← back

A second way back in

#client-side-encryption#deployment#risk-management#debugging#product-decisions

How I closed the data-loss gap I had been carrying since June, which gate I skipped on evidence, and what the first post-deploy failure taught me about blame.

Since June, my journaling app has had a gap I could describe in one sentence: your entries are encrypted with a key that lives in your own Google Drive, so if that key ever goes missing, your journal is gone and I can't get it back. I wrote that sentence to my beta testers myself. Today I shipped the thing that closes it, a recovery code that gives people a second way back in, and the deploy taught me more about my own habits than the build did.

What the server holds, and what it never will

The design question was never whether to add recovery. It was what the server would be allowed to hold. The whole point of the architecture is that entries are unlocked on the user's device, with a key we don't keep. Any recovery path that lets my side reconstruct a key on its own undoes that in one move, and no amount of policy language repairs it.

So the shape I landed on is an escrow whose key stays with the user. When someone sets up encryption, the app generates a one-time recovery code, long and random enough that guessing it is not a realistic attack, and encrypts a copy of their existing key under that code. The encrypted copy is stored on our side. The code is shown once and never stored anywhere by us. If the Drive copy of the key is ever lost, the user types the code, the app unlocks the copy on their device, and their entries come back. Nothing on the server changed hands. The key the user already had is the key they get back, which also means none of their existing entries had to be touched.

I rejected two other shapes, and I want to name why, because the rejections carry the reasoning. A passphrase the user invents was the obvious alternative, and it fails on the thing people actually do: they pick something weak, and a weak secret protecting a stored copy turns "we hold an encrypted copy" into "we hold a copy that a patient attacker can open." A random code sidesteps that entirely. The other option was a downloadable recovery file with no server copy at all, which is purer on paper and worse in practice, because the person who needs recovery is by definition the person who lost track of something. Durability won. The honest cost is that one line of my own description had to soften, from "the key never touches our servers" to "we keep an encrypted copy the code unlocks." I would rather say the second sentence truthfully than the first one wrongly.

What I keep from this is that a recovery path is defined by what the operator is unable to do, not by what they promise not to do. If the mechanism is right, the promise is just a description of it.

Skipping a gate on evidence

My deploy checklist has a step I run before anything that could touch the AI companion: the adversarial test suite, against the live version, before the code ships. Every encryption release so far has run it. It exists because the companion's safety behavior is the one thing I refuse to regress by accident. Today I skipped it, and I want to be precise about why that was the right call rather than a convenient one.

A gate protects a premise. This one protects the premise that the companion's behavior might have changed. So the question was not "am I confident" but "could this change have touched what the suite measures." I checked the diff against the list of files that make up the companion's server side: none touched. I checked when the live version of that server code was last updated: June, before the last time the suite passed. The change I was shipping is client-side encryption plumbing and a few screens. There is no path from it to the thing the suite tests.

If I had been in any doubt, the rule says run it, and I would have. But "in doubt" should mean the evidence is unclear, not that I skipped looking. The five to eight minutes the suite costs are cheap. The habit of running gates without asking what they guard is expensive in a different way, because it teaches you that gates are rituals, and rituals get dropped under pressure.

Before I skip a gate now, I name the premise it protects and show that the premise can't have moved. Before I run one out of habit, I do the same.

I made the June mistake again, and this time it went through

In June I wrote about nearly encrypting my real journal on a preview build because the screen didn't show which account I was signed in as. I caught it then, one tap early, by checking the session directly. Today I did the same thing and did not catch it. I set up the recovery code on what I believed was my throwaway test account. It was my real account.

Here is why that was fine, and it is not because I got lucky. Enrolment was built to be safe to get wrong. It adds one encrypted copy and changes nothing else. It reads the key back from Drive and checks that key against the account's stored fingerprint before it will encrypt anything, so it cannot escrow the wrong key. And I had saved the code, because the screen refuses to let you continue until you tick a box saying you did. The step I stumbled into was one I would have done deliberately an hour later anyway.

What I actually got wrong was the check. I told myself to confirm the account by its name and avatar before tapping. My real account and my test account both display the same name, because they are both me. The check I gave myself could not tell the two states apart, which means it was not a check. The thing that could have told them apart was the entry count in the history tab, fifty-one versus two, and I didn't look at it. I've now written that down as the rule.

I notice the pattern as I write this: I keep reaching for "be more careful" as the fix, and it is the weakest fix available. The strong fixes are the two that worked, a step designed to be harmless when misused and a check that actually discriminates.

When the same mistake can recur, make the step survivable and the check discriminating. Vigilance is what you fall back on when you have done neither.

The first thing that broke was not the feature

Ten minutes after the deploy, chatting with the companion stopped working on my own account. Every message came back with "couldn't reach Luna, check your connection." The feature I had just shipped sits right next to the sign-in and encryption flow, and the tempting story wrote itself: I broke something.

I did not touch the feature. I read the logs first. The database calls from my browser were all succeeding. The one call that failed was the server-side function that asks the sign-in service to confirm who is calling, and the sign-in service was answering "that session no longer exists." My session had been revoked. By me. Earlier, on the preview environment, I had signed out of my real account to switch to the test account, and signing out revokes every session that account holds, on every device and every environment, including the production tab I came back to later. The tab still had its old token. Half the system accepted it and half did not.

Signing out and back in fixed it in thirty seconds. Nothing about the feature was involved. Two things stayed with me, though. The first is that the tempting story was wrong in exactly the way tempting stories tend to be: it blamed the newest thing. The logs told a story about the oldest thing, the session. The second is that the app's message named the wrong cause. "Check your connection" sent me, and would send any user, to look in the wrong place. The honest message was "your session ended, sign in again," and the app should have said that and shown the sign-in screen. That is now a small item on my list, and I think it matters more than its size. An error that misnames its cause costs trust twice, once when it is wrong and again when the user finds out.

The principle: when something breaks right after a deploy, get the system's own account of what failed before you write yours. Then fix the message as seriously as the mechanism.

Mental Models

Design for the operator's inability. A recovery path is only as private as the things the server has no way to do. Build the mechanism so the promise you make is a plain description of it.

Gates protect premises, not habits. Before running or skipping a pre-deploy check, name the premise it guards and show, from evidence, whether that premise could have moved.

Make the step survivable, then make the check discriminating. If a mistake can recur, design the action to be harmless when misused and give yourself a check that can tell the two states apart. "Be more careful" is the fallback for when you have done neither.

Blame the newest thing last. After a deploy, read the system's account of the failure before writing your own, and treat an error message that misnames its cause as a bug in its own right.