Rewriting a Data-Retention Promise So Users Can Check It

Two days of beta preparation, and the thread running through them was not a feature. It was a habit I kept catching myself without: writing a promise in a form that nobody, including me, could check later. A retention line with no date. A security citation I had never read at the source. A test with no baseline to compare against. Each one looked finished until I asked who could verify it.

"Until the beta ends" is not a date

The privacy policy needed a new paragraph before the next testing round. People who apply through the sign-up form hand me answers in a Google Form, and the testers who get in send feedback and reply to emails. The policy has to say how long I keep that.

The first draft, written with my AI pair, said the answers would be kept until the beta ends. I read it twice and it sounded responsible. Then I asked the question I should have started with: if a tester wanted to hold me to this, what would they check? "The beta ends" is a thing I decide, on a day I have not picked, with no notice to them. It is a promise shaped like a promise.

The California Consumer Privacy Act is blunter than my draft was. As I read it, the notice has to state the retention period, or the criteria used to decide it. Period or criteria. Not a mood. The ICO's storage-limitation guidance says the same thing from the other side, and the GOV.UK user-research manual, which I read because it is one of the few places that talks about research data specifically, treats two to three years as reasonable for research records and notes that raw data loses its value fast anyway.

So the paragraph now has two clocks. Applicants who are not selected: their form answers are deleted within 30 days of the round filling, and the round fills on October 25, so November 24. Testers: their application, their feedback answers and their email replies are kept for up to 90 days after the testing window closes on November 8, so February 6, 2027, and then deleted. Counts and my own written summary are the only things I keep, and the policy names the two vendors that hold the raw material along the way, the forms tool and the gift card seller.

Both dates went into my records with a reminder on them, because a promise with a date is only better than a promise without one if someone actually keeps it.

Two other things came out in the same pass. The email to testers had promised they would "hear from me once more" after the window, which is another sentence with no date in it. Gone. And the day-14 form had a question asking for a quote I could use publicly. I re-read the FTC's endorsement guides before deciding, and a quote from someone who got the product free, from a founder they have talked to on a call, is weak evidence dressed up as social proof. The question is out, and this cohort will not be asked for testimonials at all.

Read the standard, not the summary

A week earlier I had written a note in my outside-facts register, the file where every law, vendor term and standard the product relies on gets a row with its source and a date, saying that the old OWASP list for language-model applications had been archived and a 2026 edition had replaced it. I wrote that from a summary.

This week I downloaded the 2026 edition itself, 122 pages, and read it. The note was wrong in a small way and a large way. Small: the site still shows the 2025 list, nothing is archived. Large: the numbering changed almost everywhere. Only the first two entries kept their positions. The one I cite most, prompt leakage, is now a broader category called hidden context exposure, sitting at number eight. Risks where the model acts on its own have moved out to a separate agentic list.

Then the uncomfortable part. One of my published case studies maps the app's safety layers to that list, and the mapping mixes a number from the 2023 edition with numbers from 2025. It was wrong on the day I published it. I have queued the fix rather than patching it tonight, because a correction to a public page gets its own careful pass, but I am writing it down here so the record is honest.

What I took from it is narrower than "be careful." It is that a summary of a standard is a lead, never a source, and that the day a claim goes public is the day the source gets re-read, whatever the date on my note says. The register now carries the edition name on every row that cites the list.

A baseline before the test

On October 26 a beta starts that is really one question asked of eight to twelve adults over two weeks on a frozen build. Do they come back? Before writing the weekly readout I wanted a number from the months already behind me, because a result with nothing to compare against is a story, not a measurement.

So I wrote the readout queries first and ran them once on what exists. Since the encryption launch in June, outside testers' sessions got through the unlock step in all but two of 38 sessions, and those two were left mid-step, not refused; no session was ever refused. About one in four sessions ended with a saved entry. Those two figures are now the floor the window will be read against. The queries count nights in each person's own time zone, with a 4 a.m. rule so a late entry belongs to the night it was written, and they are written to return counts and dates, not what anyone wrote.

I also cleared the feedback queue the same way, by reading each of the eight open items against what has shipped since. Seven were my own notes from May and June. Five were already resolved by later work, one was a test row, two need a re-check on a real phone. The queue is now empty of things that are not true anymore, so what arrives during the window is what the window produced.

Mental Models

A promise without a date is a mood. Write the period, or the criteria, and then write the date those criteria produce. If you cannot, you have not decided yet.

A summary is a lead. The day a claim about a law or a standard goes public, the primary text gets read again, regardless of how recently a note said it was fine.

Set the baseline before the test starts. A number measured after the fact is a story. The same number measured before is a comparison.

Clear the queue of the no-longer-true. Old open items hide the real ones. Read each against the present before the next batch lands.