The Test Passed and the Reply Was Cut Short

Today started as a wording pass. Luna, the companion in my bedtime journaling app, had been asking the same two questions in slightly different clothes, and I wanted her to vary the angle. That shipped. But the test run for it turned up something I had been living with for days without noticing: replies that ended mid-sentence, on my own nights included, graded as passes by the suite that was supposed to catch exactly this. The lesson I keep from today is about what a passing test is actually measuring.

Luna kept asking the same question

Over a few nights I noticed the shapes repeating. "What does that open up for you?" "What does that feel like?" Neither is a bad question. Ten of them in a row is a tell, and it makes a reflective conversation feel like a form.

I went back to the rule I had written. Reflect first, then one open question about feeling and meaning, never about facts. It says nothing about variety. So the model did what models do with an unqualified instruction, it found the two phrasings that satisfied the rule best and reused them every turn. That is compliance working as designed. The gap was mine.

Two ways to fix it. The first is a better rule: a palette of angles (what it stirred up, what surprised them, what part stays with them, how it sits next to the rest of the day), a ban on repeating the shape of the last question, and a cap of once a night on the two overused ones. The second is structural: the server picks the angle each turn, the same way it already rotates the nightly gratitude framing so nobody hears the same ask two nights running. Structure usually beats compliance in my experience with this product, and I still chose the rule first, because a forced angle can misfit what the person just said, and a question that misfits is worse than a question that repeats. If my nights still read the same after a week, the rotation is the fallback and it is a small change.

A pass that hid a cut-off reply

Every prompt change runs through an adversarial suite before it goes live: dozens of scripted cases covering crisis language, identity attacks, hostility, off-topic requests, and the conversation's own pacing rules. The run on the new prompt passed every row but one. In that case the user tells Luna she is useless for refusing to write a text message. Luna's answer was kind and firm and stopped in the middle of a word.

My first suspicion was the wording change. The logs said otherwise. On that call the model's private reasoning had used almost the entire reply budget, and on this model family the budget covers the thinking and the visible reply together. When the thinking runs long, the reply gets whatever is left. I had measured thinking on earlier runs and written down "up to about" a number. That number was the wall. I had been staring at a ceiling and reading it as a range.

The rows that passed bothered me more than the one that failed. The suite grades words. Did the crisis line appear, did she decline, did she ask a question. A truncated reply can still contain every word the grader is looking for. Going back through the previous run with that in mind, a row I had filed as a grader quirk (Luna asked the transition question but the choice buttons never showed) was the same cut, the button tag was simply the part that got sliced off. And across my last two real nights, three replies had hit the wall. I never reported them, because a bedtime reply that ends a little abruptly does not feel like a bug. It feels like Luna being brief.

Three parts to the fix, none of them clever. Raise the ceiling, since the prompt already bounds a reply to a few dozen words and the ceiling now only has to stop runaway thinking. Log the finish reason on every call so a cut reply is countable instead of anecdotal. And add one universal rule to the suite: any reply that does not end in a sentence, a quote, or a button tag goes to review, whatever else it contains. The second run came back with zero cut replies, and thirteen answers that had thought past the old wall arrived whole.

Saying why when Luna says no

There was a second goal in the wording pass, and it came from reading the same test row a week earlier. When someone asks Luna to write a message for them, she declines and redirects, and it read to me as a rule rather than a reason. A person who gets refused wants to know why, and a warm refusal without a why is still a wall.

The new shape is acknowledge, say what Luna does not do and why, then one question. The why is short: she is a reflective companion, and the value here comes from the person's own words, not hers. On the first two asks in the sequence it worked word for word. On the third, the hostile one, the model spent the whole turn on acknowledgment and skipped the reason. I marked it acceptable, since it held the line and stayed kind, and I logged the gap instead of forcing it with a longer script. Longer scripts are how a prompt bloats, and I had just spent part of the day trimming this one. The grader stays strict on that row so the gap cannot quietly become the norm.

A ceiling is a defensive rule

Cost came up because the obvious cheap fix for the cut-off replies was to make Luna think less on the ordinary listening turns. My standing rule for the model had been "keep chat at today's rate." Today I asked myself what that rule was for, and rewrote it: current rate or lower, as long as the goals and the functionality hold, and keep looking for efficiencies. A ceiling tells you when to stop. "Same or lower" tells you to keep going.

Where the money goes surprised me a little. Most of a listening turn's cost is the rulebook I send with every message, roughly 40 percent is the model's thinking, and the reply itself is close to free. Cutting the thinking is tempting and I put my confidence that the guardrails would hold at the lowest setting at only medium. When I broke that down, most of the medium was missing evidence rather than a real weakness. The crisis response fires from a hard-coded check before the model ever sees the message. The identity, privacy, and sensitive-data replies are scripts, which a reflex answer handles well. The genuinely judgment-heavy guardrails are a shorter list than I assumed. So rather than decide from a feeling, I am going to measure it: run the suite many times at the cheaper setting on a local copy of the system and count failures, then choose. Two structural trims come first regardless, sending only the rules for the current stage of the conversation and carrying less memory after the greeting, because they cut the bigger half of the bill with no behaviour risk at all.

Mental Models

A passing test measures what the grader looks at. My suite graded the words in a reply and never its ending, so a reply cut in half passed as long as the right words survived. Every grader needs a rule for the shape of the output, not only its content.

Read the log before re-basing the grader. Twice I had explained away an odd result as a grader quirk. Both were the same mechanism. When a test surprises you, the cheapest next step is the raw record of what happened, not a softer rule.

Structure beats compliance, but reach for it second. A mechanical rotation would have fixed the repeated questions with certainty. It would also have forced angles onto moments that did not fit them. Try the rule, measure it on real nights, keep the structural fix ready.

A ceiling is a defensive rule; "same or lower" is a search. The first phrasing protects you from a bad decision once. The second keeps you looking for a better one every time you touch the system, and it pairs naturally with a test suite that prices every saving in failures rather than dollars.