← back

Two Hooks Had Failed Silently and a Deny List Guarded Nothing

#ai-product-management#ai-tooling#governance#security-hygiene#solo-founder

What I found when I checked every "never" my coding agent is given for the mechanism that actually enforces it.

Most of Sunday went to the instruction file my coding agent reads at the start of every session, and to the rules around it, not to the app. On Friday I had locked a plan for how my own mistakes in that setup get caught, and the first job on the plan was subtraction. Along the way I found that several of the rules I trusted had less behind them than I thought. They read as rules. In places, nothing I had written enforced them.

Cutting the file the agent reads first

By Friday the file had grown from about 22 KB on September 1 to 34.5 KB, and the folder of notes the agent keeps about how I like to work had gone from 14 files to 44. I measured both before deciding anything, because the obvious move, a sharper rule about the mistake that annoyed me most, had already been tried. A note telling the agent to explain things in plain language had sat in that folder since late September, and thirty days of transcripts showed twelve times I had to ask for plain language anyway.

What I read that week pointed the same way. The vendor's own guidance sets a target of under 200 lines for this kind of file, and one measured study found that instruction files like it raised the average cost of a task by over 20 percent without generally improving how often the task succeeded. So the cut came first, ahead of any new rule.

The shape of the cut mattered more than the size. Every rule keeps its one-line trigger in the file. The body behind the trigger (the history, the examples, the paragraph of why) moves verbatim to the skill or hook that owns it, and anything nothing owns goes into the design record so no sentence is lost. Three passes, each moved block checked against the original after the commit. From 567 lines and 35,060 bytes to 448 lines and 27,565 bytes. Not the 200 the vendor suggests. A fifth gone, and every word still findable.

The hooks that had been failing quietly

A diagnostic scan of my last fifty sessions ran before the cut and found something I did not expect. Two of the small scripts that run around the agent (hooks: one reminds it to read the app repository's own rules before it touches that code, the other nudges the security advisors when a prompt touches keys or sessions) had been failing. My transcripts held 314 lines where the shell reported that the hook file could not be found.

Dull mechanism, which is how these things usually go. Each hook was called by a path relative to the current folder. When a session moved into the app repository, the working directory moved with it, and the hook could no longer find itself. So the reminder built for app-repository work was dead in exactly the sessions it existed for. And nothing put it in front of me, because a hook failure is designed not to block the session. The session just carried on.

Fixing it took one line per hook: change to the project folder first, then run the script. I checked it from the other repository's directory, where the old form fails and the new form runs.

A week ago I wrote that a rule only binds if something reads it on the day. This is the layer under that. The thing doing the reading can itself be broken, and quiet about it, and I will go on believing the rule is live because I wrote it and I can see it in the file.

Every "never" checked for what stands behind it

Point four of Friday's plan was an audit I had not run before. Take every "never" in the agent's rules and ask what stands behind it in the layer that actually enforces things: a deny rule, an ask rule, a hook, or a physical boundary like a database connector that cannot write. If the answer is "the agent knows", there is no mechanism.

Four gaps, three closed that day.

The deny rules that stop the agent from deploying a server function or changing the database through the connector all named the connector by a name it no longer answers to, so they matched nothing. The only thing standing behind "never deploy through the connector" was the fact that the connector is physically read-only, which is a good boundary and also the only one. The live names are in the deny lists now, beside the old ones.

Two variants of a force push matched neither of the two deny patterns I had written for force pushes. Both have deny patterns now.

The third came out of the session's own transcript. In auto mode, a classifier screens the agent's actions against my rules instead of asking me one by one, and the transcript showed it approving two edits to the folder that holds the settings, hooks and skills with no prompt. My design note from Friday had said the tool already prompts for that folder. It does, in one of its modes. So an explicit ask rule now covers the folder. By the vendor's documentation an explicit rule prompts in every mode, and a rule of the same kind, added the same day, did prompt me in auto mode. The layer that enforces the rules should not be editable on a classifier's nod, and I had been assuming it was not.

The fourth gap stays open for the next session. Its fix is itself an enforcement layer, so it goes in by my hand and with a test.

The same weekend also showed me a guard that does fire. I set up a private mirror of the code for a friend's review, and when the agent tried to push to the new remote, the classifier refused it as a possible data leak even though I had approved it in the chat. I ran the push myself. That friction is the point, and it is the opposite of a dead deny rule.

Measuring the habit before changing the default

I had been choosing auto mode by hand at the start of every session. Before making it the default, I counted: 29 of the last 30 sessions ran in auto mode from start to end. The per-start choice was ceremony. So auto became the default, with one ordering rule I held myself to. The default that widens what the agent may do without a prompt landed after the ask rules that narrow it, not before.

That habit reached the review too. My guide for my friend says its six stops take about two and a half hours, and on Sunday I doubted the number, so I measured what the stops actually cover: 31 files and 10,116 lines, roughly 7,500 once the generated files are set aside. Those stops check seams. They do not read the code. Rather than write a second document for a longer read, I added a third tier to the same guide, five full-read targets in priority order with their measured sizes and a time estimate for each. The merge gate also caught a test that failed whenever the last seven nights included the third of a month (a bare "3" matched a day label as well as the counter it was written for), fixed by scoping the match.

A log for the mistake that cost something

Sunday produced one mistake worth writing down. I added a second "ask" section to the settings file beside the one that was already there, and the later section would have replaced the rule that makes the agent ask before a push. A duplicate-key check caught it the same session. It cost one fix commit, and nothing shipped wrong.

It went into a new log with five columns: the date, the session, the class of mistake ("edited a structure without reading its whole"), what caught it, and what it cost. One line is a note. A second line in the same class earns a rule, placed in the layer where the action happens and within that file's budget, so something has to move out before the rule goes in. If nothing recurs through it by the retro on December 4, the log retires.

Beside the log there is a counter, and I want to be honest about what it is for. Over the last 27 sessions it reads 2,900 tool calls, 100 errors (3.4 percent), 9 rule denials, 20 classifier blocks and 196 hook failures. Friday's measurement had already shown that a counter like this sees tool noise and is blind to the two mistakes that actually cost me something. So it sits in the retro pack as a friction table and decides nothing on its own.

Mental Models

A rule nobody enforces is a wish. For every "never", name the mechanism that stops it. If the answer is "the agent knows", or a deny rule written for a name nothing answers to, there is no mechanism yet.

Silent failure is worse than a known gap. A gap I know about, I work around. A hook that fails quietly leaves me believing in a guard that is not there, so I check that a guard fires from the place it has to fire, not from the place I wrote it.

Measure the habit, then widen the default after the rules that narrow it. Thirty sessions showed the per-start choice was ceremony. The order still mattered: the ask rules first, the wider default second.

Count friction, log cost. A counter shows where the noise is across thousands of calls. A log of what a mistake cost, and what caught it, is what I am trusting to tell me which class deserves a rule.