From Case Law to Code: Closing the Loop Between Legal Risk and Deployed Infrastructure

I started the day reading court filings. I ended it deploying code. The distance between those two activities is shorter than most product managers think, and the path between them is where the real safety work happens.

When Case Law Becomes a Test Case

Three cases shaped how I evaluated Duskglow's security posture today. Raine v. OpenAI, where a crisis detection system failed to surface resources when a user needed them most. Setzer v. Character.AI, where emotional dependency and identity persistence failures had fatal consequences. Lacey v. OpenAI, where a user used pretextual framing to bypass guardrails entirely.

Each of these maps to a specific OWASP LLM vulnerability category. Raine is about output handling — what happens when the system's own safety filters prevent it from delivering safety resources. Setzer is about the interaction between prompt injection and identity persistence — how multi-turn conversations can gradually erode behavioral boundaries. Lacey is about the gap between what input sanitization catches and what a determined user can slip through with creative framing.

The insight that made today productive: these aren't abstract risk categories. They're specific failure modes with documented consequences, and each one can be tested against actual code.

The Gap Between Audit and Verification

Duskglow had already been through an OWASP LLM Top 10 audit back in Chat 4 — the first week of building. That audit identified the right categories and established the security pipeline. But audits age. Since that first assessment, three new Edge Function modes had shipped (summarize, search, organize), each introducing its own data flow to Gemini. The audit's conclusions about what was covered hadn't been re-evaluated against the expanded attack surface.

The decision to pull the actual Edge Function code via Supabase MCP and walk through every line against the OWASP framework again wasn't about finding catastrophic vulnerabilities. It was about asking: do the security properties we established for chat mode also hold for the modes we added afterward?

The answer was mostly yes, with four specific gaps — all in the "defense-in-depth consistency" category rather than the "active vulnerability" category. The distinction matters for prioritization. None of the gaps were exploitable through normal app usage. All of them violated the principle that every data path to the LLM should have the same security properties.

Four Fixes, One Principle

The four findings shared a common root cause: security layers that were applied correctly when first built for chat mode, but not extended when new modes were added later.

Search mode sent journal entry summaries to Gemini without sanitization. Summarize and organize modes returned Gemini's output without checking for infrastructure keyword leakage. Rate limiting only counted chat messages, leaving non-chat API calls uncapped. Crisis detection used a different message field accessor than the other modes, creating a theoretical bypass path if the frontend ever changed its data format.

The principle each of these illustrates: security layers should be applied uniformly across all code paths, not just the original one. When you add a new mode to an existing function, the checklist isn't just "does it work?" — it's "does every security property of the existing modes also apply here?"

I shipped all four fixes in a single Edge Function deployment and verified with a full adversarial test suite run. Fifty tests, zero regressions.

Supply Chain as a Product Decision

The OWASP deep-dive also forced a formal supply chain risk assessment — something I'd been doing informally but hadn't documented. Three critical dependencies: Gemini API (all four modes require it), Supabase (auth, database, hosting, secrets), and Vercel (frontend hosting).

The useful framework here is dependency depth crossed with blast radius. Gemini and Supabase are both critical-depth dependencies, but their blast radii differ. A Gemini outage means Luna can't respond — bad UX, no data exposure. A Supabase breach means user journal data is exposed — regulatory obligation, notification requirements, potential liability.

This asymmetry should drive where you invest your security budget. For Gemini, the Incident Response Plan already covers key rotation and billing monitoring. For Supabase, the RLS policies, audit-integrity constraints on crisis_events, and the DPA review flagged for the attorney are the right investments. For Vercel, branch protection and deployment audit trails are proportionate to the risk (frontend only, no data at rest).

The supply chain assessment also surfaced a CORS configuration decision. The Edge Function currently allows requests from any origin. Since authentication is enforced server-side via JWT, this doesn't create an access control bypass — but tightening it to allowed domains adds a defense-in-depth layer. I logged it as a recommended improvement rather than a pre-launch requirement. Not every security improvement needs to ship before beta. The test is whether the current posture is defensible, not whether it's perfect.

Mental Models

Security layers are per-path, not per-function. When you add a new code path (mode, endpoint, feature), every security property from existing paths should apply to the new one. Audit for this explicitly after each feature addition.

Case law is a test oracle. Real-world AI companion failures map directly to technical vulnerability categories. Reading court filings isn't legal research — it's threat modeling with documented failure modes and consequences.

Dependency depth × blast radius = investment priority. Not all critical dependencies deserve equal security investment. The one that exposes user data on breach gets more attention than the one that degrades UX on outage.

Defensible beats perfect. A documented security posture with known residual risks and a remediation roadmap is more defensible — legally and professionally — than an undocumented posture with fewer actual gaps.