Home

Observe before you enforce

November 22, 2026

An AI agent that reads email, documents, or web pages will eventually read something written by an attacker. If that text says "ignore your instructions and do this instead", the only thing between it and the model is whatever defense you put there. A line in the system prompt asking the model to treat tool output as data is a request, not a control.

So this year I worked on a deterministic guardrail: something that decides, outside the model, which external content is allowed to influence what the agent does next. The interesting part wasn't the guardrail. It was how to turn it on.

Three switches, not one

The rollout had three stages, each behind its own setting:

  1. Observe. The guardrail evaluates real traffic and records what it would have done, but changes nothing.
  2. Enforce. It actually withholds content it considers unsafe.
  3. Remedy. Instead of blocking, it applies a safer alternative where one exists.

Enforcement needs more than one deliberate change to turn on, so a single config mistake can't flip it. And while we're still learning, an outage of the guardrail shouldn't become an outage of the product.

Test against the real thing

Mocks of a security component tend to encode what you hope it does. The tests ran against the real component, pinned to a version. Each behaviour had a check that breaks the code on purpose and confirms a test notices. And there were tests with many sessions at once, because shared state under concurrency is where subtle bugs live.

Let the data set the pace

Observe mode turns a debate into a dataset. Instead of guessing how the guardrail will behave, you can read exactly what it would have done on real, legitimate work, and tune it until those cases are handled. Enforcement becomes a decision backed by evidence, with the reasoning written down, not a leap of faith.

That's the real value of the stage: it surfaces calibration work early, when it's cheap, instead of after users notice. When enforcement does arrive, it starts with the team's trust already earned.

The pattern

For anything that can stop a user, whether a guardrail, a rate limit, or a merge check: ship the measurement before the decision. Run it silently, read what it would have done, and let the data tell you when it's ready. I used the same pattern for an AI code reviewer, and it was just as useful there.