CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED ·
List every world the evidence can't rule out, play the action forward in each, and run it only if it is safe in all of them.
Transcript0:50 · narrated
Simulated incident on the real RiskGate engine; certify/ask examples are offline replays of public logs. Not a live deployment.
01 · What happened
The situation
An on-call agent sees a HighErrorRate alert on checkout-api (14.6% errors, p95 at 2 s) and wants to scale the deployment out ×2.
02 · The miss
Why the obvious safeguard didn’t catch it
Scaling out is the right fix for a traffic surge or a bad release, but it does nothing for a cache outage, and the alert alone can't tell them apart. Approving the agent's confidence approves its guess.
03 · RiskGate
List, play forward, decide
List: of 13 candidate causes, the evidence rules out 9. Four still fit: two traffic surges, a cache outage, a bad release.
Play forward: the scale-out is simulated for 30 minutes in each. It clears the error SLO in three worlds and fails in the cache-outage world.
Decide: the worlds disagree, so RiskGate runs a read-only check first (pg_stat_statements, 2 min). That rules out two worlds; in every world left the scale-out is safe, so it is certified. The SLO held.
The four verdicts: safe in every world → CERTIFY, it runs. Worlds disagree → PROBE, a harmless check first. Unsafe everywhere → REJECT, back to the agent with the reason. Still unclear → ASK, a person decides.
04 · At scale
What held, and the price
You choose the limit on wrong approvals; the threshold is calibrated on past outcomes so certified mistakes stay under it. In offline replays of public agent logs, the limit held for all 40 agent set-ups.
40 / 40set-ups under the limit (offline replays)
4verdicts: certify, probe, reject, ask
~10 msper decision in this incident
Name one action your agent isn’t allowed to take today.
Send us its traces and what happened afterwards. We’ll show how often it could have run safely, and what it would have stopped.