CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED ·
When you swap the AI model, yesterday's safety evidence expires. RiskGate notices the swap and asks a person until it is recalibrated.
Transcript0:54 · narrated
Offline replay of public τ²-bench (Sierra, MIT) and AgentDojo (MIT) logs · not a live deployment
01 · What happened
The situation
Calibrated on four 2025 agents on τ²-bench airline, wrong approvals stayed under the 10% limit. Then the 2026 agents arrive, and the obvious move is to keep the calibration that worked.
02 · The miss
Why the obvious safeguard didn’t catch it
Keep the old calibration and wrong approvals jump to 14.6%: the limit breaks. Nothing in the logs flags it; the new model simply makes different mistakes.
03 · RiskGate
List, play forward, decide
The certificate records the exact agent it was measured on. RiskGate fingerprints the agent; a new agent makes the calibration void.
Until it is recalibrated, every write goes to a person.
Recalibrated on 2026 runs: 6.4%, back under the limit. The price: 80% of airline writes still go to a person.
04 · At scale
What held, and the price
It doesn't always break, which is the point: retail held under the swap, airline broke, and on AgentDojo 35% of model swaps broke the limit. You can't know in advance, so RiskGate doesn't assume.
14.6%wrong approvals after a swap
6.4%after recalibration
35%of AgentDojo swaps broke the limit
Name one action your agent isn’t allowed to take today.
Send us its traces and what happened afterwards. We’ll show how often it could have run safely, and what it would have stopped.