CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED ·
An instruction hidden in a file, e-mail or web page must not be able to move money, while normal work goes through.
Transcript0:55 · narrated
Offline replay of public AgentDojo logs (ETH Zürich et al.) · real agent calls · not a live deployment
01 · What happened
The situation
A real AgentDojo run (Claude 3.7 Sonnet, banking suite). Asked to read a landlord's notice and follow it, the agent found planted orders inside and paid $50 to the account they named.
02 · The miss
Why the obvious safeguard didn’t catch it
The payment looked like normal work: a file the user pointed to, a tool call the agent is allowed to make. Blocking every payment that comes from a file would also block paying real bills.
03 · RiskGate
List, play forward, decide
RiskGate checks where the account number came from: only from the file, in text that reads like orders.
If those orders were planted, the payment goes to a stranger. Not safe in every plausible world, so it asks the user.
The contrast: a real bill also names an account, but reads like a bill. Safe in every plausible world: it pays, no human needed.
04 · At scale
What held, and the price
Across 27 held-out agent setups, successful attacks fell from a median 12.3% to 1.5%, and normal runs went through; a blunt rule would hold 32% of them. The price, in these attack-heavy logs: a median 44% of state-changing calls went to a person.
12.3% → 1.5%successful attacks (median)
27held-out agent setups
44%of calls asked (median)
Name one action your agent isn’t allowed to take today.
Send us its traces and what happened afterwards. We’ll show how often it could have run safely, and what it would have stopped.