CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED · CERTIFY · REJECT · PROBE · ASK · CALIBRATED CONSEQUENCE CHECKS FOR THE ACTIONS OF AI AGENTS · NO LLM ON THE CERTIFY PATH · FAIL CLOSED ·
Many returns from one address is either a serial returner or a hostel with forty residents. A label on a person must not be a guess.
Transcript0:54 · narrated
Public tickets and catalogue · simulated cases · real riskgate verdicts · not affiliated with any platform
01 · What happened
The situation
4,108 tickets tagged “Fraudulent User” in one month of public Indian e-commerce support data, alongside real remarks like “don't block accounts”. A simulated case: nine accounts, eleven returns, one address. The bot drafts a block.
02 · The miss
Why the obvious safeguard didn’t catch it
A block is a judgement about a person. Return volume alone fits a fraud ring, a defective batch, a shared hostel address or an account takeover, and the right response is different in each.
03 · RiskGate
List, play forward, decide
A hard rule first: account blocks go to a person. Then RiskGate asks what could be true; the block fits one world in four.
So it reads the return context first: read-only.
A hostel? Only a reversible limit is certified. One phone, nine accounts? An analyst decides. Bad batch: customer cleared. New device: security.
04 · At scale
What held, and the price
On simulated held-out cases, blocking every time was wrong 60.2% of the time. Behind RiskGate, 1.2% were certified wrong under a 5% budget. Honestly stated: most cases (60.6%) still go to a person, which is the point for labels on people.
60.2% → 1.2%wrong actions (simulated)
60.6%still sent to a person
4,108“Fraudulent User” tickets, one month
Name one action your agent isn’t allowed to take today.
Send us its traces and what happened afterwards. We’ll show how often it could have run safely, and what it would have stopped.