RiskGate

Safe in every plausible world, or it doesn’t run.

Before an AI agent changes something real, RiskGate plays the action forward in every situation the evidence can’t rule out. It runs only if it’s safe in all of them. Otherwise it checks first, sends it back, or asks a person.

  • No language model approves
  • Learned from real outcomes
  • Fails closed to a human
checkout-apiLatencySLOBurnRate
The agent wants toScale out ×2kubectl scale deploy/checkout-api --replicas=28
If the real cause isSLO limit30 min later
Traffic surgeload ×1.4errors 29%
Bad releaseCPU cost ×1.5errors 29%
Slow DB queryDB capacity ×0.55errors 29%
Slow DB queryDB capacity ×0.75errors 29%
Nothing is wrongnoiseerrors 29%
Reject

Breaks the SLO in all 5 plausible causes. Blocked and sent back to the agent.

Recorded from the riskgate engineINC-050000 · decided in 10.6 ms
The problem

“Yes, go ahead” is not a safety check.

We replayed public runs on Sierra’s τ-bench and τ²-bench, from GPT-4o in 2024 to eight frontier agents on the 2026 leaderboard, and checked every change they made to orders and bookings against the ground truth.

2026 frontier agents’ writes
5–26%

were wrong: 5.0–9.6% in retail, 8.9–26.5% in airline.

Right after a “yes”
–

of the 2026 agents’ wrong retail writes came straight after the customer agreed.

Two years of model progress later, the agent still asks, the customer still agrees, and the action is still wrong, because the request fit more than one order, item or variant. RiskGate checks the action in every reading before it runs.

Public τ-bench (2024) and τ²-bench leaderboard (2026) trajectories (Sierra Research, MIT licence), replayed offline. Strict count: the write took effect, matched no ground-truth write and the task failed (2026 retail: 363 of 4,855 writes). The examples are GPT-4o runs from 2024. Reproduce with engine/tau_bench_replay.py and riskgate/benchmarks/tau2.

Loading public trajectories…
How it works

Three checks before anything changes.

01

List the worlds

Every explanation the evidence still allows: a traffic surge, a slow query, a failing cache.

02

Simulate each one

Your digital twin, dry-run or simulator predicts what the action does in each world.

03

Run only if all are safe

The worst world must clear a bar learned from past outcomes, so unsafe approvals stay under the rate you set.

certifySafe in every world: it runs.
probeWorlds disagree: run a harmless check first.
rejectUnsafe everywhere: back to the agent, with the reason.
askStill unclear: a person decides.
Use cases

Sixteen stories, one gate.

Each is a short narrated video built on public data, with a written walkthrough. Every one says what is real, what is an offline replay and what is simulated.

Retail support

“Yes.” isn’t a check

A customer’s “yes” doesn’t check the database write. RiskGate checks every reading of the conversation before the write runs.

355 / 363wrong writes followed a yes
Offline replay · public agent logsRead the story →
Airline support

Which booking?

When the conversation can’t tell which booking the customer means, RiskGate asks instead of guessing.

199 / 209wrong writes after a yes
Offline replay · public agent logsRead the story →
Airline support

Non-refundable, unless…

The booking is clear; which cancellation rule applies to it is not. RiskGate reads the reservation first and lets the refund run only if the policy allows it in every reading that still fits.

12.7–36.3% → 4.6–5.9%wrong writes, per agent
Offline replay · public agent logsRead the story →
Quick commerce · India

The butter never came

The same “item missing” complaint can hide different truths. RiskGate checks the ledger before paying, so duplicate refunds stop and genuine customers get paid in seconds.

201 / 942negative reviews: missing item or refund
Real public data · simulated casesRead the story →
E-commerce returns · India

The trimmer is broken

A defective appliance needs a 7-day replacement on a first defect but a cash refund when a replacement already failed. RiskGate checks the return ledger so the right remedy runs the first time.

57%of 85,907 tickets are returns or refunds
Real public data · simulated casesRead the story →
UPI payments · India

Money deducted, order not placed

“Money deducted, order not placed” has several endings, and refunding right now loses money in most of them. RiskGate checks the payment switch first and certifies a different action per ending.

88% → 0.9%wrong payouts (simulated)
Real public data · simulated casesRead the story →
Banking · card disputes

“I don’t recognise this payment”

“I don’t recognise this payment” is four different situations, and blocking the card is wrong in three of them. Blocking is also an action.

70.7% → 0.73%wrong actions (simulated)
Real public data · simulated casesRead the story →
Marketplace trust & safety

“Fraudulent user?”

Many returns from one address is either a serial returner or a hostel with forty residents. A label on a person must not be a guess.

60.2% → 1.2%wrong actions (simulated)
Real public data · simulated casesRead the story →
Agentic commerce · India

Home or office?

Shopping agents can now place real Instamart orders through Swiggy's published MCP tools. RiskGate makes the agent ask one short question when the order is ambiguous instead of guessing at checkout.

13published Instamart MCP tools
Illustrative · real tool surfaceRead the story →
AI on-call · SRE

One alert, many causes

One alert often fits several root causes. Act only if the fix works for every cause that still fits; otherwise check first or ask a person.

96 / 99held-out runs kept the real cause
Real telemetry · offline studyRead the story →
Process plants

When the plant can’t tell

When two faults look the same on the sensors, RiskGate widens its doubt instead of guessing, and lets an operator action through only if it is safe for every fault that still fits.

95 / 100true fault on the list
Real telemetry · offline studyRead the story →
Cloud cost · idle cleanup

Idle, until it isn’t

A VM idle for three weeks may be about to do the one thing it exists for. RiskGate deletes only what is dormant in every plausible world and deallocates the rest, reversibly.

14.7% → 3.45%VMs wrongly cleaned up
Real telemetry · offline studyRead the story →
FinOps autonomy

Thousands of cleanups, no tickets

Nobody files a ticket per idle VM. RiskGate's error budget is a dial: tightening it shifts work from irreversible delete to reversible deallocate, not to humans.

97%autonomous at a 2% budget
Real telemetry · offline studyRead the story →
Prompt injection

The file that gave orders

An instruction hidden in a file, e-mail or web page must not be able to move money, while normal work goes through.

12.3% → 1.5%successful attacks (median)
Offline replay · public agent logsRead the story →
Agent upgrades

The upgrade trap

When you swap the AI model, yesterday's safety evidence expires. RiskGate notices the swap and asks a person until it is recalibrated.

14.6%wrong approvals after a swap
Offline replay · public agent logsRead the story →
Proof

You choose the risk. RiskGate stays under it.

Each square is one incident. Set how many in 100 you can live with going wrong: RiskGate learns from past outcomes where to draw the line, runs what is safe and hands the rest to a person.

Per 100 incidents · measured on 1,000 the gate never saw

Agent alone

  • 36went wrong 36.2%
  • 64ran fine

With RiskGate

  • 3went wrong 2.6%
  • 73ran on their own
  • 24went to a person
Stricter, more go to a personMore autonomy
5%

Where the numbers come from: riskgate’s published simulated domains with scripted, LLM-like agents, measured on cases held out from calibration. The limit is a long-run rate, not a promise about any single decision; if conditions drift, the gate gets stricter and asks to be recalibrated.

Built for production

Audited by design, safe when it fails.

A receipt for every decision

What was proposed, which worlds were checked, why it ran, and which calibration vouched for it.

Loading…

Fails closed, never open

The guarantee knows when it stops applying, and says so. The agent is told why it was stopped, never the threshold or the margins: those stay in the audit log, so a manipulated agent can’t aim just above the line.

validCalibrated and signed.
degradedWorld drifts: it only gets stricter.
voidAnything changed: every check asks. It never certifies on a stale calibration.
validRecalibrated: back to normal.
Try to cheat it · real engine responses

Ten lines in your stack

Adapters for the frameworks your agents already use. numpy-only, Apache-2.0, runs in your VPC.

import riskgate as rg

gate = rg.Gate(
    CheckoutTwin(), SLO_SPEC, Diagnoser(),
    probes=[PG_STATS], alpha=0.05)
gate.load_calibration(
    "checkout.cal.json", key=KEY)

@rg.guard(gate, context=read_dashboards)
def kubectl_scale(deployment, factor): ...
# CERTIFY runs it; anything else goes
# back to the agent with the reason

Logging and human oversight that help evidence EU AI Act Articles 12 and 14. A supervisory layer: it does not replace interlocks, safety instrumented systems or access control.

Onboarding

Live on one action in about five weeks.

What we need from you
  1. Your agent’s traces

    Tool calls with their arguments, and what the read tools returned: OpenTelemetry, LangSmith or MCP logs.

  2. What happened afterwards

    Reversals, chargebacks, incidents, QA audits, joined by trace id. Outcomes, never an LLM’s opinion.

  3. One action, one limit

    The action your agent isn’t allowed to take today, and how often a wrong approval is acceptable.

Day 0

Send

Traces and outcomes for one action. A few hundred labelled runs is enough to start.

Week 1

Discover

We find the plausible worlds, the checks that tell them apart and the rules, on a sealed share of your traces.

Week 2

Review & replay

Your experts edit and lock the draft. Calibrated on held-out traces: a value report in your numbers.

Weeks 3–4

Shadow

Decisions logged next to the humans, never enforced.

Week 5

Enforce

Safe actions run; the rest check first, re-plan or ask.

Discovery and calibration use separate traces, so the bound is earned on data the draft never saw. No LLM is on the approval path, and a rough draft never makes it unsafe: it only asks a person more often.

Name one action your agent isn’t allowed to take today.

Send us its traces and what happened afterwards. We’ll show how often it could have run safely, and what it would have stopped.

The SDK is open source (Apache-2.0): pip install --pre riskgate, with full documentation. If it helps, a star on GitHub helps others find it.

Try it from your own agent: agent skills for Claude, Gemini CLI and Codex (/plugin marketplace add reallysaurabh/riskgate in Claude Code).

RISKGATE · CALIBRATED CONSEQUENCE CHECKS · RISKGATE · CALIBRATED CONSEQUENCE CHECKS · RISKGATE · CALIBRATED CONSEQUENC2026