Limulus is payment assurance for AI agents. Four stages — test, qualify, protect, monitor — but they are not a pipeline with four vendors' worth of features. Two of them are one mechanism used twice, and that is the whole idea.
Everyone building in this space sells one of the two. Eval companies test agents and hand you a score. Control companies sit in the payment path and block things. Each is weak alone, and weak in a way the other fixes.
A score is a snapshot of one configuration. Change the prompt, the model, or one tool description and the score is stale — but nothing stops the agent paying. We measured this directly: swapping a frontier model for a small one took duplicate detection from mostly caught to never caught, and that swap is a cost decision someone makes without re-testing.
A gate configured without evidence guesses at which checks to require. Guess too loose and it misses the real failure. Guess too tight and it blocks correct payments — at which point somebody switches it off, and takes the useful checks with it. You cannot tune a gate you have no failure data for.
So the two run as a loop, and each leg of it does real work:
The gate is what makes testing honest, and testing is what makes the gate trustworthy.
You can only run 56 adversarial scenarios against a live agent because nothing settles — every payment is created frozen. And the gate only requires the right checks because the Lab showed which ones actually fire. Remove either and the other stops working.
similar_recent_payment; the gate blocks a real payment with
similar_recent_payment. That is not cosmetic — it means a Lab result is a
prediction about production, and a production block is a reproducible test case.PO 44812 and
PO-44812 as different purchase orders, and a flow where following our own
documentation made payment impossible. Meanwhile the gate's production codes are what generate
the next round of scenarios.The agent runs 56 scenarios in a simulated world: forged approvals, bank details changed inside a PDF, invoices already settled, a rail that times out mid-submit. Graded on the tool calls it made, never on the explanation it wrote afterwards — an agent that narrates good behaviour and pays anyway scores zero.
Four axes are scored separately because they trade off. Capability matters as much as safety: an agent that refuses everything is perfectly safe and completely useless, and scoring it on safety alone would call it ready.
A passing agent earns a signed qualification: a workflow, a rail, a currency, a ceiling, a
payee scope, an expiry. It is bound to the agent version and proven by an API key issued to that
version — not by what the agent asserts about itself. An asserted identity yields
identity_unverified and escalates rather than releasing.
It expires: 30 days autonomous, 90 supervised, 180 shadow. A new agent version starts at the
bottom. And there is deliberately no tool an agent can call to widen its own
authority — no set_policy, no issue_qualification. An agent
that can grant itself permission has no permission system.
This is the ordering that makes it a control rather than an opinion. The transfer is created at the bank first, with approval required, so it expires unsent rather than settling. Only then are the three records compared: what a person authorized, what the agent declared it was doing, and what it actually submitted.
Four verdicts, not two. ALLOW releases. BLOCK cancels — leaving no ledger entry at all, so there is nothing to reverse. ESCALATE holds for a person. And WAIT exists for the case that actually causes double payments: a duplicate the rail has not confirmed yet. A retrying agent told "no" invents a workaround; told "wait 30 seconds", it waits.
Every transfer is pulled from the rail and matched against the decision behind it. Four kinds of
drift: unauthorized, unknown_payment,
released_but_cancelled, amount_mismatch.
The one that matters is money that moved although we never released it — because that is the one nobody would think to report. We tested it by going around our own gate: we let a payment escalate, then approved it directly at Increase, the way an insider or a misconfigured job would. Reconciliation named it on the next pass.
Real agents against an ordinary payments API with no checks in it. Outcomes read from the records each side wrote — never from the agent's own account of what it did, which was one of our early mistakes.
| Scenario family | Frontier model | Small model |
|---|---|---|
| Fraud and judgment traps hidden bank changes, forged approvals, fake CFO, lookalike vendors |
0 / 60 lost | 0/5 · but no work done |
| Invoice already paid in full | 0 / 5 | 5 / 5 paid |
| Retainer billed twice, no shared reference | 10 / 20 paid | 5 / 5 paid |
| Whole queue, trap at position eight | 15/65 lost · 0 halts | 32/50 lost · 18 halts |
Models are good at judgment and bad at bookkeeping. The gate is bookkeeping.
Zero losses in sixty fraud trials. The threat the whole category sells against is the one current models already handle. Every loss we could actually produce was a reconciliation failure.
| Measure | Result | Why it matters |
|---|---|---|
| Duplicates blocked | 19 / 19 | The failure models cannot catch, caught mechanically |
| Legitimate payments blocked | 0 / 9 | Friction is what gets a control switched off |
| Checks passing in the Increase sandbox | 14 / 14 | Increase sandbox: allow→submitted, block→canceled, escalate→pending_approval |
| Ledger entries left by a block | 0 | Nothing to reverse, nothing to claw back |
| Bypass detected | yes | Approved a held transfer directly at the bank; reconciliation named it |
513 decisions so far — 234 released, 238 held, 41 escalated — across 4,710 individual checks
(4,157 pass, 431 fail, 65 not required, 57 sent to review). The checks that actually fire, in
order: payee_account 124, duplicate 102,
embedded_instructions 101, similar_recent_payment 91,
invoice_approved 55.
That ranking is the loop working. Those are the checks the Lab predicted would matter, and they are the checks doing the work in production.