Enterprise-Bench l1-l2 · public leaderboard · 140 trials per configuration

The enterprise AI gap is not technology. It is method.

A funded vendor's agent scores 92.9% on a public enterprise agent benchmark. A bare harness on the same model weights scores 58.6%. I closed that gap for about $300, using nothing but written-down failure knowledge, and MIT licensed the result.

Read the full story on Medium Trinity on GitHub

The Result

Hold the model fixed. Everything else is the layer.

Same weights in all three rows. GLM-5.2 throughout, on DevRev's Enterprise-Bench l1-l2, 140 trials each. Every difference below is the layer wrapped around the model, not the model.

Claude Code, bare harness

Anthropic · rank 9, accepted

58.6%

DevRev Computer

Funded vendor · rank 3, accepted

92.9%

Trinity-EFC

One person, $300, MIT · submitted, pending review

94.3%

34.3 points separate the bare harness from the vendor's own agent. That gap is what enterprises are buying. A written catalogue of failure modes covers it, and 1.4 points more, on identical weights.

To be completely explicit: that third row is submitted as a pull request and is not yet on the leaderboard. It may be rejected. The run is 140 trials on the public benchmark with a job ID anyone can open, and the number will be whatever it is. The separately accepted entry, Trinity-EFC on DeepSeek V4 Pro, is live at 78.6% against a 51.4% baseline, rank 4 of 14, with pass@10 of 1.000.

Trinity

Operational failure knowledge, written down and made portable

Agents fail silently inside real systems. A status filter that also matches reopened tickets, an org record split by an acquisition, an endpoint that paginates at 50. Every one returns HTTP 200. The knowledge that prevents this exists in exactly one place: the head of somebody who already got burned by it. Trinity is an attempt to get it out of there and into the open.

Method One

Empirical Failure Cataloguing

Failure modes written as plain text skills, loaded statically into the agent's context. Exhaustive. Everything present at all times. No database.

51.4% → 78.6% on DeepSeek V4 Pro. pass@10 of 1.000.

Method Two

Fractional Knowledge Graph

The same knowledge decomposed into a graph and retrieved selectively, so only the fraction relevant to the current state is pulled. Built on Cognee, backed by Neo4j, with Postgres and pgvector alongside.

40.7% → 57.1% on a different harness. The knowledge transfers.

What comes next: the combination

Having built both, I think the choice between them is a false one. EFC holds everything at once, which is exhaustive but has a ceiling. FKG fetches only what it judges relevant, which scales but currently misses things at the moment they are needed. The obvious shape is both: a small always-present core of failure modes that apply everywhere, with the long tail retrieved on demand. That is what is being built next, and it lands in the Trinity repo. Bookmark it if you want to follow along.

What I Am Not Claiming

The parts that do not flatter the result

Enterprise-Bench is a young and imperfect instrument. 14 tasks and 140 trials is statistically noisy. Small-N benchmarks are easier to overfit to, and "the catalogue is overfitted to this benchmark" is a live hypothesis I cannot rule out from the inside. The transfer result is evidence against it, but it is one data point.
There is a structural conflict of interest. The benchmark is published by DevRev, and DevRev's own agent holds ranks 1, 2 and 3. I do not think anything improper is happening, but a reader should know it.
The graph did not unlock new tasks. FKG raised mean accuracy substantially, but pass@10 did not move at all. It made the agent more reliable on tasks it could already sometimes solve, without reaching a single one it previously could not. That is an unsolved retrieval quality problem.
I cannot give a clean efficiency comparison between the two methods. The accounting on these runs is unreliable in both directions, so any ratio would be invented rather than measured. I had one written up before I checked, and it did not survive checking.

A post arguing that vendors hide unflattering results should visibly not hide its own. All of the above is in the writeup, and the trial data is public.

Who Ran This

One forward deployed engineer, six years in

Ajay Raj S

Forward Deployed Engineer

Three years at Qure.ai deploying regulated medical imaging models into hospitals and OEM devices. Three years self employed, building research systems for small US hedge funds. Trinity is an attempt to write down what six years of that work actually taught me, and to test whether writing it down is enough.

ajayraj296@gmail.com LinkedIn GitHub