Field notes
Thinking out loud from the corral
Notes from the corral: where we think out loud about multi-agent orchestration, model composition, and the fences that make it safe.
The fourth participant
corral keeps three models apart so none of them marks its own exam. It never asked who wrote the code — and if that was the same model, the judge is grading its own work. This was the default configuration.
Read the note →Fifty-one minutes
I was going to launch this morning. With the window still open, I finally ran corral against one of my own projects for the first time — and found out it couldn't audit it at all. The post I had written said it supported five languages.
Read the note →The knife was ours
We told Sakana they shipped the capability and left out the containment. Today I found the same shape in my own code: the router that makes 'no model marks its own exam' true could only ever move one seat of three. Named, fixed, and proven cross-vendor.
Read the note →Four words that cost me a day
corral finally proved real bugs in someone else's code — reliably, ten times out of ten. Then it spent an afternoon telling me an audit had failed without telling me why. Plus an honest accounting of how far 'audit a whole repository' actually is: on a 236-file project, corral can currently see nine.
Read the note →A judge may not certify herself. So I made corral certify corral.
The whole thesis is that no one can judge their own work. So I turned the tool on itself and asked it to audit its own kill-rate scorer. It couldn't — for six reasons, each hiding behind the last, all wearing the same mask. Here's the debugging, the six fixes, and the moment it wrote a test for its own code.
Read the note →You can delegate the labor. You can't delegate the accountability.
AI will write your tests now — by the thousand, all of them green. That feels like relief. It should feel like a question: if nobody read them and nobody cares what they do, what is actually deciding your software is fit to ship? A short answer, and a command you can run against your own code in a few minutes.
Read the note →The code is the code
We pointed corral at a real, respected, well-tested project. Its suite killed all eight planted bugs — and then our own critic flagged four of its tests as worthless. The tempting move was to hide the criticism and publish the flattering number. We didn't, and chasing down why taught us the one rule a tool built on trust cannot break: it may only assert what execution proves — including, especially, when the thing making the claim is itself an AI.
Read the note →The critic was never the point
We opened by telling you your tests suck, then spent a note learning to keep that voice honest. Both were about the critic. But the critic was only the door. The point is friendlier and, in the agentic era, urgent: an AI will write you a thousand green tests in an afternoon, and the number of passing tests has never been what makes software stronger — the number that verifiably test something is. corral makes your suite stronger from both ends: it hands you the killing test you were missing, and it proves, by execution, which of your thousand green tests have decayed into no-ops you can delete. And it's built so everyone's tests get stronger from a shared corpus of findings that are verified and signed — knowledge you're actually allowed to trust.
Read the note →Good baking means always mind the D'oh
A builder is bounded by what a model can write. A certifier is bounded by what its sandbox can actually run. Those are different constraints — and confusing them meant we spent a while certifying Go over and over while telling a five-language story. Here's the mistake, the sharper mistake hiding under it, and why the whole thing tastes better for the kneading.
Read the note →We ran our own gate as a stranger (and it bit us)
Nemo iudex in causa sua — no one may judge their own cause — is the whole thesis of the tool. So we pointed corral certify --local at a Python file as a first-time user, not as its authors. It caught the flaw it was built to catch. It also failed its own onboarding in a way only a stranger would ever hit: the absolute path our own README tells you to type. Here's the real run, the Frampton Flaw it found, and the two bugs we found in ourselves fixing it.
Read the note →Nemo iudex in causa sua
No one may be judge in their own cause. It's a maxim of natural justice — and it's the whole design. Most ways of checking AI-written code quietly violate it: the thing that did the work reports whether the work is good. Corral is built the other way, three enforced constraints deep — and here's the objection a good CISO raises, and the answer.
Read the note →Nobody Fails a Test They Never Took
We borrowed one idea from Fugu: route each task to the model that's measured-best at it. Then we built the measuring, and learned that measuring fairly is the entire problem. Three times we penalized a model for the wrong thing — a task it never ran, a success that wasn't its doing, a verdict a human hadn't blessed — and what a fair fitness signal actually takes.
Read the note →Terribly sorry to trouble you… but your tests, um, well. They suck, really
Roger Ebert collected his most withering pans into a book called Your Movie Sucks; John Cleese built a career on the flaw everyone can see and no one can bring themselves to name. I built that critic for your test suite — it mutates your code, runs your own tests against the bugs, and if the tests don't scream it says so (terribly sorry to trouble you), with a signed record and a decorrelated second opinion. Three real verdicts follow: a two-star, a needs-review, and a genuine zero-star.
Read the note →Killing the part I was proudest of
A pivot you don't commit to is just a mood. Three days ago I wrote that corral was an accountability engine, not a builder. Then a friend reviewed the code and emailed me the bottom line: an interesting mashup of the LLM 'harness' and 'router' worlds, with a couple of fun/novel features. He was right — and the couple of novel features were the only part with no incumbent. So we're deleting the builder. Here's why the demo everyone loved had to go, and what corral actually is now.
Read the note →Is any of this worth it?
I lay awake asking whether a very cool thing was a useful thing. By morning the answer was no — not the way we'd been aiming it. We can't win as a builder. But the engine we built to contain agents and run their checks isn't a builder's engine at all. It's an accountability engine. Here's the pivot, and the working proof it produced.
Read the note →What we took from Fugu — and why
We dinged Fugu for shipping the capability without the containment. Fair's fair: here's what it got right, the idea we took from it, and the one honest line about how we took it — the insight, not the mechanism.
Read the note →Fugu, and the knife it leaves unwashed
Sakana's Fugu makes the case that orchestrating frontier models is its own scaling axis. It also leaves out the one thing its own name is about — containing the poison. Field notes on where we agree, and where we diverge.
Read the note →We caught our own gate cheating
The whole pitch here is a deterministic gate you don't have to trust. Then a pre-launch audit found the gate trusting the one thing it never should: the worker's word. Here's the bug, the failing test, and the fix — while nobody's watching.
Read the note →Why we built a brain, not a bot
Agentic coding got real in 2026 — and stayed a black box. We wanted the opposite: a herd you can contain, certify, and query. This is the founding why, and why the whole thing sits on one queryable ledger.
Read the note →