The critic was never the point
Field note. The third of three, and the one that says what the first two were really for.
The door, and the room
We came in loud. The first note borrowed Roger Ebert’s precision and John Cleese’s excruciating courtesy to say the thing nobody in a code review will say out loud: your tests, terribly sorry, don’t actually test anything. It’s a good line. It gets the click. It also gets misread as the whole personality — a tool that shows up to tell you you’re bad at your job.
The second note was the reckoning: our own critic, pointed at a respected project, hallucinated the standard library and accused a perfectly good test of a crime it didn’t commit. We nearly hid it. We didn’t, and the lesson was permanent — a tool built on trust may assert only what execution proves. The critic’s opinion doesn’t get to ride first class next to a signed number just because it’s ours.
Both notes are about the critic — the voice that says the unsayable, and then the discipline that keeps that voice honest. And here’s the thing I want to say plainly, because it took us a while to say it to ourselves:
The critic was never the point. It’s the door. The room is the opposite of an insult.
The point is to make your tests stronger
When corral finds a gap — a bug your suite plants right past — it doesn’t stop at the diagnosis. A decorrelated model (not the one that graded you) writes the test you were missing, runs it against the planted bug, watches it fail, and hands the test to you. Proven, not suggested. You don’t leave with a scolding; you leave with a suite that’s measurably stronger than the one you walked in with, and a receipt that says so.
But “stronger” cuts both ways, and the other way is the one nobody talks about. Picture the codebase everyone actually has: a thousand tests, all green. That green makes you feel safe. It shouldn’t entirely, because somewhere in there are a hundred tests that no longer test anything — you deleted the feature they covered, or refactored the code out from under them, and the test quietly decayed into a no-op that passes because it can’t fail. This is test attrition, and it’s insidious precisely because it’s still green. A passing test that asserts nothing is worse than no test: it’s a false sense of security and a hundred wasted minutes on every CI run, forever, protecting nothing.
And here’s why this stopped being a slow, tolerable problem and became an acute one: agentic development. It is now trivial to ask an AI to write a thousand tests, and it will, cheerfully, in an afternoon. Some of them are sharp. Many of them assert almost nothing, or restate the implementation, or duplicate each other — and all of them are green, because a model asked to write passing tests is very, very good at writing tests that pass. The suite count goes vertical and the actual protection barely moves. We have automated the production of green checkmarks and called it coverage. The number of passing tests has never been the thing that makes software stronger. The number of tests that verifiably test something is — and in the agentic era those two numbers have come completely unmoored from each other. You can no longer eyeball a suite to tell them apart. You need to measure it, by execution, at a scale a human never could — which is exactly the machine this whole thing turned out to be.
Here is the beautiful part, and it’s the exact lesson from the second note. You do not need an opinion to find those hundred tests. You don’t need the critic to guess which ones are vacuous — the critic guesses, and the critic hallucinates. You need execution. Plant bugs across the code, run each test against them, and a test that kills zero of them is not “probably stale” — it is provably dead weight, a fact with a receipt. That’s the same tests-against-mutants machinery that grades adequacy, read down the other axis: not “does the suite catch bugs” but “does this test catch anything at all.” The hundred no-ops light up, proven, and you can delete them with a clear conscience — a leaner suite, an honest green, and a faster CI cycle that spends its minutes only on tests that are actually load-bearing.
So corral makes your suite stronger from both ends: it hands you the killing test you were missing, and it shows you — by execution, not opinion — the dead ones you can cut. Strength was never the number of tests. It’s the signal per test, and both directions raise it.
That’s real today, one run at a time. The “your tests suck” voice was only ever the attention-getter for “…here’s the one that’s missing, and here are the hundred you can safely delete.”
Now make it plural
Here’s where it stops being a clever local tool and starts being something with a floor under it. corral is client-server. The knowledge a gate produces doesn’t have to die in one run on one laptop.
A blind spot proven once — say, that length-only validators sail past the character-class rules — is a fact. Prove it, sign it, and it can become a test the whole network pulls. So your local suite gets stronger not just from what your run discovered, but from what everyone’s runs have already proven. A first-time user doesn’t start from zero; they start from a shared corpus of things that are known to catch real bugs.
The one-liner we kept circling, now that it’s finally clear:
corral makes your local tests stronger by pulling from a shared corpus of verified, signed findings.
Why “verified and signed” is the entire game
This is the part that separates a gift from a contagion, and it’s exactly the lesson the second note beat into us.
Pulling knowledge from a shared corpus is only as safe as the corpus is trustworthy. Pull from a black-box shared model, and you inherit its confident mistakes — the islice hallucination, but now baked into everyone’s tests, forever, un-inspectable. Pull from a corpus where every finding is proven by execution and signed, and you can only ever inherit proven facts — each one traceable to the exact run that proved it, tamper-evident, revocable if it’s ever shown wrong.
So verified + signed isn’t a compliance sticker on the value prop. It is the value prop. It’s the precondition that lets corral safely do the thing a learned black box structurally cannot: hand you knowledge from strangers that you are actually allowed to trust. A shared corpus of opinions is a liability. A shared corpus of signed, execution-proven findings is a commons.
The rigor and the generosity
We’ve had the rigorous half of this for a while, in Latin: nemo iudex in causa sua — no one may be judge in their own cause. It’s why you can trust the verdict.
This is its friendlier twin — the reason you’d want the verdict in the first place:
No one grades their own tests — and everyone’s tests get stronger from what’s already been proven.
One keeps it honest. One makes it worth installing.
What’s true today, and what we’re building
I’ll keep the same honesty the second note demanded. The first half is real now: point corral at your code and it will hand you the proven test your suite was missing. The second half — the shared corpus — is what we’re building: the machinery to distribute findings exists (it’s how the tool already shares knowledge between developers), but the audit corpus, and the careful work of sharing patterns and never your code across an org boundary, is ahead of us, not behind. I’m not going to sell you a flywheel that hasn’t spun. I’m going to tell you exactly what it’s shaped like and let it earn the claim.
The tool opened its mouth to tell you your tests were weak. That got your attention, and fair enough. But the reason it’s worth your keystrokes is quieter and better: it leaves your tests stronger than it found them — and it’s built so that, one signed finding at a time, it leaves everyone’s stronger too.
The critic got you in the door. The help is why you stay.