Limitations & roadmap

Corralai is v0.1 — solo-maintained and moving fast. The foundations are in place; most of what’s ahead is surfacing them, not re-architecting. This page is the honest version: what it can’t do yet, and what’s next.

What v0.1 is

• Certify by execution works end to end: corral certify -- <check> and corral certify --local’s adversarial pool (mutant-generator → your suite in the jail → test-writer → decorrelated test-critic) both mint a signed, independently-verifiable record, offline-checkable against the published key with corral certify verify. The test-writer does corrective retries — it feeds the compiler’s own error back on a non-compiling attempt so it fixes the real problem rather than blindly repeating — so real third-party libraries certify reliably.
• The repo gate and the control gate run continuously against a brain, requiring the actual check to pass — not a self-report — before branch protection lets a PR merge.
• Multi-model, multi-language (Go, Python, Ruby, JS, TS), decorrelation enforced by construction — each role can be pointed at a different vendor — key-free on a local 7B if you want it. Coverage across those five is uneven and stated plainly: Go and Python are exercised hardest, while TypeScript, JavaScript and Ruby each rest on a single third-party repository so far. The evidence table is in the README.
• The jail (bwrap/container/sandbox-exec), the human gate on admin actions and merges, and a fully attributed action ledger.
• The learning loop (findings → human-approved skills), shared memory / search_memory, and durable history + replay — the recordings you can watch on this site are real exported runs.
• Embedded DuckDB analytics and optional MotherDuck fleet sync — plus a public DuckDB integration page that runs DuckDB in your browser (WebAssembly) over the real audit ledger + execution telemetry, with a live SQL box and ?q= deep-links.

What it does not claim yet

Honesty is part of the design (see the full Trust & security page and SECURITY.md):

• It does not adjudicate a survivor as a bug. A survivor is a planted mutant your suite didn’t catch — corral discloses it unadjudicated. Whether it’s a real untested edge or an equivalent mutant (a change nothing can observe, and so no test can catch) is the human’s call, not the tool’s. corral only calls a survivor a proven gap when its test-writer authors a compiling test that actually kills it; it never over-claims an unproven survivor as a real bug. And it does not promise to prove one. Turning a survivor into a proven gap requires the test-writer to author a test that genuinely kills it — on real third-party code, some runs manage it and some don’t. Zero proven gaps means “nothing was proven in this run”, never “your tests are fine.”
• It does not make an LLM immune to instructions in the content it reads. Fencing untrusted data is hardening, not a guarantee — the control is that unvetted content can never reach an authoritative position, and a fooled agent is contained (jailed, credential-less, PR-reviewed).
• It does not defend against a malicious insider holding a valid key. Cross-swarm attestation stops identity forgery, not an authorized brain publishing false claims (mitigated by advisory-only semantics, TTLs, and revocation).
• It assumes a single trust domain. Cross-swarm coordination assumes all participating swarms share one owner; it is not multi-tenant isolation.
• It has not been battle-tested at scale. The properties are proven by design review and adversarial tests, not (yet) by a hostile production adversary.

Operationally, v0.1 has real, named ceilings, not “coming soon” hand-waving:

• One active adversarial-pool run at a time, per hosted brain. start_adversarial_run refuses a second run while one is active — a deliberate scope limit, not a bug; --local has no such limit since it runs in-process off your own key.
• The certify kill-rate threshold is fixed, not per-repo tunable yet. A non-positive threshold is refused outright — auto-certifying any suite would defeat the human gate — but there’s no per-project dial on it today.
• No pentester role in the certify pool. The adversarial pool runs mutant-generator, test-writer, and test-critic; a dedicated pentester role (as distinct from mutation-testing your suite) isn’t part of it.
• Whole-repo scanning is much weaker in JavaScript/TypeScript than in Go, Python, or Ruby. certify --local audits any single file you name, in any of the five languages — you give it the path, so nothing is discovered. certify --repo must find the files first, pairing each source file with its test by naming convention, and those conventions fit some ecosystems far better than others. Measured on real repos: rubocop/rubocop 736 candidates, gin-gonic/gin 29, pallets/flask 9 — and expressjs/express zero. If a repo scan of your JS/TS project reports 0 candidates, that’s corral’s pairing limitation, not a verdict on your tests; audit files directly with --local --code <path> --test <path>, which never pairs. The express zero is pinned in CI so a future change can’t quietly paper over it.
• An authored test only counts if your runner actually collects it. A test that compiles and passes proves nothing if the project’s own test command never ran it — a project that confines discovery to a test root (pytest’s testpaths, jest’s roots) will silently ignore a file written elsewhere. corral writes its authored test into the directory your paired test already lives in, then proves the run reached it by planting deliberately invalid source at that path and checking your unmodified command reacts. When it doesn’t, the file is reported [TEST UNSOUND] and its proven count is withheld rather than printed as a clean zero.
• The language toolchain must be jail-visible. Installed system-wide under /usr, not --user/snap/pyenv — an invisible-to-the-sandbox toolchain fails the run closed rather than degrading; see Running it.
• Production runs no worker fleet today. The hosted brain’s gates and start_adversarial_run fan-out are real, tested code paths, but with no workers connected to a given deployment they have nothing to fan out to — dormant until workers are pointed at it, not a missing feature.
• Single-operator. Many-runs-at-once, budget caps, and mid-run steering are roadmap items, not shipped features.

Where it’s heading

From the roadmap (directional, not committed) — corral’s arc is from a certify CLI + a merge gate to a full accountability plane for AI-written code:

• The trustless tier. Ship the ledger head to an external, append-only, timestamped witness (Sigstore Rekor, or a shared MotherDuck warehouse) so tampering is detectable even if you don’t trust the brain — central-trust today becomes tamper-evident-against-everyone next.
• One-command agent onboarding. Independent, dev-driven agents join with deterministic passive telemetry via their own hooks — capture by instrumentation, never by an agent choosing to self-report.
• The MotherDuck accountability warehouse. Signed records from every dev, CI runner, and project federate into one shared, queryable warehouse — the same DuckDB schema, a DSN flip.
• The shared corpus. A blind-spot pattern proven once becomes a shared, versioned, signed skill every client’s audit can pull from — execution-proven, human-gated, never opaque weights.
• Operate the gate at scale. Model management (assign models to audit roles, an earned per-role leaderboard), cross-model evaluation, a review cockpit docked beside the diff.
• Ready for teams. Cost governance, concurrency & multi-tenancy, memory hygiene on the shared corpus.

The through-line: every capability is the same pattern — brain-mediated, human-gated, attributed, share the capability and hold the credential — and increasingly just a query over one attributed ledger.

Want to shape it? Issues and verified-harness PRs are welcome on GitHub.