internal/prior at 4218eedc47, reviewed by claude-code, verified by codex.
Load merges priors on (parent sha, mutant id), but mutant ids are positional per run, so a second run's edits on an unchanged file are silently dropped, the paragraph's edit count undercounts what was tried, and empty fields of the surviving record are filled from the discarded one — producing a rendered line that places one run's hunk at another run's line, an edit that was never made.
#!/bin/sh
set -e
cat > internal/prior/zz_repro_idcollide_test.go <<'EOF'
package prior
import (
"crypto/sha256"
"encoding/hex"
"fmt"
"path/filepath"
"strings"
"testing"
"github.com/pdbethke/corralai/internal/adequacy"
"github.com/pdbethke/corralai/internal/lang"
)
// Two runs recorded edits on the SAME file at the SAME bytes. Mutant ids in
// this build are positional per run (internal/testgen/parse_diag.go:141 mints
// "m%d" by index; internal/advpool/driver.go:2152 prefixes the shard), so both
// runs' first mutant is called "s0/m1" while being a different edit.
func TestZZReproPositionalIDsCollapseAcrossRuns(t *testing.T) {
dir := t.TempDir()
code := "def f(x):\n y = 1\n if x > 10:\n return 1\n return None\n"
h := sha256.Sum256([]byte(code))
sha := hex.EncodeToString(h[:])
// Run 1's record: a constant change, with no span recorded on the row.
if err := adequacy.WriteMutantSet(filepath.Join(dir, "run1.json"), adequacy.MutantSetFile{
Format: adequacy.MutantSetFormat,
Files: map[string]adequacy.MutantSetEntry{
"f.py": {ParentSHA256: sha, Mutants: []adequacy.RecordedMutant{
{ID: "s0/m1", Search: " y = 1", Replace: " y = 2"},
}},
},
}); err != nil {
t.Fatal(err)
}
// Run 2's record: a DIFFERENT edit, at line 42, that is again the first
// mutant of shard 0.
if err := adequacy.WriteMutantSet(filepath.Join(dir, "run2.json"), adequacy.MutantSetFile{
Format: adequacy.MutantSetFormat,
Files: map[string]adequacy.MutantSetEntry{
"f.py": {ParentSHA256: sha, Mutants: []adequacy.RecordedMutant{
{ID: "s0/m1", Span: lang.LineRange{Start: 42, End: 42}, Search: " if x > 10:", Replace: " if x >= 10:"},
}},
},
}); err != nil {
t.Fatal(err)
}
p, err := Load(dir)
if err != nil {
t.Fatal(err)
}
tried, err := p.For("f.py", sha)
if err != nil {
t.Fatal(err)
}
para := Render(tried)
fmt.Printf("2 edits are on record for these exact bytes; the prior hands the generator %d.\n", len(tried))
fmt.Printf("--- the paragraph the generator reads ---\n%s----------------------------------------\n", para)
if len(tried) == 2 {
t.Fatalf("no defect: both edits survived the merge")
}
if len(tried) != 1 {
t.Fatalf("unexpected edit count %d", len(tried))
}
if strings.Contains(para, "x >= 10") {
t.Fatalf("no defect: run 2's edit is present")
}
fmt.Println("DEFECT A (dropped): run 2's edit `if x > 10:` -> `if x >= 10:` is absent from the prior; the next run is free to re-roll exactly that fault.")
if !strings.Contains(para, "(1 edit(s) from earlier runs)") {
t.Fatalf("no defect: header does not undercount:\n%s", para)
}
fmt.Println("DEFECT B (false count): the header says 1 edit from earlier runs. 2 were tried on these bytes.")
if !strings.Contains(para, "line 42") || !strings.Contains(para, "y = 2") {
t.Fatalf("no defect: no grafted span:\n%s", para)
}
fmt.Println("DEFECT C (fabricated edit): the surviving line reports run 1's hunk `y = 1` -> `y = 2` AT LINE 42, which is run 2's span, grafted by prior.go:93-95. That edit was never made at line 42.")
}
EOF
go test -count=1 -run TestZZReproPositionalIDsCollapseAcrossRuns ./internal/prior/ok github.com/pdbethke/corralai/internal/prior 0.005s
I checked Load's merge logic at internal/prior/prior.go:76-99 and the ID producers named by the reviewer. Within each path, the key is exactly ParentSHA256 plus ID, and collisions retain the incumbent while filling its empty fields from the later row. Consequently independently generated positional IDs can collapse distinct edits and can combine a hunk with another edit's span. The supplied reproduction exercises precisely that case and its assertions agree with the code.
confirmed by pdbethke — real: the merge key (sha, id) with positional ids (s0/m1 repeats every run) drops a later run's edits on an unchanged file; the script's test held and codex traced the merge. fix: the key carries the edit's span and hunk
ref d0f2993a4adf00264058ceafe5e93bf7372a3773aae0b89db90ef99e84637aa3#R1
Digest omits Search/Replace, the very text Render quotes to the generator, so two priors that produce different prompts share one fingerprint — defeating the package's stated guarantee that the digest in the cache key keeps a cached verdict from being served across the prior boundary, and making the verdict's disclosure unable to distinguish a hunk-quoting prior from a span-only one.
#!/bin/sh
set -e
cat > internal/prior/zz_repro_digest_test.go <<'EOF'
package prior
import (
"fmt"
"testing"
"github.com/pdbethke/corralai/internal/lang"
)
// The same edit as a ledger-only prior (span and shape, no hunk — see
// cmd/corral/certify_repo_bundle.go:250-255, which leaves Code empty on every
// ledger row) and as a merged document+ledger prior (the hunk restored).
func TestZZReproDigestIgnoresTheHunkItRenders(t *testing.T) {
ledgerOnly := Tried{
Path: "f.py", ParentSHA256: "e3b0c44298fc1c14", ID: "s0/m1",
Span: lang.LineRange{Start: 12, End: 12}, Shape: "constant-changed",
Outcome: "survived",
}
withHunk := ledgerOnly
withHunk.Search, withHunk.Replace = " limit = 10", " limit = 11"
a, b := []Tried{ledgerOnly}, []Tried{withHunk}
da, db := Digest(a), Digest(b)
ra, rb := Render(a), Render(b)
fmt.Printf("digest(prior WITHOUT the hunk) = %s\n", da)
fmt.Printf("digest(prior WITH the hunk) = %s\n", db)
fmt.Printf("--- paragraph A (no hunk) ---\n%s--- paragraph B (hunk) ---\n%s--------------------------\n", ra, rb)
if ra == rb {
t.Fatalf("no defect: the two priors render the same paragraph")
}
if da != db {
t.Fatalf("no defect: the digests differ (%s vs %s)", da, db)
}
fmt.Println("DEFECT: two priors that hand the generator DIFFERENT text carry ONE digest.")
fmt.Println("prior.go:217-226 hashes path, parent sha, id, span, shape, outcome, killedBy and proven")
fmt.Println("— never Search/Replace, which prior.go:263-264 is what it actually quotes to the model.")
fmt.Println("That digest is the run's whole disclosure and the cache key's prior component")
fmt.Println("(cmd/corral/certify_repo.go:5407-5411; cmd/corral/certify_local.go:1338), so a verdict")
fmt.Println("produced under paragraph B is servable to a run that only ever saw paragraph A,")
fmt.Println("contradicting this package's doc comment at prior.go:19-21.")
}
EOF
go test -count=1 -run TestZZReproDigestIgnoresTheHunkItRenders ./internal/prior/ok github.com/pdbethke/corralai/internal/prior 0.005s
Digest at internal/prior/prior.go:217-227 hashes neither Search nor Replace, while Render at lines 263-265 conditionally emits both. Thus changing only the hunk changes the generator-visible prompt without changing the digest. I found no downstream augmentation of this digest that restores the omitted prompt content before it is used for disclosure or cache separation.
confirmed by pdbethke — real: Digest omits Search/Replace while Render quotes them, so two different prompts share a fingerprint and the cache key cannot tell them apart. fix: hash the hunk
ref d0f2993a4adf00264058ceafe5e93bf7372a3773aae0b89db90ef99e84637aa3#R2
For returns ErrDifferentVersion whenever no recorded edit for the path carries the current sha, including when the records carry NO parent hash at all, so certify reports "refused: recorded against a different version of this file" about a file that never changed — an accusation the record does not support.
For at internal/prior/prior.go:194-212 treats an empty ParentSHA256 as merely unequal to the requested SHA and returns ErrDifferentVersion when no other row matches. The consumer at cmd/corral/certify_local.go:1334 renders that condition specifically as "recorded against a different version of this file." Empty parent hashes are representable and are explicitly accommodated elsewhere for pre-hash or hand-built records, so I found no invariant that converts absence of a hash into evidence of different bytes.
confirmed by pdbethke — real: an empty ParentSHA256 is reported as 'a different version of this file' when it is 'no version recorded'; the message is false for that case
ref d0f2993a4adf00264058ceafe5e93bf7372a3773aae0b89db90ef99e84637aa3#R3
The ledger reader drops every mutant whose outcome is not exactly "killed" or "survived", so an edit that was planted and errored, timed out or was ruled invalid is left out of the prior entirely and the next generator is free to re-roll it, while the paragraph's count presents the remainder as everything that was tried.
The ledger loop at internal/prior/prior.go:112-115 unconditionally skips every outcome other than killed or survived. MutantRow explicitly admits invalid and timed_out outcomes, so those recorded edits cannot reach Render and may be generated again. The reviewer's word "errored" is not a documented MutantRow outcome, and invalid does not necessarily mean the test command ran, but those qualifications do not refute the stated omission of tried-but-ungraded edits, especially timed-out edits.
confirmed by pdbethke — real: invalid and timed-out mutants were planted and are part of what was tried; skipping them lets the next generator re-roll them
ref d0f2993a4adf00264058ceafe5e93bf7372a3773aae0b89db90ef99e84637aa3#R4
Render truncates at MaxRendered after Load sorted the edits by ascending line, so the 40 disclosed edits are always the file's topmost ones and the tail of the file is never disclosed — while the same paragraph instructs the generator to plant faults elsewhere, steering it toward exactly the region whose priors were cut.
Load sorts by increasing Span.Start at internal/prior/prior.go:150-156, and Render stops after the first MaxRendered entries at lines 245-249. For a file with more than 40 positioned priors, this deterministically favors the earliest lines and summarizes later entries without their locations or shapes, while the prompt directs generation toward other places and shapes. I found no rotation, sampling, or reverse ordering in either function.
confirmed by pdbethke — real: the cut at MaxRendered after a line sort always drops the file's tail; the summary line must say where the undisclosed edits are
ref d0f2993a4adf00264058ceafe5e93bf7372a3773aae0b89db90ef99e84637aa3#R5
oneLine truncates on a byte index (s[:77]) rather than a rune boundary, so a hunk containing multi-byte characters past the cut is spliced mid-rune and the prompt carries invalid UTF-8 where the source had a character.
oneLine at internal/prior/prior.go:281-287 checks byte length and slices s[:77]. Go string slicing is byte-based, so when byte 77 falls inside a multi-byte UTF-8 encoding, the returned string contains an incomplete rune followed by the ellipsis. No rune-aware conversion or UTF-8 repair occurs afterward.
confirmed by pdbethke — real: byte-indexed truncation splits a rune
ref d0f2993a4adf00264058ceafe5e93bf7372a3773aae0b89db90ef99e84637aa3#R6
Recorded so that "found nothing here" can be told apart from "never looked here".
The package's two stated honesty rules do not both hold. The same-bytes rule (For) is enforced at both doors I could find and is sound. The disclosure rule is not: the merge key that Load uses to fold several runs together is (parent sha, mutant id), and mutant ids in this build are POSITIONAL per run — internal/testgen/parse_diag.go:141 mints "m%d" by index and internal/advpool/driver.go:2152 prefixes the shard, so run 2 on an unchanged file re-issues "s0/m1", "s0/m2", … for entirely different edits. Finding 1 shows what that costs, and it is the exact loop `certify --repo` runs by default (cmd/corral/ledger.go:76-78 — every run writes into .corral/ledger/ and reads earlier entries there as its prior): the second run's edits are silently discarded, the paragraph's "(N edit(s) from earlier runs)" undercounts, and — because the merge grafts field by field — a surviving record can end up carrying one run's hunk at another run's line, an edit that was never tried. That last part is the package telling the generator something false about the file it is examining. Finding 2 is the other half of the same disclosure promise: the package doc (lines 19-21) says the digest is in the cache key so a cached verdict is never served across the prior boundary, but Digest hashes everything about an edit EXCEPT the hunk, which is the part Render actually quotes to the model — two priors that produce visibly different prompts share one fingerprint, so the cache cannot tell them apart and neither can a reader of the verdict. Findings 3-6 are smaller: a refusal message that says the wrong thing when a row simply carries no parent hash, tried-but-not-graded edits dropped from the disclosure, a truncation that systematically hides the bottom of the file while asking for faults elsewhere, and a byte-sliced truncation. I did not find a way past the same-bytes rule itself.
The review withstands verification. R1 and R2 are directly demonstrated by the supplied tests and match the implementation. R3–R6 also follow from the relevant control flow; I found no validation, alternate ordering, rendering policy, or UTF-8 safeguard that refutes them. R4's examples are slightly loose—"errored" is not one of the documented ledger outcomes, and an invalid mutant may fail before test execution—but the core claim that recorded invalid and timed-out edits are excluded from the prior remains true.