← the record

Review e4bd04e8e47a

internal/adequacy at 201253a6e5, reviewed by codex, verified by gemini-3.6-flash.

repo
git@github.com:pdbethke/corralai
commit
201253a6e54d26c0b67e1dac9ce489c8d58a9542
scope
internal/adequacy · go
reviewer
codex (codex-cli 0.152.0)
verifier
gemini-3.6-flash different model than the reviewer different vendor: openai → google
substrate
workspace (a detached worktree at the commit; not a jail)
started
2026-09-07 14:31 UTC
files shown
44 file(s)

Findings (1)

R1 high no script

A process crash during workspace-substrate mutation can leave the checkout mutated, and the next run has no durable journal with which to detect or restore that stale mutant, allowing results to be computed from an already-corrupted baseline.

where
internal/adequacy/workspace.go
tier
HYPOTHESIS (declared HYPOTHESIS)

refutation — gemini-3.6-flash REFUTED

The claim that an abrupt process crash leaves a stale mutant on disk that causes subsequent runs to evaluate against an already-corrupted baseline is false. Before scoring any mutants, Score executes a baseline test run passing compliantCode in the candidate files map. WorkspaceRunner.RunTest applies candidate files to disk at the start of execution, which overwrites the target file with compliantCode before the baseline test command is executed. Consequently, any leftover mutation from a prior process crash is automatically cleared and overwritten with the clean baseline code before any baseline or mutant scoring occurs.

the refutation's script
cat << 'EOF' > internal/adequacy/refute_r1_test.go
package adequacy

import (
	"context"
	"os"
	"path/filepath"
	"testing"
)

func TestRefuteCrashCorruptedBaseline(t *testing.T) {
	root := wsTree(t, map[string]string{"a.py": "ORIGINAL\n"})
	w := NewWorkspaceRunner(root, 0)

	corruptedFile := filepath.Join(root, "a.py")
	if err := os.WriteFile(corruptedFile, []byte("STALE_MUTANT_CORRUPTED\n"), 0644); err != nil {
		t.Fatalf("failed to simulate stale mutant: %v", err)
	}

	mutants := []Mutant{{ID: "m1", Replace: "MUTANT\n"}}
	testCmd := []string{"sh", "-c", `grep -q ORIGINAL a.py || { echo "FAILED"; exit 1; }`}

	rep, err := Score(context.Background(), w, map[string]string{}, "a.py", "ORIGINAL\n", mutants, testCmd)
	if err != nil {
		t.Fatalf("Score failed: %v", err)
	}

	if !rep.CompliantPass {
		t.Fatalf("CompliantPass should be true despite pre-existing stale mutant on disk")
	}

	content, err := os.ReadFile(corruptedFile)
	if err != nil {
		t.Fatalf("ReadFile failed: %v", err)
	}
	if string(content) != "ORIGINAL\n" {
		t.Fatalf("expected file to be restored to ORIGINAL\\n, got %q", string(content))
	}
}
EOF
go test -v ./internal/adequacy -run TestRefuteCrashCorruptedBaseline

ref e4bd04e8e47af317c213ae57944207dcc6da7fd85e5f236fabb61df3281b05a1#R1

Looked at and could not break (1)

Recorded so that "found nothing here" can be told apart from "never looked here".

Reviewer's opinion

Finding 1 identifies a high-impact workspace-integrity risk: an abrupt process crash can leave a mutation in the real checkout and a later run may accept that stale state as its baseline. This remains a HYPOTHESIS because the review environment exposed neither repository file-reading nor command execution capability sufficient to inspect or reproduce the scoped implementation.

Verifier's opinion

Finding 1 is REFUTED: WorkspaceRunner and Score do not rely on a durable journal to recover from a stale mutant state left by an abrupt process crash, because every scoring execution begins by explicitly overwriting the checkout target with compliantCode during its baseline phase.