← the record

Review 8be2189163b0

internal/gate at 6951ca4c11, reviewed by claude-code:claude-fable-5-1, verified by codex:gpt-6-astra.

repo
git@github.com:pdbethke/corralai
commit
6951ca4c1173d359100724f0a64c64f605f57246
scope
internal/gate · go
reviewer
claude-code:claude-fable-5-1 (2.1.272 (Claude Code))
verifier
codex:gpt-6-astra (codex-cli 0.154.0) different model than the reviewer different vendor: anthropic → openai
substrate
workspace (a detached worktree at the commit; not a jail)
started
2026-09-15 13:07 UTC
audited party
Peter D Bethke · co-authored: Claude Fable 5.1
files shown
14 file(s)

Findings (7)

R1 high reproduced confirmed by a person

Two policies for the same repo that both omit context= are accepted by ParsePolicyEnv and both normalize to corral/gate, so the poller runs only the one whose variable name sorts first and skips the second on every head forever; the first policy's status is posted under the shared context and stands in for a check that never ran.

where
internal/gate/config.go:99
tier
REPRODUCED (declared REPRODUCED)
the script that was executed
#!/bin/sh
set -e
cat > internal/gate/zz_repro_samectx_test.go <<'EOF'
package gate

import (
    "context"
    "fmt"
    "path/filepath"
    "testing"
    "time"
)

type reproSameCtxLister struct{ prs []PRRef }

func (l reproSameCtxLister) ListOpenPRs(ctx context.Context, repoURL, base string) ([]PRRef, error) {
    return l.prs, nil
}

func TestReproTwoPoliciesOneRepoSharedDefaultContext(t *testing.T) {
    pols, bad := ParsePolicyEnv([]string{
        PolicyEnvPrefix + "LINT=repo=o/r,cmd=true",
        PolicyEnvPrefix + "TEST=repo=o/r,cmd=go test ./...",
    })
    if len(bad) != 0 || len(pols) != 2 {
        t.Fatalf("parser refused the pair (bad=%v n=%d): defect absent", bad, len(pols))
    }
    store, err := OpenStore(filepath.Join(t.TempDir(), "g.db"))
    if err != nil {
        t.Fatal(err)
    }
    defer store.Close()
    ran := map[string]int{}
    p := &Poller{
        Policies: pols,
        List:     reproSameCtxLister{prs: []PRRef{{Number: 1, HeadSHA: "abc", Base: "main"}}},
        Store:    store,
        Run: func(ctx context.Context, repoURL string, pol Policy, pr PRRef) error {
            ran[pol.CheckCmd]++
            return store.Save(Run{Repo: pol.Repo, HeadSHA: pr.HeadSHA, PR: pr.Number, Context: pol.Context, Passed: true, StatusPosted: true, RanAt: time.Unix(0, 0)})
        },
    }
    for i := 0; i < 3; i++ {
        _ = p.Tick(context.Background())
    }
    fmt.Printf("ParsePolicyEnv accepted 2 policies for o/r, both under context %q, bad=%v\n", pols[0].normalized().Context, bad)
    fmt.Printf("runs after 3 ticks: cmd 'true' = %d, cmd 'go test ./...' = %d (the second check never runs)\n", ran["true"], ran["go test ./..."])
    if ran["true"] != 1 || ran["go test ./..."] != 0 {
        t.Fatalf("both policies ran: defect absent")
    }
}
EOF
exec go test ./internal/gate -run TestReproTwoPoliciesOneRepoSharedDefaultContext -count=1 -v
what it printed
=== RUN   TestReproTwoPoliciesOneRepoSharedDefaultContext
ParsePolicyEnv accepted 2 policies for o/r, both under context "corral/gate", bad=[]
runs after 3 ticks: cmd 'true' = 1, cmd 'go test ./...' = 0 (the second check never runs)
--- PASS: TestReproTwoPoliciesOneRepoSharedDefaultContext (0.03s)
PASS
ok  	github.com/pdbethke/corralai/internal/gate	0.031s

refutation — codex:gpt-6-astra STANDS

I checked whether normalization or production wiring distinguishes the two policies. ParsePolicyEnv sorts and appends both independently (internal/gate/config.go:97–105), and normalizeContext gives both omitted contexts corral/gate (internal/gate/types.go:101–105). Poller.Tick looks up only repo, head and context, then skips a delivered row (internal/gate/poller.go:68–77). StartGate passes the policies directly to that poller (internal/brain/gate.go:122–128). For the named pair, the first delivered check suppresses the second.

human verdict

confirmed by pdbethke — real, reproduced, high: two policies for one repo that omit context= both became corral/gate, so the second never ran and the first's status stood in for it. Fixed in #347: ParsePolicyEnv refuses the second and the poller holds the same rule; the tests failed on main first, and a negative control shows the poller test fails without the guard.

ref 8be2189163b0bce902053cfbd385775772977e46b8424cbedeb88396a1377f49#R1

R2 medium reproduced confirmed by a person

An undelivered verdict is 'retried' by re-invoking Runner.Run, which re-checks out, re-runs the jail and appends a NEW signed record on every tick for as long as the status post fails, so a permanent forge refusal (403 missing statuses scope, 422 on the context) becomes an unbounded re-certify loop and even a transient outage produces one signed record per tick instead of one re-post.

where
internal/gate/poller.go:79
tier
REPRODUCED (declared REPRODUCED)
the script that was executed
#!/bin/sh
set -e
cat > internal/gate/zz_repro_repost_test.go <<'EOF'
package gate

import (
    "context"
    "errors"
    "fmt"
    "path/filepath"
    "testing"
    "time"
)

type reproRLister struct{ prs []PRRef }

func (l reproRLister) ListOpenPRs(ctx context.Context, repoURL, base string) ([]PRRef, error) {
    return l.prs, nil
}

type reproRCheckout struct{}

func (reproRCheckout) CheckoutPR(ctx context.Context, repoURL string, pr int, sha, dest string) error {
    return nil
}

type reproRJail struct{ calls int }

func (j *reproRJail) Run(ctx context.Context, command, ws string, net bool, to time.Duration) (int, string, error) {
    j.calls++
    return 0, "ok", nil
}

type reproRCert struct{ calls int }

func (c *reproRCert) Certify(ctx context.Context, repo, commit, command string, exit int, digest string) (int64, string, error) {
    c.calls++
    return int64(c.calls), "head", nil
}

type reproRStatus struct{ posts int }

func (s *reproRStatus) SetCommitStatus(ctx context.Context, repoURL, sha, c, state, target, desc string) error {
    s.posts++
    return errors.New("403 Resource not accessible by integration")
}

func TestReproPermanentPostFailureRerunsJailAndRecertifiesEveryTick(t *testing.T) {
    store, err := OpenStore(filepath.Join(t.TempDir(), "g.db"))
    if err != nil {
        t.Fatal(err)
    }
    defer store.Close()
    jail := &reproRJail{}
    cert := &reproRCert{}
    status := &reproRStatus{}
    r := &Runner{Checkout: reproRCheckout{}, Jail: jail, Certify: cert, Status: status, Store: store,
        RecordURL: func(a, b string) string { return "http://x" }, Now: func() time.Time { return time.Unix(0, 0) }}
    p := &Poller{
        Policies: []Policy{{Repo: "o/r", CheckCmd: "true"}},
        List:     reproRLister{prs: []PRRef{{Number: 1, HeadSHA: "abc", Base: "main"}}},
        Store:    store,
        Run:      r.Run,
    }
    ticks := 3
    for i := 0; i < ticks; i++ {
        _ = p.Tick(context.Background())
    }
    fmt.Printf("after %d ticks with a status post that always fails: jail runs=%d, signed records appended=%d, status posts=%d\n", ticks, jail.calls, cert.calls, status.posts)
    fmt.Println("a 're-delivery' of an already computed and signed verdict should re-post once per tick and run the jail / certify exactly once")
    if jail.calls < ticks || cert.calls < ticks {
        t.Fatalf("jail/certify were not re-run per tick: defect absent")
    }
}
EOF
exec go test ./internal/gate -run TestReproPermanentPostFailureRerunsJailAndRecertifiesEveryTick -count=1 -v
what it printed
=== RUN   TestReproPermanentPostFailureRerunsJailAndRecertifiesEveryTick
2026/09/15 09:11:35 gate: poller: run o/r#1@abc: 403 Resource not accessible by integration
2026/09/15 09:11:35 gate: poller: o/r@abc ctx corral/gate was gated but its status never posted — re-delivering
2026/09/15 09:11:35 gate: poller: run o/r#1@abc: 403 Resource not accessible by integration
2026/09/15 09:11:35 gate: poller: o/r@abc ctx corral/gate was gated but its status never posted — re-delivering
2026/09/15 09:11:35 gate: poller: run o/r#1@abc: 403 Resource not accessible by integration
after 3 ticks with a status post that always fails: jail runs=3, signed records appended=3, status posts=6
a 're-delivery' of an already computed and signed verdict should re-post once per tick and run the jail / certify exactly once
--- PASS: TestReproPermanentPostFailureRerunsJailAndRecertifiesEveryTick (0.03s)
PASS
ok  	github.com/pdbethke/corralai/internal/gate	0.039s

refutation — codex:gpt-6-astra STANDS

I traced the retry through the real certifier to check for cached delivery or record deduplication. An undelivered row falls through to p.Run (internal/gate/poller.go:79–82), which checks out, executes and certifies again (internal/gate/runner.go:111–123). A failed final status post returns before MarkPosted (runner.go:150–153). The production certifier calls certifyBuild (internal/brain/gate.go:28–35), whose storage path inserts a record with a fresh sequence ID (internal/buildstore/store.go:132–136). Persistent posting failure therefore repeats execution and record creation while polling continues.

human verdict

confirmed by pdbethke — real, reproduced: an undelivered status was retried by re-running Run, appending a new signed record every tick while the forge refused. Fixed in #347: a row with a signed record (RecordID != 0; build ids start at 1) is re-posted by Runner.Redeliver and never re-run; TestASignedVerdictIsRedeliveredNotRecertified failed on main first.

ref 8be2189163b0bce902053cfbd385775772977e46b8424cbedeb88396a1377f49#R2

R3 low reproduced confirmed by a person

ParsePolicy locates the command only by the exact substrings "cmd=" and ",cmd=", so a value with a space after the comma ("repo=o/r, cmd=true") — whitespace every other field and the stray-field regex tolerate — is refused as "no cmd=" and that repo's gate is silently off apart from a log line.

where
internal/gate/config.go:152
tier
REPRODUCED (declared REPRODUCED)
the script that was executed
#!/bin/sh
set -e
cat > internal/gate/zz_repro_spacedcmd_test.go <<'EOF'
package gate

import (
    "fmt"
    "strings"
    "testing"
)

func TestReproSpaceBeforeCmdIsRefused(t *testing.T) {
    // Whitespace after the comma is accepted for every other field...
    if _, reason := ParsePolicy("repo=o/r, base=main,cmd=true"); reason != "" {
        t.Fatalf("spaced base= refused (%s): premise wrong", reason)
    }
    // ...and by the stray-field guard (", base=" after cmd= is detected)...
    if _, reason := ParsePolicy("repo=o/r,cmd=true, base=main"); !strings.Contains(reason, "after cmd=") {
        t.Fatalf("spaced stray field not detected (%q): premise wrong", reason)
    }
    // ...but not for cmd= itself.
    pol, reason := ParsePolicy("repo=o/r, cmd=true")
    fmt.Printf("ParsePolicy(%q) -> reason=%q policy=%+v\n", "repo=o/r, cmd=true", reason, pol)
    if !strings.Contains(reason, "no cmd=") {
        t.Fatalf("accepted or refused for another reason: defect absent")
    }
}
EOF
exec go test ./internal/gate -run TestReproSpaceBeforeCmdIsRefused -count=1 -v
what it printed
=== RUN   TestReproSpaceBeforeCmdIsRefused
ParsePolicy("repo=o/r, cmd=true") -> reason="no cmd= (a policy with no command would report a result for a check that never ran)" policy={Repo: Base:[] Context: CheckCmd: AllowNet:false TimeoutS:0}
--- PASS: TestReproSpaceBeforeCmdIsRefused (0.00s)
PASS
ok  	github.com/pdbethke/corralai/internal/gate	0.005s

refutation — codex:gpt-6-astra STANDS

I checked for whitespace normalization before command detection and for recovery by the caller. ParsePolicy only trims the whole value before requiring an exact cmd= prefix or ,cmd= substring (internal/gate/config.go:141–157), so the specified space prevents detection. Other fields are individually trimmed at config.go:174–183. ParsePolicyEnv skips the rejected policy at config.go:101–103, and the daemon logs the rejection at internal/wranglerd/daemon.go:672–675. These paths support the finding.

human verdict

confirmed by pdbethke — real, and now FIXED: fixed in #359 (322429ca): cmd= is found with the same whitespace tolerance as every other field. Execution on main 150127f5 (2026-10-07): corral review recheck of this finding's own script -> no-longer-reproduces (ParsePolicy("repo=o/r, cmd=true") now returns the command, reason empty); TestASpaceBeforeCmdIsStillACommand passes.

ref 8be2189163b0bce902053cfbd385775772977e46b8424cbedeb88396a1377f49#R3

R4 low no script confirmed by a person

net= silently maps every value other than "true"/"1" ("yes", "True", "TRUE", "on") to no-network, unlike timeout= which refuses a bad value, so an operator who wrote net=True gets a gate that fails every network-needing check with no message pointing at the policy.

where
internal/gate/config.go:197
tier
CODE-READ (declared CODE-READ)

refutation — codex:gpt-6-astra STANDS

I checked whether subsequent normalization validates or corrects net=True. The parser sets AllowNet using only exact comparisons with true and 1 (internal/gate/config.go:196–197), returns the policy without a diagnostic, and Policy.normalized changes only Context (internal/gate/types.go:84–86). Runner passes AllowNet directly to the jail (internal/gate/runner.go:116). The default's deliberate fail-closed direction does not refute the silent acceptance of an unrecognized explicit value.

human verdict

confirmed by pdbethke — real, and now FIXED: fixed in #359 (322429ca): net= takes a boolean (strconv.ParseBool) and refuses anything else, as timeout= does. Execution on main 150127f5 (2026-10-07): TestNetRefusesAValueItDoesNotUnderstand passes. No recorded script (CODE-READ); the regression test is the evidence.

ref 8be2189163b0bce902053cfbd385775772977e46b8424cbedeb88396a1377f49#R4

R5 low no script confirmed by a person

migrateKey runs CREATE/INSERT/DROP/RENAME as four autocommitted statements: a crash after DROP TABLE gate_runs leaves every row in gate_runs_wide while the next OpenStore creates an empty gate_runs and sees a wide key, silently orphaning the dedupe history so every open head is re-gated and re-certified; a crash after the CREATE makes every later open fail on "gate_runs_wide already exists", disabling the gate until someone edits the file by hand.

where
internal/gate/store.go:117
tier
HYPOTHESIS (declared HYPOTHESIS)

refutation — codex:gpt-6-astra STANDS

I traced both proposed restart states through OpenStore. migrateKey executes the four statements through separate db.Exec calls without a transaction (internal/gate/store.go:117–140). After a committed DROP, OpenStore creates a fresh wide-key gate_runs (store.go:29–39), and migrateKey returns upon detecting its context key (store.go:114–115), leaving gate_runs_wide unused. After only the temporary CREATE commits, the retained narrow table triggers migration again and the unconditional CREATE encounters the existing temporary table. I did not execute crash injection, but the restart logic does not refute either scenario.

human verdict

confirmed by pdbethke — real, and now FIXED: fixed in #359 (322429ca): migrateKey runs in one transaction, and OpenStore repairs either interrupted state left by an older binary. Execution on main 150127f5 (2026-10-07): TestAMigrationInterruptedAfterCreateIsRetried, TestAMigrationInterruptedAfterDropKeepsItsHistory and TestAMigrationInterruptedBeforeRenameIsFinished pass.

ref 8be2189163b0bce902053cfbd385775772977e46b8424cbedeb88396a1377f49#R5

R6 low no script confirmed by a person

strayFieldAfterCmd only matches a lowercase field name preceded by a comma, so "cmd=true,Base=release" or "cmd=true\nbase=release" is swallowed into the command as a shell word/assignment and the policy gates every base while the operator named one — the exact outcome the guard exists to report — while a legitimate command containing ",timeout=" or ",base=" inside an argument is refused.

where
internal/gate/config.go:57
tier
CODE-READ (declared CODE-READ)

refutation — codex:gpt-6-astra STANDS

I checked whether another parser stage catches the named spellings or accounts for shell quoting. The guard requires a comma followed by a lowercase known field (internal/gate/config.go:50–57), and only the text before cmd= undergoes field parsing (config.go:154,174). Consequently Base= after a comma and base= after only a newline remain command text, leaving Base unset. The same unquoted regex also rejects matching text inside legitimate quoted arguments. The comma example may fail as a shell command, but that does not refute its acceptance with unrestricted policy bases; the newline example remains a valid command followed by an assignment.

human verdict

confirmed by pdbethke — real, and now FIXED: fixed in #359 (322429ca): the stray-field guard matches any case after a comma and a field at a line start. Execution on main 150127f5 (2026-10-07): TestAStrayFieldIsCaughtAfterACommaInAnyCaseAndAtALineStartInLowercase passes.

ref 8be2189163b0bce902053cfbd385775772977e46b8424cbedeb88396a1377f49#R6

R7 low no script confirmed by a person

GetBySHA, kept for the read endpoint's "was this head gated" question, returns whichever context's row has the latest ran_at, so with two policies on one repo it can report Passed=true for a head whose other context failed, depending only on which check finished last.

where
internal/gate/store.go:172
tier
CODE-READ (declared CODE-READ)

refutation — codex:gpt-6-astra STANDS

I checked whether the caller aggregates contexts or exposes the selected context to qualify Passed. GetBySHA explicitly selects the latest row by ran_at (internal/gate/store.go:169–173). GateRunHandler returns that row's Passed, PR and RecordID without aggregation or context (internal/brain/gate.go:146–150,164–175). Although the poller correctly uses GetByHead and this endpoint does not decide forge eligibility, the finding specifically concerns the read result, which can indeed report true while another context failed.

human verdict

confirmed by pdbethke — real, and now FIXED: fixed in #359 (322429ca): a head with two checks has two answers (ListBySHA), and each status links to its own check. Execution on main 150127f5 (2026-10-07): TestListBySHAReturnsEveryCheck, TestEachStatusLinksToItsOwnCheck (internal/gate), TestGateRunWithoutAContextPassesOnlyIfEveryCheckPassed and TestGateRunForOneContextAnswersForThatCheck (internal/brain) pass.

ref 8be2189163b0bce902053cfbd385775772977e46b8424cbedeb88396a1377f49#R7

Looked at and could not break (9)

Recorded so that "found nothing here" can be told apart from "never looked here".

Reviewer's opinion

The package has been hardened round after round against 'a rule at one door and not the other', and most of those doors now hold: the context normalizer is shared, the timeout floor is applied where the policy is acted on, an empty command is refused at the runner, and the primary key really is widened. What is left is one more instance of the same shape, one level up, plus a retry that does more than it says. Finding 1 (high, reproduced): ParsePolicyEnv accepts two CORRALAI_GATE_POLICY_<NAME> variables naming the same repo with no context=, both normalize to corral/gate, and the poller runs whichever sorts first and skips the other forever — the first policy's status stands for both, so a weaker check's green covers a check that never ran. The parser is the only door that can see the pair, and it does not look. Finding 2 (medium, reproduced): the R4/R5 fix says an undelivered verdict is 'retried' and 're-delivered', but Tick re-invokes Runner.Run, which re-checks out, re-runs the jail and re-certifies a new signed record on every tick for as long as the status post fails; a permanent forge refusal (403 missing statuses scope, 422 on the context) turns that into the unbounded re-certify loop round two rated high, and even a transient outage appends one signed record per tick. Finding 3 (low, reproduced): a value written 'repo=o/r, cmd=true' — a space after the comma, which every other field and the stray-field regex tolerate — is refused as 'no cmd=', so that repo's gate is silently off apart from a log line. Findings 4–7 are code-read or hypothesis, all low: net= treats any value but true/1 as false without complaint, the migration is not transactional, the stray-field guard is case- and comma-sensitive, and GetBySHA answers 'was this head gated' with whichever context ran last.

Verifier's opinion

R1–R7 stand after read-only inspection. I traced normalization and deduplication for R1, production certification for R2, parsing and daemon handling for R3–R4, migration restart paths for R5, command preservation for R6, and the read endpoint for R7. I found no refutation of the stated behaviors. R5 remains an unexecuted crash hypothesis; no tests or code were executed.