feat(review-workflow): add fail-closed proofs for blind PR queue review (#173)

Adds review_proofs.py with pure, fail-closed helpers for the reviewer-side
blind queue workflow:

- verify_pinned_head_checkout: proves HEAD == pinned PR head (full 40-hex,
  no prefix matches) and diff base == PR base; any mismatch blocks
  review/merge.
- assess_inventory_completeness: exhaustive "only PRs found" claims require
  every configured repo listed with stated filters, proven pagination, and
  reported open-PR totals.
- assess_validation_report: validation results are unclaimable unless the
  exact command was stated and its output read; missing pass/fail/skip
  counts, unjustified ignored paths, or unexplained deviation from the
  canonical command downgrade the report to weak.
- assess_self_review_contamination: contamination must be evidence-backed;
  bare same-session claims (either way) yield "unknown" with a
  choose-another-PR-or-stop action, never an assumed verdict.
- build_final_report: distinguishes identity eligibility, author/reviewer
  distinctness, session contamination, validation-on-pinned-head, merge,
  and issue verification; grades "A" only when every proof is present,
  downgrades otherwise, and marks a merge without allowing proofs as a
  blocked violation.

tests/test_review_proofs.py covers all seven harness assertions from the
issue (unchecked-out pinned head blocks merge; multi-repo inventory;
pagination; missing counts flagged weak; evidence-required contamination;
unread output cannot claim validation passed; ignored paths need
justification).

SKILL.md section F and templates/review-pr.md now require the checkout,
inventory, validation-evidence, and contamination proofs and the
distinguished final report. No review/merge safety gate is weakened.

Closes #173

Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
2026-07-05 14:30:34 -04:00
co-authored by Claude Fable 5
parent ae7a390495
commit ccc5ef79df
4 changed files with 880 additions and 8 deletions
@@ -20,13 +20,28 @@ Steps:
*If the current identity does not match the required role (or is the PR author), STOP. Relaunch/switch to the correct profile first.*
2. Verify your authenticated identity (whoami) and the active profile.
3. Fetch the PR facts: PR author, head SHA, state (must be open), base branch.
Pin the head SHA in your notes; every later step validates THAT SHA.
4. If authenticated user == PR author → STOP (no self-review).
4. scripts/worktree-review <pr-head-branch> # detached, branches/review-*
Session-contamination claims must be evidence-backed (#173): if you
cannot evidence whether this session authored/touched the PR branch,
report contamination as UNKNOWN (not contaminated, not clean) and choose
another PR or stop.
5. scripts/worktree-review <pr-head-branch> # detached, branches/review-*
cd branches/review-<pr-head-branch-slug>
5. Confirm the worktree is clean. Inspect the FULL diff; confirm scope matches
6. Checkout proof (#173) — prove and state, before any diff review or
validation:
- pinned PR head SHA (from Gitea)
- local checkout SHA (git rev-parse HEAD)
- HEAD == pinned PR head SHA
- diff base == the PR base branch
If HEAD does not match the pinned head → STOP before review/merge.
7. Confirm the worktree is clean. Inspect the FULL diff; confirm scope matches
issue #<n>; flag any unrelated files, secrets, or formatting churn. Check that the PR body correctly uses Gitea-closing keywords (`Closes #N` or `Fixes #N`) instead of non-closing ones (`Implements #N`, `Refs #N`).
6. Run the test suite; note results.
7. Post the review verdict: approve only if scope is clean and checks pass;
8. Run the test suite; report the exact command and exact results — pass/fail
plus passed/skipped/failed counts, any ignored paths and why they are safe
to ignore, and whether the command differs from the repository's canonical
validation command. Only claim a result after the output has been read.
9. Post the review verdict: approve only if scope is clean and checks pass;
otherwise request changes with specifics. Never merge from this review step.
Include a "Review Metadata" block (attribution only — docs/llm-agent-sha.md):