feat(review-workflow): add fail-closed proofs for blind PR queue review (#173)
Adds review_proofs.py with pure, fail-closed helpers for the reviewer-side blind queue workflow: - verify_pinned_head_checkout: proves HEAD == pinned PR head (full 40-hex, no prefix matches) and diff base == PR base; any mismatch blocks review/merge. - assess_inventory_completeness: exhaustive "only PRs found" claims require every configured repo listed with stated filters, proven pagination, and reported open-PR totals. - assess_validation_report: validation results are unclaimable unless the exact command was stated and its output read; missing pass/fail/skip counts, unjustified ignored paths, or unexplained deviation from the canonical command downgrade the report to weak. - assess_self_review_contamination: contamination must be evidence-backed; bare same-session claims (either way) yield "unknown" with a choose-another-PR-or-stop action, never an assumed verdict. - build_final_report: distinguishes identity eligibility, author/reviewer distinctness, session contamination, validation-on-pinned-head, merge, and issue verification; grades "A" only when every proof is present, downgrades otherwise, and marks a merge without allowing proofs as a blocked violation. tests/test_review_proofs.py covers all seven harness assertions from the issue (unchecked-out pinned head blocks merge; multi-repo inventory; pagination; missing counts flagged weak; evidence-required contamination; unread output cannot claim validation passed; ignored paths need justification). SKILL.md section F and templates/review-pr.md now require the checkout, inventory, validation-evidence, and contamination proofs and the distinguished final report. No review/merge safety gate is weakened. Closes #173 Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
@@ -20,13 +20,28 @@ Steps:
|
||||
*If the current identity does not match the required role (or is the PR author), STOP. Relaunch/switch to the correct profile first.*
|
||||
2. Verify your authenticated identity (whoami) and the active profile.
|
||||
3. Fetch the PR facts: PR author, head SHA, state (must be open), base branch.
|
||||
Pin the head SHA in your notes; every later step validates THAT SHA.
|
||||
4. If authenticated user == PR author → STOP (no self-review).
|
||||
4. scripts/worktree-review <pr-head-branch> # detached, branches/review-*
|
||||
Session-contamination claims must be evidence-backed (#173): if you
|
||||
cannot evidence whether this session authored/touched the PR branch,
|
||||
report contamination as UNKNOWN (not contaminated, not clean) and choose
|
||||
another PR or stop.
|
||||
5. scripts/worktree-review <pr-head-branch> # detached, branches/review-*
|
||||
cd branches/review-<pr-head-branch-slug>
|
||||
5. Confirm the worktree is clean. Inspect the FULL diff; confirm scope matches
|
||||
6. Checkout proof (#173) — prove and state, before any diff review or
|
||||
validation:
|
||||
- pinned PR head SHA (from Gitea)
|
||||
- local checkout SHA (git rev-parse HEAD)
|
||||
- HEAD == pinned PR head SHA
|
||||
- diff base == the PR base branch
|
||||
If HEAD does not match the pinned head → STOP before review/merge.
|
||||
7. Confirm the worktree is clean. Inspect the FULL diff; confirm scope matches
|
||||
issue #<n>; flag any unrelated files, secrets, or formatting churn. Check that the PR body correctly uses Gitea-closing keywords (`Closes #N` or `Fixes #N`) instead of non-closing ones (`Implements #N`, `Refs #N`).
|
||||
6. Run the test suite; note results.
|
||||
7. Post the review verdict: approve only if scope is clean and checks pass;
|
||||
8. Run the test suite; report the exact command and exact results — pass/fail
|
||||
plus passed/skipped/failed counts, any ignored paths and why they are safe
|
||||
to ignore, and whether the command differs from the repository's canonical
|
||||
validation command. Only claim a result after the output has been read.
|
||||
9. Post the review verdict: approve only if scope is clean and checks pass;
|
||||
otherwise request changes with specifics. Never merge from this review step.
|
||||
Include a "Review Metadata" block (attribution only — docs/llm-agent-sha.md):
|
||||
|
||||
|
||||
Reference in New Issue
Block a user