sysadminandClaude Fable 5
ccc5ef79df
Adds review_proofs.py with pure, fail-closed helpers for the reviewer-side blind queue workflow: - verify_pinned_head_checkout: proves HEAD == pinned PR head (full 40-hex, no prefix matches) and diff base == PR base; any mismatch blocks review/merge. - assess_inventory_completeness: exhaustive "only PRs found" claims require every configured repo listed with stated filters, proven pagination, and reported open-PR totals. - assess_validation_report: validation results are unclaimable unless the exact command was stated and its output read; missing pass/fail/skip counts, unjustified ignored paths, or unexplained deviation from the canonical command downgrade the report to weak. - assess_self_review_contamination: contamination must be evidence-backed; bare same-session claims (either way) yield "unknown" with a choose-another-PR-or-stop action, never an assumed verdict. - build_final_report: distinguishes identity eligibility, author/reviewer distinctness, session contamination, validation-on-pinned-head, merge, and issue verification; grades "A" only when every proof is present, downgrades otherwise, and marks a merge without allowing proofs as a blocked violation. tests/test_review_proofs.py covers all seven harness assertions from the issue (unchecked-out pinned head blocks merge; multi-repo inventory; pagination; missing counts flagged weak; evidence-required contamination; unread output cannot claim validation passed; ignored paths need justification). SKILL.md section F and templates/review-pr.md now require the checkout, inventory, validation-evidence, and contamination proofs and the distinguished final report. No review/merge safety gate is weakened. Closes #173 Co-Authored-By: Claude Fable 5 <[email protected]>
59 lines
3.1 KiB
Markdown
59 lines
3.1 KiB
Markdown
# Template: review a PR
|
|
|
|
Copy, fill the `<...>` fields, and paste as the task prompt.
|
|
|
|
```text
|
|
Task: review PR #<pr> for issue #<n>.
|
|
|
|
Rules (llm-project-workflow):
|
|
- Review in a SEPARATE detached review worktree, never the author's folder.
|
|
- You must NOT be the PR author. If the authenticated user == PR author, stop.
|
|
A different LLM-Agent-SHA does NOT make you a different actor — only a
|
|
different authenticated Gitea user does (docs/llm-agent-sha.md).
|
|
- Do not merge if any check fails.
|
|
|
|
Steps:
|
|
1. Identity Checklist: Before claiming/working on review, verify and state:
|
|
- Required identity/profile for this task: reviewer (allowed to review/approve/request_changes)
|
|
- Current authenticated identity (from whoami): <username>
|
|
- Target task role: reviewer identity (must NOT be the PR author)
|
|
*If the current identity does not match the required role (or is the PR author), STOP. Relaunch/switch to the correct profile first.*
|
|
2. Verify your authenticated identity (whoami) and the active profile.
|
|
3. Fetch the PR facts: PR author, head SHA, state (must be open), base branch.
|
|
Pin the head SHA in your notes; every later step validates THAT SHA.
|
|
4. If authenticated user == PR author → STOP (no self-review).
|
|
Session-contamination claims must be evidence-backed (#173): if you
|
|
cannot evidence whether this session authored/touched the PR branch,
|
|
report contamination as UNKNOWN (not contaminated, not clean) and choose
|
|
another PR or stop.
|
|
5. scripts/worktree-review <pr-head-branch> # detached, branches/review-*
|
|
cd branches/review-<pr-head-branch-slug>
|
|
6. Checkout proof (#173) — prove and state, before any diff review or
|
|
validation:
|
|
- pinned PR head SHA (from Gitea)
|
|
- local checkout SHA (git rev-parse HEAD)
|
|
- HEAD == pinned PR head SHA
|
|
- diff base == the PR base branch
|
|
If HEAD does not match the pinned head → STOP before review/merge.
|
|
7. Confirm the worktree is clean. Inspect the FULL diff; confirm scope matches
|
|
issue #<n>; flag any unrelated files, secrets, or formatting churn. Check that the PR body correctly uses Gitea-closing keywords (`Closes #N` or `Fixes #N`) instead of non-closing ones (`Implements #N`, `Refs #N`).
|
|
8. Run the test suite; report the exact command and exact results — pass/fail
|
|
plus passed/skipped/failed counts, any ignored paths and why they are safe
|
|
to ignore, and whether the command differs from the repository's canonical
|
|
validation command. Only claim a result after the output has been read.
|
|
9. Post the review verdict: approve only if scope is clean and checks pass;
|
|
otherwise request changes with specifics. Never merge from this review step.
|
|
Include a "Review Metadata" block (attribution only — docs/llm-agent-sha.md):
|
|
|
|
Review Metadata:
|
|
- LLM-Agent-SHA: llm-<12 lowercase hex, e.g. llm-41d0e7aa9f2c>
|
|
- LLM-Role: reviewer
|
|
- Authenticated-Gitea-User: <whoami result>
|
|
- MCP-Profile: <profile name>
|
|
- Eligibility: passed/failed
|
|
|
|
Handoff: reviewer identity, PR author, scope verdict, checks + results, decision —
|
|
formatted per SKILL.md §K (compact by default; long form if a merge happened
|
|
or a gate blocked you); if you could not merge, name the exact gate.
|
|
```
|