feat(review-workflow): raise A-bar with capability, sweep, live-state, and role-boundary proofs (#179)

Extends review_proofs.py with the four #179 proofs, the successor set to
the #173 checkout/inventory proofs:

- assess_capability_evidence: a capability claim (review_pr, merge_pr, ...)
  counts only with exact evidence citing gitea_resolve_task_capability
  output or equivalent runtime context; no claims at all fails closed.
- assess_sweep_evidence: secret/provenance sweeps must state the exact
  command/script/pattern/named method, the scope scanned, and a boolean
  result; vague summaries are downgraded and a missing sweep fails closed.
- assess_live_state_recheck: an explicit pre-mutation recheck must prove
  the PR is still open, the live head equals the pinned head (full
  40-hex), the base branch is unchanged, and blocking review state was
  checked and absent; not performing it fails closed.
- assess_role_boundary: a reviewer run using an author namespace (or vice
  versa) is clean only with an explicit justification; unreported
  namespace usage fails closed.

build_final_report now takes the four proofs as keyword arguments: any
missing or failed proof downgrades the grade, merge_allowed additionally
requires the proven live-state recheck, and a merge performed without it
is a blocked violation. Existing #173 semantics are unchanged otherwise;
gates only get stricter.

tests/test_review_proofs.py adds 29 tests covering the issue's harness
assertions: capability claims without evidence downgraded, vague sweeps
downgraded, missing/stale live-state recheck downgrades and blocks merge
(violation when a merge is claimed anyway), unjustified author-namespace
use downgraded, and the #173 positive baseline preserved.

SKILL.md sections F/G and the review-pr/merge-pr templates now require the
capability evidence, exact sweep, pre-verdict and pre-merge live-state
rechecks, and reviewer-namespace discipline.

Closes #179

Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
2026-07-05 15:44:17 -04:00
co-authored by Claude Fable 5
parent 601c608c58
commit 4a83a8e215
5 changed files with 551 additions and 13 deletions
+23 -6
View File
@@ -204,18 +204,35 @@ Worktree folder = branch with `/` replaced by `-`
validation result after the command has completed and its output has
been read (`review_proofs.assess_validation_report`).
9. **Do not merge if checks fail. Do not merge if the reviewer is the author.**
10. The final report must distinguish (`review_proofs.build_final_report`):
10. **#179 A-bar proofs** (all fail closed when missing —
`review_proofs.assess_capability_evidence`, `assess_sweep_evidence`,
`assess_live_state_recheck`, `assess_role_boundary`):
- Capability claims must cite exact `gitea_resolve_task_capability`
output (or runtime context); a bare "capability checks passed" is
downgraded.
- The secret/provenance sweep must state the exact command/script/
pattern/named method and the scope scanned.
- Immediately before submitting a review verdict (and again before any
merge), re-read live PR state and prove: still open, live head ==
pinned head, base unchanged, no unresolved blocking review state.
- Reviewer runs stay in the reviewer namespace; any author-namespace
call requires an explicit justification in the report.
11. The final report must distinguish (`review_proofs.build_final_report`):
identity eligible; PR author different from reviewer; session
contamination absent (with evidence); validation performed on the pinned
head; merge performed; issue status verified. If any proof is missing,
stop or downgrade the result instead of merging confidently.
head; capability evidence; sweep verdict; live-state recheck; role
boundary; merge performed; issue status verified. If any proof is
missing, stop or downgrade the result instead of merging confidently.
## G. Merge / cleanup workflow
Only an eligible (non-author) reviewer merges. Before merging: always verify
the authenticated identity **and** the PR author; respect runtime profile
gates; run independent validation (do not trust the author's reported
results); and merge with a **pinned head SHA** and, where supported, the
the authenticated identity **and** the PR author; cite exact capability
evidence for merge_pr (#179); respect runtime profile gates; run independent
validation (do not trust the author's reported results); perform the **final
live-state recheck** (#179 — PR still open, live head == pinned head, base
unchanged, no unresolved blocking review state) immediately before the merge
mutation; and merge with a **pinned head SHA** and, where supported, the
**expected changed-file set**, so a moved head or widened diff refuses the
merge. After a real merge:
@@ -20,10 +20,21 @@ Steps:
*If the current identity does not match the required role (or is the PR author), STOP. Relaunch/switch to the correct profile first.*
2. Verify authenticated identity + active profile.
3. Confirm PR #<pr>: author (not you), state open, mergeable, review approved. Check if PR body uses `Closes #N` or `Fixes #N`; if it uses `Implements #N` or `Refs #N`, manual closing will be needed in step 29.
4. If any gate fails → STOP and report.
4. Merge with explicit confirmation (e.g. confirmation="MERGE PR <pr>"),
optionally pinning the reviewed head SHA / changed-file set.
5. Confirm remote master now contains the merge commit (or the expected changes if squash merged).
4. Capability evidence (#179): cite the exact gitea_resolve_task_capability
output (or runtime context) proving merge_pr is allowed — a bare
"capability checks passed" claim is downgraded.
5. Final live-state recheck (#179), immediately before the merge mutation —
re-read the live PR and prove:
- PR still open
- live head SHA still equals the pinned/reviewed head SHA
- base branch unchanged
- no undismissed REQUEST_CHANGES / blocking review state remains
If any recheck fails → STOP, re-pin, re-validate.
6. If any gate fails → STOP and report.
7. Merge with explicit confirmation (e.g. confirmation="MERGE PR <pr>"),
pinning the reviewed head SHA (expected_head_sha) and, where supported,
the changed-file set.
8. Confirm remote master now contains the merge commit (or the expected changes if squash merged).
*Note: Gitea PR "closed" state is NOT equivalent to "merged". Do not assume a closed PR succeeded without verifying the actual landed changes.*
Then run the cleanup template (worktree-cleanup.md):
@@ -32,6 +32,11 @@ Steps:
- Target task role: reviewer identity (must NOT be the PR author)
*If the current identity does not match the required role (or is the PR author), STOP. Relaunch/switch to the correct profile first.*
2. Verify your authenticated identity (whoami) and the active profile.
Capability evidence (#179): cite the exact gitea_resolve_task_capability
output (or runtime context) for review_pr (and merge_pr if merging later);
a bare "capability checks passed" claim is downgraded. Stay in the
reviewer namespace: any author-namespace call must be justified in the
report (#179).
3. Fetch the PR facts: PR author, head SHA, state (must be open), base branch.
Pin the head SHA in your notes; every later step validates THAT SHA.
4. If authenticated user == PR author → STOP (no self-review).
@@ -50,12 +55,23 @@ Steps:
If HEAD does not match the pinned head → STOP before review/merge.
7. Confirm the worktree is clean. Inspect the FULL diff; confirm scope matches
issue #<n>; flag any unrelated files, secrets, or formatting churn. Check that the PR body correctly uses Gitea-closing keywords (`Closes #N` or `Fixes #N`) instead of non-closing ones (`Implements #N`, `Refs #N`).
Secret/provenance sweep must be exact (#179): state the exact command,
script, grep pattern, or named sweep method AND the scope scanned (e.g.
`git diff prgs/master...HEAD | grep -inE '<pattern>'`); "checked the diff
for secrets" alone is downgraded.
8. Run the test suite; report the exact command and exact results — pass/fail
plus passed/skipped/failed counts, any ignored paths and why they are safe
to ignore, and whether the command differs from the repository's canonical
validation command. Only claim a result after the output has been read.
9. Post the review verdict: approve only if scope is clean and checks pass;
otherwise request changes with specifics. Never merge from this review step.
9. Final live-state recheck (#179), immediately before submitting the review
verdict — re-read the live PR and prove:
- PR still open
- live head SHA still equals the pinned head SHA from step 3
- base branch unchanged
- no undismissed REQUEST_CHANGES / blocking review state left unaccounted
If anything moved → STOP, re-pin, re-validate before any verdict.
10. Post the review verdict: approve only if scope is clean and checks pass;
otherwise request changes with specifics. Never merge from this review step.
Include a "Review Metadata" block (attribution only — docs/llm-agent-sha.md):
Review Metadata: