feat(review-workflow): raise A-bar with capability, sweep, live-state, and role-boundary proofs (#179)
Extends review_proofs.py with the four #179 proofs, the successor set to the #173 checkout/inventory proofs: - assess_capability_evidence: a capability claim (review_pr, merge_pr, ...) counts only with exact evidence citing gitea_resolve_task_capability output or equivalent runtime context; no claims at all fails closed. - assess_sweep_evidence: secret/provenance sweeps must state the exact command/script/pattern/named method, the scope scanned, and a boolean result; vague summaries are downgraded and a missing sweep fails closed. - assess_live_state_recheck: an explicit pre-mutation recheck must prove the PR is still open, the live head equals the pinned head (full 40-hex), the base branch is unchanged, and blocking review state was checked and absent; not performing it fails closed. - assess_role_boundary: a reviewer run using an author namespace (or vice versa) is clean only with an explicit justification; unreported namespace usage fails closed. build_final_report now takes the four proofs as keyword arguments: any missing or failed proof downgrades the grade, merge_allowed additionally requires the proven live-state recheck, and a merge performed without it is a blocked violation. Existing #173 semantics are unchanged otherwise; gates only get stricter. tests/test_review_proofs.py adds 29 tests covering the issue's harness assertions: capability claims without evidence downgraded, vague sweeps downgraded, missing/stale live-state recheck downgrades and blocks merge (violation when a merge is claimed anyway), unjustified author-namespace use downgraded, and the #173 positive baseline preserved. SKILL.md sections F/G and the review-pr/merge-pr templates now require the capability evidence, exact sweep, pre-verdict and pre-merge live-state rechecks, and reviewer-namespace discipline. Closes #179 Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
@@ -20,10 +20,21 @@ Steps:
|
||||
*If the current identity does not match the required role (or is the PR author), STOP. Relaunch/switch to the correct profile first.*
|
||||
2. Verify authenticated identity + active profile.
|
||||
3. Confirm PR #<pr>: author (not you), state open, mergeable, review approved. Check if PR body uses `Closes #N` or `Fixes #N`; if it uses `Implements #N` or `Refs #N`, manual closing will be needed in step 29.
|
||||
4. If any gate fails → STOP and report.
|
||||
4. Merge with explicit confirmation (e.g. confirmation="MERGE PR <pr>"),
|
||||
optionally pinning the reviewed head SHA / changed-file set.
|
||||
5. Confirm remote master now contains the merge commit (or the expected changes if squash merged).
|
||||
4. Capability evidence (#179): cite the exact gitea_resolve_task_capability
|
||||
output (or runtime context) proving merge_pr is allowed — a bare
|
||||
"capability checks passed" claim is downgraded.
|
||||
5. Final live-state recheck (#179), immediately before the merge mutation —
|
||||
re-read the live PR and prove:
|
||||
- PR still open
|
||||
- live head SHA still equals the pinned/reviewed head SHA
|
||||
- base branch unchanged
|
||||
- no undismissed REQUEST_CHANGES / blocking review state remains
|
||||
If any recheck fails → STOP, re-pin, re-validate.
|
||||
6. If any gate fails → STOP and report.
|
||||
7. Merge with explicit confirmation (e.g. confirmation="MERGE PR <pr>"),
|
||||
pinning the reviewed head SHA (expected_head_sha) and, where supported,
|
||||
the changed-file set.
|
||||
8. Confirm remote master now contains the merge commit (or the expected changes if squash merged).
|
||||
*Note: Gitea PR "closed" state is NOT equivalent to "merged". Do not assume a closed PR succeeded without verifying the actual landed changes.*
|
||||
|
||||
Then run the cleanup template (worktree-cleanup.md):
|
||||
|
||||
@@ -36,6 +36,11 @@ Steps:
|
||||
- Target task role: reviewer identity (must NOT be the PR author)
|
||||
*If the current identity does not match the required role (or is the PR author), STOP. Relaunch/switch to the correct profile first.*
|
||||
2. Verify your authenticated identity (whoami) and the active profile.
|
||||
Capability evidence (#179): cite the exact gitea_resolve_task_capability
|
||||
output (or runtime context) for review_pr (and merge_pr if merging later);
|
||||
a bare "capability checks passed" claim is downgraded. Stay in the
|
||||
reviewer namespace: any author-namespace call must be justified in the
|
||||
report (#179).
|
||||
3. Fetch the PR facts: PR author, head SHA, state (must be open), base branch.
|
||||
Pin the head SHA in your notes; every later step validates THAT SHA.
|
||||
4. If authenticated user == PR author → STOP (no self-review).
|
||||
@@ -58,12 +63,23 @@ Steps:
|
||||
If HEAD does not match the pinned head → STOP before review/merge.
|
||||
7. Confirm the worktree is clean. Inspect the FULL diff; confirm scope matches
|
||||
issue #<n>; flag any unrelated files, secrets, or formatting churn. Check that the PR body correctly uses Gitea-closing keywords (`Closes #N` or `Fixes #N`) instead of non-closing ones (`Implements #N`, `Refs #N`).
|
||||
Secret/provenance sweep must be exact (#179): state the exact command,
|
||||
script, grep pattern, or named sweep method AND the scope scanned (e.g.
|
||||
`git diff prgs/master...HEAD | grep -inE '<pattern>'`); "checked the diff
|
||||
for secrets" alone is downgraded.
|
||||
8. Run the test suite; report the exact command and exact results — pass/fail
|
||||
plus passed/skipped/failed counts, any ignored paths and why they are safe
|
||||
to ignore, and whether the command differs from the repository's canonical
|
||||
validation command. Only claim a result after the output has been read.
|
||||
9. Post the review verdict: approve only if scope is clean and checks pass;
|
||||
otherwise request changes with specifics. Never merge from this review step.
|
||||
9. Final live-state recheck (#179), immediately before submitting the review
|
||||
verdict — re-read the live PR and prove:
|
||||
- PR still open
|
||||
- live head SHA still equals the pinned head SHA from step 3
|
||||
- base branch unchanged
|
||||
- no undismissed REQUEST_CHANGES / blocking review state left unaccounted
|
||||
If anything moved → STOP, re-pin, re-validate before any verdict.
|
||||
10. Post the review verdict: approve only if scope is clean and checks pass;
|
||||
otherwise request changes with specifics. Never merge from this review step.
|
||||
Include a "Review Metadata" block (attribution only — docs/llm-agent-sha.md):
|
||||
|
||||
Review Metadata:
|
||||
|
||||
Reference in New Issue
Block a user