feat(review-workflow): raise A-bar with capability, sweep, live-state, and role-boundary proofs (#179)
Extends review_proofs.py with the four #179 proofs, the successor set to the #173 checkout/inventory proofs: - assess_capability_evidence: a capability claim (review_pr, merge_pr, ...) counts only with exact evidence citing gitea_resolve_task_capability output or equivalent runtime context; no claims at all fails closed. - assess_sweep_evidence: secret/provenance sweeps must state the exact command/script/pattern/named method, the scope scanned, and a boolean result; vague summaries are downgraded and a missing sweep fails closed. - assess_live_state_recheck: an explicit pre-mutation recheck must prove the PR is still open, the live head equals the pinned head (full 40-hex), the base branch is unchanged, and blocking review state was checked and absent; not performing it fails closed. - assess_role_boundary: a reviewer run using an author namespace (or vice versa) is clean only with an explicit justification; unreported namespace usage fails closed. build_final_report now takes the four proofs as keyword arguments: any missing or failed proof downgrades the grade, merge_allowed additionally requires the proven live-state recheck, and a merge performed without it is a blocked violation. Existing #173 semantics are unchanged otherwise; gates only get stricter. tests/test_review_proofs.py adds 29 tests covering the issue's harness assertions: capability claims without evidence downgraded, vague sweeps downgraded, missing/stale live-state recheck downgrades and blocks merge (violation when a merge is claimed anyway), unjustified author-namespace use downgraded, and the #173 positive baseline preserved. SKILL.md sections F/G and the review-pr/merge-pr templates now require the capability evidence, exact sweep, pre-verdict and pre-merge live-state rechecks, and reviewer-namespace discipline. Closes #179 Co-Authored-By: Claude Fable 5 <[email protected]>
This commit is contained in:
Claude Fable 5
@@ -232,19 +232,35 @@ Worktree folder = branch with `/` replaced by `-`
|
||||
validation result after the command has completed and its output has
|
||||
been read (`review_proofs.assess_validation_report`).
|
||||
10. **Do not merge if checks fail. Do not merge if the reviewer is the author.**
|
||||
11. The final report must distinguish (`review_proofs.build_final_report`):
|
||||
11. **#179 A-bar proofs** (all fail closed when missing —
|
||||
`review_proofs.assess_capability_evidence`, `assess_sweep_evidence`,
|
||||
`assess_live_state_recheck`, `assess_role_boundary`):
|
||||
- Capability claims must cite exact `gitea_resolve_task_capability`
|
||||
output (or runtime context); a bare "capability checks passed" is
|
||||
downgraded.
|
||||
- The secret/provenance sweep must state the exact command/script/
|
||||
pattern/named method and the scope scanned.
|
||||
- Immediately before submitting a review verdict (and again before any
|
||||
merge), re-read live PR state and prove: still open, live head ==
|
||||
pinned head, base unchanged, no unresolved blocking review state.
|
||||
- Reviewer runs stay in the reviewer namespace; any author-namespace
|
||||
call requires an explicit justification in the report.
|
||||
12. The final report must distinguish (`review_proofs.build_final_report`):
|
||||
identity eligible; PR author different from reviewer; session
|
||||
contamination absent (with evidence); role boundary clean; validation
|
||||
performed on the pinned head; merge performed; issue status verified. If
|
||||
any proof is missing, stop or downgrade the result instead of merging
|
||||
confidently.
|
||||
contamination absent (with evidence); validation performed on the pinned
|
||||
head; capability evidence; sweep verdict; live-state recheck; role
|
||||
boundary; merge performed; issue status verified. If any proof is
|
||||
missing, stop or downgrade the result instead of merging confidently.
|
||||
|
||||
## G. Merge / cleanup workflow
|
||||
|
||||
Only an eligible (non-author) reviewer merges. Before merging: always verify
|
||||
the authenticated identity **and** the PR author; respect runtime profile
|
||||
gates; run independent validation (do not trust the author's reported
|
||||
results); and merge with a **pinned head SHA** and, where supported, the
|
||||
the authenticated identity **and** the PR author; cite exact capability
|
||||
evidence for merge_pr (#179); respect runtime profile gates; run independent
|
||||
validation (do not trust the author's reported results); perform the **final
|
||||
live-state recheck** (#179 — PR still open, live head == pinned head, base
|
||||
unchanged, no unresolved blocking review state) immediately before the merge
|
||||
mutation; and merge with a **pinned head SHA** and, where supported, the
|
||||
**expected changed-file set**, so a moved head or widened diff refuses the
|
||||
merge. After a real merge:
|
||||
|
||||
|
||||
@@ -20,10 +20,21 @@ Steps:
|
||||
*If the current identity does not match the required role (or is the PR author), STOP. Relaunch/switch to the correct profile first.*
|
||||
2. Verify authenticated identity + active profile.
|
||||
3. Confirm PR #<pr>: author (not you), state open, mergeable, review approved. Check if PR body uses `Closes #N` or `Fixes #N`; if it uses `Implements #N` or `Refs #N`, manual closing will be needed in step 29.
|
||||
4. If any gate fails → STOP and report.
|
||||
4. Merge with explicit confirmation (e.g. confirmation="MERGE PR <pr>"),
|
||||
optionally pinning the reviewed head SHA / changed-file set.
|
||||
5. Confirm remote master now contains the merge commit (or the expected changes if squash merged).
|
||||
4. Capability evidence (#179): cite the exact gitea_resolve_task_capability
|
||||
output (or runtime context) proving merge_pr is allowed — a bare
|
||||
"capability checks passed" claim is downgraded.
|
||||
5. Final live-state recheck (#179), immediately before the merge mutation —
|
||||
re-read the live PR and prove:
|
||||
- PR still open
|
||||
- live head SHA still equals the pinned/reviewed head SHA
|
||||
- base branch unchanged
|
||||
- no undismissed REQUEST_CHANGES / blocking review state remains
|
||||
If any recheck fails → STOP, re-pin, re-validate.
|
||||
6. If any gate fails → STOP and report.
|
||||
7. Merge with explicit confirmation (e.g. confirmation="MERGE PR <pr>"),
|
||||
pinning the reviewed head SHA (expected_head_sha) and, where supported,
|
||||
the changed-file set.
|
||||
8. Confirm remote master now contains the merge commit (or the expected changes if squash merged).
|
||||
*Note: Gitea PR "closed" state is NOT equivalent to "merged". Do not assume a closed PR succeeded without verifying the actual landed changes.*
|
||||
|
||||
Then run the cleanup template (worktree-cleanup.md):
|
||||
|
||||
@@ -36,6 +36,11 @@ Steps:
|
||||
- Target task role: reviewer identity (must NOT be the PR author)
|
||||
*If the current identity does not match the required role (or is the PR author), STOP. Relaunch/switch to the correct profile first.*
|
||||
2. Verify your authenticated identity (whoami) and the active profile.
|
||||
Capability evidence (#179): cite the exact gitea_resolve_task_capability
|
||||
output (or runtime context) for review_pr (and merge_pr if merging later);
|
||||
a bare "capability checks passed" claim is downgraded. Stay in the
|
||||
reviewer namespace: any author-namespace call must be justified in the
|
||||
report (#179).
|
||||
3. Fetch the PR facts: PR author, head SHA, state (must be open), base branch.
|
||||
Pin the head SHA in your notes; every later step validates THAT SHA.
|
||||
4. If authenticated user == PR author → STOP (no self-review).
|
||||
@@ -58,12 +63,23 @@ Steps:
|
||||
If HEAD does not match the pinned head → STOP before review/merge.
|
||||
7. Confirm the worktree is clean. Inspect the FULL diff; confirm scope matches
|
||||
issue #<n>; flag any unrelated files, secrets, or formatting churn. Check that the PR body correctly uses Gitea-closing keywords (`Closes #N` or `Fixes #N`) instead of non-closing ones (`Implements #N`, `Refs #N`).
|
||||
Secret/provenance sweep must be exact (#179): state the exact command,
|
||||
script, grep pattern, or named sweep method AND the scope scanned (e.g.
|
||||
`git diff prgs/master...HEAD | grep -inE '<pattern>'`); "checked the diff
|
||||
for secrets" alone is downgraded.
|
||||
8. Run the test suite; report the exact command and exact results — pass/fail
|
||||
plus passed/skipped/failed counts, any ignored paths and why they are safe
|
||||
to ignore, and whether the command differs from the repository's canonical
|
||||
validation command. Only claim a result after the output has been read.
|
||||
9. Post the review verdict: approve only if scope is clean and checks pass;
|
||||
otherwise request changes with specifics. Never merge from this review step.
|
||||
9. Final live-state recheck (#179), immediately before submitting the review
|
||||
verdict — re-read the live PR and prove:
|
||||
- PR still open
|
||||
- live head SHA still equals the pinned head SHA from step 3
|
||||
- base branch unchanged
|
||||
- no undismissed REQUEST_CHANGES / blocking review state left unaccounted
|
||||
If anything moved → STOP, re-pin, re-validate before any verdict.
|
||||
10. Post the review verdict: approve only if scope is clean and checks pass;
|
||||
otherwise request changes with specifics. Never merge from this review step.
|
||||
Include a "Review Metadata" block (attribution only — docs/llm-agent-sha.md):
|
||||
|
||||
Review Metadata:
|
||||
|
||||
Reference in New Issue
Block a user