Add final-report validator rules for reviewer and reconciliation handoffs #327

New Issue

2026-07-07T01:51:24-05:00

jcwalker3 commented

2026-07-07 01:51:24 -05:00

Multiple workflow-report issues define required handoff fields and proof gates, but the final-report validator in review_proofs.py does not yet enforce the full rule set consistently.

Problem

Reviewer and reconciliation final reports still pass with stale fields, vague validation claims, ambiguous mutation categories, and incomplete linked-issue proof. Individual issues specify desired behavior, but there is no single durable validator surface that fails closed on all of them.

Required behavior

Extend the final-report validator so controller handoffs and final reports are checked against task-specific rule sets:

Reviewer rules

Reject legacy Workspace mutations and vague Mutations: None when git/worktree/MCP mutations occurred (#313, #319, #320)
Require precise mutation categories: file edits, worktree/index, git ref, MCP/Gitea, review, merge, cleanup, external-state, read-only diagnostics
Classify git fetch, merge simulation, and worktree add/remove as mutations, not read-only diagnostics (#297, #317)
Require exact validation command, working directory, executable/path proof for bare commands like pytest (#311)
Verify linked issue number/status matches selected PR; reject stale issue references (#314)
Avoid unnecessary personal email disclosure; prefer username / profile (#305)
Block approval on full-suite failure unless baseline worktree proof is complete (#323, #325)
Reject main-checkout baseline comparisons and vague "same as master" claims

Reconciliation rules

Enforce dedicated reconciliation handoff schema and reject stale author/reviewer fields (#303, #307, #299)
Require inventory pagination proof and linked-issue live status before reporting it (#300, #308)
Reject eligible/reviewed wording for already-landed PRs (#298, #295)

Shared rules

Require Controller Handoff section with task-appropriate fields
Downgrade or block when required proof fields are missing, stale, or contradictory
Return structured validator findings: rule id, severity, field, reason, safe next action

Implementation notes

Build on existing build_final_report, assess_review_mutation_final_report, and related helpers in review_proofs.py
Add composable rule modules keyed by task kind: review_pr, reconcile_already_landed, author_issue, etc.
Wire validator into workflow skills/templates so agents cannot claim APPROVE/MERGE with invalid final reports

Related issues

#271, #297, #298, #299, #303, #305, #307, #311, #313, #314, #317, #319, #320, #323, #325

Acceptance criteria

Validator exposes a single entry point for final-report text + task kind + optional live proof locks.
Reviewer handoffs fail on legacy/stale mutation wording and missing validation command proof.
Reconciliation handoffs fail on author/reviewer schema leakage and unverified linked-issue claims.
Full-suite failure without baseline worktree proof blocks approval-grade reports.
Tests cover pass, downgrade, and block outcomes for each major rule category.
Validator output is suitable for controller handoff Blocker / Safe next action fields.

Multiple workflow-report issues define required handoff fields and proof gates, but the final-report validator in `review_proofs.py` does not yet enforce the full rule set consistently. ## Problem Reviewer and reconciliation final reports still pass with stale fields, vague validation claims, ambiguous mutation categories, and incomplete linked-issue proof. Individual issues specify desired behavior, but there is no single durable validator surface that fails closed on all of them. ## Required behavior Extend the final-report validator so controller handoffs and final reports are checked against task-specific rule sets: ### Reviewer rules - Reject legacy `Workspace mutations` and vague `Mutations: None` when git/worktree/MCP mutations occurred (#313, #319, #320) - Require precise mutation categories: file edits, worktree/index, git ref, MCP/Gitea, review, merge, cleanup, external-state, read-only diagnostics - Classify `git fetch`, merge simulation, and worktree add/remove as mutations, not read-only diagnostics (#297, #317) - Require exact validation command, working directory, executable/path proof for bare commands like `pytest` (#311) - Verify linked issue number/status matches selected PR; reject stale issue references (#314) - Avoid unnecessary personal email disclosure; prefer `username / profile` (#305) - Block approval on full-suite failure unless baseline worktree proof is complete (#323, #325) - Reject main-checkout baseline comparisons and vague "same as master" claims ### Reconciliation rules - Enforce dedicated reconciliation handoff schema and reject stale author/reviewer fields (#303, #307, #299) - Require inventory pagination proof and linked-issue live status before reporting it (#300, #308) - Reject eligible/reviewed wording for already-landed PRs (#298, #295) ### Shared rules - Require Controller Handoff section with task-appropriate fields - Downgrade or block when required proof fields are missing, stale, or contradictory - Return structured validator findings: rule id, severity, field, reason, safe next action ## Implementation notes - Build on existing `build_final_report`, `assess_review_mutation_final_report`, and related helpers in `review_proofs.py` - Add composable rule modules keyed by task kind: `review_pr`, `reconcile_already_landed`, `author_issue`, etc. - Wire validator into workflow skills/templates so agents cannot claim APPROVE/MERGE with invalid final reports ## Related issues #271, #297, #298, #299, #303, #305, #307, #311, #313, #314, #317, #319, #320, #323, #325 ## Acceptance criteria - [ ] Validator exposes a single entry point for final-report text + task kind + optional live proof locks. - [ ] Reviewer handoffs fail on legacy/stale mutation wording and missing validation command proof. - [ ] Reconciliation handoffs fail on author/reviewer schema leakage and unverified linked-issue claims. - [ ] Full-suite failure without baseline worktree proof blocks approval-grade reports. - [ ] Tests cover pass, downgrade, and block outcomes for each major rule category. - [ ] Validator output is suitable for controller handoff `Blocker` / `Safe next action` fields.

jcwalker3 referenced this issue

2026-07-07 02:25:31 -05:00

Extract review-merge-pr workflow from monolithic LLM project skill #334

jcwalker3 referenced this issue

2026-07-07 03:16:44 -05:00

Reject reviewer queue-status reports that violate loaded workflow proof gates #339

jcwalker3 added the status:in-progress label 2026-07-07 03:58:48 -05:00

jcwalker3 referenced this issue from a commit

2026-07-07 04:01:22 -05:00

feat: add composable final-report validator for reviewer and reconciliation handoffs (Closes #327)

jcwalker3 referenced a pull request that will close this issue

2026-07-07 04:01:26 -05:00

feat: composable final-report validator for reviewer and reconciliation handoffs (Closes #327) #348

jcwalker3 commented

2026-07-07 04:01:27 -05:00

Work complete — PR opened

Implemented composable final-report validator (#327) on branch feat/issue-327-final-report-validator-rules.

Entry point: assess_final_report_validator(report_text, task_kind, **proof_locks)

Task kinds: review_pr, reconcile_already_landed, author_issue, work_issue, issue_filing, inventory, issue_selection

Verification: pytest tests/test_final_report_validator.py tests/test_review_proofs.py -q — 163 passed

Ready for review.

## Work complete — PR opened Implemented composable final-report validator (#327) on branch `feat/issue-327-final-report-validator-rules`. **Entry point:** `assess_final_report_validator(report_text, task_kind, **proof_locks)` **Task kinds:** `review_pr`, `reconcile_already_landed`, `author_issue`, `work_issue`, `issue_filing`, `inventory`, `issue_selection` **Verification:** `pytest tests/test_final_report_validator.py tests/test_review_proofs.py -q` — 163 passed Ready for review.

jcwalker3 referenced this issue from a commit

2026-07-07 04:27:42 -05:00

feat: add composable final-report validator for reviewer and reconciliation handoffs (Closes #327)

sysadmin referenced this issue

2026-07-07 04:28:52 -05:00

feat: composable final-report validator for reviewer and reconciliation handoffs (Closes #327) #348

sysadmin closed this issue

2026-07-07 04:29:05 -05:00

sysadmin referenced this issue from a commit

2026-07-07 04:29:06 -05:00

Merge pull request 'feat: composable final-report validator for reviewer and reconciliation handoffs (Closes #327)' (#348) from feat/issue-327-final-report-validator-rules into master

sysadmin removed the status:in-progress label 2026-07-07 04:29:06 -05:00