Validation claims must be backed by a command ledger and final-report verifier #271

New Issue

2026-07-06T12:49:35-05:00

jcwalker3 commented

2026-07-06 12:49:35 -05:00

Final reports sometimes overclaim validation, omit skipped tests, confuse pending review with approved review, or contradict the actual commands run.

Required behavior:
A session can only claim validation passed if the exact command, exit code, and output summary are recorded in a validation ledger. Final reports must be verified before submission.

Acceptance criteria:

Add a validation ledger format capturing:
- command
- cwd/worktree path
- exit code
- output summary
- skipped/ignored tests
- timestamp
“Full suite passed” is allowed only when no test file was ignored unless the report explicitly says “full suite except X.”
Final-report verifier checks for:
- identity/profile/capability
- worktree path
- changed files
- validation ledger entries
- official review submitted vs finding ready
- visible APPROVED state vs PENDING state
- merge result or exact blocker
- workspace clean/classified
Reports with contradictions fail validation.
Tests cover validation overclaim, ignored test mislabeling, pending approval misreported as approved, and missing command output.

Final reports sometimes overclaim validation, omit skipped tests, confuse pending review with approved review, or contradict the actual commands run. Required behavior: A session can only claim validation passed if the exact command, exit code, and output summary are recorded in a validation ledger. Final reports must be verified before submission. Acceptance criteria: 1. Add a validation ledger format capturing: - command - cwd/worktree path - exit code - output summary - skipped/ignored tests - timestamp 2. “Full suite passed” is allowed only when no test file was ignored unless the report explicitly says “full suite except X.” 3. Final-report verifier checks for: - identity/profile/capability - worktree path - changed files - validation ledger entries - official review submitted vs finding ready - visible APPROVED state vs PENDING state - merge result or exact blocker - workspace clean/classified 4. Reports with contradictions fail validation. 5. Tests cover validation overclaim, ignored test mislabeling, pending approval misreported as approved, and missing command output.

jcwalker3 added the status:in-progress label 2026-07-06 23:21:47 -05:00

jcwalker3 commented

2026-07-06 23:21:48 -05:00

Claimed. Branch: feat/issue-271-validation-ledger-verifier. Worktree: branches/feat-issue-271-validation-ledger-verifier.

Scope: validation ledger format + final-report verifier module with tests per acceptance criteria.

LLM-Agent-SHA: llm-9a3f7c2e4d81

Claimed. Branch: `feat/issue-271-validation-ledger-verifier`. Worktree: `branches/feat-issue-271-validation-ledger-verifier`. Scope: validation ledger format + final-report verifier module with tests per acceptance criteria. LLM-Agent-SHA: llm-9a3f7c2e4d81

jcwalker3 referenced this issue from a commit

2026-07-06 23:22:35 -05:00

feat: add validation ledger and final-report verifier (Closes #271)

jcwalker3 referenced a pull request that will close this issue

2026-07-06 23:22:39 -05:00

feat: add validation ledger and final-report verifier (Closes #271) #287

jcwalker3 referenced this issue

2026-07-06 23:23:43 -05:00

Subagents must inherit task gates and be blocked for deterministic write workflows #266

jcwalker3 referenced this issue

2026-07-07 01:51:24 -05:00

Add final-report validator rules for reviewer and reconciliation handoffs #327