feat: require validation failure history in review reports (Closes #396)

Reviewer runs that observe a validation failure and later rerun to green
must report the failure and its investigation; a later pass never erases
the earlier failure:

- New reviewer_validation_failure_history.py verifier: reports must carry
  a Validation failure history section naming every observed failure with
  command, failing test or error, suspected cause, reproduced status,
  baseline master status, and PR-caused evidence; rerun-pass claims must
  explain what changed between runs, whether environmental state (such as
  stale /tmp files) was cleaned, and why the pass is trustworthy; an
  unexplained transient failure cannot be reported as a bare pass and
  must use the status 'passed after transient failure investigation'
  (exported as TRANSIENT_INVESTIGATION_STATUS).
- Re-export from review_proofs and wire into build_final_report as a
  downgrade check.
- Review-merge final-report schema gains the Validation failure history
  template section; contract test locks the markers and an export test
  locks the re-export.
- 12 tests covering the issue's scenarios: failing-then-passing rerun,
  baseline-equivalent failure, /tmp state contamination, unexplained
  transient failure, and a report omitting an earlier failure.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
2026-07-07 11:26:40 -04:00
co-authored by Claude Opus 4.8
parent fcb9944378
commit f20e0fc338
5 changed files with 367 additions and 1 deletions
+8
View File
@@ -74,6 +74,14 @@ def test_review_merge_final_report_schema_exists():
assert "Candidate head SHA:" in text
assert "Terminal review mutation:" in text
assert "Workspace mutations" in text
assert "### Validation failure history (#396)" in text
assert "passed after transient failure investigation" in text
def test_validation_failure_history_verifier_exported():
from review_proofs import assess_validation_failure_history_report
assert callable(assess_validation_failure_history_report)
def test_reconcile_landed_workflow_contract():