Add validation integrity verifier for PR-head vs diagnostic tests (#316) #358

2026-07-07T04:20:15-05:00

jcwalker3 commented

2026-07-07 04:20:15 -05:00

Summary

Closes #316

Adds assess_validation_integrity_report to enforce separation between official PR-head validation and post-edit diagnostic experiments in reviewer reports.

Changes

New reviewer_validation_integrity.py verifier
Re-export from review_proofs.py
Unit tests in tests/test_reviewer_validation_integrity.py
Export test in tests/test_llm_workflow_split.py

Behavior

Requires official validation command/result and worktree dirty state after official validation
Diagnostic runs must be labeled Diagnostic local experiment — not PR-head validation
Blocks reporting post-edit diagnostic passes as official PR-head validation
Contaminated sessions must report integrity status contaminated — recovery required

Testing

pytest tests/test_reviewer_validation_integrity.py tests/test_review_proofs.py tests/test_llm_workflow_split.py::test_validation_integrity_verifier_exported -q

166 passed

## Summary Closes #316 Adds `assess_validation_integrity_report` to enforce separation between official PR-head validation and post-edit diagnostic experiments in reviewer reports. ## Changes - New `reviewer_validation_integrity.py` verifier - Re-export from `review_proofs.py` - Unit tests in `tests/test_reviewer_validation_integrity.py` - Export test in `tests/test_llm_workflow_split.py` ## Behavior - Requires official validation command/result and worktree dirty state after official validation - Diagnostic runs must be labeled **Diagnostic local experiment — not PR-head validation** - Blocks reporting post-edit diagnostic passes as official PR-head validation - Contaminated sessions must report integrity status **contaminated — recovery required** ## Testing ``` pytest tests/test_reviewer_validation_integrity.py tests/test_review_proofs.py tests/test_llm_workflow_split.py::test_validation_integrity_verifier_exported -q ``` 166 passed

jcwalker3 added 1 commit 2026-07-07 04:20:16 -05:00

Add validation integrity verifier for PR-head vs diagnostic tests (#316 ) 4ea618e8b4

Separate official PR-head validation from post-edit diagnostic runs in
reviewer reports. Block treating diagnostic passes as official validation
and require contaminated sessions to report recovery-required status.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

jcwalker3 referenced this pull request

2026-07-07 04:20:19 -05:00

Reviewer validation reports must distinguish PR-head tests from diagnostic edited-worktree tests #316

jcwalker3 commented

2026-07-07 04:20:22 -05:00

Ready for review — implements #316 validation integrity verifier with 166 passing tests in the targeted slice.

jcwalker3 referenced this pull request

2026-07-07 04:22:56 -05:00

Fix TestIssueLockArtifactWarning profile gate setup (full suite failure) #359

jcwalker3 referenced this issue from a commit

2026-07-07 04:35:19 -05:00

fix: resolve conflicts for PR #358

jcwalker3 added 1 commit 2026-07-07 04:35:20 -05:00

fix: resolve conflicts for PR #358 15c9add3f4

jcwalker3 referenced this issue from a commit

2026-07-07 08:14:51 -05:00

fix: resolve conflicts for PR #358 against current master

jcwalker3 added 1 commit 2026-07-07 08:14:51 -05:00

fix: resolve conflicts for PR #358 against current master af806919b3

jcwalker3 referenced this issue from a commit

2026-07-07 08:27:33 -05:00

fix: resolve conflicts for PR #358 against latest master

jcwalker3 added 1 commit 2026-07-07 08:27:33 -05:00

fix: resolve conflicts for PR #358 against latest master c2226177b6

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

sysadmin approved these changes 2026-07-07 08:30:52 -05:00

sysadmin left a comment

All tests pass successfully after merging master (1323 passed, 1 failed [pre-existing]). Reviewed the implementation of assess_validation_integrity_report in reviewer_validation_integrity.py and the test coverage in tests/test_reviewer_validation_integrity.py. It correctly enforces separation between official PR-head validation and post-edit diagnostic experiments in reviewer reports. Verified that the remaining failure is pre-existing on master.

All tests pass successfully after merging master (1323 passed, 1 failed [pre-existing]). Reviewed the implementation of `assess_validation_integrity_report` in `reviewer_validation_integrity.py` and the test coverage in `tests/test_reviewer_validation_integrity.py`. It correctly enforces separation between official PR-head validation and post-edit diagnostic experiments in reviewer reports. Verified that the remaining failure is pre-existing on master.

sysadmin merged commit ce39fb52c9 into master

2026-07-07 08:31:02 -05:00

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: Scaled-Tech-Consulting/Gitea-Tools#358