feat: require true pre-merge baseline proof for non-zero validation exits (Closes #533)

Add premerge_baseline_proof.py distinguishing four validation outcomes
(clean pass / current-master failure reproduced / pre-merge baseline-proven
failure / unresolved regression risk). A non-zero exit may only be labeled
"baseline"/"pre-existing" with pre-merge proof: the failure reproduced on the
PR's pre-merge base commit, or a documented known-failure record predating the
PR. Current-master (post-merge) reproduction is explicitly rejected as
sufficient baseline proof.

- premerge_baseline_proof.py: assess_premerge_baseline_proof + labels
- final_report_validator.py: reviewer.premerge_baseline_proof rule on
  review_pr and reconcile_already_landed tasks
- skills/llm-project-workflow/templates/review-pr.md: baseline proof section
  requiring base commit, tested commit, command, exit status, failure signature
- tests/test_premerge_baseline_proof.py: reject current-master-only proof,
  accept pre-merge base proof and documented known-failure records

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
2026-07-08 23:04:44 -04:00
co-authored by Claude Opus 4.8
parent 4bcdd350f6
commit 7fb813446f
4 changed files with 319 additions and 0 deletions
+21
View File
@@ -715,6 +715,25 @@ def _rule_reviewer_main_checkout_baseline(report_text: str) -> list[dict[str, st
]
def _rule_reviewer_premerge_baseline_proof(report_text: str) -> list[dict[str, str]]:
from premerge_baseline_proof import assess_premerge_baseline_proof
result = assess_premerge_baseline_proof(report_text)
if not result.get("block"):
return []
return _findings_from_reasons(
"reviewer.premerge_baseline_proof",
result.get("reasons") or [],
field="Pre-merge baseline proof",
severity="block",
safe_next_action=result.get("safe_next_action")
or (
"prove the failure on the PR pre-merge base commit or a known-failure "
"record predating the PR; current-master reproduction is not baseline proof"
),
)
def _rule_reviewer_validation_status_vocabulary(
report_text: str,
*,
@@ -1294,6 +1313,7 @@ _RULES_BY_TASK: dict[str, list[Callable[..., list[dict[str, str]]]]] = {
_rule_reviewer_validation_structured,
_rule_reviewer_linked_issue,
_rule_reviewer_baseline_on_failure,
_rule_reviewer_premerge_baseline_proof,
_rule_reviewer_validation_status_vocabulary,
_rule_reviewer_main_checkout_baseline,
_rule_reviewer_main_checkout_path,
@@ -1318,6 +1338,7 @@ _RULES_BY_TASK: dict[str, list[Callable[..., list[dict[str, str]]]]] = {
_rule_reconcile_linked_issue_live,
_rule_reconcile_close_proof,
_rule_reconcile_pagination_proof,
_rule_reviewer_premerge_baseline_proof,
_rule_reviewer_git_fetch_readonly,
_rule_reviewer_legacy_workspace_mutations,
_rule_reviewer_vague_mutations_none,