Files
Gitea-Tools/pr_work_lease.py
T
sysadminandClaude Opus 4.8 277ec5269d feat(mcp): use a 10-minute sliding TTL for reviewer and merger PR leases (Closes #747)
Reviewer and merger PR leases minted a fixed 120-minute expiry (#407 AC6/AC7)
and derived staleness from two further activity bands: stale at 30 minutes,
reclaimable at 60. A session that died — daemon crash, transport flap, client
restart — therefore kept a PR blocked for an hour before anyone could reclaim
it, and two hours before manual cleanup was sanctioned. #718 records the
resulting deadlock: a merge stalled behind a reviewer lease that had stopped
being live long before it stopped being authoritative.

The flaw was using a long fixed expiry as a proxy for "the owner is probably
still alive" instead of making the owner continuously prove liveness.

Sliding window
--------------
`LEASE_TTL_MINUTES = 10` now governs acquisition, and every write of the lease
marker re-derives `expires_at` from the moment of the write, so each heartbeat
slides the window forward. An actively heartbeating session is never evicted
and has no maximum lifetime; a dead one releases its hold within one TTL.

`LEASE_RENEWAL_MINUTES` is named separately from the acquisition TTL. Renewal
previously had no seam at all: the heartbeat slid the expiry only as a side
effect of re-defaulting the acquisition constant, so the two durations could
not be reasoned about or tuned independently. `format_lease_body` now takes an
explicit `ttl_minutes`, and the heartbeat passes the renewal window rather than
relying on that default.

Merger leases acquire through the same lease-body formatter, so they inherit
the identical window by construction rather than by a parallel constant.

Removal of the reclaim tier
---------------------------
`classify_lease_freshness` no longer returns `reclaimable`. Beyond being an
extra waiting tier, that band is unreachable under a sliding TTL: a heartbeat
stamps `last_activity` and `expires_at` together, so a lease idle for a full
TTL is necessarily already expired. Expiry is now the only takeover gate.

No reclaim path is lost. `find_active_reviewer_lease` already ignores expired
markers, so an expired foreign lease never gated acquisition; and the
`foreign_expired` classification carries the same
`NEXT_ACTION_RELEASE_EXPIRED_LEASE` the retired `foreign_reclaimable` did. The
two updated tests in `test_reviewer_pr_lease.py` assert exactly that: the
classification label changes, the sanctioned next action does not.

`STALE_WARNING_MINUTES` drops to 5 — half the window — so the warning still
fires while the owner can heartbeat and recover.

Diagnostics
-----------
Adds `lease_seconds_remaining`, and the heartbeat tool now returns
`ttl_minutes`, `expires_at`, and `seconds_remaining`, so an operator can
distinguish "held and live" from "held and dying" instead of only seeing that
a lease exists. All additions are additive; no existing key changed.

Also removes `pr_work_lease.DEFAULT_REVIEWER_LEASE_TTL_MINUTES`, a duplicate of
the reviewer TTL with no readers anywhere in the tree, which could only drift.

Legacy markers minted under the old 120-minute TTL still parse and are judged
against their own recorded `expires_at`, so no lease is retroactively expired
by this change.

Full suite: 3425 passed, 6 skipped, 2 failed. Both failures
(`test_issue_702_review_findings_f1_f6`, `test_reconciler_supersession_close`)
reproduce identically on clean master b05075fd25 and are pre-existing.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01QxXHZ7rqXtLgTusngaWgKZ
2026-07-18 13:55:46 -04:00

595 lines
22 KiB
Python

"""Conflict-fix and reviewer PR work leases (#399, #407 reader).
Structured PR/issue comments prove exclusive phases so author conflict-fix
pushes cannot race reviewer validation/approval/merge on the same head.
"""
from __future__ import annotations
import re
from datetime import datetime, timedelta, timezone
from typing import Any
REVIEWER_LEASE_MARKER = "<!-- mcp-review-lease:v1 -->"
CONFLICT_FIX_LEASE_MARKER = "<!-- mcp-conflict-fix-lease:v1 -->"
_FULL_SHA = re.compile(r"^[0-9a-f]{40}$", re.IGNORECASE)
_FIELD_RE = re.compile(
r"^\s*([a-z_]+)\s*:\s*(.+?)\s*$",
re.IGNORECASE | re.MULTILINE,
)
# Must mirror reviewer_pr_lease._TERMINAL_PHASES: both modules read the same
# append-only lease markers, so a phase that is terminal in one and active in
# the other yields two conflicting truths for the same comment (#742 review
# 460). "abandoned" is the owner-session merger finalization outcome.
_TERMINAL_REVIEWER_PHASES = frozenset({"done", "released", "blocked", "abandoned"})
_ACTIVE_REVIEWER_PHASES = frozenset({
"claimed",
"validating",
"approved",
"request-changes",
"merging",
})
_TERMINAL_CONFLICT_FIX_PHASES = frozenset({"released", "blocked", "done"})
_ACTIVE_CONFLICT_FIX_PHASES = frozenset({"claimed", "pushing", "pushed"})
DEFAULT_CONFLICT_FIX_TTL_MINUTES = 120
# The reviewer/merger PR-lease TTL lives in reviewer_pr_lease.LEASE_TTL_MINUTES
# (#747). A second copy here had no readers and could only drift out of sync.
def _parse_timestamp(value: str | None) -> datetime | None:
if not value:
return None
text = value.strip()
if text.endswith("Z"):
text = text[:-1] + "+00:00"
try:
parsed = datetime.fromisoformat(text)
except ValueError:
return None
if parsed.tzinfo is None:
return parsed.replace(tzinfo=timezone.utc)
return parsed.astimezone(timezone.utc)
def _normalize_sha(value: str | None) -> str | None:
text = (value or "").strip().lower()
if not text:
return None
return text if _FULL_SHA.match(text) else None
def _parse_pr_ref(value: str | None) -> int | None:
digits = re.sub(r"[^\d]", "", value or "")
return int(digits) if digits.isdigit() else None
def _parse_marker_comment(body: str, marker: str) -> dict[str, str] | None:
text = body or ""
if marker not in text:
return None
fields: dict[str, str] = {}
for match in _FIELD_RE.finditer(text):
fields[match.group(1).strip().lower()] = match.group(2).strip()
return fields or None
def parse_reviewer_lease_comment(body: str) -> dict[str, Any] | None:
fields = _parse_marker_comment(body, REVIEWER_LEASE_MARKER)
if not fields:
return None
return {
"lease_kind": "reviewer",
"pr_number": _parse_pr_ref(fields.get("pr")),
"issue_number": _parse_pr_ref(fields.get("issue")),
"reviewer_identity": fields.get("reviewer_identity"),
"profile": fields.get("profile"),
"session_id": fields.get("session_id"),
"worktree": fields.get("worktree"),
"phase": (fields.get("phase") or "").strip().lower() or None,
"candidate_head": _normalize_sha(fields.get("candidate_head")),
"target_branch": fields.get("target_branch"),
"target_branch_sha": _normalize_sha(fields.get("target_branch_sha")),
"last_activity": fields.get("last_activity"),
"expires_at": fields.get("expires_at"),
"blocker": fields.get("blocker"),
"raw_fields": fields,
}
def parse_conflict_fix_lease_comment(body: str) -> dict[str, Any] | None:
fields = _parse_marker_comment(body, CONFLICT_FIX_LEASE_MARKER)
if not fields:
return None
ff = (fields.get("fast_forward") or "").strip().lower()
reviewer_active = (fields.get("reviewer_active") or "").strip().lower()
return {
"lease_kind": "conflict_fix",
"pr_number": _parse_pr_ref(fields.get("pr")),
"branch": fields.get("branch"),
"worktree": fields.get("worktree"),
"profile": fields.get("profile"),
"session_id": fields.get("session_id"),
"phase": (fields.get("phase") or "").strip().lower() or None,
"head_before": _normalize_sha(fields.get("head_before")),
"head_after": _normalize_sha(fields.get("head_after")),
"expires_at": fields.get("expires_at"),
"reviewer_active": reviewer_active in {"yes", "true", "1"},
"fast_forward": ff in {"yes", "true", "1"},
"raw_fields": fields,
}
def _comment_entries(comments: list[dict], *, pr_number: int | None) -> list[dict]:
entries: list[dict] = []
for comment in comments or []:
body = comment.get("body") or ""
for parser in (parse_reviewer_lease_comment, parse_conflict_fix_lease_comment):
parsed = parser(body)
if not parsed:
continue
if pr_number is not None and parsed.get("pr_number") not in (None, pr_number):
continue
entries.append({
**parsed,
"comment_id": comment.get("id"),
"author": (comment.get("user") or {}).get("login") or comment.get("author"),
"created_at": comment.get("created_at"),
"updated_at": comment.get("updated_at"),
})
break
return entries
def _lease_expired(lease: dict, *, now: datetime) -> bool:
expires_at = _parse_timestamp(lease.get("expires_at"))
return bool(expires_at and expires_at <= now)
def _lease_phase_active(lease: dict, *, active_phases: frozenset[str]) -> bool:
phase = (lease.get("phase") or "").strip().lower()
if phase in _TERMINAL_REVIEWER_PHASES or phase in _TERMINAL_CONFLICT_FIX_PHASES:
return False
return phase in active_phases or bool(phase and phase not in (
_TERMINAL_REVIEWER_PHASES | _TERMINAL_CONFLICT_FIX_PHASES
))
def _reviewer_chain_key(lease: dict) -> tuple | None:
"""Identity of the lease chain a reviewer marker belongs to (#742).
A chain is one session's claim → heartbeat → terminal sequence, keyed by
repository, PR, candidate head, session id, identity, and profile. Returns
None when any component is missing: an incomplete or malformed marker has
no provable chain, so it can neither be cancelled by nor cancel anything.
"""
raw = lease.get("raw_fields") or {}
repo = (raw.get("repo") or "").strip().lower()
session_id = (lease.get("session_id") or "").strip()
identity = (lease.get("reviewer_identity") or "").strip()
profile = (lease.get("profile") or "").strip()
head = lease.get("candidate_head")
pr_number = lease.get("pr_number")
if not (repo and session_id and identity and profile and head and pr_number):
return None
return (repo, pr_number, head, session_id, identity, profile)
def _chain_terminated_after(entries: list[dict], index: int) -> bool:
"""True when a later marker terminates the chain of ``entries[index]``.
Append-only newest-wins (#577 semantics, chain-scoped for #742): a terminal
marker ends only its *own* claim, so a foreign, forged, or malformed
terminal marker cannot cancel another session's valid active lease.
"""
key = _reviewer_chain_key(entries[index])
if key is None:
return False
for later in entries[index + 1:]:
if (later.get("phase") or "").strip().lower() not in _TERMINAL_REVIEWER_PHASES:
continue
if _reviewer_chain_key(later) == key:
return True
return False
def find_active_reviewer_lease(
comments: list[dict],
*,
pr_number: int,
now: datetime | None = None,
) -> dict[str, Any] | None:
"""Return the newest unexpired, non-terminated reviewer lease for *pr_number*.
Walking backward past a terminal marker used to resurrect the older claim of
the very chain that marker ended, so a released/abandoned finalization still
read as an active lease here while ``reviewer_pr_lease`` reported it ended
(#742). A claim is now skipped when a later marker terminates its own chain.
"""
now = now or datetime.now(timezone.utc)
candidates = [
entry for entry in _comment_entries(comments, pr_number=pr_number)
if entry.get("lease_kind") == "reviewer"
]
for index in range(len(candidates) - 1, -1, -1):
lease = candidates[index]
if _lease_expired(lease, now=now):
continue
phase = (lease.get("phase") or "").strip().lower()
if phase in _TERMINAL_REVIEWER_PHASES:
continue
if phase in _ACTIVE_REVIEWER_PHASES or phase:
if _chain_terminated_after(candidates, index):
continue
return lease
return None
def find_active_conflict_fix_lease(
comments: list[dict],
*,
pr_number: int,
now: datetime | None = None,
) -> dict[str, Any] | None:
"""Return the newest unexpired conflict-fix lease for *pr_number*, if any."""
now = now or datetime.now(timezone.utc)
candidates = [
entry for entry in _comment_entries(comments, pr_number=pr_number)
if entry.get("lease_kind") == "conflict_fix"
]
for lease in reversed(candidates):
if _lease_expired(lease, now=now):
continue
phase = (lease.get("phase") or "").strip().lower()
if phase in _TERMINAL_CONFLICT_FIX_PHASES:
continue
if phase in _ACTIVE_CONFLICT_FIX_PHASES or phase:
return lease
return None
def format_conflict_fix_lease_body(
*,
pr_number: int,
branch: str,
worktree: str,
profile: str,
head_before: str,
phase: str = "claimed",
session_id: str = "unknown",
expires_at: datetime | None = None,
reviewer_active: bool = False,
) -> str:
expires = expires_at or (
datetime.now(timezone.utc) + timedelta(minutes=DEFAULT_CONFLICT_FIX_TTL_MINUTES)
)
expires_text = expires.astimezone(timezone.utc).replace(microsecond=0).isoformat().replace(
"+00:00", "Z"
)
lines = [
CONFLICT_FIX_LEASE_MARKER,
f"pr: #{pr_number}",
f"branch: {branch}",
f"worktree: {worktree}",
f"profile: {profile}",
f"session_id: {session_id}",
f"phase: {phase}",
f"head_before: {head_before}",
f"expires_at: {expires_text}",
f"reviewer_active: {'yes' if reviewer_active else 'no'}",
]
return "\n".join(lines)
def assess_head_sha_equality(
reviewed_head_sha: str | None,
live_head_sha: str | None,
) -> dict[str, Any]:
"""Fail closed when reviewed and live PR heads differ."""
reviewed = _normalize_sha(reviewed_head_sha)
live = _normalize_sha(live_head_sha)
reasons: list[str] = []
if not reviewed or not live:
reasons.append(
"reviewed/live head SHA missing or not full 40-hex; fail closed"
)
elif reviewed != live:
reasons.append(
"PR head changed after validation; re-pin and re-validate before "
"approval or merge"
)
proven = not reasons
return {
"proven": proven,
"block": not proven,
"reasons": reasons,
"reviewed_head_sha": reviewed,
"live_head_sha": live,
"head_changed": bool(reviewed and live and reviewed != live),
}
def assess_conflict_fix_push(
*,
pr_number: int,
comments: list[dict],
branch_head_before: str | None,
branch_head_after: str | None,
worktree_path: str | None,
push_cwd: str | None,
is_fast_forward: bool | None,
now: datetime | None = None,
) -> dict[str, Any]:
"""Author pre-push gate: block when a reviewer holds an active lease."""
now = now or datetime.now(timezone.utc)
reasons: list[str] = []
reviewer_lease = find_active_reviewer_lease(comments, pr_number=pr_number, now=now)
conflict_lease = find_active_conflict_fix_lease(comments, pr_number=pr_number, now=now)
if reviewer_lease:
reasons.append(
f"active reviewer lease on PR #{pr_number} "
f"(phase={reviewer_lease.get('phase')}); author push blocked"
)
head_before = _normalize_sha(branch_head_before)
head_after = _normalize_sha(branch_head_after)
if not head_before:
reasons.append("branch head before push missing or invalid SHA")
if head_after and head_before and head_before == head_after:
reasons.append("branch head unchanged; no push to perform")
worktree = (worktree_path or "").strip()
cwd = (push_cwd or "").strip()
if not worktree:
reasons.append("worktree path required for conflict-fix push proof")
elif cwd and worktree and not cwd.rstrip("/").endswith(worktree.rstrip("/").split("/")[-1]):
if worktree not in cwd:
reasons.append(
f"push cwd '{cwd}' does not match session worktree '{worktree}'"
)
if is_fast_forward is False:
reasons.append("non-fast-forward push rejected for conflict-fix (fail closed)")
if conflict_lease and conflict_lease.get("phase") == "pushing":
owner = conflict_lease.get("worktree")
if owner and worktree and owner != worktree:
reasons.append(
f"sibling conflict-fix lease active from worktree '{owner}'"
)
push_allowed = not reasons
return {
"push_allowed": push_allowed,
"block": not push_allowed,
"reasons": reasons,
"active_reviewer_lease": reviewer_lease,
"active_conflict_fix_lease": conflict_lease,
"branch_head_before": head_before,
"branch_head_after": head_after,
"reviewer_was_active": bool(reviewer_lease),
"fast_forward": is_fast_forward,
}
def assess_reviewer_mutation_blocked(
*,
pr_number: int,
comments: list[dict],
reviewed_head_sha: str | None,
live_head_sha: str | None,
mutation: str,
now: datetime | None = None,
) -> dict[str, Any]:
"""Reviewer gate: block when conflict-fix lease active or head moved."""
now = now or datetime.now(timezone.utc)
reasons: list[str] = []
conflict_lease = find_active_conflict_fix_lease(comments, pr_number=pr_number, now=now)
if conflict_lease and (conflict_lease.get("phase") or "") in _ACTIVE_CONFLICT_FIX_PHASES:
reasons.append(
f"active conflict-fix lease on PR #{pr_number} "
f"(phase={conflict_lease.get('phase')}); reviewer {mutation} blocked"
)
head_check = assess_head_sha_equality(reviewed_head_sha, live_head_sha)
if head_check["block"]:
reasons.extend(head_check["reasons"])
if not _normalize_sha(reviewed_head_sha):
reasons.append(
f"reviewed head SHA required before reviewer {mutation} (fail closed)"
)
allowed = not reasons
return {
"mutation_allowed": allowed,
"block": not allowed,
"reasons": reasons,
"active_conflict_fix_lease": conflict_lease,
"head_check": head_check,
"reviewed_head_sha": head_check.get("reviewed_head_sha"),
"live_head_sha": head_check.get("live_head_sha"),
"push_during_validation": bool(
conflict_lease and conflict_lease.get("phase") in {"pushing", "pushed"}
),
}
_REVIEWED_HEAD_RE = re.compile(
r"reviewed head sha\s*:\s*([0-9a-f]{40})",
re.IGNORECASE,
)
_LIVE_HEAD_BEFORE_APPROVAL_RE = re.compile(
r"(?:live head sha before approval|final live head sha before approval)\s*:\s*([0-9a-f]{40})",
re.IGNORECASE,
)
_LIVE_HEAD_BEFORE_MERGE_RE = re.compile(
r"(?:live head sha before merge|final live head sha before merge)\s*:\s*([0-9a-f]{40})",
re.IGNORECASE,
)
_PUSH_DURING_VALIDATION_RE = re.compile(
r"push(?:es)? occurred during validation\s*:\s*(yes|no|true|false)",
re.IGNORECASE,
)
_CONFLICT_HEAD_BEFORE_RE = re.compile(
r"branch head before push\s*:\s*([0-9a-f]{40})",
re.IGNORECASE,
)
_CONFLICT_HEAD_AFTER_RE = re.compile(
r"branch head after push\s*:\s*([0-9a-f]{40})",
re.IGNORECASE,
)
_REVIEWER_LEASE_STATUS_RE = re.compile(
r"active reviewer lease status\s*:\s*(.+)$",
re.IGNORECASE | re.MULTILINE,
)
_FAST_FORWARD_RE = re.compile(
r"whether push was fast-forward\s*:\s*(yes|no|true|false)",
re.IGNORECASE,
)
_REVIEWER_ACTIVE_RE = re.compile(
r"whether any reviewer was active\s*:\s*(yes|no|true|false)",
re.IGNORECASE,
)
# #698 phase detection for phase-specific head proofs.
_NO_REVIEWED_HEAD_RE = re.compile(
r"(?:reviewed head sha|candidate head sha)\s*:\s*none\b",
re.IGNORECASE,
)
_VERDICT_RECORDED_RE = re.compile(
r"review decision\s*:\s*(?:approve[d]?|request[_ ]changes)\b"
r"|review_status\s*:\s*(?:approved|request_changes)\b"
r"|terminal review mutation\s*:\s*(?!none\b)\S",
re.IGNORECASE,
)
_MERGE_ATTEMPTED_RE = re.compile(
r"merge result\s*:\s*(?:merged|success|performed|failed|attempted)\b"
r"|merge mutations\s*:\s*(?!none\b|not applicable\b)\S",
re.IGNORECASE,
)
_VALIDATION_STARTED_RE = re.compile(
r"validation\s*:\s*(?!none\b|not run\b|not applicable\b|not started\b)"
r"[^\n]*(?:pass|fail|ran|executed|\d+\s+passed)",
re.IGNORECASE,
)
def assess_reviewer_stale_head_final_report(report_text: str) -> dict[str, Any]:
"""Final-report proof for reviewed vs live head SHAs (#399 AC 6).
#698: head proofs are phase-specific. A legitimately blocked run that
never began validation (no reviewed head, no formal verdict, no merge)
owes none of them; approval-time and merge-time live-head proofs are
owed only once the corresponding phase actually begins.
"""
text = report_text or ""
reasons: list[str] = []
reviewed = _normalize_sha(_REVIEWED_HEAD_RE.search(text).group(1) if _REVIEWED_HEAD_RE.search(text) else None)
live_approval = _normalize_sha(
_LIVE_HEAD_BEFORE_APPROVAL_RE.search(text).group(1)
if _LIVE_HEAD_BEFORE_APPROVAL_RE.search(text)
else None
)
live_merge = _normalize_sha(
_LIVE_HEAD_BEFORE_MERGE_RE.search(text).group(1)
if _LIVE_HEAD_BEFORE_MERGE_RE.search(text)
else None
)
push_during = _PUSH_DURING_VALIDATION_RE.search(text)
# Phase detection from the report's own claims.
no_head_stated = bool(_NO_REVIEWED_HEAD_RE.search(text))
verdict_recorded = bool(_VERDICT_RECORDED_RE.search(text))
merge_attempted = bool(_MERGE_ATTEMPTED_RE.search(text))
validation_started = bool(reviewed) or bool(_VALIDATION_STARTED_RE.search(text))
blocked_before_validation = (
no_head_stated
and not reviewed
and not verdict_recorded
and not merge_attempted
and not validation_started
)
if blocked_before_validation:
return {
"proven": True,
"block": False,
"reasons": [],
"reviewed_head_sha": None,
"live_head_sha_before_approval": None,
"live_head_sha_before_merge": None,
"push_during_validation": (
push_during.group(1).lower() if push_during else None
),
"phase": "blocked_before_validation",
}
if not reviewed and not no_head_stated:
# The head must always be STATED — either a SHA or an explicit
# 'none'. Silence is not a phase claim and fails closed.
reasons.append(
"reviewed head SHA not stated in final report "
"(state the SHA or an explicit 'none')"
)
elif not reviewed and (validation_started or verdict_recorded or merge_attempted):
reasons.append("reviewed head SHA not stated in final report")
if verdict_recorded and not live_approval:
reasons.append("final live head SHA before approval not stated")
if merge_attempted and not live_merge:
reasons.append("final live head SHA before merge not stated")
if validation_started and not push_during:
reasons.append("whether push occurred during validation not stated")
if reviewed and live_approval and reviewed != live_approval:
reasons.append("live head before approval differs from reviewed head SHA")
elif reviewed and live_merge and reviewed != live_merge:
reasons.append("live head before merge differs from reviewed head SHA")
proven = not reasons
return {
"proven": proven,
"block": not proven,
"reasons": reasons,
"reviewed_head_sha": reviewed,
"live_head_sha_before_approval": live_approval,
"live_head_sha_before_merge": live_merge,
"push_during_validation": (push_during.group(1).lower() if push_during else None),
}
def assess_conflict_fix_final_report(report_text: str) -> dict[str, Any]:
"""Final-report proof for conflict-fix push sessions (#399 AC 7)."""
text = report_text or ""
reasons: list[str] = []
head_before = _normalize_sha(
_CONFLICT_HEAD_BEFORE_RE.search(text).group(1)
if _CONFLICT_HEAD_BEFORE_RE.search(text)
else None
)
head_after = _normalize_sha(
_CONFLICT_HEAD_AFTER_RE.search(text).group(1)
if _CONFLICT_HEAD_AFTER_RE.search(text)
else None
)
if not head_before:
reasons.append("branch head before push not stated")
if not head_after:
reasons.append("branch head after push not stated")
if not _REVIEWER_LEASE_STATUS_RE.search(text):
reasons.append("active reviewer lease status not stated")
if not _FAST_FORWARD_RE.search(text):
reasons.append("whether push was fast-forward not stated")
if not _REVIEWER_ACTIVE_RE.search(text):
reasons.append("whether any reviewer was active not stated")
proven = not reasons
return {
"proven": proven,
"block": not proven,
"reasons": reasons,
"branch_head_before": head_before,
"branch_head_after": head_after,
}