Scaled-Tech-Consulting/Gitea-Tools
2
0
Fork 0
Code Issues 92 Pull Requests 4 Actions Packages Projects Releases Wiki Activity

fix(drain-proof): bind acknowledgement coverage to session identity (#661)

Browse Source 
Review 582 (REQUEST_CHANGES at 95178349) found a residual fail-open of the
same class the PR set out to close. Acknowledgement coverage was decided by
comparing a count against a count:

    covers_live_sessions = live_count_known and acked_count >= sessions_live_other

Nothing bound an acknowledgement to the identity of a session that actually
owed one, so acknowledgements supplied for the requesting session and for a
session that does not exist satisfied the obligations of two live sessions
that never answered - minting a clean, correctly signed proof and an allow
verdict from the restart gate.

Coverage is now derived from authoritative impact-report evidence:

- New `_required_ack_sessions()` derives the required session ids from the
  report itself, via `ack_state` keys and/or `affected_sessions` filtered on
  `live and not is_requester`. The requester is excluded only on explicit
  `is_requester` evidence, never inferred.
- When both views are present they must name the same set, and the result is
  reconciled against `counts.sessions_live_other`. Missing, malformed,
  duplicated, contradictory, or unreconcilable identity evidence fails closed
  and outranks every permitting path, including the timeout policy.
- Coverage requires every required id to carry an explicit acknowledgement
  token keyed by that id. Acknowledgements for the requester, for unknown
  ids, or for fabricated ids never increase coverage.
- Caller-supplied acknowledgement cardinality is no longer proof of anything.

Failure propagates unchanged through `acks_or_timeout` -> `proof.clean` ->
`failed_checks` -> `gate_apply_restart` verdict `deny` / `allow=False`.

The earlier missing-acknowledgement remediation is preserved in full: absent,
None, non-mapping, empty, partial, stale, and unparseable acks still fail
closed, `ack_timeout_policy_applied` stays strict `value is True`, and the
legitimate zero-live-sessions and explicit-timeout paths still pass.

Reviewer's reproduction, before and after this commit:

    sessions_live_other = 2
    report ack_state    = {'other-0': 'pending', 'other-1': 'pending'}
    supplied acks       = {'req': 'ack', 'totally-bogus-session': 'ack'}

    before: acks_or_timeout = True  | proof.clean = True  | gate allow
    after:  acks_or_timeout = False | proof.clean = False | gate deny

Tests: 22 new cases in `AcknowledgementIdentityBindingTests` covering the
reviewer's exact exploit, wrong-ids-with-sufficient-count, partial identity
match, requester-only acks, fabricated ids, unproven per-session states,
missing/malformed/contradictory identity evidence, count mismatch, and the
preserved success paths.

Verification:
- `pytest tests/test_drain_proof.py` -> 61 passed, 56 subtests
  (baseline at 95178349: 39 passed, 26 subtests)
- Restart surface (6 modules) -> 154 passed, 68 subtests, exit 0
  (baseline at 95178349: 132 passed, 38 subtests)
- Full `pytest tests/` -> 5177 passed vs baseline 5155 passed; the 23
  failures are identical in both runs and pre-exist at 95178349.

Scope: drain_proof.py, tests/test_drain_proof.py.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01VRUZAf3Fr5n3kqhhiayN6C
(cherry picked from commit 4193b63f415b066ee292386c2c89bc3d2651a0cc)
This commit is contained in:
sysadmin
2026-07-25 00:04:17 -04:00
parent 2068bae341
commit 3a9d634c17
2 changed files with 522 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
drain_proof.py
+211 -27
View File
@@ -328,6 +328,160 @@ def _is_acknowledged(value: Any) -> bool:
return value.strip().lower() in _ACK_TOKENS
def _session_ids_from_affected_sessions(
raw: Any,
) -> tuple[set[str] | None, str | None]:
"""Live, non-requester session ids from the report's ``affected_sessions``.
The requester is excluded only on authoritative ``is_requester`` evidence;
an entry whose ``live`` or ``is_requester`` flag is absent or not a real
bool is malformed, never assumed. Returns ``(ids, None)`` or
``(None, detail)``.
"""
if not isinstance(raw, (list, tuple)):
return None, (
"impact report 'affected_sessions' is malformed: expected a list, "
f"got {type(raw).__name__} (fail closed)"
)
collected: list[str] = []
for index, entry in enumerate(raw):
if not isinstance(entry, Mapping):
return None, (
f"impact report 'affected_sessions[{index}]' is malformed: "
f"expected a mapping, got {type(entry).__name__} (fail closed)"
)
session_id = entry.get("session_id")
if not isinstance(session_id, str) or not session_id.strip():
return None, (
f"impact report 'affected_sessions[{index}]' has no usable "
"session_id (fail closed)"
)
live = entry.get("live")
is_requester = entry.get("is_requester")
if not isinstance(live, bool) or not isinstance(is_requester, bool):
return None, (
f"impact report 'affected_sessions[{index}]' "
f"({session_id.strip()}) does not prove live/is_requester with "
"explicit booleans (fail closed)"
)
if live and not is_requester:
collected.append(session_id.strip())
if len(set(collected)) != len(collected):
return None, (
"impact report 'affected_sessions' names the same live session more "
"than once; required acknowledgement identities are ambiguous "
"(fail closed)"
)
return set(collected), None
def _session_ids_from_ack_state(raw: Any) -> tuple[set[str] | None, str | None]:
"""Session ids keyed by the report's ``ack_state``.
``ack_state`` is minted as ``{s.session_id: "pending" for s in
other_live_sessions}``, so its keys *are* the required set. Only the keys
are trusted here; the per-session value is the report's own placeholder and
is never read as an acknowledgement (acknowledgements come from the drain
state and must be explicit).
"""
if not isinstance(raw, Mapping):
return None, (
"impact report 'ack_state' is malformed: expected a mapping, got "
f"{type(raw).__name__} (fail closed)"
)
keys: set[str] = set()
for key in raw:
if not isinstance(key, str) or not key.strip():
return None, (
"impact report 'ack_state' carries a non-string or empty "
"session id (fail closed)"
)
keys.add(key.strip())
if len(keys) != len(raw):
return None, (
"impact report 'ack_state' names the same live session more than "
"once; required acknowledgement identities are ambiguous "
"(fail closed)"
)
return keys, None
def _required_ack_sessions(
impact_report: Mapping[str, Any],
) -> tuple[frozenset[str] | None, str | None]:
"""Identities of the live, non-requester sessions that owe an acknowledgement.
Derived only from authoritative impact-report evidence, never from the
caller-supplied drain state — the same principle checks 1 and 5 already
apply. Returns ``(ids, None)`` when the report proves the required set, or
``(None, detail)`` when the evidence is missing, malformed, contradictory,
or cannot be reconciled; every one of those fails the checklist closed.
The report carries two independent views of the same set and both are
validated: ``affected_sessions`` filtered on ``live and not is_requester``,
and ``ack_state`` whose keys are exactly those sessions. When both are
present they must name the same set, and the result is reconciled against
``counts.sessions_live_other`` — a report that disagrees with itself can
never authorise a restart.
"""
from_sessions: set[str] | None = None
if "affected_sessions" in impact_report:
from_sessions, detail = _session_ids_from_affected_sessions(
impact_report.get("affected_sessions")
)
if detail is not None:
return None, detail
from_ack_state: set[str] | None = None
if "ack_state" in impact_report:
from_ack_state, detail = _session_ids_from_ack_state(
impact_report.get("ack_state")
)
if detail is not None:
return None, detail
if from_sessions is None and from_ack_state is None:
return None, (
"impact report carries no session-identity evidence (neither "
"ack_state nor affected_sessions); required acknowledgements "
"cannot be attributed to a session (fail closed)"
)
if (
from_sessions is not None
and from_ack_state is not None
and from_sessions != from_ack_state
):
only_ack_state = sorted(from_ack_state - from_sessions)
only_sessions = sorted(from_sessions - from_ack_state)
return None, (
"impact report contradicts itself about which sessions must "
f"acknowledge: ack_state-only={only_ack_state}, "
f"affected_sessions-only={only_sessions} (fail closed)"
)
required = from_ack_state if from_ack_state is not None else from_sessions
if required is None: # unreachable; guarded above, kept fail-closed
return None, "required acknowledgement identities unresolved (fail closed)"
live_count = _live_session_count(impact_report)
if live_count is None:
return None, (
"impact report does not prove the live-session count; required "
"acknowledgement identities cannot be reconciled (fail closed)"
)
if len(required) != live_count:
return None, (
f"impact report names {len(required)} live session(s) requiring "
f"acknowledgement but counts.sessions_live_other={live_count}; "
"identity evidence cannot be reconciled (fail closed)"
)
return frozenset(required), None
def _evaluate_checks(
impact_report: Mapping[str, Any],
drain_state: Mapping[str, Any],
@@ -444,27 +598,55 @@ def _evaluate_checks(
# 6. Acknowledgements received, or an explicit timeout policy was applied.
#
# Whether acknowledgements are *required* is answered by the impact report,
# never inferred from the shape of the drain state. Previously an absent
# ``acks`` key collapsed to ``{}`` and was read as "no session needed to
# acknowledge", so a proof minted clean — and the restart gate allowed —
# while the report still showed other live sessions. Absence of evidence is
# not evidence of absence: missing, malformed, stale, or otherwise unproven
# acknowledgement data fails closed whenever live sessions require
# acknowledgement, and only explicitly verified acknowledgement evidence
# may permit the operation.
# Whether acknowledgements are *required*, and *which sessions owe them*,
# are both answered by the impact report — never inferred from the shape or
# the cardinality of the drain state. Two fail-opens of the same class were
# closed here in turn. First, an absent ``acks`` key collapsed to ``{}`` and
# was read as "no session needed to acknowledge", so a proof minted clean
# while the report still showed other live sessions. Then coverage was
# decided by comparing an acknowledgement *count* against
# ``sessions_live_other``, so acknowledgements from the requesting session
# or from ids that do not exist satisfied the obligations of the live
# sessions that never answered.
#
# Coverage is therefore bound to identity: every live, non-requester session
# the report names must itself carry an explicit acknowledgement. An
# acknowledgement from any other id — the requester, an unknown id, a
# fabricated one — is not evidence about a required session and never
# increases coverage.
sessions_live_other = _live_session_count(impact_report)
live_count_known = sessions_live_other is not None
# An unknown or malformed count fails closed: it cannot prove nobody had to
# acknowledge, so acknowledgement stays required.
acks_required = (not live_count_known) or sessions_live_other > 0
required_sessions, identity_detail = _required_ack_sessions(impact_report)
# An unknown count or unresolvable identity evidence fails closed: neither
# can prove nobody had to acknowledge, so acknowledgement stays required.
acks_required = required_sessions is None or bool(required_sessions)
raw_acks = drain_state.get("acks")
acks_present = isinstance(raw_acks, Mapping)
ack_values = list(raw_acks.values()) if acks_present else []
acked_count = sum(1 for value in ack_values if _is_acknowledged(value))
every_entry_acked = bool(ack_values) and acked_count == len(ack_values)
covers_live_sessions = live_count_known and acked_count >= sessions_live_other
# Identity-bound coverage. Only an entry keyed by a required session id and
# carrying an explicit token counts, so neither a requester acknowledgement
# nor an unknown id can stand in for a session that stayed silent.
if acks_present and required_sessions is not None:
acked_sessions = {
session_id
for session_id in required_sessions
if any(
isinstance(key, str)
and key.strip() == session_id
and _is_acknowledged(value)
for key, value in raw_acks.items()
)
}
else:
acked_sessions = set()
missing_sessions = (
sorted(required_sessions - acked_sessions)
if required_sessions is not None
else []
)
covers_live_sessions = required_sessions is not None and not missing_sessions
# Strict by construction: only an explicit ``True`` counts, so an absent,
# null, or non-boolean ``ack_timeout_policy_applied`` can never open this
# gate on its own.
@@ -476,7 +658,13 @@ def _evaluate_checks(
# is that an acknowledgement is still owed.
outstanding_entries = acks_present and bool(ack_values) and not every_entry_acked
if timeout_policy and (acks_required or outstanding_entries):
if identity_detail is not None:
# Unresolvable identity evidence outranks every permitting path,
# including the timeout policy: when the report cannot say who owed an
# acknowledgement, nothing can show the obligation was discharged.
acks_ok = False
detail = identity_detail
elif timeout_policy and (acks_required or outstanding_entries):
acks_ok = True
detail = "explicit ack timeout policy applied and recorded"
elif outstanding_entries:
@@ -492,21 +680,16 @@ def _evaluate_checks(
"impact report proves no other live sessions required to "
"acknowledge (sessions_live_other=0)"
)
elif every_entry_acked and covers_live_sessions:
elif covers_live_sessions:
acks_ok = True
detail = (
f"{acked_count} acknowledgement entr"
f"{'y' if acked_count == 1 else 'ies'} verified; covers "
f"{len(acked_sessions)} required live session(s) each acknowledged "
f"by identity: {sorted(acked_sessions)}; covers "
f"sessions_live_other={sessions_live_other}"
)
else:
acks_ok = False
if not live_count_known:
detail = (
"impact report does not prove the live-session count; "
"acknowledgement required and unproven (fail closed)"
)
elif raw_acks is None:
if raw_acks is None:
detail = (
"acknowledgement evidence absent while "
f"sessions_live_other={sessions_live_other} require "
@@ -525,9 +708,10 @@ def _evaluate_checks(
)
else:
detail = (
f"acknowledgements cover only {acked_count} session(s) but the "
f"report shows sessions_live_other={sessions_live_other} "
"(fail closed)"
"no valid acknowledgement attributed to required live "
f"session(s) {missing_sessions}; the report shows "
f"sessions_live_other={sessions_live_other} and supplied "
"acknowledgements for other ids do not count (fail closed)"
)
checks.append(DrainCheck(CHECK_ACKS_OR_TIMEOUT, acks_ok, detail))