Compare commits

...
Author SHA1 Message Date
sysadmin 22e0a41bd5 feat(restart): audit lifecycle events and durable incidents (#665)
Add restart_audit with mcp.restart.* event schema, redacted emission via
gitea_audit, correlation ids, and incident materialization for failed drain
and break-glass. Wire gitea_request_mcp_restart to always audit impact
previews and fail closed on privileged apply when the audit sink is enabled
but write fails.

Closes #665
2026-07-25 17:10:16 -04:00
sysadmin 76f293eb28 Merge pull request 'fix(gate): stop classifying stale-runtime blocks as permission denials (Closes #897)' (#901) from fix/issue-897-permission-stale-runtime-classification into master 2026-07-25 06:56:16 -05:00
jcwalker3 54559aebc3 Merge branch 'master' into fix/issue-897-permission-stale-runtime-classification 2026-07-25 06:42:27 -05:00
sysadmin 715863799f Merge pull request 'feat(webui): Runtime and session view (Phase 1) (Closes #641)' (#898) from feat/issue-641-runtime-session-view into master 2026-07-25 05:50:48 -05:00
sysadmin 6e6ca94338 fix(gate): stop classifying stale-runtime blocks as permission denials (Closes #897)
Stale-runtime and runtime-mode mutation refusals previously shared the
permission-denial channel, so permission_report claimed a missing op the
active profile already held and recommended gitea_activate_profile.
Typed blocker_kind payloads report reconnect-only recovery for staleness,
omit permission_report for non-permission gates, and fail closed when a
permission_report would invent a missing permission the profile holds.
2026-07-25 01:47:35 -04:00
6 changed files with 1724 additions and 70 deletions
+93
View File
@@ -0,0 +1,93 @@
# MCP restart audit events and incidents (#665)
Restarts and recovery attempts leave a forensic trail. Failed drains and
break-glass paths also raise durable Gitea incident issues so unsafe restarts
cannot be silently repeated.
Parent umbrella: **#655**. Related: impact coordinator **#658**, drain proof
**#661**, restart classes **#663**, break-glass **#664**, post-restart reconcile
**#662**, vision **#652**, roadmap **#653**, console recovery **#642**.
## Components
| Piece | Where | Responsibility |
|-------|-------|----------------|
| Event schema + emission | `restart_audit.py` | `mcp.restart.*` vocabulary, redacted payload builder, append-only sink via `gitea_audit` |
| Fail-closed privileged gate | `restart_audit.require_audit_or_deny` | When `GITEA_AUDIT_LOG` is set and the write fails, privileged apply is denied |
| Incident descriptors | `restart_audit.build_incident_descriptor` | Durable follow-up issues (failed drain, break-glass, reconcile unresolved, unguarded) |
| Materializer | `restart_audit.materialize_incident` | Injected `create_issue_fn` (network kept out of pure tests) |
| Wiring | `gitea_request_mcp_restart` | Correlation id, impact-preview audit, apply-gate / break-glass audit + incident creation |
## Event vocabulary
| Event type | When |
|------------|------|
| `mcp.restart.impact_preview` | Every `gitea_request_mcp_restart` evaluation |
| `mcp.restart.drain_enter` | Drain window starts (schema reserved; emit from drain path) |
| `mcp.restart.drain_exit` | Drain window ends |
| `mcp.restart.drain_proof` | Drain-proof verification result |
| `mcp.restart.apply_gate` | Apply hard gate (`dry_run=False`) |
| `mcp.restart.break_glass` | Authorized break-glass bypass |
| `mcp.restart.post_restart_reconcile` | Post-restart reconcile outcome |
| `mcp.restart.narrower_recovery` | Narrower recovery attempt recorded |
| `mcp.restart.unguarded_detected` | Unguarded restart path detected |
All free text is redacted before sink write or issue body assembly. Emission
never raises; callers decide fail-closed policy.
## Correlation
Each restart lifecycle mints a short `correlation_id` (`rst-` + 16 hex) shared
across impact preview → apply gate → incident descriptors so operators can join
the trail.
## Privileged deny-on-audit-fail
Rollout policy (issue #665):
1. Configure `GITEA_AUDIT_LOG` so writes land.
2. Only then enforce deny when a privileged restart path cannot audit.
When audit is **not** configured, privileged apply still proceeds (no false
denials during rollout). When audit **is** configured and the write fails,
`apply_authorized` is cleared.
## Incidents
| Kind | Trigger |
|------|---------|
| `restart_failed_drain` | Apply denied by drain hard gate / failed proof |
| `restart_break_glass` | Any authorized break-glass apply |
| `restart_reconcile_unresolved` | Post-restart reconcile left work unresolved |
| `restart_unguarded_detected` | Unguarded restart attempt detected |
Break-glass **always** creates an incident descriptor (and a Gitea issue when
the create path is available). Failed drain does the same. Incident bodies
include correlation id, session, class, scope, proof id, and redacted reasons.
Default labels: `mcp-health`, `safety`, `observability`, `status:ready`,
`type:bug`, `workflow-hardening`.
## Tool payload surface
`gitea_request_mcp_restart` returns:
* `correlation_id` — lifecycle join key
* `restart_audit.impact_preview_written` — sink success for the preview event
* `restart_audit.apply_gate_written` — sink success for apply/break-glass (apply only)
* `restart_audit.incident_result` — materialization outcome when an incident was required
* `incident` — durable descriptor (when gate requires follow-up)
## Security
* No secrets in audit payloads or issue bodies.
* This module never restarts a process.
* Drain proof verification remains #661; audit only records the decision.
* Incident creation failures are recorded in `incident_result.reasons` and never
crash the restart evaluation path (audit write failure still fails closed for
privileged apply when the sink is enabled).
## Tests
See `tests/test_restart_audit.py`: schema, redaction, emission, deny policy,
incident materialization mocks, break-glass / failed-drain selection.
+1
View File
@@ -33,6 +33,7 @@ recovery behavior for all nine classes.
| `ControlPlaneDB.list_sessions` | `control_plane_db.py` | Read-only session inventory (the process-level unit a restart kills). |
| `gitea_request_mcp_restart` | `gitea_mcp_server.py` | MCP tool: gathers inventory from the #613 DB, calls the coordinator, returns the report, and on `dry_run=False` runs the #661 drain-proof hard gate. Never restarts a process. |
| `drain_proof.gate_apply_restart` | `drain_proof.py` | The #661 hard gate: verifies a drain proof against the current impact fingerprint, or records an authorized break-glass bypass. |
| `restart_audit` | `restart_audit.py` | #665 forensic trail: `mcp.restart.*` events via `gitea_audit`, correlation ids, durable incidents for failed drain / break-glass. See [`mcp-restart-audit.md`](./mcp-restart-audit.md). |
## Dimensions evaluated
+409 -70
View File
@@ -2069,6 +2069,7 @@ import lease_policy # noqa: E402
import workflow_dashboard # noqa: E402 # #605 live queue/lease dashboard
import restart_coordinator # noqa: E402 # #658 MCP restart coordinator/impact
import drain_proof # noqa: E402 # #661 pre-restart drain proof and hard gate
import restart_audit # noqa: E402 # #665 restart audit events + incidents
import incident_bridge # noqa: E402
import sentry_observability # noqa: E402 (#606 optional Sentry observability)
import sentry_incident_bridge # noqa: E402 (#607 Sentry→Gitea incident bridge)
@@ -9552,15 +9553,13 @@ def gitea_edit_pr(
if closing:
gate_reasons = _profile_operation_gate("gitea.pr.close")
if gate_reasons:
return {
"success": False,
"performed": False,
"pr_number": pr_number,
"requested_state": "closed",
"required_permission": "gitea.pr.close",
"reasons": gate_reasons,
"permission_report": _permission_block_report("gitea.pr.close"),
}
return _build_operation_gate_refusal(
"gitea.pr.close",
gate_reasons,
pr_number=pr_number,
requested_state="closed",
required_permission="gitea.pr.close",
)
h, o, r = _resolve(remote, host, org, repo)
auth = _auth(h)
@@ -13823,13 +13822,17 @@ def gitea_view_issue(
def _permission_block_report(required_operation: str,
identity: str | None = None) -> dict:
"""Structured, LLM-safe explanation of a permission denial (#142).
"""Structured, LLM-safe explanation of a permission denial (#142, #897).
Built only after a gate has already refused; it adds guidance to the
refusal and never widens any permission, performs network I/O, or
raises (fail-soft: degrades to a minimal fail-closed report). Names
configured profiles only never auth references, tokens, endpoint
URLs, or keychain IDs.
#897: never fabricate a missing permission when the active profile
already allows the operation. That path is a diagnostic defect (the
refusal was not a permission denial), not a cue to switch profiles.
"""
report = {
"requested_operation": required_operation,
@@ -13841,6 +13844,7 @@ def _permission_block_report(required_operation: str,
"matching_configured_profiles": [],
"runtime_switching_supported": False,
"different_mcp_namespace_required": True,
"diagnostic_defect": False,
"exact_safe_next_action": (
"Ask the operator to fix GITEA_MCP_CONFIG/GITEA_MCP_PROFILE; "
"the active profile could not be resolved (fail closed)."),
@@ -13855,6 +13859,32 @@ def _permission_block_report(required_operation: str,
report["active_allowed_operations"] = (
profile.get("allowed_operations") or [])
# #897: fail closed as a diagnostic defect when the active profile
# already holds the operation — callers must not invent "missing".
try:
holds, _hold_reason = gitea_config.check_operation(
required_operation,
profile.get("allowed_operations") or [],
profile.get("forbidden_operations") or [],
)
except Exception:
holds = False
if holds:
report["missing_permission"] = None
report["required_permission"] = required_operation
report["diagnostic_defect"] = True
report["different_mcp_namespace_required"] = False
report["exact_safe_next_action"] = (
"Diagnostic defect: the active profile already allows "
f"{required_operation}. This is not a permission denial — "
"inspect blocker_kind / reasons (stale-runtime or runtime-mode). "
"Do not call gitea_activate_profile or switch MCP sessions."
)
report["matching_configured_profiles"] = [
p for p in [profile.get("profile_name")] if p
]
return report
matching = []
try:
config = gitea_config.load_config() or {}
@@ -13903,6 +13933,205 @@ def _permission_block_report(required_operation: str,
return report
def _reason_is_stale_runtime(reason: str) -> bool:
"""True when *reason* is a master-parity / stale-daemon refusal (#897)."""
r = (reason or "").lower()
if not r:
return False
if "stale relative to live master" in r:
return True
if "server code is stale" in r:
return True
if "daemon is stale" in r:
return True
if "started at commit" in r and "workspace master is now" in r:
return True
if "mcp server started at" in r and "stale" in r:
return True
if "restart the server to load the current capability gates" in r:
return True
if "restart/reconnect before mutating" in r:
return True
return False
def _reason_is_runtime_mode(reason: str) -> bool:
"""True when *reason* is a stable-control / runtime-mode refusal (#897)."""
r = (reason or "").lower()
if not r:
return False
if _reason_is_stale_runtime(reason):
return False
if "runtime mode could not be assessed" in r:
return True
if "runtime mode is" in r:
return True
if "stable control runtime" in r:
return True
if "dev-test" in r and ("runtime" in r or "production" in r):
return True
if "development worktree" in r or "dev worktree" in r:
return True
if "launched from a 'branches/" in r or "launched from a \"branches/" in r:
return True
if "process-root / active-workspace alignment" in r:
return True
if "namespace" in r and "reproof" in r:
return True
return False
def _reason_is_permission(reason: str) -> bool:
"""True when *reason* is a genuine profile-permission denial (#897)."""
r = (reason or "").lower()
if not r:
return False
if _reason_is_stale_runtime(reason) or _reason_is_runtime_mode(reason):
return False
if "profile could not be resolved" in r:
return True
if "profile has no configured allowed operations" in r:
return True
if "profile forbids" in r:
return True
if "profile is not allowed to" in r:
return True
if "unrecognized forbidden operation" in r:
return True
return False
def _classify_operation_gate_reasons(reasons: list[str]) -> dict:
"""Partition gate reasons into stale / runtime-mode / permission (#897)."""
stale: list[str] = []
runtime_mode: list[str] = []
permission: list[str] = []
other: list[str] = []
for reason in reasons or []:
if _reason_is_stale_runtime(reason):
stale.append(reason)
elif _reason_is_runtime_mode(reason):
runtime_mode.append(reason)
elif _reason_is_permission(reason):
permission.append(reason)
else:
other.append(reason)
return {
"stale_runtime": stale,
"runtime_mode": runtime_mode,
"permission": permission,
"other": other,
}
def _stale_runtime_reconnect_action() -> str:
"""Sanctioned recovery for a stale daemon — reconnect only (#685/#897)."""
return (
"Reconnect the IDE/client MCP session so the server reloads at the "
"current master head. Do not call gitea_activate_profile or switch "
"MCP role sessions — profile switching does not clear a stale daemon."
)
def _build_operation_gate_refusal(
required_operation: str,
reasons: list[str],
**extra_fields,
) -> dict:
"""Structured gate refusal with typed blockers (#897).
Stale-runtime and runtime-mode refusals never attach a
``permission_report`` and never recommend profile switching. True
permission denials still get ``permission_report``. When both apply,
causes are reported separately under distinct fields.
"""
classified = _classify_operation_gate_reasons(reasons)
stale = classified["stale_runtime"]
runtime_mode = classified["runtime_mode"]
permission = classified["permission"]
other = classified["other"]
blocked: dict = {
"success": False,
"performed": False,
"reasons": list(reasons),
"mutation_performed": False,
"session_context_audit": session_ctx.mutation_context_audit_fields(),
"gate_reason_classes": {
"stale_runtime": list(stale),
"runtime_mode": list(runtime_mode),
"permission": list(permission),
"other": list(other),
},
}
if stale:
parity = _current_master_parity()
blocked["blocker_kind"] = "runtime_reconnect_required"
blocked["restart_required"] = True
blocked["stop_required"] = True
blocked["startup_head"] = parity.get("startup_head")
blocked["current_head"] = parity.get("current_head")
blocked["daemon_start_head"] = (
parity.get("daemon_start_head") or parity.get("startup_head")
)
blocked["local_head"] = (
parity.get("local_head") or parity.get("current_head")
)
blocked["live_remote_head"] = parity.get("live_remote_head")
blocked["live_stale"] = bool(parity.get("live_stale"))
blocked["live_known"] = bool(parity.get("live_known"))
blocked["exact_safe_next_action"] = _stale_runtime_reconnect_action()
if permission or other:
blocked["permission_block_reasons"] = list(permission) + list(other)
blocked["stale_runtime_reasons"] = list(stale)
# Never attach permission_report for a staleness refusal.
blocked.update(extra_fields)
return blocked
if runtime_mode:
blocked["blocker_kind"] = "runtime_mode_blocked"
blocked["restart_required"] = False
blocked["stop_required"] = True
blocked["exact_safe_next_action"] = (
"Real workflow mutations run only on the promoted stable control "
"runtime. Promote/reload the stable runtime; do not call "
"gitea_activate_profile or switch MCP role sessions to clear a "
"runtime-mode block."
)
if permission or other:
blocked["permission_block_reasons"] = list(permission) + list(other)
blocked["runtime_mode_reasons"] = list(runtime_mode)
blocked.update(extra_fields)
return blocked
# Pure permission (or unclassified-as-permission) denial.
blocked["blocker_kind"] = "permission_denied"
blocked["permission_report"] = _permission_block_report(required_operation)
blocked.update(extra_fields)
return blocked
def _permission_report_for_gate_reasons(
required_operation: str,
reasons: list[str] | None,
) -> dict | None:
"""Attach ``permission_report`` only for true permission denials (#897).
Call sites that historically always attached a permission report after
``_profile_operation_gate`` should use this so stale/runtime refusals
do not emit a fabricated missing-permission payload.
"""
if not reasons:
return None
classified = _classify_operation_gate_reasons(reasons)
if classified["stale_runtime"] or classified["runtime_mode"]:
return None
if not (classified["permission"] or classified["other"]):
return None
return _permission_block_report(required_operation)
def _role_for_operation(op: str) -> str | None:
# Normalize op first
try:
@@ -14079,7 +14308,7 @@ def _master_parity_block(op: str) -> list[str]:
def _profile_operation_gate(op: str) -> list[str]:
"""Profile permission check for a single gated operation (#126, #216, #420).
"""Profile permission check for a single gated operation (#126, #216, #420, #897).
Issue discussion comments are gated separately from the gitea.pr.*
review/merge family: listing requires ``gitea.read``, creating requires
@@ -14092,21 +14321,26 @@ def _profile_operation_gate(op: str) -> list[str]:
capability gate that has since been merged, and when the runtime itself is
not the promoted stable control runtime (#615) -- a dev/test or unknown
runtime holds production credentials but has not been promoted.
#897: collect *all* independent refusal classes (stale, runtime-mode,
permission) rather than short-circuiting after the first. Callers that
only need a boolean still treat any non-empty list as blocked; typed
consumers (``_build_operation_gate_refusal``) can separate causes.
"""
stale_reasons = _master_parity_block(op)
if stale_reasons:
return stale_reasons
runtime_reasons = _runtime_mode_block(op)
if runtime_reasons:
return runtime_reasons
reasons: list[str] = []
reasons.extend(_master_parity_block(op))
reasons.extend(_runtime_mode_block(op))
try:
profile = get_profile()
except Exception as exc:
return [f"profile could not be resolved (fail closed): {_redact(str(exc))}"]
reasons.append(
f"profile could not be resolved (fail closed): {_redact(str(exc))}"
)
return reasons
op_ok, op_reason = gitea_config.check_operation(
op, profile["allowed_operations"], profile["forbidden_operations"])
if op_ok:
return []
return reasons
if _try_auto_switch_for_operation(op):
try:
@@ -14114,17 +14348,26 @@ def _profile_operation_gate(op: str) -> list[str]:
op_ok, op_reason = gitea_config.check_operation(
op, profile["allowed_operations"], profile["forbidden_operations"])
if op_ok:
return []
return reasons
except Exception as exc:
return [f"profile could not be resolved (fail closed): {_redact(str(exc))}"]
reasons.append(
f"profile could not be resolved (fail closed): {_redact(str(exc))}"
)
return reasons
if op_reason == "no-allowed-operations":
return ["profile has no configured allowed operations (fail closed)"]
if op_reason == "forbidden":
return [f"profile forbids '{op}'"]
if op_reason == "invalid-forbidden-entry":
return ["profile has an unrecognized forbidden operation entry (fail closed)"]
return [f"profile is not allowed to {op}"]
reasons.append(
"profile has no configured allowed operations (fail closed)"
)
elif op_reason == "forbidden":
reasons.append(f"profile forbids '{op}'")
elif op_reason == "invalid-forbidden-entry":
reasons.append(
"profile has an unrecognized forbidden operation entry (fail closed)"
)
else:
reasons.append(f"profile is not allowed to {op}")
return reasons
def _mutation_config_authority_block(required_operation: str) -> dict | None:
@@ -14344,10 +14587,14 @@ def _session_context_mutation_block(
def _profile_permission_block(required_operation: str, **extra_fields) -> dict | None:
"""Structured permission denial for gated tools (#69, #142).
"""Structured operation-gate denial for gated tools (#69, #142, #897).
Returns a block dict when the active profile forbids *required_operation*,
or ``None`` when the gate passes. Never performs network I/O.
the daemon is stale, or the runtime mode is not mutation-safe or
``None`` when the gate passes. Never performs network I/O.
#897: stale-runtime and runtime-mode refusals are typed
(``blocker_kind``) and never carry a ``permission_report``.
"""
req_role = "reviewer" if any(required_operation.startswith(p) for p in (
"gitea.pr.approve", "gitea.pr.merge", "gitea.pr.request_changes", "gitea.pr.review"
@@ -14357,15 +14604,9 @@ def _profile_permission_block(required_operation: str, **extra_fields) -> dict |
reasons = _profile_operation_gate(required_operation)
if reasons:
blocked = {
"success": False,
"performed": False,
"reasons": reasons,
"permission_report": _permission_block_report(required_operation),
"session_context_audit": session_ctx.mutation_context_audit_fields(),
}
blocked.update(extra_fields)
return blocked
return _build_operation_gate_refusal(
required_operation, reasons, **extra_fields
)
auth_block = _mutation_config_authority_block(required_operation)
if auth_block is not None:
@@ -14494,20 +14735,14 @@ def gitea_acquire_reviewer_pr_lease(
"""Acquire a per-PR reviewer lease before review/merge mutations (#407)."""
read_block = _profile_operation_gate("gitea.read")
if read_block:
return {
"success": False,
"acquired": False,
"reasons": read_block,
"permission_report": _permission_block_report("gitea.read"),
}
return _build_operation_gate_refusal(
"gitea.read", read_block, acquired=False
)
comment_block = _profile_operation_gate("gitea.pr.comment")
if comment_block:
return {
"success": False,
"acquired": False,
"reasons": comment_block,
"permission_report": _permission_block_report("gitea.pr.comment"),
}
return _build_operation_gate_refusal(
"gitea.pr.comment", comment_block, acquired=False
)
# task=acquire_reviewer_pr_lease so verify_preflight_purity runs shared #604
# anti-stomp for the declared lease-acquire mutation inventory entry.
@@ -14623,20 +14858,14 @@ def gitea_acquire_merger_pr_lease(
"""
read_block = _profile_operation_gate("gitea.read")
if read_block:
return {
"success": False,
"acquired": False,
"reasons": read_block,
"permission_report": _permission_block_report("gitea.read"),
}
return _build_operation_gate_refusal(
"gitea.read", read_block, acquired=False
)
comment_block = _profile_operation_gate("gitea.pr.comment")
if comment_block:
return {
"success": False,
"acquired": False,
"reasons": comment_block,
"permission_report": _permission_block_report("gitea.pr.comment"),
}
return _build_operation_gate_refusal(
"gitea.pr.comment", comment_block, acquired=False
)
merge_block = _profile_operation_gate("gitea.pr.merge")
if merge_block:
return {
@@ -19375,14 +19604,12 @@ def gitea_update_pr_branch_by_merge(
# Permission: author branch push / PR mutation surface.
push_block = _profile_operation_gate("gitea.branch.push")
if push_block:
return {
"success": False,
"performed": False,
"mutation_allowed": False,
"reasons": push_block,
"permission_report": _permission_block_report("gitea.branch.push"),
"role_kind": role,
}
return _build_operation_gate_refusal(
"gitea.branch.push",
push_block,
mutation_allowed=False,
role_kind=role,
)
if role != "author":
pre = pr_sync_status.assess_update_pr_branch_preflight(
@@ -22524,6 +22751,38 @@ def gitea_request_mcp_restart(
# and a durable incident is raised. Break-glass is the only bypass and its
# authorization is read from the environment, never self-asserted.
payload["apply_supported"] = False
# #665: correlation id threads impact preview → apply gate → incidents.
correlation_id = restart_audit.new_correlation_id()
payload["correlation_id"] = correlation_id
auth_user = None
try:
# remote-only: host override is for operator diagnostics, not required here
auth_user = (gitea_whoami(remote=remote) or {}).get("username")
except Exception: # noqa: BLE001 — identity is best-effort for audit
auth_user = None
preview_audit = restart_audit.record_restart_lifecycle(
event_type=restart_audit.EVENT_IMPACT_PREVIEW,
outcome=str(report.verdict or "unknown"),
correlation_id=correlation_id,
remote=remote,
org=o,
repo=r,
requesting_session_id=sid,
restart_class=restart_class,
profile_name=profile_name,
authenticated_username=auth_user,
reasons=list(report.reasons or []),
details={
"dry_run": True,
"allow_restart": bool(report.allow_restart),
"inventory_complete": inventory_complete,
},
privileged=False,
)
payload["restart_audit"] = {
"correlation_id": correlation_id,
"impact_preview_written": preview_audit["audit_written"],
}
if not dry_run:
proof_obj: dict | None = None
proof_parse_error: str | None = None
@@ -22583,6 +22842,86 @@ def gitea_request_mcp_restart(
)
if not gate.allow and gate.incident is not None:
payload["incident"] = gate.incident
# #665: audit apply-gate + materialize durable incidents for failed
# drain and break-glass. Privileged apply denies if audit is enabled
# and the sink write fails.
incident_desc = restart_audit.incident_from_apply_gate(
gate_payload={
**gate_payload,
"incident": gate.incident,
"allow": gate.allow,
},
break_glass=break_glass,
correlation_id=correlation_id,
requesting_session_id=sid,
restart_class=restart_class,
remote=remote,
org=o,
repo=r,
)
if incident_desc is not None:
payload["incident"] = incident_desc
def _create_restart_incident_issue(
*, title, body, labels, org=None, repo=None, **_kw
):
return gitea_create_issue(
title=title,
body=body,
labels=labels,
remote=remote,
host=h,
org=org or o,
repo=repo or r,
)
apply_audit = restart_audit.record_restart_lifecycle(
event_type=(
restart_audit.EVENT_BREAK_GLASS
if break_glass
else restart_audit.EVENT_APPLY_GATE
),
outcome=(
"break_glass"
if break_glass
else ("allow" if payload["apply_authorized"] else "deny")
),
correlation_id=correlation_id,
remote=remote,
org=o,
repo=r,
requesting_session_id=sid,
restart_class=restart_class,
profile_name=profile_name,
authenticated_username=auth_user,
reasons=list(gate_payload.get("reasons") or []),
details={
"apply_authorized": payload["apply_authorized"],
"drain_gate_allow": gate_payload.get("drain_gate_allow"),
"restart_class_authorized": restart_class_authorized,
"break_glass": break_glass,
"proof_id": gate_payload.get("proof_id"),
},
privileged=True,
create_incident=incident_desc,
create_issue_fn=_create_restart_incident_issue
if incident_desc is not None
else None,
dry_run_incident=False,
)
payload["restart_audit"] = {
"correlation_id": correlation_id,
"impact_preview_written": preview_audit["audit_written"],
"apply_gate_written": apply_audit["audit_written"],
"incident_result": apply_audit.get("incident_result"),
}
if apply_audit["deny_reasons"]:
payload["apply_authorized"] = False
payload["reasons"] = list(payload.get("reasons") or []) + list(
apply_audit["deny_reasons"]
)
payload["success"] = True
return payload
+426
View File
@@ -0,0 +1,426 @@
"""MCP restart lifecycle audit events and incident materialization (#665).
Restarts and recovery attempts must leave a forensic trail: impact previews,
drain enter/exit, drain-proof results, apply gate verdicts, break-glass, and
post-restart reconcile outcomes. Failed drains and break-glass must also raise
durable Gitea incident issues so they cannot be silently repeated.
This module is the pure + sink layer for that trail:
* **Schema** — ``mcp.restart.*`` event names and a redacted payload builder.
* **Emission** — append-only via :mod:`gitea_audit` (off when ``GITEA_AUDIT_LOG``
is unset; privileged apply can still *require* a successful write).
* **Incidents** — descriptors for failed drain / break-glass / unguarded restart,
plus an optional materializer that creates a Gitea issue through an injected
``create_issue_fn`` (keeps this module free of network I/O in tests).
Design rules:
* **No secrets.** All free text is redacted before write or issue body assembly.
* **Never raises from emission.** ``emit_restart_event`` returns False on sink
failure so callers can decide fail-closed policy for privileged restarts.
* **Does not restart.** Audit never executes a process restart.
* **Drain proof stays #661.** This module records what the gate decided; it
does not re-verify proofs.
"""
from __future__ import annotations
import uuid
from datetime import datetime, timezone
from typing import Any, Callable, Mapping, Sequence
import gitea_audit
# ── Event vocabulary (stable identifiers for operators + tests) ───────────────
EVENT_IMPACT_PREVIEW = "mcp.restart.impact_preview"
EVENT_DRAIN_ENTER = "mcp.restart.drain_enter"
EVENT_DRAIN_EXIT = "mcp.restart.drain_exit"
EVENT_DRAIN_PROOF = "mcp.restart.drain_proof"
EVENT_APPLY_GATE = "mcp.restart.apply_gate"
EVENT_BREAK_GLASS = "mcp.restart.break_glass"
EVENT_POST_RESTART_RECONCILE = "mcp.restart.post_restart_reconcile"
EVENT_NARROWER_RECOVERY = "mcp.restart.narrower_recovery"
EVENT_UNGUARDED_DETECTED = "mcp.restart.unguarded_detected"
RESTART_EVENT_TYPES: frozenset[str] = frozenset(
{
EVENT_IMPACT_PREVIEW,
EVENT_DRAIN_ENTER,
EVENT_DRAIN_EXIT,
EVENT_DRAIN_PROOF,
EVENT_APPLY_GATE,
EVENT_BREAK_GLASS,
EVENT_POST_RESTART_RECONCILE,
EVENT_NARROWER_RECOVERY,
EVENT_UNGUARDED_DETECTED,
}
)
# Incident kinds (durable Gitea issues).
INCIDENT_FAILED_DRAIN = "restart_failed_drain"
INCIDENT_BREAK_GLASS = "restart_break_glass"
INCIDENT_RECONCILE_UNRESOLVED = "restart_reconcile_unresolved"
INCIDENT_UNGUARDED = "restart_unguarded_detected"
DEFAULT_INCIDENT_LABELS: tuple[str, ...] = (
"mcp-health",
"safety",
"observability",
"status:ready",
"type:bug",
"workflow-hardening",
)
CreateIssueFn = Callable[..., dict[str, Any]]
def _utc_now_iso() -> str:
return datetime.now(timezone.utc).isoformat()
def new_correlation_id() -> str:
"""Mint a short correlation id shared across a restart lifecycle."""
return f"rst-{uuid.uuid4().hex[:16]}"
def build_restart_event(
*,
event_type: str,
outcome: str,
correlation_id: str | None = None,
remote: str | None = None,
org: str | None = None,
repo: str | None = None,
requesting_session_id: str | None = None,
restart_class: str | None = None,
profile_name: str | None = None,
authenticated_username: str | None = None,
reasons: Sequence[str] | None = None,
details: Mapping[str, Any] | None = None,
now: str | None = None,
) -> dict[str, Any]:
"""Build a redacted ``mcp.restart.*`` audit event.
Raises ``ValueError`` on unknown event types so a typo cannot silently land
under a free-form action name.
"""
name = str(event_type or "").strip()
if name not in RESTART_EVENT_TYPES:
raise ValueError(
f"unknown restart audit event_type {name!r}; expected one of "
f"{sorted(RESTART_EVENT_TYPES)}"
)
redacted_reasons = [
gitea_audit.redact(str(r)) for r in (reasons or []) if str(r).strip()
]
redacted_details = gitea_audit.redact(dict(details or {}))
if not isinstance(redacted_details, dict):
redacted_details = {"value": redacted_details}
event = gitea_audit.build_event(
action=name,
result=str(outcome or "unknown"),
remote=remote,
repository=f"{org}/{repo}" if org and repo else None,
profile_name=profile_name,
authenticated_username=authenticated_username,
reason="; ".join(redacted_reasons) if redacted_reasons else None,
request_metadata={
"event_family": "mcp.restart",
"correlation_id": correlation_id or new_correlation_id(),
"restart_class": restart_class,
"requesting_session_id": requesting_session_id,
"org": org,
"repo": repo,
"details": redacted_details,
"reasons": redacted_reasons,
},
now=now or _utc_now_iso(),
operation=name,
)
event["action_type"] = "restart_lifecycle"
event["event_type"] = name
event["correlation_id"] = (event.get("request_metadata") or {}).get(
"correlation_id"
)
return event
def emit_restart_event(event: Mapping[str, Any], *, path: str | None = None) -> bool:
"""Append *event* to the audit sink. Never raises. Returns write success."""
try:
return bool(gitea_audit.write_event(dict(event), path=path))
except Exception:
return False
def require_audit_or_deny(
*,
privileged: bool,
written: bool,
audit_enabled: bool | None = None,
) -> list[str]:
"""Return deny reasons when a privileged restart path fails to audit.
When audit is not configured (``GITEA_AUDIT_LOG`` unset), privileged apply
still proceeds under the rollout policy "enable audit before enforcing
deny-on-audit-fail" — but *if* audit is enabled and the write fails,
privileged apply is denied (fail closed).
"""
enabled = (
gitea_audit.audit_enabled() if audit_enabled is None else bool(audit_enabled)
)
if not privileged:
return []
if not enabled:
return []
if written:
return []
return [
"privileged restart path requires a successful audit write; "
"audit sink failed (fail closed, #665)"
]
# ── Incident descriptors ──────────────────────────────────────────────────────
def build_incident_descriptor(
*,
kind: str,
reasons: Sequence[str],
correlation_id: str | None = None,
requesting_session_id: str | None = None,
restart_class: str | None = None,
remote: str | None = None,
org: str | None = None,
repo: str | None = None,
proof_id: str | None = None,
at: str | None = None,
) -> dict[str, Any]:
"""Build a durable incident descriptor (no network)."""
titles = {
INCIDENT_FAILED_DRAIN: "Restart denied: drain proof failed the hard gate",
INCIDENT_BREAK_GLASS: "Break-glass MCP restart authorized",
INCIDENT_RECONCILE_UNRESOLVED: "Post-restart reconcile left unresolved work",
INCIDENT_UNGUARDED: "Unguarded MCP restart attempt detected",
}
title = titles.get(kind, f"MCP restart incident ({kind})")
redacted_reasons = [
gitea_audit.redact(str(r)) for r in reasons if str(r).strip()
]
return {
"kind": kind,
"title": title,
"labels": list(DEFAULT_INCIDENT_LABELS),
"reasons": redacted_reasons,
"correlation_id": correlation_id,
"requesting_session_id": requesting_session_id,
"restart_class": restart_class,
"remote": remote,
"org": org,
"repo": repo,
"proof_id": proof_id,
"at": at or _utc_now_iso(),
"source": "restart_audit#665",
}
def incident_body(descriptor: Mapping[str, Any]) -> str:
"""Render a redacted markdown body for a Gitea incident issue."""
reasons = descriptor.get("reasons") or []
reason_lines = "\n".join(f"- {gitea_audit.redact(str(r))}" for r in reasons) or (
"- (no reasons recorded)"
)
return "\n".join(
[
"<!-- mcp-restart-incident:v1 -->",
f"## MCP restart incident (`{descriptor.get('kind')}`)",
"",
f"**Correlation:** `{descriptor.get('correlation_id') or 'none'}`",
f"**Session:** `{descriptor.get('requesting_session_id') or 'none'}`",
f"**Class:** `{descriptor.get('restart_class') or 'none'}`",
f"**Scope:** `{descriptor.get('remote')}/{descriptor.get('org')}/"
f"{descriptor.get('repo')}`",
f"**At:** `{descriptor.get('at')}`",
f"**Proof id:** `{descriptor.get('proof_id') or 'none'}`",
"",
"### Reasons",
reason_lines,
"",
"### Operator next steps",
"- Treat this as durable follow-up work under the restart-governance umbrella (#655).",
"- Do not invent a second restart path; use sanctioned coordinator tools only.",
"- Raw secrets must never appear in this issue (already redacted).",
"",
f"_Source: {descriptor.get('source')}_",
]
)
def materialize_incident(
descriptor: Mapping[str, Any],
*,
create_issue_fn: CreateIssueFn | None,
dry_run: bool = False,
) -> dict[str, Any]:
"""Create a Gitea issue from *descriptor* when *create_issue_fn* is provided.
Returns a result dict with ``created`` / ``issue_number`` / ``dry_run`` /
``reasons``. Never raises.
"""
base: dict[str, Any] = {
"created": False,
"dry_run": bool(dry_run),
"issue_number": None,
"kind": descriptor.get("kind"),
"reasons": [],
"descriptor": dict(descriptor),
}
if dry_run:
base["reasons"] = ["dry-run only; no Gitea issue created"]
return base
if create_issue_fn is None:
base["reasons"] = [
"create_issue_fn not provided; incident descriptor retained only"
]
return base
try:
result = create_issue_fn(
title=str(descriptor.get("title") or "MCP restart incident"),
body=incident_body(descriptor),
labels=list(descriptor.get("labels") or DEFAULT_INCIDENT_LABELS),
org=descriptor.get("org"),
repo=descriptor.get("repo"),
)
number = None
if isinstance(result, dict):
number = result.get("number") or result.get("issue_number")
if number is not None:
base["created"] = True
base["issue_number"] = int(number)
base["reasons"] = [f"created incident issue #{int(number)}"]
else:
base["reasons"] = ["create_issue_fn returned no issue number"]
except Exception as exc: # noqa: BLE001 — never break restart path here
base["reasons"] = [
f"incident issue creation failed: {gitea_audit.redact(str(exc))}"
]
return base
def record_restart_lifecycle(
*,
event_type: str,
outcome: str,
correlation_id: str,
remote: str | None = None,
org: str | None = None,
repo: str | None = None,
requesting_session_id: str | None = None,
restart_class: str | None = None,
profile_name: str | None = None,
authenticated_username: str | None = None,
reasons: Sequence[str] | None = None,
details: Mapping[str, Any] | None = None,
privileged: bool = False,
create_incident: Mapping[str, Any] | None = None,
create_issue_fn: CreateIssueFn | None = None,
dry_run_incident: bool = False,
audit_path: str | None = None,
) -> dict[str, Any]:
"""Emit one restart audit event and optionally materialize an incident.
Returns ``{event, audit_written, deny_reasons, incident_result}``.
"""
event = build_restart_event(
event_type=event_type,
outcome=outcome,
correlation_id=correlation_id,
remote=remote,
org=org,
repo=repo,
requesting_session_id=requesting_session_id,
restart_class=restart_class,
profile_name=profile_name,
authenticated_username=authenticated_username,
reasons=reasons,
details=details,
)
written = emit_restart_event(event, path=audit_path)
deny = require_audit_or_deny(privileged=privileged, written=written)
incident_result = None
if create_incident is not None:
incident_result = materialize_incident(
create_incident,
create_issue_fn=create_issue_fn,
dry_run=dry_run_incident,
)
return {
"event": event,
"audit_written": written,
"deny_reasons": deny,
"incident_result": incident_result,
"correlation_id": correlation_id,
}
def incident_from_apply_gate(
*,
gate_payload: Mapping[str, Any],
break_glass: bool,
correlation_id: str,
requesting_session_id: str | None,
restart_class: str | None,
remote: str | None,
org: str | None,
repo: str | None,
) -> dict[str, Any] | None:
"""Choose an incident descriptor from an apply-gate payload, if required."""
reasons = list(gate_payload.get("reasons") or [])
proof_id = gate_payload.get("proof_id")
if break_glass:
return build_incident_descriptor(
kind=INCIDENT_BREAK_GLASS,
reasons=reasons
or ["break-glass restart path used; durable incident required (#665)"],
correlation_id=correlation_id,
requesting_session_id=requesting_session_id,
restart_class=restart_class,
remote=remote,
org=org,
repo=repo,
proof_id=proof_id if isinstance(proof_id, str) else None,
)
# Failed drain / deny path.
incident = gate_payload.get("incident")
if isinstance(incident, Mapping) and incident:
# Normalize gate-provided descriptor into our schema.
return build_incident_descriptor(
kind=INCIDENT_FAILED_DRAIN,
reasons=list(incident.get("reasons") or reasons),
correlation_id=correlation_id,
requesting_session_id=requesting_session_id
or incident.get("requesting_session_id"),
restart_class=restart_class,
remote=remote,
org=org,
repo=repo,
proof_id=incident.get("proof_id") or proof_id,
at=incident.get("at"),
)
if not gate_payload.get("allow") and not gate_payload.get("drain_gate_allow", True):
return build_incident_descriptor(
kind=INCIDENT_FAILED_DRAIN,
reasons=reasons or ["restart apply denied"],
correlation_id=correlation_id,
requesting_session_id=requesting_session_id,
restart_class=restart_class,
remote=remote,
org=org,
repo=repo,
proof_id=proof_id if isinstance(proof_id, str) else None,
)
return None
@@ -0,0 +1,453 @@
"""#897: stale-runtime / runtime-mode refusals must not look like permission denials.
Acceptance criteria (issue #897):
* Stale-runtime and runtime-mode refusals are typed distinctly from
profile-permission refusals (distinct ``blocker_kind``).
* A refusal caused by staleness or runtime mode never emits a
``permission_report`` and never names a permission the active profile holds.
* ``_permission_block_report`` verifies the active profile actually lacks the
operation before reporting it missing.
* A stale-runtime refusal reports reconnect-only recovery and never recommends
``gitea_activate_profile`` or an MCP session switch.
* The blocker payload states the observed heads (parity fields).
* Matrix across author / reviewer / merger / reconciler profiles.
* Regression: ``gitea_create_issue`` on a stale daemon under ``prgs-author``
never returns ``missing_permission: gitea.issue.create``.
"""
from __future__ import annotations
import os
import sys
import unittest
from unittest.mock import patch
sys.path.insert(0, str(__import__("pathlib").Path(__file__).resolve().parent.parent))
import gitea_config # noqa: E402
import gitea_mcp_server as mcp_server # noqa: E402
SHA_START = "7af40fb5ff7debd5e9165fe97d9c7c279358e175"
SHA_LIVE = "2f4dec832327513118f2fe92b74da25d124a01cb"
ROLE_MATRIX = (
(
"prgs-author",
"author",
"gitea.issue.create",
[
"gitea.read",
"gitea.issue.create",
"gitea.issue.comment",
"gitea.issue.close",
"gitea.branch.create",
"gitea.branch.push",
"gitea.pr.create",
"gitea.pr.comment",
"gitea.repo.commit",
],
["gitea.pr.approve", "gitea.pr.merge", "gitea.pr.request_changes"],
"gitea.pr.merge", # forbidden op for pure-permission case
),
(
"prgs-reviewer",
"reviewer",
"gitea.pr.review",
[
"gitea.read",
"gitea.pr.review",
"gitea.pr.approve",
"gitea.pr.request_changes",
"gitea.pr.comment",
"gitea.issue.comment",
],
["gitea.branch.push", "gitea.pr.create"],
"gitea.branch.push",
),
(
"prgs-merger",
"merger",
"gitea.pr.merge",
[
"gitea.read",
"gitea.pr.merge",
"gitea.pr.comment",
"gitea.issue.comment",
],
["gitea.pr.approve", "gitea.branch.push", "gitea.pr.create"],
"gitea.branch.push",
),
(
"prgs-reconciler",
"reconciler",
"gitea.branch.delete",
[
"gitea.read",
"gitea.branch.delete",
"gitea.pr.comment",
"gitea.issue.comment",
"gitea.pr.close",
"gitea.issue.close",
],
["gitea.pr.approve", "gitea.pr.merge"],
"gitea.pr.merge",
),
)
def _profile(name: str, role: str, allowed: list[str], forbidden: list[str]) -> dict:
return {
"profile_name": name,
"role": role,
"role_kind": role,
"allowed_operations": list(allowed),
"forbidden_operations": list(forbidden),
"identity": "test-user",
}
def _config(profiles: dict) -> dict:
return {
"version": 2,
"profiles": {
name: {
"role": p["role"],
"allowed_operations": p["allowed_operations"],
"forbidden_operations": p["forbidden_operations"],
}
for name, p in profiles.items()
},
"rules": {"allow_runtime_switching": True},
}
class Issue897Helpers(unittest.TestCase):
def test_classify_stale_reason_strings(self):
stale = (
f"live remote master is {SHA_LIVE[:12]} but the MCP server started "
f"at {SHA_START[:12]}; the daemon is stale relative to live master "
"-- restart/reconnect before mutating"
)
classified = mcp_server._classify_operation_gate_reasons([stale])
self.assertEqual(classified["stale_runtime"], [stale])
self.assertEqual(classified["permission"], [])
self.assertEqual(classified["runtime_mode"], [])
def test_classify_permission_reason(self):
reason = "profile is not allowed to gitea.pr.merge"
classified = mcp_server._classify_operation_gate_reasons([reason])
self.assertEqual(classified["permission"], [reason])
self.assertEqual(classified["stale_runtime"], [])
def test_classify_runtime_mode_reason(self):
reason = (
"runtime mode is 'dev-test' and the mutation targets the "
"production repository; dev/test runtimes must not mutate real "
"issues or PRs (ADR: stable control runtime vs dev runtime)"
)
classified = mcp_server._classify_operation_gate_reasons([reason])
self.assertEqual(classified["runtime_mode"], [reason])
self.assertEqual(classified["stale_runtime"], [])
class Issue897PermissionBlockReport(unittest.TestCase):
def test_holds_op_is_diagnostic_defect_not_missing_permission(self):
profile = _profile(
"prgs-author",
"author",
["gitea.read", "gitea.issue.create", "gitea.issue.comment"],
[],
)
with patch.object(mcp_server, "get_profile", return_value=profile), patch.object(
mcp_server.gitea_config, "load_config", return_value=_config({"prgs-author": profile})
), patch.object(
mcp_server.gitea_config, "is_runtime_switching_enabled", return_value=True
):
report = mcp_server._permission_block_report("gitea.issue.create")
self.assertTrue(report.get("diagnostic_defect"), report)
self.assertIsNone(report.get("missing_permission"), report)
action = (report.get("exact_safe_next_action") or "").lower()
# Must not *recommend* profile switching; mentioning the forbidden
# action in a "do not call" instruction is fine.
self.assertNotIn("call gitea_activate_profile with", action)
self.assertNotIn("switch to the author mcp session", action)
self.assertNotIn("switch to the reviewer mcp session", action)
self.assertIn("diagnostic defect", action)
def test_true_missing_permission_still_reports(self):
profile = _profile(
"prgs-author",
"author",
["gitea.read", "gitea.issue.create"],
["gitea.pr.merge"],
)
reviewer = _profile(
"prgs-reviewer",
"reviewer",
["gitea.read", "gitea.pr.merge", "gitea.pr.approve"],
[],
)
with patch.object(mcp_server, "get_profile", return_value=profile), patch.object(
mcp_server.gitea_config,
"load_config",
return_value=_config({"prgs-author": profile, "prgs-reviewer": reviewer}),
), patch.object(
mcp_server.gitea_config, "is_runtime_switching_enabled", return_value=True
):
report = mcp_server._permission_block_report("gitea.pr.merge")
self.assertFalse(report.get("diagnostic_defect"), report)
self.assertEqual(report.get("missing_permission"), "gitea.pr.merge")
self.assertIn("prgs-reviewer", report.get("matching_configured_profiles") or [])
class Issue897GateRefusalMatrix(unittest.TestCase):
def _stale_parity(self) -> dict:
return {
"in_parity": True,
"stale": False,
"restart_required": True,
"determinable": True,
"startup_head": SHA_START,
"current_head": SHA_START,
"daemon_start_head": SHA_START,
"local_head": SHA_START,
"live_remote_head": SHA_LIVE,
"live_known": True,
"live_stale": True,
"mutation_safe": False,
"reasons": [
f"live remote master is {SHA_LIVE[:12]} but the MCP server "
f"started at {SHA_START[:12]}; the daemon is stale relative "
"to live master -- restart/reconnect before mutating"
],
}
def test_stale_plus_permitted_op_all_roles(self):
for name, role, permitted_op, allowed, forbidden, _forbidden_op in ROLE_MATRIX:
with self.subTest(profile=name, op=permitted_op):
profile = _profile(name, role, allowed, forbidden)
parity = self._stale_parity()
with patch.object(mcp_server, "get_profile", return_value=profile), patch.object(
mcp_server, "_current_master_parity", return_value=parity
), patch.object(
mcp_server, "_master_parity_block", return_value=list(parity["reasons"])
), patch.object(
mcp_server, "_runtime_mode_block", return_value=[]
), patch.object(
mcp_server, "_ensure_matching_profile", return_value=None
), patch.object(
mcp_server.session_ctx,
"mutation_context_audit_fields",
return_value={"session_profile": name},
):
blocked = mcp_server._profile_permission_block(permitted_op)
self.assertIsNotNone(blocked, name)
assert blocked is not None
self.assertEqual(
blocked.get("blocker_kind"),
"runtime_reconnect_required",
blocked,
)
self.assertNotIn("permission_report", blocked, blocked)
self.assertTrue(blocked.get("restart_required"), blocked)
self.assertEqual(blocked.get("startup_head"), SHA_START, blocked)
self.assertEqual(blocked.get("live_remote_head"), SHA_LIVE, blocked)
action = (blocked.get("exact_safe_next_action") or "").lower()
self.assertIn("reconnect", action)
self.assertNotIn("call gitea_activate_profile with", action)
self.assertNotIn("switch to the author mcp session", action)
self.assertNotIn("switch to the reviewer mcp session", action)
def test_fresh_plus_forbidden_op_all_roles(self):
for name, role, _permitted, allowed, forbidden, forbidden_op in ROLE_MATRIX:
with self.subTest(profile=name, op=forbidden_op):
profile = _profile(name, role, allowed, forbidden)
with patch.object(mcp_server, "get_profile", return_value=profile), patch.object(
mcp_server, "_master_parity_block", return_value=[]
), patch.object(
mcp_server, "_runtime_mode_block", return_value=[]
), patch.object(
mcp_server, "_ensure_matching_profile", return_value=None
), patch.object(
mcp_server.session_ctx,
"mutation_context_audit_fields",
return_value={"session_profile": name},
), patch.object(
mcp_server.gitea_config,
"load_config",
return_value=_config({name: profile}),
), patch.object(
mcp_server.gitea_config, "is_runtime_switching_enabled", return_value=False
):
blocked = mcp_server._profile_permission_block(forbidden_op)
self.assertIsNotNone(blocked, name)
assert blocked is not None
self.assertEqual(blocked.get("blocker_kind"), "permission_denied", blocked)
self.assertIn("permission_report", blocked, blocked)
report = blocked["permission_report"]
self.assertEqual(report.get("missing_permission"), forbidden_op, report)
self.assertFalse(report.get("diagnostic_defect"), report)
# No runtime reconnect fields for pure permission denial
self.assertNotEqual(
blocked.get("blocker_kind"), "runtime_reconnect_required"
)
def test_stale_plus_forbidden_op_both_causes_separated(self):
for name, role, _permitted, allowed, forbidden, forbidden_op in ROLE_MATRIX:
with self.subTest(profile=name, op=forbidden_op):
profile = _profile(name, role, allowed, forbidden)
parity = self._stale_parity()
stale_reason = parity["reasons"][0]
with patch.object(mcp_server, "get_profile", return_value=profile), patch.object(
mcp_server, "_current_master_parity", return_value=parity
), patch.object(
mcp_server, "_master_parity_block", return_value=[stale_reason]
), patch.object(
mcp_server, "_runtime_mode_block", return_value=[]
), patch.object(
mcp_server, "_ensure_matching_profile", return_value=None
), patch.object(
mcp_server.session_ctx,
"mutation_context_audit_fields",
return_value={"session_profile": name},
):
# Gate collects both classes; force permission reason too.
with patch.object(
mcp_server,
"_profile_operation_gate",
return_value=[
stale_reason,
f"profile is not allowed to {forbidden_op}",
],
):
blocked = mcp_server._profile_permission_block(forbidden_op)
self.assertIsNotNone(blocked)
assert blocked is not None
self.assertEqual(
blocked.get("blocker_kind"), "runtime_reconnect_required", blocked
)
self.assertNotIn("permission_report", blocked, blocked)
self.assertIn("permission_block_reasons", blocked, blocked)
self.assertIn("stale_runtime_reasons", blocked, blocked)
classes = blocked.get("gate_reason_classes") or {}
self.assertTrue(classes.get("stale_runtime"), classes)
self.assertTrue(classes.get("permission"), classes)
def test_runtime_mode_block_no_permission_report(self):
profile = _profile(
"prgs-author",
"author",
["gitea.read", "gitea.issue.create"],
[],
)
runtime_reason = (
"runtime mode is 'dev-test' and the mutation targets the "
"production repository; dev/test runtimes must not mutate real "
"issues or PRs (ADR: stable control runtime vs dev runtime)"
)
with patch.object(mcp_server, "get_profile", return_value=profile), patch.object(
mcp_server, "_master_parity_block", return_value=[]
), patch.object(
mcp_server, "_runtime_mode_block", return_value=[runtime_reason]
), patch.object(
mcp_server, "_ensure_matching_profile", return_value=None
), patch.object(
mcp_server.session_ctx,
"mutation_context_audit_fields",
return_value={"session_profile": "prgs-author"},
):
blocked = mcp_server._profile_permission_block("gitea.issue.create")
self.assertIsNotNone(blocked)
assert blocked is not None
self.assertEqual(blocked.get("blocker_kind"), "runtime_mode_blocked", blocked)
self.assertNotIn("permission_report", blocked, blocked)
action = (blocked.get("exact_safe_next_action") or "").lower()
self.assertNotIn("call gitea_activate_profile with", action)
self.assertIn("stable control runtime", action)
class Issue897CreateIssueRegression(unittest.TestCase):
def test_create_issue_stale_daemon_never_missing_issue_create(self):
"""Regression AC: stale prgs-author create_issue must not claim missing create."""
profile = _profile(
"prgs-author",
"author",
[
"gitea.read",
"gitea.issue.create",
"gitea.issue.comment",
"gitea.branch.create",
"gitea.branch.push",
"gitea.pr.create",
"gitea.pr.comment",
"gitea.repo.commit",
],
[],
)
stale_reason = (
f"live remote master is {SHA_LIVE[:12]} but the MCP server started "
f"at {SHA_START[:12]}; the daemon is stale relative to live master "
"-- restart/reconnect before mutating"
)
parity = {
"in_parity": True,
"stale": False,
"restart_required": True,
"determinable": True,
"startup_head": SHA_START,
"current_head": SHA_START,
"daemon_start_head": SHA_START,
"local_head": SHA_START,
"live_remote_head": SHA_LIVE,
"live_known": True,
"live_stale": True,
"mutation_safe": False,
"reasons": [stale_reason],
}
with patch.object(mcp_server, "get_profile", return_value=profile), patch.object(
mcp_server, "_current_master_parity", return_value=parity
), patch.object(
mcp_server, "_master_parity_block", return_value=[stale_reason]
), patch.object(
mcp_server, "_runtime_mode_block", return_value=[]
), patch.object(
mcp_server, "_ensure_matching_profile", return_value=None
), patch.object(
mcp_server.session_ctx,
"mutation_context_audit_fields",
return_value={"session_profile": "prgs-author"},
), patch.object(
mcp_server, "_mutation_config_authority_block", return_value=None
), patch.object(
mcp_server, "_session_context_mutation_block", return_value=None
):
blocked = mcp_server._profile_permission_block(
"gitea.issue.create", remote="prgs"
)
self.assertIsNotNone(blocked)
assert blocked is not None
self.assertEqual(blocked.get("blocker_kind"), "runtime_reconnect_required")
self.assertNotIn("permission_report", blocked)
# Even if a caller still built a raw report, holds-check must not claim missing.
with patch.object(mcp_server, "get_profile", return_value=profile):
raw = mcp_server._permission_block_report("gitea.issue.create")
self.assertIsNone(raw.get("missing_permission"), raw)
self.assertNotEqual(raw.get("missing_permission"), "gitea.issue.create")
def test_permission_report_for_gate_reasons_skips_stale(self):
stale = (
f"live remote master is {SHA_LIVE[:12]} but the MCP server started "
f"at {SHA_START[:12]}; the daemon is stale relative to live master "
"-- restart/reconnect before mutating"
)
self.assertIsNone(
mcp_server._permission_report_for_gate_reasons(
"gitea.issue.comment", [stale]
)
)
if __name__ == "__main__":
unittest.main()
+342
View File
@@ -0,0 +1,342 @@
"""Tests for MCP restart lifecycle audit events and incidents (#665)."""
from __future__ import annotations
import json
import os
import tempfile
import unittest
from unittest.mock import patch
import gitea_audit
import restart_audit as ra
class TestEventSchema(unittest.TestCase):
def test_all_lifecycle_event_types_are_named(self):
expected = {
"mcp.restart.impact_preview",
"mcp.restart.drain_enter",
"mcp.restart.drain_exit",
"mcp.restart.drain_proof",
"mcp.restart.apply_gate",
"mcp.restart.break_glass",
"mcp.restart.post_restart_reconcile",
"mcp.restart.narrower_recovery",
"mcp.restart.unguarded_detected",
}
self.assertEqual(set(ra.RESTART_EVENT_TYPES), expected)
def test_build_restart_event_core_fields(self):
event = ra.build_restart_event(
event_type=ra.EVENT_IMPACT_PREVIEW,
outcome="safe",
correlation_id="rst-abc123",
remote="prgs",
org="Scaled-Tech-Consulting",
repo="Gitea-Tools",
requesting_session_id="sess-1",
restart_class="full_mcp_restart",
profile_name="prgs-author",
authenticated_username="bot",
reasons=["inventory complete"],
details={"allow_restart": True},
now="2026-07-25T12:00:00+00:00",
)
self.assertEqual(event["event_type"], ra.EVENT_IMPACT_PREVIEW)
self.assertEqual(event["action"], ra.EVENT_IMPACT_PREVIEW)
self.assertEqual(event["action_type"], "restart_lifecycle")
self.assertEqual(event["result"], "safe")
self.assertEqual(event["correlation_id"], "rst-abc123")
self.assertEqual(event["profile_name"], "prgs-author")
self.assertEqual(event["authenticated_username"], "bot")
meta = event["request_metadata"]
self.assertEqual(meta["event_family"], "mcp.restart")
self.assertEqual(meta["correlation_id"], "rst-abc123")
self.assertEqual(meta["restart_class"], "full_mcp_restart")
self.assertEqual(meta["details"]["allow_restart"], True)
def test_unknown_event_type_raises(self):
with self.assertRaises(ValueError) as ctx:
ra.build_restart_event(
event_type="mcp.restart.not_a_real_event",
outcome="x",
correlation_id="rst-1",
)
self.assertIn("unknown restart audit event_type", str(ctx.exception))
def test_reasons_and_details_are_redacted(self):
event = ra.build_restart_event(
event_type=ra.EVENT_APPLY_GATE,
outcome="deny",
correlation_id="rst-sec",
reasons=["token secret-xyz rejected", "ok"],
details={"token": "leak-token", "status": "denied"},
)
self.assertNotIn("secret-xyz", event.get("reason") or "")
meta = event["request_metadata"]
self.assertEqual(meta["details"]["token"], gitea_audit.REDACTED)
self.assertEqual(meta["details"]["status"], "denied")
for reason in meta["reasons"]:
self.assertNotIn("secret-xyz", reason)
def test_new_correlation_id_shape(self):
cid = ra.new_correlation_id()
self.assertTrue(cid.startswith("rst-"))
self.assertEqual(len(cid), len("rst-") + 16)
class TestEmitAndRequire(unittest.TestCase):
def test_emit_appends_json_line(self):
with tempfile.TemporaryDirectory() as d:
path = os.path.join(d, "audit.log")
event = ra.build_restart_event(
event_type=ra.EVENT_DRAIN_PROOF,
outcome="pass",
correlation_id="rst-write",
)
self.assertTrue(ra.emit_restart_event(event, path=path))
with open(path, encoding="utf-8") as fh:
lines = fh.read().splitlines()
self.assertEqual(len(lines), 1)
loaded = json.loads(lines[0])
self.assertEqual(loaded["event_type"], ra.EVENT_DRAIN_PROOF)
self.assertEqual(loaded["correlation_id"], "rst-write")
def test_emit_never_raises(self):
self.assertFalse(
ra.emit_restart_event({"action": "x"}, path="/no/such/dir/audit.log")
)
def test_require_audit_denies_privileged_when_write_fails_and_enabled(self):
deny = ra.require_audit_or_deny(
privileged=True, written=False, audit_enabled=True
)
self.assertEqual(len(deny), 1)
self.assertIn("fail closed", deny[0])
def test_require_audit_allows_when_audit_disabled(self):
# Rollout policy: enable audit before enforcing deny-on-audit-fail.
deny = ra.require_audit_or_deny(
privileged=True, written=False, audit_enabled=False
)
self.assertEqual(deny, [])
def test_require_audit_noop_for_non_privileged(self):
deny = ra.require_audit_or_deny(
privileged=False, written=False, audit_enabled=True
)
self.assertEqual(deny, [])
def test_require_audit_allows_when_written(self):
deny = ra.require_audit_or_deny(
privileged=True, written=True, audit_enabled=True
)
self.assertEqual(deny, [])
class TestIncidents(unittest.TestCase):
def test_break_glass_descriptor(self):
desc = ra.build_incident_descriptor(
kind=ra.INCIDENT_BREAK_GLASS,
reasons=["break-glass authorized"],
correlation_id="rst-bg",
requesting_session_id="s1",
restart_class="full_mcp_restart",
remote="prgs",
org="O",
repo="R",
)
self.assertEqual(desc["kind"], ra.INCIDENT_BREAK_GLASS)
self.assertIn("Break-glass", desc["title"])
self.assertIn("mcp-health", desc["labels"])
self.assertEqual(desc["source"], "restart_audit#665")
def test_incident_body_redacts_and_includes_correlation(self):
desc = ra.build_incident_descriptor(
kind=ra.INCIDENT_FAILED_DRAIN,
reasons=["token secret-xyz failed proof"],
correlation_id="rst-body",
remote="prgs",
org="O",
repo="R",
proof_id="proof-1",
)
body = ra.incident_body(desc)
self.assertIn("rst-body", body)
self.assertIn("proof-1", body)
self.assertIn("mcp-restart-incident:v1", body)
self.assertNotIn("secret-xyz", body)
def test_materialize_dry_run(self):
desc = ra.build_incident_descriptor(
kind=ra.INCIDENT_FAILED_DRAIN,
reasons=["denied"],
correlation_id="rst-dr",
)
result = ra.materialize_incident(desc, create_issue_fn=lambda **k: {}, dry_run=True)
self.assertFalse(result["created"])
self.assertTrue(result["dry_run"])
self.assertIn("dry-run", result["reasons"][0])
def test_materialize_without_create_fn(self):
desc = ra.build_incident_descriptor(
kind=ra.INCIDENT_FAILED_DRAIN,
reasons=["denied"],
correlation_id="rst-nfn",
)
result = ra.materialize_incident(desc, create_issue_fn=None)
self.assertFalse(result["created"])
self.assertIn("create_issue_fn not provided", result["reasons"][0])
def test_materialize_creates_issue(self):
created = {}
def _create(*, title, body, labels, org=None, repo=None, **_kw):
created["title"] = title
created["body"] = body
created["labels"] = labels
created["org"] = org
created["repo"] = repo
return {"number": 999}
desc = ra.build_incident_descriptor(
kind=ra.INCIDENT_BREAK_GLASS,
reasons=["break-glass"],
correlation_id="rst-create",
org="O",
repo="R",
)
result = ra.materialize_incident(desc, create_issue_fn=_create)
self.assertTrue(result["created"])
self.assertEqual(result["issue_number"], 999)
self.assertIn("Break-glass", created["title"])
self.assertIn("rst-create", created["body"])
self.assertEqual(created["org"], "O")
def test_materialize_never_raises_on_create_failure(self):
def _boom(**_kw):
raise RuntimeError("token secret-xyz network")
desc = ra.build_incident_descriptor(
kind=ra.INCIDENT_FAILED_DRAIN,
reasons=["x"],
correlation_id="rst-boom",
)
result = ra.materialize_incident(desc, create_issue_fn=_boom)
self.assertFalse(result["created"])
self.assertIn("failed", result["reasons"][0])
self.assertNotIn("secret-xyz", result["reasons"][0])
class TestIncidentFromApplyGate(unittest.TestCase):
def test_break_glass_always_incident(self):
desc = ra.incident_from_apply_gate(
gate_payload={"allow": True, "reasons": [], "proof_id": None},
break_glass=True,
correlation_id="rst-bg2",
requesting_session_id="s",
restart_class="full_mcp_restart",
remote="prgs",
org="O",
repo="R",
)
self.assertIsNotNone(desc)
self.assertEqual(desc["kind"], ra.INCIDENT_BREAK_GLASS)
def test_failed_drain_from_gate_incident(self):
desc = ra.incident_from_apply_gate(
gate_payload={
"allow": False,
"drain_gate_allow": False,
"reasons": ["proof expired"],
"incident": {
"reasons": ["proof expired"],
"proof_id": "p1",
},
},
break_glass=False,
correlation_id="rst-fd",
requesting_session_id="s",
restart_class="full_mcp_restart",
remote="prgs",
org="O",
repo="R",
)
self.assertIsNotNone(desc)
self.assertEqual(desc["kind"], ra.INCIDENT_FAILED_DRAIN)
self.assertEqual(desc["proof_id"], "p1")
def test_allow_without_break_glass_no_incident(self):
desc = ra.incident_from_apply_gate(
gate_payload={
"allow": True,
"drain_gate_allow": True,
"reasons": [],
},
break_glass=False,
correlation_id="rst-ok",
requesting_session_id="s",
restart_class="full_mcp_restart",
remote="prgs",
org="O",
repo="R",
)
self.assertIsNone(desc)
class TestRecordLifecycle(unittest.TestCase):
def test_record_emits_and_materializes(self):
created = []
def _create(**kwargs):
created.append(kwargs)
return {"number": 42}
with tempfile.TemporaryDirectory() as d:
path = os.path.join(d, "audit.log")
with patch.dict(os.environ, {"GITEA_AUDIT_LOG": path}, clear=False):
incident = ra.build_incident_descriptor(
kind=ra.INCIDENT_BREAK_GLASS,
reasons=["bg"],
correlation_id="rst-lc",
org="O",
repo="R",
)
out = ra.record_restart_lifecycle(
event_type=ra.EVENT_BREAK_GLASS,
outcome="break_glass",
correlation_id="rst-lc",
remote="prgs",
org="O",
repo="R",
privileged=True,
create_incident=incident,
create_issue_fn=_create,
audit_path=path,
)
self.assertTrue(out["audit_written"])
self.assertEqual(out["deny_reasons"], [])
self.assertTrue(out["incident_result"]["created"])
self.assertEqual(out["incident_result"]["issue_number"], 42)
self.assertEqual(len(created), 1)
def test_privileged_deny_when_audit_write_fails(self):
with patch.dict(
os.environ, {"GITEA_AUDIT_LOG": "/no/such/dir/a.log"}, clear=False
):
with patch("restart_audit.emit_restart_event", return_value=False):
with patch("gitea_audit.audit_enabled", return_value=True):
out = ra.record_restart_lifecycle(
event_type=ra.EVENT_APPLY_GATE,
outcome="deny",
correlation_id="rst-deny",
privileged=True,
audit_path="/no/such/dir/a.log",
)
self.assertFalse(out["audit_written"])
self.assertEqual(len(out["deny_reasons"]), 1)
if __name__ == "__main__":
unittest.main()