Merge branch 'master' into fix/issue-790-slice-a-heartbeat-policy
This commit is contained in:
@@ -0,0 +1,223 @@
|
||||
# ADR: MCP restart governance and authorization policy
|
||||
|
||||
- **Status:** Accepted (policy effective immediately for LLM and operator sessions; enforcement tooling may lag)
|
||||
- **Date:** 2026-07-23
|
||||
- **Tracking issue:** [#656](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/656)
|
||||
- **Policy version:** `restart-governance/v1`
|
||||
- **Related:**
|
||||
- Umbrella: [#655](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/655) — governed MCP restart coordination and zero-disruption recovery
|
||||
- Vision: [#652](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/652) — MCP Control Plane Web Console product vision (§A system health and process control)
|
||||
- Roadmap: [#653](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/653) — Control Plane Web Console phased delivery (Phase 2 restart controls)
|
||||
- Contamination guard: [#630](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/630) — blocks manual process-kill recovery
|
||||
- Console restart UX: [#642](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/642) — sanctioned restart and graceful reload
|
||||
- Existing restart / reconnect paths to inventory: [#591](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/591) — auto-restart on master advance (closed); [#584](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/584) — host auto-reconnect on transport flap
|
||||
- Stable-control runtime split: `docs/architecture/mcp-stable-control-runtime-policy-adr.md` (#615)
|
||||
- Client-namespace health: `docs/mcp-namespace-health.md` (#543)
|
||||
- Reconnect-only EOF recovery: `docs/mcp-namespace-eof-recovery.md`
|
||||
|
||||
## 1. Context
|
||||
|
||||
The Gitea MCP server is the **control plane** for real issue and PR mutations
|
||||
(create, comment, lock, review, merge, reconcile). The same process serves every
|
||||
role namespace (`gitea-author`, `gitea-reviewer`, `gitea-merger`,
|
||||
`gitea-reconciler`, `gitea-controller`) and holds the in-memory capability-gate
|
||||
code loaded at startup.
|
||||
|
||||
Restarting that process is destructive to concurrent work:
|
||||
|
||||
- It resets every session's identity, preflight, and capability-lease binding.
|
||||
- It can interrupt a mutation mid-critical-section (a lock acquire, a review
|
||||
submit, a merge), leaving durable state half-written.
|
||||
- Relaunching from the wrong checkout or worktree silently changes which code
|
||||
the control plane runs, defeating master-parity gates (#420 / #615).
|
||||
|
||||
Today there is **no durable written policy** stating who may restart MCP, under
|
||||
what conditions, that restart is a last resort, and how controller approval,
|
||||
automated safety gates, and break-glass interact. Operators and LLM sessions
|
||||
therefore invent restart behavior ad hoc, which makes concurrent multi-role work
|
||||
unsafe. #630 and #642 need this policy as their backbone.
|
||||
|
||||
This ADR defines that policy. It does **not** implement coordinator code or HA
|
||||
multi-instance restart (those are later children of #655).
|
||||
|
||||
## 2. Decision
|
||||
|
||||
### 2.1 v1 decision (recorded)
|
||||
|
||||
**Restart authority in v1 is `controller approval + automated safety gates`.**
|
||||
|
||||
A restart of the stable control runtime is authorized only when **both** hold:
|
||||
|
||||
1. A **controller** role explicitly approves the restart, recording an audit
|
||||
entry (who, why, scope, affected sessions), **and**
|
||||
2. The **automated safety gates** pass: a completed drain acknowledgement (no
|
||||
affected session is mid-critical-section) or a declared break-glass incident
|
||||
(§2.5).
|
||||
|
||||
Quorum among multiple controllers is **not** required day-one. It is deferred
|
||||
unless a later investigation (tracked under #653) proves single-controller
|
||||
approval is insufficient. This ADR records the v1 decision so enforcement code
|
||||
(#630) has a fixed target; changing it requires a superseding ADR.
|
||||
|
||||
### 2.2 Restart is a last resort — the recovery ladder
|
||||
|
||||
Restart is the **last** rung. Before any restart, exhaust the narrower
|
||||
recoveries, in order:
|
||||
|
||||
1. **Reconnect** the IDE/client MCP namespace (transport EOF, `client is
|
||||
closing: EOF`, transient `#584` flap). No process change. See
|
||||
`docs/mcp-namespace-eof-recovery.md`.
|
||||
2. **Refresh / rebind** the session workspace: re-run `gitea_whoami`,
|
||||
`gitea_resolve_task_capability`, and pass an explicit validated
|
||||
`worktree_path`. Fixes stale session context without touching the process.
|
||||
3. **Scoped restart** of a single misbehaving namespace/service (where the
|
||||
deployment supports per-service restart) rather than the whole control plane.
|
||||
4. **Full restart** of the stable control runtime process — operator-owned,
|
||||
controller-approved, drained.
|
||||
5. **Host / infrastructure restart** — the broadest action; same authorization
|
||||
as a full restart plus infrastructure ownership.
|
||||
|
||||
A session **must** try rungs 1–2 and record why they were insufficient before
|
||||
requesting a restart at rung 3 or above. Skipping straight to restart is a
|
||||
policy violation.
|
||||
|
||||
### 2.3 Authorization matrix
|
||||
|
||||
| Role | Reconnect (1) | Refresh/rebind (2) | Scoped restart (3) | Full restart (4) | Host restart (5) |
|
||||
|---|---|---|---|---|---|
|
||||
| **author** | self | self | request only | **forbidden** | forbidden |
|
||||
| **reviewer** | self | self | request only | **forbidden** | forbidden |
|
||||
| **merger** | self | self | request only | **forbidden** | forbidden |
|
||||
| **reconciler** | self | self | request only | **forbidden** | forbidden |
|
||||
| **controller** | self | self | **approve** (+gates) | **approve** (+gates) | request to operator |
|
||||
| **operator** | self | self | execute (controller-approved) | execute (controller-approved) | execute (controller-approved) |
|
||||
| **admin** | self | self | execute | execute | execute (break-glass) |
|
||||
|
||||
Legend: *self* = may perform for its own client session; *request only* = may
|
||||
raise a restart request but not authorize or execute it; *approve* = may
|
||||
authorize under §2.1 gates; *execute* = may perform the process action after the
|
||||
authorization is recorded.
|
||||
|
||||
Key invariants:
|
||||
|
||||
- **No LLM worker role (author/reviewer/merger/reconciler) may perform or
|
||||
authorize a full or host restart.** They may only reconnect/rebind their own
|
||||
client and file a restart request.
|
||||
- **Controller approval authorizes; operator/admin executes.** The approving
|
||||
controller and the executing operator may be the same human, but both the
|
||||
approval and the execution are audited.
|
||||
- Privileged process actions (full restart, host restart) are reserved to
|
||||
**operator/admin**, never to an automated worker.
|
||||
|
||||
### 2.4 Approved conditions
|
||||
|
||||
A restart at rung 3+ is approved only under one of these recorded conditions:
|
||||
|
||||
- **No affected sessions:** the control plane has no live session that would be
|
||||
interrupted (verified, not assumed).
|
||||
- **Full drain acknowledged:** every affected session has drained
|
||||
(no open critical section — no held mutation lease mid-write) and the drain is
|
||||
acknowledged in the audit record.
|
||||
- **Controller + gates:** controller approval plus passing automated safety
|
||||
gates (§2.1), the standard v1 path.
|
||||
- **Quorum:** not required in v1; reserved for a future superseding ADR.
|
||||
- **Break-glass:** an incident-backed emergency exception (§2.5).
|
||||
|
||||
Restart **never** bypasses mutation gates mid-critical-section. Drain before
|
||||
restart is mandatory except under break-glass with a declared incident.
|
||||
|
||||
### 2.5 Break-glass
|
||||
|
||||
Break-glass is a **separate, narrower** authorization path for emergencies where
|
||||
the normal drain-and-approve path cannot complete (e.g. the control plane is
|
||||
wedged and cannot drain).
|
||||
|
||||
Break-glass conditions:
|
||||
|
||||
- A declared incident record exists (id, timestamp, declarer) **before** the
|
||||
action.
|
||||
- The action is taken by **operator or admin** authority only — never by an LLM
|
||||
worker role, and never unilaterally by an operator with active peers when a
|
||||
controller is reachable.
|
||||
- The scope is the minimum necessary rung of the ladder.
|
||||
- A **mandatory post-hoc audit** entry is filed: what was restarted, why the
|
||||
normal path was impossible, which sessions were affected, and the incident id.
|
||||
|
||||
Break-glass suspends the drain requirement, not the audit requirement.
|
||||
|
||||
### 2.6 Explicit prohibitions
|
||||
|
||||
- **A unilateral LLM or operator full restart while active peer sessions
|
||||
exist is forbidden.** An LLM worker role must not kill, restart, or relaunch
|
||||
the MCP process; a lone operator must not full-restart over live peer work
|
||||
without controller approval or a break-glass incident.
|
||||
- Process-kill recovery is forbidden as a routine tool (#630). This ADR does not
|
||||
introduce a kill path.
|
||||
- Ambiguous policy state **denies** restart (§4).
|
||||
|
||||
## 3. Security requirements
|
||||
|
||||
- Full restart and host restart are **privileged**; only operator/admin execute
|
||||
them, only after a controller approval or break-glass incident is recorded.
|
||||
- Break-glass is a distinct authorization path with its own audit mandate; it is
|
||||
never the default and never silent.
|
||||
- **Every approval and every restart action is audited** (who approved, who
|
||||
executed, scope, affected sessions, condition, policy version). No restart is
|
||||
authorized without a durable audit entry.
|
||||
|
||||
## 4. Failure behavior
|
||||
|
||||
**Ambiguous policy → deny restart.** If it cannot be established that a
|
||||
restart is authorized under §2 — unknown affected-session state, missing
|
||||
controller approval, absent break-glass incident, or an unclassifiable request —
|
||||
the safe action is to **refuse** the restart and stop with a recovery report,
|
||||
never to restart on assumption.
|
||||
|
||||
## 5. Policy IDs (for enforcement code)
|
||||
|
||||
Enforcement code — the restart coordinator (a later child of #655), the #630
|
||||
contamination guard, and the #642 console restart UX — binds to these stable
|
||||
policy identifiers rather than to prose:
|
||||
|
||||
| Policy ID | Statement |
|
||||
|---|---|
|
||||
| `RG-01` | Restart is last resort; rungs 1–2 must be tried and recorded first (§2.2). |
|
||||
| `RG-02` | v1 authority = controller approval + automated safety gates (§2.1). |
|
||||
| `RG-03` | No LLM worker role performs or authorizes full/host restart (§2.3). |
|
||||
| `RG-04` | Full/host restart executed by operator/admin only, post approval (§2.3). |
|
||||
| `RG-05` | Drain before restart is mandatory except break-glass with incident (§2.4). |
|
||||
| `RG-06` | Break-glass requires a pre-declared incident and post-hoc audit (§2.5). |
|
||||
| `RG-07` | Unilateral LLM/operator full restart with active peers is forbidden (§2.6). |
|
||||
| `RG-08` | Ambiguous policy state denies restart (§4). |
|
||||
|
||||
The `restart-governance/v1` **policy version** field is emitted on future
|
||||
restart audit events so approvals can be reconciled against the policy revision
|
||||
in force.
|
||||
|
||||
## 6. Dogfooding
|
||||
|
||||
Gitea-Tools governs its own MCP control plane by this policy. Author, reviewer,
|
||||
merger, and reconciler sessions operating on this repository use the recovery
|
||||
ladder (§2.2) — reconnect and rebind, never self-restart — and any real restart
|
||||
of the Gitea-Tools stable control runtime follows the controller-approval +
|
||||
drain path defined here.
|
||||
|
||||
## 7. Acceptance and cross-links
|
||||
|
||||
This ADR is the authoritative restart-governance policy. It **must** stay
|
||||
cross-linked from the safety model and the web-console deployment boundary:
|
||||
|
||||
- `docs/safety-model.md` § Process restart governance references this ADR.
|
||||
- `docs/webui-deployment.md` references this ADR for restart/reload disposition.
|
||||
|
||||
It is linked to its issue lineage — umbrella **#655**, vision **#652**, roadmap
|
||||
**#653**, contamination guard **#630**, and console restart UX **#642** — in
|
||||
§ Related above.
|
||||
|
||||
## 8. Non-goals
|
||||
|
||||
- Implementing the restart coordinator or approval state machine (#630, later
|
||||
children of #655).
|
||||
- Implementing HA multi-instance restart or quorum machinery.
|
||||
- Introducing any process-kill or auto-restart tool; existing auto-restart
|
||||
behavior must be inventoried before any new restart tool is enabled.
|
||||
@@ -0,0 +1,95 @@
|
||||
# MCP restart coordinator and impact analysis (#658)
|
||||
|
||||
Before any sanctioned MCP restart, a central coordinator evaluates the live
|
||||
control-plane state and produces an **impact preview** so operators and the web
|
||||
console (#642 / #652) can see the blast radius *before* concurrent LLM work is
|
||||
disrupted. Uncoordinated restarts destroy in-flight author/reviewer/merger work
|
||||
and give operators no way to see what they are about to break.
|
||||
|
||||
This lands the coordinator + impact DTO + a dry-run MCP tool. It is the single
|
||||
sanctioned entry point for restart evaluation post-#657 (which inventoried the
|
||||
restart/reload/kill paths). The **mutative apply** path — actually performing a
|
||||
restart — is a later child gated by a drain proof and is explicitly out of
|
||||
scope here.
|
||||
|
||||
## Components
|
||||
|
||||
| Piece | Where | Responsibility |
|
||||
|-------|-------|----------------|
|
||||
| `restart_coordinator.evaluate_restart_impact` | `restart_coordinator.py` | Pure classification: inventory → impact report DTO. No I/O, no restart. |
|
||||
| `RestartImpactReport` / `SessionImpact` / `LeaseImpact` | `restart_coordinator.py` | Console-facing DTO (`.as_dict()` is JSON-serializable). |
|
||||
| `ControlPlaneDB.list_sessions` | `control_plane_db.py` | Read-only session inventory (the process-level unit a restart kills). |
|
||||
| `gitea_request_mcp_restart` | `gitea_mcp_server.py` | MCP tool: gathers inventory from the #613 DB, calls the coordinator, returns the report. Dry-run only. |
|
||||
|
||||
## Dimensions evaluated
|
||||
|
||||
The coordinator classifies the inventory across the dimensions #658 requires:
|
||||
|
||||
- **Sessions** — every active MCP session; a restart terminates all of them.
|
||||
Liveness = `status == active` **and** the owner pid is alive **and** the
|
||||
heartbeat is fresh (default window 15 min). Dead/stale sessions do not count
|
||||
toward blast radius.
|
||||
- **Leases / locks** — control-plane leases joined with work items and their
|
||||
freshness (`lease_lifecycle.classify_lease_freshness`). Only `active` (live
|
||||
owner) leases are *disruptive*; expired / released / dead-process leases never
|
||||
withhold a restart.
|
||||
- **Issue / PR work** — the issues and PRs behind disruptive leases.
|
||||
- **Mutations / critical sections** — a live lease carrying an author worktree
|
||||
or a mutating phase (`implementing`, `publishing`, `merging`, …) is a
|
||||
critical section a restart must not sever.
|
||||
- **Terminal (merge) lock** — an active terminal lock always makes a restart
|
||||
unsafe.
|
||||
- **Prior recovery attempts** — narrower recovery already tried (e.g. sanctioned
|
||||
client reconnects) is echoed so the operator sees the escalation history.
|
||||
|
||||
## Verdict
|
||||
|
||||
Exactly three verdicts, matching the acceptance criteria:
|
||||
|
||||
| Verdict | `allow_restart` | Meaning |
|
||||
|---------|-----------------|---------|
|
||||
| `safe` | `true` | No other live sessions, no live leases, no terminal lock. |
|
||||
| `unsafe` | `false` | Live work would be disrupted and no operator override is present — **or** the inventory could not be completed (fail closed). |
|
||||
| `override` | `true` | Live work present, but an operator override accepts the blast radius. |
|
||||
|
||||
`override_would_allow` tells the console whether an override path exists for the
|
||||
current state. `blast_radius` is a `none` / `low` / `medium` / `high` severity
|
||||
band derived from the affected session and work counts.
|
||||
|
||||
### Fail closed
|
||||
|
||||
If the control-plane inventory cannot be completed (DB unavailable, a listing
|
||||
failed), `inventory_complete` is `false` and the verdict is `unsafe` / deny. An
|
||||
incomplete evaluation must never green-light a restart.
|
||||
|
||||
### Operator override authority
|
||||
|
||||
Override authority is read from the environment variable
|
||||
`GITEA_OPERATOR_RESTART_OVERRIDE_AUTHORIZATION` and **never** from a tool
|
||||
argument. A worker session cannot set an environment variable on an
|
||||
already-running daemon, so override cannot be self-asserted (same pattern as the
|
||||
#630 daemon-maintenance authorization). The `request_override` tool argument only
|
||||
expresses caller intent; it takes effect solely when the environment
|
||||
authorization is present.
|
||||
|
||||
## The tool
|
||||
|
||||
```text
|
||||
gitea_request_mcp_restart(remote, host, org, repo,
|
||||
dry_run=True, request_override=False,
|
||||
session_id=None, limit=200)
|
||||
```
|
||||
|
||||
Read-only, dry-run, and it **never restarts anything**. `apply_supported` is
|
||||
always `false`; passing `dry_run=False` performs no restart and reports that
|
||||
apply is gated by a drain proof (a separate child).
|
||||
|
||||
## Audit
|
||||
|
||||
Every evaluation carries an `audit_record` (event, coordinator version, verdict,
|
||||
allow decision, blast radius, counts, timestamp) so restart decisions are
|
||||
auditable. No secrets flow through the coordinator — session ids, pids, and
|
||||
profiles are operational metadata only.
|
||||
|
||||
A representative dry-run report is in
|
||||
[`mcp-restart-impact-sample.json`](./mcp-restart-impact-sample.json).
|
||||
@@ -0,0 +1,148 @@
|
||||
{
|
||||
"coordinator_version": "1.0.0-issue-658",
|
||||
"evaluated_at": "2026-07-24T06:00:00+00:00",
|
||||
"dry_run": true,
|
||||
"restart_performed": false,
|
||||
"inventory_complete": true,
|
||||
"incomplete_reasons": [],
|
||||
"verdict": "unsafe",
|
||||
"allow_restart": false,
|
||||
"override_would_allow": true,
|
||||
"operator_override": false,
|
||||
"blast_radius": "high",
|
||||
"reasons": [
|
||||
"live work would be disrupted; restart denied without operator override",
|
||||
"1 critical section(s) in flight (active lease with a live owner)"
|
||||
],
|
||||
"affected_sessions": [
|
||||
{
|
||||
"session_id": "prgs-author-30988-d6f43c25",
|
||||
"role": "author",
|
||||
"profile": "prgs-author",
|
||||
"pid": 1,
|
||||
"status": "active",
|
||||
"alive": true,
|
||||
"heartbeat_stale": false,
|
||||
"is_requester": false,
|
||||
"live": true
|
||||
},
|
||||
{
|
||||
"session_id": "prgs-reviewer-4157-0ce9",
|
||||
"role": "reviewer",
|
||||
"profile": "prgs-reviewer",
|
||||
"pid": 1,
|
||||
"status": "active",
|
||||
"alive": true,
|
||||
"heartbeat_stale": false,
|
||||
"is_requester": true,
|
||||
"live": true
|
||||
}
|
||||
],
|
||||
"affected_leases": [
|
||||
{
|
||||
"lease_id": "lease-abc",
|
||||
"session_id": "prgs-author-30988-d6f43c25",
|
||||
"role": "author",
|
||||
"phase": "implementing",
|
||||
"freshness": "active",
|
||||
"work_kind": "issue",
|
||||
"work_number": 658,
|
||||
"worktree_path": "/repo/branches/feat-issue-658",
|
||||
"disruptive": true,
|
||||
"is_mutation": true,
|
||||
"is_critical_section": true
|
||||
},
|
||||
{
|
||||
"lease_id": "lease-dead",
|
||||
"session_id": "prgs-author-91485",
|
||||
"role": "author",
|
||||
"phase": "allocated",
|
||||
"freshness": "stale_dead_process",
|
||||
"work_kind": "issue",
|
||||
"work_number": 651,
|
||||
"worktree_path": null,
|
||||
"disruptive": false,
|
||||
"is_mutation": false,
|
||||
"is_critical_section": false
|
||||
}
|
||||
],
|
||||
"critical_sections": [
|
||||
{
|
||||
"lease_id": "lease-abc",
|
||||
"session_id": "prgs-author-30988-d6f43c25",
|
||||
"role": "author",
|
||||
"phase": "implementing",
|
||||
"freshness": "active",
|
||||
"work_kind": "issue",
|
||||
"work_number": 658,
|
||||
"worktree_path": "/repo/branches/feat-issue-658",
|
||||
"disruptive": true,
|
||||
"is_mutation": true,
|
||||
"is_critical_section": true
|
||||
}
|
||||
],
|
||||
"affected_issues": [
|
||||
658
|
||||
],
|
||||
"affected_prs": [],
|
||||
"mutations": [
|
||||
{
|
||||
"lease_id": "lease-abc",
|
||||
"session_id": "prgs-author-30988-d6f43c25",
|
||||
"role": "author",
|
||||
"phase": "implementing",
|
||||
"freshness": "active",
|
||||
"work_kind": "issue",
|
||||
"work_number": 658,
|
||||
"worktree_path": "/repo/branches/feat-issue-658",
|
||||
"disruptive": true,
|
||||
"is_mutation": true,
|
||||
"is_critical_section": true
|
||||
}
|
||||
],
|
||||
"terminal_lock": null,
|
||||
"ack_state": {
|
||||
"prgs-author-30988-d6f43c25": "pending"
|
||||
},
|
||||
"prior_recovery_attempts": [
|
||||
{
|
||||
"kind": "client_reconnect",
|
||||
"at": "2026-07-24T06:00:00+00:00",
|
||||
"outcome": "insufficient"
|
||||
}
|
||||
],
|
||||
"counts": {
|
||||
"sessions_total": 2,
|
||||
"sessions_live_other": 1,
|
||||
"leases_total": 2,
|
||||
"leases_disruptive": 1,
|
||||
"critical_sections": 1,
|
||||
"mutations": 1,
|
||||
"affected_issues": 1,
|
||||
"affected_prs": 0,
|
||||
"prior_recovery_attempts": 1
|
||||
},
|
||||
"audit_record": {
|
||||
"event": "restart_impact_evaluated",
|
||||
"coordinator_version": "1.0.0-issue-658",
|
||||
"evaluated_at": "2026-07-24T06:00:00+00:00",
|
||||
"dry_run": true,
|
||||
"operator_override": false,
|
||||
"requesting_session_id": "prgs-reviewer-4157-0ce9",
|
||||
"inventory_complete": true,
|
||||
"verdict": "unsafe",
|
||||
"allow_restart": false,
|
||||
"blast_radius": "high",
|
||||
"counts": {
|
||||
"sessions_total": 2,
|
||||
"sessions_live_other": 1,
|
||||
"leases_total": 2,
|
||||
"leases_disruptive": 1,
|
||||
"critical_sections": 1,
|
||||
"mutations": 1,
|
||||
"affected_issues": 1,
|
||||
"affected_prs": 0,
|
||||
"prior_recovery_attempts": 1
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
# MCP restart / reload / kill path inventory (#657)
|
||||
|
||||
Complete inventory of every code, script, and host path that can **restart,
|
||||
reload, reconnect, kill, or force-recreate** an MCP process in this project,
|
||||
with each path classified and linked to the guard that constrains it.
|
||||
|
||||
This document is the human-readable companion to the machine-readable registry
|
||||
in [`mcp_restart_paths.py`](../mcp_restart_paths.py). The two are kept in
|
||||
lock-step by [`tests/test_mcp_restart_paths.py`](../tests/test_mcp_restart_paths.py):
|
||||
every `path_id` below must appear in this file, and the source guards are run
|
||||
against the live tree.
|
||||
|
||||
Roadmap linkage: this inventory is the enumeration step of the restart
|
||||
governance work — parent **#655**, restart-governance ADR **#656**, vision
|
||||
**#652**, roadmap **#653**. Related detection/guard work: master-advance
|
||||
staleness **#591**/**#420**, side-effect-free resolver **#685**, transport flap
|
||||
**#584**, manual-kill contamination **#630**.
|
||||
|
||||
## Classifications
|
||||
|
||||
| Classification | Meaning |
|
||||
|---|---|
|
||||
| `sanctioned_narrow_recovery` | One-shot, safe-by-construction recovery that never targets the running daemon. |
|
||||
| `guarded_fail_closed` | Detects a restart-requiring condition, then fails mutations closed and emits reconnect guidance. Never self-restarts. |
|
||||
| `forbidden` | A workflow-safety violation; where an LLM tool could invoke it, it is marked contamination. |
|
||||
| `removed` | A previously-existing unguarded restart primitive that has been deleted; a regression guard keeps it absent. |
|
||||
| `host_residual` | Behavior owned by the host/IDE, outside this process's control. Documented, not code-guarded here. |
|
||||
|
||||
## The rule
|
||||
|
||||
**No component may perform an unguarded full restart of the MCP daemon.** The
|
||||
in-process daemon (`gitea_mcp_server.py`, `mcp_server.py`,
|
||||
`role_session_router.py`) must never replace or terminate its own process:
|
||||
replacing the process after the host has wired up the stdio pipes desyncs the
|
||||
JSON-RPC transport (observed with Antigravity/Cascade hosts). Recovery is owned
|
||||
by the host/operator via a client reconnect — the daemon only ever *detects*
|
||||
and *fails closed*.
|
||||
|
||||
## Inventory
|
||||
|
||||
| path_id | Classification | Mechanism | Guard | Refs |
|
||||
|---|---|---|---|---|
|
||||
| `cli_venv_bootstrap_execv` | sanctioned_narrow_recovery | CLI wrapper scripts re-exec into `venv/bin/python3` via `os.execv`, guarded by `sys.executable != venv_python`. | One-shot pre-import bootstrap; runs before any MCP transport exists and only when not already on the venv interpreter; idempotent guard prevents a re-exec loop. | #657 |
|
||||
| `daemon_self_replacement` | forbidden | The daemon replacing/terminating its own process (`os.execv`/`os.kill`/`os._exit`) to reload code. | Forbidden by design; enforced against the source tree by `assert_no_daemon_self_replacement()`. | #657, #584 |
|
||||
| `legacy_auto_restart_helper` | removed | A helper (`_trigger_mcp_auto_restart`) that actively restarted the server from the read-only resolver path. | Removed in #685; kept absent by `assert_auto_restart_helper_absent()`. | #685, #657 |
|
||||
| `config_touch_reload` | removed | Touching (utime) the MCP client config to make the host reload the server. | Removed from the resolver in #685: stale detection is report-only, never mutating config, spawning threads, or calling `os._exit`. | #685, #657 |
|
||||
| `master_advance_auto_restart` | guarded_fail_closed | On-disk master advancing past the running code. | `master_parity_gate` captures startup parity and blocks mutations while stale, emitting restart guidance; the process never self-restarts. | #420, #591, #657 |
|
||||
| `stale_runtime_resolver_reconnect` | guarded_fail_closed | The capability resolver detecting a stale serving process. | Report-only (#685): returns `restart_required`/`stop_required` and an exact reconnect action; no restart, thread, config touch, or `os._exit`. | #685, #657 |
|
||||
| `manual_daemon_kill` | forbidden | Shell kills of the daemon: `pkill -f mcp_server.py`, `killall`, broad `pkill -f python` sweeps, or `kill <pid>` of a daemon pid. | Forbidden (#630): `runtime_recovery_guard` classifies these as contamination and `gitea_record_daemon_process_kill_attempt` writes a durable marker that fails later mutations closed. Operator maintenance authorization is read only from the environment. | #630, #657 |
|
||||
| `conflict_marker_infra_stop` | guarded_fail_closed | The daemon entrypoint scans for unresolved merge-conflict markers at startup and stops (`sys.exit(1)`). | Fail-closed startup stop, not a restart: the process exits and waits for the operator to resolve conflicts and relaunch; never loops. | #657 |
|
||||
| `ide_client_reconnect` | host_residual | A manual `/mcp reconnect` (or equivalent host action) that recreates the MCP client connection. | Outside this process's control; the sanctioned recovery the gates point operators toward. No in-process code initiates it. | #584, #656, #657 |
|
||||
| `profile_switch_runtime` | sanctioned_narrow_recovery | Switching the active execution profile at runtime (dynamic-profile mode). | In-process and restart-free: `runtime_switching_supported` is true, so a switch rebinds capability without recreating the process. | #656, #657 |
|
||||
|
||||
## Guards enforced in CI
|
||||
|
||||
`tests/test_mcp_restart_paths.py` asserts, against the live source tree:
|
||||
|
||||
1. **Registry well-formedness** — every path has a valid classification, a
|
||||
non-empty guard description, references, and locations; ids are unique; all
|
||||
five classifications are represented.
|
||||
2. **Unknown restart attempts fail closed** —
|
||||
`assert_restart_attempt_registered()` raises `UnknownRestartPathError` for
|
||||
any path id not in this inventory, so a novel/unnamed restart primitive
|
||||
cannot slip through silently.
|
||||
3. **Daemon never self-replaces** — `assert_no_daemon_self_replacement()` scans
|
||||
the daemon modules for `os.execv`/`os.kill`/`os._exit`/`os.abort` calls
|
||||
(comment/docstring mentions are ignored) and finds none.
|
||||
4. **Legacy helper stays removed** — `assert_auto_restart_helper_absent()`
|
||||
confirms `_trigger_mcp_auto_restart` has not returned.
|
||||
5. **pkill stays forbidden** — a daemon `pkill` command still classifies as
|
||||
contamination via `runtime_recovery_guard`.
|
||||
|
||||
## Residual host behaviors (outside process control)
|
||||
|
||||
* `/mcp reconnect` in the IDE/host — the sanctioned recovery for stale-runtime,
|
||||
transport-flap (#584), and worktree-binding conditions. The daemon can only
|
||||
emit guidance toward it.
|
||||
* Host-level process management (the operator relaunching the daemon after a
|
||||
fail-closed stop, or after resolving merge conflicts).
|
||||
|
||||
These are documented rather than code-guarded because the process cannot
|
||||
observe or gate them from inside itself.
|
||||
|
||||
## Rollout
|
||||
|
||||
Per #657, guards are introduced flag-free as **regression assertions** (they
|
||||
codify invariants that already hold) before any hard runtime block is layered
|
||||
on. When the restart coordinator (#655/#656) lands, registered paths gain a
|
||||
coordinator token/capability check; unregistered attempts already fail closed
|
||||
today via `assert_restart_attempt_registered()`.
|
||||
@@ -136,6 +136,7 @@ that gates each call, not which tools exist.
|
||||
- `gitea_release_merger_pr_lease`
|
||||
- `gitea_release_reviewer_pr_lease`
|
||||
- `gitea_release_workflow_lease`
|
||||
- `gitea_request_mcp_restart`
|
||||
- `gitea_resolve_task_capability`
|
||||
- `gitea_resume_review_draft`
|
||||
- `gitea_review_pr`
|
||||
|
||||
@@ -46,3 +46,17 @@ If shell helpers are unavailable and MCP commit cannot run, stop with a recovery
|
||||
report (restart session, clear hung terminals, use MCP-native commit). See
|
||||
[`llm-workflow-runbooks.md`](llm-workflow-runbooks.md) § MCP-native commit path
|
||||
(#260) and agent temp artifact cleanup (#261).
|
||||
|
||||
## 7. Process restart governance
|
||||
|
||||
Restarting the MCP control-plane process is destructive to concurrent multi-role
|
||||
work and is governed by a dedicated policy. Restart is a **last resort** behind
|
||||
narrower recoveries (reconnect, rebind), full/host restart is reserved to
|
||||
operator/admin under **controller approval + automated safety gates**, a
|
||||
unilateral LLM or operator full restart with active peers is **forbidden**, and
|
||||
ambiguous policy state **denies** restart. Break-glass is a separate,
|
||||
incident-backed path with a mandatory audit.
|
||||
|
||||
See [`architecture/mcp-restart-governance.md`](architecture/mcp-restart-governance.md)
|
||||
(#656) for the authorization matrix, the recovery ladder, break-glass
|
||||
conditions, and the `RG-01`–`RG-08` policy IDs.
|
||||
|
||||
@@ -55,6 +55,15 @@ shipped to the browser.
|
||||
assumption paths, and the client-secret policy. Use it to verify an instance is
|
||||
configured for internal-only operation.
|
||||
|
||||
## Process restart / reload disposition
|
||||
|
||||
The console never exposes a restart or reload control; process restart of the
|
||||
MCP control-plane runtime is governed separately. Restart is a last resort behind
|
||||
reconnect/rebind, full restart is operator/admin-only under controller approval
|
||||
plus safety gates, and break-glass is an incident-backed path. See
|
||||
[`architecture/mcp-restart-governance.md`](architecture/mcp-restart-governance.md)
|
||||
(#656).
|
||||
|
||||
## Non-goals (MVP)
|
||||
|
||||
- Full SSO or session login in the UI
|
||||
|
||||
@@ -54,6 +54,7 @@ status, onboarding checklist state, and the fail-closed error payloads (#635).
|
||||
| `/` | Home / operator overview |
|
||||
| `/health` | JSON liveness (`status`, `service`, `mode`, `timestamp`, `uptime_seconds`) |
|
||||
| `/api/v1/system/health` | Structured read-only system health (#634) |
|
||||
| `/system-health` | System-health dashboard — readiness, version/uptime, dependencies, MCP namespaces, stale-runtime parity (#639) |
|
||||
| `/queue` | Live PR and issue queue dashboard (#429) |
|
||||
| `/api/queue` | JSON queue export with pagination metadata |
|
||||
| `/projects` | Project registry list with status and onboarding progress (#427, #635) |
|
||||
@@ -258,6 +259,37 @@ Not-yet-implemented surfaces (`/sessions`, `/inventory`, `/timeline`,
|
||||
surfaces are backed by #636). Mutating methods on stub routes still fail closed
|
||||
with `read-only-mvp`.
|
||||
|
||||
## System-health dashboard (#639)
|
||||
|
||||
`/system-health` renders the same snapshot the `/api/v1/system/health` API
|
||||
returns, so the page and the API can never disagree. Cards: overall readiness,
|
||||
stale-runtime parity, version and uptime, dependency probes, MCP namespaces,
|
||||
probe errors (only when present), and recovery pointers. `?deep=1` opts into
|
||||
the network probe exactly as the API does; the plain page load stays cheap.
|
||||
|
||||
Field authority and honesty rules:
|
||||
|
||||
* `ready` and `readiness_complete` are shown separately. A snapshot whose
|
||||
required probes never ran is not the same as one that ran them and passed,
|
||||
and the page never collapses the two into an unproven green.
|
||||
* A probe that did not run appears under **Not probed**, never as healthy.
|
||||
* `stale_runtime.mutation_safe` is displayed verbatim from the API. When the
|
||||
runtime is stale, or when parity is indeterminate, the page warns and does
|
||||
not claim mutation safety.
|
||||
* MCP namespaces are reported `unproven`: the web process runs outside the
|
||||
IDE-managed MCP client and cannot prove that path (#543).
|
||||
|
||||
Redaction is split by field kind. Free text — probe details, readiness and
|
||||
parity reasons, probe errors — passes through `system_health.redact`.
|
||||
Structured fields — commit SHAs, probe names, statuses, timestamps — are
|
||||
HTML-escaped only, because `redact`'s opaque-token rule matches any run of 32
|
||||
or more characters and would otherwise blank every 40-character git SHA, which
|
||||
is precisely the evidence the parity view exists to show.
|
||||
|
||||
The dashboard is read-only: no restart, reload, or process-kill control. Those
|
||||
arrive in Phase 2 (#642). Recovery guidance points at the sanctioned client
|
||||
reconnect / operator restart path — never a manual daemon kill (#630).
|
||||
|
||||
## Deployment boundary (#435)
|
||||
|
||||
MVP serves on loopback by default. Binding `0.0.0.0` or `::` is **refused**
|
||||
|
||||
Reference in New Issue
Block a user