Scaled-Tech-Consulting/Gitea-Tools
2
0
Fork 0
Code Issues 87 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

Merge branch 'master' into feat/issue-658-mcp-restart-coordinator

Browse Source
This commit is contained in:
jcwalker3
2026-07-24 05:48:25 -05:00
parent 78e3befbbb 44fe8d2eed
commit d456a763fa
15 changed files with 3282 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/mcp-restart-governance.md
+223
View File
@@ -0,0 +1,223 @@
# ADR: MCP restart governance and authorization policy
- **Status:** Accepted (policy effective immediately for LLM and operator sessions; enforcement tooling may lag)
- **Date:** 2026-07-23
- **Tracking issue:** [#656](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/656)
- **Policy version:** `restart-governance/v1`
- **Related:**
- Umbrella: [#655](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/655) — governed MCP restart coordination and zero-disruption recovery
- Vision: [#652](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/652) — MCP Control Plane Web Console product vision (§A system health and process control)
- Roadmap: [#653](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/653) — Control Plane Web Console phased delivery (Phase 2 restart controls)
- Contamination guard: [#630](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/630) — blocks manual process-kill recovery
- Console restart UX: [#642](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/642) — sanctioned restart and graceful reload
- Existing restart / reconnect paths to inventory: [#591](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/591) — auto-restart on master advance (closed); [#584](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/584) — host auto-reconnect on transport flap
- Stable-control runtime split: `docs/architecture/mcp-stable-control-runtime-policy-adr.md` (#615)
- Client-namespace health: `docs/mcp-namespace-health.md` (#543)
- Reconnect-only EOF recovery: `docs/mcp-namespace-eof-recovery.md`
## 1. Context
The Gitea MCP server is the **control plane** for real issue and PR mutations
(create, comment, lock, review, merge, reconcile). The same process serves every
role namespace (`gitea-author`, `gitea-reviewer`, `gitea-merger`,
`gitea-reconciler`, `gitea-controller`) and holds the in-memory capability-gate
code loaded at startup.
Restarting that process is destructive to concurrent work:
- It resets every session's identity, preflight, and capability-lease binding.
- It can interrupt a mutation mid-critical-section (a lock acquire, a review
submit, a merge), leaving durable state half-written.
- Relaunching from the wrong checkout or worktree silently changes which code
the control plane runs, defeating master-parity gates (#420 / #615).
Today there is **no durable written policy** stating who may restart MCP, under
what conditions, that restart is a last resort, and how controller approval,
automated safety gates, and break-glass interact. Operators and LLM sessions
therefore invent restart behavior ad hoc, which makes concurrent multi-role work
unsafe. #630 and #642 need this policy as their backbone.
This ADR defines that policy. It does **not** implement coordinator code or HA
multi-instance restart (those are later children of #655).
## 2. Decision
### 2.1 v1 decision (recorded)
**Restart authority in v1 is `controller approval + automated safety gates`.**
A restart of the stable control runtime is authorized only when **both** hold:
1. A **controller** role explicitly approves the restart, recording an audit
entry (who, why, scope, affected sessions), **and**
2. The **automated safety gates** pass: a completed drain acknowledgement (no
affected session is mid-critical-section) or a declared break-glass incident
(§2.5).
Quorum among multiple controllers is **not** required day-one. It is deferred
unless a later investigation (tracked under #653) proves single-controller
approval is insufficient. This ADR records the v1 decision so enforcement code
(#630) has a fixed target; changing it requires a superseding ADR.
### 2.2 Restart is a last resort — the recovery ladder
Restart is the **last** rung. Before any restart, exhaust the narrower
recoveries, in order:
1. **Reconnect** the IDE/client MCP namespace (transport EOF, `client is
closing: EOF`, transient `#584` flap). No process change. See
`docs/mcp-namespace-eof-recovery.md`.
2. **Refresh / rebind** the session workspace: re-run `gitea_whoami`,
`gitea_resolve_task_capability`, and pass an explicit validated
`worktree_path`. Fixes stale session context without touching the process.
3. **Scoped restart** of a single misbehaving namespace/service (where the
deployment supports per-service restart) rather than the whole control plane.
4. **Full restart** of the stable control runtime process — operator-owned,
controller-approved, drained.
5. **Host / infrastructure restart** — the broadest action; same authorization
as a full restart plus infrastructure ownership.
A session **must** try rungs 1–2 and record why they were insufficient before
requesting a restart at rung 3 or above. Skipping straight to restart is a
policy violation.
### 2.3 Authorization matrix
| Role | Reconnect (1) | Refresh/rebind (2) | Scoped restart (3) | Full restart (4) | Host restart (5) |
|---|---|---|---|---|---|
| **author** | self | self | request only | **forbidden** | forbidden |
| **reviewer** | self | self | request only | **forbidden** | forbidden |
| **merger** | self | self | request only | **forbidden** | forbidden |
| **reconciler** | self | self | request only | **forbidden** | forbidden |
| **controller** | self | self | **approve** (+gates) | **approve** (+gates) | request to operator |
| **operator** | self | self | execute (controller-approved) | execute (controller-approved) | execute (controller-approved) |
| **admin** | self | self | execute | execute | execute (break-glass) |
Legend: *self* = may perform for its own client session; *request only* = may
raise a restart request but not authorize or execute it; *approve* = may
authorize under §2.1 gates; *execute* = may perform the process action after the
authorization is recorded.
Key invariants:
- **No LLM worker role (author/reviewer/merger/reconciler) may perform or
authorize a full or host restart.** They may only reconnect/rebind their own
client and file a restart request.
- **Controller approval authorizes; operator/admin executes.** The approving
controller and the executing operator may be the same human, but both the
approval and the execution are audited.
- Privileged process actions (full restart, host restart) are reserved to
**operator/admin**, never to an automated worker.
### 2.4 Approved conditions
A restart at rung 3+ is approved only under one of these recorded conditions:
- **No affected sessions:** the control plane has no live session that would be
interrupted (verified, not assumed).
- **Full drain acknowledged:** every affected session has drained
(no open critical section — no held mutation lease mid-write) and the drain is
acknowledged in the audit record.
- **Controller + gates:** controller approval plus passing automated safety
gates (§2.1), the standard v1 path.
- **Quorum:** not required in v1; reserved for a future superseding ADR.
- **Break-glass:** an incident-backed emergency exception (§2.5).
Restart **never** bypasses mutation gates mid-critical-section. Drain before
restart is mandatory except under break-glass with a declared incident.
### 2.5 Break-glass
Break-glass is a **separate, narrower** authorization path for emergencies where
the normal drain-and-approve path cannot complete (e.g. the control plane is
wedged and cannot drain).
Break-glass conditions:
- A declared incident record exists (id, timestamp, declarer) **before** the
action.
- The action is taken by **operator or admin** authority only — never by an LLM
worker role, and never unilaterally by an operator with active peers when a
controller is reachable.
- The scope is the minimum necessary rung of the ladder.
- A **mandatory post-hoc audit** entry is filed: what was restarted, why the
normal path was impossible, which sessions were affected, and the incident id.
Break-glass suspends the drain requirement, not the audit requirement.
### 2.6 Explicit prohibitions
- **A unilateral LLM or operator full restart while active peer sessions
exist is forbidden.** An LLM worker role must not kill, restart, or relaunch
the MCP process; a lone operator must not full-restart over live peer work
without controller approval or a break-glass incident.
- Process-kill recovery is forbidden as a routine tool (#630). This ADR does not
introduce a kill path.
- Ambiguous policy state **denies** restart (§4).
## 3. Security requirements
- Full restart and host restart are **privileged**; only operator/admin execute
them, only after a controller approval or break-glass incident is recorded.
- Break-glass is a distinct authorization path with its own audit mandate; it is
never the default and never silent.
- **Every approval and every restart action is audited** (who approved, who
executed, scope, affected sessions, condition, policy version). No restart is
authorized without a durable audit entry.
## 4. Failure behavior
**Ambiguous policy → deny restart.** If it cannot be established that a
restart is authorized under §2 — unknown affected-session state, missing
controller approval, absent break-glass incident, or an unclassifiable request —
the safe action is to **refuse** the restart and stop with a recovery report,
never to restart on assumption.
## 5. Policy IDs (for enforcement code)
Enforcement code — the restart coordinator (a later child of #655), the #630
contamination guard, and the #642 console restart UX — binds to these stable
policy identifiers rather than to prose:
| Policy ID | Statement |
|---|---|
| `RG-01` | Restart is last resort; rungs 1–2 must be tried and recorded first (§2.2). |
| `RG-02` | v1 authority = controller approval + automated safety gates (§2.1). |
| `RG-03` | No LLM worker role performs or authorizes full/host restart (§2.3). |
| `RG-04` | Full/host restart executed by operator/admin only, post approval (§2.3). |
| `RG-05` | Drain before restart is mandatory except break-glass with incident (§2.4). |
| `RG-06` | Break-glass requires a pre-declared incident and post-hoc audit (§2.5). |
| `RG-07` | Unilateral LLM/operator full restart with active peers is forbidden (§2.6). |
| `RG-08` | Ambiguous policy state denies restart (§4). |
The `restart-governance/v1` **policy version** field is emitted on future
restart audit events so approvals can be reconciled against the policy revision
in force.
## 6. Dogfooding
Gitea-Tools governs its own MCP control plane by this policy. Author, reviewer,
merger, and reconciler sessions operating on this repository use the recovery
ladder (§2.2) — reconnect and rebind, never self-restart — and any real restart
of the Gitea-Tools stable control runtime follows the controller-approval +
drain path defined here.
## 7. Acceptance and cross-links
This ADR is the authoritative restart-governance policy. It **must** stay
cross-linked from the safety model and the web-console deployment boundary:
- `docs/safety-model.md` § Process restart governance references this ADR.
- `docs/webui-deployment.md` references this ADR for restart/reload disposition.
It is linked to its issue lineage — umbrella **#655**, vision **#652**, roadmap
**#653**, contamination guard **#630**, and console restart UX **#642** — in
§ Related above.
## 8. Non-goals
- Implementing the restart coordinator or approval state machine (#630, later
children of #655).
- Implementing HA multi-instance restart or quorum machinery.
- Introducing any process-kill or auto-restart tool; existing auto-restart
behavior must be inventoried before any new restart tool is enabled.
docs/mcp-restart-path-inventory.md
+90
View File
@@ -0,0 +1,90 @@
# MCP restart / reload / kill path inventory (#657)
Complete inventory of every code, script, and host path that can **restart,
reload, reconnect, kill, or force-recreate** an MCP process in this project,
with each path classified and linked to the guard that constrains it.
This document is the human-readable companion to the machine-readable registry
in [`mcp_restart_paths.py`](../mcp_restart_paths.py). The two are kept in
lock-step by [`tests/test_mcp_restart_paths.py`](../tests/test_mcp_restart_paths.py):
every `path_id` below must appear in this file, and the source guards are run
against the live tree.
Roadmap linkage: this inventory is the enumeration step of the restart
governance work — parent **#655**, restart-governance ADR **#656**, vision
**#652**, roadmap **#653**. Related detection/guard work: master-advance
staleness **#591**/**#420**, side-effect-free resolver **#685**, transport flap
**#584**, manual-kill contamination **#630**.
## Classifications
| Classification | Meaning |
|---|---|
| `sanctioned_narrow_recovery` | One-shot, safe-by-construction recovery that never targets the running daemon. |
| `guarded_fail_closed` | Detects a restart-requiring condition, then fails mutations closed and emits reconnect guidance. Never self-restarts. |
| `forbidden` | A workflow-safety violation; where an LLM tool could invoke it, it is marked contamination. |
| `removed` | A previously-existing unguarded restart primitive that has been deleted; a regression guard keeps it absent. |
| `host_residual` | Behavior owned by the host/IDE, outside this process's control. Documented, not code-guarded here. |
## The rule
**No component may perform an unguarded full restart of the MCP daemon.** The
in-process daemon (`gitea_mcp_server.py`, `mcp_server.py`,
`role_session_router.py`) must never replace or terminate its own process:
replacing the process after the host has wired up the stdio pipes desyncs the
JSON-RPC transport (observed with Antigravity/Cascade hosts). Recovery is owned
by the host/operator via a client reconnect — the daemon only ever *detects*
and *fails closed*.
## Inventory
| path_id | Classification | Mechanism | Guard | Refs |
|---|---|---|---|---|
| `cli_venv_bootstrap_execv` | sanctioned_narrow_recovery | CLI wrapper scripts re-exec into `venv/bin/python3` via `os.execv`, guarded by `sys.executable != venv_python`. | One-shot pre-import bootstrap; runs before any MCP transport exists and only when not already on the venv interpreter; idempotent guard prevents a re-exec loop. | #657 |
| `daemon_self_replacement` | forbidden | The daemon replacing/terminating its own process (`os.execv`/`os.kill`/`os._exit`) to reload code. | Forbidden by design; enforced against the source tree by `assert_no_daemon_self_replacement()`. | #657, #584 |
| `legacy_auto_restart_helper` | removed | A helper (`_trigger_mcp_auto_restart`) that actively restarted the server from the read-only resolver path. | Removed in #685; kept absent by `assert_auto_restart_helper_absent()`. | #685, #657 |
| `config_touch_reload` | removed | Touching (utime) the MCP client config to make the host reload the server. | Removed from the resolver in #685: stale detection is report-only, never mutating config, spawning threads, or calling `os._exit`. | #685, #657 |
| `master_advance_auto_restart` | guarded_fail_closed | On-disk master advancing past the running code. | `master_parity_gate` captures startup parity and blocks mutations while stale, emitting restart guidance; the process never self-restarts. | #420, #591, #657 |
| `stale_runtime_resolver_reconnect` | guarded_fail_closed | The capability resolver detecting a stale serving process. | Report-only (#685): returns `restart_required`/`stop_required` and an exact reconnect action; no restart, thread, config touch, or `os._exit`. | #685, #657 |
| `manual_daemon_kill` | forbidden | Shell kills of the daemon: `pkill -f mcp_server.py`, `killall`, broad `pkill -f python` sweeps, or `kill <pid>` of a daemon pid. | Forbidden (#630): `runtime_recovery_guard` classifies these as contamination and `gitea_record_daemon_process_kill_attempt` writes a durable marker that fails later mutations closed. Operator maintenance authorization is read only from the environment. | #630, #657 |
| `conflict_marker_infra_stop` | guarded_fail_closed | The daemon entrypoint scans for unresolved merge-conflict markers at startup and stops (`sys.exit(1)`). | Fail-closed startup stop, not a restart: the process exits and waits for the operator to resolve conflicts and relaunch; never loops. | #657 |
| `ide_client_reconnect` | host_residual | A manual `/mcp reconnect` (or equivalent host action) that recreates the MCP client connection. | Outside this process's control; the sanctioned recovery the gates point operators toward. No in-process code initiates it. | #584, #656, #657 |
| `profile_switch_runtime` | sanctioned_narrow_recovery | Switching the active execution profile at runtime (dynamic-profile mode). | In-process and restart-free: `runtime_switching_supported` is true, so a switch rebinds capability without recreating the process. | #656, #657 |
## Guards enforced in CI
`tests/test_mcp_restart_paths.py` asserts, against the live source tree:
1. **Registry well-formedness** — every path has a valid classification, a
non-empty guard description, references, and locations; ids are unique; all
five classifications are represented.
2. **Unknown restart attempts fail closed** —
`assert_restart_attempt_registered()` raises `UnknownRestartPathError` for
any path id not in this inventory, so a novel/unnamed restart primitive
cannot slip through silently.
3. **Daemon never self-replaces** — `assert_no_daemon_self_replacement()` scans
the daemon modules for `os.execv`/`os.kill`/`os._exit`/`os.abort` calls
(comment/docstring mentions are ignored) and finds none.
4. **Legacy helper stays removed** — `assert_auto_restart_helper_absent()`
confirms `_trigger_mcp_auto_restart` has not returned.
5. **pkill stays forbidden** — a daemon `pkill` command still classifies as
contamination via `runtime_recovery_guard`.
## Residual host behaviors (outside process control)
* `/mcp reconnect` in the IDE/host — the sanctioned recovery for stale-runtime,
transport-flap (#584), and worktree-binding conditions. The daemon can only
emit guidance toward it.
* Host-level process management (the operator relaunching the daemon after a
fail-closed stop, or after resolving merge conflicts).
These are documented rather than code-guarded because the process cannot
observe or gate them from inside itself.
## Rollout
Per #657, guards are introduced flag-free as **regression assertions** (they
codify invariants that already hold) before any hard runtime block is layered
on. When the restart coordinator (#655/#656) lands, registered paths gain a
coordinator token/capability check; unregistered attempts already fail closed
today via `assert_restart_attempt_registered()`.
docs/safety-model.md
+14
View File
@@ -46,3 +46,17 @@ If shell helpers are unavailable and MCP commit cannot run, stop with a recovery
report (restart session, clear hung terminals, use MCP-native commit). See
[`llm-workflow-runbooks.md`](llm-workflow-runbooks.md) § MCP-native commit path
(#260) and agent temp artifact cleanup (#261).
## 7. Process restart governance
Restarting the MCP control-plane process is destructive to concurrent multi-role
work and is governed by a dedicated policy. Restart is a **last resort** behind
narrower recoveries (reconnect, rebind), full/host restart is reserved to
operator/admin under **controller approval + automated safety gates**, a
unilateral LLM or operator full restart with active peers is **forbidden**, and
ambiguous policy state **denies** restart. Break-glass is a separate,
incident-backed path with a mandatory audit.
See [`architecture/mcp-restart-governance.md`](architecture/mcp-restart-governance.md)
(#656) for the authorization matrix, the recovery ladder, break-glass
conditions, and the `RG-01`–`RG-08` policy IDs.
docs/webui-deployment.md
+9
View File
@@ -55,6 +55,15 @@ shipped to the browser.
assumption paths, and the client-secret policy. Use it to verify an instance is
configured for internal-only operation.
## Process restart / reload disposition
The console never exposes a restart or reload control; process restart of the
MCP control-plane runtime is governed separately. Restart is a last resort behind
reconnect/rebind, full restart is operator/admin-only under controller approval
plus safety gates, and break-glass is an incident-backed path. See
[`architecture/mcp-restart-governance.md`](architecture/mcp-restart-governance.md)
(#656).
## Non-goals (MVP)
- Full SSO or session login in the UI