Gitea-Tools/docs/stable-runtime-promotion-runbook.md

# Stable control runtime — promotion runbook (#615)

Operator / release-manager procedure for promoting a revision into the **stable
control runtime**: the Gitea MCP server that performs real issue/PR mutations.

Policy source: [`architecture/mcp-stable-control-runtime-policy-adr.md`](architecture/mcp-stable-control-runtime-policy-adr.md).
Enforcement: `stable_control_runtime.py` (runtime mode classification, mutation
gates, per-namespace post-flap re-proving, promotion-record validation).

**Promotion is operator-owned.** Normal author / reviewer / merger / reconciler
sessions must never kill, restart, or relaunch the MCP server, and must never
edit the stable runtime checkout. A session that needs newer server code stops
with `BLOCKED + DIAGNOSE` and hands off to the operator.

---

## 1. When a promotion is required

- A merged PR changes MCP server code the control plane must now enforce.
- `gitea_assess_master_parity` reports `stale: true` / `restart_required: true`.
- `gitea_get_runtime_context` reports a `runtime_mode` other than
  `stable-control`, or `real_mutations_allowed: false`.

## 2. Pre-promotion checks

Run these **before** advancing the stable checkout:

1. The target revision is on remote `master` and was merged through
   `gitea_merge_pr` (never a direct stable-branch push — see #671).
2. The stable control checkout is clean (`git status --porcelain` empty) and on
   `master`. A dirty stable runtime is itself a mutation blocker.
3. The advance is strictly fast-forwardable: local `master` is an ancestor of
   `prgs/master`.
4. No active workflow lease is mid-mutation (`gitea_list_workflow_leases`).

## 3. Promotion steps

1. Record the **previous** runtime SHA (`gitea_assess_master_parity` →
   `startup_head`).
2. `git fetch --prune prgs` in the stable control checkout.
3. `git merge --ff-only prgs/master` — never rebase, reset, or force.
4. Record the **promoted** runtime SHA (`git rev-parse HEAD`).
5. Reload the runtime using the sanctioned client path (IDE/client reconnect or
   the operator's supervised service reload). Never `pkill` the daemon from a
   workflow session.
6. Re-prove **each** namespace independently (see §5).
7. Record the promotion (see §4) and post it as a durable comment on the
   tracking issue.

## 4. Promotion record (required fields)

Every promotion must record all of the following. `assess_promotion_record()`
validates them and fails closed on any missing field, or when
`previous_runtime_sha` equals `promoted_runtime_sha` (nothing was promoted).

| Field | Meaning |
|-------|---------|
| `previous_runtime_sha` | SHA the stable runtime was serving before promotion |
| `promoted_runtime_sha` | SHA the stable runtime serves after promotion |
| `source_branch` | Branch the promoted revision came from |
| `source_pr` | PR number that merged it |
| `restart_method` | Exact reload/restart mechanism the operator used |
| `health_check_proof` | `gitea_assess_mcp_namespace_health` result per namespace |
| `identity_proof` | `gitea_whoami` username + profile per namespace |
| `profile_proof` | `gitea_get_runtime_context` active profile per namespace |
| `workspace_proof` | Process root, canonical root, alignment, clean state |
| `mutation_capability_proof` | `gitea_resolve_task_capability` for the intended task |
| `rollback_instructions` | Exact steps to return to `previous_runtime_sha` |

Helper: `scripts/promote-stable-runtime` emits and validates the record. It
never restarts anything — it reads state and prints the record for the operator
to act on and archive.

## 5. Post-promotion namespace re-proving

A restart or transport flap drops every `gitea-*` namespace together. Author
proof is **not** global proof. For each of `author`, `reviewer`, `merger`,
`reconciler`, in that namespace:

1. `gitea_whoami`
2. `gitea_get_runtime_context`
3. `gitea_resolve_task_capability` immediately before the intended mutation
4. Mutate only when no reconnect / restart / stale-runtime gate is reported

Until a namespace passes all four, its mutations stay blocked with
`namespace_not_reproven_after_flap`.

## 6. Rollback

If the promoted runtime is unhealthy — namespace EOF that does not recover,
identity or profile mismatch, capability resolution failure, or an unexpected
`runtime_mode`:

1. **Stop all PR/review/merge work.** An unhealthy stable runtime fails closed;
   do not route around it.
2. Fast-forward or check out `previous_runtime_sha` in the stable checkout.
3. Reload the runtime by the same sanctioned method.
4. Re-prove every namespace (§5).
5. Record the rollback as a promotion record whose `promoted_runtime_sha` is the
   restored SHA, with the failure evidence in `health_check_proof`.

## 7. Runtime modes seen in reports

| Mode | Meaning | Real mutations |
|------|---------|----------------|
| `stable-control` | Promoted revision, stable branch, clean checkout | Allowed |
| `dev-test` | Launched from a `branches/` worktree or a feature branch | Blocked against production |
| `unknown` | Root unresolvable, not a git checkout, or detached HEAD with no declaration | Blocked |

A packaged deployment with no git checkout must declare itself explicitly with
`GITEA_MCP_RUNTIME_MODE=stable-control`; an unset or misspelled value falls back
to inference and, failing that, to `unknown`.

## 8. Related

- `architecture/mcp-stable-control-runtime-policy-adr.md` — the policy (#615)
- `mcp-namespace-health.md` — client-namespace health (#543)
- `mcp-namespace-eof-recovery.md` — reconnect-only EOF recovery
- `mcp-daemon-import-guard.md` — sanctioned daemon only (#558)
- `bootstrap-review-path.md` — controller bootstrap when the live runtime cannot
  review its own fix (#557)