4.1 KiB
4.1 KiB
Sanctioned Recovery Playbooks & Controls (Phase 2 #644)
Overview
Stale runtimes, worktree binding mismatches, and un-reconciled merged branches previously required expert manual shell recovery. Manual process kills (pkill -f mcp_server.py) are strictly forbidden and classified as runtime contamination (#630).
Phase 2 introduces sanctioned recovery playbooks and controls into the Web Console:
- Diagnose: Surface stale runtimes, worktree binding errors, contamination markers, and worktree anomalies via health & inventory APIs.
- Preview: Render mutation ledgers and exact confirmation phrases for recovery playbooks.
- Confirm & Apply: Execute sanctioned recovery actions through gated, audited paths.
- Verify: Revalidate control-plane state post-recovery before claiming clean status.
Recovery Playbook Taxonomy
| Playbook ID | Action ID | Minimum Role | Target / Scope | Description |
|---|---|---|---|---|
clear_stale_binding |
system.clear_stale_binding |
Operator | Active worktree binding | Clear provably missing or superseded GITEA_ACTIVE_WORKTREE binding (#702). |
rebind_session_worktree |
system.rebind_session_worktree |
Operator | Session worktree | Rebind or synchronize session worktree to verified lease worktree (#864). |
reconcile_cleanups |
system.reconcile_cleanups |
Controller | Worktree hygiene | Execute reconciler cleanup preview and apply for merged/superseded PR branches. |
sanctioned_restart |
system.restart_namespace |
Admin | MCP Namespace | Restart MCP daemon gracefully via host supervisor (#642). |
Wizard Workflow (Diagnose → Preview → Confirm → Verify)
1. Diagnose (GET /api/v1/system/recovery/diagnose)
Runs control-plane diagnostics:
- Stale Runtime: Mismatch between running daemon HEAD, local checkout HEAD, and remote-tracking HEAD.
- Worktree Binding: Missing path (
provably_stale_missing_path), unverified inherited binding (unverified_inherited), or superseded binding (superseded_by_session_lease). - Contamination: Checks for live contamination markers from unmanaged process kills.
- Worktree Anomalies: Scans
branches/directory for un-reconciled cleanups or missing preserved worktrees.
Returns RecoveryDiagnosis with eligible playbooks.
2. Preview (POST /api/v1/system/recovery/preview)
Takes playbook_id and optional target/params.
Returns:
- Mutation Ledger: Step-by-step sequence of actions.
- Confirmation Phrase: Exact phrase required to authorize execution (e.g.,
confirm clear_stale_binding). - Authorization Decision: RBAC check against the operator's principal.
3. Apply (POST /api/v1/system/recovery/apply)
Requires playbook_id and matching confirmation phrase.
- Validates RBAC permissions (
console_authz). - Verifies confirmation phrase (
confirmation_matches). - Enforces contamination rules (#630): A contaminated runtime must be cleared through reconciler cleanup before other playbooks run.
- Enforces master parity (#610).
- Applies sanctioned recovery logic.
- Logs audit record in
console_audit.
4. Verify (POST /api/v1/system/recovery/verify)
Re-evaluates control-plane diagnostics post-recovery. Asserts clean: true before transitioning out of recovery mode.
Safety & Governance Principles
- No Manual
pkill: Direct process killing remains forbidden and is recorded as contamination. - Auditability: Every recovery preview and execution is logged in the console audit trail.
- Master Parity & Dual Control: High-privilege recovery actions require controller/admin roles and explicit confirmation phrases.