jcwalker3andClaude Opus 4.8
42657b3b65
Adds the session_checkpoints section to the control-plane DB substrate
architecture doc: schema version and row identity, the column groups, the
public API surface, and the hard rules (redaction at write, reconcile
rather than blind restore, unknown live state is not a mismatch, drain
fails closed on an incomplete checkpoint, no transcript storage).
Satisfies acceptance criterion 1 of #660 ("Schema documented and
versioned") for the implementation added in 7e18dcc.
Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
145 lines
8.6 KiB
Markdown
145 lines
8.6 KiB
Markdown
# Control-plane DB substrate (#613)
|
|
|
|
**Status:** Implemented (SQLite single-writer MVP)
|
|
|
|
**ADR:** [`mcp-allocator-control-plane-observability-adr.md`](mcp-allocator-control-plane-observability-adr.md)
|
|
|
|
**Module:** `control_plane_db.py`
|
|
|
|
## Architecture statement
|
|
|
|
> **DB coordinates, Gitea records, Sentry/GlitchTip observe; the bridge is the only path that turns observations into Gitea work.**
|
|
|
|
## What this ships
|
|
|
|
| Capability | Notes |
|
|
|------------|--------|
|
|
| Schema | `sessions`, `work_items`, `leases`, `assignments`, `terminal_locks`, `events`, `incident_links`, `session_checkpoints` |
|
|
| Atomic assign+lease | `ControlPlaneDB.assign_and_lease` — one `BEGIN IMMEDIATE` transaction |
|
|
| Mutation gate | `require_valid_assignment` — live lease + allowed action + non-terminal work + non-stale head |
|
|
| Heartbeat / release / expire | Lease lifecycle helpers |
|
|
| Terminal-lock index | Routing signal for #600 (terminal path first) |
|
|
| `incident_links` | Provider-neutral link model for #612 — **not** assignable work; scope keys NULL-safe |
|
|
| `session_checkpoints` | Durable session resume state for #660 — redacted at write, reconciled (never restored) on boot |
|
|
|
|
## Hard rules (enforced in code)
|
|
|
|
1. Assignable `work_items.kind` ∈ {`issue`, `pr`} only — **never** raw Sentry/GlitchTip incidents.
|
|
2. Two concurrent sessions cannot both receive an active assignment on the same open work item (second gets `wait`).
|
|
3. Merged/closed work items return `no_safe_work` at assign time, and `require_valid_assignment` fails closed if the work item becomes terminal later.
|
|
4. Assignments pin `expected_head_sha`; mutations fail closed if the work item head drifts.
|
|
5. SQLite path is the **single-writer MVP** (`GITEA_CONTROL_PLANE_DB`, default under `~/.cache/gitea-tools/control-plane/`). Multi-session multi-host production requires **Postgres** or a **single allocator daemon** (ADR §6).
|
|
6. `incident_links` optional scope fields are stored as empty strings (never NULL) so UNIQUE is canonical across minimal upserts.
|
|
7. Legacy `incident_links` migration collapses NULL-scope duplicates **only** when Gitea targets **and** all meaningful observation metadata agree (fingerprint, status, event_count, permalink, timestamps, linked PRs, etc.). Conflicting metadata fails closed — no silent discard.
|
|
|
|
## Dependency chain
|
|
|
|
```text
|
|
#613 control-plane DB (this) → #600 allocator API → #612 incident bridge
|
|
```
|
|
|
|
- **#600** must call this substrate (not file locks / comment-only leases alone) for completion.
|
|
- **#612** must write `incident_links` here and create **Gitea issues**; the allocator assigns those issues, not raw incidents.
|
|
|
|
## Allocator API (#600)
|
|
|
|
Module: `allocator_service.py` · MCP tool: `gitea_allocate_next_work`
|
|
|
|
- Workers call `gitea_allocate_next_work(apply=false|true)` instead of self-selecting work.
|
|
- `apply=false` returns a dry-run selection (`outcome=preview`) with skip reasons.
|
|
- `apply=true` reserves via `ControlPlaneDB.assign_and_lease` (atomic assignment+lease).
|
|
- Coordination source is **always** the control-plane DB — not file locks or comment-only leases.
|
|
- Routing follows ADR §5.3 (REQUEST_CHANGES → author, terminal path first, foreign lease → wait).
|
|
- Raw monitoring incidents are never candidates.
|
|
|
|
|
|
## Lease lifecycle API (#601)
|
|
|
|
Module: `lease_lifecycle.py` · MCP tools: `gitea_*_workflow_lease(s)`
|
|
|
|
Active control-plane leases are **first-class workflow state**:
|
|
|
|
| Tool | Purpose |
|
|
|------|---------|
|
|
| `gitea_list_workflow_leases` | List active (or all) leases for a repo |
|
|
| `gitea_inspect_workflow_lease` | Freshness + `safe_next_action` for one lease id |
|
|
| `gitea_adopt_workflow_lease` | Owner-resume or sanctioned reclaim with provenance |
|
|
| `gitea_release_workflow_lease` | Explicit owner release (audited) |
|
|
| `gitea_expire_workflow_leases` | Deterministic expire of past-`expires_at` leases |
|
|
| `gitea_abandon_workflow_lease` | Abandon with required proof |
|
|
| `gitea_reclaim_expired_workflow_lease` | Expire + re-assign with provenance |
|
|
|
|
### Hard rules
|
|
|
|
1. Control-plane DB is the coordination authority — file locks / comment-only leases are not authoritative alone.
|
|
2. Active foreign leases cannot be stolen; inspect returns `wait_foreign_active`.
|
|
3. Abandon requires `(dead_process OR missing_worktree)` and `no_live_mutation_risk`; foreign abandon also needs `operator_authorized` or full dead+missing+no_open_pr proof.
|
|
4. Adopt provenance always records `adopted_from_session_id`, `adopted_by_session_id`, work identity, optional head SHA, and worktree path.
|
|
5. Reviewer→merger comment handoff (`gitea_adopt_merger_pr_lease`) is unchanged and remains the Gitea-thread durability path.
|
|
|
|
```bash
|
|
python3 -m pytest tests/test_lease_lifecycle.py tests/test_control_plane_db.py tests/test_allocator_service.py tests/test_merger_lease_adoption.py -q
|
|
```
|
|
|
|
## Incident bridge (#612)
|
|
|
|
Module: `incident_bridge.py` · MCP tools: `gitea_observability_*`
|
|
|
|
- Bridge converts Sentry/GlitchTip observations → **normal Gitea issues** + `incident_links`.
|
|
- Phase-1: `gitea_observability_reconcile_incident(apply=false|true)` with JSON observation.
|
|
- Dry-run performs **no** Gitea mutation and **no** DB write.
|
|
- Apply reuses existing links or creates one Gitea issue; never invents `work_items.kind=incident`.
|
|
- Config: `GITEA_OBSERVABILITY_PROJECTS_JSON` or `GITEA_OBSERVABILITY_PROJECTS_FILE`.
|
|
- Provider tokens never appear in issue bodies, links, or tool results.
|
|
- Allocator sees bridge work only after a Gitea issue exists.
|
|
|
|
## Session checkpoints (#660)
|
|
|
|
Module: `control_plane_db.py` · Table: `session_checkpoints` · Parent **#655** · Soft-depends **#659** · Vision **#652** · Roadmap **#653**
|
|
|
|
Workflow state that lived only in MCP process memory or chat did not survive a restart, so session identity, stage, lease ownership, and the next valid action had to be reconstructed by hand. The `session_checkpoints` table makes that state durable.
|
|
|
|
### Schema version
|
|
|
|
Rows carry `checkpoint_schema_version` (currently **5**, tracking the module-level `SCHEMA_VERSION`), so a reader can tell which field set a record was written under. The row identity is
|
|
`UNIQUE (remote, org, repo, session_id, work_kind, work_number)` — one current-state row per session per work unit, upserted rather than appended. A session-level checkpoint that is not bound to an issue or PR uses the `work_number = 0` sentinel with an empty `work_kind`.
|
|
|
|
| Group | Columns |
|
|
|-------|---------|
|
|
| Identity | `session_id`, `provider_identity`, `role`, `remote`/`org`/`repo` |
|
|
| Work unit | `work_kind` (`issue`/`pr`/empty), `work_number`, `worktree_path`, `branch`, `head_sha` |
|
|
| Ownership | `capabilities` (JSON), `lease_id`, `assignment_id` |
|
|
| Progress | `workflow_stage`, `last_completed_action`, `current_operation`, `pending_mutation` (JSON) |
|
|
| Recovery | `evidence` (JSON), `blocker`, `next_valid_action`, `recovery_instructions` |
|
|
| Bookkeeping | `checkpoint_schema_version`, `status`, `created_at`, `updated_at` |
|
|
|
|
### API
|
|
|
|
| Method | Purpose |
|
|
|--------|---------|
|
|
| `write_session_checkpoint` | Upsert the current-state row; redacts, and optionally enforces drain completeness |
|
|
| `get_session_checkpoint` | Read one checkpoint by session + work unit |
|
|
| `list_session_checkpoints` | List checkpoints by session or by work unit |
|
|
| `reconcile_session_checkpoint` | Pure diagnosis of a stored checkpoint against live head/lease state |
|
|
| `checkpoint_completeness` | Pure — the drain-required fields missing from a record |
|
|
|
|
### Hard rules
|
|
|
|
1. **Redaction at write.** Free-text columns are redaction-filtered and JSON columns are redacted recursively before storage, so a pasted token or credential URL cannot land in a checkpoint. Secrets are never stored (#660 AC4).
|
|
2. **Reconcile, never blind-restore.** `reconcile_session_checkpoint` returns a diagnosis (`stale`, `head_mismatch`, `lease_mismatch`, `reconcile_action`) and the caller decides. A stored head that no longer matches live Git, or a lease that is gone or reassigned, marks the checkpoint stale (#660 AC3).
|
|
3. **Unknown live state is not a mismatch.** A `None` live input means "not checked" and never flags staleness on its own.
|
|
4. **Drain fails closed.** A write with `require_complete=True` refuses when any of `session_id`, `role`, `workflow_stage`, `next_valid_action`, `recovery_instructions` is missing or blank, so drain cannot complete on a checkpoint that could not resume.
|
|
5. **No transcript storage.** Checkpoints hold resume state, not conversation history.
|
|
|
|
## Non-goals (intentionally deferred)
|
|
|
|
- Full unsupervised watchdog auto-filing (prefer explicit reconcile first)
|
|
- Assuming GlitchTip writeback API equals Sentry without verification
|
|
- Gitea comment/label mirror writers for every assignment (optional later)
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
python3 -m pytest tests/test_control_plane_db.py -q
|
|
```
|