Compare commits

...
Author SHA1 Message Date
sysadmin 2429d9d2e8 fix: fail closed on conflicting incident_links observation metadata
Legacy NULL-scope dedupe only compared Gitea targets, so duplicate rows
with the same provider key and issue but different fingerprint/status/
event_count/etc. collapsed to the lowest link_id and silently dropped
observation data. Migration now compares all meaningful observation
fields and refuses to discard conflicts (#619 RC3 / #613).
2026-07-09 23:04:17 -04:00
sysadmin 16cd871fbd fix: safe incident_links migration order; require PR head pin
Address remaining PR #619 blockers on head 783ea88:

- Deduplicate legacy NULL-scope incident_links before normalizing to ''
- Fail closed when duplicate rows disagree on Gitea target
- Require non-empty expected_head_sha for PR assign and mutation
2026-07-09 22:51:27 -04:00
sysadmin 783ea88a01 fix: fail closed on stale/terminal assignments; canonicalize incident_links
Address REQUEST_CHANGES on PR #619:

- require_valid_assignment rejects terminal work states and head drift
- incident_links scope keys normalize NULL/blank to '' for UNIQUE
- remove trailing whitespace in control-plane-db-substrate.md
2026-07-09 22:41:25 -04:00
sysadmin 036b78e31e feat: control-plane DB substrate for atomic assign/lease (Closes #613)
Add SQLite single-writer MVP control plane per the allocator ADR:
sessions, work_items (issue/pr only), atomic assign+lease, mutation
gates, terminal-lock index, and provider-neutral incident_links.

DB coordinates concurrency; Gitea remains assignable work; raw
Sentry/GlitchTip incidents are never work items. #600 and #612 build on
this substrate.
2026-07-09 22:27:51 -04:00
sysadmin 08c3488d7c Merge pull request 'docs: ADR for MCP allocator, control-plane DB, and incident bridge (#613)' (#614) from docs/issue-613-allocator-control-plane-observability-adr into master 2026-07-09 21:10:27 -05:00
sysadmin 4f2fd9a8b1 fix(mcp): resolve default remote mapping and connection check bugs 2026-07-09 20:26:58 -04:00
sysadmin 4185ee1417 docs: ADR for MCP allocator, control-plane DB, and incident bridge
Canonical architecture for #613#600#612: DB coordinates, Gitea
records, Sentry/GlitchTip observe; allocator assigns only Gitea issues/PRs.

Refs: #600 #612 #613
2026-07-09 20:00:59 -04:00
7 changed files with 2353 additions and 3 deletions
+1151
View File
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,53 @@
# Control-plane DB substrate (#613)
**Status:** Implemented (SQLite single-writer MVP)
**ADR:** [`mcp-allocator-control-plane-observability-adr.md`](mcp-allocator-control-plane-observability-adr.md)
**Module:** `control_plane_db.py`
## Architecture statement
> **DB coordinates, Gitea records, Sentry/GlitchTip observe; the bridge is the only path that turns observations into Gitea work.**
## What this ships
| Capability | Notes |
|------------|--------|
| Schema | `sessions`, `work_items`, `leases`, `assignments`, `terminal_locks`, `events`, `incident_links` |
| Atomic assign+lease | `ControlPlaneDB.assign_and_lease` — one `BEGIN IMMEDIATE` transaction |
| Mutation gate | `require_valid_assignment` — live lease + allowed action + non-terminal work + non-stale head |
| Heartbeat / release / expire | Lease lifecycle helpers |
| Terminal-lock index | Routing signal for #600 (terminal path first) |
| `incident_links` | Provider-neutral link model for #612**not** assignable work; scope keys NULL-safe |
## Hard rules (enforced in code)
1. Assignable `work_items.kind` ∈ {`issue`, `pr`} only — **never** raw Sentry/GlitchTip incidents.
2. Two concurrent sessions cannot both receive an active assignment on the same open work item (second gets `wait`).
3. Merged/closed work items return `no_safe_work` at assign time, and `require_valid_assignment` fails closed if the work item becomes terminal later.
4. Assignments pin `expected_head_sha`; mutations fail closed if the work item head drifts.
5. SQLite path is the **single-writer MVP** (`GITEA_CONTROL_PLANE_DB`, default under `~/.cache/gitea-tools/control-plane/`). Multi-session multi-host production requires **Postgres** or a **single allocator daemon** (ADR §6).
6. `incident_links` optional scope fields are stored as empty strings (never NULL) so UNIQUE is canonical across minimal upserts.
7. Legacy `incident_links` migration collapses NULL-scope duplicates **only** when Gitea targets **and** all meaningful observation metadata agree (fingerprint, status, event_count, permalink, timestamps, linked PRs, etc.). Conflicting metadata fails closed — no silent discard.
## Dependency chain
```text
#613 control-plane DB (this) → #600 allocator API → #612 incident bridge
```
- **#600** must call this substrate (not file locks / comment-only leases alone) for completion.
- **#612** must write `incident_links` here and create **Gitea issues**; the allocator assigns those issues, not raw incidents.
## Non-goals (intentionally deferred)
- Full `gitea_allocate_next_work` routing policy (REQUEST_CHANGES → author, etc.) — **#600**
- Sentry/GlitchTip provider adapters and auto-watchdog — **#612**
- Gitea comment/label mirror writers (may be added by allocator tooling later)
## Tests
```bash
python3 -m pytest tests/test_control_plane_db.py -q
```
@@ -0,0 +1,301 @@
# ADR: MCP allocator, control-plane DB, and Sentry/GlitchTip incident bridge architecture
- **Status:** Accepted (implementation pending; blocks code for #600 / #612 / #613)
- **Date:** 2026-07-09
- **Tracking issues:**
- [#613](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/613) — control-plane DB (first)
- [#600](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/600) — allocator API (second)
- [#612](https://gitea.prgs.cc/Scaled-Tech-Consulting/Gitea-Tools/issues/612) — Sentry/GlitchTip incident bridge (third)
- **Related:** existing GlitchTip contracts (`glitchtip-to-gitea-workflow-design.md`, `glitchtip-gitea-deduplication-linking-design.md`), trust boundaries (`tool-boundaries.md`, `safety-model.md`), current leases (`pr_work_lease.py`, `issue_lock_store.py`)
## 1. Context
Multiple LLM sessions can independently inspect the Gitea queue and start the same issue or PR. Existing coordination (Gitea comment leases, local issue-lock files, labels) often detects collisions **after** work has started. Parallel issues #600, #612, and #613 each describe part of a fix; without one ADR they risk overlapping stores, wrong dependency order, and trust-boundary violations.
This ADR is the canonical architecture decision **before** implementing those issues.
## 2. Decision summary (core)
| Layer | Owns | Must not |
|-------|------|----------|
| **Control-plane DB** | Sessions, atomic assignment, leases, heartbeats, terminal-lock index, events, incident links | Bypass Gitea workflow gates or replace issue/PR history |
| **Gitea** | Durable work record: issues, PRs, comments, labels, reviews, merges | Be the only concurrency lock under multi-session load |
| **Sentry / GlitchTip** | Incidents, events, provider UI | Assign work, approve/merge/close, or mutate Gitea outside the bridge |
| **Incident bridge** | Provider adapters, reconcile/create Gitea issues, link storage upsert, optional provider writeback | Hand raw provider incidents to the allocator as work items |
**One-liner:** **DB coordinates. Gitea records. Sentry/GlitchTip observe. The bridge is the only path that turns observations into Gitea work. The allocator assigns only Gitea issues/PRs, never raw monitoring incidents.**
## 3. Dependency order
Implementation **must** follow:
```text
#613 control-plane DB → #600 allocator API → #612 incident bridge
(atomic substrate) (routing policy) (feeds Gitea work)
```
| Issue | Role | Depends on |
|-------|------|------------|
| **#613** | Durable coordination DB + lease/assignment transactions | — |
| **#600** | `gitea_allocate_next_work` policy and tool surface | **#613** (hard) |
| **#612** | Multi-project Sentry/GlitchTip → Gitea bridge | **#613** for `incident_links` index; **#600** before treating bridge-created issues as allocator feed in multi-worker prod |
**Hard rules:**
1. Do **not** implement #600 on file locks / comment-only leases and call it done.
2. Do **not** ship #612 as a second assignment system for raw incidents.
3. Partial previews (read-only list tools, schema stubs) may land earlier if they do not claim “allocator complete” or “bridge complete.”
## 4. Authority boundaries
### 4.1 Gitea (durable source of truth for work)
Gitea remains authoritative for:
- Issue and PR identity, titles, bodies, state (open/closed/merged)
- Comments, reviews, approvals, REQUEST_CHANGES
- Labels and workflow status labels (`status:ready`, `status:in-progress`, …)
- Merges, closes, branch refs as recorded by Gitea/git hosting
Operators and auditors read **history** from Gitea.
### 4.2 Control-plane DB (coordination / index)
The control-plane DB is authoritative for **live multi-session coordination**:
- Which session holds which assignment/lease
- Heartbeat freshness and expiry
- Terminal-lock index for routing
- Event log of allocation/lease transitions
- `incident_links` index (see §8)
It is **not** a substitute for Gitea history. When DB and Gitea disagree on durable work state (e.g. PR already merged), **Gitea wins**; the DB is reconciled.
### 4.3 Sentry / GlitchTip (observe)
Providers own incident lifecycle and raw event data. They:
- Must **not** bypass Gitea gates
- Must **not** assign LLM work
- May receive **writeback** of resolution status only through the bridge, when configured and supported
### 4.4 Trust boundaries for credentials
Aligned with `docs/tool-boundaries.md` and `docs/safety-model.md`:
- **One MCP server process per trust boundary** remains the default.
- Monitor API tokens (Sentry/GlitchTip) live in the **bridge/orchestrator boundary** (or a dedicated observability write/read profile), **not** blindly inside every Gitea MCP author/reviewer/merger process.
- Gitea tokens stay on Gitea MCP profiles only.
- Orchestrators compose services; they must not become a single credential pool that silently mixes Gitea write + monitor tokens into every worker.
Tool names such as `gitea_observability_*` may exist as a **namespace façade** only if the runtime still enforces separate credential scopes (e.g. bridge process vs pure Gitea mutation process).
## 5. Allocator rules (#600 on top of #613)
### 5.1 Worker contract
- Workers **do not self-select** work under the standard multi-LLM workflow.
- Workers call **`gitea_allocate_next_work`** (with `apply=true` only when taking work).
- Mutations that claim exclusive work require a **valid assignment and lease** from the control-plane DB.
- Return values include at least: role, issue/PR number, expected head SHA (when applicable), lease ID, expiry, allowed actions, forbidden actions — or a non-assignment outcome (`WAIT`, terminal-path block, no safe work, needs controller, etc.).
### 5.2 Atomic reserve
- **Assignment creation and lease creation happen in one DB transaction.**
- Two concurrent sessions **must not** receive the same issue/PR as assigned work.
- On contention: second session gets **WAIT** or **owner-resume**, never a duplicate lease.
### 5.3 Routing policy
| Condition | Route |
|-----------|--------|
| Current-head **REQUEST_CHANGES** | **Author** (not reviewer/merger) |
| **Stale** approval (approval not on current head) | **Reviewer** |
| **Clean** approval on current head, mergeable | **Merger** |
| Contested / contaminated approval | **Diagnosis / reconciler** (controller path) |
| Active **foreign** lease | **WAIT** or **owner-resume** |
| **Terminal-review** lock present | **Terminal-path resolution first** before downstream review work |
| Merged / closed PR | **Never assign** |
| Issue locked by sanctioned lease | **Never assign** to another worker unless lease cleanup/adoption is sanctioned |
### 5.4 Relationship to Gitea mirrors
After a successful assignment, the allocator (or sanctioned tooling) may update Gitea comments/labels so humans and audits see state. Those mirrors **are not** the sole lock source.
## 6. Control-plane DB topology (#613)
### 6.1 Preferred architecture (multi-daemon / Proxmox)
For true multi-session concurrency (multiple MCP daemons, multiple hosts, or Proxmox VMs):
**Prefer either:**
1. **Shared Postgres** used by all Gitea MCP profiles / allocator callers, **or**
2. A **single allocator daemon** (sole writer to the coordination store) that all workers call over MCP/RPC.
Both satisfy: one transactional authority for “who has this work item.”
### 6.2 SQLite MVP
SQLite is acceptable **only** for a **single-writer MVP**:
- One process owns writes (allocator daemon or single co-located MCP server), **or**
- Documented single-host single-daemon experiments with file locking and no multi-host claims.
SQLite is **not** sufficient to declare multi-session production readiness when four independent LLM sessions talk to four MCP processes without a shared writer.
### 6.3 Suggested entities (normative shape)
Minimum logical entities (names may vary; semantics must not):
1. **sessions** — session_id, role/profile, namespace, pid, started_at, last_heartbeat_at, status
2. **work_items** — remote/org/repo, kind ∈ {`issue`, `pr`}, number, state, priority, current_head_sha, updated_at
3. **leases** — lease_id, work_item_id, session_id, role, phase, expires_at, heartbeat_at, status
4. **assignments** — assignment_id, work_item_id, session_id, allowed_action(s), expected_head_sha, status
5. **terminal_locks** — repo/org, terminal_pr, review_id, decision, status, cleanup_state
6. **events** — event_id, work_item_id, type, message, created_at
7. **incident_links** — provider-neutral link model (see §8)
**Explicit non-kind:** do **not** use `work_items.kind = sentry_incident` (or glitchtip_incident) as an assignable work unit. Incidents become work only after the bridge creates/links a **Gitea issue**.
## 7. Lease migration (avoid split-brain)
### 7.1 Target state
- **Primary** for live coordination: **control-plane DB** leases and assignments.
- **Gitea comment leases** (e.g. `<!-- mcp-review-lease:v1 -->`) and **local issue-lock files** become:
- **mirrors** of DB state for human visibility / recovery, and/or
- written **only** through the allocator (or sanctioned lease tools that update DB + mirror in one workflow).
### 7.2 Migration rules
1. New exclusive claims go through DB assignment+lease first.
2. Comment lease / file lock writers that skip the DB are **deprecated** once #613+#600 land.
3. Reconciler tooling heals: expired DB lease, orphan Gitea comment, orphan local file — fail closed when ambiguous.
4. **Split-brain is a defect:** “DB free + Gitea comment leased” or “DB leased + Gitea free” must be detectable and reconcilable; production paths must not rely on two independent writers.
### 7.3 Heartbeats
- Heartbeats update the **DB**.
- Optional Gitea summaries must avoid comment spam (periodic summary or edit-in-place policy).
## 8. Observability bridge (#612)
### 8.1 Role
The bridge:
1. Scans configured Sentry and/or GlitchTip projects (multi-project mappings).
2. Deduplicates against existing links / Gitea issues.
3. Creates or updates **normal Gitea issues** (labels, sanitized body, provider URL).
4. Upserts **one** canonical `incident_links` row.
5. Optionally writes resolution status back to the provider when supported.
6. Never approves, merges, closes, or otherwise bypasses Gitea workflow gates.
### 8.2 Allocator interaction
- The allocator sees observability work **only after** a Gitea issue exists (typically `status:ready` + observability labels).
- **Do not assign raw Sentry/GlitchTip incidents** as work items.
- Bridge AC “allocator can see linked issues” means: **as ordinary Gitea issues**, not a special incident queue.
### 8.3 Provider adapters and trust
- Separate **Sentry** and **GlitchTip** adapters; document API differences; do not assume writeback parity.
- Self-hosted base URLs supported (e.g. `https://sentry.prgs.cc`).
- Tokens from environment / secret store only.
- Prefer phase-1 **explicit reconcile / dry-run** before unsupervised watchdog auto-filing, consistent with existing GlitchTip filing safety contracts.
### 8.4 Home of implementation
Filing/orchestration may live in:
- a control-plane / bridge package or profile, and/or
- Gitea-Tools as operator-facing tools that **delegate** to that boundary,
but must not violate §4.4 credential isolation.
## 9. Incident link storage (single source of truth)
### 9.1 Canonical model: `incident_links` (provider-neutral)
Prefer a **provider-neutral** model over Sentry-only `sentry_links`:
| Field (logical) | Purpose |
|-----------------|---------|
| provider | `sentry` \| `glitchtip` \| … |
| provider_base_url | Self-hosted base |
| provider_org / provider_project | Mapping key |
| provider_issue_id / provider_short_id | Provider identity |
| provider_permalink | Human URL |
| fingerprint | Stable dedupe key when available |
| gitea_org / gitea_repo / gitea_issue_number | Linked work |
| linked_pr_numbers | Optional PR association |
| first_seen / last_seen / event_count | Summary metrics (safe) |
| status | link lifecycle |
| release_resolved_at / last_sync_at | Sync bookkeeping |
### 9.2 Gitea as human mirror
- Issue body markers / structured comments (e.g. HTML comment metadata) remain the **human- and audit-visible mirror**.
- They must stay consistent with `incident_links` but are **not** a second independent write path for inventing links without the bridge.
### 9.3 One truth
- **Do not** maintain two independent systems of record (e.g. full mapping only in #612 storage **and** a separate #613 `sentry_links` with different semantics).
- #612 implementation **must** use the same `incident_links` model introduced under #613 (or a single agreed store both reference).
## 10. Security and redaction
Mandatory for bridge payloads, Gitea issue bodies/comments, DB-stored event summaries, and logs:
- No API tokens, DSNs, passwords, cookies, auth headers
- No keychain IDs, private config contents, raw session-state files, full prompt bodies
- Sanitize stack locals and request data; store only safe tags/context
- Logs must never print provider tokens or DSNs
- Redaction tests are required for #612
## 11. Non-goals
- Replace Gitea as the durable workflow record
- Let Sentry/GlitchTip assign work or mutate Gitea workflow state outside sanctioned tools
- Let the bridge approve, merge, close, release, or bypass Gitea gates
- Implement #600 on file locks / comments alone and call allocator complete
- Treat raw monitoring incidents as `work_items` for the allocator
- Put monitor tokens into every Gitea MCP worker process by default
## 12. Consequences
### Positive
- Clear implementation order and ownership
- Multi-session safety becomes testable at the DB layer
- Observability becomes assignable work without special-casing the allocator
- Trust boundaries stay compatible with existing control-plane docs
### Costs / risks
- Migration from comment/file leases requires reconciler work
- Shared Postgres or single allocator daemon is operational cost
- Bridge must be careful not to spam Gitea issues (dedupe, caps, policy modes)
### Follow-ups (documentation only until implemented)
1. Update #600, #612, and #613 bodies to **link this ADR** and restate hard dependencies.
2. Implement #613 schema + atomic assign/lease.
3. Implement #600 against the DB.
4. Implement #612 against `incident_links` + Gitea create/update.
5. Deprecate dual writers for leases.
## 13. Acceptance criteria for *this* ADR
This document is accepted when:
1. It is merged into `docs/architecture/` on the default branch.
2. #600 / #612 / #613 (or their PRs) reference this path as the architecture source of truth.
3. No implementation PR for those issues claims completion without conforming to §§211.
## 14. Document history
| Date | Change |
|------|--------|
| 2026-07-09 | Initial ADR: dependency order, authority model, topology, lease migration, bridge, `incident_links`, security, non-goals |
+21 -2
View File
@@ -1367,8 +1367,26 @@ def cleanup_in_progress_for_pr(pr_payload: dict, remote: str, host: str | None,
# ── Helpers ───────────────────────────────────────────────────────────────────
def _effective_remote(remote: str) -> str:
"""If remote is the default ('dadeschools') but the active profile base_url maps to a known remote, use that remote instead."""
try:
profile = get_profile()
base_url = profile.get("base_url")
if remote == "dadeschools" and base_url:
import urllib.parse
url = urllib.parse.urlparse(base_url)
host = (url.netloc or url.path or "").strip().lower()
for k, v in REMOTES.items():
if v.get("host") == host:
return k
except Exception:
pass
return remote
def _resolve(remote: str, host: str | None, org: str | None, repo: str | None):
"""Resolve remote + overrides to (host, org, repo)."""
remote = _effective_remote(remote)
if remote not in REMOTES:
raise ValueError(f"Unknown remote '{remote}'. Choose from: {list(REMOTES)}")
profile = REMOTES[remote]
@@ -1402,6 +1420,7 @@ def _resolve_control_plane_guide_target(
if org is not None and repo is not None:
return _resolve(remote, host, org, repo)
remote = _effective_remote(remote)
if remote not in REMOTES:
raise ValueError(f"Unknown remote '{remote}'. Choose from: {list(REMOTES)}")
@@ -8750,6 +8769,7 @@ def gitea_whoami(
metadata: profile_name + allowed_operations; never the token).
"""
record_preflight_check("whoami")
remote = _effective_remote(remote)
if remote not in REMOTES:
raise ValueError(f"Unknown remote '{remote}'. Choose from: {list(REMOTES)}")
h = host or REMOTES[remote]["host"]
@@ -8858,8 +8878,6 @@ def gitea_get_profile(
"execution_profile": profile.get("execution_profile"),
"auth_source_type": profile.get("auth_source_type"),
# Auth is reported as a status only (#120): the token source *name*
# (env var name / keychain id) joins endpoint URLs behind the
# GITEA_MCP_REVEAL_ENDPOINTS admin opt-in. Token values never appear.
"auth_status": ("configured" if profile["token_source_name"]
else "unconfigured"),
"remote": remote if remote in REMOTES else None,
@@ -8871,6 +8889,7 @@ def gitea_get_profile(
result["base_url"] = profile["base_url"]
result["server"] = None
remote = _effective_remote(remote)
if remote not in REMOTES:
# Mark ambiguity rather than raising: the tool stays inspectable.
result["identity_status"] = "unknown"
+3
View File
@@ -6,6 +6,9 @@ Runs over stdio. All tools authenticate via macOS keychain (git credential fill)
import os
import sys
sys.stderr = open("/tmp/mcp_server_stderr.log", "a", buffering=1)
sys.stderr.write(f"\n--- MCP SERVER STARTUP (PID {os.getpid()}) ---\n")
from role_session_router import (
python_bytes_have_conflict_markers,
skip_python_scan_walk_root,
-1
View File
@@ -275,7 +275,6 @@ def main():
servers[name],
required_tool=REQUIRED_NAMESPACE_TOOLS.get(name),
config_path=config_path,
probe_source="offline_spawn",
):
failed = True
else:
+824
View File
@@ -0,0 +1,824 @@
"""Tests for control-plane DB substrate (#613)."""
from __future__ import annotations
import os
import tempfile
import threading
import unittest
from concurrent.futures import ThreadPoolExecutor, as_completed
from datetime import timedelta
from control_plane_db import (
ControlPlaneDB,
InvalidWorkKindError,
LeaseRequiredError,
WORK_KINDS,
_ts,
_utc_now,
)
class ControlPlaneDBTest(unittest.TestCase):
def setUp(self) -> None:
self._tmp = tempfile.TemporaryDirectory()
self.db_path = os.path.join(self._tmp.name, "cp.sqlite3")
self.db = ControlPlaneDB(self.db_path)
def tearDown(self) -> None:
self._tmp.cleanup()
def test_schema_and_architecture_meta(self) -> None:
import sqlite3
conn = sqlite3.connect(self.db_path)
try:
rows = dict(conn.execute("SELECT key, value FROM schema_meta").fetchall())
finally:
conn.close()
self.assertEqual(rows["schema_version"], "2")
self.assertIn("DB coordinates", rows["architecture"])
self.assertIn("bridge", rows["architecture"].lower())
def test_rejects_raw_incident_as_work_kind(self) -> None:
with self.assertRaises(InvalidWorkKindError):
self.db.upsert_work_item(
remote="prgs",
org="org",
repo="repo",
kind="sentry_incident",
number=1,
)
with self.assertRaises(InvalidWorkKindError):
self.db.assign_and_lease(
session_id="s1",
role="author",
remote="prgs",
org="org",
repo="repo",
kind="glitchtip_incident",
number=9,
)
self.assertEqual(WORK_KINDS, frozenset({"issue", "pr"}))
def test_atomic_assign_and_lease_fields(self) -> None:
self.db.upsert_session(session_id="s-a", role="author", profile="prgs-author")
result = self.db.assign_and_lease(
session_id="s-a",
role="author",
remote="prgs",
org="Scaled-Tech-Consulting",
repo="Gitea-Tools",
kind="issue",
number=613,
expected_head_sha="abc123",
allowed_actions=("implement", "comment"),
forbidden_actions=("approve", "merge"),
)
self.assertEqual(result.outcome, "assigned")
self.assertIsNotNone(result.assignment_id)
self.assertIsNotNone(result.lease_id)
self.assertEqual(result.role, "author")
self.assertEqual(result.work_kind, "issue")
self.assertEqual(result.work_number, 613)
self.assertEqual(result.expected_head_sha, "abc123")
self.assertIn("implement", result.allowed_actions)
self.assertIn("merge", result.forbidden_actions)
self.assertIsNotNone(result.expires_at)
def test_second_session_waits_on_foreign_lease(self) -> None:
self.db.upsert_session(session_id="s1", role="author")
self.db.upsert_session(session_id="s2", role="author")
first = self.db.assign_and_lease(
session_id="s1",
role="author",
remote="prgs",
org="o",
repo="r",
kind="pr",
number=100,
expected_head_sha="deadbeef",
)
self.assertEqual(first.outcome, "assigned")
second = self.db.assign_and_lease(
session_id="s2",
role="author",
remote="prgs",
org="o",
repo="r",
kind="pr",
number=100,
expected_head_sha="deadbeef",
)
self.assertEqual(second.outcome, "wait")
self.assertEqual(second.owner_session_id, "s1")
def test_owner_resume_refreshes_lease(self) -> None:
self.db.upsert_session(session_id="s1", role="reviewer")
a = self.db.assign_and_lease(
session_id="s1",
role="reviewer",
remote="prgs",
org="o",
repo="r",
kind="pr",
number=50,
expected_head_sha="head-50",
)
b = self.db.assign_and_lease(
session_id="s1",
role="reviewer",
remote="prgs",
org="o",
repo="r",
kind="pr",
number=50,
expected_head_sha="head-50",
)
self.assertEqual(b.outcome, "assigned")
self.assertEqual(b.lease_id, a.lease_id)
self.assertIn("owner-resume", b.reason)
def test_require_valid_assignment_gates_mutations(self) -> None:
self.db.upsert_session(session_id="s1", role="author")
self.db.assign_and_lease(
session_id="s1",
role="author",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=7,
allowed_actions=("implement",),
forbidden_actions=("merge",),
)
proof = self.db.require_valid_assignment(
session_id="s1",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=7,
action="implement",
)
self.assertEqual(proof["session_id"], "s1")
with self.assertRaises(LeaseRequiredError):
self.db.require_valid_assignment(
session_id="s1",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=7,
action="merge",
)
with self.assertRaises(LeaseRequiredError):
self.db.require_valid_assignment(
session_id="s-other",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=7,
action="implement",
)
def test_expired_lease_allows_reassign(self) -> None:
self.db.upsert_session(session_id="s1", role="author")
self.db.upsert_session(session_id="s2", role="author")
past = _utc_now() - timedelta(hours=1)
# Create lease already expired by using negative TTL edge via direct assign then expire
assigned = self.db.assign_and_lease(
session_id="s1",
role="author",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=3,
lease_ttl_seconds=1,
)
self.assertEqual(assigned.outcome, "assigned")
# Force expiry in DB
import sqlite3
conn = sqlite3.connect(self.db_path)
try:
conn.execute(
"UPDATE leases SET expires_at = ? WHERE lease_id = ?",
(_ts(past), assigned.lease_id),
)
conn.commit()
finally:
conn.close()
n = self.db.expire_stale_leases()
self.assertGreaterEqual(n, 1)
second = self.db.assign_and_lease(
session_id="s2",
role="author",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=3,
)
self.assertEqual(second.outcome, "assigned")
self.assertEqual(second.session_id, "s2")
def test_merged_work_never_assigned(self) -> None:
self.db.upsert_session(session_id="s1", role="merger")
self.db.upsert_work_item(
remote="prgs",
org="o",
repo="r",
kind="pr",
number=99,
state="merged",
)
result = self.db.assign_and_lease(
session_id="s1",
role="merger",
remote="prgs",
org="o",
repo="r",
kind="pr",
number=99,
expected_head_sha="merged-head",
)
self.assertEqual(result.outcome, "no_safe_work")
def test_four_concurrent_sessions_unique_assignments(self) -> None:
"""Four concurrent assigners on four different issues — all succeed uniquely.
Also two concurrent assigners on the *same* issue: at most one assigned.
"""
for i in range(4):
self.db.upsert_session(session_id=f"sess-{i}", role="author")
def claim_unique(i: int):
return self.db.assign_and_lease(
session_id=f"sess-{i}",
role="author",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=1000 + i,
)
with ThreadPoolExecutor(max_workers=4) as pool:
results = [f.result() for f in as_completed([pool.submit(claim_unique, i) for i in range(4)])]
self.assertEqual({r.outcome for r in results}, {"assigned"})
numbers = sorted(r.work_number for r in results)
self.assertEqual(numbers, [1000, 1001, 1002, 1003])
# Contention on one item
self.db.upsert_session(session_id="c1", role="author")
self.db.upsert_session(session_id="c2", role="author")
self.db.upsert_session(session_id="c3", role="author")
self.db.upsert_session(session_id="c4", role="author")
barrier = threading.Barrier(4)
outcomes: list[str] = []
lock = threading.Lock()
def contend(sid: str) -> None:
barrier.wait()
res = self.db.assign_and_lease(
session_id=sid,
role="author",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=7777,
)
with lock:
outcomes.append(res.outcome)
threads = [threading.Thread(target=contend, args=(f"c{i}",)) for i in range(1, 5)]
for t in threads:
t.start()
for t in threads:
t.join()
self.assertEqual(outcomes.count("assigned"), 1)
self.assertEqual(outcomes.count("wait"), 3)
def test_terminal_lock_index(self) -> None:
self.db.set_terminal_lock(
remote="prgs",
org="o",
repo="r",
terminal_pr=332,
review_id="rev-1",
decision="approve",
)
row = self.db.get_active_terminal_lock(remote="prgs", org="o", repo="r")
self.assertIsNotNone(row)
assert row is not None
self.assertEqual(row["terminal_pr"], 332)
self.assertEqual(row["status"], "active")
def test_incident_links_not_work_items(self) -> None:
link = self.db.upsert_incident_link(
provider="sentry",
provider_base_url="https://sentry.prgs.cc",
provider_org="prgs",
provider_project="gitea-tools-mcp",
provider_issue_id="12345",
gitea_org="Scaled-Tech-Consulting",
gitea_repo="Gitea-Tools",
gitea_issue_number=9001,
fingerprint="fp-1",
)
self.assertEqual(link["gitea_issue_number"], 9001)
found = self.db.get_incident_link_for_gitea_issue(
gitea_org="Scaled-Tech-Consulting",
gitea_repo="Gitea-Tools",
gitea_issue_number=9001,
)
self.assertIsNotNone(found)
# Linking does not create a work_item of incident kind
with self.assertRaises(InvalidWorkKindError):
self.db.upsert_work_item(
remote="prgs",
org="Scaled-Tech-Consulting",
repo="Gitea-Tools",
kind="sentry",
number=12345,
)
def test_heartbeat_and_release(self) -> None:
self.db.upsert_session(session_id="s1", role="author")
a = self.db.assign_and_lease(
session_id="s1",
role="author",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=1,
)
hb = self.db.heartbeat_lease(a.lease_id, session_id="s1")
self.assertEqual(hb["lease_id"], a.lease_id)
self.db.release_lease(a.lease_id, session_id="s1")
# After release another session can claim
self.db.upsert_session(session_id="s2", role="author")
b = self.db.assign_and_lease(
session_id="s2",
role="author",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=1,
)
self.assertEqual(b.outcome, "assigned")
self.assertEqual(b.session_id, "s2")
def test_require_valid_assignment_rejects_stale_head(self) -> None:
"""Assignment must not authorize mutations after work-item head drifts."""
self.db.upsert_session(session_id="s1", role="author")
self.db.assign_and_lease(
session_id="s1",
role="author",
remote="prgs",
org="o",
repo="r",
kind="pr",
number=42,
expected_head_sha="head-v1",
allowed_actions=("implement",),
)
# Head drifts after assignment
self.db.upsert_work_item(
remote="prgs",
org="o",
repo="r",
kind="pr",
number=42,
current_head_sha="head-v2",
)
with self.assertRaises(LeaseRequiredError) as ctx:
self.db.require_valid_assignment(
session_id="s1",
remote="prgs",
org="o",
repo="r",
kind="pr",
number=42,
action="implement",
)
self.assertIn("stale head", str(ctx.exception).lower())
def test_require_valid_assignment_rejects_terminal_state(self) -> None:
"""Assignment must not authorize mutations after work item is merged/closed."""
self.db.upsert_session(session_id="s1", role="author")
self.db.assign_and_lease(
session_id="s1",
role="author",
remote="prgs",
org="o",
repo="r",
kind="pr",
number=55,
expected_head_sha="abc",
allowed_actions=("implement",),
)
self.db.upsert_work_item(
remote="prgs",
org="o",
repo="r",
kind="pr",
number=55,
state="merged",
current_head_sha="abc",
)
with self.assertRaises(LeaseRequiredError) as ctx:
self.db.require_valid_assignment(
session_id="s1",
remote="prgs",
org="o",
repo="r",
kind="pr",
number=55,
action="implement",
)
self.assertIn("terminal", str(ctx.exception).lower())
self.db.upsert_work_item(
remote="prgs",
org="o",
repo="r",
kind="issue",
number=56,
state="open",
)
self.db.assign_and_lease(
session_id="s1",
role="author",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=56,
allowed_actions=("implement",),
)
self.db.upsert_work_item(
remote="prgs",
org="o",
repo="r",
kind="issue",
number=56,
state="closed",
)
with self.assertRaises(LeaseRequiredError):
self.db.require_valid_assignment(
session_id="s1",
remote="prgs",
org="o",
repo="r",
kind="issue",
number=56,
action="implement",
)
def test_pr_assignment_requires_expected_head_sha(self) -> None:
"""PR assign and mutation must fail closed without a head pin."""
self.db.upsert_session(session_id="s1", role="author")
with self.assertRaises(LeaseRequiredError) as ctx:
self.db.assign_and_lease(
session_id="s1",
role="author",
remote="prgs",
org="o",
repo="r",
kind="pr",
number=88,
expected_head_sha=None,
allowed_actions=("implement",),
)
self.assertIn("expected_head_sha", str(ctx.exception))
# Legacy path: force an unpinned PR assignment into the DB, then
# populate head and prove mutation is still rejected.
import sqlite3
self.db.upsert_work_item(
remote="prgs",
org="o",
repo="r",
kind="pr",
number=89,
current_head_sha=None,
)
conn = sqlite3.connect(self.db_path)
try:
wid = conn.execute(
"SELECT work_item_id FROM work_items WHERE kind='pr' AND number=89"
).fetchone()[0]
conn.execute(
"""
INSERT INTO sessions(session_id, role, started_at, last_heartbeat_at, status)
VALUES ('legacy', 'author', '2020-01-01T00:00:00Z', '2020-01-01T00:00:00Z', 'active')
"""
)
conn.execute(
"""
INSERT INTO leases(
lease_id, work_item_id, session_id, role, phase,
expires_at, heartbeat_at, status
) VALUES (
'lease-legacy', ?, 'legacy', 'author', 'claimed',
'2099-01-01T00:00:00Z', '2020-01-01T00:00:00Z', 'active'
)
""",
(wid,),
)
conn.execute(
"""
INSERT INTO assignments(
assignment_id, work_item_id, session_id, lease_id,
allowed_actions, forbidden_actions, expected_head_sha,
role, status, created_at
) VALUES (
'asn-legacy', ?, 'legacy', 'lease-legacy',
'["implement"]', '["merge"]', NULL,
'author', 'active', '2020-01-01T00:00:00Z'
)
""",
(wid,),
)
conn.commit()
finally:
conn.close()
self.db.upsert_work_item(
remote="prgs",
org="o",
repo="r",
kind="pr",
number=89,
current_head_sha="populated-head",
)
with self.assertRaises(LeaseRequiredError) as ctx2:
self.db.require_valid_assignment(
session_id="legacy",
remote="prgs",
org="o",
repo="r",
kind="pr",
number=89,
action="implement",
)
self.assertIn("expected_head_sha pin", str(ctx2.exception))
def test_migrate_duplicate_null_scope_incident_links(self) -> None:
"""Legacy NULL-scope duplicates must migrate without UNIQUE crash."""
import sqlite3
from control_plane_db import ControlPlaneDB, ControlPlaneError
# Build a v1-like table with nullable scope columns and insert dups
# that collapse under normalization, then open ControlPlaneDB on it.
path = os.path.join(self._tmp.name, "legacy_dups.sqlite3")
conn = sqlite3.connect(path)
try:
conn.executescript(
"""
CREATE TABLE schema_meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);
CREATE TABLE incident_links (
link_id INTEGER PRIMARY KEY AUTOINCREMENT,
provider TEXT NOT NULL,
provider_base_url TEXT,
provider_org TEXT,
provider_project TEXT,
provider_issue_id TEXT NOT NULL,
provider_short_id TEXT,
provider_permalink TEXT,
fingerprint TEXT,
gitea_org TEXT NOT NULL,
gitea_repo TEXT NOT NULL,
gitea_issue_number INTEGER NOT NULL,
linked_pr_numbers TEXT,
first_seen TEXT,
last_seen TEXT,
event_count INTEGER,
status TEXT NOT NULL DEFAULT 'open',
release_resolved_at TEXT,
last_sync_at TEXT,
UNIQUE (
provider, provider_base_url, provider_org,
provider_project, provider_issue_id
)
);
INSERT INTO incident_links(
provider, provider_base_url, provider_org, provider_project,
provider_issue_id, gitea_org, gitea_repo, gitea_issue_number, status
) VALUES
('sentry', NULL, NULL, NULL, 'dup-1', 'org', 'repo', 10, 'open'),
('sentry', NULL, NULL, NULL, 'dup-1', 'org', 'repo', 10, 'open');
"""
)
# SQLite allows two NULL-scope rows with same provider/issue under UNIQUE.
n = conn.execute("SELECT COUNT(*) FROM incident_links").fetchone()[0]
self.assertEqual(n, 2)
conn.commit()
finally:
conn.close()
db = ControlPlaneDB(path)
conn2 = sqlite3.connect(path)
try:
n2 = conn2.execute("SELECT COUNT(*) FROM incident_links").fetchone()[0]
rows = conn2.execute(
"SELECT provider_base_url, provider_org, provider_project, gitea_issue_number "
"FROM incident_links"
).fetchall()
finally:
conn2.close()
self.assertEqual(n2, 1)
self.assertEqual(rows[0][0], "")
self.assertEqual(rows[0][1], "")
self.assertEqual(rows[0][2], "")
self.assertEqual(rows[0][3], 10)
# Touch to silence unused import in type checkers if needed
self.assertTrue(issubclass(ControlPlaneError, Exception))
del db
def test_migrate_conflicting_duplicate_incident_links_fails_closed(self) -> None:
"""Conflicting Gitea targets for the same provider key must fail closed."""
import sqlite3
from control_plane_db import ControlPlaneDB, ControlPlaneError
path = os.path.join(self._tmp.name, "legacy_conflict.sqlite3")
conn = sqlite3.connect(path)
try:
conn.executescript(
"""
CREATE TABLE incident_links (
link_id INTEGER PRIMARY KEY AUTOINCREMENT,
provider TEXT NOT NULL,
provider_base_url TEXT,
provider_org TEXT,
provider_project TEXT,
provider_issue_id TEXT NOT NULL,
gitea_org TEXT NOT NULL,
gitea_repo TEXT NOT NULL,
gitea_issue_number INTEGER NOT NULL,
status TEXT NOT NULL DEFAULT 'open',
UNIQUE (
provider, provider_base_url, provider_org,
provider_project, provider_issue_id
)
);
INSERT INTO incident_links(
provider, provider_base_url, provider_org, provider_project,
provider_issue_id, gitea_org, gitea_repo, gitea_issue_number
) VALUES
('sentry', NULL, NULL, NULL, 'dup-c', 'org', 'repo', 1),
('sentry', NULL, NULL, NULL, 'dup-c', 'org', 'repo', 2);
"""
)
conn.commit()
finally:
conn.close()
with self.assertRaises(ControlPlaneError) as ctx:
ControlPlaneDB(path)
self.assertIn("conflicting", str(ctx.exception).lower())
def test_migrate_conflicting_observation_metadata_fails_closed(self) -> None:
"""Same provider key + same Gitea target but differing obs metadata must fail closed.
Regression for silent data loss: migration used to keep lowest link_id and
delete peers after comparing only Gitea targets (#619 RC3).
"""
import sqlite3
from control_plane_db import ControlPlaneDB, ControlPlaneError
path = os.path.join(self._tmp.name, "legacy_meta_conflict.sqlite3")
conn = sqlite3.connect(path)
try:
conn.executescript(
"""
CREATE TABLE schema_meta (key TEXT PRIMARY KEY, value TEXT NOT NULL);
CREATE TABLE incident_links (
link_id INTEGER PRIMARY KEY AUTOINCREMENT,
provider TEXT NOT NULL,
provider_base_url TEXT,
provider_org TEXT,
provider_project TEXT,
provider_issue_id TEXT NOT NULL,
provider_short_id TEXT,
provider_permalink TEXT,
fingerprint TEXT,
gitea_org TEXT NOT NULL,
gitea_repo TEXT NOT NULL,
gitea_issue_number INTEGER NOT NULL,
linked_pr_numbers TEXT,
first_seen TEXT,
last_seen TEXT,
event_count INTEGER,
status TEXT NOT NULL DEFAULT 'open',
release_resolved_at TEXT,
last_sync_at TEXT,
UNIQUE (
provider, provider_base_url, provider_org,
provider_project, provider_issue_id
)
);
INSERT INTO incident_links(
provider, provider_base_url, provider_org, provider_project,
provider_issue_id, provider_permalink, fingerprint,
gitea_org, gitea_repo, gitea_issue_number,
event_count, status, first_seen, last_seen
) VALUES
(
'sentry', NULL, NULL, NULL, 'dup-meta',
'https://sentry.example/issues/1', 'fingerprint-A',
'org', 'repo', 42,
1, 'open', '2026-01-01T00:00:00Z', '2026-01-01T01:00:00Z'
),
(
'sentry', NULL, NULL, NULL, 'dup-meta',
'https://sentry.example/issues/1', 'fingerprint-B',
'org', 'repo', 42,
99, 'resolved', '2026-01-01T00:00:00Z', '2026-01-02T00:00:00Z'
);
"""
)
n = conn.execute("SELECT COUNT(*) FROM incident_links").fetchone()[0]
self.assertEqual(n, 2)
conn.commit()
finally:
conn.close()
with self.assertRaises(ControlPlaneError) as ctx:
ControlPlaneDB(path)
msg = str(ctx.exception).lower()
self.assertIn("conflicting", msg)
self.assertIn("observation metadata", msg)
# Rows must still be present — migration must not delete before failing.
conn2 = sqlite3.connect(path)
try:
remaining = conn2.execute("SELECT COUNT(*) FROM incident_links").fetchone()[0]
fps = {
r[0]
for r in conn2.execute(
"SELECT fingerprint FROM incident_links ORDER BY link_id"
).fetchall()
}
finally:
conn2.close()
self.assertEqual(remaining, 2)
self.assertEqual(fps, {"fingerprint-A", "fingerprint-B"})
def test_incident_links_minimal_upsert_is_canonical(self) -> None:
"""Repeated minimal upserts must update one row (NULL-safe uniqueness)."""
a = self.db.upsert_incident_link(
provider="sentry",
provider_issue_id="inc-1",
gitea_org="org",
gitea_repo="repo",
gitea_issue_number=1,
)
b = self.db.upsert_incident_link(
provider="sentry",
provider_issue_id="inc-1",
gitea_org="org",
gitea_repo="repo",
gitea_issue_number=2,
# omit optional scope fields again
)
c = self.db.upsert_incident_link(
provider="sentry",
provider_issue_id="inc-1",
gitea_org="org",
gitea_repo="repo",
gitea_issue_number=3,
provider_base_url=None,
provider_org="",
provider_project=" ",
)
self.assertEqual(a["link_id"], b["link_id"])
self.assertEqual(b["link_id"], c["link_id"])
self.assertEqual(c["gitea_issue_number"], 3)
self.assertEqual(c["provider_base_url"], "")
self.assertEqual(c["provider_org"], "")
self.assertEqual(c["provider_project"], "")
import sqlite3
conn = sqlite3.connect(self.db_path)
try:
n = conn.execute(
"SELECT COUNT(*) FROM incident_links WHERE provider_issue_id = ?",
("inc-1",),
).fetchone()[0]
finally:
conn.close()
self.assertEqual(n, 1)
if __name__ == "__main__":
unittest.main()