Scaled-Tech-Consulting/Gitea-Tools
2
0
Fork 0
Code Issues 101 Pull Requests 11 Actions Packages Projects Releases Wiki Activity

feat(control-plane): add authoritative native MCP fleet inventory (#949)

Browse Source 
Every pre-existing runtime surface is per-process. gitea_get_runtime_context
and gitea_assess_master_parity describe only the server answering the call,
and gitea_assess_mcp_namespace_health accepts process, probe_result and
registered_tools from the caller, so it cannot constrain the caller. The
control-plane sessions table records allocator task sessions, not server
processes. Five self-reports of one revision therefore never proved that five
processes exist, that no sixth exists, or that all five share one client
cohort.

Add gitea_assess_fleet_inventory, a strictly read-only capability that takes
no evidence parameters. It combines two sources that must agree:

- a control-plane runtime registry row each server writes about itself,
  from the official entrypoint, immediately after the native transport bind;
- a process observation the answering server performs, never the caller.

A member is running only when both agree and the observed process predates
its registration, so configuration alone never counts and a recycled PID
cannot impersonate an exited server. Classification is a pure function of the
snapshot, so gitea-controller and gitea-reconciler return the same verdict.

Missing, duplicate, unexpected, stale and unregistered members are reported
in separate fields rather than collapsed into one error, because each needs a
different operator action. single_cohort is derived only from recorded cohort
identity and stays null when unknown, so matching revisions never establish a
cohort. Any unreadable registry, unavailable listing, unregistered process,
unknown cohort or unknown revision sets inventory_complete and
mutation_gate_satisfied false with a specific blocked_reason.

The capability performs no restart, reconnect, drain, lease mutation, issue
mutation or process termination, never terminates a duplicate, and never
manufactures restart evidence. The only signal sent is signal 0.

Also resolves the fleet namespace from the expected roster:
role_namespace_gate.infer_mcp_namespace recognises only author and reviewer
and echoes the profile name otherwise, which would have mislabelled the
controller, merger and reconciler members. Widening that shared helper is
controller role-metadata work owned by #950 and is deliberately not done here.

Schema v5 to v6 is additive and idempotent: one new table, no existing table,
tool signature or result field changed. Two control-plane tests that pinned
the schema version to the literal 5 now assert against SCHEMA_VERSION.

#950 (controller role metadata), #951 (restart receipts) and #952 (stale-lease
consistency) remain separate and untouched.

Tests: 118 new across tests/test_mcp_fleet_inventory.py,
tests/test_control_plane_db_server_runtimes.py and
tests/test_fleet_inventory_tool.py, covering every acceptance criterion
including the multi-LLM duplicate-server regression that motivated the issue.

Closes #949

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
Claude-Session: https://claude.ai/code/session_01NBaQKcRRKeKbZuhk72MhEf
This commit is contained in:
sysadmin
2026-07-27 23:25:45 -04:00
co-authored by Claude Opus 5
parent 82d71b7702
commit 92615f474b
12 changed files with 2718 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/test_control_plane_db.py
+6 -2
View File
@@ -14,6 +14,7 @@ from control_plane_db import (
ControlPlaneError,
InvalidWorkKindError,
LeaseRequiredError,
SCHEMA_VERSION,
WORK_KINDS,
_ts,
_utc_now,
@@ -37,7 +38,10 @@ class ControlPlaneDBTest(unittest.TestCase):
rows = dict(conn.execute("SELECT key, value FROM schema_meta").fetchall())
finally:
conn.close()
self.assertEqual(rows["schema_version"], "5")
# Pinned to the constant, not a literal: every additive migration bumps
# SCHEMA_VERSION, and the invariant under test is that the meta row
# records the version the code actually wrote (#949 added v6).
self.assertEqual(rows["schema_version"], str(SCHEMA_VERSION))
self.assertIn("DB coordinates", rows["architecture"])
self.assertIn("bridge", rows["architecture"].lower())
@@ -868,7 +872,7 @@ class SessionCheckpointTest(unittest.TestCase):
conn.close()
self.assertIn("session_checkpoints", names)
record = self._write()
self.assertEqual(record["checkpoint_schema_version"], 5)
self.assertEqual(record["checkpoint_schema_version"], SCHEMA_VERSION)
# AC2 — checkpoints written for multi-role session fixtures.
def test_multi_role_fixtures_each_get_a_row(self) -> None:
tests/test_control_plane_db_server_runtimes.py
+282
View File
@@ -0,0 +1,282 @@
"""Control-plane MCP server runtime registry (#949).
The registry is the half of the fleet evidence a caller cannot supply: each
server writes exactly one row about itself at native transport bind. These tests
pin the storage contract — additive migration, deterministic ordering, and
PID-reuse/retention pruning that cannot hide a live duplicate.
"""
import os
import tempfile
import unittest
from datetime import datetime, timedelta, timezone
from unittest import mock
import control_plane_db
import mcp_fleet_inventory as mfi
def _stamp(delta_seconds=0):
moment = datetime.now(timezone.utc) + timedelta(seconds=delta_seconds)
return moment.replace(microsecond=0).strftime("%Y-%m-%dT%H:%M:%SZ")
class _DBCase(unittest.TestCase):
def setUp(self):
self._tmp = tempfile.TemporaryDirectory()
self.addCleanup(self._tmp.cleanup)
self.db_path = os.path.join(self._tmp.name, "control_plane.sqlite3")
self.db = control_plane_db.ControlPlaneDB(db_path=self.db_path)
def record(self, namespace, profile, role, pid, **overrides):
base = mfi.build_process_runtime_record(
namespace=namespace,
profile=profile,
role=role,
remote="prgs",
org="Scaled-Tech-Consulting",
repo="Gitea-Tools",
repository_root="/checkout/Gitea-Tools",
pid=pid,
startup_head="82d71b77028a7abd4f8ab4a4e4d89658a187f73d",
transport="stdio",
client_provenance="client_managed",
env={mfi.COHORT_ID_ENV: "ppid:40990"},
boot_id=f"boot{pid}",
)
base.update(overrides)
return base
class TestSchema(_DBCase):
def test_schema_version_is_bumped(self):
self.assertGreaterEqual(control_plane_db.SCHEMA_VERSION, 6)
def test_runtime_table_exists_on_a_fresh_database(self):
self.assertEqual(self.db.list_mcp_server_runtimes(), [])
def test_migration_is_additive_and_idempotent(self):
"""Re-opening an existing DB must not disturb the other tables."""
self.db.upsert_session(session_id="s-a", role="author", profile="prgs-author")
self.db.register_mcp_server_runtime(
self.record("gitea-author", "prgs-author", "author", 41000)
)
reopened = control_plane_db.ControlPlaneDB(db_path=self.db_path)
self.assertEqual(len(reopened.list_sessions()), 1)
self.assertEqual(len(reopened.list_mcp_server_runtimes()), 1)
class TestRegistration(_DBCase):
def test_registration_round_trips_every_field(self):
record = self.record("gitea-controller", "prgs-controller", "controller", 41001)
stored = self.db.register_mcp_server_runtime(record)
for key in (
"runtime_id",
"namespace",
"profile",
"role",
"remote",
"org",
"repo",
"repository_root",
"pid",
"cohort_id",
"cohort_source",
"client_provenance",
"boot_id",
"startup_head",
"transport",
"status",
):
self.assertEqual(stored[key], record[key], key)
def test_registration_requires_a_runtime_id(self):
record = self.record("gitea-author", "prgs-author", "author", 41002)
record["runtime_id"] = ""
with self.assertRaises(ValueError):
self.db.register_mcp_server_runtime(record)
def test_registration_requires_a_namespace(self):
record = self.record("gitea-author", "prgs-author", "author", 41003)
record["namespace"] = " "
with self.assertRaises(ValueError):
self.db.register_mcp_server_runtime(record)
def test_registration_requires_a_pid(self):
record = self.record("gitea-author", "prgs-author", "author", 41004)
record["pid"] = None
with self.assertRaises(ValueError):
self.db.register_mcp_server_runtime(record)
def test_five_members_register_independently(self):
for index, entry in enumerate(mfi.EXPECTED_PRGS_FLEET):
self.db.register_mcp_server_runtime(
self.record(
entry["namespace"], entry["profile"], entry["role"], 41100 + index
)
)
rows = self.db.list_mcp_server_runtimes()
self.assertEqual(len(rows), 5)
self.assertEqual(
sorted(r["profile"] for r in rows),
[
"prgs-author",
"prgs-controller",
"prgs-merger",
"prgs-reconciler",
"prgs-reviewer",
],
)
class TestPruning(_DBCase):
def test_reusing_a_pid_replaces_the_stale_row(self):
first = self.record("gitea-author", "prgs-author", "author", 41200)
self.db.register_mcp_server_runtime(first)
second = self.record("gitea-reviewer", "prgs-reviewer", "reviewer", 41200)
self.db.register_mcp_server_runtime(second)
rows = self.db.list_mcp_server_runtimes()
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0]["runtime_id"], second["runtime_id"])
def test_pruning_never_removes_a_live_duplicate_on_another_pid(self):
"""The defect this must not have: hiding a second running server."""
first = self.record("gitea-author", "prgs-author", "author", 41300)
self.db.register_mcp_server_runtime(first)
second = self.record("gitea-author", "prgs-author", "author", 41301)
self.db.register_mcp_server_runtime(
second, retention_seconds=mfi.RUNTIME_RETENTION_SECONDS
)
rows = self.db.list_mcp_server_runtimes()
self.assertEqual(len(rows), 2)
self.assertEqual(sorted(r["pid"] for r in rows), [41300, 41301])
def test_retention_drops_rows_older_than_the_window(self):
old = self.record(
"gitea-merger",
"prgs-merger",
"merger",
41400,
registered_at=_stamp(-(mfi.RUNTIME_RETENTION_SECONDS + 3600)),
)
self.db.register_mcp_server_runtime(old)
fresh = self.record("gitea-author", "prgs-author", "author", 41401)
self.db.register_mcp_server_runtime(
fresh, retention_seconds=mfi.RUNTIME_RETENTION_SECONDS
)
rows = self.db.list_mcp_server_runtimes()
self.assertEqual([r["pid"] for r in rows], [41401])
def test_retention_is_skipped_when_not_requested(self):
old = self.record(
"gitea-merger",
"prgs-merger",
"merger",
41500,
registered_at=_stamp(-(mfi.RUNTIME_RETENTION_SECONDS + 3600)),
)
self.db.register_mcp_server_runtime(old)
self.db.register_mcp_server_runtime(
self.record("gitea-author", "prgs-author", "author", 41501)
)
self.assertEqual(len(self.db.list_mcp_server_runtimes()), 2)
class TestListing(_DBCase):
def test_listing_is_deterministically_ordered(self):
for index in range(5):
self.db.register_mcp_server_runtime(
self.record(
"gitea-author",
"prgs-author",
"author",
41600 + index,
registered_at="2026-07-28T02:00:00Z",
)
)
first = [r["runtime_id"] for r in self.db.list_mcp_server_runtimes()]
second = [r["runtime_id"] for r in self.db.list_mcp_server_runtimes()]
self.assertEqual(first, second)
self.assertEqual(first, sorted(first))
def test_status_filter_selects_running_rows(self):
running = self.record("gitea-author", "prgs-author", "author", 41700)
self.db.register_mcp_server_runtime(running)
stopped = self.record("gitea-merger", "prgs-merger", "merger", 41701)
self.db.register_mcp_server_runtime(stopped)
self.db.mark_mcp_server_runtime_stopped(stopped["runtime_id"])
rows = self.db.list_mcp_server_runtimes(statuses=("running",))
self.assertEqual([r["pid"] for r in rows], [41700])
def test_listing_without_a_filter_returns_stopped_rows_too(self):
stopped = self.record("gitea-merger", "prgs-merger", "merger", 41800)
self.db.register_mcp_server_runtime(stopped)
self.db.mark_mcp_server_runtime_stopped(stopped["runtime_id"])
rows = self.db.list_mcp_server_runtimes()
self.assertEqual(len(rows), 1)
self.assertEqual(rows[0]["status"], "stopped")
class TestHeartbeat(_DBCase):
def test_heartbeat_advances_only_the_timestamp(self):
record = self.record(
"gitea-author",
"prgs-author",
"author",
41900,
last_heartbeat_at="2026-07-28T02:00:00Z",
)
self.db.register_mcp_server_runtime(record)
self.db.heartbeat_mcp_server_runtime(record["runtime_id"])
row = self.db.list_mcp_server_runtimes()[0]
self.assertNotEqual(row["last_heartbeat_at"], "2026-07-28T02:00:00Z")
self.assertEqual(row["status"], "running")
self.assertEqual(row["pid"], 41900)
def test_heartbeat_for_an_unknown_runtime_is_a_no_op(self):
self.db.heartbeat_mcp_server_runtime("does-not-exist")
self.assertEqual(self.db.list_mcp_server_runtimes(), [])
class TestEndToEndSnapshot(_DBCase):
"""Registry rows feed the classifier without reshaping."""
def test_registered_fleet_classifies_as_healthy(self):
pids = []
for index, entry in enumerate(mfi.EXPECTED_PRGS_FLEET):
pid = 42000 + index
pids.append(pid)
self.db.register_mcp_server_runtime(
self.record(
entry["namespace"],
entry["profile"],
entry["role"],
pid,
registered_at="2026-07-28T02:00:00Z",
)
)
rows = self.db.list_mcp_server_runtimes(statuses=("running",))
scan = {
"available": True,
"processes": [
{"pid": pid, "started_at": "2026-07-28T01:00:00Z"} for pid in pids
],
"reason": None,
}
with mock.patch.object(mfi, "probe_pid_alive", return_value=True):
result = mfi.classify_fleet_inventory(
runtime_rows=rows,
process_scan=scan,
expected_binding={
"remote": "prgs",
"org": "Scaled-Tech-Consulting",
"repo": "Gitea-Tools",
},
)
self.assertTrue(result["inventory_complete"])
self.assertTrue(result["mutation_gate_satisfied"])
self.assertEqual(result["running_member_count"], 5)
if __name__ == "__main__":
unittest.main()
tests/test_fleet_inventory_tool.py
+337
View File
@@ -0,0 +1,337 @@
"""Native exposure of the fleet inventory capability (#949).
Covers the parts the pure classifier cannot: that the tool is registered and
documented, that it is gated on ``gitea.read`` rather than a role, that it
accepts no caller-supplied evidence, that it fails closed on an unreadable
registry, and that the workflow documentation explains gate consumption.
"""
import inspect
import os
import pathlib
import unittest
from unittest import mock
import gitea_mcp_server
import mcp_fleet_inventory as mfi
import task_capability_map
REPO_ROOT = pathlib.Path(__file__).resolve().parent.parent
TOOL_NAME = "gitea_assess_fleet_inventory"
class TestCapabilityMapExposure(unittest.TestCase):
"""AC14: the invariant is provable through sanctioned native calls."""
def test_task_resolves_to_a_read_permission(self):
self.assertEqual(
task_capability_map.required_permission("assess_fleet_inventory"),
"gitea.read",
)
self.assertEqual(
task_capability_map.required_permission(TOOL_NAME), "gitea.read"
)
def test_task_and_tool_keys_agree(self):
self.assertEqual(
task_capability_map.TASK_CAPABILITY_MAP["assess_fleet_inventory"],
task_capability_map.TASK_CAPABILITY_MAP[TOOL_NAME],
)
def test_task_is_not_role_exclusive(self):
"""Controller and reconciler both need it; so does any read-only gate."""
self.assertNotIn(
"assess_fleet_inventory", task_capability_map.ROLE_EXCLUSIVE_TASKS
)
self.assertNotIn(TOOL_NAME, task_capability_map.ROLE_EXCLUSIVE_TASKS)
def test_task_is_not_registered_as_an_issue_mutation(self):
self.assertNotIn(TOOL_NAME, task_capability_map.ISSUE_MUTATION_TOOL_TASKS)
def test_unknown_task_still_fails_closed(self):
with self.assertRaises(KeyError):
task_capability_map.required_permission("assess_fleet_inventory_typo")
class TestToolRegistration(unittest.TestCase):
def test_tool_is_registered_on_the_server(self):
self.assertTrue(hasattr(gitea_mcp_server, TOOL_NAME))
def test_tool_accepts_no_caller_supplied_evidence(self):
"""The defect #949 names: evidence parameters the caller controls."""
signature = inspect.signature(gitea_mcp_server.gitea_assess_fleet_inventory)
self.assertEqual(sorted(signature.parameters), ["host", "org", "remote", "repo"])
for forbidden in ("process", "probe_result", "registered_tools", "processes"):
self.assertNotIn(forbidden, signature.parameters)
def test_tool_is_documented_in_the_canonical_inventory(self):
doc = (REPO_ROOT / "docs" / "mcp-tool-inventory.md").read_text()
self.assertIn(f"`{TOOL_NAME}`", doc)
def test_docstring_states_the_read_only_guarantee(self):
doc = (gitea_mcp_server.gitea_assess_fleet_inventory.__doc__ or "").lower()
self.assertIn("read-only", doc)
self.assertIn("fails closed", doc)
class TestNamespaceResolution(unittest.TestCase):
"""Every configured profile must resolve to its real fleet namespace.
``role_namespace_gate.infer_mcp_namespace`` recognises only author and
reviewer and echoes the profile name for the rest, which would label the
controller, merger, and reconciler members with namespaces that do not
exist — and then flag all three as role mismatches.
"""
def test_every_expected_profile_maps_to_its_namespace(self):
for entry in mfi.EXPECTED_PRGS_FLEET:
self.assertEqual(
mfi.namespace_for_profile(entry["profile"]),
entry["namespace"],
entry["profile"],
)
def test_server_helper_agrees_with_the_roster(self):
for entry in mfi.EXPECTED_PRGS_FLEET:
self.assertEqual(
gitea_mcp_server._fleet_namespace_for_profile(entry["profile"]),
entry["namespace"],
entry["profile"],
)
def test_unknown_profile_falls_back_to_the_supplied_default(self):
self.assertEqual(
mfi.namespace_for_profile("prgs-shadow", default="gitea-shadow"),
"gitea-shadow",
)
def test_unknown_profile_without_a_default_returns_the_profile(self):
self.assertEqual(mfi.namespace_for_profile("prgs-shadow"), "prgs-shadow")
def test_registered_row_uses_the_roster_namespace(self):
fake_db = mock.Mock()
with mock.patch.object(
gitea_mcp_server, "_control_plane_db_or_error", return_value=(fake_db, [])
), mock.patch.object(
gitea_mcp_server, "get_profile", return_value={"allowed_operations": []}
), mock.patch.object(
gitea_mcp_server.gitea_config,
"selected_profile_name",
return_value="prgs-reconciler",
):
record = gitea_mcp_server._register_fleet_runtime(transport="stdio")
self.assertEqual(record["namespace"], "gitea-reconciler")
class TestToolBehavior(unittest.TestCase):
"""The tool wires the registry and the process scan into the classifier."""
@staticmethod
def _healthy_rows():
return [
{
"runtime_id": f"{entry['namespace']}:{43000 + index}:boot",
"namespace": entry["namespace"],
"profile": entry["profile"],
"role": entry["role"],
"remote": "prgs",
"org": "Scaled-Tech-Consulting",
"repo": "Gitea-Tools",
"repository_root": "/checkout/Gitea-Tools",
"pid": 43000 + index,
"cohort_id": "ppid:40990",
"cohort_source": "parent_process",
"client_provenance": "client_managed",
"boot_id": "boot",
"startup_head": "82d71b77028a7abd4f8ab4a4e4d89658a187f73d",
"daemon_start_head": "82d71b77028a7abd4f8ab4a4e4d89658a187f73d",
"transport": "stdio",
"registered_at": "2026-07-28T02:00:00Z",
"last_heartbeat_at": "2026-07-28T02:00:00Z",
"status": "running",
}
for index, entry in enumerate(mfi.EXPECTED_PRGS_FLEET)
]
def _call(self, rows, *, scan=None, db=None, db_errors=None):
fake_db = db
if fake_db is None and db_errors is None:
fake_db = mock.Mock()
fake_db.list_mcp_server_runtimes.return_value = rows
scan = scan or {
"available": True,
"processes": [
{"pid": r["pid"], "started_at": "2026-07-28T01:00:00Z"} for r in rows
],
"reason": None,
}
with mock.patch.object(
gitea_mcp_server, "_profile_operation_gate", return_value=[]
), mock.patch.object(
gitea_mcp_server,
"_control_plane_db_or_error",
return_value=(fake_db, db_errors or []),
), mock.patch.object(
gitea_mcp_server, "_active_profile_name", return_value="prgs-controller"
), mock.patch.object(
mfi, "scan_mcp_server_processes", return_value=scan
), mock.patch.object(
mfi, "probe_pid_alive", return_value=True
):
return gitea_mcp_server.gitea_assess_fleet_inventory(
remote="prgs", org="Scaled-Tech-Consulting", repo="Gitea-Tools"
)
def test_healthy_fleet_satisfies_the_gate_through_the_tool(self):
result = self._call(self._healthy_rows())
self.assertTrue(result["inventory_complete"])
self.assertTrue(result["mutation_gate_satisfied"])
self.assertEqual(result["running_member_count"], 5)
self.assertEqual(result["mutations_performed"], [])
def test_tool_reports_the_expected_repository_binding(self):
result = self._call(self._healthy_rows())
self.assertEqual(
result["expected_repository_binding"],
{"remote": "prgs", "org": "Scaled-Tech-Consulting", "repo": "Gitea-Tools"},
)
def test_tool_reads_only_running_rows(self):
rows = self._healthy_rows()
fake_db = mock.Mock()
fake_db.list_mcp_server_runtimes.return_value = rows
self._call(rows, db=fake_db)
fake_db.list_mcp_server_runtimes.assert_called_once_with(statuses=("running",))
def test_tool_never_writes_to_the_registry(self):
rows = self._healthy_rows()
fake_db = mock.Mock()
fake_db.list_mcp_server_runtimes.return_value = rows
self._call(rows, db=fake_db)
fake_db.register_mcp_server_runtime.assert_not_called()
fake_db.heartbeat_mcp_server_runtime.assert_not_called()
fake_db.mark_mcp_server_runtime_stopped.assert_not_called()
def test_unavailable_control_plane_fails_closed(self):
result = self._call([], db_errors=["control-plane DB substrate unavailable"])
self.assertFalse(result["inventory_complete"])
self.assertFalse(result["mutation_gate_satisfied"])
self.assertFalse(result["registry"]["available"])
self.assertIn("unavailable", result["blocked_reason"])
def test_registry_read_failure_fails_closed(self):
rows = self._healthy_rows()
fake_db = mock.Mock()
fake_db.list_mcp_server_runtimes.side_effect = RuntimeError("db locked")
result = self._call(rows, db=fake_db)
self.assertFalse(result["inventory_complete"])
self.assertFalse(result["mutation_gate_satisfied"])
self.assertIn("could not be read", result["registry"]["error"])
def test_missing_read_permission_blocks_without_touching_the_registry(self):
fake_db = mock.Mock()
with mock.patch.object(
gitea_mcp_server,
"_profile_operation_gate",
return_value=["profile may not read"],
), mock.patch.object(
gitea_mcp_server, "_control_plane_db_or_error", return_value=(fake_db, [])
):
result = gitea_mcp_server.gitea_assess_fleet_inventory(remote="prgs")
self.assertFalse(result["success"])
self.assertFalse(result["mutation_gate_satisfied"])
self.assertIn("permission_report", result)
self.assertEqual(result["mutations_performed"], [])
fake_db.list_mcp_server_runtimes.assert_not_called()
def test_answering_namespace_is_reported(self):
result = self._call(self._healthy_rows())
self.assertEqual(result["answering_namespace"], "gitea-controller")
def test_summary_is_present(self):
result = self._call(self._healthy_rows())
self.assertIn("5 of 5", result["summary"])
class TestStartupRegistration(unittest.TestCase):
"""The registry row is written by the process it describes."""
def test_registration_helper_exists_on_the_entrypoint_module(self):
self.assertTrue(hasattr(gitea_mcp_server, "_register_fleet_runtime"))
def test_registration_writes_one_row_for_this_process(self):
fake_db = mock.Mock()
with mock.patch.object(
gitea_mcp_server, "_control_plane_db_or_error", return_value=(fake_db, [])
), mock.patch.object(
gitea_mcp_server, "get_profile", return_value={"allowed_operations": []}
):
record = gitea_mcp_server._register_fleet_runtime(transport="stdio")
self.assertIsNotNone(record)
fake_db.register_mcp_server_runtime.assert_called_once()
written = fake_db.register_mcp_server_runtime.call_args.args[0]
self.assertEqual(written["pid"], os.getpid())
self.assertEqual(written["transport"], "stdio")
def test_registration_failure_never_blocks_startup(self):
with mock.patch.object(
gitea_mcp_server,
"_control_plane_db_or_error",
side_effect=RuntimeError("boom"),
):
self.assertIsNone(gitea_mcp_server._register_fleet_runtime())
def test_registration_is_skipped_when_the_control_plane_is_unavailable(self):
with mock.patch.object(
gitea_mcp_server,
"_control_plane_db_or_error",
return_value=(None, ["unavailable"]),
):
self.assertIsNone(gitea_mcp_server._register_fleet_runtime())
def test_entrypoint_registers_after_binding_native_transport(self):
"""Order matters: only a transport-bound process may claim a row."""
source = (REPO_ROOT / "gitea_mcp_server.py").read_text()
bind_at = source.index('bind_native_mcp_transport(transport="stdio")')
register_at = source.index('_register_fleet_runtime(transport="stdio")')
run_at = source.index('mcp.run(transport="stdio")')
self.assertLess(bind_at, register_at)
self.assertLess(register_at, run_at)
class TestWorkflowDocumentation(unittest.TestCase):
"""AC13: documentation explains how the gates consume the result."""
def setUp(self):
self.doc = (REPO_ROOT / "docs" / "mcp-fleet-inventory.md").read_text()
self.skill = (
REPO_ROOT / "skills" / "llm-project-workflow" / "SKILL.md"
).read_text()
def test_dedicated_document_exists(self):
self.assertIn("# Authoritative MCP fleet inventory", self.doc)
def test_document_names_both_consuming_namespaces(self):
self.assertIn("gitea-controller", self.doc)
self.assertIn("gitea-reconciler", self.doc)
def test_document_explains_gate_consumption(self):
self.assertIn("mutation_gate_satisfied", self.doc)
self.assertIn("blocked_reason", self.doc)
def test_document_states_the_non_inferences(self):
self.assertIn("Configuration is not existence", self.doc)
self.assertIn("Matching revisions are not a cohort", self.doc)
def test_document_preserves_the_neighbouring_issue_boundaries(self):
for issue in ("#950", "#951", "#952"):
self.assertIn(issue, self.doc)
def test_canonical_workflow_skill_links_the_document(self):
self.assertIn("docs/mcp-fleet-inventory.md", self.skill)
self.assertIn(TOOL_NAME, self.skill)
if __name__ == "__main__":
unittest.main()
tests/test_mcp_fleet_inventory.py
+788
View File
@@ -0,0 +1,788 @@
"""Authoritative fleet inventory classification (#949).
One test group per acceptance criterion. The classifier is pure, so every
scenario is expressed as a snapshot: registry rows plus a process observation.
PID liveness is the one impure input, so it is patched per test rather than
depending on whatever happens to be running on the machine.
"""
import os
import unittest
from datetime import datetime, timedelta, timezone
from unittest import mock
import mcp_fleet_inventory as mfi
NOW = datetime(2026, 7, 28, 3, 0, 0, tzinfo=timezone.utc)
HEAD_A = "82d71b77028a7abd4f8ab4a4e4d89658a187f73d"
HEAD_B = "35ed8a2fcb11134a37c862ca6eaca26e3028902a"
COHORT_A = "ppid:40990"
COHORT_B = "ppid:51022"
BINDING = {"remote": "prgs", "org": "Scaled-Tech-Consulting", "repo": "Gitea-Tools"}
_BASE_PID = 41000
def row(
namespace,
profile,
role,
pid,
*,
cohort_id=COHORT_A,
startup_head=HEAD_A,
registered_at="2026-07-28T02:00:00Z",
**overrides,
):
"""One control-plane runtime registry row."""
record = {
"runtime_id": f"{namespace}:{pid}:boot{pid}",
"namespace": namespace,
"profile": profile,
"role": role,
"remote": BINDING["remote"],
"org": BINDING["org"],
"repo": BINDING["repo"],
"repository_root": "/checkout/Gitea-Tools",
"pid": pid,
"cohort_id": cohort_id,
"cohort_source": "parent_process",
"client_provenance": "client_managed",
"boot_id": f"boot{pid}",
"startup_head": startup_head,
"daemon_start_head": startup_head,
"transport": "stdio",
"registered_at": registered_at,
"last_heartbeat_at": registered_at,
"status": "running",
}
record.update(overrides)
return record
def healthy_rows():
"""One live row per expected member, all one cohort, all one revision."""
return [
row(entry["namespace"], entry["profile"], entry["role"], _BASE_PID + index)
for index, entry in enumerate(mfi.EXPECTED_PRGS_FLEET)
]
def scan_for(rows, *, available=True, extra_pids=(), started_at="2026-07-28T01:00:00Z"):
"""A process observation that corroborates *rows* (plus any extra PIDs)."""
processes = [
{"pid": r["pid"], "started_at": started_at, "command": "python mcp_server.py"}
for r in rows
]
processes.extend(
{"pid": pid, "started_at": started_at, "command": "python mcp_server.py"}
for pid in extra_pids
)
return {"available": available, "processes": processes, "reason": None}
def classify(rows, scan=None, **kwargs):
kwargs.setdefault("expected_binding", BINDING)
kwargs.setdefault("now", NOW)
return mfi.classify_fleet_inventory(
runtime_rows=rows,
process_scan=scan if scan is not None else scan_for(rows),
**kwargs,
)
class _AliveMixin:
"""Treat every PID as alive unless a test declares it dead or unknown."""
def setUp(self):
super().setUp()
self.dead_pids = set()
self.unknown_pids = set()
def probe(pid):
if pid in self.unknown_pids:
return None
return pid not in self.dead_pids
patcher = mock.patch.object(mfi, "probe_pid_alive", side_effect=probe)
patcher.start()
self.addCleanup(patcher.stop)
class TestHealthyFleet(_AliveMixin, unittest.TestCase):
"""AC1: a healthy five-server fleet reports all five members exactly once."""
def test_five_members_each_reported_once(self):
result = classify(healthy_rows())
self.assertEqual(len(result["configured_members"]), 5)
self.assertEqual(len(result["running_members"]), 5)
self.assertEqual(result["running_member_count"], 5)
self.assertEqual(result["missing_members"], [])
self.assertEqual(result["duplicate_members"], [])
self.assertEqual(result["unexpected_members"], [])
self.assertTrue(result["exactly_one_per_profile"])
for member in result["configured_members"]:
self.assertEqual(member["instance_count"], 1)
self.assertEqual(member["health"], mfi.HEALTH_RUNNING)
def test_healthy_fleet_satisfies_the_mutation_gate(self):
result = classify(healthy_rows())
self.assertTrue(result["inventory_complete"])
self.assertTrue(result["mutation_gate_satisfied"])
self.assertIsNone(result["blocked_reason"])
self.assertTrue(result["single_cohort"])
self.assertFalse(result["mixed_cohort"])
self.assertFalse(result["mixed_revision"])
def test_every_expected_namespace_and_profile_is_present(self):
result = classify(healthy_rows())
self.assertEqual(
sorted(m["profile"] for m in result["configured_members"]),
[
"prgs-author",
"prgs-controller",
"prgs-merger",
"prgs-reconciler",
"prgs-reviewer",
],
)
class TestDuplicateMember(_AliveMixin, unittest.TestCase):
"""AC2: two processes on one profile are a duplicate and fail the gate."""
def test_duplicate_is_reported_with_every_pid(self):
rows = healthy_rows()
rows.append(row("gitea-author", "prgs-author", "author", 49999))
result = classify(rows)
self.assertEqual(len(result["duplicate_members"]), 1)
duplicate = result["duplicate_members"][0]
self.assertEqual(duplicate["profile"], "prgs-author")
self.assertEqual(duplicate["instance_count"], 2)
self.assertEqual(duplicate["pids"], [_BASE_PID, 49999])
def test_duplicate_fails_the_mutation_gate(self):
rows = healthy_rows()
rows.append(row("gitea-author", "prgs-author", "author", 49999))
result = classify(rows)
self.assertFalse(result["exactly_one_per_profile"])
self.assertFalse(result["mutation_gate_satisfied"])
self.assertIn("duplicate", result["blocked_reason"])
def test_duplicate_is_not_reported_as_missing_or_unexpected(self):
rows = healthy_rows()
rows.append(row("gitea-author", "prgs-author", "author", 49999))
result = classify(rows)
self.assertEqual(result["missing_members"], [])
self.assertEqual(result["unexpected_members"], [])
class TestMissingMember(_AliveMixin, unittest.TestCase):
"""AC3: a missing expected server is identified and fails the gate."""
def test_missing_member_is_named(self):
rows = [r for r in healthy_rows() if r["profile"] != "prgs-merger"]
result = classify(rows)
self.assertEqual(len(result["missing_members"]), 1)
self.assertEqual(result["missing_members"][0]["profile"], "prgs-merger")
self.assertEqual(result["missing_members"][0]["health"], mfi.HEALTH_MISSING)
def test_missing_member_fails_the_mutation_gate(self):
rows = [r for r in healthy_rows() if r["profile"] != "prgs-merger"]
result = classify(rows)
self.assertFalse(result["exactly_one_per_profile"])
self.assertFalse(result["mutation_gate_satisfied"])
self.assertIn("prgs-merger", result["blocked_reason"])
def test_missing_member_still_lists_all_configured_members(self):
rows = [r for r in healthy_rows() if r["profile"] != "prgs-merger"]
result = classify(rows)
self.assertEqual(len(result["configured_members"]), 5)
merger = [
m for m in result["configured_members"] if m["profile"] == "prgs-merger"
][0]
self.assertEqual(merger["instance_count"], 0)
self.assertEqual(merger["health"], mfi.HEALTH_MISSING)
class TestUnexpectedMember(_AliveMixin, unittest.TestCase):
"""AC4: an unexpected PRGS server is reported explicitly."""
def test_unexpected_member_is_its_own_category(self):
rows = healthy_rows()
rows.append(row("gitea-shadow", "prgs-shadow", "author", 47777))
result = classify(rows)
self.assertEqual(len(result["unexpected_members"]), 1)
self.assertEqual(result["unexpected_members"][0]["profile"], "prgs-shadow")
self.assertEqual(
result["unexpected_members"][0]["health"], mfi.HEALTH_UNEXPECTED
)
def test_unexpected_member_is_not_collapsed_into_duplicates_or_missing(self):
rows = healthy_rows()
rows.append(row("gitea-shadow", "prgs-shadow", "author", 47777))
result = classify(rows)
self.assertEqual(result["duplicate_members"], [])
self.assertEqual(result["missing_members"], [])
self.assertTrue(result["exactly_one_per_profile"])
self.assertFalse(result["no_unexpected_members"])
self.assertFalse(result["mutation_gate_satisfied"])
def test_unexpected_member_blocks_with_its_own_reason(self):
rows = healthy_rows()
rows.append(row("gitea-shadow", "prgs-shadow", "author", 47777))
result = classify(rows)
self.assertTrue(
any("unexpected" in reason for reason in result["blocked_reasons"])
)
class TestMixedCohort(_AliveMixin, unittest.TestCase):
"""AC5: members from different client cohorts are detected."""
def test_two_cohorts_are_detected(self):
rows = healthy_rows()
rows[0]["cohort_id"] = COHORT_B
result = classify(rows)
self.assertTrue(result["mixed_cohort"])
self.assertFalse(result["single_cohort"])
self.assertEqual(result["cohort_ids"], sorted([COHORT_A, COHORT_B]))
def test_mixed_cohort_fails_the_mutation_gate(self):
rows = healthy_rows()
rows[0]["cohort_id"] = COHORT_B
result = classify(rows)
self.assertFalse(result["mutation_gate_satisfied"])
self.assertTrue(any("cohort" in reason for reason in result["blocked_reasons"]))
def test_mixed_cohort_is_not_a_duplicate_or_missing_report(self):
rows = healthy_rows()
rows[0]["cohort_id"] = COHORT_B
result = classify(rows)
self.assertEqual(result["duplicate_members"], [])
self.assertEqual(result["missing_members"], [])
class TestMixedRevision(_AliveMixin, unittest.TestCase):
"""AC6: members running different startup revisions are detected."""
def test_two_revisions_are_detected(self):
rows = healthy_rows()
rows[0]["startup_head"] = HEAD_B
result = classify(rows)
self.assertTrue(result["mixed_revision"])
self.assertEqual(result["startup_revisions"], sorted([HEAD_A, HEAD_B]))
def test_mixed_revision_fails_the_mutation_gate(self):
rows = healthy_rows()
rows[0]["startup_head"] = HEAD_B
result = classify(rows)
self.assertFalse(result["mutation_gate_satisfied"])
self.assertTrue(
any("revision" in reason for reason in result["blocked_reasons"])
)
def test_mixed_revision_does_not_by_itself_imply_mixed_cohort(self):
rows = healthy_rows()
rows[0]["startup_head"] = HEAD_B
result = classify(rows)
self.assertTrue(result["single_cohort"])
self.assertFalse(result["mixed_cohort"])
class TestRevisionIsNotCohort(_AliveMixin, unittest.TestCase):
"""AC7: matching Git revisions alone do not establish a single cohort."""
def test_identical_revisions_with_unknown_cohort_stay_unknown(self):
rows = healthy_rows()
for r in rows:
r["cohort_id"] = None
result = classify(rows)
self.assertEqual(len({r["startup_head"] for r in rows}), 1)
self.assertIsNone(result["single_cohort"])
self.assertIsNone(result["mixed_cohort"])
def test_identical_revisions_with_unknown_cohort_fail_closed(self):
rows = healthy_rows()
for r in rows:
r["cohort_id"] = None
result = classify(rows)
self.assertFalse(result["inventory_complete"])
self.assertFalse(result["mutation_gate_satisfied"])
self.assertTrue(
any("cohort" in reason for reason in result["incomplete_reasons"])
)
def test_one_unknown_cohort_among_known_ones_still_fails_closed(self):
rows = healthy_rows()
rows[0]["cohort_id"] = None
result = classify(rows)
self.assertIsNone(result["single_cohort"])
self.assertFalse(result["mutation_gate_satisfied"])
def test_cohort_derivation_never_consults_revisions(self):
"""The cohort helper takes no revision input at all."""
identity = mfi.derive_cohort_identity({mfi.COHORT_ID_ENV: "cohort-x"})
self.assertEqual(identity["cohort_id"], "cohort-x")
self.assertEqual(identity["cohort_source"], "explicit_env")
def test_orphaned_process_reports_unknown_cohort(self):
with mock.patch("os.getppid", return_value=1):
identity = mfi.derive_cohort_identity({})
self.assertIsNone(identity["cohort_id"])
self.assertEqual(identity["cohort_source"], "unknown")
def test_parent_process_is_the_cohort_when_no_env_is_set(self):
with mock.patch("os.getppid", return_value=40990):
identity = mfi.derive_cohort_identity({})
self.assertEqual(identity["cohort_id"], COHORT_A)
self.assertEqual(identity["cohort_source"], "parent_process")
class TestConfigurationIsNotRunning(_AliveMixin, unittest.TestCase):
"""AC8: configuration without a live worker is not a running member."""
def test_no_registry_rows_means_every_member_is_missing(self):
result = classify([], scan=scan_for([]))
self.assertEqual(len(result["missing_members"]), 5)
self.assertEqual(result["running_members"], [])
self.assertFalse(result["mutation_gate_satisfied"])
def test_dead_pid_is_stale_not_running(self):
rows = healthy_rows()
self.dead_pids = {rows[0]["pid"]}
result = classify(rows)
self.assertEqual(len(result["running_members"]), 4)
self.assertEqual(len(result["stale_members"]), 1)
self.assertEqual(result["stale_members"][0]["liveness"], mfi.LIVENESS_DEAD)
self.assertEqual(result["stale_members"][0]["health"], mfi.HEALTH_STALE)
self.assertEqual(len(result["missing_members"]), 1)
def test_registry_row_without_a_matching_process_is_not_running(self):
rows = healthy_rows()
scan = scan_for(rows[1:]) # first member's process is absent
result = classify(rows, scan=scan)
self.assertEqual(len(result["running_members"]), 4)
self.assertEqual(
result["stale_members"][0]["liveness"], mfi.LIVENESS_UNOBSERVED
)
self.assertFalse(result["mutation_gate_satisfied"])
def test_recycled_pid_does_not_impersonate_a_dead_server(self):
rows = healthy_rows()
scan = scan_for(rows)
# The process now holding the first PID started *after* registration.
scan["processes"][0]["started_at"] = "2026-07-28T02:30:00Z"
result = classify(rows, scan=scan)
stale = [
m
for m in result["stale_members"]
if m["liveness"] == mfi.LIVENESS_PID_RECYCLED
]
self.assertEqual(len(stale), 1)
self.assertEqual(len(result["running_members"]), 4)
self.assertFalse(result["mutation_gate_satisfied"])
class TestIncompleteEvidenceFailsClosed(_AliveMixin, unittest.TestCase):
"""AC9: unknown or unavailable evidence produces a fail-closed result."""
def test_unavailable_process_listing_fails_closed(self):
rows = healthy_rows()
result = classify(
rows,
scan={"available": False, "processes": [], "reason": "ps unavailable"},
)
self.assertFalse(result["inventory_complete"])
self.assertFalse(result["mutation_gate_satisfied"])
self.assertIn("ps unavailable", result["incomplete_reasons"])
self.assertEqual(result["running_members"], [])
def test_unreadable_registry_fails_closed(self):
result = classify(
[],
scan=scan_for([]),
registry_available=False,
registry_error="registry unreadable",
)
self.assertFalse(result["inventory_complete"])
self.assertFalse(result["mutation_gate_satisfied"])
self.assertIn("registry unreadable", result["incomplete_reasons"])
def test_unregistered_running_process_fails_closed(self):
rows = healthy_rows()
result = classify(rows, scan=scan_for(rows, extra_pids=[59999]))
self.assertEqual([p["pid"] for p in result["unregistered_processes"]], [59999])
self.assertFalse(result["inventory_complete"])
self.assertFalse(result["mutation_gate_satisfied"])
self.assertTrue(
any("59999" in reason for reason in result["incomplete_reasons"])
)
def test_undeterminable_pid_liveness_is_unknown_not_healthy(self):
rows = healthy_rows()
self.unknown_pids = {rows[0]["pid"]}
result = classify(rows)
self.assertEqual(result["stale_members"][0]["liveness"], mfi.LIVENESS_UNKNOWN)
self.assertEqual(result["stale_members"][0]["health"], mfi.HEALTH_UNKNOWN)
self.assertFalse(result["inventory_complete"])
self.assertFalse(result["mutation_gate_satisfied"])
def test_unknown_startup_revision_fails_closed(self):
rows = healthy_rows()
rows[0]["startup_head"] = None
result = classify(rows)
self.assertFalse(result["inventory_complete"])
self.assertFalse(result["mutation_gate_satisfied"])
self.assertTrue(
any("startup revision" in r for r in result["incomplete_reasons"])
)
def test_unknown_is_distinguished_from_healthy(self):
rows = healthy_rows()
for r in rows:
r["cohort_id"] = None
result = classify(rows)
# Not "unhealthy" in the sense of a named defect: nothing is missing,
# duplicated or unexpected. It is *unknown*, and that still fails closed.
self.assertEqual(result["missing_members"], [])
self.assertEqual(result["duplicate_members"], [])
self.assertEqual(result["unexpected_members"], [])
self.assertIsNone(result["single_cohort"])
self.assertFalse(result["mutation_gate_satisfied"])
class TestBindingAndRoleConsistency(_AliveMixin, unittest.TestCase):
"""Repository-binding and role/profile mismatches fail closed."""
def test_repository_binding_mismatch_is_reported(self):
rows = healthy_rows()
rows[0]["repo"] = "Some-Other-Repo"
result = classify(rows)
self.assertEqual(len(result["repository_binding_mismatches"]), 1)
self.assertEqual(
result["repository_binding_mismatches"][0]["repo"], "Some-Other-Repo"
)
self.assertFalse(result["mutation_gate_satisfied"])
def test_incomplete_repository_binding_is_reported(self):
rows = healthy_rows()
rows[0]["org"] = None
result = classify(rows)
self.assertEqual(len(result["repository_binding_mismatches"]), 1)
self.assertIn("complete", result["repository_binding_mismatches"][0]["reason"])
self.assertFalse(result["mutation_gate_satisfied"])
def test_role_mismatch_is_reported(self):
rows = healthy_rows()
rows[0]["role"] = "merger" # prgs-author is configured as author
result = classify(rows)
self.assertEqual(len(result["role_mismatches"]), 1)
self.assertEqual(result["role_mismatches"][0]["expected_role"], "author")
self.assertEqual(result["role_mismatches"][0]["declared_role"], "merger")
self.assertFalse(result["mutation_gate_satisfied"])
def test_profile_served_from_the_wrong_namespace_is_reported(self):
rows = healthy_rows()
rows[0]["namespace"] = "gitea-reviewer"
result = classify(rows)
self.assertTrue(
any(
m.get("expected_namespace") == "gitea-author"
for m in result["role_mismatches"]
)
)
self.assertFalse(result["mutation_gate_satisfied"])
def test_matching_binding_produces_no_mismatch(self):
result = classify(healthy_rows())
self.assertEqual(result["repository_binding_mismatches"], [])
self.assertEqual(result["role_mismatches"], [])
class TestDeterminism(_AliveMixin, unittest.TestCase):
"""AC10 / stable ordering and deterministic structured output."""
def test_identical_snapshots_produce_identical_results(self):
rows = healthy_rows()
first = classify(rows, scan=scan_for(rows))
second = classify(healthy_rows(), scan=scan_for(healthy_rows()))
self.assertEqual(first, second)
def test_row_order_does_not_change_the_result(self):
rows = healthy_rows()
shuffled = list(reversed(healthy_rows()))
forward = classify(rows, scan=scan_for(rows))
backward = classify(shuffled, scan=scan_for(shuffled))
self.assertEqual(forward["running_members"], backward["running_members"])
self.assertEqual(forward["configured_members"], backward["configured_members"])
self.assertEqual(
forward["mutation_gate_satisfied"], backward["mutation_gate_satisfied"]
)
def test_members_are_sorted_by_namespace_then_profile_then_pid(self):
rows = healthy_rows()
rows.append(row("gitea-author", "prgs-author", "author", 40001))
result = classify(rows)
keys = [
(m["namespace"], m["profile"], m["pid"]) for m in result["running_members"]
]
self.assertEqual(keys, sorted(keys))
def test_answering_namespace_does_not_change_the_verdict(self):
"""AC10: controller and reconciler agree for one fleet snapshot."""
rows = healthy_rows()
controller = classify(
rows, scan=scan_for(rows), answering_namespace="gitea-controller"
)
reconciler = classify(
healthy_rows(),
scan=scan_for(healthy_rows()),
answering_namespace="gitea-reconciler",
)
self.assertEqual(controller["answering_namespace"], "gitea-controller")
self.assertEqual(reconciler["answering_namespace"], "gitea-reconciler")
for key in sorted(set(controller) - {"answering_namespace"}):
self.assertEqual(controller[key], reconciler[key], f"{key} disagreed")
def test_controller_and_reconciler_agree_on_an_unhealthy_fleet(self):
rows = [r for r in healthy_rows() if r["profile"] != "prgs-merger"]
controller = classify(
rows, scan=scan_for(rows), answering_namespace="gitea-controller"
)
reconciler = classify(
rows, scan=scan_for(rows), answering_namespace="gitea-reconciler"
)
self.assertEqual(controller["blocked_reason"], reconciler["blocked_reason"])
self.assertEqual(
controller["mutation_gate_satisfied"],
reconciler["mutation_gate_satisfied"],
)
class TestReadOnly(_AliveMixin, unittest.TestCase):
"""AC11: the capability mutates nothing."""
def test_classification_reports_no_mutations(self):
result = classify(healthy_rows())
self.assertEqual(result["mutations_performed"], [])
self.assertTrue(result["read_only"])
def test_classification_does_not_write_the_input_rows_back(self):
rows = healthy_rows()
snapshot = [dict(r) for r in rows]
classify(rows)
self.assertEqual(rows, snapshot)
class TestLivenessProbe(unittest.TestCase):
"""AC11: the real probe never sends a terminating signal.
Deliberately *not* using ``_AliveMixin`` — these tests exercise
``probe_pid_alive`` itself, which the mixin replaces.
"""
def test_only_signal_zero_is_ever_sent(self):
with mock.patch("os.kill") as killer:
mfi.probe_pid_alive(4242)
killer.assert_called_once_with(4242, 0)
def test_classifying_a_duplicate_never_terminates_it(self):
rows = healthy_rows()
rows.append(row("gitea-author", "prgs-author", "author", 49999))
with mock.patch("os.kill") as killer:
result = mfi.classify_fleet_inventory(
runtime_rows=rows,
process_scan=scan_for(rows),
expected_binding=BINDING,
now=NOW,
)
self.assertTrue(killer.call_args_list, "liveness must actually be probed")
for call in killer.call_args_list:
self.assertEqual(call.args[1], 0, "only signal 0 may ever be sent")
self.assertEqual(result["mutations_performed"], [])
def test_liveness_probe_tolerates_a_missing_process(self):
with mock.patch("os.kill", side_effect=ProcessLookupError):
self.assertFalse(mfi.probe_pid_alive(4242))
def test_liveness_probe_treats_permission_error_as_alive(self):
with mock.patch("os.kill", side_effect=PermissionError):
self.assertTrue(mfi.probe_pid_alive(4242))
def test_liveness_probe_returns_unknown_on_other_os_errors(self):
with mock.patch("os.kill", side_effect=OSError):
self.assertIsNone(mfi.probe_pid_alive(4242))
def test_invalid_pid_is_unknown_and_probes_nothing(self):
with mock.patch("os.kill") as killer:
self.assertIsNone(mfi.probe_pid_alive(None))
self.assertIsNone(mfi.probe_pid_alive(0))
killer.assert_not_called()
class TestMultiClientRegression(_AliveMixin, unittest.TestCase):
"""The multi-LLM duplicate-server scenario that motivated #949."""
@staticmethod
def _two_client_rows():
first = healthy_rows()
second = [
row(
entry["namespace"],
entry["profile"],
entry["role"],
50000 + index,
cohort_id=COHORT_B,
)
for index, entry in enumerate(mfi.EXPECTED_PRGS_FLEET)
]
return first + second
def test_second_client_running_the_same_five_profiles_is_caught(self):
"""Two clients, ten servers, same profiles, same revision.
Every member self-reports the same parity, which is exactly why the old
per-process surfaces reported success. The fleet inventory must report
five duplicates, two cohorts, and a closed gate.
"""
rows = self._two_client_rows()
result = classify(rows, scan=scan_for(rows))
self.assertEqual(len(result["duplicate_members"]), 5)
self.assertFalse(result["exactly_one_per_profile"])
self.assertTrue(result["mixed_cohort"])
self.assertFalse(result["single_cohort"])
self.assertFalse(result["mixed_revision"], "both clients share a revision")
self.assertFalse(result["mutation_gate_satisfied"])
self.assertEqual(result["missing_members"], [])
def test_identical_parity_across_ten_servers_is_not_health(self):
rows = self._two_client_rows()
result = classify(rows, scan=scan_for(rows))
self.assertEqual(result["startup_revisions"], [HEAD_A])
self.assertFalse(result["mutation_gate_satisfied"])
def test_every_duplicate_pid_is_named_for_the_operator(self):
rows = self._two_client_rows()
result = classify(rows, scan=scan_for(rows))
for duplicate in result["duplicate_members"]:
self.assertEqual(len(duplicate["pids"]), 2)
class TestProcessScan(unittest.TestCase):
"""The process observation is server-side and fails closed."""
def test_scan_parses_mcp_server_processes(self):
stdout = (
" PID STARTED COMMAND\n"
" 41000 Mon Jul 27 20:00:00 2026 python /path/mcp_server.py\n"
" 41001 Mon Jul 27 20:00:01 2026 python /path/other_server.py\n"
)
result = mfi.scan_mcp_server_processes(
runner=lambda *a, **k: mock.Mock(stdout=stdout)
)
self.assertTrue(result["available"])
self.assertEqual([p["pid"] for p in result["processes"]], [41000])
def test_scan_failure_reports_unavailable_rather_than_empty(self):
def boom(*args, **kwargs):
raise OSError("ps missing")
result = mfi.scan_mcp_server_processes(runner=boom)
self.assertFalse(result["available"])
self.assertEqual(result["processes"], [])
self.assertIn("ps missing", result["reason"])
def test_scan_results_are_sorted_by_pid(self):
stdout = (
" PID STARTED COMMAND\n"
" 41005 Mon Jul 27 20:00:00 2026 python /path/mcp_server.py\n"
" 41001 Mon Jul 27 20:00:01 2026 python /path/mcp_server.py\n"
)
result = mfi.scan_mcp_server_processes(
runner=lambda *a, **k: mock.Mock(stdout=stdout)
)
self.assertEqual([p["pid"] for p in result["processes"]], [41001, 41005])
class TestRuntimeRecord(unittest.TestCase):
"""The row a server writes about itself."""
def test_record_describes_the_calling_process(self):
record = mfi.build_process_runtime_record(
namespace="gitea-controller",
profile="prgs-controller",
role="controller",
remote="prgs",
org=BINDING["org"],
repo=BINDING["repo"],
pid=41022,
startup_head=HEAD_A,
transport="stdio",
client_provenance="client_managed",
env={mfi.COHORT_ID_ENV: COHORT_A},
boot_id="deadbeefcafe0001",
)
self.assertEqual(
record["runtime_id"], "gitea-controller:41022:deadbeefcafe0001"
)
self.assertEqual(record["namespace"], "gitea-controller")
self.assertEqual(record["cohort_id"], COHORT_A)
self.assertEqual(record["cohort_source"], "explicit_env")
self.assertEqual(record["startup_head"], HEAD_A)
self.assertEqual(record["status"], "running")
def test_record_defaults_pid_to_the_current_process(self):
record = mfi.build_process_runtime_record(
namespace="gitea-author", profile="prgs-author", role="author"
)
self.assertEqual(record["pid"], os.getpid())
def test_each_boot_gets_a_distinct_runtime_id(self):
first = mfi.build_process_runtime_record(
namespace="gitea-author", profile="prgs-author", role="author", pid=1
)
second = mfi.build_process_runtime_record(
namespace="gitea-author", profile="prgs-author", role="author", pid=1
)
self.assertNotEqual(first["runtime_id"], second["runtime_id"])
def test_registered_at_uses_the_control_plane_timestamp_format(self):
record = mfi.build_process_runtime_record(
namespace="gitea-author", profile="prgs-author", role="author", pid=1
)
datetime.strptime(record["registered_at"], "%Y-%m-%dT%H:%M:%SZ")
class TestSummary(_AliveMixin, unittest.TestCase):
def test_healthy_summary_names_the_counts(self):
result = classify(healthy_rows())
self.assertIn("5 of 5", mfi.summarize(result))
def test_blocked_summary_repeats_the_blocked_reason(self):
rows = [r for r in healthy_rows() if r["profile"] != "prgs-merger"]
result = classify(rows)
self.assertIn(result["blocked_reason"], mfi.summarize(result))
class TestHeartbeatAge(_AliveMixin, unittest.TestCase):
def test_heartbeat_age_is_reported_for_diagnosis(self):
rows = healthy_rows()
stamp = (NOW - timedelta(minutes=30)).strftime("%Y-%m-%dT%H:%M:%SZ")
rows[0]["last_heartbeat_at"] = stamp
result = classify(rows)
member = [m for m in result["running_members"] if m["pid"] == rows[0]["pid"]][0]
self.assertEqual(member["heartbeat_age_seconds"], 1800)
def test_missing_heartbeat_is_reported_as_unknown_age(self):
rows = healthy_rows()
rows[0]["last_heartbeat_at"] = None
result = classify(rows)
member = [m for m in result["running_members"] if m["pid"] == rows[0]["pid"]][0]
self.assertIsNone(member["heartbeat_age_seconds"])
if __name__ == "__main__":
unittest.main()