Reviewer REQUEST_CHANGES on PR #903 at head1c88b87raised five blockers, all reproduced by executing that head. The shared shape: a write path that declared itself gated, audited, and verified, but never armed the gate, mutated a copy of the state it claimed to fix, and then verified against that same copy. B1 - the apply path never asked the execution gate. execute_recovery_playbook called console_authz.authorize with the default for_execution=False, and the phase branch only fires when it is True. ACTIVE_PHASE is 1 and every new action is phase 2, so an operator executed a phase-2 write through POST /api/v1/system/recovery/apply while build_recovery_preview reported execution_enabled false. The call now passes for_execution=True and surfaces the phase_not_active refusal. Preview reports the same decision under execution_authorization / execution_blocked_reason instead of a hardcoded False it could not explain. B2 - both env playbooks mutated a discarded copy and verified against it. source_env = dict(os.environ) meant clear_stale_binding and rebind_session_worktree never touched the running process, and verify_post_recovery(env=source_env) re-diagnosed the same copy, confirming a change that had not happened. Mutations now target the live mapping (apply_recovery's sanctioned env=None -> os.environ path, #702 AC2) and verification re-reads state rather than the mutated input. binding_before / binding_after / binding_changed are returned, and a playbook that changed nothing reports performed: false. verify_post_recovery no longer reads an unverified_inherited binding as clean, because unproven is not clean. B3 - the reconcile playbook called a function that does not exist. merged_cleanup_reconcile.reconcile_merged_cleanups is absent from that module and a bare except turned the AttributeError into a generic failure, so the playbook could never succeed. It now calls gitea_mcp_server.gitea_reconcile_merged_cleanups, the real orchestrator, imported lazily; failures carry error_type. task_capability_map declared gitea.pr.close for reconcile_cleanups while the entry point gates on gitea.read; the two authority statements are reconciled to the one that is enforced. B4 - the #630 contamination integration could not block. assess_contamination_gate was fed marker=None, which short-circuits to block: False on its first statement; the task passed was a console action id outside CONTAMINATION_GATED_TASKS; and the result was read through a "contaminated" key the gate never returns, making STATUS_BLOCKED_CONTAMINATION unreachable. The live marker now comes from the #641 session inventory reader, the gated task key console_recovery_apply is added to CONTAMINATION_GATED_TASKS, every read uses the "block" key the gate actually returns, and the marker is forwarded to sanctioned_restart.execute_restart so a restart cannot launder a contaminated runtime. The reconciler cleanup playbook stays exempt as the designated remedy. B5 - the parity baseline was captured from the head it was compared against. capture_startup_parity(root, head=checkout_head) stores the head verbatim, so in_parity was structurally incapable of being false, and live_remote_head was never passed. The baseline is now the daemon start head that assess_stale_runtime already returns, and the #610 live-remote dimension is restored. Also: _recovery_card was the one renderer in system_health_views.py interpolating without _esc(), and it is where a marker's operator-supplied command_summary lands once B4 is wired; it now escapes, including the except branch. Docs no longer claim apply enforces master parity or that verify asserts clean: true, and the absolute file:///Users/... links are relative. Tests: the two that asserted the defects as intended are inverted - test_api_recovery_apply_with_dev_auth asserted the phase-gate bypass, and the rebind test asserted the input echoed back. Added coverage per blocker, including a no-op detection test that fails when a playbook reports success without changing anything, the previously untested reconcile playbook, contamination block and remedy-exemption tests, and parity baseline/live-remote tests. Both new guards were mutation-verified: disarming for_execution fails 2 tests, restoring the env copy fails 2 tests. Validation: WEBUI_TEST_OFFLINE=1 ../../venv/bin/python -m pytest tests/ -q from branches/feat-issue-644 gives 27 failed / 5291 passed / 6 skipped / 953 subtests; the same command from branches/baseline-master-76f293e at76f293eb28gives 28 failed / 5262 passed / 6 skipped / 926 subtests. Suites run one at a time. comm of the sorted FAILED lines shows no new signature at the head. The single absent signature, test_workspace_guard_alignment.py:: TestRuntimeContextGuardAlignment::test_declared_branches_worktree_passes_when_mcp_root_differs, is suite-order dependent: that file passes 9/9 in isolation at both revisions. Closes #644 Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
337 lines
13 KiB
Python
337 lines
13 KiB
Python
"""HTML views for the system-health dashboard (#639).
|
|
|
|
Renders the read-only :class:`~webui.system_health.SystemHealthSnapshot`
|
|
produced by the Phase 1 system-health API (#634). The page offers no restart,
|
|
reload, or process-kill control: those are Phase 2 work, and manual process
|
|
kills are the contamination path #630 exists to prevent.
|
|
|
|
Every free-text field passes through :func:`webui.system_health.redact` before
|
|
it reaches HTML, so a probe detail that captured a token or a credentialed URL
|
|
cannot leak through the dashboard even though the API redacts it already.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
import html
|
|
|
|
from webui.system_health import (
|
|
STATUS_DEGRADED,
|
|
STATUS_DOWN,
|
|
STATUS_OK,
|
|
STATUS_SKIPPED,
|
|
STATUS_UNPROVEN,
|
|
DependencyProbe,
|
|
SystemHealthSnapshot,
|
|
redact,
|
|
)
|
|
|
|
_STATUS_BADGE_CLASS = {
|
|
STATUS_OK: "badge-health-ok",
|
|
STATUS_DEGRADED: "badge-health-degraded",
|
|
STATUS_DOWN: "badge-health-down",
|
|
STATUS_SKIPPED: "badge-health-skipped",
|
|
STATUS_UNPROVEN: "badge-health-unproven",
|
|
}
|
|
|
|
|
|
def _safe(value: object) -> str:
|
|
"""Escape free text for HTML after redacting anything secret-shaped.
|
|
|
|
Use this for every value that can carry arbitrary text — probe details,
|
|
reasons, probe errors — because those are where a credential could ride
|
|
along.
|
|
"""
|
|
return html.escape(redact(str(value)))
|
|
|
|
|
|
def _esc(value: object) -> str:
|
|
"""Escape a structured field for HTML without redacting it.
|
|
|
|
Commit SHAs, probe names, statuses, and timestamps are enumerated or
|
|
machine-generated, never credential-bearing. They must not go through
|
|
:func:`redact`: its opaque-token rule matches any 32-plus-character run,
|
|
so a 40-character git SHA would render as ``[redacted]`` and the parity
|
|
view — the one thing an operator reads this page for — would be blank.
|
|
"""
|
|
return html.escape(str(value))
|
|
|
|
|
|
def _status_badge(status: str) -> str:
|
|
css = _STATUS_BADGE_CLASS.get(status, "badge-health-unproven")
|
|
return f'<span class="badge {css}">{_esc(status)}</span>'
|
|
|
|
|
|
def _reason_list(reasons: tuple[str, ...], *, empty: str) -> str:
|
|
if not reasons:
|
|
return f"<p class='muted'>{html.escape(empty)}</p>"
|
|
items = "".join(f"<li>{_safe(reason)}</li>" for reason in reasons)
|
|
return f"<ul class='reasons'>{items}</ul>"
|
|
|
|
|
|
def _readiness_card(snapshot: SystemHealthSnapshot) -> str:
|
|
"""Overall readiness.
|
|
|
|
``ready`` and ``readiness_complete`` are shown separately on purpose: a
|
|
snapshot whose required probes never ran is not the same as one that ran
|
|
them and passed, and collapsing the two would render an unproven green.
|
|
"""
|
|
if snapshot.ready and snapshot.readiness_complete:
|
|
headline = "Ready"
|
|
elif snapshot.ready:
|
|
headline = "Ready (incomplete evidence)"
|
|
else:
|
|
headline = "Not ready"
|
|
|
|
return (
|
|
"<section class='health-card'>"
|
|
f"<h3>Readiness {_status_badge(snapshot.status)}</h3>"
|
|
f"<p class='health-headline'>{html.escape(headline)}</p>"
|
|
"<table class='detail'>"
|
|
f"<tr><th>Service</th><td><code>{_esc(snapshot.service)}</code></td></tr>"
|
|
f"<tr><th>Mode</th><td>{_esc(snapshot.mode)}</td></tr>"
|
|
f"<tr><th>Ready</th><td>{_esc(snapshot.ready)}</td></tr>"
|
|
"<tr><th>Readiness evidence complete</th>"
|
|
f"<td>{_esc(snapshot.readiness_complete)}</td></tr>"
|
|
"<tr><th>Deep probes requested</th>"
|
|
f"<td>{_esc(snapshot.deep_probes_requested)}</td></tr>"
|
|
f"<tr><th>Observed at</th><td><code>{_esc(snapshot.timestamp)}</code></td></tr>"
|
|
"</table>"
|
|
"<h4>Readiness reasons</h4>"
|
|
f"{_reason_list(snapshot.readiness_reasons, empty='No readiness objections recorded.')}"
|
|
"</section>"
|
|
)
|
|
|
|
|
|
def _version_card(snapshot: SystemHealthSnapshot) -> str:
|
|
version = snapshot.version
|
|
uptime_hours = snapshot.uptime_seconds / 3600.0
|
|
known = (
|
|
"resolved"
|
|
if version.known
|
|
else "unresolved — version fields could not be read from the checkout"
|
|
)
|
|
schema = version.control_plane_schema_version
|
|
return (
|
|
"<section class='health-card'>"
|
|
"<h3>Version and uptime</h3>"
|
|
"<table class='detail'>"
|
|
f"<tr><th>Git SHA</th><td><code>{_esc(version.git_sha or 'unknown')}</code></td></tr>"
|
|
"<tr><th>Git describe</th>"
|
|
f"<td><code>{_esc(version.git_describe or 'unknown')}</code></td></tr>"
|
|
"<tr><th>Control-plane schema</th>"
|
|
f"<td>{_esc(schema if schema is not None else 'unknown')}</td></tr>"
|
|
f"<tr><th>Python</th><td><code>{_esc(version.python_version)}</code></td></tr>"
|
|
f"<tr><th>Version status</th><td>{html.escape(known)}</td></tr>"
|
|
f"<tr><th>Started at</th><td><code>{_esc(snapshot.started_at)}</code></td></tr>"
|
|
"<tr><th>Uptime</th>"
|
|
f"<td>{snapshot.uptime_seconds:.3f}s ({uptime_hours:.2f}h)</td></tr>"
|
|
"</table>"
|
|
"</section>"
|
|
)
|
|
|
|
|
|
def _dependency_rows(probes: tuple[DependencyProbe, ...]) -> str:
|
|
if not probes:
|
|
return "<p class='muted'>No dependency probes were reported.</p>"
|
|
rows = []
|
|
for probe in probes:
|
|
latency = (
|
|
f"{probe.latency_ms:.1f} ms" if probe.latency_ms is not None else "n/a"
|
|
)
|
|
rows.append(
|
|
"<tr>"
|
|
f"<td><code>{_esc(probe.name)}</code></td>"
|
|
f"<td>{_esc(probe.kind)}</td>"
|
|
f"<td>{_status_badge(probe.status)}</td>"
|
|
f"<td>{_esc('required' if probe.required else 'optional')}</td>"
|
|
f"<td>{html.escape(latency)}</td>"
|
|
f"<td>{_safe(probe.detail)}</td>"
|
|
"</tr>"
|
|
)
|
|
return (
|
|
"<table class='registry'><thead><tr>"
|
|
"<th>Dependency</th><th>Kind</th><th>Status</th><th>Requirement</th>"
|
|
"<th>Latency</th><th>Detail</th>"
|
|
"</tr></thead><tbody>"
|
|
f"{''.join(rows)}</tbody></table>"
|
|
)
|
|
|
|
|
|
def _dependency_card(snapshot: SystemHealthSnapshot) -> str:
|
|
degraded = [probe for probe in snapshot.dependencies if probe.ran and not probe.healthy]
|
|
not_run = [probe for probe in snapshot.dependencies if not probe.ran]
|
|
|
|
banner = ""
|
|
if degraded:
|
|
names = ", ".join(sorted(probe.name for probe in degraded))
|
|
banner += (
|
|
"<div class='stub health-degraded'><p><strong>Degraded dependencies:</strong> "
|
|
f"{_esc(names)}</p></div>"
|
|
)
|
|
if not_run:
|
|
names = ", ".join(sorted(probe.name for probe in not_run))
|
|
banner += (
|
|
"<div class='stub'><p><strong>Not probed:</strong> "
|
|
f"{_esc(names)} — these contribute no evidence and are not counted "
|
|
"as healthy.</p></div>"
|
|
)
|
|
|
|
return (
|
|
"<section class='health-card'>"
|
|
"<h3>Dependencies</h3>"
|
|
f"{banner}"
|
|
f"{_dependency_rows(snapshot.dependencies)}"
|
|
"<p class='muted'>Details are redacted at the API boundary and again "
|
|
"before rendering; credentials are never displayed.</p>"
|
|
"</section>"
|
|
)
|
|
|
|
|
|
def _namespace_card(snapshot: SystemHealthSnapshot) -> str:
|
|
if not snapshot.mcp_namespaces:
|
|
body = "<p class='muted'>No MCP namespaces are declared.</p>"
|
|
else:
|
|
rows = []
|
|
for entry in snapshot.mcp_namespaces:
|
|
rows.append(
|
|
"<tr>"
|
|
f"<td><code>{_esc(entry.get('namespace'))}</code></td>"
|
|
f"<td><code>{_esc(entry.get('required_tool'))}</code></td>"
|
|
f"<td>{_status_badge(str(entry.get('status') or STATUS_UNPROVEN))}</td>"
|
|
f"<td>{_esc(entry.get('ide_namespace_proven'))}</td>"
|
|
f"<td>{_safe(entry.get('reason'))}</td>"
|
|
"</tr>"
|
|
)
|
|
body = (
|
|
"<table class='registry'><thead><tr>"
|
|
"<th>Namespace</th><th>Required tool</th><th>Status</th>"
|
|
"<th>IDE-proven</th><th>Reason</th>"
|
|
"</tr></thead><tbody>"
|
|
f"{''.join(rows)}</tbody></table>"
|
|
)
|
|
return (
|
|
"<section class='health-card'>"
|
|
"<h3>MCP namespaces</h3>"
|
|
f"{body}"
|
|
"<p class='muted'>The web process runs outside the IDE-managed MCP "
|
|
"client, so namespace health is reported as unproven rather than "
|
|
"guessed (#543).</p>"
|
|
"</section>"
|
|
)
|
|
|
|
|
|
def _stale_runtime_card(snapshot: SystemHealthSnapshot) -> str:
|
|
stale = snapshot.stale_runtime
|
|
if stale.stale:
|
|
warning = (
|
|
"<div class='stub health-stale'><p><strong>Stale runtime:</strong> "
|
|
"the running code, the checkout, and the remote-tracking commit "
|
|
"disagree. Capability gates may be evaluating obsolete code — "
|
|
"do not treat this runtime as mutation-safe.</p></div>"
|
|
)
|
|
elif not stale.determinable:
|
|
warning = (
|
|
"<div class='stub health-stale'><p><strong>Staleness "
|
|
"indeterminate:</strong> parity could not be proven, so this "
|
|
"runtime is not reported as mutation-safe.</p></div>"
|
|
)
|
|
else:
|
|
warning = ""
|
|
|
|
return (
|
|
"<section class='health-card'>"
|
|
"<h3>Stale-runtime parity</h3>"
|
|
f"{warning}"
|
|
"<table class='detail'>"
|
|
"<tr><th>Daemon head</th>"
|
|
f"<td><code>{_esc(stale.daemon_head or 'unknown')}</code></td></tr>"
|
|
"<tr><th>Checkout head</th>"
|
|
f"<td><code>{_esc(stale.checkout_head or 'unknown')}</code></td></tr>"
|
|
"<tr><th>Remote head</th>"
|
|
f"<td><code>{_esc(stale.remote_head or 'unknown')}</code></td></tr>"
|
|
f"<tr><th>Stale</th><td>{_esc(stale.stale)}</td></tr>"
|
|
f"<tr><th>Determinable</th><td>{_esc(stale.determinable)}</td></tr>"
|
|
f"<tr><th>Mutation safe</th><td>{_esc(stale.mutation_safe)}</td></tr>"
|
|
"</table>"
|
|
f"{_reason_list(stale.reasons, empty='Runtime, checkout, and remote agree.')}"
|
|
"</section>"
|
|
)
|
|
|
|
|
|
def _probe_error_card(snapshot: SystemHealthSnapshot) -> str:
|
|
if not snapshot.probe_errors:
|
|
return ""
|
|
return (
|
|
"<section class='health-card'>"
|
|
"<h3>Probe errors</h3>"
|
|
f"{_reason_list(snapshot.probe_errors, empty='')}"
|
|
"</section>"
|
|
)
|
|
|
|
|
|
def _recovery_card() -> str:
|
|
"""Sanctioned recovery controls & playbooks (#644, Phase 2)."""
|
|
try:
|
|
from webui import console_recovery
|
|
diag = console_recovery.diagnose_recovery()
|
|
# Every other card in this file escapes at the interpolation boundary.
|
|
# This one did not, and it is where a #630 marker's operator-supplied
|
|
# command_summary lands once the contamination gate is wired.
|
|
status_badge = (
|
|
f"<span class='status-pill {_esc(diag.status)}'>{_esc(diag.status)}</span>"
|
|
)
|
|
playbook_lis = ""
|
|
for pb in diag.playbooks:
|
|
elig = "eligible" if pb.eligible else "disabled"
|
|
playbook_lis += (
|
|
f"<li><strong>{_esc(pb.label)}</strong> "
|
|
f"(<code>{_esc(pb.playbook_id)}</code>) — "
|
|
f"<span class='badge {elig}'>{elig}</span>: {_esc(pb.description)} "
|
|
f"<em class='muted'>({_esc(pb.reason)})</em></li>"
|
|
)
|
|
reasons_html = ""
|
|
if diag.reasons:
|
|
items = "".join(f"<li>{_esc(r)}</li>" for r in diag.reasons)
|
|
reasons_html = f"<ul class='reasons'>{items}</ul>"
|
|
else:
|
|
reasons_html = "<p class='clean-note'>No recovery actions currently required. Control plane is healthy.</p>"
|
|
|
|
return (
|
|
"<section class='health-card recovery-card'>"
|
|
f"<h3>Sanctioned Recovery Controls (Phase 2 #644) {status_badge}</h3>"
|
|
"<p class='muted'>Guided recovery wizard: Diagnose → Preview → Confirm → Verify. "
|
|
"Reconnect the MCP client from the IDE, then re-run the blocked cycle. "
|
|
"Never kill the daemon process manually: unmanaged kills are recorded as runtime contamination (#630).</p>"
|
|
f"{reasons_html}"
|
|
"<h4>Available Recovery Playbooks</h4>"
|
|
f"<ul class='playbooks-list'>{playbook_lis}</ul>"
|
|
"<p class='meta'>APIs: <code>/api/v1/system/recovery/diagnose</code>, "
|
|
"<code>/api/v1/system/recovery/preview</code>, <code>/api/v1/system/recovery/apply</code>, "
|
|
"<code>/api/v1/system/recovery/verify</code>.</p>"
|
|
"</section>"
|
|
)
|
|
except Exception as exc:
|
|
return (
|
|
"<section class='health-card'>"
|
|
"<h3>Sanctioned Recovery Controls (Phase 2 #644)</h3>"
|
|
f"<p class='error'>Recovery diagnostics unavailable: {_esc(exc)}</p>"
|
|
"</section>"
|
|
)
|
|
|
|
|
|
def render_system_health_page(snapshot: SystemHealthSnapshot) -> str:
|
|
"""Render the full system-health dashboard body."""
|
|
return (
|
|
"<h2>System health</h2>"
|
|
"<p class='meta'>Read-only view of the Phase 1 system-health API "
|
|
"(<code>/api/v1/system/health</code>). Reload this page to refresh; "
|
|
"nothing here polls or mutates on your behalf.</p>"
|
|
f"{_readiness_card(snapshot)}"
|
|
f"{_stale_runtime_card(snapshot)}"
|
|
f"{_version_card(snapshot)}"
|
|
f"{_dependency_card(snapshot)}"
|
|
f"{_namespace_card(snapshot)}"
|
|
f"{_probe_error_card(snapshot)}"
|
|
f"{_recovery_card()}"
|
|
)
|