fix(leases): align stale-lease inspection, dashboard, and cleanup enforcement #952

New Issue

2026-07-27T21:38:44-05:00

jcwalker3 commented

2026-07-27 21:38:44 -05:00

Related: #601, #691, #790, #792, #793, #755, #753, #772, #747, #745, #702, #947, #949, #951, #636, #937

Problem

For one authoritative workflow-lease state, the sanctioned native surfaces return mutually incompatible cleanup guidance. The lease inspector advertises an action the executor cannot perform, the dashboard omits a lease it has already classified as dead-process stale, and capability resolution reports the cleanup tool as ready without distinguishing "the tool is permitted" from "this target is eligible."

This is a diagnostic and workflow-contract defect. It does not prove the executor should have abandoned the lease. The executor's refusal was correct; the inspector's and dashboard's answers were not consistent with it.

Observed evidence

Recorded natively during a read-only assessment of the dead-process lease associated with PR #906. The assessment intentionally performed no cleanup, and no lease was mutated.

Target lease at the time of assessment:

Dimension	Value
Lease ID	`lease-6d38e6cb69904976`
Repository	`Scaled-Tech-Consulting/Gitea-Tools`
Work type and number	PR #906
Associated issue	#650
Role	`author`
Owning profile	`prgs-author`
Owning session	`prgs-author-92989-cd7e0310`
Status	`active`
Freshness	`stale_dead_process`
Native process evidence	`owner_pid_alive=false`
Evidence source	authoritative control-plane state, not caller-supplied process evidence
Expiration observed	`2026-07-28T03:52:06Z`
PR #906 state	open
`worktree_path`	`null`
`worktree_present`	`null`
Adoption / provenance / abandonment proof	none recorded

Contract contradiction

Four sanctioned native surfaces disagreed about the same lease:

gitea_inspect_workflow_lease reported safe_next_action="abandon_allowed" with block=false.
The workflow dashboard did not include the lease in stale_or_expired_leases.
Capability resolution reported abandon_workflow_lease as configured, allowed, and ready.
The executor's foreign-abandon contract could not authorize the operation. It required either an authenticated operator authorization path, or dead_process AND missing_worktree AND no_open_pr.

Only dead_process was proven:

missing_worktree was unknown, not true — worktree_path and worktree_present were both null.
no_open_pr was false — PR #906 remained open.

So the inspector advertised an action the executor could not safely perform, while the dashboard omitted a lease already classified as dead-process stale.

Required behavior

For one authoritative lease state, inspection, dashboard reporting, capability resolution, canonical workflow guidance, and mutation execution must consume a single consistent decision model.

That model must distinguish at least:

Active and healthy
Active but owner process dead
Stale but not yet expired
Expired
Orphaned
Reclaimable
Abandonable by the owner
Abandonable by a foreign reconciler
Blocked by an open PR
Blocked because worktree state is unknown
Blocked because a live workflow dependency may remain
Awaiting authenticated operator authorization
Cleanup unsupported
Cleanup evidence incomplete

safe_next_action="abandon_allowed" must be returned only when the exact executor can perform the action using evidence already available through the sanctioned contract.

When additional proof or authorization is required, inspection must say so explicitly. Example dispositions:

abandon_blocked_open_pr
missing_worktree_evidence_required
operator_authorization_required
wait_for_expiration
reclaim_after_expiration
live_dependency_unknown

Exact names may follow existing repository conventions; equivalent structure is required.

Safety requirements

Do not weaken foreign-lease abandonment protections merely to make the surfaces agree. The fix is to correct the advertised disposition, not to relax the executor.
worktree_path=null or worktree_present=null means unknown, never absent, unless authoritative discovery proves absence.
An open PR must not be silently treated as proof of either a live owner or a safely abandoned owner.
Dead-process evidence must come from an authoritative source, never from caller-supplied process lists.
Caller-supplied operator_authorized=true must not function as an unauthenticated force-clear.
Any operator override must be represented by an authenticated, role-authorized, auditable control-plane operation.
Inspection must report the exact unsatisfied executor predicates rather than a single collapsed verdict.
Capability resolution must distinguish "tool permitted" from "specific target currently eligible."
Expiration and reclamation paths must be explicit and observable.
A stale lease omitted from a cleanup queue must carry a structured reason for the omission.
No read-only inspection call may mutate the lease.

Acceptance criteria

Inspector, dashboard, resolver, workflow recommendation, and executor use one authoritative eligibility model.
The same lease cannot simultaneously be reported as abandon_allowed and ineligible for abandonment.
Capability permission and target eligibility are reported separately.
safe_next_action identifies an action executable under the current evidence.
All unsatisfied cleanup predicates are returned structurally.
Dead-process stale leases appear in the dashboard even when they are not yet reclaimable.
Dashboard entries distinguish actionable leases from visible-but-blocked leases.
Null worktree evidence is classified as unknown, not missing.
An open PR produces the documented preservation or escalation result.
The owning-lease and foreign-lease paths are distinguished.
Operator authorization is authenticated, role-checked, target-specific, and auditable.
A caller cannot fabricate authorization with an unverified Boolean.
When waiting for expiry is the canonical action, the result identifies the expiration timestamp and the sanctioned follow-up capability.
After expiry, reclamation eligibility is reported consistently across every native surface.
Cleanup cannot affect unrelated leases.
Read-only inspection and dashboard calls perform no mutation.
Automated tests cover the full matrix of process state, expiration, PR state, worktree evidence, ownership, and operator authorization.
Regression tests reproduce the PR #906 evidence combination: status=active, freshness=stale_dead_process, owner_pid_alive=false, open PR, worktree_path=null, worktree_present=null, no adoption or provenance proof.
Canonical workflow documentation explains permission versus target eligibility.
An LLM can determine the safe action using sanctioned native calls without attempting a forbidden cleanup.

Non-goals

Clearing the specific PR #906 lease
Closing or modifying PR #906 or issue Web Console: AI-provider connections and evidence-backed operational insights (Phase 4) (#650)
Weakening foreign-abandon safety gates
Treating every dead-process lease as immediately safe to remove
Implementing fleet inventory from issue #949
Implementing controller role alignment from issue #950
Implementing restart receipts from issue #951
Restarting or reconciling the MCP fleet
Resuming issue Remote-MCP 02: Introduce a transport-neutral bind seam accepting a remote transport (#931)
Changing normal PR review or merge rules

Duplicate verdict

NOT a duplicate. Verified by reading the bodies and acceptance criteria of #601, #691, #790, #792, #793, #755, #636, #947, #949, and #951.

Issue	State	Why it does not cover this
#691	closed	Closest neighbour. Owns the missing non-owner cleanup path for a comment-only reviewer lease pinned to a superseded head on an open PR, and records that DB foreign-abandon proof was unsatisfiable there. But its defect is "no cleanup path exists at all, so the diagnostic can only say wait" — the diagnostic and executor agreed. Here they disagree: the inspector affirmatively advertised `abandon_allowed` for a lease the executor's own predicates refuse. #691 also concerns a comment lease with no `lease_id` in the control-plane DB; this lease has one (`lease-6d38e6cb69904976`). #691 defines no dashboard-inclusion requirement and no permission-versus-target-eligibility separation.
#601	closed	Established canonical adopt / release / expire / abandon handling and required a "safe next action" field. It made the surfaces exist; it never required that the inspector's `safe_next_action` be executable by the specific executor that would run it, nor that stale leases appear in cleanup queues with structured omission reasons. This issue constrains the consistency of the surfaces #601 created.
#790	closed	Shared heartbeat lease lifecycle and safe terminal dead-owner recovery — the acquire/renew/expire/reclaim/release semantics. Slice A only. It governs how state transitions, not how the diagnostic, dashboard, resolver, and executor report agreement about one state.
#792	open	Slice B: terminal retirement of author issue leases, and it is explicitly scoped to work whose PR has completed integration and whose issue is closed. PR #906 is open and issue #650 is open, so the terminal-retirement path does not reach this lease at all.
#793	open	Slice C: generation fencing on the read side, re-scoping the #760 renewal path, and migrating remaining task classes onto the shared lifecycle. Fencing correctness, not cross-surface diagnostic consistency. Neither AC-C1 through AC-C8 touches inspector disposition, dashboard membership, or resolver eligibility reporting.
#755 / #753 / #772	closed	The dead-session author lock recovery family, all clean-worktree paths, concerned with re-establishing ownership so work can continue. This issue is about cleanup guidance for a lease nobody is resuming, and specifically about null worktree evidence being misread.
#947	open	A different closed set: the dirty same-claimant rebind writes `rebind_record` that no enforcement path consumes, so continuation evidence cannot be minted. That is a missing-evidence-writer defect on the author continuation path. This issue is a disagreement between existing readers of evidence that is already present.
#636	closed	Web Console HTTP inventory API. Not a native MCP surface; a session under the native-only rail cannot call it, and its non-goals exclude lease mutations entirely.
#949	open	Authoritative native fleet inventory and duplicate-process detection. Its non-goals name "Clearing workflow leases" and "Cleaning the stale PR #906 author lease" explicitly, so it disclaims this scope by construction.
#951	open	Durable synchronization and restart receipts. Its non-goals likewise name "Automatically clearing workflow leases" and "Cleaning the stale PR #906 author lease" explicitly. It records provenance of fleet operations, not lease cleanup eligibility.

Canonical issue state

STATE: ready-for-author
WHO_IS_NEXT: author
NEXT_ACTION: Implement a single authoritative lease-eligibility model consumed by the inspector, dashboard, capability resolver, canonical workflow guidance, and the abandon/reclaim executors, per the acceptance criteria
NEXT_PROMPT:
Role: AUTHOR
Repo: prgs / Scaled-Tech-Consulting / Gitea-Tools
Issue: this issue

1. Native MCP only; gitea_whoami + gitea_resolve_task_capability(work_issue) on prgs-author; bind an issue-backed worktree under branches/.
2. Define one authoritative lease-eligibility model covering the full state list, including unknown-worktree and open-PR blocked states, and make it the single source for inspector disposition, dashboard membership, resolver target eligibility, workflow recommendation, and executor predicates.
3. Return unsatisfied executor predicates structurally; separate tool permission from target eligibility; classify null worktree evidence as unknown; never weaken the foreign-abandon gates.
4. Require authenticated, role-checked, target-specific, auditable operator authorization; reject unverified caller Booleans.
5. Add regression tests for the full matrix and reproduce the PR #906 evidence combination exactly.
6. Update canonical workflow documentation on permission versus target eligibility.
7. Open a PR with Closes; stop; hand off to an independent reviewer.

Related: #601, #691, #790, #792, #793, #755, #753, #772, #747, #745, #702, #947, #949, #951, #636, #937 ## Problem For one authoritative workflow-lease state, the sanctioned native surfaces return mutually incompatible cleanup guidance. The lease inspector advertises an action the executor cannot perform, the dashboard omits a lease it has already classified as dead-process stale, and capability resolution reports the cleanup tool as ready without distinguishing "the tool is permitted" from "this target is eligible." This is a diagnostic and workflow-contract defect. It does **not** prove the executor should have abandoned the lease. The executor's refusal was correct; the inspector's and dashboard's answers were not consistent with it. ## Observed evidence Recorded natively during a read-only assessment of the dead-process lease associated with PR #906. The assessment intentionally performed **no cleanup**, and no lease was mutated. Target lease at the time of assessment: | Dimension | Value | | --- | --- | | Lease ID | `lease-6d38e6cb69904976` | | Repository | `Scaled-Tech-Consulting/Gitea-Tools` | | Work type and number | PR #906 | | Associated issue | #650 | | Role | `author` | | Owning profile | `prgs-author` | | Owning session | `prgs-author-92989-cd7e0310` | | Status | `active` | | Freshness | `stale_dead_process` | | Native process evidence | `owner_pid_alive=false` | | Evidence source | authoritative control-plane state, not caller-supplied process evidence | | Expiration observed | `2026-07-28T03:52:06Z` | | PR #906 state | open | | `worktree_path` | `null` | | `worktree_present` | `null` | | Adoption / provenance / abandonment proof | none recorded | ## Contract contradiction Four sanctioned native surfaces disagreed about the same lease: 1. `gitea_inspect_workflow_lease` reported `safe_next_action="abandon_allowed"` with `block=false`. 2. The workflow dashboard did **not** include the lease in `stale_or_expired_leases`. 3. Capability resolution reported `abandon_workflow_lease` as configured, allowed, and ready. 4. The executor's foreign-abandon contract could not authorize the operation. It required either an authenticated operator authorization path, or `dead_process AND missing_worktree AND no_open_pr`. Only `dead_process` was proven: * `missing_worktree` was **unknown**, not true — `worktree_path` and `worktree_present` were both `null`. * `no_open_pr` was **false** — PR #906 remained open. So the inspector advertised an action the executor could not safely perform, while the dashboard omitted a lease already classified as dead-process stale. ## Required behavior For one authoritative lease state, inspection, dashboard reporting, capability resolution, canonical workflow guidance, and mutation execution must consume a single consistent decision model. That model must distinguish at least: * Active and healthy * Active but owner process dead * Stale but not yet expired * Expired * Orphaned * Reclaimable * Abandonable by the owner * Abandonable by a foreign reconciler * Blocked by an open PR * Blocked because worktree state is unknown * Blocked because a live workflow dependency may remain * Awaiting authenticated operator authorization * Cleanup unsupported * Cleanup evidence incomplete `safe_next_action="abandon_allowed"` must be returned only when the exact executor can perform the action using evidence already available through the sanctioned contract. When additional proof or authorization is required, inspection must say so explicitly. Example dispositions: * `abandon_blocked_open_pr` * `missing_worktree_evidence_required` * `operator_authorization_required` * `wait_for_expiration` * `reclaim_after_expiration` * `live_dependency_unknown` Exact names may follow existing repository conventions; equivalent structure is required. ## Safety requirements * Do not weaken foreign-lease abandonment protections merely to make the surfaces agree. The fix is to correct the advertised disposition, not to relax the executor. * `worktree_path=null` or `worktree_present=null` means **unknown**, never **absent**, unless authoritative discovery proves absence. * An open PR must not be silently treated as proof of either a live owner or a safely abandoned owner. * Dead-process evidence must come from an authoritative source, never from caller-supplied process lists. * Caller-supplied `operator_authorized=true` must not function as an unauthenticated force-clear. * Any operator override must be represented by an authenticated, role-authorized, auditable control-plane operation. * Inspection must report the exact unsatisfied executor predicates rather than a single collapsed verdict. * Capability resolution must distinguish "tool permitted" from "specific target currently eligible." * Expiration and reclamation paths must be explicit and observable. * A stale lease omitted from a cleanup queue must carry a structured reason for the omission. * No read-only inspection call may mutate the lease. ## Acceptance criteria 1. Inspector, dashboard, resolver, workflow recommendation, and executor use one authoritative eligibility model. 2. The same lease cannot simultaneously be reported as `abandon_allowed` and ineligible for abandonment. 3. Capability permission and target eligibility are reported separately. 4. `safe_next_action` identifies an action executable under the current evidence. 5. All unsatisfied cleanup predicates are returned structurally. 6. Dead-process stale leases appear in the dashboard even when they are not yet reclaimable. 7. Dashboard entries distinguish actionable leases from visible-but-blocked leases. 8. Null worktree evidence is classified as unknown, not missing. 9. An open PR produces the documented preservation or escalation result. 10. The owning-lease and foreign-lease paths are distinguished. 11. Operator authorization is authenticated, role-checked, target-specific, and auditable. 12. A caller cannot fabricate authorization with an unverified Boolean. 13. When waiting for expiry is the canonical action, the result identifies the expiration timestamp and the sanctioned follow-up capability. 14. After expiry, reclamation eligibility is reported consistently across every native surface. 15. Cleanup cannot affect unrelated leases. 16. Read-only inspection and dashboard calls perform no mutation. 17. Automated tests cover the full matrix of process state, expiration, PR state, worktree evidence, ownership, and operator authorization. 18. Regression tests reproduce the PR #906 evidence combination: `status=active`, `freshness=stale_dead_process`, `owner_pid_alive=false`, open PR, `worktree_path=null`, `worktree_present=null`, no adoption or provenance proof. 19. Canonical workflow documentation explains permission versus target eligibility. 20. An LLM can determine the safe action using sanctioned native calls without attempting a forbidden cleanup. ## Non-goals * Clearing the specific PR #906 lease * Closing or modifying PR #906 or issue #650 * Weakening foreign-abandon safety gates * Treating every dead-process lease as immediately safe to remove * Implementing fleet inventory from issue #949 * Implementing controller role alignment from issue #950 * Implementing restart receipts from issue #951 * Restarting or reconciling the MCP fleet * Resuming issue #931 * Changing normal PR review or merge rules ## Duplicate verdict NOT a duplicate. Verified by reading the bodies and acceptance criteria of #601, #691, #790, #792, #793, #755, #636, #947, #949, and #951. | Issue | State | Why it does not cover this | | --- | --- | --- | | #691 | closed | Closest neighbour. Owns the missing **non-owner cleanup path** for a comment-only reviewer lease pinned to a superseded head on an open PR, and records that DB foreign-abandon proof was unsatisfiable there. But its defect is "no cleanup path exists at all, so the diagnostic can only say wait" — the diagnostic and executor **agreed**. Here they disagree: the inspector affirmatively advertised `abandon_allowed` for a lease the executor's own predicates refuse. #691 also concerns a comment lease with no `lease_id` in the control-plane DB; this lease has one (`lease-6d38e6cb69904976`). #691 defines no dashboard-inclusion requirement and no permission-versus-target-eligibility separation. | | #601 | closed | Established canonical adopt / release / expire / abandon handling and required a "safe next action" field. It made the surfaces exist; it never required that the inspector's `safe_next_action` be executable by the specific executor that would run it, nor that stale leases appear in cleanup queues with structured omission reasons. This issue constrains the consistency of the surfaces #601 created. | | #790 | closed | Shared heartbeat lease lifecycle and safe terminal dead-owner recovery — the acquire/renew/expire/reclaim/release **semantics**. Slice A only. It governs how state transitions, not how the diagnostic, dashboard, resolver, and executor report agreement about one state. | | #792 | open | Slice B: terminal retirement of author issue leases, and it is explicitly scoped to work whose PR has **completed integration** and whose issue is **closed**. PR #906 is open and issue #650 is open, so the terminal-retirement path does not reach this lease at all. | | #793 | open | Slice C: generation fencing on the read side, re-scoping the #760 renewal path, and migrating remaining task classes onto the shared lifecycle. Fencing correctness, not cross-surface diagnostic consistency. Neither AC-C1 through AC-C8 touches inspector disposition, dashboard membership, or resolver eligibility reporting. | | #755 / #753 / #772 | closed | The dead-session **author lock** recovery family, all clean-worktree paths, concerned with re-establishing ownership so work can continue. This issue is about cleanup guidance for a lease nobody is resuming, and specifically about null worktree evidence being misread. | | #947 | open | A different closed set: the dirty same-claimant rebind writes `rebind_record` that no enforcement path consumes, so continuation evidence cannot be minted. That is a missing-evidence-writer defect on the author continuation path. This issue is a disagreement between existing readers of evidence that is already present. | | #636 | closed | Web Console HTTP inventory API. Not a native MCP surface; a session under the native-only rail cannot call it, and its non-goals exclude lease mutations entirely. | | #949 | open | Authoritative native fleet inventory and duplicate-process detection. Its non-goals name "Clearing workflow leases" and "Cleaning the stale PR #906 author lease" explicitly, so it disclaims this scope by construction. | | #951 | open | Durable synchronization and restart receipts. Its non-goals likewise name "Automatically clearing workflow leases" and "Cleaning the stale PR #906 author lease" explicitly. It records provenance of fleet operations, not lease cleanup eligibility. | ## Canonical issue state ```text STATE: ready-for-author WHO_IS_NEXT: author NEXT_ACTION: Implement a single authoritative lease-eligibility model consumed by the inspector, dashboard, capability resolver, canonical workflow guidance, and the abandon/reclaim executors, per the acceptance criteria NEXT_PROMPT: Role: AUTHOR Repo: prgs / Scaled-Tech-Consulting / Gitea-Tools Issue: this issue 1. Native MCP only; gitea_whoami + gitea_resolve_task_capability(work_issue) on prgs-author; bind an issue-backed worktree under branches/. 2. Define one authoritative lease-eligibility model covering the full state list, including unknown-worktree and open-PR blocked states, and make it the single source for inspector disposition, dashboard membership, resolver target eligibility, workflow recommendation, and executor predicates. 3. Return unsatisfied executor predicates structurally; separate tool permission from target eligibility; classify null worktree evidence as unknown; never weaken the foreign-abandon gates. 4. Require authenticated, role-checked, target-specific, auditable operator authorization; reject unverified caller Booleans. 5. Add regression tests for the full matrix and reproduce the PR #906 evidence combination exactly. 6. Update canonical workflow documentation on permission versus target eligibility. 7. Open a PR with Closes; stop; hand off to an independent reviewer. ```