Centralized management for scheduled multi-LLM workers #797

New Issue

2026-07-22T05:17:17-05:00

jcwalker3 commented

2026-07-22 05:17:17 -05:00

Problem

We are operating multiple scheduled LLM workers for Gitea-Tools, including Claude, Grok, Codex, AGY, and Kimi K. These workers may perform distinct author, reviewer, merger, reconciler, cleanup, or other sanctioned workflow roles.

The workers currently use separate processes, schedules, logs, locks, prompts, MCP namespaces, and LaunchAgents. We need one safe control surface for configuring, monitoring, and managing the complete worker fleet without weakening role separation.

Every implementation task must begin with an existing open Gitea issue. Scheduled author workers must never invent and implement untracked work during the same cycle.

Required scope

Maintain a central registry for all scheduled workers.
Record each worker’s:
- LLM and worker name
- assigned lane and role
- MCP namespace and profile
- executable
- schedule
- timeout
- LaunchAgent label
- enabled or paused state
Provide safe operations for:
- status
- logs
- run-now
- pause
- resume
- restart
- doctor
Display:
- current issue and PR
- branch and worktree
- PID and elapsed runtime
- last start and completion time
- last terminal classification
- next scheduled run
Standardize terminal classifications:
- COMPLETE
- NO_WORK
- BLOCKED
- TIMEOUT
- FAILED
Detect:
- overlapping worker ownership
- conflicting lanes
- schedule collisions
- stale locks
- dead processes
- missing executables
- unhealthy MCP namespaces
- wrong profiles or identities
- runtime/master drift
Preserve strict author, reviewer, merger, and reconciler boundaries.
Fail closed on wrong-role access or ambiguous task ownership.
Keep credentials and tokens inaccessible to LLM processes.
Produce machine-readable run records and human-readable logs.
Support safe installation, validation, upgrades, rollback, and removal.
Continue using isolated worker processes while managing them through a unified control surface.

Issue-first workflow requirements

An author cycle may work only on an existing open issue selected through canonical routing.
An author cycle may not create a new issue and implement it in the same cycle.
Issue-intake cycles may search for and create issues but may not acquire an implementation lock or worktree.
Every author implementation requires a durable issue lock and validated author worktree.
Reviewer, merger, reconciler, and cleanup work must use their dedicated namespaces and workflows.
All mutations require read-after-write verification.

Acceptance criteria

All five LLM workers can be inspected through one control surface.
Worker schedules and responsibilities can be validated before activation.
Operators can safely pause, resume, restart, and run individual workers.
Current work and recent outcomes are visible for every worker.
Duplicate ownership, conflicting roles, and stale processes are detected.
Wrong-role operations fail closed.
Credentials are never exposed to prompts, logs, or LLM subprocesses.
Author workers reject tasks that lack an existing open issue.
Automated tests cover configuration validation, role enforcement, scheduling conflicts, locks, health checks, and process controls.
Operator documentation covers installation, daily operation, recovery, rollback, and removal.

## Problem We are operating multiple scheduled LLM workers for Gitea-Tools, including Claude, Grok, Codex, AGY, and Kimi K. These workers may perform distinct author, reviewer, merger, reconciler, cleanup, or other sanctioned workflow roles. The workers currently use separate processes, schedules, logs, locks, prompts, MCP namespaces, and LaunchAgents. We need one safe control surface for configuring, monitoring, and managing the complete worker fleet without weakening role separation. Every implementation task must begin with an existing open Gitea issue. Scheduled author workers must never invent and implement untracked work during the same cycle. ## Required scope - Maintain a central registry for all scheduled workers. - Record each worker’s: - LLM and worker name - assigned lane and role - MCP namespace and profile - executable - schedule - timeout - LaunchAgent label - enabled or paused state - Provide safe operations for: - status - logs - run-now - pause - resume - restart - doctor - Display: - current issue and PR - branch and worktree - PID and elapsed runtime - last start and completion time - last terminal classification - next scheduled run - Standardize terminal classifications: - COMPLETE - NO_WORK - BLOCKED - TIMEOUT - FAILED - Detect: - overlapping worker ownership - conflicting lanes - schedule collisions - stale locks - dead processes - missing executables - unhealthy MCP namespaces - wrong profiles or identities - runtime/master drift - Preserve strict author, reviewer, merger, and reconciler boundaries. - Fail closed on wrong-role access or ambiguous task ownership. - Keep credentials and tokens inaccessible to LLM processes. - Produce machine-readable run records and human-readable logs. - Support safe installation, validation, upgrades, rollback, and removal. - Continue using isolated worker processes while managing them through a unified control surface. ## Issue-first workflow requirements 1. An author cycle may work only on an existing open issue selected through canonical routing. 2. An author cycle may not create a new issue and implement it in the same cycle. 3. Issue-intake cycles may search for and create issues but may not acquire an implementation lock or worktree. 4. Every author implementation requires a durable issue lock and validated author worktree. 5. Reviewer, merger, reconciler, and cleanup work must use their dedicated namespaces and workflows. 6. All mutations require read-after-write verification. ## Acceptance criteria - All five LLM workers can be inspected through one control surface. - Worker schedules and responsibilities can be validated before activation. - Operators can safely pause, resume, restart, and run individual workers. - Current work and recent outcomes are visible for every worker. - Duplicate ownership, conflicting roles, and stale processes are detected. - Wrong-role operations fail closed. - Credentials are never exposed to prompts, logs, or LLM subprocesses. - Author workers reject tasks that lack an existing open issue. - Automated tests cover configuration validation, role enforcement, scheduling conflicts, locks, health checks, and process controls. - Operator documentation covers installation, daily operation, recovery, rollback, and removal.