4 Commits

Author SHA1 Message Date
sysadmin e67932fcc3 docs: propose labels for Jenkins/GlitchTip workflow (#80) 2026-07-02 15:07:13 -04:00
sysadmin 007b5dad14 Merge pull request 'docs: Jenkins read-only build status tools design (#72)' (#88) from docs/issue-72-jenkins-readonly-design into master 2026-07-02 14:01:36 -05:00
sysadmin 104907e311 docs: LLM-Agent-SHA opaque attribution convention, Phase 0 (#86) (#87)
Merge PR 87

Co-authored-by: Jason Walker <913443@dadeschools.net>
Co-committed-by: Jason Walker <913443@dadeschools.net>
2026-07-02 14:00:46 -05:00
sysadmin 74a7e8f792 docs: Jenkins read-only build status tools design (#72)
Add docs/architecture/jenkins-readonly-build-status-design.md:
implementation-ready design notes for the jenkins-mcp read-only tool set —
minimum tools (whoami, list_jobs, latest_build, build_status, get_build,
gated console_tail), safe return-field allowlist (url, number, timestamp,
duration, branch, result, commit), fail-closed failure behavior (unknown
job, unreachable, 5xx, auth, malformed JSON), bounded+redacted console tail
behind a distinct jenkins.console.read operation, per-service credential/
profile requirements (token by reference, fail closed), explicit exclusions
(build/deploy triggers, parameterized launches), job addressing with mapping
deferred to #77, and a mocked-Jenkins testing strategy.

Design only; no implementation, no code behavior changed, no Jenkins code
in mcp_server.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-02 14:34:18 -04:00
10 changed files with 561 additions and 13 deletions
@@ -0,0 +1,151 @@
# Jenkins Read-Only Build Status Tools — Design Notes
- **Status:** Design (implementation-ready notes; **no implementation in this repo**)
- **Issue:** #72 (parent umbrella: #75; boundary decision: ADR-0001, #71)
- **Related:** #77 (repo/branch/PR → job mapping, designed separately)
- **Date:** 2026-07-02
## 1. Purpose and scope
Define the minimum **read-only** Jenkins MCP tool set that lets an LLM answer:
*"Did the latest build for this project/branch succeed or fail?"* — plus enough
detail (build URL, number, timing, result) to report or investigate.
Phase 1 is **strictly read-only**, per ADR-0001
([`adr-0001-mcp-control-plane-boundaries.md`](adr-0001-mcp-control-plane-boundaries.md)):
- **Excluded: build triggers.**
- **Excluded: deploy triggers.**
- **Excluded: parameterized job launches.**
- Excluded: job creation/deletion/config changes, queue manipulation, node
management — any Jenkins mutation whatsoever.
## 2. Boundary placement
These tools belong to the **`jenkins-mcp`** package/server of the MCP Control
Plane — **never** inside `gitea-mcp` (`mcp_server.py` in this repo).
Consequences (from `tool-boundaries.md`, `credential-isolation.md`, ADR-0001):
- `jenkins-mcp` runs as its own server process with its own `.env`.
- **Jenkins credentials never enter the Gitea MCP runtime**, and Gitea
credentials never enter `jenkins-mcp`.
- This document lands in this repo only because the repo currently hosts the
Control Plane's architecture docs; the code ships elsewhere (owner decision
#1 of ADR-0001).
## 3. Minimum read-only tool set
| Tool | Purpose |
|---|---|
| `jenkins_whoami` | Verify authenticated Jenkins identity + active profile (mirror of `gitea_whoami`; fail-closed identity proof before anything else) |
| `jenkins_list_jobs` | List visible jobs (supports folder paths), with pagination bounds |
| `jenkins_latest_build` | The primary question: latest build of a job (or job+branch for multibranch) → status summary |
| `jenkins_build_status` | Status of a specific build number (job, number) |
| `jenkins_get_build` | Full safe detail of a build (fields in §4) |
| `jenkins_console_tail` | Bounded, redacted tail of a build's console log (§6) — optional, approval-gated addition |
All tools are `GET`-only against the Jenkins JSON API (`/api/json`,
`.../lastBuild/api/json`, `.../consoleText`). No tool issues POST/PUT/DELETE.
## 4. Return payloads (safe fields)
`jenkins_latest_build` / `jenkins_build_status` / `jenkins_get_build` return:
| Field | Source | Notes |
|---|---|---|
| `job` | request | Fully-qualified job path (folders joined with `/`) |
| `build_number` | `number` | int |
| `result` | `result` | `SUCCESS` / `FAILURE` / `UNSTABLE` / `ABORTED` / `NOT_BUILT`; `null``IN_PROGRESS` when `building=true` |
| `building` | `building` | bool |
| `url` | `url` | Build URL |
| `branch` | multibranch job name / SCM action | Best-effort; omitted when unknown |
| `timestamp` | `timestamp` | ISO-8601 UTC (converted from epoch ms) |
| `duration_seconds` | `duration` | 0/omitted while building |
| `commit_sha` | SCM build action | Best-effort; omitted when unknown |
Rules: no raw Jenkins payload passthrough (allowlist projection only); no
`Authorization` header, token, or crumb material in any output or error
(reuse the shared redaction approach of `safety-model.md` §3 / `gitea_audit`).
## 5. Failure behavior (fail closed, clear, safe)
| Condition | Behavior |
|---|---|
| Unknown job | Explicit `{"found": false, "job": "<path>", "error": "job not found"}` — never guess or fuzzy-match a job name (hard rule; see also #77) |
| Jenkins unreachable (DNS/timeout/conn refused) | Clear `"network error contacting Jenkins: <redacted reason>"`; no retry storm — mirror `gitea_auth.api_request` timeout + failure conversion |
| 502/503/504 | Explicit "Jenkins upstream unavailable" |
| 401/403 | "Jenkins auth failed / insufficient permissions" — **without** echoing credentials or the request's auth material |
| Malformed JSON | "malformed JSON response from Jenkins" (no raw-body dump) |
| Missing profile/creds | Fail closed before any network call (§7) |
## 6. Console tail safety (`jenkins_console_tail`)
Console logs are the highest-risk surface (secrets, tokens, internal hosts
routinely leak into build logs). If included at all (owner may defer it):
- **Bounded:** hard server-side cap (default: last 200 lines AND ≤ 64 KiB,
whichever is smaller; caller may request less, never more).
- **Redacted:** pass through the shared secret redactor (token/`Basic`/`Bearer`/
password/key-value patterns) before returning; redaction failure ⇒ return an
error, never the raw text.
- **Default off:** summary fields (`result`, failing stage if cheaply available)
are preferred; the tail requires an explicit `allowed_operations` entry
(`jenkins.console.read`) distinct from plain `jenkins.build.read`.
## 7. Credentials and profile requirements
Follows the per-service profile model (`gitea-execution-profiles.md`, extended
by #76):
- Env/config: `JENKINS_URL`, `JENKINS_USER`, `JENKINS_TOKEN_SOURCE_NAME`
(name-of-secret only — value resolved at runtime, never logged/committed).
- Profile: e.g. `jenkins-readonly` with namespaced
`allowed_operations: ["jenkins.read", "jenkins.build.read"]`
(+ `jenkins.console.read` only if the tail tool is approved);
`forbidden_operations: ["jenkins.build.trigger", "jenkins.deploy", "jenkins.job.configure"]`
as belt-and-braces even though no mutating tool exists.
- Missing URL/user/token/profile ⇒ **fail closed** with a clear message.
- Since every tool is read-only, no confirmation gates are needed — but
identity (`jenkins_whoami`) must still work so workflows can prove which
Jenkins account they act as.
## 8. Job addressing and mapping
Tools accept an explicit fully-qualified job path (folder-aware:
`folder/subfolder/job`). How a *repo/branch/PR* resolves to that job path is
**out of scope here** and designed in **#77**, with these fixed constraints:
- No silent guessing of job names — unmapped input returns an explicit
"no mapping" result.
- Multibranch pipelines address a branch job as `<job>/<branch>` with proper
URL-encoding of branch names (e.g. `feature%2Fx`).
## 9. Testing strategy (for the implementing package)
Mocked-Jenkins unit tests only (no live Jenkins in unit CI), mirroring this
repo's conventions (`docs/developer-testing-guidelines.md`):
- Patch the HTTP client; assert method is always `GET` and URL shape is correct
(folders, multibranch encoding).
- Success projections: field allowlist exactly as §4; unknown fields dropped.
- `result=null + building=true``IN_PROGRESS`.
- Unknown job ⇒ found:false, no fuzzy match, no API retry.
- Timeout/DNS/5xx/malformed-JSON ⇒ safe errors, no secret/credential leakage
(explicit no-token-in-error assertions).
- Console tail: cap enforcement (lines and bytes), redaction applied, redaction
failure ⇒ error not raw text, gated behind `jenkins.console.read`.
- Profile gate: missing/insufficient profile ⇒ no network call
(`mock_api.assert_not_called()` pattern).
## 10. Implementation-readiness checklist
Ready to implement in `jenkins-mcp` once:
1. ADR-0001 owner decision #1 (where `jenkins-mcp` lives) is made.
2. #76 profile schema exists (or a minimal `jenkins-readonly` profile is
hand-rolled to the same rules).
3. #77 mapping design is accepted (or tools ship path-addressed only, mapping
deferred).
Explicitly **not** unlocked by this document: build triggers, deploys,
parameterized launches, any Jenkins code in `mcp_server.py`.
+1 -1
View File
@@ -5,5 +5,5 @@ This document describes how credentials and sensitive environment variables are
## Separate Credentials ## Separate Credentials
Even though multiple MCP servers share the same monorepo, they **must** have separate credentials and runtimes. Even though multiple MCP servers share the same monorepo, they **must** have separate credentials and runtimes.
- **No Shared Environments**: Each MCP server (`gitea-mcp`, `jenkins-mcp`, `glitchtip-mcp`, `ops-mcp`, etc.) must be instantiated as an independent service with its own dedicated `.env` configuration file. - **No Shared Environments**: Each MCP server (`gitea-mcp`, `jenkins-mcp`, `ops-mcp`, etc.) must be instantiated as an independent service with its own dedicated `.env` configuration file.
- **Strict Isolation**: A server will only have access to the credentials required for its specific trust boundary. For instance, `gitea-mcp` has no access to Jenkins or Ops authentication tokens. - **Strict Isolation**: A server will only have access to the credentials required for its specific trust boundary. For instance, `gitea-mcp` has no access to Jenkins or Ops authentication tokens.
+34
View File
@@ -0,0 +1,34 @@
# Label Taxonomy
This document catalogs the issue labels used for MCP workflows, including Jenkins and GlitchTip (observability).
> **Approval Required:** Do not create or apply new labels in `manage_labels.py` without explicit owner approval of this document.
## Existing Labels
* **`jenkins`**
* Description: Jenkins integration
* Color: `d93f0b`
* Use: Used to mark issues, PRs, or tasks that involve the `jenkins-mcp` boundaries, CI/CD designs, or build failures.
* **`glitchtip`**
* Description: GlitchTip integration
* Color: `b60205`
* Use: Used to mark issues related to the `glitchtip-mcp` boundary and observability integration.
## Proposed / Missing Labels
* **`observability`**
* Proposed Description: Observability, metrics, and monitoring tasks
* Proposed Color: `5319e7`
* Use: Broader than GlitchTip alone; covers logging, metrics, traces, and general observability pipeline improvements.
* **`source:glitchtip`**
* Proposed Description: Issue filed automatically by GlitchTip orchestration
* Proposed Color: `b60205`
* Use: Applied automatically by the orchestrator when a GlitchTip error event is converted into a Gitea issue.
* **`status:triage`**
* Proposed Description: Issue needs human or orchestrator triage
* Proposed Color: `fbca04`
* Use: Used for incoming issues (especially automated ones like `source:glitchtip`) that have not yet been evaluated for priority or resolution.
+138
View File
@@ -0,0 +1,138 @@
# LLM-Agent-SHA — Opaque Agent Attribution (Phase 0)
Convention for attributing work to a specific LLM session/workstream across
issues, branches, PRs, and review handoffs, without exposing a human or model
identity. Approved by the owner decision on issue #86
(`#issuecomment-1354`); this document implements **Phase 0 only**.
## The one rule that matters
`LLM-Agent-SHA` is **informational attribution metadata only**. It must never
be used for authentication, authorization, review eligibility, merge
eligibility, profile permissions, or any other security decision.
The security gates remain, unchanged:
- the **authenticated Gitea user** (self-review/self-merge protection),
- the **active MCP profile** and its `allowed_operations`
(see [`gitea-execution-profiles.md`](gitea-execution-profiles.md)),
- the fail-closed eligibility checks in `gitea_check_pr_eligibility`.
Two sessions with different `LLM-Agent-SHA` values that authenticate as the
same Gitea user are **the same actor** for review/merge safety. A different
SHA never unlocks self-review or self-merge. `tests/test_llm_agent_sha.py`
proves the eligibility logic never consults the SHA.
## Format
```text
LLM-Agent-SHA: llm-<12 lowercase hex chars>
```
Validation regex:
```text
^llm-[0-9a-f]{12}$
```
Examples: `llm-8f3a9c2d6b41`, `llm-41d0e7aa9f2c`, `llm-b7c93d441a08`.
### Generation
Generate 48 random bits, e.g. `python3 -c "import secrets; print('llm-' +
secrets.token_hex(6))"`, or hash a non-secret session UUID. An
operator-provided opaque ID is also fine.
Do **not** derive the value from any of:
- a Gitea token or other secret,
- an email address or username,
- a machine hostname or private filesystem path,
- a model or provider name,
- conversation contents.
The SHA must contain no model name, provider name, human name, email,
hostname, token, private path, or conversation-derived content. It is safe to
include in PR bodies, issue comments, and audit logs — and only there.
## Lifetime
Canonical lifetime is **per PR/workstream**: pick one SHA when starting an
issue and keep it through the branch, PR, and handoff for that workstream. A
per-session SHA is acceptable when the session maps cleanly to one
workstream. Do not reuse a SHA across unrelated workstreams.
## Placement
Phase 0 uses **visible markdown metadata blocks** (not hidden HTML
comments). Include the block in PR bodies and review handoffs; keep it out of
ordinary comments unless attribution is genuinely useful there.
**Never put the SHA in branch or worktree names.** Branches stay
issue-linked and human-readable (`docs/issue-86-llm-agent-sha-phase0`), per
the branch standard.
### Handoff metadata block (implementer → PR body / handoff report)
```markdown
LLM Handoff Metadata:
- LLM-Agent-SHA: llm-8f3a9c2d6b41
- LLM-Role: implementer
- Authenticated-Gitea-User: jcwalker3
- MCP-Profile: gitea-default
- Branch: docs/example-branch
- Worktree: branches/docs-example-branch
- Self-review allowed: no
```
### Review metadata block (reviewer → review comment)
```markdown
Review Metadata:
- LLM-Agent-SHA: llm-41d0e7aa9f2c
- LLM-Role: reviewer
- Authenticated-Gitea-User: sysadmin
- MCP-Profile: prgs-reviewer
- Eligibility: passed
```
## Same SHA vs same user vs same profile
Reviewers and operators must keep three distinct identities straight:
| Comparison | Meaning | Effect on eligibility |
|---|---|---|
| same `LLM-Agent-SHA` | same LLM session/workstream wrote both artifacts | **none — attribution only** |
| same authenticated Gitea user | same Gitea actor | **blocks** self-review / self-merge, regardless of SHA |
| same MCP profile | same capability set | governs `allowed_operations` (what actions are permitted at all) |
Concretely: an implementer session (`llm-8f3a…`, user `jcwalker3`) and a
would-be reviewer session (`llm-41d0…`, also user `jcwalker3`) have different
SHAs but the **same Gitea user** — the reviewer session is still the PR
author to Gitea and must not review, approve, or merge. Review handoffs
require a genuinely different authenticated user (e.g. `sysadmin` /
`prgs-reviewer`).
## Phase 0 scope (and what is deferred)
Phase 0 is documentation, handoff/review templates, and negative tests only.
Deferred to later owner-approved phases; none of this exists yet:
- launcher-enforced SHA generation,
- `LLM_AGENT_SHA` / `LLM_AGENT_ROLE` environment injection,
- `gitea_whoami` returning SHA/role,
- automatic PR body injection by MCP tools,
- audit schema changes requiring the SHA,
- release/orchestrator lineage tracking.
MCP tools neither read nor emit the SHA. Setting an `LLM_AGENT_SHA`
environment variable has no effect on any tool; the negative tests assert
eligibility results are byte-identical with and without it.
## Related documents
- [`llm-workflow-runbooks.md`](llm-workflow-runbooks.md) — the runbooks whose
handoffs carry these blocks
- [`gitea-execution-profiles.md`](gitea-execution-profiles.md) — profiles and
`allowed_operations` (the real permission gate)
- [`safety-model.md`](safety-model.md) — audit, redaction, confirmation gates
+20 -2
View File
@@ -45,6 +45,18 @@ Use any eligible reviewer profile to review PR #N.
Use any eligible merger profile to merge PR #N if checks pass. Use any eligible merger profile to merge PR #N if checks pass.
``` ```
### Attribution: `LLM-Agent-SHA` (metadata only)
Sessions may attribute their work with an opaque `LLM-Agent-SHA`
(`llm-<12 lowercase hex>`, e.g. `llm-8f3a9c2d6b41`) in PR-body and
review-handoff metadata blocks — see
[`llm-agent-sha.md`](llm-agent-sha.md) for the full convention. It is
**attribution only**: eligibility is decided solely by the authenticated
Gitea user and the profile's allowed operations. Two sessions with different
SHAs under the same Gitea user are the same actor — a different SHA never
permits self-review or self-merge. Keep the SHA out of branch and worktree
names.
## Prerequisites: canonical config + thin launchers ## Prerequisites: canonical config + thin launchers
Runtime profiles live in **one canonical JSON file**, referenced by every LLM Runtime profiles live in **one canonical JSON file**, referenced by every LLM
@@ -274,7 +286,8 @@ touching anything.
`fix/...` / `docs/...`); `cd` into that worktree; implement narrowly; add or `fix/...` / `docs/...`); `cd` into that worktree; implement narrowly; add or
update tests if behavior changes; run the full suite; commit with an update tests if behavior changes; run the full suite; commit with an
issue-linked message; open a PR to `master`. **Do not** review or merge your issue-linked message; open a PR to `master`. **Do not** review or merge your
own PR. own PR. Include an `LLM Handoff Metadata` block (with `LLM-Agent-SHA`) in
the PR body — see [`llm-agent-sha.md`](llm-agent-sha.md).
- **Prompt:** `Use an author profile to implement issue #N and open a PR to - **Prompt:** `Use an author profile to implement issue #N and open a PR to
master. Do not self-review or self-merge.` master. Do not self-review or self-merge.`
@@ -285,7 +298,11 @@ touching anything.
- **Steps:** confirm identity + eligibility (menu eligibility check or - **Steps:** confirm identity + eligibility (menu eligibility check or
`gitea_check_pr_eligibility`); read the diff; confirm scope matches the linked `gitea_check_pr_eligibility`); read the diff; confirm scope matches the linked
issue; post the review (`comment` / `request_changes` / `approve`) via the issue; post the review (`comment` / `request_changes` / `approve`) via the
gated review tool. Pin the reviewed head SHA where supported. gated review tool. Pin the reviewed head SHA where supported. Include a
`Review Metadata` block (with your own `LLM-Agent-SHA`) in the review —
and remember: a different `LLM-Agent-SHA` does **not** make you a different
actor; only a different authenticated Gitea user does
([`llm-agent-sha.md`](llm-agent-sha.md)).
- **Prompt:** `Use any eligible reviewer profile to review PR #N. Approve only - **Prompt:** `Use any eligible reviewer profile to review PR #N. Approve only
if scope matches issue #M and checks pass; otherwise request changes.` if scope matches issue #M and checks pass; otherwise request changes.`
@@ -391,6 +408,7 @@ scripts/release-tag v0.4.0 --notes-file /tmp/release-notes.md --push
- [`../skills/llm-project-workflow/SKILL.md`](../skills/llm-project-workflow/SKILL.md) — portable cross-project LLM workflow skill. - [`../skills/llm-project-workflow/SKILL.md`](../skills/llm-project-workflow/SKILL.md) — portable cross-project LLM workflow skill.
- [`gitea-execution-profiles.md`](gitea-execution-profiles.md) — the profile model. - [`gitea-execution-profiles.md`](gitea-execution-profiles.md) — the profile model.
- [`llm-agent-sha.md`](llm-agent-sha.md) — opaque agent attribution metadata (never an eligibility input).
- [`safety-model.md`](safety-model.md) — trust boundaries and audit logging. - [`safety-model.md`](safety-model.md) — trust boundaries and audit logging.
- [`tool-boundaries.md`](tool-boundaries.md) — per-tool allowed operations. - [`tool-boundaries.md`](tool-boundaries.md) — per-tool allowed operations.
- [`credential-isolation.md`](credential-isolation.md) — credential handling. - [`credential-isolation.md`](credential-isolation.md) — credential handling.
-8
View File
@@ -13,11 +13,3 @@ To maintain a secure environment, all secrets, tokens, passwords, and sensitive
- System and application logs - System and application logs
- Tool return values/outputs - Tool return values/outputs
- Any form of persistent storage or console output - Any form of persistent storage or console output
## 4. Read-Only First Policy
By default, MCP servers (such as `jenkins-mcp` and `ops-mcp`) operate in a **read-only** mode. Mutation capabilities are deny-by-default and fail-closed.
## 5. Mutation Gating
Any mutating action (e.g., Gitea issue creation from GlitchTip, or Jenkins builds) must be explicitly allowed by the execution profile.
- **Jenkins build triggers** are explicitly deferred for phase 1.
- **GlitchTip to Gitea issue filing** is documented as a gated, orchestrated workflow, not a direct unprompted automatic action.
+1 -2
View File
@@ -2,7 +2,7 @@
This document defines the strict boundaries between the different MCP server packages within the monorepo. This document defines the strict boundaries between the different MCP server packages within the monorepo.
The project is named **MCP Control Plane** and lives in the `mcp-control-plane` repository. It groups the following packages: `common`, `gitea-mcp`, `jenkins-mcp`, `glitchtip-mcp`, `ops-mcp`, and `release-mcp`. The project is named **MCP Control Plane** and lives in the `mcp-control-plane` repository. It groups the following packages: `common`, `gitea-mcp`, `jenkins-mcp`, `ops-mcp`, and `release-mcp`.
## 1. Architectural Philosophy ## 1. Architectural Philosophy
- **One MCP Server per Trust Boundary**: While the packages share a monorepo, their runtime services must remain entirely separate. There is no single "everything" server. - **One MCP Server per Trust Boundary**: While the packages share a monorepo, their runtime services must remain entirely separate. There is no single "everything" server.
@@ -10,5 +10,4 @@ The project is named **MCP Control Plane** and lives in the `mcp-control-plane`
## 2. Package-Specific Boundaries ## 2. Package-Specific Boundaries
- **gitea-mcp**: Restricted to source-control and work-item capabilities (issues, PRs, comments). This package **must not** have Jenkins or Ops credentials, nor can it execute deploy operations. - **gitea-mcp**: Restricted to source-control and work-item capabilities (issues, PRs, comments). This package **must not** have Jenkins or Ops credentials, nor can it execute deploy operations.
- **jenkins-mcp**: Focused on CI/CD capabilities. This package **must not** have Ops credentials unless explicitly configured for a specific, isolated pipeline later. - **jenkins-mcp**: Focused on CI/CD capabilities. This package **must not** have Ops credentials unless explicitly configured for a specific, isolated pipeline later.
- **glitchtip-mcp**: Dedicated to observability and error reporting. This package **must not** have Gitea write credentials or Jenkins deploy capabilities.
- **ops-mcp**: Dedicated to live environment and host checks. In its initial state, this package starts as strictly read-only (e.g., health checks, status checks, log reading). - **ops-mcp**: Dedicated to live environment and host checks. In its initial state, this package starts as strictly read-only (e.g., health checks, status checks, log reading).
@@ -8,6 +8,8 @@ Task: review PR #<pr> for issue #<n>.
Rules (llm-project-workflow): Rules (llm-project-workflow):
- Review in a SEPARATE detached review worktree, never the author's folder. - Review in a SEPARATE detached review worktree, never the author's folder.
- You must NOT be the PR author. If the authenticated user == PR author, stop. - You must NOT be the PR author. If the authenticated user == PR author, stop.
A different LLM-Agent-SHA does NOT make you a different actor — only a
different authenticated Gitea user does (docs/llm-agent-sha.md).
- Do not merge if any check fails. - Do not merge if any check fails.
Steps: Steps:
@@ -21,6 +23,14 @@ Steps:
6. Run the test suite; note results. 6. Run the test suite; note results.
7. Post the review verdict: approve only if scope is clean and checks pass; 7. Post the review verdict: approve only if scope is clean and checks pass;
otherwise request changes with specifics. Never merge from this review step. otherwise request changes with specifics. Never merge from this review step.
Include a "Review Metadata" block (attribution only — docs/llm-agent-sha.md):
Review Metadata:
- LLM-Agent-SHA: llm-<12 lowercase hex, e.g. llm-41d0e7aa9f2c>
- LLM-Role: reviewer
- Authenticated-Gitea-User: <whoami result>
- MCP-Profile: <profile name>
- Eligibility: passed/failed
Handoff: reviewer identity, PR author, scope verdict, checks + results, decision. Handoff: reviewer identity, PR author, scope verdict, checks + results, decision.
``` ```
@@ -23,6 +23,17 @@ Steps:
6. Checks: run the test suite, compile/lint changed files, git diff --check, 6. Checks: run the test suite, compile/lint changed files, git diff --check,
and scan the diff for secrets. and scan the diff for secrets.
7. Commit (issue-linked message), push the branch, open a PR to master. 7. Commit (issue-linked message), push the branch, open a PR to master.
Include an "LLM Handoff Metadata" block in the PR body (attribution only;
never an eligibility input — docs/llm-agent-sha.md):
LLM Handoff Metadata:
- LLM-Agent-SHA: llm-<12 lowercase hex, e.g. llm-8f3a9c2d6b41>
- LLM-Role: implementer
- Authenticated-Gitea-User: <whoami result>
- MCP-Profile: <profile name>
- Branch: <branch>
- Worktree: <worktree path>
- Self-review allowed: no
8. Stop before review/merge — you are the author. 8. Stop before review/merge — you are the author.
Handoff: issue #, branch, worktree path, files changed, checks + results, PR URL. Handoff: issue #, branch, worktree path, files changed, checks + results, PR URL.
+195
View File
@@ -0,0 +1,195 @@
"""Negative tests for LLM-Agent-SHA attribution (#86, Phase 0).
``LLM-Agent-SHA`` (docs/llm-agent-sha.md) is attribution metadata ONLY. These
tests prove it can never bypass the review/merge safety gates:
1. Same Gitea user + different LLM-Agent-SHA still fails self-review/approval.
2. Same Gitea user + different LLM-Agent-SHA still fails self-merge.
3. The eligibility logic does not consult SHA metadata at all — results are
identical with no SHA, one SHA, or a different SHA in the environment, and
no gate accepts an agent-SHA input.
Phase 0 adds no SHA support to any MCP tool; the environment variables set
here (``LLM_AGENT_SHA`` / ``LLM_AGENT_ROLE``) simulate a future launcher and
must be ignored by every gate.
"""
import inspect
import os
import re
import sys
import unittest
from unittest.mock import patch
sys.path.insert(0, str(__import__("pathlib").Path(__file__).resolve().parent.parent))
from mcp_server import ( # noqa: E402
gitea_check_pr_eligibility,
gitea_merge_pr,
gitea_review_pr,
gitea_submit_pr_review,
)
FAKE_AUTH = "Basic dGVzdDp0ZXN0"
SHA_PATTERN = re.compile(r"^llm-[0-9a-f]{12}$")
# Two distinct, well-formed agent SHAs "belonging" to the same Gitea user.
SHA_IMPLEMENTER = "llm-8f3a9c2d6b41"
SHA_WOULD_BE_REVIEWER = "llm-41d0e7aa9f2c"
def _pr(author, state="open", sha="abc123", mergeable=True):
return {
"user": {"login": author},
"state": state,
"head": {"sha": sha},
"mergeable": mergeable,
}
class TestShaFormatConvention(unittest.TestCase):
"""The documented format: llm-<12 lowercase hex>, nothing identifying."""
def test_documented_examples_are_valid(self):
for value in (SHA_IMPLEMENTER, SHA_WOULD_BE_REVIEWER, "llm-b7c93d441a08"):
self.assertRegex(value, SHA_PATTERN)
def test_identifying_or_malformed_values_are_rejected(self):
for bad in (
"llm-8F3A9C2D6B41", # uppercase
"llm-8f3a9c2d6b4", # too short
"llm-8f3a9c2d6b411", # too long
"llm-opus4", # model name, not hex
"claude-8f3a9c2d6b41", # provider prefix
"jcwalker3", # username
"llm-user@example.com", # email
"8f3a9c2d6b41", # missing prefix
"",
):
self.assertNotRegex(bad, SHA_PATTERN)
class TestShaCannotBypassSelfReview(unittest.TestCase):
"""Scenario: user A (SHA 1) authored the PR; user A (SHA 2) tries to act."""
def _env(self, agent_sha, role):
# Reviewer-capable profile + a simulated launcher-injected agent SHA.
return {
"GITEA_PROFILE_NAME": "gitea-reviewer",
"GITEA_ALLOWED_OPERATIONS": "read,review,approve,merge",
"LLM_AGENT_SHA": agent_sha,
"LLM_AGENT_ROLE": role,
}
@patch("mcp_server.api_request")
@patch("mcp_server.get_auth_header", return_value=FAKE_AUTH)
def test_same_user_different_sha_cannot_approve(self, _auth, mock_api):
mock_api.side_effect = [{"login": "jcwalker3"}, _pr("jcwalker3")]
env = self._env(SHA_WOULD_BE_REVIEWER, "reviewer")
with patch.dict(os.environ, env, clear=True):
r = gitea_check_pr_eligibility(pr_number=9, action="approve", remote="prgs")
self.assertFalse(r["eligible"])
self.assertIn("authenticated user is PR author", r["reasons"])
@patch("mcp_server.api_request")
@patch("mcp_server.get_auth_header", return_value=FAKE_AUTH)
def test_same_user_different_sha_cannot_merge(self, _auth, mock_api):
mock_api.side_effect = [{"login": "jcwalker3"}, _pr("jcwalker3")]
env = self._env(SHA_WOULD_BE_REVIEWER, "merger")
with patch.dict(os.environ, env, clear=True):
r = gitea_check_pr_eligibility(pr_number=9, action="merge", remote="prgs")
self.assertFalse(r["eligible"])
self.assertIn("authenticated user is PR author", r["reasons"])
@patch("mcp_server.api_request")
@patch("mcp_server.get_auth_header", return_value=FAKE_AUTH)
def test_gated_merge_tool_refuses_self_merge_despite_sha(self, _auth, mock_api):
# Even the fully-confirmed gated merge path must refuse: correct
# confirmation string, mergeable PR, merge-capable profile — but the
# authenticated user is the PR author, whatever the agent SHA says.
mock_api.side_effect = [{"login": "jcwalker3"}, _pr("jcwalker3")]
env = self._env(SHA_WOULD_BE_REVIEWER, "merger")
with patch.dict(os.environ, env, clear=True):
r = gitea_merge_pr(
pr_number=9, confirmation="MERGE PR 9", remote="prgs")
self.assertFalse(r.get("performed"))
for call in mock_api.call_args_list:
method, url = call.args[0], call.args[1]
self.assertFalse(
method == "POST" and url.endswith("/merge"),
f"self-merge mutation reached the API: {method} {url}",
)
@patch("mcp_server.api_request")
@patch("mcp_server.get_auth_header", return_value=FAKE_AUTH)
def test_review_tool_refuses_self_approval_despite_sha(self, _auth, mock_api):
mock_api.side_effect = [{"login": "jcwalker3"}, _pr("jcwalker3")]
env = self._env(SHA_WOULD_BE_REVIEWER, "reviewer")
with patch.dict(os.environ, env, clear=True):
r = gitea_review_pr(
pr_number=9, event="APPROVE", body="self approve", merge=False,
remote="prgs")
self.assertFalse(r["success"])
self.assertIn("authenticated user is PR author", r["message"])
for call in mock_api.call_args_list:
method, url = call.args[0], call.args[1]
self.assertFalse(
method == "POST" and url.endswith("/reviews"),
f"self-review mutation reached the API: {method} {url}",
)
class TestEligibilityNeverConsultsSha(unittest.TestCase):
"""The gates have no SHA input: not via env, not via parameters."""
def _run_eligibility(self, extra_env):
env = {
"GITEA_PROFILE_NAME": "gitea-reviewer",
"GITEA_ALLOWED_OPERATIONS": "read,review,approve",
}
env.update(extra_env)
with patch("mcp_server.get_auth_header", return_value=FAKE_AUTH), \
patch("mcp_server.api_request") as mock_api:
mock_api.side_effect = [{"login": "reviewer-bot"}, _pr("author-bot")]
with patch.dict(os.environ, env, clear=True):
return gitea_check_pr_eligibility(
pr_number=5, action="approve", remote="prgs")
def test_result_identical_with_without_and_across_shas(self):
baseline = self._run_eligibility({})
with_one = self._run_eligibility(
{"LLM_AGENT_SHA": SHA_IMPLEMENTER, "LLM_AGENT_ROLE": "implementer"})
with_other = self._run_eligibility(
{"LLM_AGENT_SHA": SHA_WOULD_BE_REVIEWER, "LLM_AGENT_ROLE": "reviewer"})
self.assertEqual(baseline, with_one)
self.assertEqual(baseline, with_other)
self.assertTrue(baseline["eligible"]) # sanity: a real decision ran
def test_eligibility_result_carries_no_agent_sha(self):
r = self._run_eligibility(
{"LLM_AGENT_SHA": SHA_IMPLEMENTER, "LLM_AGENT_ROLE": "implementer"})
blob = repr(r)
self.assertNotIn(SHA_IMPLEMENTER, blob)
self.assertNotIn("LLM_AGENT", blob)
def test_gate_functions_accept_no_agent_sha_parameter(self):
for fn in (gitea_check_pr_eligibility, gitea_merge_pr,
gitea_review_pr, gitea_submit_pr_review):
for param in inspect.signature(fn).parameters:
lowered = param.lower()
self.assertNotIn("agent", lowered,
f"{fn.__name__} accepts agent param {param!r}")
self.assertNotIn("llm", lowered,
f"{fn.__name__} accepts llm param {param!r}")
def test_gate_sources_never_read_agent_sha(self):
# Phase 0 guarantee: no gate reads LLM_AGENT_* metadata at all.
for fn in (gitea_check_pr_eligibility, gitea_merge_pr,
gitea_review_pr, gitea_submit_pr_review):
src = inspect.getsource(fn)
self.assertNotIn("LLM_AGENT", src,
f"{fn.__name__} reads LLM_AGENT metadata")
self.assertNotIn("agent_sha", src,
f"{fn.__name__} consults an agent SHA")
if __name__ == "__main__":
unittest.main()