Scaled-Tech-Consulting/Gitea-Tools
2
0
Fork 0
Code Issues 73 Pull Requests 9 Actions Packages Projects Releases Wiki Activity

Harden issue-filing final reports: Controller Handoff, full SHAs, and exact mutation capability proof #191

New Issue
Closed
opened 2026-07-05 14:42:56 -05:00 by jcwalker3 · 4 comments
jcwalker3 commented 2026-07-05 14:42:56 -05:00
Owner
Copy Link
Copy Source

Problem:
The creation of Gitea-Tools issue #188/189 was a strong issue-filing run. It correctly performed a duplicate check, explained why a new issue was needed instead of expanding #183, avoided premature status:in-progress, and created only one issue.
However, a few issue-filing/reporting details should be made mandatory so future runs are consistently A-level.

Required behavior:

  1. Every issue-filing task must include a Controller Handoff section.
    Required fields:

    • Task
    • Repo
    • Role
    • Identity
    • Issue created or updated
    • Related issues
    • Mutations
    • Current status
    • Blockers
    • Next
    • Safety

  2. Issue-filing reports must include the exact issue number and title.
    Example:

    • Created Gitea-Tools #189 — Implement fail-closed continuation mode for issues already represented by open PRs

  3. When old/new commit references are used as evidence, reports must include full 40-character SHAs, not abbreviated SHAs.
    Example requirement:

    • old head: <40-char SHA>
    • new head: <40-char SHA>

  4. Every mutation must have exact capability proof.
    Examples:

    • issue creation requires create_issue
    • issue comment requires comment_issue
    • label change requires exact label/set-label capability proof
    • PR comment requires comment_pr

  5. If only one mutation is performed, the report should explicitly say:

    • Only mutation: issue creation
    • or Only mutations: issue creation + comment
    • and confirm no labels, comments, PRs, reviews, merges, or closes were performed.

  6. Duplicate-check evidence must be summarized:

    • number of open issues searched
    • closest existing issue(s)
    • why update was rejected
    • why new issue was justified

Tests / harness assertions:

  • issue-filing final report without Controller Handoff is downgraded
  • abbreviated commit SHAs in evidence are downgraded
  • mutation without exact capability proof is downgraded
  • label mutation without label-specific capability proof is blocked/downgraded
  • new issue creation without duplicate-search summary is downgraded
  • report must distinguish “new issue created” from “existing issue updated”

Acceptance criteria:
An issue-filing run earns an A only when it proves duplicate-search decision quality, exact mutation capability, full evidence identifiers, no unintended mutations, and includes a compact Controller Handoff.

Problem: The creation of Gitea-Tools issue #188/189 was a strong issue-filing run. It correctly performed a duplicate check, explained why a new issue was needed instead of expanding #183, avoided premature `status:in-progress`, and created only one issue. However, a few issue-filing/reporting details should be made mandatory so future runs are consistently A-level. Required behavior: 1. Every issue-filing task must include a `Controller Handoff` section. Required fields: * Task * Repo * Role * Identity * Issue created or updated * Related issues * Mutations * Current status * Blockers * Next * Safety 2. Issue-filing reports must include the exact issue number and title. Example: * Created Gitea-Tools #189 — `Implement fail-closed continuation mode for issues already represented by open PRs` 3. When old/new commit references are used as evidence, reports must include full 40-character SHAs, not abbreviated SHAs. Example requirement: * old head: `<40-char SHA>` * new head: `<40-char SHA>` 4. Every mutation must have exact capability proof. Examples: * issue creation requires `create_issue` * issue comment requires `comment_issue` * label change requires exact label/set-label capability proof * PR comment requires `comment_pr` 5. If only one mutation is performed, the report should explicitly say: * `Only mutation: issue creation` * or `Only mutations: issue creation + comment` * and confirm no labels, comments, PRs, reviews, merges, or closes were performed. 6. Duplicate-check evidence must be summarized: * number of open issues searched * closest existing issue(s) * why update was rejected * why new issue was justified Tests / harness assertions: * issue-filing final report without `Controller Handoff` is downgraded * abbreviated commit SHAs in evidence are downgraded * mutation without exact capability proof is downgraded * label mutation without label-specific capability proof is blocked/downgraded * new issue creation without duplicate-search summary is downgraded * report must distinguish “new issue created” from “existing issue updated” Acceptance criteria: An issue-filing run earns an A only when it proves duplicate-search decision quality, exact mutation capability, full evidence identifiers, no unintended mutations, and includes a compact Controller Handoff.
sysadmin commented 2026-07-05 14:47:25 -05:00
Owner
Copy Link
Copy Source

Positive Author-Run Reporting Example

Evidence:
Claude implemented Gitea-Tools issue #179 and opened PR #193 as jcwalker3 / prgs-author.

Positive behavior observed:

  • Included a complete Controller Handoff.
  • Stayed in author/implementer role.
  • Did not review, approve, request changes, merge, or close issues.
  • Reported selected issue, PR number, branch, head SHA, and base SHA:
  • Reported exact files changed.
  • Reported exact validation:
    • targeted tests: 72 passed
    • full suite: 712 passed, 6 skipped in 8.41s
    • git diff --check prgs/master...HEAD: clean
    • python3 -m py_compile $(git ls-files '*.py'): OK
  • Reported exact secret/provenance sweep command:
    • git diff --cached | grep -inE "password|api[_-]?key|authorization|bearer|BEGIN (RSA|EC|OPENSSH)|https?://"
    • single hit was synthetic fixture text only
  • Proved exact capability for each mutation:
    • comment_issue
    • push_branch
    • create_pr
  • Correctly failed closed on unmapped mark_issue and used comment claim instead.
  • Reported base movement and branch/drift proof application.
  • Ended with next action and safety notes.

Remaining improvement notes:

  • Future A+ reports should list complete open PR inventory counts or explicitly state inventory scope.
  • Force-with-lease / branch-push reports should include old remote head, expected lease head, and new remote head when applicable.
  • Related workflow-hardening issues should be mentioned when they are close enough to the selected issue.

Use this run as a positive baseline for author-side final reports under #191.

### Positive Author-Run Reporting Example **Evidence:** Claude implemented Gitea-Tools issue #179 and opened PR #193 as `jcwalker3 / prgs-author`. **Positive behavior observed:** * Included a complete `Controller Handoff`. * Stayed in author/implementer role. * Did not review, approve, request changes, merge, or close issues. * Reported selected issue, PR number, branch, head SHA, and base SHA: * PR #193 * branch `feat/issue-179-reviewer-proof-tightening` * head `4a83a8e215342c7c18c9f2b117515e1b4a5a310f` * base master `601c608` * Reported exact files changed. * Reported exact validation: * targeted tests: `72 passed` * full suite: `712 passed, 6 skipped in 8.41s` * `git diff --check prgs/master...HEAD`: clean * `python3 -m py_compile $(git ls-files '*.py')`: OK * Reported exact secret/provenance sweep command: * `git diff --cached | grep -inE "password|api[_-]?key|authorization|bearer|BEGIN (RSA|EC|OPENSSH)|https?://"` * single hit was synthetic fixture text only * Proved exact capability for each mutation: * `comment_issue` * `push_branch` * `create_pr` * Correctly failed closed on unmapped `mark_issue` and used comment claim instead. * Reported base movement and branch/drift proof application. * Ended with next action and safety notes. **Remaining improvement notes:** * Future A+ reports should list complete open PR inventory counts or explicitly state inventory scope. * Force-with-lease / branch-push reports should include old remote head, expected lease head, and new remote head when applicable. * Related workflow-hardening issues should be mentioned when they are close enough to the selected issue. Use this run as a positive baseline for author-side final reports under #191.
jcwalker3 commented 2026-07-05 14:47:57 -05:00
Author
Owner
Copy Link
Copy Source

Evidence: Grok author run (issue #150 → PR #190) — author-side reporting gaps

A Grok author run selected Gitea-Tools issue #150, created branch feat/issue-150-add-integration-discoverability-coverage, and opened PR #190. The run was safe: it used jcwalker3 / prgs-author, checked both configured repos, inventoried open PRs before issue selection, selected one issue, did not review/approve/request changes/merge, and stopped after opening the PR.

Observed workflow gaps (extending this issue's required behaviors from issue-filing runs to author implementation runs):

  1. Missing Controller Handoff. The final report did not include a compact Controller Handoff block (this issue's requirement 1).

  2. Unproven label mutation — internal contradiction. The report says mark_issue resolved unknown and therefore a comment claim was used, but later says status:in-progress was set via gitea_set_issue_labels. If a label mutation is performed, the workflow must prove exact capability for label mutation; comment_issue capability is not enough (this issue's requirement 4 / label-specific assertion). If mark_issue is unknown, the workflow must not use label mutation unless gitea_set_issue_labels or an equivalent capability is explicitly resolved.

  3. Missing PR head SHA. The report gives the branch name but not the final head commit SHA for PR #190 (this issue's requirement 3: full 40-char identifiers).

  4. Incomplete validation counts. Targeted result was exact (23 passed), but the broader "full relevant" suite was reported only as exit 0 — no pass/skip/fail counts. Every claimed suite needs exact counts.

  5. Ambiguous sweep result. Sweep command was git diff | grep -iE 'token|secret|password|key|credential|http' || echo "SWEEP_CLEAN", and the report acknowledges matches (expected http:// and keychain: test strings) that were manually judged benign — yet called the sweep "clean". Sweeps must distinguish: no matches / matches reviewed and benign (enumerated or allowlisted) / unreviewed matches / real findings.

Additional tests/harness assertions this evidence motivates:

  • label mutation without exact capability proof is blocked/downgraded
  • a report claiming comment-only claim while also performing a label mutation is downgraded (contradictory mutation account)
  • author PR report without head SHA is incomplete
  • a validation suite reported only as exit 0 is downgraded
  • a sweep with matches cannot be reported as simply "clean" unless benign matches are enumerated or allowlisted
  • final report without Controller Handoff is downgraded

Acceptance criterion (author-run analogue of this issue's): an author issue run earns an "A" only when it proves exact capability for every mutation (including label changes), includes PR number + branch + exact head SHA, reports exact validation counts for every claimed suite, reports secret/provenance sweep results with the four-way match taxonomy, and ends with a Controller Handoff.

Cross-references: #183 (author-run reporting hardening — its gaps 1/4/5 overlap the handoff/sweep/head-SHA items here; the label-mutation capability contradiction and the sweep match-taxonomy are new evidence beyond #183); review_proofs.assess_sweep_evidence (merged via PR #193's branch, pending review) already requires exact method+scope and could be extended with the benign-match enumeration taxonomy when this issue is implemented. The unresolved mark_issue resolver gap itself remains tracked in #183.

## Evidence: Grok author run (issue #150 → PR #190) — author-side reporting gaps A Grok author run selected Gitea-Tools issue #150, created branch `feat/issue-150-add-integration-discoverability-coverage`, and opened PR #190. The run was safe: it used `jcwalker3 / prgs-author`, checked both configured repos, inventoried open PRs before issue selection, selected one issue, did not review/approve/request changes/merge, and stopped after opening the PR. Observed workflow gaps (extending this issue's required behaviors from issue-filing runs to author implementation runs): 1. **Missing Controller Handoff.** The final report did not include a compact `Controller Handoff` block (this issue's requirement 1). 2. **Unproven label mutation — internal contradiction.** The report says `mark_issue` resolved unknown and therefore a comment claim was used, but later says `status:in-progress` was set via `gitea_set_issue_labels`. If a label mutation is performed, the workflow must prove exact capability for label mutation; `comment_issue` capability is not enough (this issue's requirement 4 / label-specific assertion). If `mark_issue` is unknown, the workflow must not use label mutation unless `gitea_set_issue_labels` or an equivalent capability is explicitly resolved. 3. **Missing PR head SHA.** The report gives the branch name but not the final head commit SHA for PR #190 (this issue's requirement 3: full 40-char identifiers). 4. **Incomplete validation counts.** Targeted result was exact (`23 passed`), but the broader "full relevant" suite was reported only as `exit 0` — no pass/skip/fail counts. Every claimed suite needs exact counts. 5. **Ambiguous sweep result.** Sweep command was `git diff | grep -iE 'token|secret|password|key|credential|http' || echo "SWEEP_CLEAN"`, and the report acknowledges matches (expected `http://` and `keychain:` test strings) that were manually judged benign — yet called the sweep "clean". Sweeps must distinguish: no matches / matches reviewed and benign (enumerated or allowlisted) / unreviewed matches / real findings. Additional tests/harness assertions this evidence motivates: * label mutation without exact capability proof is blocked/downgraded * a report claiming comment-only claim while also performing a label mutation is downgraded (contradictory mutation account) * author PR report without head SHA is incomplete * a validation suite reported only as `exit 0` is downgraded * a sweep with matches cannot be reported as simply "clean" unless benign matches are enumerated or allowlisted * final report without Controller Handoff is downgraded Acceptance criterion (author-run analogue of this issue's): an author issue run earns an "A" only when it proves exact capability for every mutation (including label changes), includes PR number + branch + exact head SHA, reports exact validation counts for every claimed suite, reports secret/provenance sweep results with the four-way match taxonomy, and ends with a Controller Handoff. Cross-references: #183 (author-run reporting hardening — its gaps 1/4/5 overlap the handoff/sweep/head-SHA items here; the label-mutation capability contradiction and the sweep match-taxonomy are new evidence beyond #183); `review_proofs.assess_sweep_evidence` (merged via PR #193's branch, pending review) already requires exact method+scope and could be extended with the benign-match enumeration taxonomy when this issue is implemented. The unresolved `mark_issue` resolver gap itself remains tracked in #183.
jcwalker3 commented 2026-07-05 14:48:59 -05:00
Author
Owner
Copy Link
Copy Source

Positive Author-Run Reporting Example (Codex)

Evidence:
Codex was explicitly forced into author/implementer mode and successfully avoided PR review behavior.

Positive behavior observed:

  • Stayed in jcwalker3 / prgs-author.
  • Did not review, approve, request changes, merge, or manually close issues.
  • Checked both configured repos:
    • Scaled-Tech-Consulting/Gitea-Tools
    • Scaled-Tech-Consulting/mcp-control-plane
  • Used open PR inventory only for duplicate avoidance.
  • Selected issue #175 because it was unclaimed, testable, and not represented by an open PR.
  • Opened PR #192:
    • title: Add role-boundary proofs for reviewer queue workflows (Issue #175)
    • branch: feat/issue-175-role-pivot-wall
    • head: 574a9ea7c109d04de170b7004d2b8f2b9e2f30cf
  • Included a Controller Handoff.
  • Reported exact validation:
    • python3 -m py_compile review_proofs.py tests/test_review_proofs.py: passed
    • python3 -m py_compile $(git ls-files '*.py'): passed
    • pytest tests/test_review_proofs.py: 49 passed
    • pytest tests/test_review_proofs.py tests/test_pr_queue_inventory.py: 65 passed
    • full suite: 689 passed, 6 skipped
    • git diff --check prgs/master...HEAD: passed
  • Reported exact secret/provenance sweep command:
    • git diff prgs/master...HEAD | grep -iE 'token|secret|password|key|credential|http|Authorization:|Bearer |BEGIN (RSA|OPENSSH|PRIVATE) KEY' || true
    • result: no matches

Remaining A-level improvements:

  • Mutation capability proof should be explicit for each mutation:
    • issue comment / claim
    • branch create
    • push branch
    • PR create
  • Final PR creation report should verify live PR head SHA matches the pushed commit.
  • Final reports should avoid noisy internal pseudo-action lines such as ::git-create-branch unless they are part of a formal trace.

Use this as a positive example that the author-only prompt can prevent Codex from drifting into PR review work.

### Positive Author-Run Reporting Example (Codex) **Evidence:** Codex was explicitly forced into author/implementer mode and successfully avoided PR review behavior. **Positive behavior observed:** * Stayed in `jcwalker3 / prgs-author`. * Did not review, approve, request changes, merge, or manually close issues. * Checked both configured repos: * `Scaled-Tech-Consulting/Gitea-Tools` * `Scaled-Tech-Consulting/mcp-control-plane` * Used open PR inventory only for duplicate avoidance. * Selected issue #175 because it was unclaimed, testable, and not represented by an open PR. * Opened PR #192: * title: `Add role-boundary proofs for reviewer queue workflows (Issue #175)` * branch: `feat/issue-175-role-pivot-wall` * head: `574a9ea7c109d04de170b7004d2b8f2b9e2f30cf` * Included a Controller Handoff. * Reported exact validation: * `python3 -m py_compile review_proofs.py tests/test_review_proofs.py`: passed * `python3 -m py_compile $(git ls-files '*.py')`: passed * `pytest tests/test_review_proofs.py`: `49 passed` * `pytest tests/test_review_proofs.py tests/test_pr_queue_inventory.py`: `65 passed` * full suite: `689 passed, 6 skipped` * `git diff --check prgs/master...HEAD`: passed * Reported exact secret/provenance sweep command: * `git diff prgs/master...HEAD | grep -iE 'token|secret|password|key|credential|http|Authorization:|Bearer |BEGIN (RSA|OPENSSH|PRIVATE) KEY' || true` * result: no matches **Remaining A-level improvements:** * Mutation capability proof should be explicit for each mutation: * issue comment / claim * branch create * push branch * PR create * Final PR creation report should verify live PR head SHA matches the pushed commit. * Final reports should avoid noisy internal pseudo-action lines such as `::git-create-branch` unless they are part of a formal trace. Use this as a positive example that the author-only prompt can prevent Codex from drifting into PR review work.
jcwalker3 added the status:in-progress label 2026-07-06 10:33:09 -05:00
jcwalker3 commented 2026-07-06 10:35:30 -05:00
Author
Owner
Copy Link
Copy Source

Claimed and implemented on feat/issue-191-issue-filing-reports.

Approach: assess_issue_filing_final_report composes handoff, issue reference, SHA evidence, mutation capability/scope, and duplicate-search summary checks. Issue-filing handoff role fields added to assess_controller_handoff.

Tests: 5 unit tests covering A-grade report, missing handoff, abbreviated SHA, missing capability proof, and label-specific proof. All pass.

PR opened for review.

Claimed and implemented on `feat/issue-191-issue-filing-reports`. **Approach:** `assess_issue_filing_final_report` composes handoff, issue reference, SHA evidence, mutation capability/scope, and duplicate-search summary checks. Issue-filing handoff role fields added to `assess_controller_handoff`. **Tests:** 5 unit tests covering A-grade report, missing handoff, abbreviated SHA, missing capability proof, and label-specific proof. All pass. PR opened for review.
sysadmin added bug and removed status:in-progress labels 2026-07-06 12:28:36 -05:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
allocator
Controller-owned work allocator
 anti-stomp
Prevent concurrent LLM session stomping
 architecture
Architecture / structural design
 bug
chore
codex
OpenAI Codex client / workflow session surface
 concurrency
Concurrent session safety
 contamination
Workflow or session contamination incident
 control-plane
MCP control-plane coordination and allocation authority
 dashboard
MCP operational dashboard/queue view
 database
Internal coordination storage (SQLite/Postgres)
 design
Design / investigation, no implementation
 documentation
Docs / runbooks
 enhancement
New feature or improvement
 gitea
Gitea MCP workflow
 glitchtip
GlitchTip integration
 important
incident
Operational or process incident requiring durable audit trail
 incident-bridge
Sentry-to-Gitea incident bridging
 integration
Integration testing
 jenkins
Jenkins integration
 labels
Label taxonomy management
 leases
Lease adopt/release/expire lifecycle
 mcp
MCP server / tooling
 mcp-health
MCP namespace and runtime health
 mcp-menu
MCP menu surface
 multi-project
Work spanning multiple monitoring projects or Gitea repos
 mutating
Mutating action; requires gating
 nice-to-have
observability
Observability, metrics, traces, error reporting
 portability
Cross-platform / portability
 preflight
Shared preflight gates before mutation
 protected-branch
Protected branch / stable-branch policy concern
 queue
Work queue visibility and allocation
 read-only
Read-only, no mutation
 reconnect
MCP client reconnect/reload recovery path
 recovery
Recovery paths for stale/foreign leases
 refactor
Code refactor / restructure
 release
Release / versioning
 reliability
Reliability / failure handling
 resumable-review
Persist and resume prepared review verdicts across sessions
 reviewer
Reviewer workflow tooling
 roadmap
Roadmap / umbrella issue
 safety
Safety rails and fail-closed mutation guards
 security
Security / trust boundary
 self-hosted
Self-hosted infrastructure integration
 sentry
Sentry error monitoring integration
 stale-runtime
Stale backend daemon / runtime-vs-master parity failures
 status:blocked
Issue is blocked
 status:in-progress
Issue is being worked on
 status:pr-open
Issue has an open pull request
 status:ready
Issue is ready for work
 terminal-lock
Terminal review lock (#332) path
 testing
Tests / test coverage
 tracker
Issue tracker hygiene / meta
 type:bug
Bug or defect
 type:feature
Feature or enhancement
 type:feature
Feature or enhancement
 type:guardrail
Safety gate or guardrail
 visibility
Workflow state visibility for LLMs/operators
 workflow
Cross-tool workflow
 workflow-hardening
LLM workflow coordination hardening
 workflow-hardening
LLM workflow coordination hardening
No labels bug
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
jcwalker3 sysadmin
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Scaled-Tech-Consulting/Gitea-Tools#191