Reviewer final reports must prove inventory completeness and worktree state consistently #293

Closed
opened 2026-07-07 00:14:47 -05:00 by jcwalker3 · 1 comment
Owner

Reviewer workflows need stricter proof and final-report validation for PR inventory and worktree reporting.

Observed failure patterns:

  1. A reviewer report claimed the PR inventory was complete because the first page returned 10 open PRs and assumed the default Gitea API cap was 50.
  2. This is unsafe unless the tool proves the requested page size, the returned page size, and no-next-page/final-page evidence.
  3. The report used a review worktree under branches/, but the controller handoff still said Scratch worktree used: False.
  4. The report used contradictory checkout wording such as Detached HEAD / Branch master.
  5. These inconsistencies make it harder to trust merge reports.

Required behavior:

  1. PR inventory completeness must be proven by one of:

    • explicit no-next-page / has_more=false / final-page response from the MCP tool
    • paginated traversal until an empty or final page is returned
    • explicit requested page size and response count proving the page limit was not hit
  2. A report may not claim oldest eligible PR or next eligible PR without inventory completeness proof.

  3. Final reports must distinguish:

    • main checkout branch/state
    • review worktree path
    • whether the review worktree was detached
    • whether the review worktree was inside branches/
  4. Scratch worktree used: False must be rejected if any branches/ worktree was created or used.

  5. Replace ambiguous field names:

    • use Review worktree used: true/false
    • use Review worktree path: ...
    • use Main checkout branch: master/main/dev
    • use Review worktree HEAD state: detached/branch
  6. Final-report validation must reject contradictory checkout wording and inconsistent worktree flags.

Acceptance criteria:

  • Reviewer reports cannot claim inventory completeness from a partial first-page result.
  • Reviewer reports cannot claim oldest/next eligible PR without final-page proof.
  • Reports using a branches/ review worktree must say Review worktree used: true.
  • Reports distinguish main checkout state from review worktree state.
  • Tests cover exactly-full first page, incomplete pagination, no-next-page proof, review worktree used, and contradictory branch/detached wording.
Reviewer workflows need stricter proof and final-report validation for PR inventory and worktree reporting. Observed failure patterns: 1. A reviewer report claimed the PR inventory was complete because the first page returned 10 open PRs and assumed the default Gitea API cap was 50. 2. This is unsafe unless the tool proves the requested page size, the returned page size, and no-next-page/final-page evidence. 3. The report used a review worktree under `branches/`, but the controller handoff still said `Scratch worktree used: False`. 4. The report used contradictory checkout wording such as `Detached HEAD / Branch master`. 5. These inconsistencies make it harder to trust merge reports. Required behavior: 1. PR inventory completeness must be proven by one of: * explicit no-next-page / has_more=false / final-page response from the MCP tool * paginated traversal until an empty or final page is returned * explicit requested page size and response count proving the page limit was not hit 2. A report may not claim `oldest eligible PR` or `next eligible PR` without inventory completeness proof. 3. Final reports must distinguish: * main checkout branch/state * review worktree path * whether the review worktree was detached * whether the review worktree was inside `branches/` 4. `Scratch worktree used: False` must be rejected if any `branches/` worktree was created or used. 5. Replace ambiguous field names: * use `Review worktree used: true/false` * use `Review worktree path: ...` * use `Main checkout branch: master/main/dev` * use `Review worktree HEAD state: detached/branch` 6. Final-report validation must reject contradictory checkout wording and inconsistent worktree flags. Acceptance criteria: * Reviewer reports cannot claim inventory completeness from a partial first-page result. * Reviewer reports cannot claim oldest/next eligible PR without final-page proof. * Reports using a `branches/` review worktree must say `Review worktree used: true`. * Reports distinguish main checkout state from review worktree state. * Tests cover exactly-full first page, incomplete pagination, no-next-page proof, review worktree used, and contradictory branch/detached wording.
jcwalker3 added the status:in-progress label 2026-07-07 04:40:16 -05:00
Author
Owner

Implementation ready for review in PR #369.

Adds assess_inventory_worktree_report() verifier with tests covering partial pagination, exactly-full first pages, branches/ worktree flags, and contradictory checkout wording. Wired into build_final_report.

Implementation ready for review in PR #369. Adds `assess_inventory_worktree_report()` verifier with tests covering partial pagination, exactly-full first pages, branches/ worktree flags, and contradictory checkout wording. Wired into `build_final_report`.
sysadmin removed the status:in-progress label 2026-07-07 08:47:08 -05:00
Sign in to join this conversation.
No labels
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: Scaled-Tech-Consulting/Gitea-Tools#293