docs: re-land release version SOP with v1.1.0 audit lessons (#111 )

2026-07-03 03:24:26 -04:00
4 changed files with 192 additions and 421 deletions
@@ -423,6 +423,9 @@ with the profile and authenticated user when `GITEA_AUDIT_LOG` is set (see
 ## Releases and version tags
 All release tagging, version bumps, and validation must comply with the [Release / Version Process SOP](release-version-sop.md).
 Versions follow SemVer — **`vMAJOR.MINOR.PATCH`**, using **`v0.x.y`** while
 unstable. Pick the bump by the largest change since the last tag:
@@ -0,0 +1,189 @@
 # Release / Version Process SOP
 Operator standard operating procedure for cutting a versioned release of
 Gitea-Tools: version bump, checks, merge, tag, and cleanup.
 > **Scope.** This is the **human/operator** SOP. It is deliberately distinct
 > from [`release-workflows.md`](release-workflows.md), which describes the
 > **future `release-mcp` orchestrator** boundary (a coordination concept), not
 > the day-to-day tagging process. When they disagree, this document governs how
 > a release is actually cut today.
 ---
 ## 1. Branch flow
 The repo is **`master`-based**. Releases are cut from `master`; there is no
 separate `dev`/`release` branch unless and until that is explicitly introduced
 and this SOP is updated to match. All work lands on `master` via reviewed PRs
 from short-lived, issue-linked branches (e.g. `docs/issue-68-...`).
 ## 2. Where "the version" lives
 There is **no `VERSION` file and no `CHANGELOG` file** in the repo today. The
 released version is expressed **only as an annotated git tag** of the form
 `vMAJOR.MINOR.PATCH` (existing tags: `v1.0.0`, `v1.0.1`). Release notes are
 carried as the **annotated tag's message** (via `--notes-file`), not a tracked
 changelog.
 > Do **not** confuse this with `SUPPORTED_VERSION` in `gitea_config.py` — that is
 > the **config-schema** version, unrelated to the application release version.
 If a `VERSION`/`CHANGELOG` file is added later, update this SOP to list it under
 "files to update".
 ## 3. Deciding the version bump (SemVer)
 Pick the bump against the last tag using semantic-versioning intent:
 * **PATCH** (`v1.0.1 → v1.0.2`): bug fixes, docs, tests, internal cleanups — no
  change to tool names, parameters, return payloads, or behavior.
 * **MINOR** (`v1.0.1 → v1.1.0`): backward-compatible additions — new MCP tool,
  new optional parameter, new script, additive behavior.
 * **MAJOR** (`v1.1.0 → v2.0.0`): backward-**incompatible** changes — renamed or
  removed tools, changed return-payload shape, changed default behavior, or a
  tightened safety gate that rejects previously-accepted input.
 When unsure between two levels, choose the higher one.
 ## 4. Preparing a version-bump / release PR
 Releases are still gated by the normal issue-first, PR-reviewed flow.
 1. Open (or use) a tracking issue for the release and **claim it** with
   `status:in-progress` (see §9).
 2. Create an isolated, issue-linked branch + worktree from latest `master`
   (e.g. `chore/issue-63-v1.1.0`). Never commit directly to `master`.
 3. Include in the PR:
   * Any code/docs changes that belong to the release.
   * The **release notes** for the annotated tag (draft them in the PR body or a
     notes file you will pass to `scripts/release-tag --notes-file`).
   * If a `VERSION`/`CHANGELOG` file exists at that time, its update.
 4. Open the PR **targeting `master`**.
 The tag is **not** created in the PR. Tagging happens only after merge (§6).
 ## 5. Required checks before release
 Run all of these green before merging the release PR and before tagging:
 ```bash
 python3 -m py_compile mcp_server.py
 python3 -m py_compile manage_labels.py
 bash -n scripts/clear-provenance
 ./venv/bin/python -m pytest tests/ -q
 git diff --check
 ```
 Plus a secret sweep (there is no third-party scanner wired in; do a staged-diff
 sweep — see [`developer-testing-guidelines.md`](developer-testing-guidelines.md)
 §7):
 ```bash
 git diff --cached | grep -nEi "authorization: (basic|bearer)|password[:=]|token=[A-Za-z0-9]" || echo "clean"
 ```
 `scripts/release-tag` **also** runs the test suite itself before tagging (unless
 `--skip-tests` is passed), so tests are enforced twice by default.
 ## 6. Running `scripts/release-tag`
 Tag **only after** the release PR is merged to `master`. `scripts/release-tag`
 enforces the tagging policy and is **safe by default** (creates nothing on a
 dry-run; never pushes without `--push`).
 Before it tags, it requires **all** of:
 * version matches `vMAJOR.MINOR.PATCH` (SemVer);
 * `fetch --prune` has run;
 * you are **on `master`**;
 * the worktree is **clean** (no uncommitted changes);
 * local `master` **equals** `<remote>/master`;
 * `HEAD` is that same commit (the commit is present on remote master);
 * the tag does **not** already exist locally or on the remote;
 * the test suite passes (unless `--skip-tests`, which warns).
 Typical sequence:
 ```bash
 # 1. Dry-run to confirm the plan (changes nothing)
 scripts/release-tag --dry-run v1.1.0
 # 2. Create the annotated tag locally, with release notes
 scripts/release-tag v1.1.0 --notes-file /path/to/release-notes.md
 # 3. Push the tag only when ready
 scripts/release-tag v1.1.0 --notes-file /path/to/release-notes.md --push
 ```
 Env injection points (mainly for CI/tests):
 `RELEASE_TAG_REMOTE` (default `prgs`), `RELEASE_TAG_TEST_CMD`
 (default `./venv/bin/python -m pytest tests/ -q`).
 ## 7. Who may merge / tag
 * The release PR must be **merged by someone other than its author** — the
  author-cannot-merge safety gate applies to releases exactly as to any other PR.
 * Merge uses the gated `gitea_merge_pr` workflow; CLI/legacy merge is disabled.
 * Whoever tags must operate on clean master synced to the remote (enforced by
  `scripts/release-tag`). Tagging is an operator action performed after merge.
 ## 8. Self-review / self-merge restrictions
 Release PRs are **not** exempt from the safety model:
 * No self-review — the author may not approve their own release PR.
 * No self-merge — a different eligible identity merges.
 * These gates are enforced by the MCP tooling and must not be bypassed.
 ## 9. Handling `status:in-progress` during release work
 * **Claim** the release tracking issue with `status:in-progress` before starting.
 * Keep it claimed while the release PR is open and under review.
 * On merge/close, the tracker-hygiene automation releases `status:in-progress`
  for issues the PR closes; if it remains after the release lands, release it
  explicitly. Do not leave a shipped release issue marked in-progress.
 ## 10. Branch / worktree cleanup after merge
 After the release PR merges and the tag is pushed:
 * Delete the remote release branch (if repo policy allows).
 * Remove the local worktree and delete the local branch:
  ```bash
  git worktree remove branches/<release-worktree>
  git branch -d <release-branch>
  git worktree prune
  ```
 * Confirm the root repo is clean and on `master` synced to the remote.
 ## 11. What NOT to do
 * **No direct commits to `master`.** All changes land via reviewed PRs.
 * **No force-push** (to `master` or to tags).
 * **No self-merge** of a release PR.
 * **No tagging before merge** — tag only commits already on remote `master`.
 * **No release from a dirty worktree** — `scripts/release-tag` refuses, and so
  should you.
 * **No `--skip-tests`** for a real release unless there is an explicit,
  documented reason.
 * **No re-tagging / moving an existing tag** — pick the next version instead.
 ## 12. Post-Merge Verification & Audit Lessons (v1.1.0)
 During the v1.1.0 release audit, we identified a critical reconciliation issue (captured in historical PRs/issues #68 and #82):
 * **The "Closed" State Trap:** Gitea PRs marked as `closed` are not guaranteed to be `merged` (they can be closed without merging, leading to silent omissions of code/documentation changes).
 * **Mandatory Post-Merge File/Commit Presence Probe:** Reviewers/mergers must perform explicit post-merge validation. Do not assume a merge succeeded.
  - Check that the merged branch head is an ancestor of the target branch (`master`):
    ```bash
    git fetch <remote> --prune
    git merge-base --is-ancestor <pr-head-sha> <remote>/master
    ```
  - Probe file presence for expected modifications/additions:
    ```bash
    git log --oneline -- <expected-file>
    # and confirm file presence:
    ls -la docs/release-version-sop.md
    ```
 * **Verify in Handoff:** Final report blocks must explicitly document the verification method and probe results.
@@ -1,224 +0,0 @@
 #!/usr/bin/env python3
 """Migration helper to convert profiles.json from version 1 to version 2 environments shape.
 This script preserves existing keychain references (auth.id) and maps old profile
 names as aliases so that existing IDE configurations continue to function.
 """
 import os
 import sys
 import json
 import argparse
 import shutil
 import tempfile
 # Resolve path to import gitea_config
 PROJECT_ROOT = os.path.dirname(os.path.abspath(__file__))
 if PROJECT_ROOT not in sys.path:
    sys.path.insert(0, PROJECT_ROOT)
 import gitea_config
 def migrate_v1_to_v2(v1_data):
    """Convert version 1 profiles.json format to version 2 environments format."""
    environments = {}
    aliases = {}
    profiles = v1_data.get("profiles", {})
    if not isinstance(profiles, dict):
        raise ValueError("Malformed input: 'profiles' field must be a JSON object")
    for name, prof in profiles.items():
        if not isinstance(prof, dict):
            raise ValueError(f"Malformed input: profile '{name}' must be a JSON object")
        # Infer environment and identity name
        if "-" in name:
            parts = name.split("-", 1)
            env_name = parts[0]
            ident_name = parts[1]
        else:
            env_name = name
            ident_name = "author"
        # Determine role and identity based on name / execution_profile
        role = "author"
        exec_prof = prof.get("execution_profile") or ""
        if "reviewer" in name or "reviewer" in exec_prof:
            role = "reviewer"
            ident_name = "reviewer"
        elif "author" in name or "author" in exec_prof:
            role = "author"
            ident_name = "author"
        # Construct identity block
        identity_data = {
            "role": role,
            "username": prof.get("username"),
            "auth": prof.get("auth"),
        }
        if prof.get("execution_profile"):
            identity_data["execution_profile"] = prof["execution_profile"]
        # Set audit label (default to old name to preserve context)
        identity_data["audit_label"] = prof.get("audit_label") or name
        # Populate capabilities based on role
        if role == "author":
            identity_data["allowed_operations"] = [
                "read", "branch", "commit", "push", "open_pr", "comment"
            ]
            identity_data["forbidden_operations"] = [
                "approve", "request_changes", "merge"
            ]
        else:
            identity_data["allowed_operations"] = [
                "read", "review", "comment", "approve", "request_changes", "merge"
            ]
            identity_data["forbidden_operations"] = [
                "branch", "commit", "push", "open_pr"
            ]
        # Nest inside environments/services structure
        env = environments.setdefault(env_name, {})
        services = env.setdefault("services", {})
        gitea_svc = services.setdefault("gitea", {})
        # Copy service-level attributes
        if prof.get("base_url"):
            gitea_svc["base_url"] = prof["base_url"]
        if prof.get("default_owner"):
            gitea_svc["default_owner"] = prof["default_owner"]
        if prof.get("default_repo"):
            gitea_svc["default_repo"] = prof["default_repo"]
        identities = gitea_svc.setdefault("identities", {})
        identities[ident_name] = identity_data
        # Alias resolution targets
        alias_target = f"{env_name}.gitea.{ident_name}"
        if name != alias_target:
            aliases[name] = alias_target
        # Extra convenience alias for standard old-profile compatibility (e.g. prgs-author)
        convenience_alias = f"{env_name}-{ident_name}"
        if convenience_alias != alias_target and convenience_alias not in aliases:
            aliases[convenience_alias] = alias_target
    v2_data = {
        "version": 2,
        "environments": environments,
        "aliases": aliases
    }
    return v2_data
 def validate_v2_data(v2_data):
    """Validate generated v2 structure using gitea_config parser."""
    fd, temp_path = tempfile.mkstemp(suffix=".json")
    os.close(fd)
    try:
        with open(temp_path, "w") as f:
            json.dump(v2_data, f)
        # Attempt to load using load_config to run all validation rules
        gitea_config.load_config(temp_path)
        return True
    except Exception as e:
        raise ValueError(f"Generated v2 config failed validation: {e}")
    finally:
        try:
            os.remove(temp_path)
        except OSError:
            pass
 def main():
    parser = argparse.ArgumentParser(
        description="Migrate profiles.json from version 1 to version 2 environments shape."
    )
    parser.add_argument(
        "-i", "--input",
        default=gitea_config.DEFAULT_CONFIG_PATH,
        help="Path to the version 1 profiles.json file (default: ~/.config/gitea-tools/profiles.json)"
    )
    parser.add_argument(
        "-o", "--output",
        help="Path to write the migrated version 2 profiles.json file (default: overwrite input)"
    )
    parser.add_argument(
        "-w", "--write",
        action="store_true",
        help="Actually write the migrated config and create a backup (default is dry-run)"
    )
    parser.add_argument(
        "--backup",
        help="Path to write the backup file (default: <input_path>.bak)"
    )
    args = parser.parse_args()
    input_path = os.path.abspath(args.input)
    output_path = os.path.abspath(args.output or input_path)
    backup_path = args.backup or f"{input_path}.bak"
    if not os.path.isfile(input_path):
        print(f"Error: Input file not found: {input_path}", file=sys.stderr)
        sys.exit(1)
    try:
        with open(input_path, "r") as f:
            v1_data = json.load(f)
    except json.JSONDecodeError as e:
        print(f"Error: Input file is not valid JSON: {e}", file=sys.stderr)
        sys.exit(1)
    except Exception as e:
        print(f"Error reading input file: {e}", file=sys.stderr)
        sys.exit(1)
    # Validate version
    version = v1_data.get("version")
    if version is not None and version != 1:
        print(f"Error: Unsupported profiles.json version: {version}. Expected version 1.", file=sys.stderr)
        sys.exit(1)
    try:
        v2_data = migrate_v1_to_v2(v1_data)
        validate_v2_data(v2_data)
    except Exception as e:
        print(f"Error: {e}", file=sys.stderr)
        sys.exit(1)
    if not args.write:
        print("=== DRY-RUN MODE (No files modified) ===")
        print(f"Would read from: {input_path}")
        print(f"Would create backup at: {backup_path}")
        print(f"Would write v2 config to: {output_path}")
        print("\nGenerated v2 config:")
        print(json.dumps(v2_data, indent=2))
        sys.exit(0)
    # Write Mode: Create Backup first
    try:
        print(f"Creating backup: {backup_path}")
        shutil.copy2(input_path, backup_path)
    except Exception as e:
        print(f"Error creating backup: {e}", file=sys.stderr)
        sys.exit(1)
    # Write migrated config
    try:
        print(f"Writing migrated version 2 config: {output_path}")
        # Ensure target directory exists
        os.makedirs(os.path.dirname(output_path), exist_ok=True)
        with open(output_path, "w") as f:
            json.dump(v2_data, f, indent=2)
            f.write("\n")
        print("Migration completed successfully!")
    except Exception as e:
        print(f"Error writing output file: {e}", file=sys.stderr)
        sys.exit(1)
 if __name__ == "__main__":
    main()
@@ -1,197 +0,0 @@
 """Unit tests for migrate_profiles.py migration helper."""
 import os
 import sys
 import json
 import unittest
 import tempfile
 import shutil
 from unittest.mock import patch
 from io import StringIO
 # Add project root to sys.path
 PROJECT_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
 if PROJECT_ROOT not in sys.path:
    sys.path.insert(0, PROJECT_ROOT)
 import migrate_profiles
 class TestMigrateProfiles(unittest.TestCase):
    def setUp(self):
        self.temp_dir = tempfile.mkdtemp()
        self.v1_content = {
            "version": 1,
            "profiles": {
                "prgs": {
                  "base_url": "https://gitea.prgs.cc",
                  "username": "jcwalker3",
                  "auth": {
                    "type": "keychain",
                    "id": "prgs-gitea-token"
                  },
                  "default_owner": "Scaled-Tech-Consulting",
                  "execution_profile": "personal-prgs"
                },
                "mdcps": {
                  "base_url": "https://gitea.dadeschools.net",
                  "username": "913443",
                  "auth": {
                    "type": "keychain",
                    "id": "mdcps-gitea-token"
                  },
                  "default_owner": "Contractor",
                  "execution_profile": "mdcps"
                },
                "prgs-reviewer": {
                  "base_url": "https://gitea.prgs.cc",
                  "username": "sysadmin",
                  "auth": {
                    "type": "keychain",
                    "id": "prgs-gitea-reviewer-token"
                  },
                  "default_owner": "Scaled-Tech-Consulting",
                  "execution_profile": "prgs-reviewer"
                }
            }
        }
        self.input_file = os.path.join(self.temp_dir, "profiles.json")
        with open(self.input_file, "w") as f:
            json.dump(self.v1_content, f)
    def tearDown(self):
        shutil.rmtree(self.temp_dir)
    def test_migration_logic(self):
        """Test the structural transformation and capability mapping."""
        v2_data = migrate_profiles.migrate_v1_to_v2(self.v1_content)
        self.assertEqual(v2_data["version"], 2)
        # Check environment structure
        envs = v2_data["environments"]
        self.assertIn("prgs", envs)
        self.assertIn("mdcps", envs)
        # Check service and identity structure
        prgs_gitea = envs["prgs"]["services"]["gitea"]
        self.assertEqual(prgs_gitea["base_url"], "https://gitea.prgs.cc")
        self.assertEqual(prgs_gitea["default_owner"], "Scaled-Tech-Consulting")
        author = prgs_gitea["identities"]["author"]
        self.assertEqual(author["role"], "author")
        self.assertEqual(author["username"], "jcwalker3")
        self.assertEqual(author["auth"]["id"], "prgs-gitea-token")
        self.assertIn("push", author["allowed_operations"])
        reviewer = prgs_gitea["identities"]["reviewer"]
        self.assertEqual(reviewer["role"], "reviewer")
        self.assertEqual(reviewer["username"], "sysadmin")
        self.assertEqual(reviewer["auth"]["id"], "prgs-gitea-reviewer-token")
        self.assertIn("merge", reviewer["allowed_operations"])
    def test_alias_generation(self):
        """Test that aliases are correctly generated to support old profile names."""
        v2_data = migrate_profiles.migrate_v1_to_v2(self.v1_content)
        aliases = v2_data["aliases"]
        self.assertEqual(aliases["prgs"], "prgs.gitea.author")
        self.assertEqual(aliases["prgs-author"], "prgs.gitea.author")
        self.assertEqual(aliases["prgs-reviewer"], "prgs.gitea.reviewer")
        self.assertEqual(aliases["mdcps"], "mdcps.gitea.author")
    def test_no_secret_behavior(self):
        """Ensure secrets are never extracted, printed, or processed."""
        v2_data = migrate_profiles.migrate_v1_to_v2(self.v1_content)
        # Check that auth structures only contain keychain references, not credentials
        for env in v2_data["environments"].values():
            for svc in env["services"].values():
                for ident in svc["identities"].values():
                    auth = ident["auth"]
                    self.assertEqual(auth["type"], "keychain")
                    self.assertIn("id", auth)
                    self.assertNotIn("token", auth)
                    self.assertNotIn("password", auth)
    def test_validation(self):
        """Test that the generated v2 configuration validates against Gitea-Tools v2 loader."""
        v2_data = migrate_profiles.migrate_v1_to_v2(self.v1_content)
        self.assertTrue(migrate_profiles.validate_v2_data(v2_data))
    @patch("sys.stdout", new_callable=StringIO)
    def test_dry_run_default(self, mock_stdout):
        """Verify that running without -w prints generated config without modifying files."""
        output_file = os.path.join(self.temp_dir, "migrated_dry.json")
        test_args = [
            "migrate_profiles.py",
            "-i", self.input_file,
            "-o", output_file
        ]
        with patch.object(sys, "argv", test_args):
            with self.assertRaises(SystemExit) as cm:
                migrate_profiles.main()
            self.assertEqual(cm.exception.code, 0)
        self.assertFalse(os.path.exists(output_file))
        self.assertFalse(os.path.exists(f"{self.input_file}.bak"))
        stdout_output = mock_stdout.getvalue()
        self.assertIn("DRY-RUN MODE", stdout_output)
        self.assertIn("version", stdout_output)
        self.assertIn("environments", stdout_output)
    def test_write_mode_and_backup(self):
        """Verify that write mode creates a backup and correctly saves the validated config."""
        output_file = os.path.join(self.temp_dir, "migrated.json")
        backup_file = os.path.join(self.temp_dir, "profiles_backup.json.bak")
        test_args = [
            "migrate_profiles.py",
            "-i", self.input_file,
            "-o", output_file,
            "--backup", backup_file,
            "-w"
        ]
        with patch.object(sys, "argv", test_args):
            migrate_profiles.main()
        # Verify backup exists and matches original v1 config
        self.assertTrue(os.path.exists(backup_file))
        with open(backup_file, "r") as f:
            backup_data = json.load(f)
        self.assertEqual(backup_data["version"], 1)
        self.assertIn("prgs", backup_data["profiles"])
        # Verify migrated v2 config exists and validates
        self.assertTrue(os.path.exists(output_file))
        with open(output_file, "r") as f:
            v2_data = json.load(f)
        self.assertEqual(v2_data["version"], 2)
        self.assertIn("environments", v2_data)
        self.assertEqual(v2_data["aliases"]["prgs"], "prgs.gitea.author")
    def test_malformed_input_fails_safely(self):
        """Test that malformed JSON or invalid version numbers cause a clean exit with code 1."""
        bad_json_file = os.path.join(self.temp_dir, "bad.json")
        with open(bad_json_file, "w") as f:
            f.write("{invalid-json}")
        test_args = ["migrate_profiles.py", "-i", bad_json_file]
        with patch.object(sys, "argv", test_args):
            with self.assertRaises(SystemExit) as cm:
                migrate_profiles.main()
            self.assertEqual(cm.exception.code, 1)
        bad_version_file = os.path.join(self.temp_dir, "bad_version.json")
        with open(bad_version_file, "w") as f:
            json.dump({"version": 3, "profiles": {}}, f)
        test_args = ["migrate_profiles.py", "-i", bad_version_file]
        with patch.object(sys, "argv", test_args):
            with self.assertRaises(SystemExit) as cm:
                migrate_profiles.main()
            self.assertEqual(cm.exception.code, 1)
 if __name__ == "__main__":
    unittest.main()