Files

T

sysadmin 625f835aa7 docs: fix invalid pytest node IDs in testing guide

2026-07-02 13:06:31 -04:00

9.0 KiB

Raw Blame History

Developer Testing Guidelines

How to write and run tests for Gitea-Tools. This guide reflects the current repository behavior and the safety model documented in safety-model.md, credential-isolation.md, and gitea-execution-profiles.md.

Core principle: tests never make real network calls and never touch real credentials. Every test mocks the HTTP client and the keychain/auth lookup.

1. Standard test commands

The test suite needs the project virtualenv (it provides the MCP SDK):

# From the repository root
source venv/bin/activate
python3 -m pytest tests/ -q

Or invoke the venv interpreter directly without activating:

./venv/bin/python -m pytest tests/ -q

Use -q for a compact summary and -v to see individual test names.

Run the full suite

./venv/bin/python -m pytest tests/ -q

Run targeted tests

# One file
./venv/bin/python -m pytest tests/test_mcp_server.py -q

# One class
./venv/bin/python -m pytest tests/test_merge_pr.py::TestMergeDisabled -q

# One test, by node id
./venv/bin/python -m pytest tests/test_review_pr.py::TestAPIPayload::test_payload_fields_and_workflow -q

# By keyword expression
./venv/bin/python -m pytest tests/ -q -k "merge and fails_closed"

2. Syntax and formatting checks

These are fast and belong in any pre-PR loop:

# Byte-compile the main modules (catches syntax errors)
python3 -m py_compile mcp_server.py
python3 -m py_compile manage_labels.py

# Lint shell scripts without executing them
bash -n scripts/clear-provenance

# Detect stray conflict markers and whitespace errors in the diff
git diff --check

Run git diff --check before every commit; it flags leftover merge-conflict markers and trailing-whitespace/whitespace-error lines.

3. How to add an MCP tool test

MCP tools live in mcp_server.py and are exercised in tests/test_mcp_server.py. Tests call the underlying tool function directly with the network layer and auth mocked. The established pattern:

from unittest.mock import patch, MagicMock

FAKE_AUTH = "Basic ZmFrZTpmYWtl"  # not a real credential

class TestCreateIssue(unittest.TestCase):
    @patch("mcp_server.api_request")
    @patch("mcp_server.get_auth_header", return_value=FAKE_AUTH)
    def test_creates_issue(self, _auth, mock_api):
        mock_api.return_value = {"number": 1, "html_url": "https://gitea.example.com/issues/1"}

        result = mcp_server.gitea_create_issue(title="Add tests", remote="prgs")

        # Assert on the request the tool would have made
        mock_api.assert_called_once()
        method, url = mock_api.call_args[0][0], mock_api.call_args[0][1]
        self.assertEqual(method, "POST")
        self.assertIn("/issues", url)

Checklist when adding a tool test:

Patch mcp_server.api_request — never hit the network.
Patch mcp_server.get_auth_header to return a fake header — never read the keychain.
Assert on the method, URL, and payload the tool builds, and on the shape of the returned payload.
Cover both dadeschools and prgs remotes when the tool takes remote, and confirm the correct host/org/repo are used.
Cover the error path (e.g. api_request raising) and confirm the tool surfaces a clear message without leaking secrets.

4. How to mock API requests safely

Always patch mcp_server.api_request (or gitea_auth.api_request for the CLI/auth-level tests). No test should open a socket.
Always patch the auth lookup (get_auth_header / get_credentials) and return an obviously fake value. Do not put a real token or password in a test, a fixture, or an environment file.
Prefer asserting on mock_api.call_args (method/URL/payload) over asserting on a real response body.
For keychain behavior specifically, see tests/test_credentials.py, which mocks the git credential fill subprocess (Popen) and the environment.

5. How to test profile / allowed-operation failures

The execution-profile model (see gitea-execution-profiles.md) enforces that a tool may only perform operations in its profile's allowed_operations, and that forbidden_operations always override allowed_operations. Mutating tools must fail closed when the active profile does not permit the operation.

When adding or changing a gated tool, add tests that:

Configure a profile whose allowed_operations does not include the requested operation, and assert the tool refuses without calling api_request (assert mock_api.assert_not_called()).
Configure a profile where the operation is in both allowed and forbidden, and assert forbidden wins (still refused).
Confirm the refusal message names the missing operation and does not include any secret material.
Confirm the happy path (operation allowed) still reaches api_request.

The guiding assertion is: no mutation path may reach api_request unless the profile/allowed-operation check passed first.

6. How to test self-review / self-merge gates

Author-cannot-review and author-cannot-merge are hard safety gates. The merge path is gated (gitea_merge_pr), the legacy review wrapper fails closed on merge=True, and gitea_submit_pr_review never merges. Existing coverage lives in tests/test_merge_pr.py and tests/test_review_pr.py.

Patterns to follow (see those files for concrete examples):

Self-merge blocked: authenticated user == PR author → the tool returns a refusal and never calls the merge endpoint (mock_api.assert_not_called() or assert no POST .../merge).
Fail-closed inputs: missing confirmation string, or an unexpected expected_head_sha/changed-file set → refuse before any API call.
Legacy wrapper: merge=True on the review wrapper fails closed and points to the gated workflow, with no API call (test_merge_flag_fails_closed_without_api_call).
Self-approval blocked: authenticated user == PR author → approve / request_changes refused.

Every new gate should have a test proving the mutating endpoint is not reached when the gate should block.

7. No-secret / no-token regression expectations

Secrets must never appear in logs, tool return values, audit records, or test output (see safety-model.md §3). The audit module (gitea_audit.py) redacts secret-like keys and value prefixes; see tests/test_audit.py.

Expectations for new tests:

Assert that token/authorization/password fields are replaced with gitea_audit.REDACTED in any structured output or audit record (test_redacts_secret_keys, test_redacts_nested_and_lists).
Assert that credential-looking substrings in free-text (error messages, reasons) are redacted (test_redacts_credential_value_prefixes, test_metadata_and_reason_redacted).
Never commit a real token/password, even in a fixture. Use obviously fake values (e.g. FAKE_AUTH above).
When a tool returns identity/profile metadata, assert it contains the non-secret fields (username, profile name) and not the token.

There is no third-party secret scanner wired into this repo today; secret safety is enforced by gitea_audit.redact plus the regression tests above. A quick manual sweep before a PR:

# Look for accidentally committed credentials in the diff
git diff --cached | grep -nEi "authorization: (basic|bearer)|password|token=[A-Za-z0-9]" || echo "clean"

8. Unit tests vs. future Docker integration tests

Unit tests (today, default): fast, fully mocked, no network, no keychain. This is where the vast majority of coverage lives and where new tests should go. They must stay fast and must not require credentials.
Docker/local-Gitea integration tests (planned, see #66): opt-in and skipped by default, gated behind an explicit environment variable and run against a pinned, disposable Gitea container. They validate real API behavior (pagination, permissions, label/PR-review endpoints, error payloads) that mocks cannot prove. They must not require production credentials and must not leak tokens.

Rule of thumb: prove logic and request-shaping with unit tests; reserve integration tests for real-server compatibility. Do not convert unit tests into network tests.

9. Read-only vs. mutating tool expectations

Read-only tools (e.g. gitea_whoami, gitea_view_*, gitea_list_*, gitea_get_profile): test that they never issue a mutating HTTP method and never require a mutation gate. Assert the request method is GET.
Mutating tools (create/edit/close/label, review, merge, mirror): test that they (a) pass the profile/allowed-operation gate, (b) honor confirmation and self-action gates, (c) emit an audit record with the authenticated identity and outcome, and (d) fail closed — no api_request call — when any gate fails.

Keep this split explicit in test names and assertions so a reviewer can see, per tool, which category it belongs to and which gates it must respect.

9.0 KiB Raw Blame History