feat: add root-level run-tests.sh full-validation runner (Closes #473)
Adds a canonical, discoverable full-validation entry point so sessions stop guessing between pytest invocations. - run-tests.sh: executable root runner; runs `venv/bin/python -m pytest "$@"`; forwards args; fails closed with a clear setup message when the venv Python is missing (no silent wrong-interpreter fallback); set -euo pipefail; no network, no Gitea, no lock files. - docs/developer-testing-guidelines.md: name ./run-tests.sh as the canonical full/focused validation command. - tests/test_run_tests_script.py: contract checks (exists, executable, strict bash flags, venv pytest + arg forwarding, fail-closed guard, repo-local, and the guide naming the runner). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
@@ -13,6 +13,25 @@ credentials.** Every test mocks the HTTP client and the keychain/auth lookup.
|
||||
|
||||
## 1. Standard test commands
|
||||
|
||||
### Canonical runner: `./run-tests.sh`
|
||||
|
||||
The canonical full-validation command is the root-level runner. It invokes the
|
||||
project virtualenv interpreter and passes any extra arguments straight through
|
||||
to `pytest`:
|
||||
|
||||
```bash
|
||||
# Full validation
|
||||
./run-tests.sh
|
||||
|
||||
# Focused validation (extra args forward to pytest)
|
||||
./run-tests.sh tests/test_mcp_server.py -q
|
||||
```
|
||||
|
||||
`run-tests.sh` runs `venv/bin/python -m pytest "$@"` and fails with a clear
|
||||
setup message if the virtualenv Python is missing (so a session never silently
|
||||
falls back to the wrong interpreter). The explicit `venv/bin/python -m pytest`
|
||||
forms below remain valid and equivalent.
|
||||
|
||||
The test suite needs the project virtualenv (it provides the MCP SDK):
|
||||
|
||||
```bash
|
||||
|
||||
Executable
+13
@@ -0,0 +1,13 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
PYTHON="$ROOT_DIR/venv/bin/python"
|
||||
|
||||
if [[ ! -x "$PYTHON" ]]; then
|
||||
echo "ERROR: expected virtualenv Python at $PYTHON" >&2
|
||||
echo "Create the venv first, then run: venv/bin/python -m pytest" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exec "$PYTHON" -m pytest "$@"
|
||||
@@ -0,0 +1,65 @@
|
||||
"""Contract checks for the root-level `run-tests.sh` convenience runner (#473).
|
||||
|
||||
`run-tests.sh` is the canonical full-validation entry point. It must invoke the
|
||||
project virtualenv interpreter, forward extra args to pytest, fail closed when
|
||||
the venv Python is missing, and stay repo-local (no network, no lock files).
|
||||
These checks pin that behavior so it cannot silently regress, and confirm the
|
||||
developer testing guide names the runner.
|
||||
"""
|
||||
import stat
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
SCRIPT = REPO_ROOT / "run-tests.sh"
|
||||
GUIDE = REPO_ROOT / "docs" / "developer-testing-guidelines.md"
|
||||
|
||||
|
||||
def _script_text() -> str:
|
||||
return SCRIPT.read_text(encoding="utf-8")
|
||||
|
||||
|
||||
def test_run_tests_script_exists():
|
||||
assert SCRIPT.is_file(), "run-tests.sh must exist at the repository root"
|
||||
|
||||
|
||||
def test_run_tests_script_is_executable():
|
||||
mode = SCRIPT.stat().st_mode
|
||||
assert mode & stat.S_IXUSR, "run-tests.sh must be executable (chmod +x)"
|
||||
|
||||
|
||||
def test_run_tests_script_has_strict_bash_flags():
|
||||
text = _script_text()
|
||||
assert text.startswith("#!/usr/bin/env bash"), "must use the bash shebang"
|
||||
assert "set -euo pipefail" in text, "must set -euo pipefail"
|
||||
|
||||
|
||||
def test_run_tests_script_uses_venv_pytest_and_forwards_args():
|
||||
text = _script_text()
|
||||
# Resolves the venv interpreter relative to the script's own directory.
|
||||
assert "venv/bin/python" in text, "must use the project virtualenv Python"
|
||||
assert "-m pytest" in text, "must run pytest via the module"
|
||||
assert '"$@"' in text, "must forward extra CLI args to pytest"
|
||||
|
||||
|
||||
def test_run_tests_script_fails_closed_without_venv():
|
||||
text = _script_text()
|
||||
# Missing venv must be an explicit, non-zero-exit error, not a silent
|
||||
# fallback to the wrong interpreter.
|
||||
assert "if [[ ! -x" in text, "must guard on an executable venv Python"
|
||||
assert "exit 1" in text, "must exit non-zero when the venv is missing"
|
||||
assert "ERROR" in text, "must print a clear error message"
|
||||
|
||||
|
||||
def test_run_tests_script_stays_repo_local():
|
||||
text = _script_text()
|
||||
# No network calls, no lock-file writes from the runner itself.
|
||||
for forbidden in ("curl", "wget", "gitea_issue_lock.json"):
|
||||
assert forbidden not in text, f"runner must not reference {forbidden!r}"
|
||||
|
||||
|
||||
def test_guide_names_canonical_runner():
|
||||
text = " ".join(GUIDE.read_text(encoding="utf-8").split())
|
||||
assert "./run-tests.sh" in text, "testing guide must name ./run-tests.sh"
|
||||
assert "./run-tests.sh tests/test_mcp_server.py -q" in text, (
|
||||
"testing guide must show the focused-validation example"
|
||||
)
|
||||
Reference in New Issue
Block a user