feat: add root-level run-tests.sh full-validation runner (Closes #473)
Adds a canonical, discoverable full-validation entry point so sessions stop guessing between pytest invocations. - run-tests.sh: executable root runner; runs `venv/bin/python -m pytest "$@"`; forwards args; fails closed with a clear setup message when the venv Python is missing (no silent wrong-interpreter fallback); set -euo pipefail; no network, no Gitea, no lock files. - docs/developer-testing-guidelines.md: name ./run-tests.sh as the canonical full/focused validation command. - tests/test_run_tests_script.py: contract checks (exists, executable, strict bash flags, venv pytest + arg forwarding, fail-closed guard, repo-local, and the guide naming the runner). Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
@@ -13,6 +13,25 @@ credentials.** Every test mocks the HTTP client and the keychain/auth lookup.
|
|||||||
|
|
||||||
## 1. Standard test commands
|
## 1. Standard test commands
|
||||||
|
|
||||||
|
### Canonical runner: `./run-tests.sh`
|
||||||
|
|
||||||
|
The canonical full-validation command is the root-level runner. It invokes the
|
||||||
|
project virtualenv interpreter and passes any extra arguments straight through
|
||||||
|
to `pytest`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Full validation
|
||||||
|
./run-tests.sh
|
||||||
|
|
||||||
|
# Focused validation (extra args forward to pytest)
|
||||||
|
./run-tests.sh tests/test_mcp_server.py -q
|
||||||
|
```
|
||||||
|
|
||||||
|
`run-tests.sh` runs `venv/bin/python -m pytest "$@"` and fails with a clear
|
||||||
|
setup message if the virtualenv Python is missing (so a session never silently
|
||||||
|
falls back to the wrong interpreter). The explicit `venv/bin/python -m pytest`
|
||||||
|
forms below remain valid and equivalent.
|
||||||
|
|
||||||
The test suite needs the project virtualenv (it provides the MCP SDK):
|
The test suite needs the project virtualenv (it provides the MCP SDK):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
Executable
+13
@@ -0,0 +1,13 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
PYTHON="$ROOT_DIR/venv/bin/python"
|
||||||
|
|
||||||
|
if [[ ! -x "$PYTHON" ]]; then
|
||||||
|
echo "ERROR: expected virtualenv Python at $PYTHON" >&2
|
||||||
|
echo "Create the venv first, then run: venv/bin/python -m pytest" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
exec "$PYTHON" -m pytest "$@"
|
||||||
@@ -0,0 +1,65 @@
|
|||||||
|
"""Contract checks for the root-level `run-tests.sh` convenience runner (#473).
|
||||||
|
|
||||||
|
`run-tests.sh` is the canonical full-validation entry point. It must invoke the
|
||||||
|
project virtualenv interpreter, forward extra args to pytest, fail closed when
|
||||||
|
the venv Python is missing, and stay repo-local (no network, no lock files).
|
||||||
|
These checks pin that behavior so it cannot silently regress, and confirm the
|
||||||
|
developer testing guide names the runner.
|
||||||
|
"""
|
||||||
|
import stat
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||||
|
SCRIPT = REPO_ROOT / "run-tests.sh"
|
||||||
|
GUIDE = REPO_ROOT / "docs" / "developer-testing-guidelines.md"
|
||||||
|
|
||||||
|
|
||||||
|
def _script_text() -> str:
|
||||||
|
return SCRIPT.read_text(encoding="utf-8")
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_tests_script_exists():
|
||||||
|
assert SCRIPT.is_file(), "run-tests.sh must exist at the repository root"
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_tests_script_is_executable():
|
||||||
|
mode = SCRIPT.stat().st_mode
|
||||||
|
assert mode & stat.S_IXUSR, "run-tests.sh must be executable (chmod +x)"
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_tests_script_has_strict_bash_flags():
|
||||||
|
text = _script_text()
|
||||||
|
assert text.startswith("#!/usr/bin/env bash"), "must use the bash shebang"
|
||||||
|
assert "set -euo pipefail" in text, "must set -euo pipefail"
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_tests_script_uses_venv_pytest_and_forwards_args():
|
||||||
|
text = _script_text()
|
||||||
|
# Resolves the venv interpreter relative to the script's own directory.
|
||||||
|
assert "venv/bin/python" in text, "must use the project virtualenv Python"
|
||||||
|
assert "-m pytest" in text, "must run pytest via the module"
|
||||||
|
assert '"$@"' in text, "must forward extra CLI args to pytest"
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_tests_script_fails_closed_without_venv():
|
||||||
|
text = _script_text()
|
||||||
|
# Missing venv must be an explicit, non-zero-exit error, not a silent
|
||||||
|
# fallback to the wrong interpreter.
|
||||||
|
assert "if [[ ! -x" in text, "must guard on an executable venv Python"
|
||||||
|
assert "exit 1" in text, "must exit non-zero when the venv is missing"
|
||||||
|
assert "ERROR" in text, "must print a clear error message"
|
||||||
|
|
||||||
|
|
||||||
|
def test_run_tests_script_stays_repo_local():
|
||||||
|
text = _script_text()
|
||||||
|
# No network calls, no lock-file writes from the runner itself.
|
||||||
|
for forbidden in ("curl", "wget", "gitea_issue_lock.json"):
|
||||||
|
assert forbidden not in text, f"runner must not reference {forbidden!r}"
|
||||||
|
|
||||||
|
|
||||||
|
def test_guide_names_canonical_runner():
|
||||||
|
text = " ".join(GUIDE.read_text(encoding="utf-8").split())
|
||||||
|
assert "./run-tests.sh" in text, "testing guide must name ./run-tests.sh"
|
||||||
|
assert "./run-tests.sh tests/test_mcp_server.py -q" in text, (
|
||||||
|
"testing guide must show the focused-validation example"
|
||||||
|
)
|
||||||
Reference in New Issue
Block a user