Files
github-spec-kit/tests/test_check_prerequisites_python_parity.py
Marsel Safin 2a0ada9a6a feat(scripts): port create-new-feature, setup-plan and setup-tasks to Python (#3386)
* feat(scripts): port create-new-feature, setup-plan and setup-tasks to Python

Ports the three core workflow scripts to Python as part of #3280,
following the check-prerequisites PoC pattern from #3302. Adds
resolve_template() to the shared common.py module and parity tests
that run bash and Python side by side.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(tests): treat only None env as unset in parity run helper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): fall back to directory scan on any registry error, skip hidden preset dirs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(templates): add py: lines for setup_plan and setup_tasks

Ships with the scripts they reference; the remaining templates got
their py: lines in #3403.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: support py variant in skills placeholder resolver

resolve_skill_placeholders only accepted sh/ps, so a py init option
fell into the fallback path and {SCRIPT} rendered without an
interpreter prefix. Accept py and prefix the resolved interpreter,
matching process_template. Also guard ps_cmd against a missing
PowerShell with a clear assert.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: pin clean-error behavior for invalid --number

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(scripts): reword unused-arg comment to match implementation

The loop accepts and silently ignores extra positional args (it doesn't
build a collected list); match the wording to what the code and
setup-plan.sh actually do.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: fall back when configured script variant is missing from frontmatter

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): reject signed/whitespace --number values to match bash 10# parity

The bash twin uses $((10#$BRANCH_NUMBER)), which rejects signed and
whitespace-padded values. Python's int() accepted them (e.g. -1),
producing a malformed -01-... prefix that sequential scans ignore.
Restrict --number to unsigned decimal digits before conversion, and
pin the parity with a bash-comparison test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): complete Python port installation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(integrations): fall back for missing script variants

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: make Python script checks platform-aware

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix Windows Python command invocation parity

Use PowerShell's call operator for spaced Python interpreter paths and align setup-tasks missing-template errors across script variants.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): preserve cross-platform Python parity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: reject signed PowerShell feature numbers

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): align feature number range

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): reject exhausted feature numbers

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): complete create feature parity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): align create feature outputs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): harden cross-platform parity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): keep truncation JSON clean

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): align setup failure parity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): close parity edge cases

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): propagate PowerShell setup errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): harden fallback resolution

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): stabilize PowerShell fallbacks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): complete setup-plan parity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(cli): require runnable script fallbacks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(cli): preserve shell fallback without preference

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): restore help and symlink parity

- setup-tasks.ps1: check -Help before unknown-argument validation so
  '-Help --bogus' exits 0 like the Bash/Python variants
- common.py: strip the repo root prefix lexically in persist_feature_json
  instead of resolve(), so a symlinked specs/ still persists the relative
  'specs/NNN-name' path the Bash/PowerShell helpers store

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): align persist-hint quoting with shlex.quote

- create-new-feature.sh: replace printf %q with a shell_quote helper that
  emits shlex.quote-identical output, so the persistence hints stay
  byte-identical between the Bash and Python variants (printf %q output
  also varies between bash versions)
- promote the negative --number test to an all-variants parity test now
  that Bash and PowerShell reject signed values consistently
- add a spaced-repo-path parity test for the persistence hints

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-21 11:40:15 -05:00

373 lines
13 KiB
Python

"""Parity tests for the Python check-prerequisites PoC."""
from __future__ import annotations
import json
import os
import shutil
import subprocess
import sys
from pathlib import Path
import pytest
from tests.conftest import requires_bash
PROJECT_ROOT = Path(__file__).resolve().parent.parent
COMMON_SH = PROJECT_ROOT / "scripts" / "bash" / "common.sh"
CHECK_PREREQS_SH = PROJECT_ROOT / "scripts" / "bash" / "check-prerequisites.sh"
COMMON_PS = PROJECT_ROOT / "scripts" / "powershell" / "common.ps1"
CHECK_PREREQS_PS = PROJECT_ROOT / "scripts" / "powershell" / "check-prerequisites.ps1"
COMMON_PY = PROJECT_ROOT / "scripts" / "python" / "common.py"
CHECK_PREREQS_PY = PROJECT_ROOT / "scripts" / "python" / "check_prerequisites.py"
HAS_PWSH = shutil.which("pwsh") is not None
_WINDOWS_POWERSHELL = (
shutil.which("powershell.exe") or shutil.which("powershell")
) if os.name == "nt" else None
def _install_scripts(repo: Path) -> None:
bash_dir = repo / ".specify" / "scripts" / "bash"
bash_dir.mkdir(parents=True, exist_ok=True)
shutil.copy(COMMON_SH, bash_dir / "common.sh")
shutil.copy(CHECK_PREREQS_SH, bash_dir / "check-prerequisites.sh")
ps_dir = repo / ".specify" / "scripts" / "powershell"
ps_dir.mkdir(parents=True, exist_ok=True)
shutil.copy(COMMON_PS, ps_dir / "common.ps1")
shutil.copy(CHECK_PREREQS_PS, ps_dir / "check-prerequisites.ps1")
py_dir = repo / ".specify" / "scripts" / "python"
py_dir.mkdir(parents=True, exist_ok=True)
shutil.copy(COMMON_PY, py_dir / "common.py")
shutil.copy(CHECK_PREREQS_PY, py_dir / "check_prerequisites.py")
def _write_feature_json(
repo: Path, feature_directory: str = "specs/001-my-feature"
) -> None:
(repo / ".specify" / "feature.json").write_text(
json.dumps({"feature_directory": feature_directory}, separators=(",", ":"))
+ "\n",
encoding="utf-8",
)
def _clean_env() -> dict[str, str]:
env = os.environ.copy()
for key in list(env):
if key.startswith("SPECIFY_"):
env.pop(key)
return env
def _git_init(repo: Path) -> None:
subprocess.run(["git", "init", "-q"], cwd=repo, check=True)
subprocess.run(
["git", "config", "user.email", "test@example.com"], cwd=repo, check=True
)
subprocess.run(["git", "config", "user.name", "Test User"], cwd=repo, check=True)
subprocess.run(
["git", "commit", "--allow-empty", "-m", "init", "-q"], cwd=repo, check=True
)
@pytest.fixture
def prereq_repo(tmp_path: Path) -> Path:
repo = tmp_path / "proj"
repo.mkdir()
_git_init(repo)
(repo / ".specify").mkdir()
_install_scripts(repo)
return repo
def _py_cmd(repo: Path, *args: str) -> list[str]:
script = repo / ".specify" / "scripts" / "python" / "check_prerequisites.py"
return [sys.executable, str(script), *args]
def _repo_copy_py_cmd(repo: Path, *args: str) -> list[str]:
script = repo / "scripts" / "python" / "check_prerequisites.py"
return [sys.executable, str(script), *args]
def _bash_cmd(repo: Path, *args: str) -> list[str]:
script = repo / ".specify" / "scripts" / "bash" / "check-prerequisites.sh"
return ["bash", str(script), *args]
def _ps_cmd(repo: Path, *args: str) -> list[str]:
script = repo / ".specify" / "scripts" / "powershell" / "check-prerequisites.ps1"
exe = "pwsh" if HAS_PWSH else _WINDOWS_POWERSHELL
return [exe, "-NoProfile", "-File", str(script), *args]
def _run(
cmd: list[str], repo: Path, env: dict[str, str] | None = None
) -> subprocess.CompletedProcess[str]:
return subprocess.run(
cmd,
cwd=repo,
capture_output=True,
text=True,
check=False,
env=env or _clean_env(),
)
def _json_stdout(result: subprocess.CompletedProcess[str]) -> object:
return json.loads(result.stdout)
def _normalize_status_text(text: str) -> str:
return (
text.replace("", " [OK] ")
.replace("", " [FAIL] ")
.replace("\r\n", "\n")
)
def _normalize_help_text(text: str) -> str:
normalized = text.replace("\r\n", "\n").replace(
"check-prerequisites.sh", "check_prerequisites.py"
)
return "\n".join("" if not line.strip() else line for line in normalized.split("\n"))
@requires_bash
@pytest.mark.parametrize(
"args",
[
("--json",),
("--json", "--include-tasks"),
("--json", "--require-tasks", "--include-tasks"),
("--json", "--paths-only"),
],
)
def test_python_json_output_matches_bash(prereq_repo: Path, args: tuple[str, ...]) -> None:
feat = prereq_repo / "specs" / "001-my-feature"
feat.mkdir(parents=True)
(feat / "plan.md").write_text("# plan\n", encoding="utf-8")
(feat / "tasks.md").write_text("# tasks\n", encoding="utf-8")
(feat / "research.md").write_text("# research\n", encoding="utf-8")
(feat / "data-model.md").write_text("# model\n", encoding="utf-8")
(feat / "quickstart.md").write_text("# quickstart\n", encoding="utf-8")
(feat / "contracts" / "v1").mkdir(parents=True)
_write_feature_json(prereq_repo)
bash = _run(_bash_cmd(prereq_repo, *args), prereq_repo)
py = _run(_py_cmd(prereq_repo, *args), prereq_repo)
assert py.returncode == bash.returncode == 0
assert py.stderr == bash.stderr == ""
assert _json_stdout(py) == _json_stdout(bash)
@requires_bash
def test_python_text_output_matches_bash(prereq_repo: Path) -> None:
feat = prereq_repo / "specs" / "001-my-feature"
feat.mkdir(parents=True)
(feat / "plan.md").write_text("# plan\n", encoding="utf-8")
(feat / "contracts").mkdir()
_write_feature_json(prereq_repo)
bash = _run(_bash_cmd(prereq_repo, "--include-tasks"), prereq_repo)
py = _run(_py_cmd(prereq_repo, "--include-tasks"), prereq_repo)
assert py.returncode == bash.returncode == 0
assert py.stderr == bash.stderr == ""
assert _normalize_status_text(py.stdout) == _normalize_status_text(bash.stdout)
@requires_bash
def test_python_help_output_matches_bash(prereq_repo: Path) -> None:
bash = _run(_bash_cmd(prereq_repo, "--help"), prereq_repo)
py = _run(_py_cmd(prereq_repo, "--help"), prereq_repo)
assert py.returncode == bash.returncode == 0
assert py.stderr == bash.stderr == ""
assert _normalize_help_text(py.stdout) == _normalize_help_text(bash.stdout)
@requires_bash
def test_python_unknown_option_matches_bash_error_shape(prereq_repo: Path) -> None:
bash = _run(_bash_cmd(prereq_repo, "--bogus"), prereq_repo)
py = _run(_py_cmd(prereq_repo, "--bogus"), prereq_repo)
assert py.returncode == bash.returncode == 1
assert py.stdout == bash.stdout == ""
assert py.stderr == bash.stderr
@pytest.mark.skipif(not (HAS_PWSH or _WINDOWS_POWERSHELL), reason="no PowerShell available")
@pytest.mark.parametrize(
("py_args", "ps_args"),
[
(("--json",), ("-Json",)),
(("--json", "--include-tasks"), ("-Json", "-IncludeTasks")),
(
("--json", "--require-tasks", "--include-tasks"),
("-Json", "-RequireTasks", "-IncludeTasks"),
),
(("--json", "--paths-only"), ("-Json", "-PathsOnly")),
],
ids=[
"json",
"json_include_tasks",
"json_require_tasks_include_tasks",
"json_paths_only",
],
)
def test_python_json_output_matches_powershell(
prereq_repo: Path, py_args: tuple[str, ...], ps_args: tuple[str, ...]
) -> None:
feat = prereq_repo / "specs" / "001-my-feature"
feat.mkdir(parents=True)
(feat / "plan.md").write_text("# plan\n", encoding="utf-8")
(feat / "tasks.md").write_text("# tasks\n", encoding="utf-8")
(feat / "research.md").write_text("# research\n", encoding="utf-8")
(feat / "data-model.md").write_text("# model\n", encoding="utf-8")
(feat / "quickstart.md").write_text("# quickstart\n", encoding="utf-8")
(feat / "contracts" / "v1").mkdir(parents=True)
_write_feature_json(prereq_repo)
ps = _run(_ps_cmd(prereq_repo, *ps_args), prereq_repo)
py = _run(_py_cmd(prereq_repo, *py_args), prereq_repo)
assert py.returncode == ps.returncode == 0
assert py.stderr == ps.stderr == ""
assert _json_stdout(py) == _json_stdout(ps)
def test_python_repo_copy_script_file_fallback_finds_repo_root(tmp_path: Path) -> None:
repo = tmp_path / "proj"
outside = tmp_path / "outside"
repo.mkdir()
outside.mkdir()
_git_init(repo)
(repo / ".specify").mkdir()
_write_feature_json(repo)
(repo / "specs" / "001-my-feature").mkdir(parents=True)
py_dir = repo / "scripts" / "python"
py_dir.mkdir(parents=True)
shutil.copy(COMMON_PY, py_dir / "common.py")
shutil.copy(CHECK_PREREQS_PY, py_dir / "check_prerequisites.py")
py = _run(_repo_copy_py_cmd(repo, "--json", "--paths-only"), outside)
assert py.returncode == 0, py.stderr
assert Path(_json_stdout(py)["REPO_ROOT"]) == repo
def test_python_paths_only_does_not_persist_feature_json(prereq_repo: Path) -> None:
(prereq_repo / "specs" / "001-my-feature").mkdir(parents=True)
(prereq_repo / "specs" / "002-other").mkdir(parents=True)
_write_feature_json(prereq_repo, "specs/001-my-feature")
feature_json = prereq_repo / ".specify" / "feature.json"
before = feature_json.read_text(encoding="utf-8")
env = _clean_env()
env["SPECIFY_FEATURE_DIRECTORY"] = "specs/002-other"
py = _run(_py_cmd(prereq_repo, "--json", "--paths-only"), prereq_repo, env=env)
assert py.returncode == 0, py.stderr
assert "002-other" in _json_stdout(py)["FEATURE_DIR"]
assert feature_json.read_text(encoding="utf-8") == before
def test_python_normal_mode_persists_feature_json(prereq_repo: Path) -> None:
(prereq_repo / "specs" / "001-my-feature").mkdir(parents=True)
feat = prereq_repo / "specs" / "002-other"
feat.mkdir(parents=True)
(feat / "plan.md").write_text("# plan\n", encoding="utf-8")
_write_feature_json(prereq_repo, "specs/001-my-feature")
env = _clean_env()
env["SPECIFY_FEATURE_DIRECTORY"] = "specs/002-other"
py = _run(_py_cmd(prereq_repo, "--json"), prereq_repo, env=env)
assert py.returncode == 0, py.stderr
data = json.loads(
(prereq_repo / ".specify" / "feature.json").read_text(encoding="utf-8")
)
assert data["feature_directory"] == "specs/002-other"
@requires_bash
def test_persisted_feature_json_is_lexical_when_specs_is_symlink(
prereq_repo: Path, tmp_path: Path
) -> None:
"""A symlinked specs/ dir must persist "specs/NNN" like Bash does with its
lexical prefix strip — resolve() would escape the repo and store a
machine-specific absolute path."""
real_specs = tmp_path / "real-specs"
feat = real_specs / "002-other"
feat.mkdir(parents=True)
(feat / "plan.md").write_text("# plan\n", encoding="utf-8")
repo = prereq_repo.resolve()
try:
(repo / "specs").symlink_to(real_specs, target_is_directory=True)
except OSError:
pytest.skip("symlinks not supported on this platform")
env = _clean_env()
env["SPECIFY_FEATURE_DIRECTORY"] = str(repo / "specs" / "002-other")
feature_json = repo / ".specify" / "feature.json"
bash = _run(_bash_cmd(prereq_repo, "--json"), prereq_repo, env=env)
assert bash.returncode == 0, bash.stderr
bash_persisted = json.loads(feature_json.read_text(encoding="utf-8"))
feature_json.unlink()
py = _run(_py_cmd(prereq_repo, "--json"), prereq_repo, env=env)
assert py.returncode == 0, py.stderr
py_persisted = json.loads(feature_json.read_text(encoding="utf-8"))
assert py_persisted == bash_persisted
assert py_persisted["feature_directory"] == "specs/002-other"
@pytest.mark.parametrize(
("args", "expected"),
[
(("--json",), "Feature directory not found"),
(("--json",), "plan.md not found"),
(("--json", "--require-tasks"), "tasks.md not found"),
],
ids=["missing_feature_context", "missing_plan", "missing_tasks"],
)
def test_python_negative_errors_are_stderr_only(
tmp_path: Path, args: tuple[str, ...], expected: str
) -> None:
repo = tmp_path / "proj"
repo.mkdir()
_git_init(repo)
(repo / ".specify").mkdir()
_install_scripts(repo)
if expected in {"plan.md not found", "tasks.md not found"}:
feat = repo / "specs" / "001-my-feature"
feat.mkdir(parents=True)
_write_feature_json(repo)
if expected == "tasks.md not found":
(feat / "plan.md").write_text("# plan\n", encoding="utf-8")
py = _run(_py_cmd(repo, *args), repo)
assert py.returncode != 0
assert expected in py.stderr
assert expected not in py.stdout
assert py.stdout.strip() == ""
def test_python_branch_falls_back_to_feature_dir_basename(prereq_repo: Path) -> None:
(prereq_repo / "specs" / "001-my-feature").mkdir(parents=True)
_write_feature_json(prereq_repo)
py = _run(_py_cmd(prereq_repo, "--json", "--paths-only"), prereq_repo)
assert py.returncode == 0, py.stderr
assert _json_stdout(py)["BRANCH"] == "001-my-feature"