Files
github-spec-kit/tests/test_workflow_run_without_project.py
Pascal THUET 490566847c feat(cli): honor SPECIFY_INIT_DIR in the specify CLI project resolver (#3186)
* feat(cli): honor SPECIFY_INIT_DIR in the specify CLI project resolver

The shell resolver honors SPECIFY_INIT_DIR (#2892), but the Python CLI did
not: it resolved the project as Path.cwd() + a .specify/ check and never read
the override. So setup-plan.sh respected it while `specify integration install`
ignored it, and you still had to cd into the member project.

Route project resolution through a shared _resolve_init_dir_override() that
applies the shell resolver's validation rules (relative to cwd, must exist and
contain .specify/, hard error, no fallback, same error strings). It's wired into
_require_specify_project() — the chokepoint for every project-scoped subcommand
(integration/extension/workflow/preset/...) — and the `workflow run <file>`
standalone path, which re-applies its symlinked-.specify guard on the override
branch too. init is unchanged: it creates .specify/, so the must-pre-exist rule
doesn't apply.

The resolver canonicalizes symlinks via Path.resolve() while the shell keeps the
logical path; they agree for non-symlinked paths (documented in the resolver).

Tests in tests/test_init_dir_cli.py mirror the strict cases from test_init_dir.py
through the CLI; conftest now strips SPECIFY_* for the whole suite so a stray
export can't perturb the now-env-reading resolver. Docs note the CLI applies the
same rules.

Discussion: github/spec-kit#2834

(Disclosure: I used an AI coding agent to audit the call sites and resolver,
draft the change, and run an adversarial code review; reviewed by me.)

* fix(cli): honor SPECIFY_INIT_DIR for bundle commands

Assisted-by: Codex (model: GPT-5, autonomous)

* fix(bundler): refuse symlinked .specify on the SPECIFY_INIT_DIR override path

find_project_root refuses a symlinked .specify (following it could read/write
outside the tree, and a test pins that), but the SPECIFY_INIT_DIR override added
for bundle commands returned early and skipped that guard:
_resolve_init_dir_override validates .specify with is_dir(), which follows
symlinks. So `specify bundle` accepted via the override a layout the cwd path
rejects. Re-check the override result with the same guard, plus a regression test.

(Disclosure: found via an AI code review and fixed with an AI coding agent;
reviewed by me.)

* fix(cli): keep SPECIFY_INIT_DIR strict for bundles

Treat an explicit symlinked SPECIFY_INIT_DIR project as a hard bundle error instead of returning no project, which could initialize the current directory. Align the docs with the actual unset resolver behavior.

Assisted-by: Codex (model: GPT-5, autonomous)

* docs(core): note symlinked .specify handling differs across CLI surfaces

A symlinked .specify is followed by integration/extension/workflow (matching the
shell resolver) but refused by bundle and workflow run <file> (write
confinement). Document the asymmetry so it reads as intentional.

(Disclosure: AI-assisted; reviewed by me.)

* docs(core): reframe symlinked .specify note around the override invariant

Per maintainer feedback on #3186: SPECIFY_INIT_DIR relocates where the project
is, not how a surface treats symlinks. Each surface keeps its cwd-path stance
(write surfaces refuse a symlinked .specify, read/config surfaces follow it),
so the split is one policy relocated, not an inconsistency.

* docs: address Copilot review on resolver docstrings

- _project.py: the error messages "mirror" the shell wording rather than
  "match" it (the CLI renders a Rich `Error:` line, the shell a plain `ERROR:`).
- find_project_root: document that honoring SPECIFY_INIT_DIR when start is None
  can raise typer.Exit / BundlerError, so the Path | None signature isn't
  surprising to direct callers.

* docs(bundler): note require_project_root inherits the override raise behavior

find_project_root can raise typer.Exit / BundlerError under the SPECIFY_INIT_DIR
override (start=None); require_project_root inherits that, so document it
alongside its own BundlerError-on-missing-project.

* docs: clarify symlinked project root behavior

Assisted-by: OpenAI Codex (model: GPT-5, autonomous)

* Address SPECIFY_INIT_DIR review feedback

Assisted-by: OpenAI Codex (model: GPT-5, autonomous)

* Route workflow JSON errors to stderr

Assisted-by: OpenAI Codex (model: GPT-5, autonomous)
2026-07-01 15:55:18 -05:00

325 lines
12 KiB
Python

"""Tests for running workflow YAML files without a project."""
import os
import pytest
import yaml
class TestWorkflowRunWithoutProject:
"""Tests that specify workflow run works with YAML files without .specify/ dir."""
def test_workflow_run_yaml_without_project(self, tmp_path):
"""Running a .yml file should work without a .specify/ directory."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
# Create a minimal workflow YAML with a shell step
workflow_file = tmp_path / "test-workflow.yml"
workflow_content = {
"schema_version": "1.0",
"workflow": {
"id": "standalone-test",
"name": "Standalone Test",
"version": "1.0.0",
"description": "A workflow that runs without a project",
},
"steps": [
{
"id": "create-marker",
"type": "shell",
"run": "echo done > marker.txt",
},
],
}
workflow_file.write_text(yaml.dump(workflow_content), encoding="utf-8")
old_cwd = os.getcwd()
try:
os.chdir(tmp_path)
result = runner.invoke(app, [
"workflow", "run", str(workflow_file),
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"workflow run failed: {result.output}"
assert "completed" in result.output
assert (tmp_path / "marker.txt").exists()
assert (tmp_path / ".specify" / "workflows" / "runs").is_dir()
def test_workflow_run_yaml_with_tilde_and_uppercase_suffix(self, tmp_path, monkeypatch):
"""Running ~/file.YML should work without a .specify/ directory."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
home_dir = tmp_path / "home"
home_dir.mkdir()
monkeypatch.setenv("HOME", str(home_dir))
monkeypatch.setenv("USERPROFILE", str(home_dir))
workflow_file = home_dir / "test-workflow.YML"
workflow_content = {
"schema_version": "1.0",
"workflow": {
"id": "standalone-test-uppercase",
"name": "Standalone Test Uppercase",
"version": "1.0.0",
"description": "A workflow that runs from ~/ with an uppercase suffix",
},
"steps": [
{
"id": "create-marker",
"type": "shell",
"run": "echo done > marker.txt",
},
],
}
workflow_file.write_text(yaml.dump(workflow_content), encoding="utf-8")
old_cwd = os.getcwd()
try:
os.chdir(tmp_path)
result = runner.invoke(app, [
"workflow", "run", "~/test-workflow.YML",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"workflow run failed: {result.output}"
assert "Status: completed" in result.output
assert (tmp_path / "marker.txt").exists()
def test_workflow_run_id_still_requires_project(self, tmp_path):
"""Running a workflow by ID should still require a .specify/ directory."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
old_cwd = os.getcwd()
try:
os.chdir(tmp_path)
result = runner.invoke(app, [
"workflow", "run", "some-workflow-id",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code != 0
assert "Not a Spec Kit project" in result.output
def test_workflow_run_missing_yaml_file(self, tmp_path):
"""Running a non-existent .yml file should still require a project."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
old_cwd = os.getcwd()
try:
os.chdir(tmp_path)
result = runner.invoke(app, [
"workflow", "run", "nonexistent.yml",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
# non-existent .yml files fall through to project check or file-not-found
assert result.exit_code != 0
def test_workflow_run_failing_yaml_without_project(self, tmp_path):
"""A failing workflow YAML should report failure status."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
workflow_file = tmp_path / "fail-workflow.yml"
workflow_content = {
"schema_version": "1.0",
"workflow": {
"id": "fail-test",
"name": "Fail Test",
"version": "1.0.0",
"description": "A workflow that fails",
},
"steps": [
{
"id": "fail-step",
"type": "shell",
"run": "exit 1",
},
],
}
workflow_file.write_text(yaml.dump(workflow_content), encoding="utf-8")
old_cwd = os.getcwd()
try:
os.chdir(tmp_path)
result = runner.invoke(app, [
"workflow", "run", str(workflow_file),
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
# A failed workflow now maps to a non-zero process exit code so
# scripts and CI can rely on $? (the CLI itself still ran fine).
assert result.exit_code == 1, f"expected exit 1 on failed run: {result.output}"
assert "Status: failed" in result.output
def test_workflow_run_yaml_rejects_symlinked_specify_dir(self, tmp_path):
"""Running local YAML should fail when .specify is a symlink."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
workflow_file = tmp_path / "test-workflow.yml"
workflow_content = {
"schema_version": "1.0",
"workflow": {
"id": "symlink-test",
"name": "Symlink Test",
"version": "1.0.0",
"description": "A workflow for symlink guard testing",
},
"steps": [{"id": "noop", "type": "shell", "run": "echo done"}],
}
workflow_file.write_text(yaml.dump(workflow_content), encoding="utf-8")
target_dir = tmp_path / "real-specify-dir"
target_dir.mkdir()
try:
(tmp_path / ".specify").symlink_to(target_dir, target_is_directory=True)
except (OSError, NotImplementedError):
pytest.skip("Symlinks are not available in this environment")
old_cwd = os.getcwd()
try:
os.chdir(tmp_path)
result = runner.invoke(app, [
"workflow", "run", str(workflow_file),
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code != 0
assert "Refusing to use symlinked .specify path" in result.output
def test_workflow_run_yaml_rejects_symlinked_workflows_dir(self, tmp_path):
"""Running local YAML should fail when .specify/workflows is a symlink."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
workflow_file = tmp_path / "test-workflow.yml"
workflow_content = {
"schema_version": "1.0",
"workflow": {
"id": "symlink-workflows-test",
"name": "Symlink Workflows Test",
"version": "1.0.0",
"description": "A workflow for symlink guard testing",
},
"steps": [{"id": "noop", "type": "shell", "run": "echo done"}],
}
workflow_file.write_text(yaml.dump(workflow_content), encoding="utf-8")
(tmp_path / ".specify").mkdir()
target_dir = tmp_path / "real-workflows-dir"
target_dir.mkdir()
try:
(tmp_path / ".specify" / "workflows").symlink_to(
target_dir, target_is_directory=True
)
except (OSError, NotImplementedError):
pytest.skip("Symlinks are not available in this environment")
old_cwd = os.getcwd()
try:
os.chdir(tmp_path)
result = runner.invoke(app, [
"workflow", "run", str(workflow_file),
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code != 0
assert "Refusing to use symlinked .specify/workflows path" in result.output
def test_workflow_run_yaml_rejects_symlinked_runs_dir(self, tmp_path):
"""Running local YAML should fail when .specify/workflows/runs is a symlink."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
workflow_file = tmp_path / "test-workflow.yml"
workflow_content = {
"schema_version": "1.0",
"workflow": {
"id": "symlink-runs-test",
"name": "Symlink Runs Test",
"version": "1.0.0",
"description": "A workflow for symlink guard testing",
},
"steps": [{"id": "noop", "type": "shell", "run": "echo done"}],
}
workflow_file.write_text(yaml.dump(workflow_content), encoding="utf-8")
(tmp_path / ".specify" / "workflows").mkdir(parents=True)
target_dir = tmp_path / "real-runs-dir"
target_dir.mkdir()
try:
(tmp_path / ".specify" / "workflows" / "runs").symlink_to(
target_dir, target_is_directory=True
)
except (OSError, NotImplementedError):
pytest.skip("Symlinks are not available in this environment")
old_cwd = os.getcwd()
try:
os.chdir(tmp_path)
result = runner.invoke(app, [
"workflow", "run", str(workflow_file),
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code != 0
assert "Refusing to use symlinked .specify/workflows/runs path" in result.output
def test_workflow_run_yaml_rejects_non_directory_specify_path(self, tmp_path):
"""Running local YAML should fail when .specify is not a directory."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
workflow_file = tmp_path / "test-workflow.yml"
workflow_content = {
"schema_version": "1.0",
"workflow": {
"id": "nondir-test",
"name": "Non-directory Test",
"version": "1.0.0",
"description": "A workflow for non-directory guard testing",
},
"steps": [{"id": "noop", "type": "shell", "run": "echo done"}],
}
workflow_file.write_text(yaml.dump(workflow_content), encoding="utf-8")
(tmp_path / ".specify").write_text("not a directory", encoding="utf-8")
old_cwd = os.getcwd()
try:
os.chdir(tmp_path)
result = runner.invoke(app, [
"workflow", "run", str(workflow_file),
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code != 0
assert ".specify path exists but is not a directory" in result.output