github/github-spec-kit
1
0
Fork 0
mirror of https://github.com/github/spec-kit.git synced 2026-07-03 12:28:06 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
fc3d1244c07c612e146cfad65d9e36542ee14175
github-spec-kit/tests/integrations/test_cli.py
Manfred Riem fc3d1244c0 fix: replace shell-based context updates with marker-based upsert (#2259) 
* Replace shell-based context updates with marker-based upsert

Replace ~3500 lines of bash/PowerShell agent context update scripts
with a Python-based approach using <!-- SPECKIT START/END --> markers.

IntegrationBase now manages the agent context file directly:
- upsert_context_section(): creates or updates the marked section at
  init/install/switch time with a directive to read the current plan
- remove_context_section(): removes the section at uninstall, deleting
  the file only if it becomes empty
- __CONTEXT_FILE__ placeholder in command templates is resolved per
  integration so the plan command references the correct agent file
- context_file is persisted in init-options.json for extension access

The plan command template instructs the LLM to update the plan
reference between the markers in the agent context file.

Removed:
- scripts/bash/update-agent-context.sh (857 lines)
- scripts/powershell/update-agent-context.ps1 (515 lines)
- 56 integration wrapper scripts (update-context.sh/.ps1)
- templates/agent-file-template.md
- agent_scripts frontmatter key and {AGENT_SCRIPT} replacement logic
- update-context reference from integration.json
- tests/test_cursor_frontmatter.py (tested deleted scripts)

Added:
- upsert/remove context section methods on IntegrationBase
- __CONTEXT_FILE__ placeholder support in process_template()
- context_file field in init-options.json (init/switch/uninstall)
- Per-integration tests: context file correctness, plan reference,
  init-options persistence (78 new context_file tests)
- End-to-end CLI validation across all 28 integrations

* fix: search for end marker after start marker in context section methods

Address Copilot review: content.find(CONTEXT_MARKER_END) searched from
the start of the file rather than after the located start marker. If
the file contained a stray end marker before the start marker, the
wrong slice could be replaced.

Now both upsert_context_section() and remove_context_section() pass
start_idx as the second argument to find() and validate end_idx >
start_idx before performing the replacement.

* fix: address Copilot review feedback on context section handling

1. Fix grammar in _build_context_section() directive text — add commas
   for a complete sentence.

2. Resolve __CONTEXT_FILE__ in resolve_skill_placeholders() — skills
   generated via extensions/presets for codex/kimi now replace the
   placeholder using the context_file value from init-options.json.

3. Handle Cursor .mdc frontmatter — when creating a new .mdc context
   file, prepend alwaysApply: true YAML frontmatter so Cursor
   auto-loads the rules.

4. Fix empty-file leading newline — when the context file exists but
   is empty, write the section directly instead of prepending a blank
   line.

* fix: address second round of Copilot review feedback

1. Ensure .mdc frontmatter on existing files — upsert_context_section()
   now checks for missing YAML frontmatter on .mdc files during updates
   (not just creation), so pre-existing Cursor files get alwaysApply.

2. Guard against context_file=None — use 'or ""' instead of a default
   arg so explicit null values in init-options.json don't cause a
   TypeError in str.replace().

3. Clean up .mdc files on removal — remove_context_section() treats
   files containing only the Speckit-generated frontmatter block as
   empty, deleting them rather than leaving orphaned frontmatter.

* fix: address third round of Copilot review feedback

1. CRLF-safe .mdc frontmatter check — use lstrip().startswith('---')
   instead of startswith('---\n') so CRLF files don't get duplicate
   frontmatter.

2. CRLF-safe .mdc removal check — normalize line endings before
   comparing against the sentinel frontmatter string.

3. Call remove_context_section() during integration_uninstall() — the
   manifest-only uninstall was leaving the managed SPECKIT markers
   behind in the agent context file.

4. Fix stale docstring — remove 'agent_scripts' mention from
   test_lean_commands_have_no_scripts().

* fix: address fourth round of Copilot review feedback

1. Remove unused script_type parameter from _write_integration_json()
   and all 3 call sites — the parameter was no longer referenced after
   the update-context script removal.

2. Fix _build_context_section() docstring — correct example path from
   '.specify/plans/plan.md' to 'specs/<feature>/plan.md'.

3. Improve .mdc frontmatter-only detection in remove_context_section()
   — use regex to match any YAML frontmatter block (not just the exact
   Speckit-generated one), so .mdc files with additional frontmatter
   keys are also cleaned up when no body content remains.

* fix: handle corrupted markers and parse .mdc frontmatter robustly

1. Handle partial/corrupted markers in upsert_context_section() —
   if only the START marker exists (no END), replace from START
   through EOF. If only the END marker exists, replace from BOF
   through END. This keeps upsert idempotent even when a user
   accidentally deletes one marker.

2. Parse .mdc YAML frontmatter properly — new _ensure_mdc_frontmatter()
   helper parses existing frontmatter and ensures alwaysApply: true is
   set, rather than just checking for the --- delimiter. Handles
   missing frontmatter, existing frontmatter without alwaysApply, and
   already-correct frontmatter.

* fix: preserve .mdc frontmatter, add tests, clean up on switch

1. Rewrite _ensure_mdc_frontmatter() with regex — preserves comments,
   formatting, and custom keys in existing frontmatter instead of
   destructively re-serializing via yaml.safe_dump(). Inserts or
   fixes alwaysApply: true in place.

2. Add 6 focused .mdc frontmatter tests to cursor-agent test file:
   new file creation, missing frontmatter, preserved custom keys,
   wrong alwaysApply value, idempotent upserts, removal cleanup.

3. Call remove_context_section() during integration switch Phase 1 —
   prevents stale SPECKIT markers from being left in the old
   integration's context file. Also clear context_file from
   init-options during the metadata reset.

* fix: remove unused MDC_FRONTMATTER, preserve inline comments, normalize bare CR

1. Remove unused MDC_FRONTMATTER class variable — dead code after
   _ensure_mdc_frontmatter() was rewritten with regex.

2. Preserve inline comments when fixing alwaysApply — the regex
   substitution now captures trailing '# comment' text and keeps it.

3. Normalize bare CR in upsert_context_section() — match the
   behavior of remove_context_section() which already normalizes
   both CRLF and bare CR.

4. Clarify .mdc removal comment — 'treat frontmatter-only as empty'
   instead of misleading 'strip frontmatter'.

* fix: handle corrupted markers in remove, CRLF-safe end-marker consumption

1. Handle corrupted markers in remove_context_section() — mirror
   upsert's behavior: start-only removes start→EOF, end-only removes
   BOF→end. Previously bailed out leaving partial markers behind.

2. CRLF-safe end-marker consumption — both upsert and remove now
   handle \r\n after the end marker, not just \n. Prevents extra
   blank lines at replacement boundaries in CRLF files.

3. Clarify path rule in plan template — distinguish filesystem
   operations (absolute paths) from documentation/agent context
   references (project-relative paths).

* fix: only remove context section when both markers are well-ordered

remove_context_section() previously treated mismatched markers as
corruption and aggressively removed from BOF→end-marker or
start-marker→EOF, which could delete user-authored content if only
one marker remained. Now it only removes when both START and END
markers exist and are properly ordered, returning False otherwise.
2026-04-17 13:57:51 -05:00

357 lines
14 KiB
Python
Raw Blame History

"""Tests for --integration flag on specify init (CLI-level)."""
import json
import os
import yaml
from tests.conftest import strip_ansi
def _normalize_cli_output(output: str) -> str:
output = strip_ansi(output)
output = " ".join(output.split())
return output.strip()
class TestInitIntegrationFlag:
def test_integration_and_ai_mutually_exclusive(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
result = runner.invoke(app, [
"init", str(tmp_path / "test-project"), "--ai", "claude", "--integration", "copilot",
])
assert result.exit_code != 0
assert "mutually exclusive" in result.output
def test_unknown_integration_rejected(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
result = runner.invoke(app, [
"init", str(tmp_path / "test-project"), "--integration", "nonexistent",
])
assert result.exit_code != 0
assert "Unknown integration" in result.output
def test_integration_copilot_creates_files(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
project = tmp_path / "int-test"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
result = runner.invoke(app, [
"init", "--here", "--integration", "copilot", "--script", "sh", "--no-git",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"init failed: {result.output}"
assert (project / ".github" / "agents" / "speckit.plan.agent.md").exists()
assert (project / ".github" / "prompts" / "speckit.plan.prompt.md").exists()
assert (project / ".specify" / "scripts" / "bash" / "common.sh").exists()
data = json.loads((project / ".specify" / "integration.json").read_text(encoding="utf-8"))
assert data["integration"] == "copilot"
opts = json.loads((project / ".specify" / "init-options.json").read_text(encoding="utf-8"))
assert opts["integration"] == "copilot"
assert opts["context_file"] == ".github/copilot-instructions.md"
assert (project / ".specify" / "integrations" / "copilot.manifest.json").exists()
# Context section should be upserted into the copilot instructions file
ctx_file = project / ".github" / "copilot-instructions.md"
assert ctx_file.exists()
ctx_content = ctx_file.read_text(encoding="utf-8")
assert "<!-- SPECKIT START -->" in ctx_content
assert "<!-- SPECKIT END -->" in ctx_content
shared_manifest = project / ".specify" / "integrations" / "speckit.manifest.json"
assert shared_manifest.exists()
def test_ai_copilot_auto_promotes(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "promote-test"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(app, [
"init", "--here", "--ai", "copilot", "--script", "sh", "--no-git",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0
assert (project / ".github" / "agents" / "speckit.plan.agent.md").exists()
def test_ai_emits_deprecation_warning_with_integration_replacement(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "warn-ai"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(app, [
"init", "--here", "--ai", "copilot", "--script", "sh", "--no-git",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
normalized_output = _normalize_cli_output(result.output)
assert result.exit_code == 0, result.output
assert "Deprecation Warning" in normalized_output
assert "--ai" in normalized_output
assert "deprecated" in normalized_output
assert "no longer be available" in normalized_output
assert "1.0.0" in normalized_output
assert "--integration copilot" in normalized_output
assert normalized_output.index("Deprecation Warning") < normalized_output.index("Next Steps")
assert (project / ".github" / "agents" / "speckit.plan.agent.md").exists()
def test_ai_generic_warning_suggests_integration_options_equivalent(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "warn-generic"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(app, [
"init", "--here", "--ai", "generic", "--ai-commands-dir", ".myagent/commands",
"--script", "sh", "--no-git",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
normalized_output = _normalize_cli_output(result.output)
assert result.exit_code == 0, result.output
assert "Deprecation Warning" in normalized_output
assert "--integration generic" in normalized_output
assert "--integration-options" in normalized_output
assert ".myagent/commands" in normalized_output
assert normalized_output.index("Deprecation Warning") < normalized_output.index("Next Steps")
assert (project / ".myagent" / "commands" / "speckit.plan.md").exists()
def test_ai_claude_here_preserves_preexisting_commands(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "claude-here-existing"
project.mkdir()
commands_dir = project / ".claude" / "skills"
commands_dir.mkdir(parents=True)
skill_dir = commands_dir / "speckit-specify"
skill_dir.mkdir(parents=True)
command_file = skill_dir / "SKILL.md"
command_file.write_text("# preexisting command\n", encoding="utf-8")
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(app, [
"init", "--here", "--force", "--ai", "claude", "--ai-skills", "--script", "sh", "--no-git", "--ignore-agent-tools",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, result.output
assert command_file.exists()
# init replaces skills (not additive); verify the file has valid skill content
assert command_file.exists()
assert "speckit-specify" in command_file.read_text(encoding="utf-8")
assert (project / ".claude" / "skills" / "speckit-plan" / "SKILL.md").exists()
def test_shared_infra_skips_existing_files(self, tmp_path):
"""Pre-existing shared files are not overwritten by _install_shared_infra."""
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "skip-test"
project.mkdir()
# Pre-create a shared script with custom content
scripts_dir = project / ".specify" / "scripts" / "bash"
scripts_dir.mkdir(parents=True)
custom_content = "# user-modified common.sh\n"
(scripts_dir / "common.sh").write_text(custom_content, encoding="utf-8")
# Pre-create a shared template with custom content
templates_dir = project / ".specify" / "templates"
templates_dir.mkdir(parents=True)
custom_template = "# user-modified spec-template\n"
(templates_dir / "spec-template.md").write_text(custom_template, encoding="utf-8")
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(app, [
"init", "--here", "--force",
"--integration", "copilot",
"--script", "sh",
"--no-git",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0
# User's files should be preserved
assert (scripts_dir / "common.sh").read_text(encoding="utf-8") == custom_content
assert (templates_dir / "spec-template.md").read_text(encoding="utf-8") == custom_template
# Other shared files should still be installed
assert (scripts_dir / "setup-plan.sh").exists()
assert (templates_dir / "plan-template.md").exists()
class TestForceExistingDirectory:
"""Tests for --force merging into an existing named directory."""
def test_force_merges_into_existing_dir(self, tmp_path):
"""specify init <dir> --force succeeds when the directory already exists."""
from typer.testing import CliRunner
from specify_cli import app
target = tmp_path / "existing-proj"
target.mkdir()
# Place a pre-existing file to verify it survives the merge
marker = target / "user-file.txt"
marker.write_text("keep me", encoding="utf-8")
runner = CliRunner()
result = runner.invoke(app, [
"init", str(target), "--integration", "copilot", "--force",
"--no-git", "--script", "sh",
], catch_exceptions=False)
assert result.exit_code == 0, f"init --force failed: {result.output}"
# Pre-existing file should survive
assert marker.read_text(encoding="utf-8") == "keep me"
# Spec Kit files should be installed
assert (target / ".specify" / "init-options.json").exists()
assert (target / ".specify" / "templates" / "spec-template.md").exists()
def test_without_force_errors_on_existing_dir(self, tmp_path):
"""specify init <dir> without --force errors when directory exists."""
from typer.testing import CliRunner
from specify_cli import app
target = tmp_path / "existing-proj"
target.mkdir()
runner = CliRunner()
result = runner.invoke(app, [
"init", str(target), "--integration", "copilot",
"--no-git", "--script", "sh",
], catch_exceptions=False)
assert result.exit_code == 1
assert "already exists" in result.output
class TestGitExtensionAutoInstall:
"""Tests for auto-installation of the git extension during specify init."""
def test_git_extension_auto_installed(self, tmp_path):
"""Without --no-git, the git extension is installed during init."""
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "git-auto"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(app, [
"init", "--here", "--ai", "claude", "--script", "sh",
"--ignore-agent-tools",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"init failed: {result.output}"
# Check that the tracker didn't report a git error
assert "install failed" not in result.output, f"git extension install failed: {result.output}"
# Git extension files should be installed
ext_dir = project / ".specify" / "extensions" / "git"
assert ext_dir.exists(), "git extension directory not installed"
assert (ext_dir / "extension.yml").exists()
assert (ext_dir / "scripts" / "bash" / "create-new-feature.sh").exists()
assert (ext_dir / "scripts" / "bash" / "initialize-repo.sh").exists()
# Hooks should be registered
extensions_yml = project / ".specify" / "extensions.yml"
assert extensions_yml.exists(), "extensions.yml not created"
hooks_data = yaml.safe_load(extensions_yml.read_text(encoding="utf-8"))
assert "hooks" in hooks_data
assert "before_specify" in hooks_data["hooks"]
assert "before_constitution" in hooks_data["hooks"]
def test_no_git_skips_extension(self, tmp_path):
"""With --no-git, the git extension is NOT installed."""
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "no-git"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(app, [
"init", "--here", "--ai", "claude", "--script", "sh",
"--no-git", "--ignore-agent-tools",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"init failed: {result.output}"
# Git extension should NOT be installed
ext_dir = project / ".specify" / "extensions" / "git"
assert not ext_dir.exists(), "git extension should not be installed with --no-git"
def test_git_extension_commands_registered(self, tmp_path):
"""Git extension commands are registered with the agent during init."""
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "git-cmds"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(app, [
"init", "--here", "--ai", "claude", "--script", "sh",
"--ignore-agent-tools",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"init failed: {result.output}"
# Git extension commands should be registered with the agent
claude_skills = project / ".claude" / "skills"
assert claude_skills.exists(), "Claude skills directory was not created"
git_skills = [f for f in claude_skills.iterdir() if f.name.startswith("speckit-git-")]
assert len(git_skills) > 0, "no git extension commands registered"
Reference in New Issue View Git Blame Copy Permalink