Files
github-spec-kit/tests/integrations/test_integration_claude.py
Manfred Riem 53d9543355 feat: make agent-context extension a full opt-in (#3097)
* docs: add Spec Kit spec for agent-context full opt-in

Use Spec Kit's own specify workflow to author the spec that makes the
agent-context extension a full opt-in, removing all agent-context
configuration/support from the Python codebase and removing the
deprecation message. Force-added despite specs/ being gitignored; the
generated artifact will be purged prior to merge.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: add Spec Kit plan artifacts for agent-context full opt-in

Phase 0/1 of the SDD plan workflow: plan.md, research.md, data-model.md,
quickstart.md, and contracts/cli-behavior.md. Constitution Check is a
documented no-op (repo has no ratified constitution). Force-added despite
specs/ being gitignored; generated artifacts will be purged prior to merge.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: correct Constitution Check against ratified v1.0.0

Earlier draft wrongly treated the gate as a no-op; the fork's main is 16
commits behind upstream/main, which carries .specify/memory/constitution.md.
Re-evaluate the feature against Principles I-V (all PASS) and note that
Principle I mandates keeping context_file as a declared class attribute,
validating the R1 metadata decision.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: refresh plan artifacts against synced upstream/main

After syncing fork main to upstream and rebasing, re-scan the current
agent-context surface. Upstream generalized the single context_file into a
plural context_files concept with new resolver helpers
(_resolve_context_files, _resolve_context_file_values,
_format_context_file_values) and upsert/remove now loop over multiple
files. Update research.md, data-model.md, contracts, quickstart grep
guards, and the plan summary to cover the expanded removal scope.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: add Spec Kit tasks for agent-context full opt-in

Phase 2 of SDD: dependency-ordered tasks.md (30 tasks) organized by the
three user stories, with mandatory test tasks (Constitution Principle II)
and a foundational phase decoupling __CONTEXT_FILE__ resolution from the
extension config. Includes the extension self-seeding task (T015) and a
static guard test (T002) enforcing zero agent-context references in the CLI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat!: remove agent-context lifecycle from the Specify CLI

Make the agent-context extension a full opt-in. The CLI no longer
installs the extension during init, writes agent-context-config.yml,
or creates/updates/removes the managed Spec Kit section in agent
context files. Context-section upsert/remove, marker resolution,
extension-enabled gating, the config helpers, and the obsolete inline
deprecation warning are all removed. Integration context_file stays as
inert metadata; __CONTEXT_FILE__ now resolves from registry metadata.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(agent-context): self-seed context file from the active integration

When agent-context-config.yml has no context_file/context_files, the
bundled bash and PowerShell update scripts now resolve the context file
from the active integration in .specify/init-options.json via the
integration registry, so the extension no longer depends on the CLI
writing its config.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test+docs: update suite and docs for agent-context opt-in

Update integration/extension tests to expect no agent-context install,
config, or context-section writes during init. Add a static guard test
(test_agent_context_cli_free.py) asserting the CLI source is free of
agent-context lifecycle symbols, plus backward-compatibility tests for
legacy projects. Refresh AGENTS.md, the extension README, and add a
CHANGELOG entry describing the opt-in behavior change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(agent-context): warn on self-seed failure, correct docs, speed up guard test

Address PR review feedback:
- Self-seed scripts (bash + PowerShell) now emit an actionable warning when
  an active integration is configured but specify_cli cannot be imported by
  the chosen Python (e.g. pipx installs), or when the integration declares no
  context file, instead of silently falling through to 'nothing to do'.
- Correct the extension README disable note: command rendering never reads the
  extension config; __CONTEXT_FILE__ is always substituted from integration
  metadata, so a stale context_files value cannot affect rendering.
- Cache CLI source reads in the static guard test via a module-scoped fixture
  so the directory walk happens once instead of once per forbidden symbol.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(agent-context): ship self-owned per-agent context-file defaults

The extension now bundles agent-context-defaults.json (key→context_file
map) and self-seeds from it, dropping any dependency on the Specify CLI
registry. Both the bash and PowerShell update scripts read the bundled
JSON map keyed by the active integration from init-options.json.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat!: remove all agent-context state from the Specify CLI

Strip every context_file reference from the CLI: the field on all 35
integration classes, the IntegrationBase plumbing (process_template
param/step, _context_file_display, docstrings), the __CONTEXT_FILE__
resolution in agents.py, the legacy context_file/context_markers
popping in _helpers.py, and the context_file template in
integration_scaffold.py. Also drop the Agent context update step and
__CONTEXT_FILE__ placeholder from templates/commands/plan.md.

The agent-context extension now solely owns all context-file knowledge,
including the per-agent default mapping.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: drop context_file coverage and guard against CLI reintroduction

Remove CONTEXT_FILE attrs and context_file assertions across the base
mixins, all 35 per-integration test files, shared integration tests, and
conftest stubs. Rewrite the base-mixin context tests to assert no managed
section is written and no __CONTEXT_FILE__ placeholder survives. Extend
the CLI-free static guard to forbid context_file, __CONTEXT_FILE__, and
_context_file_display in src/specify_cli, and have the extension tests
copy the bundled defaults JSON so self-seed runs without the CLI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: reflect full removal of agent-context state from the CLI

Update AGENTS.md (integration examples, required-fields table, context
behavior section, pitfalls), CHANGELOG, and the SDD spec artifacts
(FR-007, SC-002, data-model) to state that the CLI carries no
context_file and the extension fully owns the per-agent default mapping
via agent-context-defaults.json.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: align SDD artifacts with full context_file removal

Update research.md (R1, R2, R4, summary table), contracts/cli-behavior.md
(C3, C5), tasks.md (Phase 2, T026, notes), plan.md (Principle I, source
map), and checklists/requirements.md so the spec artifacts reflect the
implemented decision: the CLI carries no context_file attribute or
__CONTEXT_FILE__ resolution, and the per-agent defaults map lives in the
extension. Resolves PR review #4548130110.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: scrub stale context-file mentions from CLI docstrings

Update the multi_install_safe docstring (drop the removed "context file"
invariant), the RovoDev setup docstring (no longer upserts a context
section), the Copilot module docstring (drop the context-file line), and
tighten the _update_init_options_for_integration note. Pure docstring
changes — no behavioral impact. Resolves PR review #4548237085.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test+docs: harden agent-context test helper and fix stale docs

- base.py: document multi_install_safe as an optional subclass attribute
  in the IntegrationBase docstring.
- test_cli.py: clarify the init-options assertion is guarding against
  leftover legacy agent-context keys, not relocation.
- test_extension_agent_context.py: _install_agent_context_config now
  asserts the bundled agent-context-defaults.json exists and always
  copies it, so self-seeding tests fail loudly instead of silently
  skipping when the map is missing.
- test_integration_cursor_agent.py: drop Path/IntegrationManifest imports
  left unused after removing the context-section frontmatter tests.

Resolves PR review #4548293116.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: remove gitignored SDD artifacts from specs/

The specs/001-agent-context-full-optin/ artifacts were force-added for
dogfooding visibility, but specs/ is gitignored and these were always
intended to be purged before merge. Remove them so merging does not add
an intentionally-untracked directory to repo history.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: keep CHANGELOG.md identical to upstream

CHANGELOG.md is auto-generated at release time, so the branch should not
carry a manual entry. Restore it to match upstream/main exactly.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: preserve Cursor .mdc frontmatter in agent-context updater scripts

The bundled agent-context updater scripts wrote the managed section as
plain text. For Cursor-style `.mdc` targets this dropped the required
`---\nalwaysApply: true\n---` frontmatter, reintroducing the rule-loading
bug originally fixed in #1699. Port the `_ensure_mdc_frontmatter` logic
into both the bash and PowerShell updaters: prepend frontmatter when
missing, repair `alwaysApply` when set to the wrong value, and leave
non-`.mdc` targets untouched. Add regression tests covering both shells.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: scope CLI-free guard to agent-context-specific symbols

Drop the bare "context_file" substring from FORBIDDEN_SYMBOLS so the
guard no longer fails on unrelated future CLI fields named context_file.
The list still covers agent-context-specific identifiers (__CONTEXT_FILE__,
_context_file_display, _resolve_context_files, _resolve_context_file_values).

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: harden agent-context bash self-seed against malformed init JSON

Two robustness fixes in the embedded Python self-seed logic:
- Coerce the integration value from init-options.json to a string only when
  it is actually a string; otherwise treat it as unset so a corrupted
  dict/list value degrades to the existing nothing-to-do behavior instead of
  breaking the agents-map lookup.
- Normalize agent-context-defaults.json: only use 'agents' when both the JSON
  root and the 'agents' value are dicts, so a wrong-shaped (but valid) JSON
  falls back to the warning path instead of raising on .get.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: correct PowerShell hyphenated key lookup and regex replace count

- Self-seed now reads the defaults mapping via
  $defaults.agents.PSObject.Properties[$integrationKey].Value instead of
  member access ($defaults.agents.$integrationKey), which parsed hyphenated
  keys like 'cursor-agent'/'kiro-cli' as subtraction and failed to resolve.
- Replace the static [regex]::Replace(..., 1) call, whose trailing 1 was
  interpreted as RegexOptions.IgnoreCase rather than a replacement count, with
  an instance Regex whose Replace(input, replacement, 1) limits to the first
  match as intended.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make bash .mdc frontmatter guard case-insensitive

The bash updater only injected Cursor .mdc frontmatter when ctx_path ended
in lowercase '.mdc', so a mixed/upper-case extension (e.g. specify-rules.MDC)
was skipped and Cursor would not auto-load the rule file. Compare against the
casefolded path. The PowerShell variant already uses -match, which is
case-insensitive by default, so no change is needed there.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: document separator-agnostic agent-context update invocation

The README hard-coded the dot-notation slash command
(/speckit.agent-context.update), which hyphen-separator agents like Forge and
Cline do not recognize. Document the canonical command ID plus both slash
invocations so users copy the form their agent accepts.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-29 15:27:26 -05:00

870 lines
36 KiB
Python

"""Tests for ClaudeIntegration."""
import json
import os
from pathlib import Path
from unittest.mock import patch
import yaml
from specify_cli.integrations import INTEGRATION_REGISTRY, get_integration
from specify_cli.integrations.base import IntegrationBase, SkillsIntegration
from specify_cli.integrations.claude import ARGUMENT_HINTS, FORK_CONTEXT_COMMANDS
from specify_cli.integrations.manifest import IntegrationManifest
class TestClaudeIntegration:
def test_registered(self):
assert "claude" in INTEGRATION_REGISTRY
assert get_integration("claude") is not None
def test_is_base_integration(self):
assert isinstance(get_integration("claude"), IntegrationBase)
def test_config_uses_skills(self):
integration = get_integration("claude")
assert integration.config["folder"] == ".claude/"
assert integration.config["commands_subdir"] == "skills"
def test_registrar_config_uses_skill_layout(self):
integration = get_integration("claude")
assert integration.registrar_config["dir"] == ".claude/skills"
assert integration.registrar_config["format"] == "markdown"
assert integration.registrar_config["args"] == "$ARGUMENTS"
assert integration.registrar_config["extension"] == "/SKILL.md"
def test_setup_creates_skill_files(self, tmp_path):
integration = get_integration("claude")
manifest = IntegrationManifest("claude", tmp_path)
created = integration.setup(tmp_path, manifest, script_type="sh")
skill_files = [path for path in created if path.name == "SKILL.md"]
assert skill_files
skills_dir = tmp_path / ".claude" / "skills"
assert skills_dir.is_dir()
plan_skill = skills_dir / "speckit-plan" / "SKILL.md"
assert plan_skill.exists()
content = plan_skill.read_text(encoding="utf-8")
assert "{SCRIPT}" not in content
assert "{ARGS}" not in content
assert "__AGENT__" not in content
assert "__SPECKIT_COMMAND_" not in content, "unprocessed __SPECKIT_COMMAND_*__"
assert "/speckit." not in content, "skills agent must use /speckit-<name> not /speckit.<name>"
parts = content.split("---", 2)
parsed = yaml.safe_load(parts[1])
assert parsed["name"] == "speckit-plan"
assert parsed["user-invocable"] is True
assert parsed["disable-model-invocation"] is False
assert parsed["metadata"]["source"] == "templates/commands/plan.md"
def test_render_skill_unicode(self):
"""Test rendering a skill preserves non-ASCII characters."""
integration = get_integration("claude")
rendered = integration._render_skill(
"constitution",
{"description": "Prüfe Konformität der Implementierung"},
"Body",
)
assert "Prüfe Konformität" in rendered
def test_setup_does_not_write_context_section(self, tmp_path):
"""The CLI no longer manages the agent context file — that is owned by
the opt-in agent-context extension. Setup must not create or touch it."""
integration = get_integration("claude")
manifest = IntegrationManifest("claude", tmp_path)
integration.setup(tmp_path, manifest, script_type="sh")
for path in tmp_path.rglob("*"):
if path.is_file():
text = path.read_text(encoding="utf-8", errors="ignore")
assert "<!-- SPECKIT START -->" not in text
def test_teardown_does_not_touch_existing_context_file(self, tmp_path):
"""A user-authored context file is left intact on teardown."""
integration = get_integration("claude")
ctx_path = tmp_path / "CLAUDE.md"
original = "# CLAUDE.md\n\nUser content.\n"
ctx_path.write_text(original, encoding="utf-8")
manifest = IntegrationManifest("claude", tmp_path)
integration.setup(tmp_path, manifest, script_type="sh")
integration.teardown(tmp_path, manifest)
assert ctx_path.read_text(encoding="utf-8") == original
def test_integration_flag_creates_skill_files_cli(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "claude-promote"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(
app,
[
"init",
"--here",
"--integration",
"claude",
"--script",
"sh",
"--ignore-agent-tools",
],
catch_exceptions=False,
)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, result.output
assert (project / ".claude" / "skills" / "speckit-plan" / "SKILL.md").exists()
assert not (project / ".claude" / "commands").exists()
init_options = json.loads(
(project / ".specify" / "init-options.json").read_text(encoding="utf-8")
)
assert init_options["ai"] == "claude"
assert init_options["ai_skills"] is True
assert init_options["integration"] == "claude"
def test_integration_flag_creates_skill_files(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "claude-integration"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(
app,
[
"init",
"--here",
"--integration",
"claude",
"--script",
"sh",
"--ignore-agent-tools",
],
catch_exceptions=False,
)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, result.output
assert (project / ".claude" / "skills" / "speckit-specify" / "SKILL.md").exists()
assert (project / ".specify" / "integrations" / "claude.manifest.json").exists()
def test_interactive_claude_selection_uses_integration_path(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "claude-interactive"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
with (
patch("specify_cli.commands.init._stdin_is_interactive", return_value=True),
patch("specify_cli.commands.init.select_with_arrows", return_value="claude"),
):
result = runner.invoke(
app,
[
"init",
"--here",
"--script",
"sh",
"--ignore-agent-tools",
],
catch_exceptions=False,
)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, result.output
assert (project / ".specify" / "integration.json").exists()
assert (project / ".specify" / "integrations" / "claude.manifest.json").exists()
skill_file = project / ".claude" / "skills" / "speckit-plan" / "SKILL.md"
assert skill_file.exists()
skill_content = skill_file.read_text(encoding="utf-8")
assert "user-invocable: true" in skill_content
assert "disable-model-invocation: false" in skill_content
init_options = json.loads(
(project / ".specify" / "init-options.json").read_text(encoding="utf-8")
)
assert init_options["ai"] == "claude"
assert init_options["ai_skills"] is True
assert init_options["integration"] == "claude"
def test_claude_init_remains_usable_when_converter_fails(self, tmp_path):
"""Claude init should succeed even without install_skills."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
target = tmp_path / "fail-proj"
result = runner.invoke(
app,
["init", str(target), "--integration", "claude", "--script", "sh", "--ignore-agent-tools"],
)
assert result.exit_code == 0
assert (target / ".claude" / "skills" / "speckit-specify" / "SKILL.md").exists()
def test_claude_hooks_render_skill_invocation(self, tmp_path):
from specify_cli.extensions import HookExecutor
project = tmp_path / "claude-hooks"
project.mkdir()
init_options = project / ".specify" / "init-options.json"
init_options.parent.mkdir(parents=True, exist_ok=True)
init_options.write_text(json.dumps({"ai": "claude", "ai_skills": True}))
hook_executor = HookExecutor(project)
message = hook_executor.format_hook_message(
"before_plan",
[
{
"extension": "test-ext",
"command": "speckit.plan",
"optional": False,
}
],
)
assert "Executing: `/speckit-plan`" in message
assert "EXECUTE_COMMAND: speckit.plan" in message
assert "EXECUTE_COMMAND_INVOCATION: /speckit-plan" in message
def test_claude_preset_creates_new_skill_without_commands_dir(self, tmp_path):
from specify_cli import save_init_options
from specify_cli.presets import PresetManager
project = tmp_path / "claude-preset-skill"
project.mkdir()
save_init_options(project, {"ai": "claude", "ai_skills": True, "script": "sh"})
skills_dir = project / ".claude" / "skills"
skills_dir.mkdir(parents=True, exist_ok=True)
preset_dir = tmp_path / "claude-skill-command"
preset_dir.mkdir()
(preset_dir / "commands").mkdir()
(preset_dir / "commands" / "speckit.research.md").write_text(
"---\n"
"description: Research workflow\n"
"---\n\n"
"preset:claude-skill-command\n"
)
manifest_data = {
"schema_version": "1.0",
"preset": {
"id": "claude-skill-command",
"name": "Claude Skill Command",
"version": "1.0.0",
"description": "Test",
},
"requires": {"speckit_version": ">=0.1.0"},
"provides": {
"templates": [
{
"type": "command",
"name": "speckit.research",
"file": "commands/speckit.research.md",
}
]
},
}
with open(preset_dir / "preset.yml", "w") as f:
yaml.dump(manifest_data, f)
manager = PresetManager(project)
manager.install_from_directory(preset_dir, "0.1.5")
skill_file = skills_dir / "speckit-research" / "SKILL.md"
assert skill_file.exists()
content = skill_file.read_text(encoding="utf-8")
assert "preset:claude-skill-command" in content
assert "name: speckit-research" in content
assert "user-invocable: true" in content
assert "disable-model-invocation: false" in content
metadata = manager.registry.get("claude-skill-command")
assert "speckit-research" in metadata.get("registered_skills", [])
class TestClaudeArgumentHints:
"""Verify that argument-hint frontmatter is injected for Claude skills."""
def test_converge_has_no_argument_hint(self):
"""Converge should not advertise unsupported feature-name arguments."""
assert "converge" not in ARGUMENT_HINTS
def test_all_skills_have_hints(self, tmp_path):
"""Every skill with a configured hint must contain an argument-hint line."""
i = get_integration("claude")
m = IntegrationManifest("claude", tmp_path)
created = i.setup(tmp_path, m, script_type="sh")
skill_files = [f for f in created if f.name == "SKILL.md"]
assert len(skill_files) > 0
for f in skill_files:
stem = f.parent.name
if stem.startswith("speckit-"):
stem = stem[len("speckit-"):]
content = f.read_text(encoding="utf-8")
if stem in ARGUMENT_HINTS:
assert "argument-hint:" in content, (
f"{f.parent.name}/SKILL.md is missing argument-hint frontmatter"
)
else:
assert "argument-hint:" not in content, (
f"{f.parent.name}/SKILL.md unexpectedly has argument-hint frontmatter"
)
def test_hints_match_expected_values(self, tmp_path):
"""Each skill's argument-hint must match the expected text."""
i = get_integration("claude")
m = IntegrationManifest("claude", tmp_path)
created = i.setup(tmp_path, m, script_type="sh")
skill_files = [f for f in created if f.name == "SKILL.md"]
for f in skill_files:
# Extract stem: speckit-plan -> plan
stem = f.parent.name
if stem.startswith("speckit-"):
stem = stem[len("speckit-"):]
expected_hint = ARGUMENT_HINTS.get(stem)
content = f.read_text(encoding="utf-8")
if expected_hint is None:
assert "argument-hint:" not in content, (
f"{f.parent.name}/SKILL.md unexpectedly has argument-hint frontmatter"
)
else:
assert f'argument-hint: "{expected_hint}"' in content, (
f"{f.parent.name}/SKILL.md: expected hint '{expected_hint}' not found"
)
def test_hint_is_inside_frontmatter(self, tmp_path):
"""argument-hint must appear between the --- delimiters, not in the body."""
i = get_integration("claude")
m = IntegrationManifest("claude", tmp_path)
created = i.setup(tmp_path, m, script_type="sh")
skill_files = [f for f in created if f.name == "SKILL.md"]
for f in skill_files:
content = f.read_text(encoding="utf-8")
parts = content.split("---", 2)
assert len(parts) >= 3, f"No frontmatter in {f.parent.name}/SKILL.md"
frontmatter = parts[1]
body = parts[2]
stem = f.parent.name
if stem.startswith("speckit-"):
stem = stem[len("speckit-"):]
if stem in ARGUMENT_HINTS:
assert "argument-hint:" in frontmatter, (
f"{f.parent.name}/SKILL.md: argument-hint not in frontmatter section"
)
assert "argument-hint:" not in body, (
f"{f.parent.name}/SKILL.md: argument-hint leaked into body"
)
else:
assert "argument-hint:" not in content, (
f"{f.parent.name}/SKILL.md unexpectedly has argument-hint frontmatter"
)
def test_hint_appears_after_description(self, tmp_path):
"""argument-hint must immediately follow the description line."""
i = get_integration("claude")
m = IntegrationManifest("claude", tmp_path)
created = i.setup(tmp_path, m, script_type="sh")
skill_files = [f for f in created if f.name == "SKILL.md"]
for f in skill_files:
content = f.read_text(encoding="utf-8")
lines = content.splitlines()
stem = f.parent.name
if stem.startswith("speckit-"):
stem = stem[len("speckit-"):]
if stem not in ARGUMENT_HINTS:
assert "argument-hint:" not in content, (
f"{f.parent.name}/SKILL.md unexpectedly has argument-hint frontmatter"
)
continue
found_description = False
for idx, line in enumerate(lines):
if line.startswith("description:"):
found_description = True
assert idx + 1 < len(lines), (
f"{f.parent.name}/SKILL.md: description is last line"
)
assert lines[idx + 1].startswith("argument-hint:"), (
f"{f.parent.name}/SKILL.md: argument-hint does not follow description"
)
break
assert found_description, (
f"{f.parent.name}/SKILL.md: no description: line found in output"
)
def test_inject_argument_hint_only_in_frontmatter(self):
"""inject_argument_hint must not modify description: lines in the body."""
from specify_cli.integrations.claude import ClaudeIntegration
content = (
"---\n"
"description: My command\n"
"---\n"
"\n"
"description: this is body text\n"
)
result = ClaudeIntegration.inject_argument_hint(content, "Test hint")
lines = result.splitlines()
hint_count = sum(1 for ln in lines if ln.startswith("argument-hint:"))
assert hint_count == 1, (
f"Expected exactly 1 argument-hint line, found {hint_count}"
)
def test_inject_argument_hint_skips_if_already_present(self):
"""inject_argument_hint must not duplicate if argument-hint already exists."""
from specify_cli.integrations.claude import ClaudeIntegration
content = (
"---\n"
"description: My command\n"
'argument-hint: "Existing hint"\n'
"---\n"
"\n"
"Body text\n"
)
result = ClaudeIntegration.inject_argument_hint(content, "New hint")
assert result == content, "Content should be unchanged when hint already exists"
lines = result.splitlines()
hint_count = sum(1 for ln in lines if ln.startswith("argument-hint:"))
assert hint_count == 1
class TestClaudeDisableModelInvocation:
"""Verify disable-model-invocation is false for Claude skills."""
def test_setup_sets_disable_model_invocation_false(self, tmp_path):
"""Generated SKILL.md files must have disable-model-invocation: false."""
i = get_integration("claude")
m = IntegrationManifest("claude", tmp_path)
created = i.setup(tmp_path, m, script_type="sh")
skill_files = [f for f in created if f.name == "SKILL.md"]
assert len(skill_files) > 0
for f in skill_files:
content = f.read_text(encoding="utf-8")
parts = content.split("---", 2)
parsed = yaml.safe_load(parts[1])
assert parsed["disable-model-invocation"] is False, (
f"{f.parent.name}: expected disable-model-invocation: false"
)
def test_disable_model_invocation_not_true(self, tmp_path):
"""No Claude skill should have disable-model-invocation: true."""
i = get_integration("claude")
m = IntegrationManifest("claude", tmp_path)
created = i.setup(tmp_path, m, script_type="sh")
for f in created:
if f.name != "SKILL.md":
continue
content = f.read_text(encoding="utf-8")
assert "disable-model-invocation: true" not in content, (
f"{f.parent.name}: must not have disable-model-invocation: true"
)
def test_non_claude_agents_lack_disable_model_invocation(self, tmp_path):
"""Non-Claude skill agents should not get disable-model-invocation."""
from specify_cli.agents import CommandRegistrar
fm = CommandRegistrar.build_skill_frontmatter(
"codex", "speckit-plan", "desc", "templates/commands/plan.md"
)
assert "disable-model-invocation" not in fm
assert "user-invocable" not in fm
def test_skills_default_post_process_preserves_content_without_hooks(self, tmp_path):
"""SkillsIntegration agents without an override preserve non-hook content."""
# ``agy`` is a plain SkillsIntegration with no post-process override,
# so it stands in for the base-class default behavior.
agy = get_integration("agy")
if agy is None:
return # agy not registered in this build
content = "---\nname: test\n---\nBody"
assert agy.post_process_skill_content(content) == content
class TestClaudeForkContext:
"""Verify context: fork is injected only for commands listed in FORK_CONTEXT_COMMANDS."""
def test_no_commands_fork_by_default(self):
"""FORK_CONTEXT_COMMANDS is empty: no command opts into context: fork.
``analyze`` was removed (#3185) because its verbose report defeated the
purpose of forking and compounded context overhead across repeated runs.
"""
assert FORK_CONTEXT_COMMANDS == {}
def test_analyze_skill_does_not_fork(self, tmp_path):
"""speckit-analyze must run in the main session, not a forked subagent (#3185)."""
i = get_integration("claude")
m = IntegrationManifest("claude", tmp_path)
i.setup(tmp_path, m, script_type="sh")
analyze_skill = tmp_path / ".claude/skills/speckit-analyze/SKILL.md"
assert analyze_skill.exists()
content = analyze_skill.read_text(encoding="utf-8")
parts = content.split("---", 2)
parsed = yaml.safe_load(parts[1])
assert "context" not in parsed
assert "agent" not in parsed
def test_no_skills_fork(self, tmp_path):
"""Skills not in FORK_CONTEXT_COMMANDS must not get context: fork."""
i = get_integration("claude")
m = IntegrationManifest("claude", tmp_path)
created = i.setup(tmp_path, m, script_type="sh")
skill_files = [f for f in created if f.name == "SKILL.md"]
for f in skill_files:
stem = f.parent.name
if stem.startswith("speckit-"):
stem = stem[len("speckit-"):]
if stem in FORK_CONTEXT_COMMANDS:
continue
content = f.read_text(encoding="utf-8")
parts = content.split("---", 2)
parsed = yaml.safe_load(parts[1])
assert "context" not in parsed, (
f"{f.parent.name}: must not have context frontmatter"
)
assert "agent" not in parsed, (
f"{f.parent.name}: must not have agent frontmatter"
)
def test_post_process_no_fork_for_skills(self):
"""With FORK_CONTEXT_COMMANDS empty, post_process must not add context/agent."""
i = get_integration("claude")
for name in ("speckit-analyze", "speckit-plan"):
content = f'---\nname: "{name}"\ndescription: "x"\n---\n\nBody\n'
result = i.post_process_skill_content(content)
parsed = yaml.safe_load(result.split("---", 2)[1])
assert "context" not in parsed
assert "agent" not in parsed
def test_fork_mechanism_injects_when_configured(self, monkeypatch):
"""The injection mechanism still works for any command added to
FORK_CONTEXT_COMMANDS, even though none ships enabled by default."""
import specify_cli.integrations.claude as claude_mod
monkeypatch.setitem(
claude_mod.FORK_CONTEXT_COMMANDS,
"analyze",
{"context": "fork", "agent": "general-purpose"},
)
i = get_integration("claude")
content = '---\nname: "speckit-analyze"\ndescription: "x"\n---\n\nBody\n'
result = i.post_process_skill_content(content)
parts = result.split("---", 2)
parsed = yaml.safe_load(parts[1])
assert parsed.get("context") == "fork"
assert parsed.get("agent") == "general-purpose"
# Flags must land in the frontmatter, not the body.
assert "context: fork" in parts[1]
assert "context: fork" not in parts[2]
# Re-running must not duplicate the injected keys.
twice = i.post_process_skill_content(result)
assert result == twice
assert twice.count("context: fork") == 1
assert twice.count("agent: general-purpose") == 1
class TestClaudeHookCommandNote:
"""Verify dot-to-hyphen normalization note is injected in hook sections."""
def test_hook_note_injected_in_skills_with_hooks(self, tmp_path):
"""Skills that have hook sections should get the normalization note."""
i = get_integration("claude")
m = IntegrationManifest("claude", tmp_path)
i.setup(tmp_path, m, script_type="sh")
specify_skill = tmp_path / ".claude/skills/speckit-specify/SKILL.md"
assert specify_skill.exists()
content = specify_skill.read_text(encoding="utf-8")
# specify.md has hook sections
assert "replace dots" in content, (
"speckit-specify should have dot-to-hyphen hook note"
)
def test_hook_note_not_in_skills_without_hooks(self, tmp_path):
"""Skills without hook sections should not get the note."""
content = "---\nname: test\ndescription: test\n---\n\nNo hooks here.\n"
result = SkillsIntegration._inject_hook_command_note(content)
assert "replace dots" not in result
def test_hook_note_idempotent(self, tmp_path):
"""Injecting the note twice should not duplicate it."""
content = (
"---\nname: test\n---\n\n"
"- For each executable hook, output the following based on its flag:\n"
)
once = SkillsIntegration._inject_hook_command_note(content)
twice = SkillsIntegration._inject_hook_command_note(once)
assert once == twice, "Hook note injection should be idempotent"
def test_hook_note_fills_missing_repeated_instructions(self, tmp_path):
"""Already-noted hook sections should not suppress later sections."""
from specify_cli.integrations.base import _HOOK_COMMAND_NOTE
content = (
"---\nname: test\n---\n\n"
f"{_HOOK_COMMAND_NOTE}"
"- For each executable hook, output the following based on its flag:\n"
"\n"
" - For each executable hook, output the following based on its flag:\n"
)
result = SkillsIntegration._inject_hook_command_note(content)
assert result.count("replace dots (`.`) with hyphens") == 2
def test_hook_note_not_suppressed_by_unrelated_phrase(self, tmp_path):
"""Unrelated text should not trip the hook-note idempotence guard."""
content = (
"---\nname: test\n---\n\n"
"This paragraph says replace dots in a different context.\n"
"- For each executable hook, output the following based on its flag:\n"
)
result = SkillsIntegration._inject_hook_command_note(content)
assert "This paragraph says replace dots in a different context." in result
assert result.count("replace dots (`.`) with hyphens") == 1
def test_hook_note_preserves_indentation(self, tmp_path):
"""The injected note should match the indentation of the target line."""
content = (
"---\nname: test\n---\n\n"
" - For each executable hook, output the following\n"
)
result = SkillsIntegration._inject_hook_command_note(content)
lines = result.splitlines()
note_line = [line for line in lines if "replace dots" in line][0]
assert note_line.startswith(" "), "Note should preserve indentation"
def test_post_process_injects_all_claude_flags(self):
"""post_process_skill_content should inject all Claude-specific fields."""
i = get_integration("claude")
content = (
"---\nname: test\ndescription: test\n---\n\n"
"- For each executable hook, output the following\n"
)
result = i.post_process_skill_content(content)
assert "user-invocable: true" in result
assert "disable-model-invocation: false" in result
assert "replace dots" in result
class TestSpeckitManifestRecordsSkippedFiles:
"""Regression test for issue #2107.
``install_shared_infra`` must record every shared-infrastructure file
under ``.specify/`` in ``speckit.manifest.json``, including files that
were *skipped* because they already existed on disk and ``force=False``.
Before the fix, the skip branches in the scripts and templates loops
appended to ``skipped_files`` without calling ``manifest.record_existing``.
So when ``install_shared_infra`` ran with a fresh (or lost) manifest
against an already-populated ``.specify/`` tree, every file went down the
skip path, ``planned_copies`` and ``planned_templates`` stayed empty, and
``manifest.save()`` wrote an empty ``files`` field — leaving the
integration believing nothing was installed.
Reproduction (without the fix) using ``install_shared_infra`` directly:
install_shared_infra(p, "sh", ..., force=False) # 1st run → 10 files
(p / ".specify/integrations/speckit.manifest.json").unlink()
install_shared_infra(p, "sh", ..., force=False) # 2nd run → 0 files
# ^^ BUG: empty
"""
def _read_manifest_files(self, project_path: Path) -> dict:
manifest_path = (
project_path / ".specify" / "integrations" / "speckit.manifest.json"
)
assert manifest_path.exists(), (
f"speckit.manifest.json not written at {manifest_path}"
)
data = json.loads(manifest_path.read_text(encoding="utf-8"))
# ``IntegrationManifest.save`` serialises a ``files`` dict — assert
# the schema explicitly so a regression to a different key (e.g.
# the internal ``_files`` attribute name) fails loudly instead of
# being masked by a silent fallback.
assert isinstance(data, dict), (
f"manifest root is not a dict, got {type(data).__name__}"
)
assert "files" in data, (
f"manifest missing 'files' key, got keys: {sorted(data.keys())}"
)
files = data["files"]
assert isinstance(files, dict), (
f"manifest 'files' is not a dict, got {type(files).__name__}"
)
return files
def test_install_shared_infra_records_skipped_files(self, tmp_path):
"""With ``force=False`` and ``.specify/`` already populated, the
manifest must still record every file — the skip branches are not
allowed to drop files from the manifest."""
from rich.console import Console
from specify_cli.shared_infra import install_shared_infra
# Resolve the project's own packaged sources by walking up from this
# test file to the repo root (which contains ``scripts/`` and
# ``templates/`` that ``shared_scripts_source`` looks for).
repo_root = Path(__file__).resolve().parents[2]
console = Console(quiet=True)
# First run — fresh project, manifest gets populated normally.
install_shared_infra(
tmp_path,
"sh",
version="0.0.0",
core_pack=None,
repo_root=repo_root,
console=console,
force=False,
)
first_files = self._read_manifest_files(tmp_path)
assert first_files, "first install produced an empty manifest"
# Simulate a lost manifest while ``.specify/`` is still on disk
# (e.g. the manifest was deleted, corrupted, or the layout was
# extracted out-of-band).
manifest_path = (
tmp_path / ".specify" / "integrations" / "speckit.manifest.json"
)
manifest_path.unlink()
# Second run — every file already exists, so every iteration takes
# the skip branch. With the fix, those files are still recorded.
install_shared_infra(
tmp_path,
"sh",
version="0.0.0",
core_pack=None,
repo_root=repo_root,
console=console,
force=False,
)
second_files = self._read_manifest_files(tmp_path)
assert second_files, (
"speckit.manifest.json files dict is empty after install with "
"skipped files (issue #2107) — every file went down the skip "
"branch but none were recorded"
)
# The recovered manifest must cover everything the first run tracked.
missing = set(first_files) - set(second_files)
assert not missing, (
f"these files were tracked on the first install but missing after "
f"the skipped-files re-install: {sorted(missing)[:5]}"
)
def test_install_shared_infra_handles_directory_at_script_destination(
self, tmp_path
):
"""A non-file (directory) at a script's destination must NOT crash
``install_shared_infra`` and must NOT be recorded in the manifest —
the path still appears in the user-visible skipped-paths warning.
"""
from io import StringIO
from rich.console import Console
from specify_cli.shared_infra import install_shared_infra
repo_root = Path(__file__).resolve().parents[2]
output = StringIO()
console = Console(file=output, force_terminal=False, width=200)
# Pre-create the .specify/scripts/bash tree, then plant a directory
# where a script file is expected so the skip branch hits a
# non-regular-file path.
bash_dir = tmp_path / ".specify" / "scripts" / "bash"
bash_dir.mkdir(parents=True)
(bash_dir / "common.sh").mkdir() # collision: dir where file expected
# Must not crash.
install_shared_infra(
tmp_path,
"sh",
version="0.0.0",
core_pack=None,
repo_root=repo_root,
console=console,
force=False,
)
files = self._read_manifest_files(tmp_path)
assert ".specify/scripts/bash/common.sh" not in files, (
"directory at script dst must not be recorded in the manifest"
)
text = output.getvalue()
assert "common.sh" in text, (
"directory-at-script-dst path must surface in the skipped warning"
)
def test_install_shared_infra_handles_directory_at_template_destination(
self, tmp_path
):
"""Symmetric coverage for the templates loop: a directory at a
template's destination must NOT crash install nor be recorded."""
from io import StringIO
from rich.console import Console
from specify_cli.shared_infra import install_shared_infra
repo_root = Path(__file__).resolve().parents[2]
output = StringIO()
console = Console(file=output, force_terminal=False, width=200)
templates_dir = tmp_path / ".specify" / "templates"
templates_dir.mkdir(parents=True)
src_templates = repo_root / "templates"
real_template = next(
(
p.name
for p in src_templates.iterdir()
if p.is_file()
and not p.name.startswith(".")
and p.name != "vscode-settings.json"
),
None,
)
assert real_template, (
"no real template found in repo to collide against"
)
(templates_dir / real_template).mkdir() # collision
install_shared_infra(
tmp_path,
"sh",
version="0.0.0",
core_pack=None,
repo_root=repo_root,
console=console,
force=False,
)
files = self._read_manifest_files(tmp_path)
template_rel = f".specify/templates/{real_template}"
assert template_rel not in files, (
"directory at template dst must not be recorded in manifest"
)
text = output.getvalue()
assert real_template in text, (
"directory-at-template-dst path must surface in the skipped warning"
)