Files
github-spec-kit/tests/integrations/test_integration_base_markdown.py
Ben Buttigieg 353851e966 fix(presets): seed constitution from preset constitution-template (#3272) (#3276)
* fix(presets): seed constitution from preset constitution-template (#3272)

The constitution is the only template materialized to a live file
(.specify/memory/constitution.md) rather than resolved on demand, yet
ensure_constitution_from_template hardcoded a copy from the core template
and ignored PresetResolver. Combined with init seeding the constitution
before preset installation, a preset's constitution-template (e.g.
strategy: replace with a ratified constitution) could never go live.

Changes:
- ensure_constitution_from_template now resolves constitution-template
  through PresetResolver, so a preset/override/extension wins and core is
  the fallback.
- init seeds the constitution after preset installation so init --preset
  uses the resolved stack.
- install_from_directory re-seeds memory/constitution.md from the resolved
  preset template, guarded to only act when the memory file is missing or
  still contains generic placeholder tokens — authored constitutions are
  never overwritten. Covers preset add and install_from_zip.
- Tests for preset seeding, placeholder re-seed, authored-constitution
  preservation, override resolution, and resolver-aware init seeding.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix(presets): compose constitution-template when seeding memory

Take on review feedback from Copilot and gglachant:
- constitution seeding previously copied the top layer file path verbatim
  even when the winning layer used a composing strategy
  (prepend/append/wrap), which could leave {CORE_TEMPLATE} unresolved.
- both seeding paths now inspect resolver layers and only copy verbatim for
  replace; non-replace strategies materialize composed content via
  PresetResolver.resolve_content().
- add regression tests for wrap strategy composition in both
  PresetManager seeding and ensure_constitution_from_template.
- add a drift-guard test pinning _CONSTITUTION_PLACEHOLDER_TOKENS to the
  placeholders in templates/constitution-template.md.

Assisted-by: GitHub Copilot (model: GPT-5.3-Codex, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* refactor(presets): unify constitution template materialization

Address latest Copilot feedback on the constitution seeding path:
- moved resolver/layer I/O behind the existing-memory fast path in init
- corrected tracker output for composed materialization
- deduplicated materialization logic shared by init and preset install seeding
  into presets._materialize_constitution_template()

Behavior is unchanged for replace strategies (copy verbatim) and remains
composed for prepend/append/wrap via resolve_content().

Assisted-by: GitHub Copilot (model: GPT-5.3-Codex, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix(init): restore shutil import

The constitution materialization refactor removed the module import, but init
still uses shutil.rmtree when cleaning up a failed new-project initialization.
Restore the import so the required ruff check passes.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix(presets): harden constitution materialization

Address the outstanding review batch for preset constitution seeding:
- use checked atomic writes and reject symlinked memory paths
- replace placeholder heuristics with hash/source provenance
- rematerialize unchanged generated constitutions by resolver priority
- preserve authored or edited constitutions, including placeholder mentions
- warn non-fatally when post-install materialization cannot complete
- retain exact core-template comparison for legacy projects without provenance

Add focused provenance, priority, symlink, and failure-path coverage, and
update integration inventories for the generated provenance sidecar.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1b2c095d-b45c-4d52-8d56-bd6121d96ab6

* fix(presets): reconcile constitution after removal

When the removed preset supplied constitution-template, rematerialize the
winning remaining resolver layer only if provenance proves the live file is
still generated and unchanged. Preserve edited constitutions and report
post-removal reconciliation failures as non-fatal warnings.

Add coverage for restoring the core layer, falling back from a removed
higher-priority preset, and preserving edited generated content.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1b2c095d-b45c-4d52-8d56-bd6121d96ab6

* fix(presets): tighten legacy constitution provenance

Trust only the immutable bundled/source constitution template when migrating
legacy projects without provenance. Do not infer core provenance from mutable
project templates or preset source labels, including IDs beginning with core.

Also detect convention-based constitution-template files before preset removal
so unchanged generated constitutions reconcile to the next resolver layer.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1b2c095d-b45c-4d52-8d56-bd6121d96ab6

* fix(presets): preserve files with invalid provenance

Use immutable-core legacy migration only when the provenance sidecar is absent.
If a sidecar is malformed or its hash does not match the live constitution,
treat the file as edited and preserve it.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 1b2c095d-b45c-4d52-8d56-bd6121d96ab6

* fix(presets): reconcile constitution on stack changes

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 1b2c095d-b45c-4d52-8d56-bd6121d96ab6

* fix(presets): guard constitution reconciliation edges

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 1b2c095d-b45c-4d52-8d56-bd6121d96ab6

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-07-15 11:42:50 +01:00

315 lines
13 KiB
Python

"""Reusable test mixin for standard MarkdownIntegration subclasses.
Each per-agent test file sets ``KEY``, ``FOLDER``, ``COMMANDS_SUBDIR``,
and ``REGISTRAR_DIR``, then inherits all verification logic from
``MarkdownIntegrationTests``.
"""
import os
from specify_cli.integrations import INTEGRATION_REGISTRY, get_integration
from specify_cli.integrations.base import MarkdownIntegration
from specify_cli.integrations.manifest import IntegrationManifest
class MarkdownIntegrationTests:
"""Mixin — set class-level constants and inherit these tests.
Required class attrs on subclass::
KEY: str — integration registry key
FOLDER: str — e.g. ".claude/"
COMMANDS_SUBDIR: str — e.g. "commands"
REGISTRAR_DIR: str — e.g. ".claude/commands"
"""
KEY: str
FOLDER: str
COMMANDS_SUBDIR: str
REGISTRAR_DIR: str
# -- Registration -----------------------------------------------------
def test_registered(self):
assert self.KEY in INTEGRATION_REGISTRY
assert get_integration(self.KEY) is not None
def test_is_markdown_integration(self):
assert isinstance(get_integration(self.KEY), MarkdownIntegration)
# -- Config -----------------------------------------------------------
def test_config_folder(self):
i = get_integration(self.KEY)
assert i.config["folder"] == self.FOLDER
def test_config_commands_subdir(self):
i = get_integration(self.KEY)
assert i.config["commands_subdir"] == self.COMMANDS_SUBDIR
def test_registrar_config(self):
i = get_integration(self.KEY)
assert i.registrar_config["dir"] == self.REGISTRAR_DIR
assert i.registrar_config["format"] == "markdown"
assert i.registrar_config["args"] == "$ARGUMENTS"
assert i.registrar_config["extension"] == ".md"
# -- Setup / teardown -------------------------------------------------
def test_setup_creates_files(self, tmp_path):
i = get_integration(self.KEY)
m = IntegrationManifest(self.KEY, tmp_path)
created = i.setup(tmp_path, m)
assert len(created) > 0
cmd_files = [f for f in created if "scripts" not in f.parts]
for f in cmd_files:
assert f.exists()
assert f.name.startswith("speckit.")
assert f.name.endswith(".md")
def test_setup_writes_to_correct_directory(self, tmp_path):
i = get_integration(self.KEY)
m = IntegrationManifest(self.KEY, tmp_path)
created = i.setup(tmp_path, m)
expected_dir = i.commands_dest(tmp_path)
assert expected_dir.exists(), f"Expected directory {expected_dir} was not created"
cmd_files = [f for f in created if "scripts" not in f.parts]
assert len(cmd_files) > 0, "No command files were created"
for f in cmd_files:
assert f.resolve().parent == expected_dir.resolve(), (
f"{f} is not under {expected_dir}"
)
def test_templates_are_processed(self, tmp_path):
"""Command files must have placeholders replaced, not raw templates."""
i = get_integration(self.KEY)
m = IntegrationManifest(self.KEY, tmp_path)
created = i.setup(tmp_path, m)
cmd_files = [f for f in created if "scripts" not in f.parts]
assert len(cmd_files) > 0
for f in cmd_files:
content = f.read_text(encoding="utf-8")
assert "{SCRIPT}" not in content, f"{f.name} has unprocessed {{SCRIPT}}"
assert "__AGENT__" not in content, f"{f.name} has unprocessed __AGENT__"
assert "{ARGS}" not in content, f"{f.name} has unprocessed {{ARGS}}"
assert "__SPECKIT_COMMAND_" not in content, f"{f.name} has unprocessed __SPECKIT_COMMAND_*__"
assert "\nscripts:\n" not in content, f"{f.name} has unstripped scripts: block"
def test_plan_command_has_no_context_placeholder(self, tmp_path):
"""The generated plan command must not carry a context-file placeholder.
Agent context files are owned entirely by the opt-in agent-context
extension, so the core plan command must not reference one.
"""
i = get_integration(self.KEY)
m = IntegrationManifest(self.KEY, tmp_path)
i.setup(tmp_path, m)
plan_file = i.commands_dest(tmp_path) / i.command_filename("plan")
assert plan_file.exists(), f"Plan file {plan_file} not created"
content = plan_file.read_text(encoding="utf-8")
assert "__CONTEXT_FILE__" not in content, (
f"Plan command has unprocessed __CONTEXT_FILE__ placeholder in {plan_file.name}"
)
def test_all_files_tracked_in_manifest(self, tmp_path):
i = get_integration(self.KEY)
m = IntegrationManifest(self.KEY, tmp_path)
created = i.setup(tmp_path, m)
for f in created:
rel = f.resolve().relative_to(tmp_path.resolve()).as_posix()
assert rel in m.files, f"{rel} not tracked in manifest"
def test_install_uninstall_roundtrip(self, tmp_path):
i = get_integration(self.KEY)
m = IntegrationManifest(self.KEY, tmp_path)
created = i.install(tmp_path, m)
assert len(created) > 0
m.save()
for f in created:
assert f.exists()
removed, skipped = i.uninstall(tmp_path, m)
assert len(removed) == len(created)
assert skipped == []
def test_modified_file_survives_uninstall(self, tmp_path):
i = get_integration(self.KEY)
m = IntegrationManifest(self.KEY, tmp_path)
created = i.install(tmp_path, m)
m.save()
modified_file = created[0]
modified_file.write_text("user modified this", encoding="utf-8")
removed, skipped = i.uninstall(tmp_path, m)
assert modified_file.exists()
assert modified_file in skipped
# -- Context file ownership (extension-owned, opt-in) -----------------
def test_setup_does_not_write_context_section(self, tmp_path):
"""Setup must not create or manage any agent context file — that is
owned entirely by the opt-in agent-context extension."""
i = get_integration(self.KEY)
m = IntegrationManifest(self.KEY, tmp_path)
i.setup(tmp_path, m)
for path in tmp_path.rglob("*"):
if path.is_file():
text = path.read_text(encoding="utf-8", errors="ignore")
assert "<!-- SPECKIT START -->" not in text, (
f"Setup wrote a managed context section into {path} for {self.KEY}"
)
def test_teardown_leaves_existing_context_file_intact(self, tmp_path):
"""A user-authored context file must survive setup + teardown untouched."""
i = get_integration(self.KEY)
m = IntegrationManifest(self.KEY, tmp_path)
ctx_path = tmp_path / "AGENTS.md"
original = "# My Rules\n\nUser content.\n"
ctx_path.write_text(original, encoding="utf-8")
i.setup(tmp_path, m)
m.save()
i.teardown(tmp_path, m)
assert ctx_path.read_text(encoding="utf-8") == original
# -- CLI integration flag -------------------------------------------------
def test_integration_flag_auto_promotes(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / f"promote-{self.KEY}"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(app, [
"init", "--here", "--integration", self.KEY, "--script", "sh",
"--ignore-agent-tools",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"init --integration {self.KEY} failed: {result.output}"
i = get_integration(self.KEY)
cmd_dir = i.commands_dest(project)
assert cmd_dir.is_dir(), f"--integration {self.KEY} did not create commands directory"
def test_integration_flag_creates_files(self, tmp_path):
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / f"int-{self.KEY}"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
result = runner.invoke(app, [
"init", "--here", "--integration", self.KEY, "--script", "sh",
"--ignore-agent-tools",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"init --integration {self.KEY} failed: {result.output}"
i = get_integration(self.KEY)
cmd_dir = i.commands_dest(project)
assert cmd_dir.is_dir(), f"Commands directory {cmd_dir} not created"
commands = sorted(cmd_dir.glob("speckit.*"))
assert len(commands) > 0, f"No command files in {cmd_dir}"
# -- Complete file inventory ------------------------------------------
COMMAND_STEMS = [
"analyze", "clarify", "constitution", "converge", "implement",
"plan", "checklist", "specify", "tasks", "taskstoissues",
]
def _expected_files(self, script_variant: str) -> list[str]:
"""Build the expected file list for this integration + script variant."""
i = get_integration(self.KEY)
cmd_dir = i.registrar_config["dir"]
files = []
# Command files
for stem in self.COMMAND_STEMS:
files.append(f"{cmd_dir}/speckit.{stem}.md")
# Framework files
files.append(".specify/integration.json")
files.append(".specify/init-options.json")
files.append(f".specify/integrations/{self.KEY}.manifest.json")
files.append(".specify/integrations/speckit.manifest.json")
if script_variant == "sh":
for name in ["check-prerequisites.sh", "common.sh", "create-new-feature.sh",
"setup-plan.sh", "setup-tasks.sh"]:
files.append(f".specify/scripts/bash/{name}")
else:
for name in ["check-prerequisites.ps1", "common.ps1", "create-new-feature.ps1",
"setup-plan.ps1", "setup-tasks.ps1"]:
files.append(f".specify/scripts/powershell/{name}")
for name in ["checklist-template.md",
"constitution-template.md", "plan-template.md",
"spec-template.md", "tasks-template.md"]:
files.append(f".specify/templates/{name}")
files.append(".specify/memory/.constitution-template.json")
files.append(".specify/memory/constitution.md")
# Bundled workflow
files.append(".specify/workflows/speckit/workflow.yml")
files.append(".specify/workflows/workflow-registry.json")
return sorted(files)
def test_complete_file_inventory_sh(self, tmp_path):
"""Every file produced by specify init --integration <key> --script sh."""
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / f"inventory-sh-{self.KEY}"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
result = CliRunner().invoke(app, [
"init", "--here", "--integration", self.KEY, "--script", "sh",
"--ignore-agent-tools",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"init failed: {result.output}"
actual = sorted(p.relative_to(project).as_posix()
for p in project.rglob("*") if p.is_file() and ".git" not in p.parts)
expected = self._expected_files("sh")
assert actual == expected, (
f"Missing: {sorted(set(expected) - set(actual))}\n"
f"Extra: {sorted(set(actual) - set(expected))}"
)
def test_complete_file_inventory_ps(self, tmp_path):
"""Every file produced by specify init --integration <key> --script ps."""
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / f"inventory-ps-{self.KEY}"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
result = CliRunner().invoke(app, [
"init", "--here", "--integration", self.KEY, "--script", "ps",
"--ignore-agent-tools",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"init failed: {result.output}"
actual = sorted(p.relative_to(project).as_posix()
for p in project.rglob("*") if p.is_file() and ".git" not in p.parts)
expected = self._expected_files("ps")
assert actual == expected, (
f"Missing: {sorted(set(expected) - set(actual))}\n"
f"Extra: {sorted(set(actual) - set(expected))}"
)