Files
github-spec-kit/tests/integrations/test_integration_generic.py
Ben Buttigieg 0c29d890ab feat: add /speckit.converge command (#3001)
* Add /speckit.converge SDD artifacts and project scaffolding

Dogfood the converge feature through Spec Kit's own workflow:

- spec.md, plan.md, tasks.md, research, data-model, contracts, quickstart
- requirements checklist for the feature
- ratified constitution v1.0.0 (.specify/memory)
- Specify project scaffolding (.specify/, .github agent + prompt files)

Defines a built-in /speckit.converge command that assesses spec/plan/tasks
against the codebase and appends remaining work as new tasks (no git, no
change tracking, append-only). Implementation not yet started.

Excludes unrelated working-tree changes to agents.py, extensions.py,
test_extensions.py, catalog.community.json, and README.md.

* Implement /speckit.converge command

Add the built-in converge command that assesses the codebase against a
feature's spec.md, plan.md, and tasks.md and appends remaining unbuilt work
as new traceable tasks to tasks.md (append-only; no git, no change tracking).

- templates/commands/converge.md: full command body (load artifacts, assess
  code, classify findings missing/partial/contradicts/unrequested, append
  '## Phase N — Convergence' tasks with source-ref + gap-type, read-only
  guardrails, converged branch, handoff, before/after_converge hooks)
- Register converge as a core command across all enumeration sites
  (SKILL_DESCRIPTIONS, _FALLBACK_CORE_COMMAND_NAMES, ARGUMENT_HINTS, and the
  integration test command lists incl. copilot/generic file inventories)
- init.py Next Steps panel + README Core Commands table
- tasks.md: T001-T024 complete (T025 manual quickstart pending)

Full suite green: 2343 passed.

* Record quickstart validation results for /speckit.converge (T025)

All six quickstart scenarios validated (GitHub Copilot agent, macOS/zsh):
S1 gap->appended traceable task, S2 implement+re-converge, S3 converged leaves
tasks.md unchanged, S4 read-only boundaries, S5 missing-prereq stop, S6 cross-
integration install (copilot + windsurf). Automated suite: 2343 passed.

* Record 2026-06-16 re-verification results for /speckit.converge (T025)

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix integration upgrade deleting settings.json and dropping script +x

Two upgrade-path bugs surfaced during converge E2E validation:

- copilot upgrade stale-deleted .vscode/settings.json because setup() only tracks the file when it creates it; on upgrade the pre-existing file is merged and left untracked, so Phase 2 stale cleanup removed it. Add an integration-level stale_cleanup_exclusions() hook (CopilotIntegration returns {.vscode/settings.json}) and subtract it from stale_keys.

- shared .specify/scripts/*.sh lost their execute bit because the managed refresh rewrites them with the bundled source mode (often 0o644) and nothing restored perms. Call ensure_executable_scripts() after the managed-refresh block (POSIX only).

Add regression tests in TestIntegrationUpgrade covering both fixes (validated to fail without the fixes).

* fix: resolve markdownlint errors in PR files

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: clean up runtime state files from PR

Remove .specify state files that are per-project runtime artifacts:
- feature.json, init-options.json, integration.json
- manifest files, extension registry, bug artifacts

These are generated by 'specify init' and should not be committed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat: fold converge artifacts from #3003 and #3005

- Add speckit.converge Copilot agent and prompt files (#3003)
- Add regression test for Claude argument hints (#3005)
- Remove invalid converge entry from Claude argument hints
- Fix documentation removing branch-prefix fallback claims

Supersedes: #3003, #3005

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: remove non-converge specify scaffolding from PR

Remove .specify/ artifacts, non-converge .github/agents and prompts,
and copilot-instructions.md that were generated by 'specify init'
and are not part of the converge command feature.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: remove SDD spec artifacts from PR

Remove specs/001-converge-command/ — the spec/plan/tasks/research SDD
artifacts produced while building this feature. spec-kit does not track
a specs/ directory on main (those are outputs of running the workflow on
the repo, not part of the shipped tool).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: remove generated Copilot converge command files

Remove .github/agents/speckit.converge.agent.md and
.github/prompts/speckit.converge.prompt.md — these are generated by
'specify init --integration copilot' from templates/commands/converge.md
(all __SPECKIT_COMMAND_*__/{SCRIPT} tokens are resolved). main tracks no
.github/agents or .github/prompts files; the template is the source of truth.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: split out unrelated integration-upgrade fix

Move the stale_cleanup_exclusions / executable-bit upgrade fix
(base.py, copilot, _migrate_commands.py, test_integration_subcommand.py)
out of this PR into its own change. This PR is now scoped purely to the
/speckit.converge command.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: add converge to core command template ordering

converge is a core command in SKILL_DESCRIPTIONS but was missing from
_CORE_COMMAND_TEMPLATE_ORDER, so it sorted with the fallback rank. Add it
after 'implement' to keep core-command ordering consistent across
integrations.

Addresses review feedback on #3001.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: make converge findings example neutral

Replace the self-referential sample evidence text in the Convergence
Findings table with a neutral placeholder so agents are less likely to copy
nonsensical template-specific findings into real output.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* docs: clarify converge scope and hook outcome wording

- Remove FR-specific parenthetical from code-scope rule so it doesn't imply
  a hard FR-001 reference exists in every feature
- Replace unsupported 'pass outcome to hook context' instruction with explicit
  in-session outcome reporting before hook listing

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: align converge task example with tasks format

Use  (no colon) in the convergence task example so it
matches tasks-template formatting and downstream expectations.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Clarification of usage

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* docs: align converge phase/task-id format with tasks template

- Use  (colon) for consistency with tasks template
- Clarify appended task IDs must be zero-padded ( style)
- Update checklist example to a concrete zero-padded ID ()

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: standardize converge phase heading format

Use  consistently in converge.md (including the
append-only contract section) to match Step 7 and tasks template style.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 14:47:00 -05:00

411 lines
18 KiB
Python

"""Tests for GenericIntegration."""
import os
import pytest
from specify_cli.integrations import get_integration
from specify_cli.integrations.base import MarkdownIntegration
from specify_cli.integrations.manifest import IntegrationManifest
class TestGenericIntegration:
"""Tests for GenericIntegration — requires --commands-dir option."""
# -- Registration -----------------------------------------------------
def test_registered(self):
from specify_cli.integrations import INTEGRATION_REGISTRY
assert "generic" in INTEGRATION_REGISTRY
def test_is_markdown_integration(self):
assert isinstance(get_integration("generic"), MarkdownIntegration)
# -- Config -----------------------------------------------------------
def test_config_folder_is_none(self):
i = get_integration("generic")
assert i.config["folder"] is None
def test_config_requires_cli_false(self):
i = get_integration("generic")
assert i.config["requires_cli"] is False
def test_context_file_is_agents_md(self):
i = get_integration("generic")
assert i.context_file == "AGENTS.md"
# -- Options ----------------------------------------------------------
def test_options_include_commands_dir(self):
i = get_integration("generic")
opts = i.options()
assert len(opts) == 1
assert opts[0].name == "--commands-dir"
assert opts[0].required is True
assert opts[0].is_flag is False
# -- Setup / teardown -------------------------------------------------
def test_setup_requires_commands_dir(self, tmp_path):
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
with pytest.raises(ValueError, match="--commands-dir is required"):
i.setup(tmp_path, m, parsed_options={})
def test_setup_requires_nonempty_commands_dir(self, tmp_path):
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
with pytest.raises(ValueError, match="--commands-dir is required"):
i.setup(tmp_path, m, parsed_options={"commands_dir": ""})
def test_setup_writes_to_correct_directory(self, tmp_path):
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
created = i.setup(
tmp_path, m,
parsed_options={"commands_dir": ".myagent/commands"},
)
expected_dir = tmp_path / ".myagent" / "commands"
assert expected_dir.exists(), f"Expected directory {expected_dir} was not created"
cmd_files = [f for f in created if "scripts" not in f.parts]
assert len(cmd_files) > 0, "No command files were created"
for f in cmd_files:
assert f.resolve().parent == expected_dir.resolve(), (
f"{f} is not under {expected_dir}"
)
def test_setup_creates_md_files(self, tmp_path):
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
created = i.setup(
tmp_path, m,
parsed_options={"commands_dir": ".custom/cmds"},
)
cmd_files = [f for f in created if "scripts" not in f.parts]
assert len(cmd_files) > 0
for f in cmd_files:
assert f.name.startswith("speckit.")
assert f.name.endswith(".md")
def test_templates_are_processed(self, tmp_path):
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
created = i.setup(
tmp_path, m,
parsed_options={"commands_dir": ".custom/cmds"},
)
cmd_files = [f for f in created if "scripts" not in f.parts]
for f in cmd_files:
content = f.read_text(encoding="utf-8")
assert "{SCRIPT}" not in content, f"{f.name} has unprocessed {{SCRIPT}}"
assert "__AGENT__" not in content, f"{f.name} has unprocessed __AGENT__"
assert "{ARGS}" not in content, f"{f.name} has unprocessed {{ARGS}}"
assert "__SPECKIT_COMMAND_" not in content, f"{f.name} has unprocessed __SPECKIT_COMMAND_*__"
def test_all_files_tracked_in_manifest(self, tmp_path):
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
created = i.setup(
tmp_path, m,
parsed_options={"commands_dir": ".custom/cmds"},
)
for f in created:
rel = f.resolve().relative_to(tmp_path.resolve()).as_posix()
assert rel in m.files, f"{rel} not tracked in manifest"
def test_install_uninstall_roundtrip(self, tmp_path):
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
created = i.install(
tmp_path, m,
parsed_options={"commands_dir": ".custom/cmds"},
)
assert len(created) > 0
m.save()
for f in created:
assert f.exists()
removed, skipped = i.uninstall(tmp_path, m)
assert len(removed) == len(created)
assert skipped == []
def test_modified_file_survives_uninstall(self, tmp_path):
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
created = i.install(
tmp_path, m,
parsed_options={"commands_dir": ".custom/cmds"},
)
m.save()
modified = created[0]
modified.write_text("user modified this", encoding="utf-8")
removed, skipped = i.uninstall(tmp_path, m)
assert modified.exists()
assert modified in skipped
def test_different_commands_dirs(self, tmp_path):
"""Generic should work with various user-specified paths."""
for path in [".agent/commands", "tools/ai-cmds", ".custom/prompts"]:
project = tmp_path / path.replace("/", "-")
project.mkdir()
i = get_integration("generic")
m = IntegrationManifest("generic", project)
created = i.setup(
project, m,
parsed_options={"commands_dir": path},
)
expected = project / path
assert expected.is_dir(), f"Dir {expected} not created for {path}"
cmd_files = [f for f in created if "scripts" not in f.parts]
assert len(cmd_files) > 0
# -- Context section ---------------------------------------------------
def test_setup_upserts_context_section(self, tmp_path):
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
i.setup(tmp_path, m, parsed_options={"commands_dir": ".custom/cmds"})
if i.context_file:
ctx_path = tmp_path / i.context_file
assert ctx_path.exists()
content = ctx_path.read_text(encoding="utf-8")
assert "<!-- SPECKIT START -->" in content
assert "<!-- SPECKIT END -->" in content
def test_plan_references_correct_context_file(self, tmp_path):
"""The generated plan command must reference generic's context file."""
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
i.setup(tmp_path, m, parsed_options={"commands_dir": ".custom/cmds"})
plan_file = tmp_path / ".custom" / "cmds" / "speckit.plan.md"
assert plan_file.exists()
content = plan_file.read_text(encoding="utf-8")
assert i.context_file in content, (
f"Plan command should reference {i.context_file!r}"
)
assert "__CONTEXT_FILE__" not in content
def test_plan_defines_quickstart_as_validation_guide(self, tmp_path):
"""The generated plan command should keep quickstart.md out of implementation scope."""
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
i.setup(tmp_path, m, parsed_options={"commands_dir": ".custom/cmds"})
plan_file = tmp_path / ".custom" / "cmds" / "speckit.plan.md"
assert plan_file.exists()
content = plan_file.read_text(encoding="utf-8")
assert "Create quickstart validation guide" in content
assert "runnable validation scenarios" in content
assert "Do not include full implementation code" in content
assert "implementation details belong in `tasks.md` and the implementation phase" in content
def test_implement_loads_constitution_context(self, tmp_path):
"""The generated implement command should load constitution governance context."""
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
i.setup(tmp_path, m, parsed_options={"commands_dir": ".custom/cmds"})
implement_file = tmp_path / ".custom" / "cmds" / "speckit.implement.md"
assert implement_file.exists()
content = implement_file.read_text(encoding="utf-8")
assert ".specify/memory/constitution.md" in content
@pytest.mark.parametrize(
"command_stem",
[
"analyze",
"clarify",
"converge",
"implement",
"plan",
"checklist",
"specify",
"tasks",
"taskstoissues",
],
)
def test_command_loads_constitution_context(self, tmp_path, command_stem):
"""Every command except constitution must reference constitution.md."""
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
i.setup(tmp_path, m, parsed_options={"commands_dir": ".custom/cmds"})
cmd_file = tmp_path / ".custom" / "cmds" / f"speckit.{command_stem}.md"
assert cmd_file.exists(), f"Command file missing: {cmd_file.name}"
content = cmd_file.read_text(encoding="utf-8")
assert "constitution.md" in content, (
f"speckit.{command_stem}.md must reference constitution.md"
)
def test_constitution_command_exists(self, tmp_path):
"""The constitution command itself must exist but is not required to load itself."""
i = get_integration("generic")
m = IntegrationManifest("generic", tmp_path)
i.setup(tmp_path, m, parsed_options={"commands_dir": ".custom/cmds"})
cmd_file = tmp_path / ".custom" / "cmds" / "speckit.constitution.md"
assert cmd_file.exists()
# -- CLI --------------------------------------------------------------
def test_cli_generic_without_commands_dir_fails(self, tmp_path):
"""--integration generic without --integration-options should fail."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
result = runner.invoke(app, [
"init", str(tmp_path / "test-generic"), "--integration", "generic",
])
# Generic requires --commands-dir via --integration-options
assert result.exit_code != 0
def test_init_options_includes_context_file(self, tmp_path):
"""agent-context extension config must include context_file for the generic integration."""
import yaml
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "opts-generic"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
result = CliRunner().invoke(app, [
"init", "--here", "--integration", "generic",
"--integration-options=--commands-dir .myagent/commands",
"--script", "sh",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0
ext_cfg_path = project / ".specify" / "extensions" / "agent-context" / "agent-context-config.yml"
ext_cfg = yaml.safe_load(ext_cfg_path.read_text(encoding="utf-8")) if ext_cfg_path.exists() else {}
assert ext_cfg.get("context_file") == "AGENTS.md"
def test_complete_file_inventory_sh(self, tmp_path):
"""Every file produced by specify init --integration generic --integration-options=--commands-dir ... --script sh."""
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "inventory-generic-sh"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
result = CliRunner().invoke(app, [
"init", "--here", "--integration", "generic",
"--integration-options=--commands-dir .myagent/commands",
"--script", "sh",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"init failed: {result.output}"
actual = sorted(
p.relative_to(project).as_posix()
for p in project.rglob("*") if p.is_file() and ".git" not in p.parts
)
expected = sorted([
"AGENTS.md",
".myagent/commands/speckit.analyze.md",
".myagent/commands/speckit.checklist.md",
".myagent/commands/speckit.clarify.md",
".myagent/commands/speckit.constitution.md",
".myagent/commands/speckit.converge.md",
".myagent/commands/speckit.implement.md",
".myagent/commands/speckit.plan.md",
".myagent/commands/speckit.specify.md",
".myagent/commands/speckit.tasks.md",
".myagent/commands/speckit.taskstoissues.md",
".specify/extensions.yml",
".specify/extensions/.registry",
".specify/extensions/agent-context/README.md",
".specify/extensions/agent-context/agent-context-config.yml",
".specify/extensions/agent-context/commands/speckit.agent-context.update.md",
".specify/extensions/agent-context/extension.yml",
".specify/extensions/agent-context/scripts/bash/update-agent-context.sh",
".specify/extensions/agent-context/scripts/powershell/update-agent-context.ps1",
".specify/init-options.json",
".specify/integration.json",
".specify/integrations/generic.manifest.json",
".specify/integrations/speckit.manifest.json",
".specify/memory/constitution.md",
".specify/scripts/bash/check-prerequisites.sh",
".specify/scripts/bash/common.sh",
".specify/scripts/bash/create-new-feature.sh",
".specify/scripts/bash/setup-plan.sh",
".specify/scripts/bash/setup-tasks.sh",
".specify/templates/checklist-template.md",
".specify/templates/constitution-template.md",
".specify/templates/plan-template.md",
".specify/templates/spec-template.md",
".specify/templates/tasks-template.md",
".specify/workflows/speckit/workflow.yml",
".specify/workflows/workflow-registry.json",
])
assert actual == expected, (
f"Missing: {sorted(set(expected) - set(actual))}\n"
f"Extra: {sorted(set(actual) - set(expected))}"
)
def test_complete_file_inventory_ps(self, tmp_path):
"""Every file produced by specify init --integration generic --integration-options=--commands-dir ... --script ps."""
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / "inventory-generic-ps"
project.mkdir()
old_cwd = os.getcwd()
try:
os.chdir(project)
result = CliRunner().invoke(app, [
"init", "--here", "--integration", "generic",
"--integration-options=--commands-dir .myagent/commands",
"--script", "ps",
], catch_exceptions=False)
finally:
os.chdir(old_cwd)
assert result.exit_code == 0, f"init failed: {result.output}"
actual = sorted(
p.relative_to(project).as_posix()
for p in project.rglob("*") if p.is_file() and ".git" not in p.parts
)
expected = sorted([
"AGENTS.md",
".myagent/commands/speckit.analyze.md",
".myagent/commands/speckit.checklist.md",
".myagent/commands/speckit.clarify.md",
".myagent/commands/speckit.constitution.md",
".myagent/commands/speckit.converge.md",
".myagent/commands/speckit.implement.md",
".myagent/commands/speckit.plan.md",
".myagent/commands/speckit.specify.md",
".myagent/commands/speckit.tasks.md",
".myagent/commands/speckit.taskstoissues.md",
".specify/extensions.yml",
".specify/extensions/.registry",
".specify/extensions/agent-context/README.md",
".specify/extensions/agent-context/agent-context-config.yml",
".specify/extensions/agent-context/commands/speckit.agent-context.update.md",
".specify/extensions/agent-context/extension.yml",
".specify/extensions/agent-context/scripts/bash/update-agent-context.sh",
".specify/extensions/agent-context/scripts/powershell/update-agent-context.ps1",
".specify/init-options.json",
".specify/integration.json",
".specify/integrations/generic.manifest.json",
".specify/integrations/speckit.manifest.json",
".specify/memory/constitution.md",
".specify/scripts/powershell/check-prerequisites.ps1",
".specify/scripts/powershell/common.ps1",
".specify/scripts/powershell/create-new-feature.ps1",
".specify/scripts/powershell/setup-plan.ps1",
".specify/scripts/powershell/setup-tasks.ps1",
".specify/templates/checklist-template.md",
".specify/templates/constitution-template.md",
".specify/templates/plan-template.md",
".specify/templates/spec-template.md",
".specify/templates/tasks-template.md",
".specify/workflows/speckit/workflow.yml",
".specify/workflows/workflow-registry.json",
])
assert actual == expected, (
f"Missing: {sorted(set(expected) - set(actual))}\n"
f"Extra: {sorted(set(actual) - set(expected))}"
)