mirror of
https://github.com/github/spec-kit.git
synced 2026-07-03 20:36:23 +08:00
* Add /speckit.converge SDD artifacts and project scaffolding Dogfood the converge feature through Spec Kit's own workflow: - spec.md, plan.md, tasks.md, research, data-model, contracts, quickstart - requirements checklist for the feature - ratified constitution v1.0.0 (.specify/memory) - Specify project scaffolding (.specify/, .github agent + prompt files) Defines a built-in /speckit.converge command that assesses spec/plan/tasks against the codebase and appends remaining work as new tasks (no git, no change tracking, append-only). Implementation not yet started. Excludes unrelated working-tree changes to agents.py, extensions.py, test_extensions.py, catalog.community.json, and README.md. * Implement /speckit.converge command Add the built-in converge command that assesses the codebase against a feature's spec.md, plan.md, and tasks.md and appends remaining unbuilt work as new traceable tasks to tasks.md (append-only; no git, no change tracking). - templates/commands/converge.md: full command body (load artifacts, assess code, classify findings missing/partial/contradicts/unrequested, append '## Phase N — Convergence' tasks with source-ref + gap-type, read-only guardrails, converged branch, handoff, before/after_converge hooks) - Register converge as a core command across all enumeration sites (SKILL_DESCRIPTIONS, _FALLBACK_CORE_COMMAND_NAMES, ARGUMENT_HINTS, and the integration test command lists incl. copilot/generic file inventories) - init.py Next Steps panel + README Core Commands table - tasks.md: T001-T024 complete (T025 manual quickstart pending) Full suite green: 2343 passed. * Record quickstart validation results for /speckit.converge (T025) All six quickstart scenarios validated (GitHub Copilot agent, macOS/zsh): S1 gap->appended traceable task, S2 implement+re-converge, S3 converged leaves tasks.md unchanged, S4 read-only boundaries, S5 missing-prereq stop, S6 cross- integration install (copilot + windsurf). Automated suite: 2343 passed. * Record 2026-06-16 re-verification results for /speckit.converge (T025) * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Fix integration upgrade deleting settings.json and dropping script +x Two upgrade-path bugs surfaced during converge E2E validation: - copilot upgrade stale-deleted .vscode/settings.json because setup() only tracks the file when it creates it; on upgrade the pre-existing file is merged and left untracked, so Phase 2 stale cleanup removed it. Add an integration-level stale_cleanup_exclusions() hook (CopilotIntegration returns {.vscode/settings.json}) and subtract it from stale_keys. - shared .specify/scripts/*.sh lost their execute bit because the managed refresh rewrites them with the bundled source mode (often 0o644) and nothing restored perms. Call ensure_executable_scripts() after the managed-refresh block (POSIX only). Add regression tests in TestIntegrationUpgrade covering both fixes (validated to fail without the fixes). * fix: resolve markdownlint errors in PR files Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: clean up runtime state files from PR Remove .specify state files that are per-project runtime artifacts: - feature.json, init-options.json, integration.json - manifest files, extension registry, bug artifacts These are generated by 'specify init' and should not be committed. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat: fold converge artifacts from #3003 and #3005 - Add speckit.converge Copilot agent and prompt files (#3003) - Add regression test for Claude argument hints (#3005) - Remove invalid converge entry from Claude argument hints - Fix documentation removing branch-prefix fallback claims Supersedes: #3003, #3005 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: remove non-converge specify scaffolding from PR Remove .specify/ artifacts, non-converge .github/agents and prompts, and copilot-instructions.md that were generated by 'specify init' and are not part of the converge command feature. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: remove SDD spec artifacts from PR Remove specs/001-converge-command/ — the spec/plan/tasks/research SDD artifacts produced while building this feature. spec-kit does not track a specs/ directory on main (those are outputs of running the workflow on the repo, not part of the shipped tool). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: remove generated Copilot converge command files Remove .github/agents/speckit.converge.agent.md and .github/prompts/speckit.converge.prompt.md — these are generated by 'specify init --integration copilot' from templates/commands/converge.md (all __SPECKIT_COMMAND_*__/{SCRIPT} tokens are resolved). main tracks no .github/agents or .github/prompts files; the template is the source of truth. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * chore: split out unrelated integration-upgrade fix Move the stale_cleanup_exclusions / executable-bit upgrade fix (base.py, copilot, _migrate_commands.py, test_integration_subcommand.py) out of this PR into its own change. This PR is now scoped purely to the /speckit.converge command. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: add converge to core command template ordering converge is a core command in SKILL_DESCRIPTIONS but was missing from _CORE_COMMAND_TEMPLATE_ORDER, so it sorted with the fallback rank. Add it after 'implement' to keep core-command ordering consistent across integrations. Addresses review feedback on #3001. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: make converge findings example neutral Replace the self-referential sample evidence text in the Convergence Findings table with a neutral placeholder so agents are less likely to copy nonsensical template-specific findings into real output. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * docs: clarify converge scope and hook outcome wording - Remove FR-specific parenthetical from code-scope rule so it doesn't imply a hard FR-001 reference exists in every feature - Replace unsupported 'pass outcome to hook context' instruction with explicit in-session outcome reporting before hook listing Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: align converge task example with tasks format Use (no colon) in the convergence task example so it matches tasks-template formatting and downstream expectations. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Clarification of usage Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * Potential fix for pull request finding Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> * docs: align converge phase/task-id format with tasks template - Use (colon) for consistency with tasks template - Clarify appended task IDs must be zero-padded ( style) - Update checklist example to a concrete zero-padded ID () Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs: standardize converge phase heading format Use consistently in converge.md (including the append-only contract section) to match Step 7 and tasks template style. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
361 lines
15 KiB
Python
361 lines
15 KiB
Python
"""Reusable test mixin for standard MarkdownIntegration subclasses.
|
|
|
|
Each per-agent test file sets ``KEY``, ``FOLDER``, ``COMMANDS_SUBDIR``,
|
|
``REGISTRAR_DIR``, and ``CONTEXT_FILE``, then inherits all verification
|
|
logic from ``MarkdownIntegrationTests``.
|
|
"""
|
|
|
|
import os
|
|
|
|
from specify_cli.integrations import INTEGRATION_REGISTRY, get_integration
|
|
from specify_cli.integrations.base import MarkdownIntegration
|
|
from specify_cli.integrations.manifest import IntegrationManifest
|
|
|
|
|
|
class MarkdownIntegrationTests:
|
|
"""Mixin — set class-level constants and inherit these tests.
|
|
|
|
Required class attrs on subclass::
|
|
|
|
KEY: str — integration registry key
|
|
FOLDER: str — e.g. ".claude/"
|
|
COMMANDS_SUBDIR: str — e.g. "commands"
|
|
REGISTRAR_DIR: str — e.g. ".claude/commands"
|
|
CONTEXT_FILE: str — e.g. "CLAUDE.md"
|
|
"""
|
|
|
|
KEY: str
|
|
FOLDER: str
|
|
COMMANDS_SUBDIR: str
|
|
REGISTRAR_DIR: str
|
|
CONTEXT_FILE: str
|
|
|
|
# -- Registration -----------------------------------------------------
|
|
|
|
def test_registered(self):
|
|
assert self.KEY in INTEGRATION_REGISTRY
|
|
assert get_integration(self.KEY) is not None
|
|
|
|
def test_is_markdown_integration(self):
|
|
assert isinstance(get_integration(self.KEY), MarkdownIntegration)
|
|
|
|
# -- Config -----------------------------------------------------------
|
|
|
|
def test_config_folder(self):
|
|
i = get_integration(self.KEY)
|
|
assert i.config["folder"] == self.FOLDER
|
|
|
|
def test_config_commands_subdir(self):
|
|
i = get_integration(self.KEY)
|
|
assert i.config["commands_subdir"] == self.COMMANDS_SUBDIR
|
|
|
|
def test_registrar_config(self):
|
|
i = get_integration(self.KEY)
|
|
assert i.registrar_config["dir"] == self.REGISTRAR_DIR
|
|
assert i.registrar_config["format"] == "markdown"
|
|
assert i.registrar_config["args"] == "$ARGUMENTS"
|
|
assert i.registrar_config["extension"] == ".md"
|
|
|
|
def test_context_file(self):
|
|
i = get_integration(self.KEY)
|
|
assert i.context_file == self.CONTEXT_FILE
|
|
|
|
# -- Setup / teardown -------------------------------------------------
|
|
|
|
def test_setup_creates_files(self, tmp_path):
|
|
i = get_integration(self.KEY)
|
|
m = IntegrationManifest(self.KEY, tmp_path)
|
|
created = i.setup(tmp_path, m)
|
|
assert len(created) > 0
|
|
cmd_files = [f for f in created if "scripts" not in f.parts]
|
|
for f in cmd_files:
|
|
assert f.exists()
|
|
assert f.name.startswith("speckit.")
|
|
assert f.name.endswith(".md")
|
|
|
|
def test_setup_writes_to_correct_directory(self, tmp_path):
|
|
i = get_integration(self.KEY)
|
|
m = IntegrationManifest(self.KEY, tmp_path)
|
|
created = i.setup(tmp_path, m)
|
|
expected_dir = i.commands_dest(tmp_path)
|
|
assert expected_dir.exists(), f"Expected directory {expected_dir} was not created"
|
|
cmd_files = [f for f in created if "scripts" not in f.parts]
|
|
assert len(cmd_files) > 0, "No command files were created"
|
|
for f in cmd_files:
|
|
assert f.resolve().parent == expected_dir.resolve(), (
|
|
f"{f} is not under {expected_dir}"
|
|
)
|
|
|
|
def test_templates_are_processed(self, tmp_path):
|
|
"""Command files must have placeholders replaced, not raw templates."""
|
|
i = get_integration(self.KEY)
|
|
m = IntegrationManifest(self.KEY, tmp_path)
|
|
created = i.setup(tmp_path, m)
|
|
cmd_files = [f for f in created if "scripts" not in f.parts]
|
|
assert len(cmd_files) > 0
|
|
for f in cmd_files:
|
|
content = f.read_text(encoding="utf-8")
|
|
assert "{SCRIPT}" not in content, f"{f.name} has unprocessed {{SCRIPT}}"
|
|
assert "__AGENT__" not in content, f"{f.name} has unprocessed __AGENT__"
|
|
assert "{ARGS}" not in content, f"{f.name} has unprocessed {{ARGS}}"
|
|
assert "__SPECKIT_COMMAND_" not in content, f"{f.name} has unprocessed __SPECKIT_COMMAND_*__"
|
|
assert "\nscripts:\n" not in content, f"{f.name} has unstripped scripts: block"
|
|
|
|
def test_plan_references_correct_context_file(self, tmp_path):
|
|
"""The generated plan command must reference this integration's context file."""
|
|
i = get_integration(self.KEY)
|
|
if not i.context_file:
|
|
return
|
|
m = IntegrationManifest(self.KEY, tmp_path)
|
|
i.setup(tmp_path, m)
|
|
plan_file = i.commands_dest(tmp_path) / i.command_filename("plan")
|
|
assert plan_file.exists(), f"Plan file {plan_file} not created"
|
|
content = plan_file.read_text(encoding="utf-8")
|
|
assert i.context_file in content, (
|
|
f"Plan command should reference {i.context_file!r} but it was not found in {plan_file.name}"
|
|
)
|
|
assert "__CONTEXT_FILE__" not in content, (
|
|
f"Plan command has unprocessed __CONTEXT_FILE__ placeholder in {plan_file.name}"
|
|
)
|
|
|
|
def test_all_files_tracked_in_manifest(self, tmp_path):
|
|
i = get_integration(self.KEY)
|
|
m = IntegrationManifest(self.KEY, tmp_path)
|
|
created = i.setup(tmp_path, m)
|
|
for f in created:
|
|
rel = f.resolve().relative_to(tmp_path.resolve()).as_posix()
|
|
assert rel in m.files, f"{rel} not tracked in manifest"
|
|
|
|
def test_install_uninstall_roundtrip(self, tmp_path):
|
|
i = get_integration(self.KEY)
|
|
m = IntegrationManifest(self.KEY, tmp_path)
|
|
created = i.install(tmp_path, m)
|
|
assert len(created) > 0
|
|
m.save()
|
|
for f in created:
|
|
assert f.exists()
|
|
removed, skipped = i.uninstall(tmp_path, m)
|
|
assert len(removed) == len(created)
|
|
assert skipped == []
|
|
|
|
def test_modified_file_survives_uninstall(self, tmp_path):
|
|
i = get_integration(self.KEY)
|
|
m = IntegrationManifest(self.KEY, tmp_path)
|
|
created = i.install(tmp_path, m)
|
|
m.save()
|
|
modified_file = created[0]
|
|
modified_file.write_text("user modified this", encoding="utf-8")
|
|
removed, skipped = i.uninstall(tmp_path, m)
|
|
assert modified_file.exists()
|
|
assert modified_file in skipped
|
|
|
|
# -- Context section ---------------------------------------------------
|
|
|
|
def test_setup_upserts_context_section(self, tmp_path):
|
|
i = get_integration(self.KEY)
|
|
m = IntegrationManifest(self.KEY, tmp_path)
|
|
i.setup(tmp_path, m)
|
|
if i.context_file:
|
|
ctx_path = tmp_path / i.context_file
|
|
assert ctx_path.exists(), f"Context file {i.context_file} not created for {self.KEY}"
|
|
content = ctx_path.read_text(encoding="utf-8")
|
|
assert "<!-- SPECKIT START -->" in content
|
|
assert "<!-- SPECKIT END -->" in content
|
|
assert "read the current plan" in content
|
|
|
|
def test_teardown_removes_context_section(self, tmp_path):
|
|
i = get_integration(self.KEY)
|
|
m = IntegrationManifest(self.KEY, tmp_path)
|
|
i.setup(tmp_path, m)
|
|
m.save()
|
|
if i.context_file:
|
|
ctx_path = tmp_path / i.context_file
|
|
# Add user content around the section
|
|
content = ctx_path.read_text(encoding="utf-8")
|
|
ctx_path.write_text("# My Rules\n\n" + content + "\n# Footer\n", encoding="utf-8")
|
|
i.teardown(tmp_path, m)
|
|
remaining = ctx_path.read_text(encoding="utf-8")
|
|
assert "<!-- SPECKIT START -->" not in remaining
|
|
assert "<!-- SPECKIT END -->" not in remaining
|
|
assert "# My Rules" in remaining
|
|
|
|
# -- CLI integration flag -------------------------------------------------
|
|
|
|
def test_integration_flag_auto_promotes(self, tmp_path):
|
|
from typer.testing import CliRunner
|
|
from specify_cli import app
|
|
|
|
project = tmp_path / f"promote-{self.KEY}"
|
|
project.mkdir()
|
|
old_cwd = os.getcwd()
|
|
try:
|
|
os.chdir(project)
|
|
runner = CliRunner()
|
|
result = runner.invoke(app, [
|
|
"init", "--here", "--integration", self.KEY, "--script", "sh",
|
|
"--ignore-agent-tools",
|
|
], catch_exceptions=False)
|
|
finally:
|
|
os.chdir(old_cwd)
|
|
assert result.exit_code == 0, f"init --integration {self.KEY} failed: {result.output}"
|
|
i = get_integration(self.KEY)
|
|
cmd_dir = i.commands_dest(project)
|
|
assert cmd_dir.is_dir(), f"--integration {self.KEY} did not create commands directory"
|
|
|
|
def test_integration_flag_creates_files(self, tmp_path):
|
|
from typer.testing import CliRunner
|
|
from specify_cli import app
|
|
|
|
project = tmp_path / f"int-{self.KEY}"
|
|
project.mkdir()
|
|
old_cwd = os.getcwd()
|
|
try:
|
|
os.chdir(project)
|
|
runner = CliRunner()
|
|
result = runner.invoke(app, [
|
|
"init", "--here", "--integration", self.KEY, "--script", "sh",
|
|
"--ignore-agent-tools",
|
|
], catch_exceptions=False)
|
|
finally:
|
|
os.chdir(old_cwd)
|
|
assert result.exit_code == 0, f"init --integration {self.KEY} failed: {result.output}"
|
|
i = get_integration(self.KEY)
|
|
cmd_dir = i.commands_dest(project)
|
|
assert cmd_dir.is_dir(), f"Commands directory {cmd_dir} not created"
|
|
commands = sorted(cmd_dir.glob("speckit.*"))
|
|
assert len(commands) > 0, f"No command files in {cmd_dir}"
|
|
|
|
def test_init_options_includes_context_file(self, tmp_path):
|
|
"""agent-context extension config must include context_file for the active integration."""
|
|
import yaml
|
|
from typer.testing import CliRunner
|
|
from specify_cli import app
|
|
|
|
project = tmp_path / f"opts-{self.KEY}"
|
|
project.mkdir()
|
|
old_cwd = os.getcwd()
|
|
try:
|
|
os.chdir(project)
|
|
result = CliRunner().invoke(app, [
|
|
"init", "--here", "--integration", self.KEY, "--script", "sh",
|
|
"--ignore-agent-tools",
|
|
], catch_exceptions=False)
|
|
finally:
|
|
os.chdir(old_cwd)
|
|
assert result.exit_code == 0
|
|
ext_cfg_path = project / ".specify" / "extensions" / "agent-context" / "agent-context-config.yml"
|
|
ext_cfg = yaml.safe_load(ext_cfg_path.read_text(encoding="utf-8")) if ext_cfg_path.exists() else {}
|
|
i = get_integration(self.KEY)
|
|
assert ext_cfg.get("context_file") == i.context_file, (
|
|
f"Expected context_file={i.context_file!r}, got {ext_cfg.get('context_file')!r}"
|
|
)
|
|
|
|
# -- Complete file inventory ------------------------------------------
|
|
|
|
COMMAND_STEMS = [
|
|
"agent-context.update",
|
|
"analyze", "clarify", "constitution", "converge", "implement",
|
|
"plan", "checklist", "specify", "tasks", "taskstoissues",
|
|
]
|
|
|
|
def _expected_files(self, script_variant: str) -> list[str]:
|
|
"""Build the expected file list for this integration + script variant."""
|
|
i = get_integration(self.KEY)
|
|
cmd_dir = i.registrar_config["dir"]
|
|
files = []
|
|
|
|
# Command files
|
|
for stem in self.COMMAND_STEMS:
|
|
files.append(f"{cmd_dir}/speckit.{stem}.md")
|
|
|
|
# Framework files
|
|
files.append(".specify/integration.json")
|
|
files.append(".specify/init-options.json")
|
|
files.append(f".specify/integrations/{self.KEY}.manifest.json")
|
|
files.append(".specify/integrations/speckit.manifest.json")
|
|
|
|
if script_variant == "sh":
|
|
for name in ["check-prerequisites.sh", "common.sh", "create-new-feature.sh",
|
|
"setup-plan.sh", "setup-tasks.sh"]:
|
|
files.append(f".specify/scripts/bash/{name}")
|
|
else:
|
|
for name in ["check-prerequisites.ps1", "common.ps1", "create-new-feature.ps1",
|
|
"setup-plan.ps1", "setup-tasks.ps1"]:
|
|
files.append(f".specify/scripts/powershell/{name}")
|
|
|
|
for name in ["checklist-template.md",
|
|
"constitution-template.md", "plan-template.md",
|
|
"spec-template.md", "tasks-template.md"]:
|
|
files.append(f".specify/templates/{name}")
|
|
|
|
files.append(".specify/memory/constitution.md")
|
|
# Bundled workflow
|
|
files.append(".specify/workflows/speckit/workflow.yml")
|
|
files.append(".specify/workflows/workflow-registry.json")
|
|
|
|
# Bundled agent-context extension
|
|
files.append(".specify/extensions.yml")
|
|
files.append(".specify/extensions/.registry")
|
|
files.append(".specify/extensions/agent-context/README.md")
|
|
files.append(".specify/extensions/agent-context/agent-context-config.yml")
|
|
files.append(".specify/extensions/agent-context/commands/speckit.agent-context.update.md")
|
|
files.append(".specify/extensions/agent-context/extension.yml")
|
|
files.append(".specify/extensions/agent-context/scripts/bash/update-agent-context.sh")
|
|
files.append(".specify/extensions/agent-context/scripts/powershell/update-agent-context.ps1")
|
|
|
|
# Agent context file (if set)
|
|
if i.context_file:
|
|
files.append(i.context_file)
|
|
|
|
return sorted(files)
|
|
|
|
def test_complete_file_inventory_sh(self, tmp_path):
|
|
"""Every file produced by specify init --integration <key> --script sh."""
|
|
from typer.testing import CliRunner
|
|
from specify_cli import app
|
|
|
|
project = tmp_path / f"inventory-sh-{self.KEY}"
|
|
project.mkdir()
|
|
old_cwd = os.getcwd()
|
|
try:
|
|
os.chdir(project)
|
|
result = CliRunner().invoke(app, [
|
|
"init", "--here", "--integration", self.KEY, "--script", "sh",
|
|
"--ignore-agent-tools",
|
|
], catch_exceptions=False)
|
|
finally:
|
|
os.chdir(old_cwd)
|
|
assert result.exit_code == 0, f"init failed: {result.output}"
|
|
actual = sorted(p.relative_to(project).as_posix()
|
|
for p in project.rglob("*") if p.is_file() and ".git" not in p.parts)
|
|
expected = self._expected_files("sh")
|
|
assert actual == expected, (
|
|
f"Missing: {sorted(set(expected) - set(actual))}\n"
|
|
f"Extra: {sorted(set(actual) - set(expected))}"
|
|
)
|
|
|
|
def test_complete_file_inventory_ps(self, tmp_path):
|
|
"""Every file produced by specify init --integration <key> --script ps."""
|
|
from typer.testing import CliRunner
|
|
from specify_cli import app
|
|
|
|
project = tmp_path / f"inventory-ps-{self.KEY}"
|
|
project.mkdir()
|
|
old_cwd = os.getcwd()
|
|
try:
|
|
os.chdir(project)
|
|
result = CliRunner().invoke(app, [
|
|
"init", "--here", "--integration", self.KEY, "--script", "ps",
|
|
"--ignore-agent-tools",
|
|
], catch_exceptions=False)
|
|
finally:
|
|
os.chdir(old_cwd)
|
|
assert result.exit_code == 0, f"init failed: {result.output}"
|
|
actual = sorted(p.relative_to(project).as_posix()
|
|
for p in project.rglob("*") if p.is_file() and ".git" not in p.parts)
|
|
expected = self._expected_files("ps")
|
|
assert actual == expected, (
|
|
f"Missing: {sorted(set(expected) - set(actual))}\n"
|
|
f"Extra: {sorted(set(actual) - set(expected))}"
|
|
)
|