Files
github-spec-kit/tests/integrations/test_integration_rovodev.py
Manfred Riem 53d9543355 feat: make agent-context extension a full opt-in (#3097)
* docs: add Spec Kit spec for agent-context full opt-in

Use Spec Kit's own specify workflow to author the spec that makes the
agent-context extension a full opt-in, removing all agent-context
configuration/support from the Python codebase and removing the
deprecation message. Force-added despite specs/ being gitignored; the
generated artifact will be purged prior to merge.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: add Spec Kit plan artifacts for agent-context full opt-in

Phase 0/1 of the SDD plan workflow: plan.md, research.md, data-model.md,
quickstart.md, and contracts/cli-behavior.md. Constitution Check is a
documented no-op (repo has no ratified constitution). Force-added despite
specs/ being gitignored; generated artifacts will be purged prior to merge.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: correct Constitution Check against ratified v1.0.0

Earlier draft wrongly treated the gate as a no-op; the fork's main is 16
commits behind upstream/main, which carries .specify/memory/constitution.md.
Re-evaluate the feature against Principles I-V (all PASS) and note that
Principle I mandates keeping context_file as a declared class attribute,
validating the R1 metadata decision.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: refresh plan artifacts against synced upstream/main

After syncing fork main to upstream and rebasing, re-scan the current
agent-context surface. Upstream generalized the single context_file into a
plural context_files concept with new resolver helpers
(_resolve_context_files, _resolve_context_file_values,
_format_context_file_values) and upsert/remove now loop over multiple
files. Update research.md, data-model.md, contracts, quickstart grep
guards, and the plan summary to cover the expanded removal scope.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: add Spec Kit tasks for agent-context full opt-in

Phase 2 of SDD: dependency-ordered tasks.md (30 tasks) organized by the
three user stories, with mandatory test tasks (Constitution Principle II)
and a foundational phase decoupling __CONTEXT_FILE__ resolution from the
extension config. Includes the extension self-seeding task (T015) and a
static guard test (T002) enforcing zero agent-context references in the CLI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat!: remove agent-context lifecycle from the Specify CLI

Make the agent-context extension a full opt-in. The CLI no longer
installs the extension during init, writes agent-context-config.yml,
or creates/updates/removes the managed Spec Kit section in agent
context files. Context-section upsert/remove, marker resolution,
extension-enabled gating, the config helpers, and the obsolete inline
deprecation warning are all removed. Integration context_file stays as
inert metadata; __CONTEXT_FILE__ now resolves from registry metadata.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(agent-context): self-seed context file from the active integration

When agent-context-config.yml has no context_file/context_files, the
bundled bash and PowerShell update scripts now resolve the context file
from the active integration in .specify/init-options.json via the
integration registry, so the extension no longer depends on the CLI
writing its config.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test+docs: update suite and docs for agent-context opt-in

Update integration/extension tests to expect no agent-context install,
config, or context-section writes during init. Add a static guard test
(test_agent_context_cli_free.py) asserting the CLI source is free of
agent-context lifecycle symbols, plus backward-compatibility tests for
legacy projects. Refresh AGENTS.md, the extension README, and add a
CHANGELOG entry describing the opt-in behavior change.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(agent-context): warn on self-seed failure, correct docs, speed up guard test

Address PR review feedback:
- Self-seed scripts (bash + PowerShell) now emit an actionable warning when
  an active integration is configured but specify_cli cannot be imported by
  the chosen Python (e.g. pipx installs), or when the integration declares no
  context file, instead of silently falling through to 'nothing to do'.
- Correct the extension README disable note: command rendering never reads the
  extension config; __CONTEXT_FILE__ is always substituted from integration
  metadata, so a stale context_files value cannot affect rendering.
- Cache CLI source reads in the static guard test via a module-scoped fixture
  so the directory walk happens once instead of once per forbidden symbol.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(agent-context): ship self-owned per-agent context-file defaults

The extension now bundles agent-context-defaults.json (key→context_file
map) and self-seeds from it, dropping any dependency on the Specify CLI
registry. Both the bash and PowerShell update scripts read the bundled
JSON map keyed by the active integration from init-options.json.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat!: remove all agent-context state from the Specify CLI

Strip every context_file reference from the CLI: the field on all 35
integration classes, the IntegrationBase plumbing (process_template
param/step, _context_file_display, docstrings), the __CONTEXT_FILE__
resolution in agents.py, the legacy context_file/context_markers
popping in _helpers.py, and the context_file template in
integration_scaffold.py. Also drop the Agent context update step and
__CONTEXT_FILE__ placeholder from templates/commands/plan.md.

The agent-context extension now solely owns all context-file knowledge,
including the per-agent default mapping.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: drop context_file coverage and guard against CLI reintroduction

Remove CONTEXT_FILE attrs and context_file assertions across the base
mixins, all 35 per-integration test files, shared integration tests, and
conftest stubs. Rewrite the base-mixin context tests to assert no managed
section is written and no __CONTEXT_FILE__ placeholder survives. Extend
the CLI-free static guard to forbid context_file, __CONTEXT_FILE__, and
_context_file_display in src/specify_cli, and have the extension tests
copy the bundled defaults JSON so self-seed runs without the CLI.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: reflect full removal of agent-context state from the CLI

Update AGENTS.md (integration examples, required-fields table, context
behavior section, pitfalls), CHANGELOG, and the SDD spec artifacts
(FR-007, SC-002, data-model) to state that the CLI carries no
context_file and the extension fully owns the per-agent default mapping
via agent-context-defaults.json.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: align SDD artifacts with full context_file removal

Update research.md (R1, R2, R4, summary table), contracts/cli-behavior.md
(C3, C5), tasks.md (Phase 2, T026, notes), plan.md (Principle I, source
map), and checklists/requirements.md so the spec artifacts reflect the
implemented decision: the CLI carries no context_file attribute or
__CONTEXT_FILE__ resolution, and the per-agent defaults map lives in the
extension. Resolves PR review #4548130110.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: scrub stale context-file mentions from CLI docstrings

Update the multi_install_safe docstring (drop the removed "context file"
invariant), the RovoDev setup docstring (no longer upserts a context
section), the Copilot module docstring (drop the context-file line), and
tighten the _update_init_options_for_integration note. Pure docstring
changes — no behavioral impact. Resolves PR review #4548237085.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test+docs: harden agent-context test helper and fix stale docs

- base.py: document multi_install_safe as an optional subclass attribute
  in the IntegrationBase docstring.
- test_cli.py: clarify the init-options assertion is guarding against
  leftover legacy agent-context keys, not relocation.
- test_extension_agent_context.py: _install_agent_context_config now
  asserts the bundled agent-context-defaults.json exists and always
  copies it, so self-seeding tests fail loudly instead of silently
  skipping when the map is missing.
- test_integration_cursor_agent.py: drop Path/IntegrationManifest imports
  left unused after removing the context-section frontmatter tests.

Resolves PR review #4548293116.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: remove gitignored SDD artifacts from specs/

The specs/001-agent-context-full-optin/ artifacts were force-added for
dogfooding visibility, but specs/ is gitignored and these were always
intended to be purged before merge. Remove them so merging does not add
an intentionally-untracked directory to repo history.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* chore: keep CHANGELOG.md identical to upstream

CHANGELOG.md is auto-generated at release time, so the branch should not
carry a manual entry. Restore it to match upstream/main exactly.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: preserve Cursor .mdc frontmatter in agent-context updater scripts

The bundled agent-context updater scripts wrote the managed section as
plain text. For Cursor-style `.mdc` targets this dropped the required
`---\nalwaysApply: true\n---` frontmatter, reintroducing the rule-loading
bug originally fixed in #1699. Port the `_ensure_mdc_frontmatter` logic
into both the bash and PowerShell updaters: prepend frontmatter when
missing, repair `alwaysApply` when set to the wrong value, and leave
non-`.mdc` targets untouched. Add regression tests covering both shells.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: scope CLI-free guard to agent-context-specific symbols

Drop the bare "context_file" substring from FORBIDDEN_SYMBOLS so the
guard no longer fails on unrelated future CLI fields named context_file.
The list still covers agent-context-specific identifiers (__CONTEXT_FILE__,
_context_file_display, _resolve_context_files, _resolve_context_file_values).

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: harden agent-context bash self-seed against malformed init JSON

Two robustness fixes in the embedded Python self-seed logic:
- Coerce the integration value from init-options.json to a string only when
  it is actually a string; otherwise treat it as unset so a corrupted
  dict/list value degrades to the existing nothing-to-do behavior instead of
  breaking the agents-map lookup.
- Normalize agent-context-defaults.json: only use 'agents' when both the JSON
  root and the 'agents' value are dicts, so a wrong-shaped (but valid) JSON
  falls back to the warning path instead of raising on .get.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: correct PowerShell hyphenated key lookup and regex replace count

- Self-seed now reads the defaults mapping via
  $defaults.agents.PSObject.Properties[$integrationKey].Value instead of
  member access ($defaults.agents.$integrationKey), which parsed hyphenated
  keys like 'cursor-agent'/'kiro-cli' as subtraction and failed to resolve.
- Replace the static [regex]::Replace(..., 1) call, whose trailing 1 was
  interpreted as RegexOptions.IgnoreCase rather than a replacement count, with
  an instance Regex whose Replace(input, replacement, 1) limits to the first
  match as intended.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make bash .mdc frontmatter guard case-insensitive

The bash updater only injected Cursor .mdc frontmatter when ctx_path ended
in lowercase '.mdc', so a mixed/upper-case extension (e.g. specify-rules.MDC)
was skipped and Cursor would not auto-load the rule file. Compare against the
casefolded path. The PowerShell variant already uses -match, which is
case-insensitive by default, so no change is needed there.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs: document separator-agnostic agent-context update invocation

The README hard-coded the dot-notation slash command
(/speckit.agent-context.update), which hyphen-separator agents like Forge and
Cline do not recognize. Document the canonical command ID plus both slash
invocations so users copy the form their agent accepts.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-29 15:27:26 -05:00

296 lines
12 KiB
Python

"""Tests for RovodevIntegration."""
from __future__ import annotations
import os
import pytest
import yaml
from click.testing import Result
from typer.testing import CliRunner
from specify_cli import app
from specify_cli.integrations import get_integration
from specify_cli.integrations.manifest import IntegrationManifest
def _run_init(project, *flags: str) -> Result:
"""Run ``specify init --here`` in *project* with the given extra flags.
Centralises the cwd-management boilerplate so individual tests just
declare the flags they care about.
"""
old_cwd = os.getcwd()
try:
os.chdir(project)
return CliRunner().invoke(
app,
["init", "--here", *flags, "--script", "sh", "--ignore-agent-tools"],
catch_exceptions=False,
)
finally:
os.chdir(old_cwd)
@pytest.fixture
def rovodev_init_project(tmp_path):
"""Run ``specify init --integration rovodev`` once and return the project root.
Shared across the slow init-inventory tests so we pay the full-CLI cost
only once instead of three times.
"""
project = tmp_path / "rovodev-init"
project.mkdir()
result = _run_init(project, "--integration", "rovodev")
assert result.exit_code == 0, result.output
return project
class TestRovodevIntegration:
"""Rovodev-specific tests (not inherited from SkillsIntegrationTests because
rovodev's setup() emits prompt wrappers + prompts.yml in addition to skills,
which violates the base mixin's pure-skills assumptions)."""
KEY = "rovodev"
# -- ACLI dispatch -----------------------------------------------------
def test_build_exec_args(self):
impl = get_integration(self.KEY)
args = impl.build_exec_args("/speckit.plan add OAuth")
assert args[0:3] == ["acli", "rovodev", "run"]
assert args[3] == "/speckit.plan add OAuth"
assert "--output-schema" in args
def test_build_exec_args_without_json(self):
impl = get_integration(self.KEY)
args = impl.build_exec_args("/speckit.plan add OAuth", output_json=False)
assert args == ["acli", "rovodev", "run", "/speckit.plan add OAuth"]
def test_build_exec_args_executable_env_override(self, monkeypatch):
"""SPECKIT_INTEGRATION_ROVODEV_EXECUTABLE overrides the binary path.
Lets operators pin a specific ``acli`` build or relocate the binary
without modifying the integration. Mirrors codex/devin/claude/etc.
"""
monkeypatch.setenv("SPECKIT_INTEGRATION_ROVODEV_EXECUTABLE", "/opt/atl/bin/acli")
impl = get_integration(self.KEY)
args = impl.build_exec_args("hello", output_json=False)
assert args == ["/opt/atl/bin/acli", "rovodev", "run", "hello"]
def test_build_exec_args_executable_env_blank_falls_back(self, monkeypatch):
"""Whitespace/empty env override is treated as unset → default ``acli``."""
monkeypatch.setenv("SPECKIT_INTEGRATION_ROVODEV_EXECUTABLE", " ")
impl = get_integration(self.KEY)
args = impl.build_exec_args("hello", output_json=False)
assert args[0] == "acli"
def test_build_exec_args_extra_args_env_injection(self, monkeypatch):
"""SPECKIT_INTEGRATION_ROVODEV_EXTRA_ARGS injects extra CLI flags.
Useful for CI or non-interactive contexts that need to pass flags
the integration doesn't expose. Mirrors the contract on every other
CLI integration (claude, codex, devin, …).
"""
monkeypatch.setenv("SPECKIT_INTEGRATION_ROVODEV_EXTRA_ARGS", "--quiet --no-color")
impl = get_integration(self.KEY)
args = impl.build_exec_args("hello", output_json=False)
assert args == [
"acli", "rovodev", "run", "hello", "--quiet", "--no-color",
]
# -- Setup-level: prompt wrappers + prompts.yml ------------------------
def test_setup_creates_prompts_and_manifest(self, tmp_path):
impl = get_integration(self.KEY)
manifest = IntegrationManifest(self.KEY, tmp_path)
created = impl.setup(tmp_path, manifest)
prompts_manifest = tmp_path / ".rovodev" / "prompts.yml"
assert prompts_manifest in created
assert prompts_manifest.exists()
prompts_dir = tmp_path / ".rovodev" / "prompts"
skills_dir = tmp_path / ".rovodev" / "skills"
assert prompts_dir.is_dir()
assert skills_dir.is_dir()
templates = impl.list_command_templates()
prompt_files = sorted(prompts_dir.glob("speckit-*.prompt.md"))
skill_dirs = sorted(d for d in skills_dir.iterdir() if d.is_dir() and d.name.startswith("speckit-"))
assert len(prompt_files) == len(templates)
assert len(skill_dirs) == len(templates)
for skill_dir in skill_dirs:
assert (skill_dir / "SKILL.md").exists()
def test_prompts_manifest_entries_well_formed(self, tmp_path):
impl = get_integration(self.KEY)
manifest = IntegrationManifest(self.KEY, tmp_path)
impl.setup(tmp_path, manifest)
prompts_manifest = tmp_path / ".rovodev" / "prompts.yml"
data = yaml.safe_load(prompts_manifest.read_text(encoding="utf-8"))
assert list(data) == ["prompts"]
entries = data["prompts"]
assert entries
for entry in entries:
assert entry["name"].startswith("speckit-")
assert entry["description"]
content_file = tmp_path / ".rovodev" / entry["content_file"]
assert content_file.exists(), f"Missing prompt file {content_file}"
def test_prompt_wrapper_format(self, tmp_path):
"""Every prompt wrapper delegates to its paired skill via 'use skill ...'."""
impl = get_integration(self.KEY)
manifest = IntegrationManifest(self.KEY, tmp_path)
impl.setup(tmp_path, manifest)
prompts_dir = tmp_path / ".rovodev" / "prompts"
prompt_files = sorted(prompts_dir.glob("speckit-*.prompt.md"))
assert prompt_files
for prompt_file in prompt_files:
skill_name = prompt_file.name.removesuffix(".prompt.md")
content = prompt_file.read_text(encoding="utf-8")
assert content == f"use skill {skill_name} $ARGUMENTS\n", (
f"{prompt_file} has unexpected wrapper format"
)
def test_prompts_manifest_merge_preserves_user_entries(self, tmp_path):
impl = get_integration(self.KEY)
manifest = IntegrationManifest(self.KEY, tmp_path)
prompts_manifest = tmp_path / ".rovodev" / "prompts.yml"
prompts_manifest.parent.mkdir(parents=True, exist_ok=True)
user_entry = {
"name": "my-custom-prompt",
"description": "User-added prompt",
"content_file": "prompts/my-custom-prompt.md",
}
prompts_manifest.write_text(
yaml.safe_dump({"prompts": [user_entry]}, sort_keys=False),
encoding="utf-8",
)
impl.setup(tmp_path, manifest)
data = yaml.safe_load(prompts_manifest.read_text(encoding="utf-8"))
names = {entry.get("name") for entry in data.get("prompts", [])}
assert "my-custom-prompt" in names
assert "speckit-plan" in names
def test_modified_prompts_yml_survives_uninstall(self, tmp_path):
impl = get_integration(self.KEY)
manifest = IntegrationManifest(self.KEY, tmp_path)
impl.install(tmp_path, manifest)
manifest.save()
modified = tmp_path / ".rovodev" / "prompts.yml"
modified.write_text("user modified this", encoding="utf-8")
_, skipped = impl.uninstall(tmp_path, manifest)
assert modified.exists()
assert modified in skipped
# -- Full-CLI init: skills + prompts integration with extensions -------
def test_init_inventory(self, rovodev_init_project):
"""Rovodev + extensions produce the expected skill / prompt set.
Contract:
- Rovodev.setup() emits one SKILL.md + one .prompt.md per core template.
- Extensions install additional SKILL.md directories with NO prompt wrapper.
"""
project = rovodev_init_project
impl = get_integration(self.KEY)
core_skill_names = {
f"speckit-{t.stem.replace('.', '-')}"
for t in impl.list_command_templates()
}
prompt_files = sorted((project / ".rovodev" / "prompts").glob("speckit-*.prompt.md"))
prompt_stems = {p.name.removesuffix(".prompt.md") for p in prompt_files}
skills_dir = project / ".rovodev" / "skills"
skill_names = {
d.name for d in skills_dir.iterdir()
if d.is_dir() and d.name.startswith("speckit-")
}
# Prompts: exactly the core template set.
assert prompt_stems == core_skill_names
# Skills: exactly the core template set (no extension auto-install).
assert skill_names == core_skill_names
# prompts.yml mirrors the prompt files exactly.
prompts_manifest = project / ".rovodev" / "prompts.yml"
data = yaml.safe_load(prompts_manifest.read_text(encoding="utf-8"))
assert {e["name"] for e in data["prompts"]} == core_skill_names
def test_init_skill_files_well_formed(self, rovodev_init_project):
"""Every speckit-* SKILL.md from full init has valid frontmatter +
processed body, including extension-installed skills."""
project = rovodev_init_project
skills_dir = project / ".rovodev" / "skills"
skill_dirs = sorted(
d for d in skills_dir.iterdir()
if d.is_dir() and d.name.startswith("speckit-")
)
assert skill_dirs
for skill_dir in skill_dirs:
skill_file = skill_dir / "SKILL.md"
assert skill_file.exists(), f"Missing {skill_file}"
content = skill_file.read_text(encoding="utf-8")
# Frontmatter delimited by leading '---\n' ... '\n---\n'
assert content.startswith("---\n"), f"{skill_file} missing frontmatter"
fm_end = content.find("\n---\n", 4)
assert fm_end != -1, f"{skill_file} has unterminated frontmatter"
fm = yaml.safe_load(content[4:fm_end])
body = content[fm_end + len("\n---\n"):]
assert fm.get("name") == skill_dir.name
assert fm.get("description")
assert body.strip(), f"{skill_file} has empty body"
for placeholder in ("{SCRIPT}", "__AGENT__", "__CONTEXT_FILE__", "__SPECKIT_COMMAND_"):
assert placeholder not in body, (
f"{skill_file} body contains unprocessed placeholder {placeholder!r}"
)
# Skills agents must use hyphen-style refs in body.
assert "/speckit." not in body, (
f"{skill_file} body contains dot-notation /speckit. reference"
)
# -- Full-CLI init: integration metadata -------------------------------
def test_init_writes_integration_manifest_and_options(self, rovodev_init_project):
"""Full init must produce an integration manifest and well-formed
init-options.json — used by extensions, presets, and uninstall."""
import json
project = rovodev_init_project
manifest_path = project / ".specify" / "integrations" / "rovodev.manifest.json"
speckit_manifest = project / ".specify" / "integrations" / "speckit.manifest.json"
assert manifest_path.exists(), "rovodev integration manifest missing"
assert speckit_manifest.exists(), "speckit shared manifest missing"
init_options = json.loads(
(project / ".specify" / "init-options.json").read_text(encoding="utf-8")
)
assert init_options["integration"] == self.KEY
assert init_options["ai"] == self.KEY
# Rovodev is a SkillsIntegration, so ai_skills is auto-set.
assert init_options.get("ai_skills") is True
assert init_options.get("script") == "sh"
def test_integration_flag_creates_expected_files(self, tmp_path):
"""``--integration rovodev`` should create all expected rovodev files."""
project = tmp_path / "rovodev-int"
project.mkdir()
result = _run_init(project, "--integration", "rovodev")
assert result.exit_code == 0, result.output
assert (project / ".rovodev" / "skills" / "speckit-plan" / "SKILL.md").exists()
assert (project / ".rovodev" / "prompts.yml").exists()
assert (project / ".specify" / "integrations" / "rovodev.manifest.json").exists()