Files
github-spec-kit/tests/integrations/test_integration_devin.py
vishal-gandhi 237e918f11 feat(integrations): add Devin for Terminal skills-based integration (#2364)
* feat(integrations): add Devin for Terminal skills-based integration

- Register DevinIntegration as a SkillsIntegration with .devin/skills/ layout
- Add catalog entry, docs row, and supported-agents listing
- Display /speckit-<command> hyphen syntax in init "Next Steps" panel
  (matches Claude/Cursor/Copilot skills mode, since Devin invokes skills
  by directory name)

Closes #2346

* fix(devin): implement -p non-interactive dispatch; clarify skills comment

Addresses Copilot review on PR #2364:

- Override build_exec_args() in DevinIntegration to emit
  'devin -p <prompt> [--model X]' for non-interactive text dispatch
  (verified Devin CLI supports -p / --print). Returns None when
  output_json=True since Devin has no structured-output flag, so
  CommandStep workflows that require JSON cleanly raise
  NotImplementedError instead of crashing on an unknown CLI flag.
  requires_cli=True is retained for tool detection.

- Extend the skills-integrations enumeration comment in
  specify_cli/__init__.py to include copilot and devin so the
  comment matches the code below it.

* fix(devin): always return exec args; document plain-text stdout

Addresses third Copilot review comment on PR #2364.

Returning None from build_exec_args() when output_json=True
incorrectly used the codebase's IDE-only sentinel: workflow
CommandStep checks 'impl.build_exec_args("test") is None' to
detect non-dispatchable integrations (test_workflows.py exercises
this with WindsurfIntegration). The previous implementation made
Devin appear non-dispatchable to all command steps even though it
runs fine via 'devin -p'.

Always return the args list. When output_json is requested, Devin
is still dispatched and returns plain-text stdout instead of
structured JSON; the docstring documents this explicitly.

* docs(devin): include claude in skills-integrations enumeration comment

Addresses Copilot review on PR #2364: the comment listing skills
integrations omitted Claude, which is also a SkillsIntegration
subclass. Updated to keep the comment accurate for future readers.

* test(devin): add build_exec_args regression tests; bump catalog updated_at

Addresses Copilot review on PR #2364, per @mnriem's request to
'address the Copilot feedback, especially the testing ask':

- tests/integrations/test_integration_devin.py: add TestDevinBuildExecArgs
  with three regression assertions:
    * build_exec_args returns args (not the None IDE-only sentinel)
    * --output-format is never emitted, regardless of output_json
    * --model flag is passed through correctly
- integrations/catalog.json: bump top-level updated_at to reflect the
  Devin entry addition so downstream catalog consumers can detect the
  change reliably.
2026-04-29 16:22:06 -05:00

75 lines
2.9 KiB
Python

"""Tests for DevinIntegration."""
from .test_integration_base_skills import SkillsIntegrationTests
class TestDevinIntegration(SkillsIntegrationTests):
KEY = "devin"
FOLDER = ".devin/"
COMMANDS_SUBDIR = "skills"
REGISTRAR_DIR = ".devin/skills"
CONTEXT_FILE = "AGENTS.md"
class TestDevinBuildExecArgs:
"""Regression tests for DevinIntegration.build_exec_args.
Devin's CLI has no --output-format flag, so build_exec_args must
omit it regardless of the output_json argument. The integration
must also remain dispatchable (must not return None, which is the
codebase's IDE-only sentinel checked by CommandStep).
"""
def test_returns_args_not_none_for_dispatch(self):
"""Devin is CLI-dispatchable; build_exec_args must not return None."""
from specify_cli.integrations.devin import DevinIntegration
impl = DevinIntegration()
args = impl.build_exec_args("test prompt")
assert args is not None, (
"DevinIntegration.build_exec_args must not return None. "
"None is the codebase sentinel for IDE-only integrations "
"(see WindsurfIntegration); Devin is dispatchable via 'devin -p'."
)
assert args[:3] == ["devin", "-p", "test prompt"]
def test_output_json_does_not_emit_output_format_flag(self):
"""Devin has no --output-format flag; output_json=True must not add it."""
from specify_cli.integrations.devin import DevinIntegration
impl = DevinIntegration()
args_json = impl.build_exec_args("hello", output_json=True)
args_text = impl.build_exec_args("hello", output_json=False)
assert "--output-format" not in args_json
assert "json" not in args_json[3:]
# The two should be identical: output_json is documented as having
# no effect on the command line for Devin (plain-text stdout).
assert args_json == args_text
def test_model_flag_passed_through(self):
"""--model is supported and should appear when provided."""
from specify_cli.integrations.devin import DevinIntegration
impl = DevinIntegration()
args = impl.build_exec_args("hi", model="claude-sonnet-4")
assert args == ["devin", "-p", "hi", "--model", "claude-sonnet-4"]
class TestDevinAutoPromote:
"""--ai devin auto-promotes to integration path."""
def test_ai_devin_without_ai_skills_auto_promotes(self, tmp_path):
"""--ai devin should work the same as --integration devin."""
from typer.testing import CliRunner
from specify_cli import app
runner = CliRunner()
target = tmp_path / "test-proj"
result = runner.invoke(
app,
["init", str(target), "--ai", "devin", "--no-git", "--ignore-agent-tools", "--script", "sh"],
)
assert result.exit_code == 0, f"init --ai devin failed: {result.output}"
assert (target / ".devin" / "skills" / "speckit-plan" / "SKILL.md").exists()