mirror of
https://github.com/github/spec-kit.git
synced 2026-07-03 12:28:06 +08:00
fix(workflow): support integration: auto to follow project's initialized AI (#2421)
* fix(workflow): support integration: auto to follow project's initialized AI Closes #2406 (squashed) * fix(workflow): combine JSONDecodeError and UnicodeDecodeError handling Address Copilot feedback: UnicodeDecodeError can be raised by both read_text() and json.loads(), so combining the handlers ensures both cases produce a consistent, clear error message. * fix(workflows): honor integration_state schema guard and modern state in 'integration: auto' Three Copilot follow-ups on PR #2421: 1. engine.py:799 — `_load_project_integration` was bypassing the same schema guard `_read_integration_json` enforces. It now reads the schema field directly, returns None on a future schema (so the workflow falls back to the literal 'auto' default rather than guessing), and routes through `normalize_integration_state` / `default_integration_key` so modern installs that record `default_integration` / `installed_integrations` (without the legacy top-level `integration` field) resolve correctly. 2. test_workflows.py — added two regression cases: - `integration: auto` resolves a modern normalized state file - `integration: auto` falls back when the state file declares a newer `integration_state_schema` than this CLI supports 3. test_cli.py — added a CLI-level regression for the `UnicodeDecodeError` branch in `_read_integration_json` to match the existing malformed-JSON coverage. * refactor(integration): extract shared try_read_integration_json helper Address Copilot review on PR #2421: Both `_read_integration_json` (CLI) and `_load_project_integration` (workflow engine) were parsing `.specify/integration.json` independently, duplicating the schema guard and risking drift between the two readers. Extract the parse + schema validation into a single low-level helper `try_read_integration_json` in `integration_state.py` that returns either the normalized state or a structured `IntegrationReadError`. Both callers now delegate to this helper: - CLI keeps its loud-fail UX: each error kind ("decode", "os", "not_object", "schema_too_new") is translated into the existing console message + typer.Exit(1). - Engine keeps its silent fallback: any error simply returns None so `integration: auto` falls back to the workflow's literal default. This eliminates the divergence Copilot flagged without changing observable behavior for either caller. * fix(integration): distinguish missing file from non-regular path Address Copilot review on PR #2421: `try_read_integration_json` was collapsing two distinct cases into a single `(None, None)` return: 1. `.specify/integration.json` truly missing — silent fallback is correct. 2. Path exists but is a directory, socket, or other non-regular file — this is a misconfiguration the CLI should surface loudly. Split the check: `exists()` falsey returns `(None, None)`; existing-but- not-a-regular-file returns `(None, IntegrationReadError(kind="os", ...))` so the CLI's loud-fail path produces an actionable error while the engine still treats it as a fallback to the workflow's literal default. * docs(workflow): clarify version pin, advisory integrations list, enum exemption - workflow.yml: fix comment that said 0.8.3 was first release with auto resolution; the pin is >=0.8.5 so the comment now matches the pin. - workflow.yml: clarify that requires.integrations.any is an advisory, non-exhaustive compatibility hint, not a closed set. - engine.py: clarify that the auto-sentinel exemption only skips enum membership; declared type is still enforced through _coerce_input. * fix(workflow): resolve auto sentinel for provided values; report stat errors Two Copilot findings fixed: 1. _resolve_inputs only resolved the ``integration: auto`` sentinel when it came from the input default. A caller explicitly providing ``{"integration": "auto"}`` (which the workflow prompt advertises as a valid value) bypassed _resolve_default and the literal "auto" leaked to dispatch. Provided values now go through the same resolution path as defaults, and the enum-membership exemption applies in both cases. Regression test added. 2. try_read_integration_json used Path.exists() / Path.is_file() as a pre-check. Both return False on some OSErrors (e.g. permission errors during stat), which silently treated an unreadable-but-present file as missing — the engine fell back without warning and the CLI failed to surface the loud error. The pre-check is gone: read_text() is attempted directly, FileNotFoundError means missing (silent fallback), IsADirectoryError and other OSErrors become loud IntegrationReadError. * fix(workflow): enforce declared type for string inputs, reject bool-as-number Two Copilot findings fixed: 1. _coerce_input previously coerced/validated only ``number`` and ``boolean`` types, so ``type: string`` silently accepted any Python value (numbers, lists, dicts). A YAML authoring mistake like ``type: string`` + ``default: 5`` slipped through. Strings are now required to actually be strings; non-strings raise ValueError, which surfaces as an ``invalid default`` error from validate_workflow. 2. ``type: number`` accepted ``default: true`` because ``bool`` is a subclass of ``int`` (``float(True) == 1.0``). Bools are now rejected explicitly in the number path so the YAML mistake fails fast. The boolean path is also tightened to reject non-bool / non-string values for symmetry. Comment on the auto-sentinel enum exemption updated to reflect the stronger guarantee. Regression tests added for both rejections. * fix(cli): drop unused normalize_integration_state import to satisfy ruff CI's `uvx ruff check src/` flagged this as F401: the symbol was imported under a private alias but never referenced. Tests stay green after removal.
This commit is contained in:
Quratulain-bilal
@@ -1495,6 +1495,394 @@ steps:
|
||||
with pytest.raises(ValueError, match="Required input"):
|
||||
engine.execute(definition, {})
|
||||
|
||||
def test_integration_auto_default_uses_project_integration(self, project_dir):
|
||||
"""`integration: auto` should resolve to .specify/integration.json's integration."""
|
||||
from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition
|
||||
|
||||
specify_dir = project_dir / ".specify"
|
||||
specify_dir.mkdir(parents=True, exist_ok=True)
|
||||
(specify_dir / "integration.json").write_text(
|
||||
json.dumps({"integration": "opencode", "version": "0.7.4"}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "auto-default"
|
||||
name: "Auto Default"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
integration:
|
||||
type: string
|
||||
default: "auto"
|
||||
""")
|
||||
engine = WorkflowEngine(project_dir)
|
||||
resolved = engine._resolve_inputs(definition, {})
|
||||
assert resolved["integration"] == "opencode"
|
||||
|
||||
def test_integration_auto_default_falls_back_when_no_integration_json(self, project_dir):
|
||||
"""`integration: auto` should keep the literal "auto" when project state is missing.
|
||||
|
||||
The engine itself must not invent an integration when
|
||||
``.specify/integration.json`` is absent; any later validation or
|
||||
command resolution will handle an unresolved ``"auto"`` value.
|
||||
"""
|
||||
from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "auto-fallback"
|
||||
name: "Auto Fallback"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
integration:
|
||||
type: string
|
||||
default: "auto"
|
||||
""")
|
||||
engine = WorkflowEngine(project_dir)
|
||||
resolved = engine._resolve_inputs(definition, {})
|
||||
assert resolved["integration"] == "auto"
|
||||
|
||||
def test_integration_explicit_input_overrides_auto(self, project_dir):
|
||||
"""An explicit --input integration=X must win over `auto` even when integration.json exists."""
|
||||
from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition
|
||||
|
||||
specify_dir = project_dir / ".specify"
|
||||
specify_dir.mkdir(parents=True, exist_ok=True)
|
||||
(specify_dir / "integration.json").write_text(
|
||||
json.dumps({"integration": "opencode"}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "explicit-wins"
|
||||
name: "Explicit Wins"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
integration:
|
||||
type: string
|
||||
default: "auto"
|
||||
""")
|
||||
engine = WorkflowEngine(project_dir)
|
||||
resolved = engine._resolve_inputs(definition, {"integration": "claude"})
|
||||
assert resolved["integration"] == "claude"
|
||||
|
||||
def test_integration_explicit_auto_resolves_like_default(self, project_dir):
|
||||
"""Passing ``integration=auto`` explicitly must resolve the sentinel,
|
||||
not pass it through as a literal — the workflow prompt advertises
|
||||
``auto`` as a valid value, so the dispatch path must never see it.
|
||||
"""
|
||||
from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition
|
||||
|
||||
specify_dir = project_dir / ".specify"
|
||||
specify_dir.mkdir(parents=True, exist_ok=True)
|
||||
(specify_dir / "integration.json").write_text(
|
||||
json.dumps({"integration": "opencode"}),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "explicit-auto"
|
||||
name: "Explicit Auto"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
integration:
|
||||
type: string
|
||||
default: "auto"
|
||||
""")
|
||||
engine = WorkflowEngine(project_dir)
|
||||
resolved = engine._resolve_inputs(definition, {"integration": "auto"})
|
||||
assert resolved["integration"] == "opencode"
|
||||
|
||||
def test_integration_auto_ignores_malformed_integration_json(self, project_dir):
|
||||
"""A malformed integration.json must not crash — fall back to the literal default."""
|
||||
from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition
|
||||
|
||||
specify_dir = project_dir / ".specify"
|
||||
specify_dir.mkdir(parents=True, exist_ok=True)
|
||||
(specify_dir / "integration.json").write_text("{not json", encoding="utf-8")
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "auto-malformed"
|
||||
name: "Auto Malformed"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
integration:
|
||||
type: string
|
||||
default: "auto"
|
||||
""")
|
||||
engine = WorkflowEngine(project_dir)
|
||||
resolved = engine._resolve_inputs(definition, {})
|
||||
assert resolved["integration"] == "auto"
|
||||
|
||||
def test_integration_auto_ignores_non_utf8_integration_json(self, project_dir):
|
||||
"""A non-UTF8 integration.json must not crash — fall back to the literal default."""
|
||||
from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition
|
||||
|
||||
specify_dir = project_dir / ".specify"
|
||||
specify_dir.mkdir(parents=True, exist_ok=True)
|
||||
# 0xFF is invalid as the leading byte of a UTF-8 sequence, so
|
||||
# ``Path.read_text(encoding="utf-8")`` raises UnicodeDecodeError.
|
||||
(specify_dir / "integration.json").write_bytes(b"\xff\xfe\x00\x00")
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "auto-non-utf8"
|
||||
name: "Auto Non UTF-8"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
integration:
|
||||
type: string
|
||||
default: "auto"
|
||||
""")
|
||||
engine = WorkflowEngine(project_dir)
|
||||
resolved = engine._resolve_inputs(definition, {})
|
||||
assert resolved["integration"] == "auto"
|
||||
|
||||
def test_integration_auto_resolves_modern_normalized_state(self, project_dir):
|
||||
"""`integration: auto` must resolve modern state files that record
|
||||
``default_integration`` / ``installed_integrations`` and omit the
|
||||
legacy ``integration`` field."""
|
||||
from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition
|
||||
|
||||
specify_dir = project_dir / ".specify"
|
||||
specify_dir.mkdir(parents=True, exist_ok=True)
|
||||
(specify_dir / "integration.json").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"version": "0.8.3",
|
||||
"integration_state_schema": 1,
|
||||
"default_integration": "claude",
|
||||
"installed_integrations": ["claude", "copilot"],
|
||||
"integration_settings": {},
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "auto-modern"
|
||||
name: "Auto Modern"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
integration:
|
||||
type: string
|
||||
default: "auto"
|
||||
""")
|
||||
engine = WorkflowEngine(project_dir)
|
||||
resolved = engine._resolve_inputs(definition, {})
|
||||
assert resolved["integration"] == "claude"
|
||||
|
||||
def test_integration_auto_rejects_future_state_schema(self, project_dir):
|
||||
"""`integration: auto` must not silently use a state file written by a newer
|
||||
CLI (``integration_state_schema`` greater than the current supported value);
|
||||
the resolver falls back to the literal default rather than guessing."""
|
||||
from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition
|
||||
from specify_cli.integration_state import INTEGRATION_STATE_SCHEMA
|
||||
|
||||
specify_dir = project_dir / ".specify"
|
||||
specify_dir.mkdir(parents=True, exist_ok=True)
|
||||
(specify_dir / "integration.json").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"version": "99.0.0",
|
||||
"integration_state_schema": INTEGRATION_STATE_SCHEMA + 1,
|
||||
"default_integration": "claude",
|
||||
"installed_integrations": ["claude"],
|
||||
"integration_settings": {},
|
||||
}
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "auto-future-schema"
|
||||
name: "Auto Future Schema"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
integration:
|
||||
type: string
|
||||
default: "auto"
|
||||
""")
|
||||
engine = WorkflowEngine(project_dir)
|
||||
resolved = engine._resolve_inputs(definition, {})
|
||||
assert resolved["integration"] == "auto"
|
||||
|
||||
def test_default_value_is_validated_against_enum(self, project_dir):
|
||||
"""Defaults must run through the same coercion/enum check as provided inputs."""
|
||||
from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "default-enum"
|
||||
name: "Default Enum"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
scope:
|
||||
type: string
|
||||
default: "not-in-enum"
|
||||
enum: ["full", "backend-only", "frontend-only"]
|
||||
""")
|
||||
engine = WorkflowEngine(project_dir)
|
||||
with pytest.raises(ValueError, match="not in allowed values"):
|
||||
engine._resolve_inputs(definition, {})
|
||||
|
||||
def test_default_value_is_coerced_to_declared_type(self, project_dir):
|
||||
"""A numeric default declared as a string should still be coerced like a provided input."""
|
||||
from specify_cli.workflows.engine import WorkflowEngine, WorkflowDefinition
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "default-coerce"
|
||||
name: "Default Coerce"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
retries:
|
||||
type: number
|
||||
default: "3"
|
||||
""")
|
||||
engine = WorkflowEngine(project_dir)
|
||||
resolved = engine._resolve_inputs(definition, {})
|
||||
assert resolved["retries"] == 3
|
||||
assert isinstance(resolved["retries"], int)
|
||||
|
||||
def test_validate_workflow_rejects_invalid_default(self):
|
||||
"""Authoring-time validation should reject defaults that violate enum."""
|
||||
from specify_cli.workflows.engine import WorkflowDefinition, validate_workflow
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "bad-default"
|
||||
name: "Bad Default"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
scope:
|
||||
type: string
|
||||
default: "not-in-enum"
|
||||
enum: ["full", "backend-only", "frontend-only"]
|
||||
steps:
|
||||
- id: noop
|
||||
type: gate
|
||||
message: "noop"
|
||||
options: [approve]
|
||||
""")
|
||||
errors = validate_workflow(definition)
|
||||
assert any("invalid default" in e for e in errors), errors
|
||||
|
||||
def test_validate_workflow_exempts_integration_auto_sentinel(self):
|
||||
"""``integration: auto`` is a runtime-resolved sentinel and must not fail validation."""
|
||||
from specify_cli.workflows.engine import WorkflowDefinition, validate_workflow
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "auto-ok"
|
||||
name: "Auto OK"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
integration:
|
||||
type: string
|
||||
default: "auto"
|
||||
enum: ["copilot", "claude", "gemini"]
|
||||
steps:
|
||||
- id: noop
|
||||
type: gate
|
||||
message: "noop"
|
||||
options: [approve]
|
||||
""")
|
||||
errors = validate_workflow(definition)
|
||||
assert not any("invalid default" in e for e in errors), errors
|
||||
|
||||
def test_validate_workflow_still_checks_type_for_auto_sentinel(self):
|
||||
"""The ``auto`` exemption only skips enum-membership; declared type is still enforced."""
|
||||
from specify_cli.workflows.engine import WorkflowDefinition, validate_workflow
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "auto-bad-type"
|
||||
name: "Auto Bad Type"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
integration:
|
||||
type: number
|
||||
default: "auto"
|
||||
steps:
|
||||
- id: noop
|
||||
type: gate
|
||||
message: "noop"
|
||||
options: [approve]
|
||||
""")
|
||||
errors = validate_workflow(definition)
|
||||
assert any("invalid default" in e for e in errors), errors
|
||||
|
||||
def test_validate_workflow_rejects_bool_default_for_number_type(self):
|
||||
"""``type: number`` paired with a bool default must fail — bool is a
|
||||
subclass of int so ``float(True)`` would otherwise silently coerce
|
||||
``true`` to ``1``.
|
||||
"""
|
||||
from specify_cli.workflows.engine import WorkflowDefinition, validate_workflow
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "bool-as-number"
|
||||
name: "Bool As Number"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
count:
|
||||
type: number
|
||||
default: true
|
||||
steps:
|
||||
- id: noop
|
||||
type: gate
|
||||
message: "noop"
|
||||
options: [approve]
|
||||
""")
|
||||
errors = validate_workflow(definition)
|
||||
assert any("invalid default" in e for e in errors), errors
|
||||
|
||||
def test_validate_workflow_rejects_non_string_default_for_string_type(self):
|
||||
"""``type: string`` must require an actual string — a numeric YAML
|
||||
default like ``5`` would otherwise slip through unvalidated.
|
||||
"""
|
||||
from specify_cli.workflows.engine import WorkflowDefinition, validate_workflow
|
||||
|
||||
definition = WorkflowDefinition.from_string("""
|
||||
schema_version: "1.0"
|
||||
workflow:
|
||||
id: "number-as-string"
|
||||
name: "Number As String"
|
||||
version: "1.0.0"
|
||||
inputs:
|
||||
label:
|
||||
type: string
|
||||
default: 5
|
||||
steps:
|
||||
- id: noop
|
||||
type: gate
|
||||
message: "noop"
|
||||
options: [approve]
|
||||
""")
|
||||
errors = validate_workflow(definition)
|
||||
assert any("invalid default" in e for e in errors), errors
|
||||
|
||||
|
||||
# ===== State Persistence Tests =====
|
||||
|
||||
|
||||
Reference in New Issue
Block a user