Files
github-spec-kit/src/specify_cli/workflows/__init__.py
Copilot 00bff788c9 Add init workflow step to bootstrap projects like specify init (#2838)
* Initial plan

* Add init workflow step to bootstrap projects like `specify init`

* Address review: simplify stderr capture and extract VALID_SCRIPT_TYPES

* Address review: fail fast on non-empty dir, stdout fallback, README force fix

* Populate exit_code/stdout/stderr in non-empty-dir fast-fail

* fix: address three unresolved review comments in InitStep

- Use `with os.scandir(...)` context manager so the iterator is always
  closed even when `any()` short-circuits, preventing file-descriptor
  leaks in long-running workflow runs.
- Guard `os.chdir(prev_cwd)` in the `finally` block with a try/except
  so an `OSError` (e.g. directory deleted) doesn't bypass returning
  the captured `StepResult`.
- Reject non-string `script` values in `validate()` with a clear error
  message, rather than silently passing them through to become
  `--script True` at runtime.

* Potential fix for pull request finding 'Empty except'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* fix: remove no_git and branch_numbering options removed upstream

The --no-git and --branch-numbering flags were removed from `specify init`
on main. Update InitStep to drop these unsupported config fields and fix
tests accordingly.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: address review — integration defaults, integration_options, engine-owned dirs

- Apply DEFAULT_INIT_INTEGRATION fallback when neither step config nor
  workflow context provides an integration, so output.integration always
  reflects the actual integration used.
- Add integration_options config field to support --integration-options
  passthrough (required for generic integration and --skills mode).
- Exclude .specify/ from the non-empty directory fast-fail check so that
  here: true works when the engine has already created its run-state
  directory before steps execute.
- Note: mix_stderr=False is not needed — Click 8.2+ captures stderr
  separately by default and the existing try/except handles access.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: implicitly add --force when only engine-owned dirs exist

When the workflow engine creates .specify/workflows/runs/ before steps
execute, the directory is technically non-empty. Previously, specify init
would prompt for confirmation (hanging in unattended mode) unless the
user explicitly set force: true. Now the step detects that only
engine-owned directories (.specify/) are present and implicitly adds
--force so init proceeds without user interaction.

Also fixes the test to exercise the implicit-force path rather than
passing force: True explicitly (which bypassed the check entirely).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: derive VALID_SCRIPT_TYPES from shared constant, fail fast on OSError, include all resolved fields in output

- Derive VALID_SCRIPT_TYPES from SCRIPT_TYPE_CHOICES in _agent_config
  so the valid set cannot drift from the specify init CLI.
- Fail fast with a clear error when os.scandir() raises OSError (e.g.
  permission denied) instead of silently treating the directory as empty.
- Include preset, force, and ignore_agent_tools in all output dicts
  (both fast-fail and normal paths) for consistent interpolation and
  debugging downstream.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: populate stderr from stdout on older Click, fix force comment wording

- When Click does not expose result.stderr (older versions where stderr
  is mixed into stdout), use stdout as stderr on non-zero exit so
  workflows can consistently read steps.<id>.output.stderr for errors.
- Update README inline comment for force: wording to say 'when target
  directory already exists' rather than 'non-empty directory', matching
  the actual specify init behavior for the project: form.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: build argv flags before early returns, use any() for dir scan

- Move argv flag-building (--integration, --script, --preset,
  --ignore-agent-tools) before the non-empty-dir and OSError early
  returns so output['argv'] always reflects the complete command.
- --force is appended after the check since it may be set implicitly.
- Replace list comprehension with any() generator expression to
  short-circuit without allocating a full list of DirEntry objects.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: only treat .specify as engine-owned when it is a real directory

A file or symlink named .specify should not be excluded from the
non-empty check. Use entry.is_dir(follow_symlinks=False) to ensure
only an actual directory is considered engine-owned content.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: guard implicit force for engine dirs only, fix integration fallback order

- Only set implicit --force when engine-owned directories (.specify/)
  are actually present. A completely empty directory no longer gets
  --force added unnecessarily.
- Fix integration resolution precedence: resolve step config expression
  first, then fall back to workflow default (also resolved), then to
  DEFAULT_INIT_INTEGRATION. Previously, a step expression resolving to
  falsy would bypass the workflow default entirely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Manfred Riem <15701806+mnriem@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Manfred Riem <mnriem@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-17 11:46:51 -05:00

204 lines
7.7 KiB
Python

"""Workflow engine for multi-step, resumable automation workflows.
Provides:
- ``StepBase`` — abstract base every step type must implement.
- ``StepContext`` — execution context passed to each step.
- ``StepResult`` — return value from step execution.
- ``STEP_REGISTRY`` — maps ``type_key`` to ``StepBase`` subclass instances.
- ``WorkflowEngine`` — orchestrator that loads, validates, and executes
workflow YAML definitions.
- ``load_custom_steps`` — loads community-installed step types into STEP_REGISTRY.
"""
from __future__ import annotations
from pathlib import Path
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from .base import StepBase
# Maps step type_key → StepBase instance.
STEP_REGISTRY: dict[str, StepBase] = {}
def _register_step(step: StepBase) -> None:
"""Register a step type instance in the global registry.
Raises ``ValueError`` for falsy keys and ``KeyError`` for duplicates.
"""
key = step.type_key
if not key:
raise ValueError("Cannot register step type with an empty type_key.")
if key in STEP_REGISTRY:
raise KeyError(f"Step type with key {key!r} is already registered.")
STEP_REGISTRY[key] = step
def get_step_type(type_key: str) -> StepBase | None:
"""Return the step type for *type_key*, or ``None`` if not registered."""
return STEP_REGISTRY.get(type_key)
# -- Register built-in step types ----------------------------------------
def _register_builtin_steps() -> None:
"""Register all built-in step types."""
from .steps.command import CommandStep
from .steps.do_while import DoWhileStep
from .steps.fan_in import FanInStep
from .steps.fan_out import FanOutStep
from .steps.gate import GateStep
from .steps.if_then import IfThenStep
from .steps.init import InitStep
from .steps.prompt import PromptStep
from .steps.shell import ShellStep
from .steps.switch import SwitchStep
from .steps.while_loop import WhileStep
_register_step(CommandStep())
_register_step(DoWhileStep())
_register_step(FanInStep())
_register_step(FanOutStep())
_register_step(GateStep())
_register_step(IfThenStep())
_register_step(InitStep())
_register_step(PromptStep())
_register_step(ShellStep())
_register_step(SwitchStep())
_register_step(WhileStep())
_register_builtin_steps()
def load_custom_steps(project_root: Path) -> list[str]:
"""Load community-installed custom step types into STEP_REGISTRY.
Scans ``.specify/workflows/steps/`` for installed step packages.
Each valid package must contain ``step.yml`` (with a ``step.type_key``
field) and ``__init__.py`` (a ``StepBase`` subclass).
Returns a list of type_keys that were successfully loaded.
Silently skips packages that fail to import or validate.
"""
import hashlib as _hashlib
import importlib.util as _importlib_util
import re as _re
import sys as _sys
steps_dir = Path(project_root) / ".specify" / "workflows" / "steps"
# Defense-in-depth: refuse to execute step code from a symlinked
# parent directory under .specify/workflows/steps, which could redirect
# the import outside the project root and bypass the install-time
# symlink guard. Check symlinks *before* is_dir() since the latter
# follows symlinks and would stat an external target.
_current = Path(project_root)
for _part in (".specify", "workflows", "steps"):
_current = _current / _part
if _current.is_symlink():
return []
if not steps_dir.is_dir():
return []
loaded: list[str] = []
for step_dir in steps_dir.iterdir():
# Check symlinks before is_dir() since the latter follows symlinks
# and would stat an external target through a symlinked directory.
if step_dir.is_symlink():
continue
if not step_dir.is_dir():
continue
step_yml = step_dir / "step.yml"
init_py = step_dir / "__init__.py"
if step_yml.is_symlink() or init_py.is_symlink():
continue
if not step_yml.is_file() or not init_py.is_file():
continue
try:
import yaml as _yaml
meta = _yaml.safe_load(step_yml.read_text(encoding="utf-8")) or {}
step_meta = meta.get("step", {})
type_key = step_meta.get("type_key", "")
if not type_key:
continue
# Skip if already registered (e.g. built-in or previously loaded)
if type_key in STEP_REGISTRY:
continue
# Sanitize type_key so the synthetic module name is a valid identifier
# (e.g. "test-custom" → "_speckit_custom_step_test_custom_<hash>").
# The 8-char SHA-256 hash of the original type_key makes the name
# collision-resistant when different type_keys produce the same
# sanitized form (e.g. "a-b" and "a_b" both sanitize to "a_b" but
# have different hashes).
safe_key = _re.sub(r"[^A-Za-z0-9_]", "_", type_key)
key_hash = _hashlib.sha256(type_key.encode()).hexdigest()[:8]
module_name = f"_speckit_custom_step_{safe_key}_{key_hash}"
# Treat the step directory as a proper package so that relative
# imports inside the step (e.g. ``from .helpers import …``) work.
spec = _importlib_util.spec_from_file_location(
module_name,
init_py,
submodule_search_locations=[str(step_dir)],
)
if spec is None or spec.loader is None:
continue
module = _importlib_util.module_from_spec(spec)
module.__package__ = module_name
# Register before exec so relative imports resolve correctly.
_sys.modules[module_name] = module
registered = False
try:
spec.loader.exec_module(module) # type: ignore[union-attr]
# Find the StepBase subclass in the module
from .base import StepBase as _StepBase
step_class = None
for attr_name in dir(module):
attr = getattr(module, attr_name)
try:
if (
isinstance(attr, type)
and issubclass(attr, _StepBase)
and attr is not _StepBase
and getattr(attr, "type_key", "") == type_key
):
step_class = attr
break
except TypeError:
continue
if step_class is None:
continue
_register_step(step_class())
loaded.append(type_key)
registered = True
finally:
# If the step wasn't successfully registered (failed import,
# no matching StepBase subclass, or registration error), remove
# the synthetic module — and any submodules loaded via relative
# imports (e.g. ``from .helpers import …``) — from sys.modules so
# a broken/skipped step package leaves no lingering import state
# behind.
if not registered:
_sys.modules.pop(module_name, None)
submodule_prefix = module_name + "."
for _mod_key in [
k for k in _sys.modules if k.startswith(submodule_prefix)
]:
_sys.modules.pop(_mod_key, None)
except Exception: # noqa: BLE001
# Silently skip broken step packages at load time
continue
return loaded