mirror of
https://github.com/github/spec-kit.git
synced 2026-07-05 21:49:47 +08:00
* Initial plan * Add init workflow step to bootstrap projects like `specify init` * Address review: simplify stderr capture and extract VALID_SCRIPT_TYPES * Address review: fail fast on non-empty dir, stdout fallback, README force fix * Populate exit_code/stdout/stderr in non-empty-dir fast-fail * fix: address three unresolved review comments in InitStep - Use `with os.scandir(...)` context manager so the iterator is always closed even when `any()` short-circuits, preventing file-descriptor leaks in long-running workflow runs. - Guard `os.chdir(prev_cwd)` in the `finally` block with a try/except so an `OSError` (e.g. directory deleted) doesn't bypass returning the captured `StepResult`. - Reject non-string `script` values in `validate()` with a clear error message, rather than silently passing them through to become `--script True` at runtime. * Potential fix for pull request finding 'Empty except' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * fix: remove no_git and branch_numbering options removed upstream The --no-git and --branch-numbering flags were removed from `specify init` on main. Update InitStep to drop these unsupported config fields and fix tests accordingly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: address review — integration defaults, integration_options, engine-owned dirs - Apply DEFAULT_INIT_INTEGRATION fallback when neither step config nor workflow context provides an integration, so output.integration always reflects the actual integration used. - Add integration_options config field to support --integration-options passthrough (required for generic integration and --skills mode). - Exclude .specify/ from the non-empty directory fast-fail check so that here: true works when the engine has already created its run-state directory before steps execute. - Note: mix_stderr=False is not needed — Click 8.2+ captures stderr separately by default and the existing try/except handles access. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: implicitly add --force when only engine-owned dirs exist When the workflow engine creates .specify/workflows/runs/ before steps execute, the directory is technically non-empty. Previously, specify init would prompt for confirmation (hanging in unattended mode) unless the user explicitly set force: true. Now the step detects that only engine-owned directories (.specify/) are present and implicitly adds --force so init proceeds without user interaction. Also fixes the test to exercise the implicit-force path rather than passing force: True explicitly (which bypassed the check entirely). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: derive VALID_SCRIPT_TYPES from shared constant, fail fast on OSError, include all resolved fields in output - Derive VALID_SCRIPT_TYPES from SCRIPT_TYPE_CHOICES in _agent_config so the valid set cannot drift from the specify init CLI. - Fail fast with a clear error when os.scandir() raises OSError (e.g. permission denied) instead of silently treating the directory as empty. - Include preset, force, and ignore_agent_tools in all output dicts (both fast-fail and normal paths) for consistent interpolation and debugging downstream. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: populate stderr from stdout on older Click, fix force comment wording - When Click does not expose result.stderr (older versions where stderr is mixed into stdout), use stdout as stderr on non-zero exit so workflows can consistently read steps.<id>.output.stderr for errors. - Update README inline comment for force: wording to say 'when target directory already exists' rather than 'non-empty directory', matching the actual specify init behavior for the project: form. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: build argv flags before early returns, use any() for dir scan - Move argv flag-building (--integration, --script, --preset, --ignore-agent-tools) before the non-empty-dir and OSError early returns so output['argv'] always reflects the complete command. - --force is appended after the check since it may be set implicitly. - Replace list comprehension with any() generator expression to short-circuit without allocating a full list of DirEntry objects. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: only treat .specify as engine-owned when it is a real directory A file or symlink named .specify should not be excluded from the non-empty check. Use entry.is_dir(follow_symlinks=False) to ensure only an actual directory is considered engine-owned content. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix: guard implicit force for engine dirs only, fix integration fallback order - Only set implicit --force when engine-owned directories (.specify/) are actually present. A completely empty directory no longer gets --force added unnecessarily. - Fix integration resolution precedence: resolve step config expression first, then fall back to workflow default (also resolved), then to DEFAULT_INIT_INTEGRATION. Previously, a step expression resolving to falsy would bypass the workflow default entirely. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Manfred Riem <15701806+mnriem@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> Co-authored-by: Manfred Riem <mnriem@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
204 lines
7.7 KiB
Python
204 lines
7.7 KiB
Python
"""Workflow engine for multi-step, resumable automation workflows.
|
|
|
|
Provides:
|
|
- ``StepBase`` — abstract base every step type must implement.
|
|
- ``StepContext`` — execution context passed to each step.
|
|
- ``StepResult`` — return value from step execution.
|
|
- ``STEP_REGISTRY`` — maps ``type_key`` to ``StepBase`` subclass instances.
|
|
- ``WorkflowEngine`` — orchestrator that loads, validates, and executes
|
|
workflow YAML definitions.
|
|
- ``load_custom_steps`` — loads community-installed step types into STEP_REGISTRY.
|
|
"""
|
|
|
|
from __future__ import annotations
|
|
|
|
from pathlib import Path
|
|
from typing import TYPE_CHECKING
|
|
|
|
if TYPE_CHECKING:
|
|
from .base import StepBase
|
|
|
|
# Maps step type_key → StepBase instance.
|
|
STEP_REGISTRY: dict[str, StepBase] = {}
|
|
|
|
|
|
def _register_step(step: StepBase) -> None:
|
|
"""Register a step type instance in the global registry.
|
|
|
|
Raises ``ValueError`` for falsy keys and ``KeyError`` for duplicates.
|
|
"""
|
|
key = step.type_key
|
|
if not key:
|
|
raise ValueError("Cannot register step type with an empty type_key.")
|
|
if key in STEP_REGISTRY:
|
|
raise KeyError(f"Step type with key {key!r} is already registered.")
|
|
STEP_REGISTRY[key] = step
|
|
|
|
|
|
def get_step_type(type_key: str) -> StepBase | None:
|
|
"""Return the step type for *type_key*, or ``None`` if not registered."""
|
|
return STEP_REGISTRY.get(type_key)
|
|
|
|
|
|
# -- Register built-in step types ----------------------------------------
|
|
|
|
def _register_builtin_steps() -> None:
|
|
"""Register all built-in step types."""
|
|
from .steps.command import CommandStep
|
|
from .steps.do_while import DoWhileStep
|
|
from .steps.fan_in import FanInStep
|
|
from .steps.fan_out import FanOutStep
|
|
from .steps.gate import GateStep
|
|
from .steps.if_then import IfThenStep
|
|
from .steps.init import InitStep
|
|
from .steps.prompt import PromptStep
|
|
from .steps.shell import ShellStep
|
|
from .steps.switch import SwitchStep
|
|
from .steps.while_loop import WhileStep
|
|
|
|
_register_step(CommandStep())
|
|
_register_step(DoWhileStep())
|
|
_register_step(FanInStep())
|
|
_register_step(FanOutStep())
|
|
_register_step(GateStep())
|
|
_register_step(IfThenStep())
|
|
_register_step(InitStep())
|
|
_register_step(PromptStep())
|
|
_register_step(ShellStep())
|
|
_register_step(SwitchStep())
|
|
_register_step(WhileStep())
|
|
|
|
|
|
_register_builtin_steps()
|
|
|
|
|
|
def load_custom_steps(project_root: Path) -> list[str]:
|
|
"""Load community-installed custom step types into STEP_REGISTRY.
|
|
|
|
Scans ``.specify/workflows/steps/`` for installed step packages.
|
|
Each valid package must contain ``step.yml`` (with a ``step.type_key``
|
|
field) and ``__init__.py`` (a ``StepBase`` subclass).
|
|
|
|
Returns a list of type_keys that were successfully loaded.
|
|
Silently skips packages that fail to import or validate.
|
|
"""
|
|
import hashlib as _hashlib
|
|
import importlib.util as _importlib_util
|
|
import re as _re
|
|
import sys as _sys
|
|
|
|
steps_dir = Path(project_root) / ".specify" / "workflows" / "steps"
|
|
|
|
# Defense-in-depth: refuse to execute step code from a symlinked
|
|
# parent directory under .specify/workflows/steps, which could redirect
|
|
# the import outside the project root and bypass the install-time
|
|
# symlink guard. Check symlinks *before* is_dir() since the latter
|
|
# follows symlinks and would stat an external target.
|
|
_current = Path(project_root)
|
|
for _part in (".specify", "workflows", "steps"):
|
|
_current = _current / _part
|
|
if _current.is_symlink():
|
|
return []
|
|
|
|
if not steps_dir.is_dir():
|
|
return []
|
|
|
|
loaded: list[str] = []
|
|
for step_dir in steps_dir.iterdir():
|
|
# Check symlinks before is_dir() since the latter follows symlinks
|
|
# and would stat an external target through a symlinked directory.
|
|
if step_dir.is_symlink():
|
|
continue
|
|
if not step_dir.is_dir():
|
|
continue
|
|
step_yml = step_dir / "step.yml"
|
|
init_py = step_dir / "__init__.py"
|
|
if step_yml.is_symlink() or init_py.is_symlink():
|
|
continue
|
|
if not step_yml.is_file() or not init_py.is_file():
|
|
continue
|
|
|
|
try:
|
|
import yaml as _yaml
|
|
|
|
meta = _yaml.safe_load(step_yml.read_text(encoding="utf-8")) or {}
|
|
step_meta = meta.get("step", {})
|
|
type_key = step_meta.get("type_key", "")
|
|
if not type_key:
|
|
continue
|
|
|
|
# Skip if already registered (e.g. built-in or previously loaded)
|
|
if type_key in STEP_REGISTRY:
|
|
continue
|
|
|
|
# Sanitize type_key so the synthetic module name is a valid identifier
|
|
# (e.g. "test-custom" → "_speckit_custom_step_test_custom_<hash>").
|
|
# The 8-char SHA-256 hash of the original type_key makes the name
|
|
# collision-resistant when different type_keys produce the same
|
|
# sanitized form (e.g. "a-b" and "a_b" both sanitize to "a_b" but
|
|
# have different hashes).
|
|
safe_key = _re.sub(r"[^A-Za-z0-9_]", "_", type_key)
|
|
key_hash = _hashlib.sha256(type_key.encode()).hexdigest()[:8]
|
|
module_name = f"_speckit_custom_step_{safe_key}_{key_hash}"
|
|
|
|
# Treat the step directory as a proper package so that relative
|
|
# imports inside the step (e.g. ``from .helpers import …``) work.
|
|
spec = _importlib_util.spec_from_file_location(
|
|
module_name,
|
|
init_py,
|
|
submodule_search_locations=[str(step_dir)],
|
|
)
|
|
if spec is None or spec.loader is None:
|
|
continue
|
|
module = _importlib_util.module_from_spec(spec)
|
|
module.__package__ = module_name
|
|
# Register before exec so relative imports resolve correctly.
|
|
_sys.modules[module_name] = module
|
|
registered = False
|
|
try:
|
|
spec.loader.exec_module(module) # type: ignore[union-attr]
|
|
|
|
# Find the StepBase subclass in the module
|
|
from .base import StepBase as _StepBase
|
|
|
|
step_class = None
|
|
for attr_name in dir(module):
|
|
attr = getattr(module, attr_name)
|
|
try:
|
|
if (
|
|
isinstance(attr, type)
|
|
and issubclass(attr, _StepBase)
|
|
and attr is not _StepBase
|
|
and getattr(attr, "type_key", "") == type_key
|
|
):
|
|
step_class = attr
|
|
break
|
|
except TypeError:
|
|
continue
|
|
|
|
if step_class is None:
|
|
continue
|
|
|
|
_register_step(step_class())
|
|
loaded.append(type_key)
|
|
registered = True
|
|
finally:
|
|
# If the step wasn't successfully registered (failed import,
|
|
# no matching StepBase subclass, or registration error), remove
|
|
# the synthetic module — and any submodules loaded via relative
|
|
# imports (e.g. ``from .helpers import …``) — from sys.modules so
|
|
# a broken/skipped step package leaves no lingering import state
|
|
# behind.
|
|
if not registered:
|
|
_sys.modules.pop(module_name, None)
|
|
submodule_prefix = module_name + "."
|
|
for _mod_key in [
|
|
k for k in _sys.modules if k.startswith(submodule_prefix)
|
|
]:
|
|
_sys.modules.pop(_mod_key, None)
|
|
except Exception: # noqa: BLE001
|
|
# Silently skip broken step packages at load time
|
|
continue
|
|
|
|
return loaded
|