mirror of
https://github.com/github/spec-kit.git
synced 2026-07-08 15:06:10 +08:00
Add workflow engine with catalog system (#2158)
* Initial plan * Add workflow engine with step registry, expression engine, catalog system, and CLI commands Agent-Logs-Url: https://github.com/github/spec-kit/sessions/72a7bb5d-071f-4d67-a507-7e1abae2384d Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com> * Add comprehensive tests for workflow engine (94 tests) Agent-Logs-Url: https://github.com/github/spec-kit/sessions/72a7bb5d-071f-4d67-a507-7e1abae2384d Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com> * Address review feedback: do-while condition preservation and URL scheme validation Agent-Logs-Url: https://github.com/github/spec-kit/sessions/72a7bb5d-071f-4d67-a507-7e1abae2384d Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com> * Address review feedback, add CLI dispatch, interactive gates, and docs Review comments (7/7): - Add explanatory comment to empty except block - Implement workflow catalog download with cleanup on failure - Add input type coercion for number/boolean/enum - Fix example workflow to remove non-existent output references - Fix while_loop and if_then condition defaults (string 'false' → bool False) - Fix resume step index tracking with step_offset parameter CLI dispatch: - Add build_exec_args() and dispatch_command() to IntegrationBase - Override for Claude (skills: /speckit-specify), Gemini (-m flag), Codex (codex exec), Copilot (--agent speckit.specify) - CommandStep invokes installed commands by name via integration CLI - Add PromptStep for arbitrary inline prompts (10th step type) - Stream CLI output live to terminal (no silent blocking) - Remove timeout when streaming (user can Ctrl+C) - Ctrl+C saves state as PAUSED for clean resume Interactive gates: - Gate steps prompt [1] approve [2] reject in TTY - Fall back to PAUSED in non-interactive environments - Resume re-executes the gate for interactive prompting Documentation: - workflows/README.md — user guide - workflows/ARCHITECTURE.md — internals with Mermaid diagrams - workflows/PUBLISHING.md — catalog submission guide Tests: 94 → 122 workflow tests, 1362 total (all passing) * Fix ruff lint errors: unused imports, f-string placeholders, undefined name * Address second review: registry-backed validation, shell failures, loop/fan-out execution, URL validation - VALID_STEP_TYPES now queries STEP_REGISTRY dynamically - Shell step returns FAILED on non-zero exit code - Persist workflow YAML in run directory for reliable resume - Resume loads from run copy, falls back to installed workflow - Engine iterates while/do-while loops up to max_iterations - Engine expands fan-out per item with context.item - HTTPS URL validation for catalog workflow installs (HTTP allowed for localhost) - Fix catalog merge priority docstring (lower number wins) - Fix dispatch_command docstring (no build_exec_args_for_command) - Gate on_reject=retry pauses for re-prompt on resume - Update docs to 10 step types, add prompt step to tables and README * Potential fix for pull request finding 'Empty except' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * Address third review: fan-out IDs, catalog guards, shell coercion, docs - Fan-out generates unique per-item step IDs and collects results - Catalog merge skips non-dict workflow entries (malformed data guard) - Shell step coerces run_cmd to str after expression evaluation - urlopen timeout=30 for catalog workflow installs - yaml.dump with sort_keys=False, allow_unicode=True for catalog configs - Document streaming timeout as intentionally unbounded (user Ctrl+C) - Document --allow-all-tools as required for non-interactive + future enhancement - Update test docstring and PUBLISHING.md to 10 step types with prompt * Validate final URL after redirects in catalog fetch urlopen follows redirects, so validate the response URL against the same HTTPS/localhost rules to prevent redirect-based downgrade attacks. * Address fourth review: filter arg eval, tags normalization, install redirect check - Filter arguments now evaluated via _evaluate_simple_expression() so default(42) returns int not string - Tags normalized: non-list/non-string values handled gracefully - Install URL redirect validation (same as catalog fetch) - Remove unused 'skipped' variable in catalog config parsing - Author 'github' → 'GitHub' in example workflow - Document nested step resume limitation (re-runs parent step) * Add explanatory comment to empty except ValueError block * Address fifth review: expression parsing, fan-out output, URL install, gate options - Move string literal parsing before operator detection in expressions so quoted strings with operators (e.g. 'a in b') are not mis-parsed - Fan-out: remove max_concurrency from persisted output, fix docstring to reflect sequential execution - workflow add: support URL sources with HTTPS/redirect validation, validate workflow ID is non-empty before writing files - Deduplicate local install logic via _validate_and_install_local() - Remove 'edit' gate option from speckit workflow (not implemented) * Add comments to empty except ValueError blocks in URL install * Address sixth review: operator precedence, fan_in cleanup, registry resilience, docs - Fix or/and operator precedence (or parsed first = lower precedence) - Restore context.fan_in after fan-in step completes - Catch JSONDecodeError in registry load for corrupted files - Replace print() with on_step_start callback (library-safe) - Gate validation warns when on_reject set but no reject option - Shell step: document shell=True security tradeoff - README: sdd-pipeline → speckit, parallel → sequential for fan-out - ARCHITECTURE.md: parallel → fan-out/fan-in in diagram * Address seventh review: string literal before pipe, type annotations, validate on install - Move string literal check above pipe filter parsing so 'a | b' works - Fix type annotations: input_values list[str] | None, run_id str | None - Run validate_workflow() before installing from local path/URL - Remove duplicate string literal check from expression parser * Address eighth review: fan-out namespaced IDs, early return, catalog validation - Fan-out per-item step IDs use _fanout_{step_id}_{base}_{idx} namespace to avoid collisions with user-defined step IDs - Early return after fan-out loop when state is paused/failed/aborted - Catalog installs parse + validate downloaded YAML before registering, using definition metadata instead of catalog entry for registry * Address ninth review: populate catalog, fix indentation, priority, README - Add speckit workflow entry to catalog.json so it's discoverable - Fix shell step output dict indentation - Catalog add_catalog priority derived from max existing + 1 - README Quick Start clarified with install + local file examples * Address tenth review: max_iterations validation, catalog config guard, version alignment - Validate max_iterations is int >= 1 in while and do-while steps - Guard add_catalog against corrupted config (non-dict/non-list) - Align speckit_version requirement to >=0.6.1 (current package version) - Fan-out template validation uses separate seen_ids set to avoid false duplication errors with user-defined step IDs * Address eleventh review: command step fails without CLI, ID mismatch warning, state persistence - Command step returns FAILED when CLI not installed (was silent COMPLETED) - Catalog install warns on workflow ID vs catalog key mismatch - Engine persists state.save() before returning on unknown step type - Update tests to expect FAILED for command steps without CLI - Integration tests use shell steps for CLI-independent execution * Address twelfth review: type annotations, version examples, streaming docs, requires - Fix workflow_search type annotations (str | None) - PUBLISHING.md: speckit_version >=0.15.0 → >=0.6.1 - Document that exit_code is captured and referenceable by later steps - Mark requires as declared-but-not-enforced (planned enhancement) - Note full stdout/stderr capture as planned enhancement * Enforce catalog key matches workflow ID (fail instead of warn) * Bundle speckit workflow: auto-install during specify init - Add workflows/speckit to pyproject.toml force-include for wheel builds - Add _locate_bundled_workflow() helper (mirrors _locate_bundled_extension) - Auto-install speckit workflow during specify init (after git extension) - Update all integration file inventory tests to expect workflow files * Address fourteenth review: prompt fails without CLI, resolved step data, fan-out normalization - PromptStep returns FAILED when CLI not installed (was silent COMPLETED) - Engine step_data prefers resolved values from step output - Fan-out normalizes output.results=[] for empty item lists - subprocess.run inherits stdout/stderr (no explicit sys.stdout) - Registry tests use issubset for extensibility * Address fifteenth review: fan_in docstring, gate defaults, validation guards, reserved prefix - FanInStep docstring: aggregate-only, no blocking semantics - FanInStep: guard output_config as dict, handle None - Gate validate: use same default options as execute - Validate inputs is dict and steps is list before iterating - Reserve _fanout_ prefix in step ID validation - PUBLISHING.md: remove unenforced checklist items, add _fanout_ note * Address sixteenth review: docs regex, fan_in try/finally, hyphenated dot-path keys - PUBLISHING.md: update ID regex docs to match implementation (single-char OK) - FanInStep: wrap expression evaluation in try/finally for context.fan_in - Expression dot-path: allow hyphens in keys before list index (e.g. run-tests[0]) * Make speckit workflow integration-agnostic, document Copilot CLI requirement - Workflow integration selectable via input (default: claude) - Each command step uses {{ inputs.integration }} instead of hardcoded copilot - Copilot docstring documents CLI requirement for workflow dispatch - Added install_url for Copilot CLI docs * Address seventeenth review: project checks, catalog robustness - Add .specify/ project check to workflow run/resume/status/search/info - remove_catalog validates config shape (dict + list) before indexing - _fetch_single_catalog validates response is a dict - _get_merged_workflows raises when all catalogs fail to fetch - add_catalog guards against non-dict catalog entries in config * Address eighteenth review: condition coercion, gate abort result, while default, cache guard, resume log - evaluate_condition treats plain 'false'/'true' strings as booleans - Gate abort returns StepResult(FAILED) instead of raising exception so step output is persisted in state for inspection - while_loop max_iterations optional (default 10), validation aligned - Catalog cache fallback catches invalid JSON gracefully - resume() appends workflow_finished log entry like execute() * Address nineteenth review: allow-all-tools opt-in, empty catalogs, abort dead code, while docstring - --allow-all-tools controlled by SPECKIT_ALLOW_ALL_TOOLS env var (default: 1) Set to 0 to disable automatic tool approval for Copilot CLI - Empty catalogs list falls back to built-in defaults (not an error) - Remove unreachable WorkflowAbortError catches from execute/resume (gate abort now returns StepResult(FAILED) instead of raising) - while_loop docstring updated: max_iterations is optional (default 10) * Address twentieth review: gate abort maps to ABORTED status, do-while max_iterations optional - Engine detects output.aborted from gate step and sets RunStatus.ABORTED (was unreachable — gate abort returned FAILED but status was always FAILED) - do-while max_iterations now optional (default 10), aligned with while_loop - do-while docstring and validation updated accordingly * Coerce default_options to dict, align bundled workflow ID regex with validator * Gate validates string options, prompt uses resolved integration, loop normalizes max_iterations * Use parentId:childId convention for nested step IDs - Fan-out per-item IDs use parentId:templateId:index (e.g. parallel:impl:0) - Reserve ':' in user step IDs (validation rejects) - Replaces _fanout_ prefix with cleaner namespacing - Expressions like {{ steps.parallel:impl:0.output.file }} work naturally * Validate workflow version is semantic versioning (X.Y.Z) * Schema version validation, strict semver, load_workflow docstring, preserve max_concurrency - Validate schema_version is '1.0' (reject unknown future schemas) - Strict semver regex: ^\d+\.\d+\.\d+$ (rejects 1.0.0beta etc.) - load_workflow docstring: 'parsed' not 'validated' - Keep max_concurrency in fan-out output (was dropped) - do_while docstring: engine re-evaluates step_config condition - ARCHITECTURE.md: document nested resume limitation * Path traversal prevention, loop step ID namespacing - RunState validates run_id is alphanumeric+hyphens (no path separators) - workflow_add validates catalog source doesn't escape workflows_dir - Loop iterations namespace nested step IDs as parentId:childId:iteration so multiple iterations don't overwrite each other in context/state --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
This commit is contained in:
Copilot
@@ -621,6 +621,31 @@ def _locate_bundled_extension(extension_id: str) -> Path | None:
|
||||
return None
|
||||
|
||||
|
||||
def _locate_bundled_workflow(workflow_id: str) -> Path | None:
|
||||
"""Return the path to a bundled workflow directory, or None.
|
||||
|
||||
Checks the wheel's core_pack first, then falls back to the
|
||||
source-checkout ``workflows/<id>/`` directory.
|
||||
"""
|
||||
import re as _re
|
||||
if not _re.match(r'^[a-z0-9](?:[a-z0-9-]*[a-z0-9])?$', workflow_id):
|
||||
return None
|
||||
|
||||
core = _locate_core_pack()
|
||||
if core is not None:
|
||||
candidate = core / "workflows" / workflow_id
|
||||
if (candidate / "workflow.yml").is_file():
|
||||
return candidate
|
||||
|
||||
# Source-checkout / editable install: look relative to repo root
|
||||
repo_root = Path(__file__).parent.parent.parent
|
||||
candidate = repo_root / "workflows" / workflow_id
|
||||
if (candidate / "workflow.yml").is_file():
|
||||
return candidate
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def _locate_bundled_preset(preset_id: str) -> Path | None:
|
||||
"""Return the path to a bundled preset, or None.
|
||||
|
||||
@@ -1159,6 +1184,7 @@ def init(
|
||||
("chmod", "Ensure scripts executable"),
|
||||
("constitution", "Constitution setup"),
|
||||
("git", "Install git extension"),
|
||||
("workflow", "Install bundled workflow"),
|
||||
("final", "Finalize"),
|
||||
]:
|
||||
tracker.add(key, label)
|
||||
@@ -1262,6 +1288,37 @@ def init(
|
||||
else:
|
||||
tracker.skip("git", "--no-git flag")
|
||||
|
||||
# Install bundled speckit workflow
|
||||
try:
|
||||
bundled_wf = _locate_bundled_workflow("speckit")
|
||||
if bundled_wf:
|
||||
from .workflows.catalog import WorkflowRegistry
|
||||
from .workflows.engine import WorkflowDefinition
|
||||
wf_registry = WorkflowRegistry(project_path)
|
||||
if wf_registry.is_installed("speckit"):
|
||||
tracker.complete("workflow", "already installed")
|
||||
else:
|
||||
import shutil as _shutil
|
||||
dest_wf = project_path / ".specify" / "workflows" / "speckit"
|
||||
dest_wf.mkdir(parents=True, exist_ok=True)
|
||||
_shutil.copy2(
|
||||
bundled_wf / "workflow.yml",
|
||||
dest_wf / "workflow.yml",
|
||||
)
|
||||
definition = WorkflowDefinition.from_yaml(dest_wf / "workflow.yml")
|
||||
wf_registry.add("speckit", {
|
||||
"name": definition.name,
|
||||
"version": definition.version,
|
||||
"description": definition.description,
|
||||
"source": "bundled",
|
||||
})
|
||||
tracker.complete("workflow", "speckit installed")
|
||||
else:
|
||||
tracker.skip("workflow", "bundled workflow not found")
|
||||
except Exception as wf_err:
|
||||
sanitized_wf = str(wf_err).replace('\n', ' ').strip()
|
||||
tracker.error("workflow", f"install failed: {sanitized_wf[:120]}")
|
||||
|
||||
# Fix permissions after all installs (scripts + extensions)
|
||||
ensure_executable_scripts(project_path, tracker=tracker)
|
||||
|
||||
@@ -4136,6 +4193,668 @@ def extension_set_priority(
|
||||
console.print("\n[dim]Lower priority = higher precedence in template resolution[/dim]")
|
||||
|
||||
|
||||
# ===== Workflow Commands =====
|
||||
|
||||
workflow_app = typer.Typer(
|
||||
name="workflow",
|
||||
help="Manage and run automation workflows",
|
||||
add_completion=False,
|
||||
)
|
||||
app.add_typer(workflow_app, name="workflow")
|
||||
|
||||
workflow_catalog_app = typer.Typer(
|
||||
name="catalog",
|
||||
help="Manage workflow catalogs",
|
||||
add_completion=False,
|
||||
)
|
||||
workflow_app.add_typer(workflow_catalog_app, name="catalog")
|
||||
|
||||
|
||||
@workflow_app.command("run")
|
||||
def workflow_run(
|
||||
source: str = typer.Argument(..., help="Workflow ID or YAML file path"),
|
||||
input_values: list[str] | None = typer.Option(
|
||||
None, "--input", "-i", help="Input values as key=value pairs"
|
||||
),
|
||||
):
|
||||
"""Run a workflow from an installed ID or local YAML path."""
|
||||
from .workflows.engine import WorkflowEngine
|
||||
|
||||
project_root = Path.cwd()
|
||||
if not (project_root / ".specify").exists():
|
||||
console.print("[red]Error:[/red] Not a spec-kit project (no .specify/ directory)")
|
||||
raise typer.Exit(1)
|
||||
engine = WorkflowEngine(project_root)
|
||||
engine.on_step_start = lambda sid, label: console.print(f" \u25b8 [{sid}] {label} \u2026")
|
||||
|
||||
try:
|
||||
definition = engine.load_workflow(source)
|
||||
except FileNotFoundError:
|
||||
console.print(f"[red]Error:[/red] Workflow not found: {source}")
|
||||
raise typer.Exit(1)
|
||||
except ValueError as exc:
|
||||
console.print(f"[red]Error:[/red] Invalid workflow: {exc}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Validate
|
||||
errors = engine.validate(definition)
|
||||
if errors:
|
||||
console.print("[red]Workflow validation failed:[/red]")
|
||||
for err in errors:
|
||||
console.print(f" • {err}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Parse inputs
|
||||
inputs: dict[str, Any] = {}
|
||||
if input_values:
|
||||
for kv in input_values:
|
||||
if "=" not in kv:
|
||||
console.print(f"[red]Error:[/red] Invalid input format: {kv!r} (expected key=value)")
|
||||
raise typer.Exit(1)
|
||||
key, _, value = kv.partition("=")
|
||||
inputs[key.strip()] = value.strip()
|
||||
|
||||
console.print(f"\n[bold cyan]Running workflow:[/bold cyan] {definition.name} ({definition.id})")
|
||||
console.print(f"[dim]Version: {definition.version}[/dim]\n")
|
||||
|
||||
try:
|
||||
state = engine.execute(definition, inputs)
|
||||
except ValueError as exc:
|
||||
console.print(f"[red]Error:[/red] {exc}")
|
||||
raise typer.Exit(1)
|
||||
except Exception as exc:
|
||||
console.print(f"[red]Workflow failed:[/red] {exc}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
status_colors = {
|
||||
"completed": "green",
|
||||
"paused": "yellow",
|
||||
"failed": "red",
|
||||
"aborted": "red",
|
||||
}
|
||||
color = status_colors.get(state.status.value, "white")
|
||||
console.print(f"\n[{color}]Status: {state.status.value}[/{color}]")
|
||||
console.print(f"[dim]Run ID: {state.run_id}[/dim]")
|
||||
|
||||
if state.status.value == "paused":
|
||||
console.print(f"\nResume with: [cyan]specify workflow resume {state.run_id}[/cyan]")
|
||||
|
||||
|
||||
@workflow_app.command("resume")
|
||||
def workflow_resume(
|
||||
run_id: str = typer.Argument(..., help="Run ID to resume"),
|
||||
):
|
||||
"""Resume a paused or failed workflow run."""
|
||||
from .workflows.engine import WorkflowEngine
|
||||
|
||||
project_root = Path.cwd()
|
||||
if not (project_root / ".specify").exists():
|
||||
console.print("[red]Error:[/red] Not a spec-kit project (no .specify/ directory)")
|
||||
raise typer.Exit(1)
|
||||
engine = WorkflowEngine(project_root)
|
||||
engine.on_step_start = lambda sid, label: console.print(f" \u25b8 [{sid}] {label} \u2026")
|
||||
|
||||
try:
|
||||
state = engine.resume(run_id)
|
||||
except FileNotFoundError:
|
||||
console.print(f"[red]Error:[/red] Run not found: {run_id}")
|
||||
raise typer.Exit(1)
|
||||
except ValueError as exc:
|
||||
console.print(f"[red]Error:[/red] {exc}")
|
||||
raise typer.Exit(1)
|
||||
except Exception as exc:
|
||||
console.print(f"[red]Resume failed:[/red] {exc}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
status_colors = {
|
||||
"completed": "green",
|
||||
"paused": "yellow",
|
||||
"failed": "red",
|
||||
"aborted": "red",
|
||||
}
|
||||
color = status_colors.get(state.status.value, "white")
|
||||
console.print(f"\n[{color}]Status: {state.status.value}[/{color}]")
|
||||
|
||||
|
||||
@workflow_app.command("status")
|
||||
def workflow_status(
|
||||
run_id: str | None = typer.Argument(None, help="Run ID to inspect (shows all if omitted)"),
|
||||
):
|
||||
"""Show workflow run status."""
|
||||
from .workflows.engine import WorkflowEngine
|
||||
|
||||
project_root = Path.cwd()
|
||||
if not (project_root / ".specify").exists():
|
||||
console.print("[red]Error:[/red] Not a spec-kit project (no .specify/ directory)")
|
||||
raise typer.Exit(1)
|
||||
engine = WorkflowEngine(project_root)
|
||||
|
||||
if run_id:
|
||||
try:
|
||||
from .workflows.engine import RunState
|
||||
state = RunState.load(run_id, project_root)
|
||||
except FileNotFoundError:
|
||||
console.print(f"[red]Error:[/red] Run not found: {run_id}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
status_colors = {
|
||||
"completed": "green",
|
||||
"paused": "yellow",
|
||||
"failed": "red",
|
||||
"aborted": "red",
|
||||
"running": "blue",
|
||||
"created": "dim",
|
||||
}
|
||||
color = status_colors.get(state.status.value, "white")
|
||||
|
||||
console.print(f"\n[bold cyan]Workflow Run: {state.run_id}[/bold cyan]")
|
||||
console.print(f" Workflow: {state.workflow_id}")
|
||||
console.print(f" Status: [{color}]{state.status.value}[/{color}]")
|
||||
console.print(f" Created: {state.created_at}")
|
||||
console.print(f" Updated: {state.updated_at}")
|
||||
|
||||
if state.current_step_id:
|
||||
console.print(f" Current: {state.current_step_id}")
|
||||
|
||||
if state.step_results:
|
||||
console.print(f"\n [bold]Steps ({len(state.step_results)}):[/bold]")
|
||||
for step_id, step_data in state.step_results.items():
|
||||
s = step_data.get("status", "unknown")
|
||||
sc = {"completed": "green", "failed": "red", "paused": "yellow"}.get(s, "white")
|
||||
console.print(f" [{sc}]●[/{sc}] {step_id}: {s}")
|
||||
else:
|
||||
runs = engine.list_runs()
|
||||
if not runs:
|
||||
console.print("[yellow]No workflow runs found.[/yellow]")
|
||||
return
|
||||
|
||||
console.print("\n[bold cyan]Workflow Runs:[/bold cyan]\n")
|
||||
for run_data in runs:
|
||||
s = run_data.get("status", "unknown")
|
||||
sc = {"completed": "green", "failed": "red", "paused": "yellow", "running": "blue"}.get(s, "white")
|
||||
console.print(
|
||||
f" [{sc}]●[/{sc}] {run_data['run_id']} "
|
||||
f"{run_data.get('workflow_id', '?')} "
|
||||
f"[{sc}]{s}[/{sc}] "
|
||||
f"[dim]{run_data.get('updated_at', '?')}[/dim]"
|
||||
)
|
||||
|
||||
|
||||
@workflow_app.command("list")
|
||||
def workflow_list():
|
||||
"""List installed workflows."""
|
||||
from .workflows.catalog import WorkflowRegistry
|
||||
|
||||
project_root = Path.cwd()
|
||||
specify_dir = project_root / ".specify"
|
||||
if not specify_dir.exists():
|
||||
console.print("[red]Error:[/red] Not a spec-kit project (no .specify/ directory)")
|
||||
raise typer.Exit(1)
|
||||
|
||||
registry = WorkflowRegistry(project_root)
|
||||
installed = registry.list()
|
||||
|
||||
if not installed:
|
||||
console.print("[yellow]No workflows installed.[/yellow]")
|
||||
console.print("\nInstall a workflow with:")
|
||||
console.print(" [cyan]specify workflow add <workflow-id>[/cyan]")
|
||||
return
|
||||
|
||||
console.print("\n[bold cyan]Installed Workflows:[/bold cyan]\n")
|
||||
for wf_id, wf_data in installed.items():
|
||||
console.print(f" [bold]{wf_data.get('name', wf_id)}[/bold] ({wf_id}) v{wf_data.get('version', '?')}")
|
||||
desc = wf_data.get("description", "")
|
||||
if desc:
|
||||
console.print(f" {desc}")
|
||||
console.print()
|
||||
|
||||
|
||||
@workflow_app.command("add")
|
||||
def workflow_add(
|
||||
source: str = typer.Argument(..., help="Workflow ID, URL, or local path"),
|
||||
):
|
||||
"""Install a workflow from catalog, URL, or local path."""
|
||||
from .workflows.catalog import WorkflowCatalog, WorkflowRegistry, WorkflowCatalogError
|
||||
from .workflows.engine import WorkflowDefinition
|
||||
|
||||
project_root = Path.cwd()
|
||||
specify_dir = project_root / ".specify"
|
||||
if not specify_dir.exists():
|
||||
console.print("[red]Error:[/red] Not a spec-kit project (no .specify/ directory)")
|
||||
raise typer.Exit(1)
|
||||
|
||||
registry = WorkflowRegistry(project_root)
|
||||
workflows_dir = project_root / ".specify" / "workflows"
|
||||
|
||||
def _validate_and_install_local(yaml_path: Path, source_label: str) -> None:
|
||||
"""Validate and install a workflow from a local YAML file."""
|
||||
try:
|
||||
definition = WorkflowDefinition.from_yaml(yaml_path)
|
||||
except (ValueError, yaml.YAMLError) as exc:
|
||||
console.print(f"[red]Error:[/red] Invalid workflow YAML: {exc}")
|
||||
raise typer.Exit(1)
|
||||
if not definition.id or not definition.id.strip():
|
||||
console.print("[red]Error:[/red] Workflow definition has an empty or missing 'id'")
|
||||
raise typer.Exit(1)
|
||||
|
||||
from .workflows.engine import validate_workflow
|
||||
errors = validate_workflow(definition)
|
||||
if errors:
|
||||
console.print("[red]Error:[/red] Workflow validation failed:")
|
||||
for err in errors:
|
||||
console.print(f" \u2022 {err}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
dest_dir = workflows_dir / definition.id
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
import shutil
|
||||
shutil.copy2(yaml_path, dest_dir / "workflow.yml")
|
||||
registry.add(definition.id, {
|
||||
"name": definition.name,
|
||||
"version": definition.version,
|
||||
"description": definition.description,
|
||||
"source": source_label,
|
||||
})
|
||||
console.print(f"[green]✓[/green] Workflow '{definition.name}' ({definition.id}) installed")
|
||||
|
||||
# Try as URL (http/https)
|
||||
if source.startswith("http://") or source.startswith("https://"):
|
||||
from ipaddress import ip_address
|
||||
from urllib.parse import urlparse
|
||||
from urllib.request import urlopen # noqa: S310
|
||||
|
||||
parsed_src = urlparse(source)
|
||||
src_host = parsed_src.hostname or ""
|
||||
src_loopback = src_host == "localhost"
|
||||
if not src_loopback:
|
||||
try:
|
||||
src_loopback = ip_address(src_host).is_loopback
|
||||
except ValueError:
|
||||
# Host is not an IP literal (e.g., a DNS name); keep default non-loopback.
|
||||
pass
|
||||
if parsed_src.scheme != "https" and not (parsed_src.scheme == "http" and src_loopback):
|
||||
console.print("[red]Error:[/red] Only HTTPS URLs are allowed, except HTTP for localhost.")
|
||||
raise typer.Exit(1)
|
||||
|
||||
import tempfile
|
||||
try:
|
||||
with urlopen(source, timeout=30) as resp: # noqa: S310
|
||||
final_url = resp.geturl()
|
||||
final_parsed = urlparse(final_url)
|
||||
final_host = final_parsed.hostname or ""
|
||||
final_lb = final_host == "localhost"
|
||||
if not final_lb:
|
||||
try:
|
||||
final_lb = ip_address(final_host).is_loopback
|
||||
except ValueError:
|
||||
# Redirect host is not an IP literal; keep loopback as determined above.
|
||||
pass
|
||||
if final_parsed.scheme != "https" and not (final_parsed.scheme == "http" and final_lb):
|
||||
console.print(f"[red]Error:[/red] URL redirected to non-HTTPS: {final_url}")
|
||||
raise typer.Exit(1)
|
||||
with tempfile.NamedTemporaryFile(suffix=".yml", delete=False) as tmp:
|
||||
tmp.write(resp.read())
|
||||
tmp_path = Path(tmp.name)
|
||||
except typer.Exit:
|
||||
raise
|
||||
except Exception as exc:
|
||||
console.print(f"[red]Error:[/red] Failed to download workflow: {exc}")
|
||||
raise typer.Exit(1)
|
||||
try:
|
||||
_validate_and_install_local(tmp_path, source)
|
||||
finally:
|
||||
tmp_path.unlink(missing_ok=True)
|
||||
return
|
||||
|
||||
# Try as a local file/directory
|
||||
source_path = Path(source)
|
||||
if source_path.exists():
|
||||
if source_path.is_file() and source_path.suffix in (".yml", ".yaml"):
|
||||
_validate_and_install_local(source_path, str(source_path))
|
||||
return
|
||||
elif source_path.is_dir():
|
||||
wf_file = source_path / "workflow.yml"
|
||||
if not wf_file.exists():
|
||||
console.print(f"[red]Error:[/red] No workflow.yml found in {source}")
|
||||
raise typer.Exit(1)
|
||||
_validate_and_install_local(wf_file, str(source_path))
|
||||
return
|
||||
|
||||
# Try from catalog
|
||||
catalog = WorkflowCatalog(project_root)
|
||||
try:
|
||||
info = catalog.get_workflow_info(source)
|
||||
except WorkflowCatalogError as exc:
|
||||
console.print(f"[red]Error:[/red] {exc}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
if not info:
|
||||
console.print(f"[red]Error:[/red] Workflow '{source}' not found in catalog")
|
||||
raise typer.Exit(1)
|
||||
|
||||
if not info.get("_install_allowed", True):
|
||||
console.print(f"[yellow]Warning:[/yellow] Workflow '{source}' is from a discovery-only catalog")
|
||||
console.print("Direct installation is not enabled for this catalog source.")
|
||||
raise typer.Exit(1)
|
||||
|
||||
workflow_url = info.get("url")
|
||||
if not workflow_url:
|
||||
console.print(f"[red]Error:[/red] Workflow '{source}' does not have an install URL in the catalog")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Validate URL scheme (HTTPS required, HTTP allowed for localhost only)
|
||||
from ipaddress import ip_address
|
||||
from urllib.parse import urlparse
|
||||
|
||||
parsed_url = urlparse(workflow_url)
|
||||
url_host = parsed_url.hostname or ""
|
||||
is_loopback = False
|
||||
if url_host == "localhost":
|
||||
is_loopback = True
|
||||
else:
|
||||
try:
|
||||
is_loopback = ip_address(url_host).is_loopback
|
||||
except ValueError:
|
||||
# Host is not an IP literal (e.g., a regular hostname); treat as non-loopback.
|
||||
pass
|
||||
if parsed_url.scheme != "https" and not (parsed_url.scheme == "http" and is_loopback):
|
||||
console.print(
|
||||
f"[red]Error:[/red] Workflow '{source}' has an invalid install URL. "
|
||||
"Only HTTPS URLs are allowed, except HTTP for localhost/loopback."
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
workflow_dir = workflows_dir / source
|
||||
# Validate that source is a safe directory name (no path traversal)
|
||||
try:
|
||||
workflow_dir.resolve().relative_to(workflows_dir.resolve())
|
||||
except ValueError:
|
||||
console.print(f"[red]Error:[/red] Invalid workflow ID: {source!r}")
|
||||
raise typer.Exit(1)
|
||||
workflow_file = workflow_dir / "workflow.yml"
|
||||
|
||||
try:
|
||||
from urllib.request import urlopen # noqa: S310 — URL comes from catalog
|
||||
|
||||
workflow_dir.mkdir(parents=True, exist_ok=True)
|
||||
with urlopen(workflow_url, timeout=30) as response: # noqa: S310
|
||||
# Validate final URL after redirects
|
||||
final_url = response.geturl()
|
||||
final_parsed = urlparse(final_url)
|
||||
final_host = final_parsed.hostname or ""
|
||||
final_loopback = final_host == "localhost"
|
||||
if not final_loopback:
|
||||
try:
|
||||
final_loopback = ip_address(final_host).is_loopback
|
||||
except ValueError:
|
||||
# Host is not an IP literal (e.g., a regular hostname); treat as non-loopback.
|
||||
pass
|
||||
if final_parsed.scheme != "https" and not (final_parsed.scheme == "http" and final_loopback):
|
||||
if workflow_dir.exists():
|
||||
import shutil
|
||||
shutil.rmtree(workflow_dir, ignore_errors=True)
|
||||
console.print(
|
||||
f"[red]Error:[/red] Workflow '{source}' redirected to non-HTTPS URL: {final_url}"
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
workflow_file.write_bytes(response.read())
|
||||
except Exception as exc:
|
||||
if workflow_dir.exists():
|
||||
import shutil
|
||||
shutil.rmtree(workflow_dir, ignore_errors=True)
|
||||
console.print(f"[red]Error:[/red] Failed to install workflow '{source}' from catalog: {exc}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Validate the downloaded workflow before registering
|
||||
try:
|
||||
definition = WorkflowDefinition.from_yaml(workflow_file)
|
||||
except (ValueError, yaml.YAMLError) as exc:
|
||||
import shutil
|
||||
shutil.rmtree(workflow_dir, ignore_errors=True)
|
||||
console.print(f"[red]Error:[/red] Downloaded workflow is invalid: {exc}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
from .workflows.engine import validate_workflow
|
||||
errors = validate_workflow(definition)
|
||||
if errors:
|
||||
import shutil
|
||||
shutil.rmtree(workflow_dir, ignore_errors=True)
|
||||
console.print("[red]Error:[/red] Downloaded workflow validation failed:")
|
||||
for err in errors:
|
||||
console.print(f" \u2022 {err}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Enforce that the workflow's internal ID matches the catalog key
|
||||
if definition.id and definition.id != source:
|
||||
import shutil
|
||||
shutil.rmtree(workflow_dir, ignore_errors=True)
|
||||
console.print(
|
||||
f"[red]Error:[/red] Workflow ID in YAML ({definition.id!r}) "
|
||||
f"does not match catalog key ({source!r}). "
|
||||
f"The catalog entry may be misconfigured."
|
||||
)
|
||||
raise typer.Exit(1)
|
||||
|
||||
registry.add(source, {
|
||||
"name": definition.name or info.get("name", source),
|
||||
"version": definition.version or info.get("version", "0.0.0"),
|
||||
"description": definition.description or info.get("description", ""),
|
||||
"source": "catalog",
|
||||
"catalog_name": info.get("_catalog_name", ""),
|
||||
"url": workflow_url,
|
||||
})
|
||||
console.print(f"[green]✓[/green] Workflow '{info.get('name', source)}' installed from catalog")
|
||||
|
||||
|
||||
@workflow_app.command("remove")
|
||||
def workflow_remove(
|
||||
workflow_id: str = typer.Argument(..., help="Workflow ID to uninstall"),
|
||||
):
|
||||
"""Uninstall a workflow."""
|
||||
from .workflows.catalog import WorkflowRegistry
|
||||
|
||||
project_root = Path.cwd()
|
||||
specify_dir = project_root / ".specify"
|
||||
if not specify_dir.exists():
|
||||
console.print("[red]Error:[/red] Not a spec-kit project (no .specify/ directory)")
|
||||
raise typer.Exit(1)
|
||||
|
||||
registry = WorkflowRegistry(project_root)
|
||||
|
||||
if not registry.is_installed(workflow_id):
|
||||
console.print(f"[red]Error:[/red] Workflow '{workflow_id}' is not installed")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Remove workflow files
|
||||
workflow_dir = project_root / ".specify" / "workflows" / workflow_id
|
||||
if workflow_dir.exists():
|
||||
import shutil
|
||||
shutil.rmtree(workflow_dir)
|
||||
|
||||
registry.remove(workflow_id)
|
||||
console.print(f"[green]✓[/green] Workflow '{workflow_id}' removed")
|
||||
|
||||
|
||||
@workflow_app.command("search")
|
||||
def workflow_search(
|
||||
query: str | None = typer.Argument(None, help="Search query"),
|
||||
tag: str | None = typer.Option(None, "--tag", help="Filter by tag"),
|
||||
):
|
||||
"""Search workflow catalogs."""
|
||||
from .workflows.catalog import WorkflowCatalog, WorkflowCatalogError
|
||||
|
||||
project_root = Path.cwd()
|
||||
if not (project_root / ".specify").exists():
|
||||
console.print("[red]Error:[/red] Not a spec-kit project (no .specify/ directory)")
|
||||
raise typer.Exit(1)
|
||||
catalog = WorkflowCatalog(project_root)
|
||||
|
||||
try:
|
||||
results = catalog.search(query=query, tag=tag)
|
||||
except WorkflowCatalogError as exc:
|
||||
console.print(f"[red]Error:[/red] {exc}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
if not results:
|
||||
console.print("[yellow]No workflows found.[/yellow]")
|
||||
return
|
||||
|
||||
console.print(f"\n[bold cyan]Workflows ({len(results)}):[/bold cyan]\n")
|
||||
for wf in results:
|
||||
console.print(f" [bold]{wf.get('name', wf.get('id', '?'))}[/bold] ({wf.get('id', '?')}) v{wf.get('version', '?')}")
|
||||
desc = wf.get("description", "")
|
||||
if desc:
|
||||
console.print(f" {desc}")
|
||||
tags = wf.get("tags", [])
|
||||
if tags:
|
||||
console.print(f" [dim]Tags: {', '.join(tags)}[/dim]")
|
||||
console.print()
|
||||
|
||||
|
||||
@workflow_app.command("info")
|
||||
def workflow_info(
|
||||
workflow_id: str = typer.Argument(..., help="Workflow ID"),
|
||||
):
|
||||
"""Show workflow details and step graph."""
|
||||
from .workflows.catalog import WorkflowCatalog, WorkflowRegistry, WorkflowCatalogError
|
||||
from .workflows.engine import WorkflowEngine
|
||||
|
||||
project_root = Path.cwd()
|
||||
if not (project_root / ".specify").exists():
|
||||
console.print("[red]Error:[/red] Not a spec-kit project (no .specify/ directory)")
|
||||
raise typer.Exit(1)
|
||||
|
||||
# Check installed first
|
||||
registry = WorkflowRegistry(project_root)
|
||||
installed = registry.get(workflow_id)
|
||||
|
||||
engine = WorkflowEngine(project_root)
|
||||
|
||||
definition = None
|
||||
try:
|
||||
definition = engine.load_workflow(workflow_id)
|
||||
except FileNotFoundError:
|
||||
# Local workflow definition not found on disk; fall back to
|
||||
# catalog/registry lookup below.
|
||||
pass
|
||||
|
||||
if definition:
|
||||
console.print(f"\n[bold cyan]{definition.name}[/bold cyan] ({definition.id})")
|
||||
console.print(f" Version: {definition.version}")
|
||||
if definition.author:
|
||||
console.print(f" Author: {definition.author}")
|
||||
if definition.description:
|
||||
console.print(f" Description: {definition.description}")
|
||||
if definition.default_integration:
|
||||
console.print(f" Integration: {definition.default_integration}")
|
||||
if installed:
|
||||
console.print(" [green]Installed[/green]")
|
||||
|
||||
if definition.inputs:
|
||||
console.print("\n [bold]Inputs:[/bold]")
|
||||
for name, inp in definition.inputs.items():
|
||||
if isinstance(inp, dict):
|
||||
req = "required" if inp.get("required") else "optional"
|
||||
console.print(f" {name} ({inp.get('type', 'string')}) — {req}")
|
||||
|
||||
if definition.steps:
|
||||
console.print(f"\n [bold]Steps ({len(definition.steps)}):[/bold]")
|
||||
for step in definition.steps:
|
||||
stype = step.get("type", "command")
|
||||
console.print(f" → {step.get('id', '?')} [{stype}]")
|
||||
return
|
||||
|
||||
# Try catalog
|
||||
catalog = WorkflowCatalog(project_root)
|
||||
try:
|
||||
info = catalog.get_workflow_info(workflow_id)
|
||||
except WorkflowCatalogError:
|
||||
info = None
|
||||
|
||||
if info:
|
||||
console.print(f"\n[bold cyan]{info.get('name', workflow_id)}[/bold cyan] ({workflow_id})")
|
||||
console.print(f" Version: {info.get('version', '?')}")
|
||||
if info.get("description"):
|
||||
console.print(f" Description: {info['description']}")
|
||||
if info.get("tags"):
|
||||
console.print(f" Tags: {', '.join(info['tags'])}")
|
||||
console.print(" [yellow]Not installed[/yellow]")
|
||||
else:
|
||||
console.print(f"[red]Error:[/red] Workflow '{workflow_id}' not found")
|
||||
raise typer.Exit(1)
|
||||
|
||||
|
||||
@workflow_catalog_app.command("list")
|
||||
def workflow_catalog_list():
|
||||
"""List configured workflow catalog sources."""
|
||||
from .workflows.catalog import WorkflowCatalog, WorkflowCatalogError
|
||||
|
||||
project_root = Path.cwd()
|
||||
catalog = WorkflowCatalog(project_root)
|
||||
|
||||
try:
|
||||
configs = catalog.get_catalog_configs()
|
||||
except WorkflowCatalogError as exc:
|
||||
console.print(f"[red]Error:[/red] {exc}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
console.print("\n[bold cyan]Workflow Catalog Sources:[/bold cyan]\n")
|
||||
for i, cfg in enumerate(configs):
|
||||
install_status = "[green]install allowed[/green]" if cfg["install_allowed"] else "[yellow]discovery only[/yellow]"
|
||||
console.print(f" [{i}] [bold]{cfg['name']}[/bold] — {install_status}")
|
||||
console.print(f" {cfg['url']}")
|
||||
if cfg.get("description"):
|
||||
console.print(f" [dim]{cfg['description']}[/dim]")
|
||||
console.print()
|
||||
|
||||
|
||||
@workflow_catalog_app.command("add")
|
||||
def workflow_catalog_add(
|
||||
url: str = typer.Argument(..., help="Catalog URL to add"),
|
||||
name: str = typer.Option(None, "--name", help="Catalog name"),
|
||||
):
|
||||
"""Add a workflow catalog source."""
|
||||
from .workflows.catalog import WorkflowCatalog, WorkflowValidationError
|
||||
|
||||
project_root = Path.cwd()
|
||||
specify_dir = project_root / ".specify"
|
||||
if not specify_dir.exists():
|
||||
console.print("[red]Error:[/red] Not a spec-kit project (no .specify/ directory)")
|
||||
raise typer.Exit(1)
|
||||
|
||||
catalog = WorkflowCatalog(project_root)
|
||||
try:
|
||||
catalog.add_catalog(url, name)
|
||||
except WorkflowValidationError as exc:
|
||||
console.print(f"[red]Error:[/red] {exc}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
console.print(f"[green]✓[/green] Catalog source added: {url}")
|
||||
|
||||
|
||||
@workflow_catalog_app.command("remove")
|
||||
def workflow_catalog_remove(
|
||||
index: int = typer.Argument(..., help="Catalog index to remove (from 'catalog list')"),
|
||||
):
|
||||
"""Remove a workflow catalog source by index."""
|
||||
from .workflows.catalog import WorkflowCatalog, WorkflowValidationError
|
||||
|
||||
project_root = Path.cwd()
|
||||
specify_dir = project_root / ".specify"
|
||||
if not specify_dir.exists():
|
||||
console.print("[red]Error:[/red] Not a spec-kit project (no .specify/ directory)")
|
||||
raise typer.Exit(1)
|
||||
|
||||
catalog = WorkflowCatalog(project_root)
|
||||
try:
|
||||
removed_name = catalog.remove_catalog(index)
|
||||
except WorkflowValidationError as exc:
|
||||
console.print(f"[red]Error:[/red] {exc}")
|
||||
raise typer.Exit(1)
|
||||
|
||||
console.print(f"[green]✓[/green] Catalog source '{removed_name}' removed")
|
||||
|
||||
|
||||
def main():
|
||||
app()
|
||||
|
||||
|
||||
Reference in New Issue
Block a user