github/github-spec-kit
1
0
Fork 0
mirror of https://github.com/github/spec-kit.git synced 2026-08-03 06:26:30 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,652 Commits 4 Branches 208 Tags
3b683c2292acbc0e18bd6b6767bb56ab5f6c78a2
Commit Graph

11 Commits

Author SHA1 Message Date
Manfred Riem
781a14a6d2 docs: clarify shell-step interpolation safety (#3719) 
* docs: clarify shell-step interpolation safety

Shell step `run` fields are executed by the system shell and `{{ ... }}`
expressions are substituted as raw, unquoted text. Document that untrusted
sources — workflow `inputs.*` and prior-step output, including AI-generated
`prompt` output — must be quoted, enum-constrained, validated, or gated before
they reach a `run` field.

- docs/reference/workflows.md: add an "Interpolation and shell safety" section.
- workflows/README.md: add a warning under the Shell Steps example, link to the
  new section, and quote the `inputs.project_dir` example.
- workflows/PUBLISHING.md: strengthen the interpolation guidance and call out
  prior-step/agent output as untrusted.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0fd6900-69bf-4fcb-b377-de37f98f5835

* docs: correct shell-step interpolation guidance

Address review feedback that the previous wording over-promised. Clarify that
none of the mitigations neutralise a hostile interpolated value:

- Quoting is not a security boundary — there is no shell-escaping filter, and a
  value containing the matching quote can break out. Present quoting as
  correctness handling for already-constrained values only.
- Remove the "pass data via environment or files" guidance: ShellStep has no
  `env` mapping (it only copies the process environment and sets
  SPECKIT_WORKFLOW_DIR), so that transport does not exist.
- Drop the claim that routing through a command/prompt step validates or safely
  binds a value; it does not.
- Correct the gate guidance: a gate renders only its own message/show_file and
  does not inspect, resolve, or sanitise the following step. Authors must
  surface the exact command/data in the gate themselves, and approval does not
  neutralise an injectable interpolation.

Frame constraining values at the source (enum/allowlist) as the only reliable
control, and keeping unconstrained values out of `run` fields entirely.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0fd6900-69bf-4fcb-b377-de37f98f5835

* docs: remove unsafe interpolation from example and gate guidance

Address further review feedback:

- workflows/README.md: the shell example interpolated an unconstrained path
  into shell source, which contradicted the warning beneath it. Shell steps
  already run from the project root, so drop the `cd '{{ inputs.project_dir }}'`
  prefix and model a plain `run: "npm test"` with no interpolation.
- docs/reference/workflows.md: GateStep prints `message` verbatim with no
  control-character stripping (stripping applies only to `show_file` path and
  contents), so recommending that authors surface untrusted data in `message`
  was itself unsafe — agent/caller output could inject terminal escapes to
  alter or hide the prompt. Direct authors to keep `message` to trusted text
  and surface untrusted material via `show_file`, whose contents are stripped.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0fd6900-69bf-4fcb-b377-de37f98f5835

* docs: use correct prompt-step output key in example

A `prompt` step stores agent-generated text under `output.stdout`, not
`output.value`, so the example expression `{{ steps.plan.output.value }}`
would resolve to None. Reference `output.stdout` so the example correctly
demonstrates untrusted agent output flowing into a shell step.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0fd6900-69bf-4fcb-b377-de37f98f5835

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c0fd6900-69bf-4fcb-b377-de37f98f5835
 2026-07-24 10:40:26 -05:00
Markus Wondrak
d6fa0460ed feat(workflows): WorkflowResolver standalone (PR 1) (#3557) 
* feat(workflows): add standalone WorkflowResolver and overlay subsystem

Implement PR 1 of the workflow-overlays plan: a concrete, standalone
WorkflowResolver for downstream workflow extensibility without touching the
Preset subsystem.

- Add overlay manifest schema (Overlay, OverlayEdit, validate_overlay_yaml)
- Add pure-function merge engine (find_step, apply_edit, merge_steps,
  validate_edits) with recursive anchor search and higher-wins semantics
- Add StepListComposer and tiered layer sources (project, installed, base)
- Add WorkflowResolver facade with inline HIGHER_WINS priority sorting
- Add CLI verbs: workflow overlay add/set-priority/enable/disable/remove/list
  and workflow resolve <id>
- Wire WorkflowEngine.load_workflow through WorkflowResolver
- Extend workflow add to copy optional overlays/ subdirectory from local
  workflow directories
- Add comprehensive unit, integration, and security tests

Refs: discussion #3473 (https://github.com/github/spec-kit/discussions/3473)

Assisted-by: Kimi (model: opencode-go/kimi-k2.7-code, autonomous)

* fix(workflows): reject symlinked overlay directories in layer sources

Address PR #3557 review comments r3594064534 and r3594064563:

- ProjectOverlaySource.collect now rejects symlinked per-workflow overlay
  directories (.specify/workflows/overlays/<id>) before iterating
- InstalledOverlaySource.collect now rejects symlinked installed overlay
  directories (.specify/workflows/<id>/overlays) before iterating
- workflow_overlay_list catches ValueError from resolver and exits with
  code 1 instead of crashing on unhandled exceptions
- Added .specify/workflows/overlays to _reject_unsafe_workflow_storage
  chokepoint for defense-in-depth

These guards prevent symlinked overlay directories from redirecting
auto-loaded overlay YAML to attacker-controlled content outside the
project, which could inject executable shell steps into trusted workflows.

Refs: PR #3557 review comments r3594064534, r3594064563

Assisted-by: opencode-go/qwen3.7-max (autonomous)

* fix(workflows): address Copilot review findings in merge engine

- Apply inserts before winning replace to prevent anchor-not-found errors
  when replace changes step ID (r3594064604)
- Track attribution recursively for nested steps in composite inserts/replaces
  so workflow resolve attributes all child steps correctly (r3594064638)
- Add regression tests for both fixes

Refs: PR #3557 review discussion

Assisted-by: GitHub Copilot (model: qwen3.7-plus, autonomous)

* refactor(workflows): simplify overlay architecture to 2-tier

Remove installed overlays tier to enforce clean separation of concerns:
- workflow add installs workflows only (no overlay copying)
- workflow overlay add installs overlays only (project-local)

Changes:
- Remove InstalledOverlaySource class and all references
- Remove overlay-copying logic from _validate_and_install_local()
- Update WorkflowResolver to 2-tier: project overlays + base workflow
- Fix --priority override timing: apply before validation, not after
- Remove tests for installed overlays (no longer applicable)

Rationale: If upstream controls both base workflow and shipped overlays,
and both get overwritten on bundle update, there's no reason to ship
overlays separately. Overlays only make sense when someone other than
the base author adds them.

Resolves all three review findings from PR #3557:
- r3594064677: workflow add no longer copies overlays from all call sites
- r3594064705: --priority override now applied before validation
- r3594064726: no stale installed overlays (tier removed entirely)

Assisted-by: Claude (model: claude-opus-4-7, autonomous)

* fix(workflows): harden overlay symlink handling

Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(workflows): remove stale installed-overlay references from workflows.md

The 2-tier refactor (cc28185) removed the installed-overlay tier entirely,
but docs/reference/workflows.md was not updated. This commit addresses all
four Cluster 2 findings from the PR review:

- workflow add: remove sentence about copying overlays/ subdirectory
- How Overlays Work: drop installed-overlay table row and precedence prose;
  rewrite to 2-tier model (project overlays only, source-order tie-break)
- overlay remove: drop trailing sentence about installed overlays
- Interaction with Bundles: rewrite to say workflow add installs only
  workflow.yml; remove installed-overlay discovery language

Fixes: r3596368791, r3596368831, r3596368873, r3596368919

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(overlays): detect ancestor-conflict anchors in merge_steps

When two overlay edits target anchors that share a parent/descendant
relationship (e.g. remove an if-step + insert_after a nested child),
merge_steps processed them independently and in dict-insertion order,
making the outcome non-deterministic.

Add two private helpers to merge.py:
- _descendant_ids(step): returns all step IDs nested inside a step dict
  by delegating to the existing _all_base_step_ids helper on children.
- _check_anchor_conflicts(anchors, base_steps): for each targeted anchor
  finds its descendants and checks whether any other targeted anchor is
  among them; returns human-readable error strings.

Wire _check_anchor_conflicts into merge_steps immediately after
edits_by_anchor is built, before any tree mutation occurs. Raises
ValueError listing the conflicting anchor pair(s) so overlay authors
know exactly what to fix.

Add TestMergeStepsAncestorConflicts (6 cases):
- remove parent + insert_after child raises ValueError
- replace parent + remove child raises ValueError
- conflict across multiple overlays raises ValueError
- sibling anchors (not ancestor/descendant) pass
- single anchor passes
- parent targeted but child not targeted passes

Closes review comment r3596368746 (PR #3557, round 2, cluster 3).

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(overlays): fix over-broad conflict detection and non-deterministic ID collision

Finding 1.1 — _check_anchor_conflicts was rejecting any ancestor/descendant
anchor pair, including insert-only edits that are perfectly safe. Only
replace/remove on an ancestor can destroy its subtree and make a descendant
anchor unresolvable. Change the signature to accept a dict[str, str]
(anchor → winning operation) and skip the check for insert_after/insert_before.

Finding 1.2 — merge_steps was calling find_step on the already-mutated tree,
so a replacement step that reused a base step ID could be accidentally targeted
by a later edit group (non-deterministic result depending on dict iteration
order). Replace the anchor-group loop with a single-pass _traverse_and_apply
that walks the original tree structure and applies edits as each step is
encountered. Anchors are never re-looked up in a mutated tree.

Design invariant enforced: overlays always apply to the original base tree and
cannot target steps introduced by other overlays. Non-remove edits on non-base
anchors now raise ValueError early.

Also removes apply_edit (no production callers, only tested in isolation) and
its test class — the new traversal inlines the same mechanics without the
find_step round-trip.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(overlays): reject ID trailing newlines and reuse existing .yaml path

Fix two input validation bugs in the overlay layer (Group 2 of copilot
review PR #3557):

1. _validate_safe_id in schema.py used re.match() which anchors only at
   the start of the string, so IDs like 'overlay\n' passed validation
   and could produce newline-containing file paths. Changed to fullmatch()
   so the entire string must satisfy the pattern.

2. workflow_overlay_add always wrote <id>.yml without checking whether
   <id>.yaml already existed. Since the resolver loads both extensions,
   this created two active layers whose edits applied twice. Now uses
   the existing _find_overlay_file() to detect a pre-existing file and
   reuse its path, falling back to .yml only for new overlays.

Tests added for both fixes.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(overlays): fix display order inversion and wrap file-read errors

Finding group 3 from copilot-review-v2.md:

3.1 — Precedence display inverted (overlays/__init__.py)
collect_all_layers used a single-pass sort by (-priority, source_asc),
which placed the *losing* equal-priority source first in the display
while claiming "highest first". Fix: two-pass stable sort — source
descending then priority descending — so the actual winner (last applied
by the composer) rises to the top of the display.

3.2 — Unwrapped file-read errors (overlays/layer_sources.py)
Only yaml.YAMLError was caught around path.read_text(), so an
unreadable or non-UTF-8 overlay produced a raw traceback. Fix: widen
the except clause to (yaml.YAMLError, OSError, UnicodeDecodeError),
matching the pattern used throughout catalog.py.

Tests:
- test_workflow_resolve_equal_priority_winner_shown_first: verifies
  project:zzz (the winner) appears before project:aaa in workflow resolve
  output when both overlays share the same priority.
- tests/workflows/test_overlay_layer_sources.py (new): OSError and
  non-UTF-8 bytes both produce OverlayLoadError, not raw tracebacks.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: rename misleading overlay test

Assisted-by: GitHub Copilot (model: gpt-5.3-codex, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: remove EOF blank line in overlay resolver

Assisted-by: GitHub Copilot (model: gpt-5.3-codex, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix: handle overlay read and enumeration errors

Assisted-by: GitHub Copilot (model: gpt-5.3-codex, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(overlays): validate resolver workflow IDs

Reject unsafe and reserved workflow IDs before overlay or base sources construct paths, preventing traversal through resolver and engine fallback paths.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix(overlays): drop _remove_sources_recursively from remove branch

In _traverse_and_apply, the remove branch called _remove_sources_recursively
to clean up attribution entries for the deleted step.  This was inherited from
the old apply_edit loop (c70a5d6) where it was needed because the sources dict
was queried exhaustively.

In the current single-pass design, _build_attribution only traverses the result
list, so stale sources entries for removed steps are never read.  The cleanup
call is therefore unnecessary — and actively harmful when another overlay has
replaced a different step with a new step that reuses the same ID: the pop
clobbers the replacement's attribution entry, causing workflow resolve to report
the surviving step as 'unknown'.

Fix: simply remove the _remove_sources_recursively call from the remove branch.
Add an attribution assertion to the existing reused-ID regression test to catch
this case.

Fixes: r3604242050 (Copilot review finding)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

* Fix CLI overlay ID validation anchoring

Use fullmatch for CLI workflow/overlay ID validation so trailing newlines are rejected consistently with manifest validation.

Add regression coverage for newline-suffixed workflow and overlay IDs in overlay set-priority.

Assisted-by: GitHub Copilot (model: gpt-5.3-codex, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): validate workflow_id in layer sources before path construction

ProjectOverlaySource.collect() and BaseWorkflowSource.collect() joined
workflow_id directly onto storage paths without validation, enabling
path traversal (e.g. '../../outside') when called outside the
WorkflowResolver.

Add _validate_workflow_id() to layer_sources.py — mirrors the same
_SAFE_ID_PATTERN / _RESERVED_WORKFLOW_IDS check used by WorkflowResolver
in overlays/__init__.py — and call it at the top of both collect()
methods before any path is constructed.

Adds parametrised tests covering unsafe IDs and verifying no filesystem
access occurs for an invalid ID.

Closes review finding r3604772700.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): align layer source validation with _safe_workflow_id_dir

Plan §4.1 requires that Workflow-ID-Validierung, Symlink-/Containment-
Prüfungen and Fehlerübersetzung must not diverge between workflow
management and the overlay resolver.

My previous fix added ID pattern + reserved-name validation to both
collect() methods but was missing the containment step and the
BaseWorkflowSource directory/file checks that _safe_workflow_id_dir
performs.

Changes:
- Add _ensure_contained_dir(path, root) to layer_sources.py — pure
  domain mirror of overlays/_commands.py::_ensure_contained_dir that
  raises OverlayLoadError instead of typer.Exit
- ProjectOverlaySource.collect(): replace two inline symlink/dir checks
  with _ensure_contained_dir(workflow_overlay_dir, self.overlays_dir),
  adding the missing resolve().relative_to() containment step
- BaseWorkflowSource.collect(): add _ensure_contained_dir on the
  workflow directory, and add workflow.yml symlink check before is_file()

The same logic now lives in three places (workflow CLI, overlay CLI,
layer sources). The DRY extraction to workflows/_validation.py is
deferred to PR 3 per plan §4.1.

Tests: add containment and symlink tests for both sources.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix(overlays): resolve identity from manifest field, not filename

Align overlay identity resolution with the project-wide convention:
presets use preset.id, extensions use extension.id, workflows use
workflow.id, and workflow steps use step.type_key. Overlays must
derive identity from the manifest id field, not the filename.

Rewrite _find_overlay_file() to scan all YAML files in the overlay
directory and match on the manifest id field, fixing the bug where
enable/disable/remove/set-priority failed when filename != manifest id.

Closes: PR #3557 discussion r3605010197

Assisted-by: opencode-go/qwen3.7-max (autonomous)

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix(workflows): make validation behavior consistent across YAML loading paths

Address PR #3557 review finding r3607632921:

- Wrap yaml.YAMLError → ValueError in from_yaml() and from_string() so
  malformed YAML matches the documented exception contract
- Add except ValueError to workflow_info to handle composition errors
  cleanly instead of crashing with a raw traceback
- Remove validate_workflow() from compose() so the resolver path is
  parse-only like all other YAML loading mechanisms; callers validate
  explicitly via engine.validate()
- Update test to reflect new behavior: resolve() returns composed
  definition, caller validates separately

Assisted-by: opencode-go/qwen3.7-max (autonomous)

* fix(overlays): list disabled overlays in management view

Keep disabled overlays visible in workflow overlay list while leaving resolution behavior unchanged.

- add an include_disabled opt-in to overlay source/resolver collection
- use include_disabled=True for workflow overlay list
- add regression tests for list visibility and default filtering

Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* docs: align overlay extends and resolver contract

Assisted-by: GitHub Copilot (model: GPT-5.3-Codex, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: use atomic write for overlay file updates to prevent hard-link attack

Replace in-place write_text() calls in workflow_overlay_add() and
_update_overlay_field() with the same mkstemp → write → os.replace()
pattern used by the workflow installer (_stage_workflow_file /
_commit_workflow_file / _discard_staged_workflow_file).

The prior code rejected symlinks and validated path containment, but a
hard-linked destination file passes both checks while sharing an inode
with an external file. write_text() would then truncate and overwrite
that external inode. The atomic staging approach never opens the
existing destination for writing, eliminating the hard-link vector.

Fixes findings r3608669512 and r3608669517 on PR #3557.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, supervised)

* fix(composer): preserve invalid base definition instead of coercing steps to []

When 'steps' is not a list, returning early with the unmodified
WorkflowDefinition lets validate_workflow surface the proper error
("'steps' must be a list.") to the caller. The previous silent
coercion to [] masked the validation error entirely.

Fixes: https://github.com/github/spec-kit/pull/3557#discussion_r3608669506

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, supervised)

* fix: align workflow overlay priority semantics

Assisted-by: GitHub Copilot (model: GPT-5.6 Terra, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: validate overlay priority presentation

Assisted-by: GitHub Copilot (model: GPT-5.6 Terra, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: catch OverflowError in normalize_priority for float infinity values

YAML values like `priority: .inf` parse to float('inf'), causing
int() to raise OverflowError. This broke validate_overlay_yaml()'s
'validation never raises' contract. Adding OverflowError to the
except clause makes it fall back to the default priority (10),
consistent with other invalid value handling.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Markus <markus@example.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
 2026-07-21 08:35:25 -05:00
Roland Huss
6688b447b7 feat(workflows): expose workflow source directory to steps (#3469) 
* feat(workflows): expose workflow source directory to steps (#3467)

Propagate WorkflowDefinition.source_path to steps via
{{ context.workflow_dir }} in template expressions and
SPECKIT_WORKFLOW_DIR env var for shell steps. The original
source directory is persisted in state.json so resume
restores the correct value instead of the run-directory copy path.

Closes #3467

Assisted-By: 🤖 Claude Code

* fix: apply bot review suggestions (#2)

Applied fixes from bot review comments:
- Comment #3563319058: prevent stale SPECKIT_WORKFLOW_DIR leak from parent env
- Comment #3563319094: use cross-platform Python one-liner instead of printenv
- Comment #3563319103: add monkeypatch.delenv for deterministic env var test
- Comment #3563319116: same env leak fix as #3563319058

Assisted-By: 🤖 Claude Code

* fix: use YAML single-quotes and forward-slash paths for Windows CI (#2)

sys.executable on Windows returns backslash paths (D:\a\...) which YAML
double-quoted strings interpret as escape sequences. Switch to
single-quoted YAML strings and normalize paths with replace("\\", "/").

Assisted-By: 🤖 Claude Code

* fix: resolve workflow_dir to absolute path and add installed-by-ID test (#3469)

Applied fixes from bot review comments:
- Comment #3563382853: resolve source_path before taking parent to ensure absolute paths
- Comment #3563382864: add test for installed-by-ID workflow_dir semantics

Assisted-By: 🤖 Claude Code

* docs: document context.workflow_dir and SPECKIT_WORKFLOW_DIR

Add reference documentation for the new workflow_dir runtime value in
both workflows/README.md and docs/reference/workflows.md so workflow
authors can discover the feature and its semantics.

Assisted-By: 🤖 Claude Code

* fix: clarify installed workflow_dir is an absolute path (#3469)

The documentation for context.workflow_dir described the installed-by-ID
case as ".specify/workflows/<id>/" which appears relative, contradicting
the "resolved absolute path" semantics. Clarified that it is the absolute
path to the installation directory.

Assisted-By: 🤖 Claude Code

* fix: apply bot review suggestions (#3469)

Applied fixes from bot review comments:
- Comment #3580005128: Quote sys.executable in shell step env var test
- Comment #3580005174: Quote sys.executable in no-env-var test

Assisted-By: 🤖 Claude Code

* fix: apply bot review suggestions (#3469)

Applied fixes from bot review comments:
- Comment #3587146944: Quote interpolated workflow_dir path in example

Assisted-By: 🤖 Claude Code
 2026-07-15 08:09:35 -05:00
Marsel Safin
d7b6626218 feat(workflows): align workflow CLI with extension command surface (#3419) 
* feat(workflows): align workflow CLI with extension command surface

Adds the missing workflow commands and flags so the workflow CLI
matches the extension/preset pattern: add --dev and --from, search
--author, update, enable and disable. Disabled workflows are blocked
from running and marked in list output.

Fixes #2342

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): preserve disabled state on update, guard corrupted registry entries

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): guard list against corrupted registry entries, re-raise typer.Exit in catalog install

workflow list now skips non-dict registry entries with a warning instead
of crashing, matching update/enable/disable. The broad except in
_install_workflow_from_catalog no longer swallows typer.Exit, so precise
errors like the non-HTTPS redirect message are not duplicated.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): escape rich markup in id-mismatch errors and validate --from source early

The two id-mismatch error paths interpolated repr() into Rich markup, so
a stray bracket in a user typo could be parsed as markup. Route both
through rich.markup.escape.

`workflow add <source> --from <url>` also validated the source only
after downloading. Validate it up front so a URL/path/typo fails
without a network fetch.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): escape rich markup in list output and catalog install errors, isolate update failures

workflow list now escapes id/name/version/description before printing,
matching how extensions render user-editable fields. The catalog install
helper computes safe_wf_id once and uses it for every early error path
plus the final failure message.

workflow update wraps _safe_workflow_id_dir and the backup read inside
the try/except typer.Exit block so an unsafe id in a corrupted registry
fails that one workflow and the rest continue.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): escape rich markup in --from download exception message

Matches how the catalog install path escapes exception strings.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): catch OSError in per-workflow update loop and make restore best-effort

Transient FS errors (perms, disk full) from backup read or write no
longer abort the whole update run. The restore is wrapped in its own
try/except so a failed write only warns, and the offending workflow
is reported via 'Failed to update' like other per-workflow failures.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): escape rich markup in search output

workflow search now escapes catalog-derived name/id/version/description/
tags before printing, matching extension search and workflow list.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: escape workflow validation errors before Rich output

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): escape remaining unescaped Rich markup paths

Covers the last few review threads not yet addressed:
- Escape yaml.YAMLError text in the local workflow add install path
  (matches the already-escaped download/catalog paths).
- Escape the non---dev local directory fallback's "No workflow.yml
  found in <path>" message (the --dev branch already escaped it).
- Escape the redirected final_url in the --from non-HTTPS redirect
  error (IPv6 literals like http://[::1]/... are legal and contain
  brackets).
- Escape the "Downloaded workflow is invalid" exception message in
  _install_workflow_from_catalog, matching the sibling catalog-install
  exception handler a few lines above it.

Adds regression tests for each in TestWorkflowCliAlignment, following
the existing escaping-test pattern in this class.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): escape workflow name/id in install success messages

Workflow names and ids come from user-controlled YAML or external catalog
data; printing them unescaped lets bracket characters be interpreted as
Rich tags. Escape them in the add/catalog-install success messages and the
remaining catalog error paths, matching the rest of the output hardening.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): fail cleanly on unparseable catalog install URLs

urlparse raises ValueError on e.g. an unbalanced IPv6 literal before the
invalid-URL branch is reached; on workflow update that also bypassed the
per-workflow handler and aborted the whole command. Convert the parse
failure into a clean error so add fails cleanly and update skips just the
affected workflow.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): reject catalog updates whose downloaded version mismatches

The update path never verified the downloaded workflow carries the catalog
version that triggered the update, so a stale or misconfigured URL could
report success while leaving the old version installed or downgrading it.
Pass the expected version into the install helper and fail the update when
the downloaded definition does not match.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): validate workflow ID in run command and document new CLI flags

Path-equivalent spellings like "align-wf/" previously bypassed the
registry disabled check because the engine normalizes the path while the
registry matches the raw string. workflow run now validates non-file
sources against the workflow ID pattern before lookup.

Also updates docs/reference/workflows.md with --dev/--from install
options, update/enable/disable commands, and the search --author flag.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): enforce disabled state for direct paths to installed workflows

Running the installed copy's YAML directly (specify workflow run
.specify/workflows/align-wf/workflow.yml) skipped the registry check.
File sources resolving inside .specify/workflows/<id>/ now map back to
the workflow ID and refuse to run while disabled.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): reject explicit empty --from URL instead of catalog fallback

'workflow add foo --from ""' fell through 'from_url or ...' to a
catalog install. Distinguish None from empty string so explicit values
stay on the URL-validation path and fail closed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): registry rollback on save failure, consistent disabled check, honest update summary

- WorkflowRegistry.add now rolls back its in-memory mutation when save()
  raises, so a later successful save cannot persist metadata for a
  failed update alongside the restored YAML backup.
- workflow run uses the same truthiness check for 'enabled' as list and
  disable, so malformed values like 0 or null refuse to run.
- workflow update reports 'No workflows were eligible for update' when
  every target was skipped instead of claiming all are up to date.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): guard non-string catalog URL and keep enable/disable rollback intact

- A truthy non-string catalog url (e.g. 123) reached urlparse and raised
  AttributeError, escaping the clean error path; validate it is a string.
- enable/disable mutated the live registry entry before add(), so add's
  rollback snapshot captured the already-toggled object; pass a fresh
  mapping instead.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): tolerate non-dict registry entries in add and clarify test docstrings

A corrupted-but-parseable registry entry (e.g. a string value) crashed
WorkflowRegistry.add with AttributeError on existing.get. Guard the
non-dict case while still restoring the original raw value on rollback.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): atomic registry save and accurate mixed-target update summary

- save() wrote the registry with open('w'), so a failed dump truncated
  the file and the next load reset every entry. Write to a sibling temp
  file and os.replace into place.
- workflow update no longer claims all workflows are up to date when
  some targets were skipped; it reports checked-only status with a
  skipped count.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): exclusive temp file for registry save and cwd-independent disabled guard

- save() now uses tempfile.mkstemp in the workflows dir (matching the
  engine's atomic writer), so a pre-created symlink at a predictable
  .tmp path cannot redirect the write and concurrent processes cannot
  collide.
- The direct-path disabled guard derives the owning project from the
  resolved file path instead of the caller's cwd, so running an
  installed workflow's YAML from outside the project still refuses when
  disabled.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): symlink guards and shape validation in workflow registry, dev-dir file check

- WorkflowRegistry now mirrors StepRegistry: _load refuses symlinked
  parents/registry file and normalizes a non-dict workflows field;
  save() rejects symlinked paths before writing.
- workflow add --dev requires workflow.yml to be a regular file so a
  directory named workflow.yml gets the documented CLI error instead of
  an uncaught IsADirectoryError.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): validate download redirects before following them

All three workflow download sites (add --from, catalog install, step
install) passed no redirect_validator to open_url, so an HTTPS URL
redirecting to cleartext HTTP issued the insecure request before the
post-hoc geturl() check reported it. Shared validator now rejects
non-HTTPS redirects (loopback HTTP allowed) pre-follow, matching the
preset download path.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test(workflows): accept redirect_validator kwarg in step-add open_url fakes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): guard directory-shaped workflow.yml and unreadable registry

- workflow add's plain local-path fallback (no --dev) checked wf_file.exists()
  before installing, so a directory literally named workflow.yml passed the
  guard and _validate_and_install_local() leaked an uncaught
  IsADirectoryError instead of the documented CLI error. Use is_file(),
  matching the --dev branch's existing guard.
- WorkflowRegistry._load() treated any OSError while reading an existing
  registry the same as corrupted JSON, resetting to an empty in-memory
  registry. A later save() would then silently persist that empty state via
  os.replace, discarding every previously installed workflow entry. Track a
  _load_error flag on OSError-during-read and have save() refuse to write
  when it is set, so a transient I/O failure can no longer overwrite intact
  data on disk.
- docs/reference/workflows.md: document `--from <url>` with its value
  placeholder, matching extensions.md and presets.md.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflow): rollback registry.remove() and guard OSError at CLI boundaries

Critical: WorkflowRegistry.remove() deleted the in-memory entry then
called save() with no rollback, unlike add(). Combined with
workflow_remove deleting the workflow directory before calling
registry.remove(), a save failure permanently destroyed the workflow's
files, left the on-disk registry still claiming it installed, and
surfaced a raw unhandled OSError with no CLI message.

- WorkflowRegistry.remove() now rolls back the in-memory entry on a
  save() OSError, mirroring add()'s existing rollback pattern.
- workflow_remove persists the registry removal (registry.remove(),
  wrapped in try/except OSError -> clean escaped message) before
  deleting any files, so a save failure never touches the workflow
  directory.

Important sibling paths: workflow add (local/--dev/--from and catalog),
enable, and disable all called registry.add() without catching its
deliberate OSError, so a save failure surfaced either an orphaned
install directory (fresh local/catalog installs) or a raw/unhandled
exception with no clean CLI output.

- _validate_and_install_local (backs local/--dev/--from) now removes
  the freshly created directory on a fresh install, or restores the
  prior workflow.yml bytes on a reinstall-over-existing-local install,
  before raising a clean escaped error.
- _install_workflow_from_catalog wraps the final registry.add() using
  the function's own established convention (rmtree the just-downloaded
  workflow_dir, then a clean escaped error) -- workflow_update's
  existing backup/restore around this function is unaffected.
- workflow_enable/workflow_disable catch registry.add()'s OSError and
  print a clean escaped message instead of leaking the exception.

Added failing-first tests proving each behavior (registry-unit rollback
test, CLI-level remove/add/enable/disable save-failure tests
parametrized where they share one root cause), all confirmed red before
the fix and green after.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflow): preserve prior catalog install on reinstall registry-save failure

_install_workflow_from_catalog's final registry.add() failure handler
unconditionally rmtree'd workflow_dir. That's safe for a brand-new
install, but plain `workflow add <catalog-id>` also allows re-adding an
already-installed workflow, downloading the new version over the
existing directory first. If registry.add() then failed to save, the
unconditional rmtree deleted the prior working install while the
registry (after its own rollback) still reported it installed -- data
loss with no way back. workflow_update already avoids this via an outer
backup/restore around this function, but plain add has no such caller.

Fix mirrors _validate_and_install_local's existed-before/backup-aware
handling: capture whether workflow_dir existed and back up its
workflow.yml bytes before any download write, then on a registry.add()
OSError, restore those bytes for a reinstall or rmtree only a
brand-new directory. Only one file (workflow.yml) is ever written by
this path, so no further per-file bookkeeping is needed.

Added a failing-first regression: install a catalog workflow, re-add it
with a simulated registry save OSError, and assert a clean error, the
original workflow.yml restored byte-for-byte, and the registry still
reporting the original version installed. Confirmed red (prior file
deleted) before the fix, green after.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflow): centralize catalog-install cleanup across all failure branches

_install_workflow_from_catalog is new in this PR and has seven failure
branches after the mkdir/download step, each independently rmtree'ing
workflow_dir: redirect-to-non-HTTPS rejection, a generic download
exception, invalid downloaded YAML, a validate_workflow failure, a
workflow-id/catalog-key mismatch, a version mismatch, and (fixed in the
prior commit) a registry.add() OSError. Only the last one had been
special-cased to spare a prior working install on reinstall; the other
six still unconditionally deleted the whole directory, so re-adding an
already-installed catalog workflow and hitting any of those six earlier
failures destroyed the working install even though nothing about it had
actually changed.

Replaced all seven ad hoc rmtree call sites with a single local
_cleanup_failed_install() helper that closes over the existed_before /
prior_workflow_bytes captured once at the top of the function: restore
the prior workflow.yml for a reinstall, or rmtree only a directory that
this attempt itself created. Every failure branch now calls this one
helper, so the fix is structural rather than duplicated, and every
existing error message/exit code is unchanged -- only the cleanup
performed before each message is different.

Added a parametrized regression test covering the four early-failure
trigger points reachable from plain workflow add (redirect rejection,
download exception, invalid YAML, ID mismatch): each installs a catalog
workflow, re-adds it while forcing that specific failure, and asserts a
clean error plus the original workflow.yml surviving byte-for-byte.
Confirmed red against the unfixed code (all four raised FileNotFoundError
reading the deleted file) before applying the helper, green after.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflow): restore registry entry verbatim on post-removal rmtree failure

workflow_remove now persists registry.remove() before deleting any
files (fixed previously), but if the registry write succeeds and the
subsequent shutil.rmtree(workflow_dir) then fails, the registry was
left claiming the workflow uninstalled while its directory remained on
disk -- an orphaned install with no path back to a clean state.
workflow_step_remove already handles this exact sequencing by capturing
the registry entry before removal and restoring it directly into
registry.data plus save() (bypassing add(), which would stamp a new
updated_at) if the directory removal fails afterwards.

Applied the same pattern to workflow_remove: capture registry_metadata
via registry.get() before registry.remove(), and on an rmtree OSError,
write it straight back into registry.data["workflows"][workflow_id] and
save(), matching workflow_step_remove's restore-failure handling (a
yellow warning, not a hard failure, since the primary error is already
about to be reported). Existing error message and exit behavior for the
rmtree failure are unchanged.

Added a failing-first regression: install a workflow, monkeypatch
shutil.rmtree to raise OSError, and assert a clean existing error
message, the directory remaining (rmtree never actually deleted
anything), and the registry entry restored byte-for-byte identical
(including installed_at/updated_at) -- proving the fix bypasses add()
and doesn't re-stamp timestamps. Confirmed red (registry entry stayed
None) before the fix, green after.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix 4 current Copilot review findings on workflow run/registry/install

1. workflow run ownership check followed symlinks via Path.resolve()
   before mapping a direct YAML path back to its installed workflow ID.
   A symlinked .specify/workflows/<id>/workflow.yml resolved outside the
   tree, missed the ownership match entirely, and let the disabled-workflow
   guard be silently skipped while engine.load_workflow still followed the
   symlink. Now maps ownership from a lexically-normalized path (os.path.
   normpath, no symlink following) and explicitly refuses to run if the
   installed <id> directory or workflow.yml leaf is itself a symlink.
   Direct external workflow paths that don't match .specify/workflows/...
   are unaffected.

2. WorkflowRegistry._load() caught a read OSError and silently fell back
   to an empty in-memory registry, only blocking a later save(). Callers
   that only query is_installed()/get()/list() before writing a file
   (e.g. commands/init.py's bundled speckit install, which overwrites
   workflow.yml once is_installed() reports false) could act on that
   false-empty state and destroy real data before ever reaching save().
   _load() now raises OSError immediately so an unreadable registry fails
   closed at construction, before any query or side effect is possible.
   Added _open_workflow_registry() to give every CLI command a consistent
   clean-error boundary around registry construction.

3. _validate_and_install_local's mkdir/copy2 ran before the try/except
   that protected registry.add(); a copy2 failure (e.g. a truncating
   partial write on a reinstall) was not caught at all, so the existing
   backup-restore cleanup never ran and the prior working workflow.yml
   was corrupted with a raw traceback surfaced to the user. mkdir/copy2
   now run inside the same rollback-protected section as registry.add(),
   sharing one _cleanup_failed_install() helper.

4. workflow update's skip message claimed any non-catalog source was
   installed "from a local path or URL", which is wrong for the bundled
   speckit workflow (source: "bundled"). Message is now source-neutral.

Verified all 4 threads are current (not outdated) via GraphQL review
thread query on PR #3419, HEAD 812050a.

Tests: strict TDD per fix (red test proving each bug, minimal production
change, green). tests/test_workflows.py: 474 passed. Full suite: 3976
passed, 110 skipped. ruff check: all checks passed on touched files and
full src tree.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix disabled-workflow bypass via symlinked .specify project root

workflow run's ownership check derived registry_root/registered_id from
the lexical path, then checked the id directory and workflow.yml leaf for
symlinks -- but never checked .specify or .specify/workflows themselves
for that derived root. _reject_unsafe_workflow_storage only guards the
cwd's project_root, which can differ from the path-derived registry_root
(a direct path into an unrelated project, or that project's own .specify
being a symlink to an attacker-controlled tree). WorkflowRegistry's own
symlinked-parent handling silently substitutes an empty registry instead
of raising, so a query against it (is_installed/get returning "not
found") is not a safety signal a caller can rely on: with a symlinked
.specify, the disabled check saw no registry entry and let a disabled
workflow run anyway.

Fix: reject an unsafe .specify/.specify-workflows for the actual derived
registry_root before ever consulting the registry, reusing the existing
_reject_unsafe_dir helper already used by _reject_unsafe_workflow_storage.

Red-first end-to-end repro: victim project's .specify symlinked to an
attacker-controlled tree containing a disabled workflow entry, run
invoked with a direct path from an unrelated cwd -- confirmed the
disabled workflow executed (exit 0) before the fix, now refused cleanly.

Tests: tests/test_workflows.py 475 passed. Full suite: 3977 passed, 110
skipped. ruff check: all checks passed.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix raw exception leak in bundle remove primitive boundary

remove_bundle() had no exception handling around its component
removal loop, unlike install_bundle() which converts any raw
exception into a clean BundlerError. Since WorkflowRegistry now
fails closed (raises OSError) on an unreadable registry file,
and _WorkflowKindManager.__init__ constructs WorkflowRegistry
with no try/except, an unreadable workflow registry surfaced as
a raw OSError through remove_bundle(). The bundle_remove CLI
command only catches BundlerError, so the raw OSError propagated
uncaught, producing exit_code=1 with empty output instead of a
clean, actionable message.

Wrap remove_bundle()'s component loop in the same
try/except BundlerError: raise / except Exception: raise
BundlerError(...) from exc pattern already used by
install_bundle(), converting any raw exception at this shared
boundary. save_records() remains outside the try block, so a
failure still leaves the bundle's record untouched (no removal
side effects recorded).

Tests:
- tests/integration/test_bundler_install_flow.py::test_remove_converts_raw_installer_exception_to_bundler_error
  (function-level regression: a raw OSError from installer.is_installed
  must become a clean BundlerError, and the bundle record must survive)
- tests/contract/test_bundle_cli.py::test_remove_reports_clean_error_when_primitive_raises_raw_exception
  (CLI-level regression: `specify bundle remove` must print a clean
  actionable message and exit non-zero instead of raw/empty output)

Both tests were confirmed red beforehand: the raw OSError propagated
uncaught out of remove_bundle(), and the CLI-level CliRunner result
showed exit_code=1 with empty output.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix 8 current Copilot review findings on registry fail-closed, rollback orphans, backup-read boundaries, and Rich escaping

1. WorkflowRegistry._load(): a symlinked .specify/.specify/workflows
   parent (or a symlinked registry file) silently returned an empty
   registry instead of raising, unlike an unreadable-file read failure.
   A read-only caller (notably the bundler's remove path) querying
   is_installed() before ever writing could conclude an installed
   workflow is absent, skip removing it, then delete the bundle
   record -- leaving the workflow untracked but still on disk. Now
   raises OSError immediately, matching the existing unreadable-file
   fail-closed behavior.

2/8. _validate_and_install_local and _install_workflow_from_catalog:
   when the destination directory already existed but had no prior
   workflow.yml (e.g. a leftover empty dir), existed_before was True
   but there were no backup bytes to restore, so the rollback closure
   did nothing on a later failure -- leaving the newly copied/
   downloaded file behind. Both now unlink the newly created file in
   this case, restoring the pre-existing directory to its prior
   (empty) state.

3/4. Both install paths read the prior workflow.yml bytes (to seed
   the reinstall rollback) *before* any try/except boundary: a read
   failure on the existing file (e.g. a transient permission/FS
   issue) leaked a raw, unescaped OSError instead of the same clean
   CLI error used by every other failure branch in these functions.
   Both reads are now guarded by their own try/except OSError, with
   no writes attempted before the read succeeds (so there is nothing
   to roll back on this specific failure).

5. remove_bundle's exception-conversion message unconditionally
   claimed "No changes were recorded," even though a failure can
   occur after earlier components in the same bundle have already
   been removed from disk (save_records never runs on this path, so
   the record is left claiming the bundle fully installed). The
   message now reports how many components were already removed
   when that happened, instead of asserting no changes occurred.

6/7. workflow_remove's new post-registry-removal directory-failure
   error and its restore-failure warning interpolated workflow_dir
   and the exception values into Rich markup unescaped. A project
   path or OS error message containing Rich-markup-like brackets
   could be parsed as markup and hide/corrupt the displayed text.
   Both now use the existing _escape_markup helper, consistent with
   every other error path in this file.

Tests (tests/test_workflows.py unless noted):
- TestWorkflowRegistry::test_load_symlinked_workflows_dir_fails_closed_not_silently_empty (1)
- TestWorkflowCliAlignment::test_add_dev_fresh_install_into_preexisting_empty_dir_cleans_new_file (2)
- TestWorkflowCliAlignment::test_add_catalog_fresh_install_into_preexisting_empty_dir_cleans_new_file (8)
- TestWorkflowCliAlignment::test_add_dev_reinstall_backup_read_failure_gives_clean_error (3)
- TestWorkflowCliAlignment::test_add_catalog_reinstall_backup_read_failure_gives_clean_error (4)
- tests/integration/test_bundler_install_flow.py::test_remove_partial_failure_message_reflects_partial_state (5)
- TestWorkflowRemoveGuard::test_remove_directory_and_restore_failure_escapes_rich_markup (6/7)

All seven were confirmed red beforehand, matching each thread's
described failure mode exactly (silent empty registry instead of a
raise; orphaned new file left behind; raw unescaped OSError leaking;
a misleading "no changes were recorded" claim; Rich markup consuming
bracketed path/exception text). Also updated
test_registry_save_refuses_symlinked_parent, a pre-existing test that
asserted the symlinked-parent raise at add()/save() time -- it now
raises at construction instead, per fix #1, so the test was adjusted
to match without weakening its guarantee (still asserts no writes
occur under the symlinked target).

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix 3 current Copilot review findings: bookkeeping-aware BundlerError removal, bounded workflow downloads

1. bundle remove: BundlerError raised by the primitive installer itself
   (e.g. from a kind manager) bypassed the partial-removal bookkeeping
   message added previously via a bare `except BundlerError: raise`. Now
   routes through the same detail-construction logic as generic
   exceptions, so a mid-loop BundlerError after an earlier successful
   removal still reports that the project may be partially uninstalled,
   while a zero-removal BundlerError still reports "No components were
   removed." Both preserve the original exception message and chain
   `from exc`.

2/3. workflow add --from and catalog install/update downloads used
   unbounded `response.read()`, buffering the entire server-controlled
   body into memory before any size check, and trusted Content-Length
   alone where checked at all. Added a single shared
   `_read_response_within_limit()` helper reused by both call sites: it
   fails fast on an oversized declared Content-Length, and separately
   enforces the same cap while streaming in 64KiB chunks so a chunked or
   Content-Length-less response cannot bypass the limit by lying about or
   omitting its size. Chose 5 MiB as the cap: workflow YAML definitions
   are small step/metadata text, not binaries, so this is generous
   headroom against a malicious/misbehaving server without affecting any
   legitimate workflow definition. Both call sites already route any
   raised exception through their existing clean-error and rollback
   (`_cleanup_failed_install`) paths, so no additional error-handling
   plumbing was needed.

Tests: extended the shared `_FakeResponse` test helper (and 5 duplicate
per-test FakeResponse classes) to support `.read(amt)` chunked reads with
an internal cursor (backward compatible with existing bare `.read()`
callers) plus header simulation. Added red-first tests for: BundlerError
after partial removal reporting partial state, BundlerError with zero
removals reporting no changes, --from oversized-Content-Length rejection,
--from oversized-streamed-body-without-Content-Length rejection, and the
same two cases for the catalog install path (asserting no orphan
directory/registry mutation on rejection).

tests/integration/test_bundler_install_flow.py: 17 passed
tests/test_workflows.py: 485 passed
tests -q: 3992 passed, 110 skipped
ruff check: clean on all touched files

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix temp-file leak in workflow add --from and strengthen size-limit test assertions

workflow_add's --from download path opened a NamedTemporaryFile(delete=False)
-- which creates the file on disk immediately -- then wrote the size-limited
response body before assigning `tmp_path`. If `_read_response_within_limit`
raised (oversized declared Content-Length, or an over-cap streamed body with
no/understated Content-Length), the exception propagated out of the `with`
block before `tmp_path` was ever set, so the outer except handler had no
path to clean up: a 0-byte `.yml` temp file was left behind permanently on
every rejected/failed --from download. Fixed by assigning `tmp_path`
immediately after the file is opened (before the size-limited read/write),
and unlinking it in the except branch when set. Normal post-download cleanup
in the existing `finally: tmp_path.unlink(missing_ok=True)` is unchanged.

Verified (not assumed) the catalog install path has no equivalent leak: it
writes the response bytes directly to `workflow_file` inside `workflow_dir`
(no separate temp file), and any read/size-limit failure is already caught
by the existing `except Exception: _cleanup_failed_install()` handler, which
correctly restores a reinstalled file or removes a freshly-created directory.

While investigating, found the previous round's 4 size-limit tests were
false positives: `_read_response_within_limit`'s `max_bytes` parameter had
its default bound to `_MAX_WORKFLOW_YAML_BYTES` at function-definition time,
so monkeypatching the module attribute in tests had no effect on the
function's actual behavior -- the tests were passing because the oversized
mock bodies failed downstream YAML/id validation instead of the size check.
Fixed by resolving `max_bytes` from the module attribute at call time
(default `None`, resolved inside the function body) so tests can actually
override the effective limit, and strengthened all 4 tests' assertions to
match the specific size-limit error text (whitespace-collapsed to tolerate
Rich's line-wrapping), so they now prove the real code path fires.

Tests: added 2 red-first regression tests (oversized-streamed-body and
oversized-Content-Length --from downloads leave no leftover temp file,
verified against a scratch tempfile.tempdir), confirmed red (real 0-byte
file found) before the fix and green after. Strengthened the pre-existing
4 --from/catalog size-limit tests to assert on the actual error message
instead of generic exit-code/non-empty-output checks.

tests/test_workflows.py: 487 passed
tests -k bundler: 186 passed
tests -q: 3994 passed, 110 skipped
ruff check: clean on all touched files

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Harden workflow install/remove transactions with atomic staging

Addresses 5 Copilot review findings on HEAD b8269c8, all centered on
transaction integrity around workflow install/remove/registry writes,
following the atomic_write_json pattern already used in _utils.py:

1. WorkflowRegistry.save() now preserves the existing registry file's
   mode (e.g. 0640/0644) across a save instead of silently downgrading
   it to mkstemp's 0600 default; a brand-new registry still gets the
   secure 0600 default.

2. workflow_remove now stages the install directory out of the way via
   an atomic rename *before* the registry write, rather than deleting
   it directly with shutil.rmtree after the registry already claims it
   removed. This closes a real data-integrity gap: a partially-failed
   rmtree could no longer leave a damaged directory re-marked
   "installed" by the old manual restore-after-rmtree-failure code
   (now deleted -- it's structurally impossible to need it). A
   registry-write failure renames the staged directory back
   (guarded, with an explicit warning if the restore-back rename
   itself fails); a registry-write success is durable, so a later
   failure to delete the staged directory is now a warning (exit 0),
   not a contradictory "Error: Failed to remove" (exit 1) that used to
   claim failure while the registry already recorded success.

3. Local (--dev/--from/plain path) and catalog install/reinstall now
   write new content to a same-directory staging file and commit it
   onto the destination workflow.yml via a single atomic swap, instead
   of writing/downloading directly into the destination file. A prior
   file (reinstall) is renamed aside rather than overwritten in place,
   so it can be restored via rename -- never a content rewrite -- if
   registry.add() subsequently fails; a rollback failure is now
   explicitly reported as a warning instead of escaping unguarded and
   masking the original clean error. This also removes the need to
   read the prior file's bytes into memory before installing (that
   read-before-write step and its failure mode are now unreachable),
   and both local and catalog installs share the same four small
   helpers (_stage_workflow_file / _commit_workflow_file /
   _discard_staged_workflow_file / _rollback_committed_workflow_file,
   plus guarded wrappers) rather than duplicating the logic.

4. Updated a stale comment (workflow_run's ownership-guard rationale)
   that still described WorkflowRegistry._load() as silently
   substituting an empty registry; it now fails closed by raising
   OSError, which the comment now states plainly.

Tests: rewrote the two workflow_remove tests whose assertions encoded
the old (incoherent) rmtree-then-restore contract to instead prove the
new stage-then-commit contract (post-registry-success cleanup failure
is a warning+exit 0; pre-registry-success stage-restore failure is
guarded and escapes markup correctly). Rewrote the local/catalog
"backup read failure" tests, which tested a step the new design no
longer performs, into "restore-rename failure" tests proving the new
guarded rollback boundary. Added registry file-mode preservation tests.
All other existing install/remove/reinstall tests (save-failure
cleanup, pre-existing-empty-dir handling, early-failure-during-
reinstall parametrized cases, Rich markup escaping) continue to pass
unmodified against the new implementation.

Verified via GraphQL that all 5 threads are current (not outdated/
resolved) before fixing. Full suite: 3996 passed, 110 skipped. Ruff
clean on all touched files.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Discard reinstall backup file after registry.add() succeeds

_commit_workflow_file() renames a prior workflow.yml aside to
workflow.yml.bak so it can be restored if registry.add() subsequently
fails. Neither the local install/reinstall path nor the catalog
install/reinstall path ever cleaned up that backup after a successful
registry.add() -- every successful reinstall permanently left a
workflow.yml.bak sibling, which later reinstalls would silently
overwrite/re-orphan.

Add a shared _discard_committed_backup_file() helper, called from both
success paths right after registry.add() durably succeeds (and before
the final "installed" message, preserving output ordering). A fresh
install (backup_file is None) is a no-op. A cleanup failure is reported
as a warning (exit 0), not a failure, since the install itself already
succeeded -- consistent with workflow_remove's post-commit cleanup
warning semantics.

Add red-first regression tests proving: (1) successful local reinstall
leaves no workflow.yml.bak sibling, (2) successful catalog reinstall
leaves no workflow.yml.bak sibling, (3) a cleanup failure on the backup
file after a successful reinstall reports a warning and still exits 0
with the registry correctly reflecting the new install.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Clean up freshly-created dest_dir when staging mkstemp fails

_stage_workflow_file() does dest_dir.mkdir(parents=True, exist_ok=True)
then tempfile.mkstemp(dir=dest_dir, ...). For a fresh install (no prior
directory), if mkdir succeeds but mkstemp then raises (disk
full/EMFILE/quota), the exception previously propagated straight past
both the local-install and catalog-install call sites without any
cleanup, leaving the newly-created empty workflow directory orphaned
on disk with no error indicating why.

Fix at the shared _stage_workflow_file() boundary instead of duplicating
cleanup at each call site: track whether this call created dest_dir: on
a mkstemp failure, remove that directory via a guarded rmdir (never a
broad rmtree, so any concurrently written content would be left
untouched) before re-raising the original OSError unchanged. A
pre-existing (reinstall) dest_dir is never touched by this cleanup,
and a cleanup failure is reported as its own warning without masking
the original error.

Add red-first regression tests proving: a fresh local install (--dev,
plain local path, --from) and a fresh catalog install both clean up the
orphaned directory on a simulated mkstemp failure, and a reinstall over
a pre-existing directory is left untouched by the same failure.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix installed-workflow ownership/disabled bypass and resume enforcement

Address 3 current Copilot review findings on the disabled-workflow guard
in `workflow run`/`workflow resume`:

- The lexical `.specify/workflows/<id>` ownership scan stopped at the
  first match scanning from the start of the path. A nested project
  living beneath an outer installed workflow's own directory tree (reusing
  the same segment names) was attributed to the wrong (outer) workflow
  and ID, gating the run on an unrelated workflow's disabled state.
  `_scan_for_workflow_owner` now scans from the end so the nearest
  (innermost) owner always wins.

- A path with no `.specify/workflows` segments of its own (e.g.
  `/tmp/alias.yml`) that is itself a symlink resolving *into* installed
  storage bypassed the disabled check entirely, since only the raw
  lexical path was inspected. `_resolve_installed_workflow_ownership` now
  additionally resolves the real path when the lexical scan finds no
  owner and re-runs the same scan against it, so an outward-pointing
  alias into a disabled workflow is caught too. Genuinely standalone
  external files (no symlink anywhere on the path) are unaffected.

- `workflow resume` bypassed the disabled check altogether: engine.resume()
  replays a persisted run directly from disk with no registry awareness.
  RunState now optionally persists `installed_workflow_id` and
  `installed_registry_root` at run start (set by workflow_run when the
  source resolved to an installed ID); `workflow_resume` pre-loads the
  run state and re-checks the registry's *current* disabled state before
  calling engine.resume(), mirroring workflow_run's own guard. Both new
  fields default to None via RunState.load()'s `.get()`, so runs from a
  direct/non-installed source, and any run persisted before this schema
  addition, resume exactly as before.

The ownership-mapping logic (previously inlined in workflow_run) is
extracted into `_resolve_installed_workflow_ownership` /
`_scan_for_workflow_owner` so both the lexical and resolved-path cases
share the same scan and the existing inward-symlink-component refusal.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Guard --from temp cleanup; drop redundant update rollback; mark POSIX-only tests

Two more current Copilot review findings, both in workflow_add/update:

- `workflow add --from`'s `finally: tmp_path.unlink(missing_ok=True)` ran
  unguarded after `_validate_and_install_local` had already committed the
  file and registry entry (success) or already raised its own clean
  `typer.Exit` (failure). An OSError from that cleanup unlink would
  surface as an unhandled failure even though the install itself
  succeeded. It is now wrapped in try/except OSError, printing a neutral
  warning that doesn't claim success or failure (the finally runs on both
  outcomes) instead of propagating.

- `workflow_update`'s per-item loop performed its own outer backup
  (`wf_file.read_bytes()`) and restore (`wf_file.write_bytes(backup)`)
  around `_install_workflow_from_catalog`, which is itself fully
  transactional (staged download, atomic rename-based commit, its own
  rollback on registry failure) and never leaves a raw OSError or a
  partially-written workflow.yml. The outer restore was therefore dead
  weight for its stated purpose, and — being an unguarded byte-level
  write — was itself an unnecessary place a second failure could truncate
  an already-safely-preserved file. Removed; the loop now only records
  success/failure.

Also marks 3 registry-save file-mode tests
(`test_registry_save_preserves_existing_file_mode`,
`test_registry_save_on_new_registry_uses_secure_default_mode`,
`test_registry_save_failure_preserves_file_on_disk`) as POSIX-only via
the repo's existing `skipif(sys.platform == "win32", ...)` pattern, since
they assert exact POSIX permission bits that don't hold on Windows.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Report possible partial changes on zero-removed bundle removal failure

The final Copilot review finding: `remove_bundle`'s zero-removed-components
error message claimed "No components were removed." even when the failing
installer component may have deleted files before raising -- prior review
rounds already established that DefaultPrimitiveInstaller's removal paths
are not atomic and can leave partial filesystem changes despite raising
before `result.uninstalled` is populated. The zero-count message is now a
conservative caution ("...but the failing component may have made partial
changes before raising, so the project may be partially uninstalled.")
instead of an unconditional claim of no side effects. The >0-removed path
(which already reports the confirmed partial list) is unchanged.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix workflow resume disabled-check bypass after project move/rename

RunState.installed_registry_root previously persisted the creation-time
absolute project path unconditionally whenever a run belonged to an
installed workflow. After the whole project directory was renamed or
moved, workflow_resume would open a WorkflowRegistry at that now
nonexistent path, get back an empty/default registry, and silently skip
the disabled-workflow check -- a paused run for a disabled workflow could
be resumed successfully from the new location.

Fix persists installed_registry_root only when the owning root genuinely
differs from the current project_root (true cross-project direct-file-
source invocations). The common same-project case now persists None and
is re-derived from the live project_root at resume time via a new
_resolve_run_owner_root() helper, which also falls back to project_root
if a stored root no longer exists on disk -- covering both the common
case transparently surviving project moves and the cross-project case
degrading safely if its owner project vanishes, rather than silently
skipping the disabled check.

Backward compatible: state files missing the new fields, and states with
a still-existing distinct cross-project root, behave unchanged.

Added regression tests:
- resume blocked after project moved then disabled at new location
- resume still works after project moved while workflow stays enabled
- cross-project registry root is still correctly honored when it exists
- resume falls back to current project's registry when a stored
  cross-project root no longer exists

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix silenced cleanup failures and malformed run-state type validation

Four fixes from Copilot review on HEAD 4f24735:

1. _discard_staged_workflow_file's fresh-install directory removal used
   shutil.rmtree(dest_dir, ignore_errors=True), so a genuine cleanup
   failure there could never reach _safe_discard_staged_workflow_file's
   warning -- an orphaned directory was left behind with zero report.
   Now removes only if dest_dir still exists and lets a real OSError
   propagate to the existing safe wrapper, which warns while the
   already-printed original install error remains primary.

2. _rollback_committed_workflow_file's fresh-install directory removal
   (post registry.add() failure) had the same ignore_errors=True gap;
   fixed identically so _safe_rollback_committed_workflow_file's warning
   can actually fire.

3. In the --from download-failure branch, tmp_path.unlink(missing_ok=
   True) was unguarded: if it raised (e.g. read-only tempdir), it
   replaced the original "Failed to download workflow" error with a raw
   unhandled OSError instead of a clean typer.Exit. Now guarded exactly
   like the later post-install finally cleanup: a cleanup failure prints
   a warning and the original download error is still reported cleanly.

4. RunState.load() trusted installed_workflow_id/installed_registry_root
   straight out of state.json with no type validation. A malformed value
   (int/list/dict/bool instead of str-or-null) would crash deep inside
   _resolve_run_owner_root or the registry lookup (TypeError building a
   Path, unhashable dict/list as a mapping key) instead of failing
   cleanly. Both fields are now validated as str | None during load,
   raising a clear ValueError that workflow_resume's existing ValueError
   boundary already converts into a clean CLI error with no traceback.
   Valid values (including the empty-string fallback already handled by
   _resolve_run_owner_root) continue to load unchanged.

Added red-first regression tests for each: staged-discard cleanup
warning, rollback cleanup warning, download-failure cleanup-vs-original-
error precedence, and parameterized malformed/valid run-state field
coverage.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add ValueError boundary to workflow status single-run lookup

QUALITY re-review flagged that fa410d3's new RunState.load() type
validation (malformed installed_workflow_id/installed_registry_root
raising ValueError) leaked as a raw unhandled traceback through
`workflow status <run_id>`, which only caught FileNotFoundError.
`workflow resume` already had the matching ValueError boundary.

Adds an `except ValueError as exc: console.print(f"[red]Error:[/red]
{exc}"); raise typer.Exit(1)` clause mirroring resume's exact pattern
(unescaped interpolation, consistent with the existing convention at
every other ValueError boundary in this file). FileNotFoundError
behavior and the no-run-id list-all-runs path are unchanged.

Added parametrized regression covering malformed installed_workflow_id/
installed_registry_root (int/list) via `workflow status`, plus
regressions locking in the unaffected FileNotFoundError and no-run-id
list-path behaviors.

Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: bound workflow step downloads

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: preserve workflow reinstall state

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: fail closed on workflow registry state

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: make workflow installs transactional

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: close workflow transaction races

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: clean up failed workflow transactions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: clean up workflow removal state

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: guard workflow update transactions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: harden workflow lifecycle edge cases

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: verify installed workflow ownership

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: isolate workflow rollback cleanup

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: preserve unique workflow backups

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: bind workflow staging to file descriptors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: fail closed on corrupt workflow registry

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: bind workflow ownership and source identity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): harden redirects and Windows tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): restore staged removals on serialization errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): preserve state across interrupted writes

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): harden resume ownership checks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): validate persisted run state

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(workflows): validate origin and release metadata

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
 2026-07-14 08:03:33 -05:00
Ali jawwad
5367f69f6c docs(workflows): add the built-in 'init' step type to the Step Types table (#3234) 
The Step Types table in docs/reference/workflows.md listed command, prompt, shell, gate, if, switch, while, do-while, fan-out, and fan-in, but omitted 'init' -- which IS a registered built-in (workflows/__init__.py _register_builtin_steps registers InitStep) and is documented in steps/init/__init__.py as bootstrapping a project (equivalent to 'specify init'). Add the missing row so the reference matches the registry.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-29 15:08:49 -05:00
Zied Jlassi
f846d6526c fix(workflows): validate requires keys and reject phantom permissions gate (#3079) 
* fix(workflows): validate requires keys and reject phantom permissions gate

A workflow's `requires` block was parsed but its keys were never
validated, so a typo or an unsupported key was silently ignored. Most
importantly, authors could write `requires.permissions.shell: true`
expecting a runtime capability gate — but no such gate exists: a `shell`
step always runs with the user's privileges. The declaration gave a
false sense of sandboxing.

`validate_workflow` now accepts only the recognised keys
(`speckit_version`, `integrations`, `tools`, `mcp`) and rejects anything
else, with an explicit error for `requires.permissions` pointing authors
to `gate` steps for approval. Docs and the model comment are updated to
state that `requires` is advisory, not a security boundary.

- Reject non-mapping `requires`, unknown keys, and `requires.permissions`
- Clarify workflows reference + PUBLISHING.md shell-step guidance
- Tests for valid keys, non-mapping, unknown key, and permissions

Signed-off-by: Zied Jlassi <6190550+zied-jlassi@users.noreply.github.com>
Assisted-by: AI

* fix(workflows): address review feedback on requires validation

Follow-up to the review on #3079:

- Guard `requires` validation on `is not None` instead of truthiness so a
  falsy non-mapping value (e.g. `requires: []` or `requires: ''`) is
  reported as an error instead of being silently skipped; `requires:`
  (YAML null) is still treated as an omitted block. Add a regression test.
- Reword the workflows security note so `requires.permissions` is shown
  as rejected/unsupported rather than as a valid example of `requires`.
- Standardize on US spelling (`_RECOGNIZED_REQUIRES_KEYS`, "recognized")
  to match the surrounding code and ease searching.
- Tighten the permissions-rejection test to assert on specific message
  markers (`requires.permissions` and the `gate` guidance) so it fails if
  the validation path or wording drifts.

Assisted-by: AI
Signed-off-by: Zied Jlassi (Architect AI) <6190550+zied-jlassi@users.noreply.github.com>

* fix(workflows): scope requires validation to workflow keys (drop tools/mcp)

tools and mcp belong to the bundle manifest requires schema (bundler/models/manifest.py, resolved in bundler/services/resolver.py), not the workflow requires validated here. Drop them from _RECOGNIZED_REQUIRES_KEYS and revert the PUBLISHING.md claim that this PR had introduced, so workflow requires only recognizes speckit_version and integrations.

This keeps the existing docs accurate and resolves the inline doc-consistency review comments.

Signed-off-by: Zied Jlassi <6190550+zied-jlassi@users.noreply.github.com>

* refactor(workflows): type WorkflowDefinition.requires as Any pre-validation

self.requires holds the raw parsed value, which before validate_workflow()
runs may be a non-mapping (None for a bare 'requires:', a list for
'requires: []', etc.). Annotating it dict[str, Any] was misleading for
editors/type-checkers; use Any and document that validate_workflow() enforces
the mapping shape.

Addresses Copilot review feedback on engine.py.

Signed-off-by: Zied Jlassi <6190550+zied-jlassi@users.noreply.github.com>

* fix(workflows): reject YAML-null requires: as a non-mapping

Address Copilot review: validate requires the same way as inputs. A
bare requires: parses as YAML null and was previously treated as an
omitted block, which is inconsistent with inputs and lets a stray
requires: line be silently ignored.

Drop the is-not-None guard and check isinstance(..., dict) directly: an
omitted block still defaults to {} (valid), but a present-but-non-mapping
value -- YAML null, [] or '' -- is now an authoring error that surfaces.

Tests: add YAML-null rejection + an omitted-is-still-valid guard test.
Signed-off-by: Zied Jlassi <6190550+zied-jlassi@users.noreply.github.com>

---------

Signed-off-by: Zied Jlassi <6190550+zied-jlassi@users.noreply.github.com>
Signed-off-by: Zied Jlassi (Architect AI) <6190550+zied-jlassi@users.noreply.github.com>
 2026-06-24 14:49:43 -05:00
Huy Do
affbf5ead5 feat(workflows): add from_json expression filter (#2961) 
* feat(workflows): add from_json expression filter

Step outputs captured as strings could never become typed values in
templates - the filter set was default/join/map/contains only, so e.g.
a fan-out items: could never consume a step's JSON stdout. Add an
arg-less from_json pipe filter with parse-or-raise semantics: invalid
JSON or non-string input raises a clear ValueError rather than passing
through silently.

Fixes #2960

* fix(expressions): make from_json strict — reject any arguments

Address review (#2961): from_json('x') and from_json() previously fell through to a silent passthrough of the unparsed value. Reject any parenthesized form with a clear error so mis-wired templates fail loudly. Rename test to ...parses_object (JSON under test is an object) and add coverage for the strict no-arguments behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(workflows): document the from_json expression filter

Address Copilot review: the user-facing filter references omitted the
newly added `from_json` filter. Add it to the ARCHITECTURE.md filter table
(with the `{{ steps.emit.output.stdout | from_json }}` example) and to the
filter enumerations in workflows/README.md and docs/reference/workflows.md
so the docs match the evaluator's capabilities.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(workflows): make from_json strictness reject trailing tokens; fix docstring

Address Copilot review:
- Strictness only rejected parenthesized forms, so typos like
  `| from_json)` or `| from_json extra` still fell through to the
  unknown-filter path and silently returned the unparsed value. Match on
  the leading filter token and require the whole filter to be exactly
  `from_json`, so every mis-wired form raises. Extend the rejection test to
  cover the trailing-token cases.
- The module docstring claimed "no imports", which is misleading now that
  the module imports `json`. Reword to state the actual sandbox guarantee:
  templates cannot do file I/O, import modules, or run arbitrary code.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-17 13:43:26 -05:00
Huy Do
141119efea feat(workflows): add JSON output for workflow run resume and status (#2814) 
* feat(workflows): add --json output to workflow run, resume, and status

Adds an opt-in `--json` flag to `workflow run`, `workflow resume`, and
`workflow status` that emits a single machine-readable object (run_id,
workflow_id, status, current step; status also reports per-step states
and a runs list) for automation and external orchestrators.

JSON is written via a small `_emit_workflow_json` helper using plain
stdout, so Rich markup, highlighting, and line-wrapping can never alter
the emitted object. Default human-readable output and exit codes are
unchanged when `--json` is omitted. Reference docs updated.

Closes #2811.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(workflows): keep --json stdout clean while steps write output

Suppressing the banner and the step-start callback was not enough to
guarantee a single parseable JSON object on stdout: individual steps still
write there while the engine runs. The gate step prints its prompt, and the
prompt step runs a CLI subprocess that inherits the process's stdout file
descriptor — either can corrupt the JSON stream for interactive runs or
integration-backed workflows.

Wrap engine.execute()/engine.resume() in a file-descriptor-level redirect
(dup2) when --json is set, so both Python-level writes and inherited-fd
subprocess output go to stderr while stdout carries only the emitted JSON.
Step progress stays visible on stderr. status does not run the engine, so
it is unaffected.

Tests cover both pollution channels (a Python print and a real subprocess)
via fd-level capture, and the inactive no-op path. Docs note the
stdout/stderr split.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(workflows): fix stray escape sequence in --json redirect comments

The redirect helper's docstring and its test comment wrote ``print``\s,
which renders as "print\s" rather than "prints". Replace with plain
"prints".

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-04 11:11:39 -05:00
Copilot
40d832f90a Allow specify workflow run to execute YAML files without a project (#2825) 
* Initial plan

* feat: add --workflow option to init command for post-init workflow execution

* chore: remove unused import in test file

* refactor: allow workflow run without project when given a YAML file path

Instead of adding --workflow to init, make `specify workflow run ./file.yml`
work without requiring a .specify/ project directory. When the source is a
YAML file that exists on disk, cwd is used as the project root. When it's a
workflow ID, the .specify/ project requirement is preserved.

* Handle standalone workflow path edge cases

* Fix USERPROFILE env var portability and docs notation

* Fix workflow YAML path detection to require regular files

* Harden workflow run against unsafe .specify paths

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 2026-06-04 07:09:54 -05:00
Huy Do
1732b9b62e feat(workflows): allow resume to accept updated workflow inputs (#2815) 
`workflow resume` now accepts `--input key=value` (the same flag and
parsing as `workflow run`, via a shared `_parse_input_values` helper).
Supplied values are merged over the run's persisted inputs and
re-resolved through the existing typed-validation path
(`_resolve_inputs`), so a resumed/re-run step sees the updated inputs
and ill-typed values fail fast. Keys not supplied keep their persisted
values; resuming without `--input` is unchanged. Reference docs updated.

Distinct from #2405 (file-reference inputs at run time): this is about
supplying inputs at resume time, reusing the existing input model.

Closes #2812.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-03 12:04:07 -05:00
Manfred Riem
02a1d610df docs: add workflows reference, reorganize into docs/reference/, and add --version flag (#2244) 
* docs: add workflows reference, reorganize into docs/reference/, and add --version flag

- Move integrations.md, extensions.md, presets.md into docs/reference/
- New docs/reference/workflows.md: command reference for all workflow
  commands, built-in SDD Cycle workflow with Mermaid diagram, step types,
  expressions, input types, state/resume, and FAQ
- Rename workflow input feature_name to spec with prompt 'Describe what
  you want to build' to match speckit.specify command terminology
- Add --version / -V flag to root specify command with tests
- Update docs/toc.yml, README.md links, and docs/upgrade.md cross-reference
  to use reference/ paths
- Add workflow command to README CLI reference table

* docs: update speckit_version requirement to >=0.7.2 in workflow example
 2026-04-16 13:34:08 -05:00