All three workflow download sites (add --from, catalog install, step
install) passed no redirect_validator to open_url, so an HTTPS URL
redirecting to cleartext HTTP issued the insecure request before the
post-hoc geturl() check reported it. Shared validator now rejects
non-HTTPS redirects (loopback HTTP allowed) pre-follow, matching the
preset download path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- WorkflowRegistry now mirrors StepRegistry: _load refuses symlinked
parents/registry file and normalizes a non-dict workflows field;
save() rejects symlinked paths before writing.
- workflow add --dev requires workflow.yml to be a regular file so a
directory named workflow.yml gets the documented CLI error instead of
an uncaught IsADirectoryError.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- save() now uses tempfile.mkstemp in the workflows dir (matching the
engine's atomic writer), so a pre-created symlink at a predictable
.tmp path cannot redirect the write and concurrent processes cannot
collide.
- The direct-path disabled guard derives the owning project from the
resolved file path instead of the caller's cwd, so running an
installed workflow's YAML from outside the project still refuses when
disabled.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- save() wrote the registry with open('w'), so a failed dump truncated
the file and the next load reset every entry. Write to a sibling temp
file and os.replace into place.
- workflow update no longer claims all workflows are up to date when
some targets were skipped; it reports checked-only status with a
skipped count.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A corrupted-but-parseable registry entry (e.g. a string value) crashed
WorkflowRegistry.add with AttributeError on existing.get. Guard the
non-dict case while still restoring the original raw value on rollback.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- A truthy non-string catalog url (e.g. 123) reached urlparse and raised
AttributeError, escaping the clean error path; validate it is a string.
- enable/disable mutated the live registry entry before add(), so add's
rollback snapshot captured the already-toggled object; pass a fresh
mapping instead.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- WorkflowRegistry.add now rolls back its in-memory mutation when save()
raises, so a later successful save cannot persist metadata for a
failed update alongside the restored YAML backup.
- workflow run uses the same truthiness check for 'enabled' as list and
disable, so malformed values like 0 or null refuse to run.
- workflow update reports 'No workflows were eligible for update' when
every target was skipped instead of claiming all are up to date.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
'workflow add foo --from ""' fell through 'from_url or ...' to a
catalog install. Distinguish None from empty string so explicit values
stay on the URL-validation path and fail closed.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Running the installed copy's YAML directly (specify workflow run
.specify/workflows/align-wf/workflow.yml) skipped the registry check.
File sources resolving inside .specify/workflows/<id>/ now map back to
the workflow ID and refuse to run while disabled.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Path-equivalent spellings like "align-wf/" previously bypassed the
registry disabled check because the engine normalizes the path while the
registry matches the raw string. workflow run now validates non-file
sources against the workflow ID pattern before lookup.
Also updates docs/reference/workflows.md with --dev/--from install
options, update/enable/disable commands, and the search --author flag.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The update path never verified the downloaded workflow carries the catalog
version that triggered the update, so a stale or misconfigured URL could
report success while leaving the old version installed or downgrading it.
Pass the expected version into the install helper and fail the update when
the downloaded definition does not match.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
urlparse raises ValueError on e.g. an unbalanced IPv6 literal before the
invalid-URL branch is reached; on workflow update that also bypassed the
per-workflow handler and aborted the whole command. Convert the parse
failure into a clean error so add fails cleanly and update skips just the
affected workflow.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Covers the last few review threads not yet addressed:
- Escape yaml.YAMLError text in the local workflow add install path
(matches the already-escaped download/catalog paths).
- Escape the non---dev local directory fallback's "No workflow.yml
found in <path>" message (the --dev branch already escaped it).
- Escape the redirected final_url in the --from non-HTTPS redirect
error (IPv6 literals like http://[::1]/... are legal and contain
brackets).
- Escape the "Downloaded workflow is invalid" exception message in
_install_workflow_from_catalog, matching the sibling catalog-install
exception handler a few lines above it.
Adds regression tests for each in TestWorkflowCliAlignment, following
the existing escaping-test pattern in this class.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Transient FS errors (perms, disk full) from backup read or write no
longer abort the whole update run. The restore is wrapped in its own
try/except so a failed write only warns, and the offending workflow
is reported via 'Failed to update' like other per-workflow failures.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
workflow list now escapes id/name/version/description before printing,
matching how extensions render user-editable fields. The catalog install
helper computes safe_wf_id once and uses it for every early error path
plus the final failure message.
workflow update wraps _safe_workflow_id_dir and the backup read inside
the try/except typer.Exit block so an unsafe id in a corrupted registry
fails that one workflow and the rest continue.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The two id-mismatch error paths interpolated repr() into Rich markup, so
a stray bracket in a user typo could be parsed as markup. Route both
through rich.markup.escape.
`workflow add <source> --from <url>` also validated the source only
after downloading. Validate it up front so a URL/path/typo fails
without a network fetch.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
workflow list now skips non-dict registry entries with a warning instead
of crashing, matching update/enable/disable. The broad except in
_install_workflow_from_catalog no longer swallows typer.Exit, so precise
errors like the non-HTTPS redirect message are not duplicated.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the missing workflow commands and flags so the workflow CLI
matches the extension/preset pattern: add --dev and --from, search
--author, update, enable and disable. Disabled workflows are blocked
from running and marked in list output.
Fixes#2342
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflows): validate scalar types before string operations in workflow validation
YAML parses unquoted scalars like version: 1.0 and id: 123 as
float/int, which crashed validate_workflow and workflow add with raw
tracebacks. Type-check id, name, version and step ids before regex
and string operations so these surface as validation errors. Accept
an unquoted schema_version: 1.0 instead of printing a self-identical
rejection message.
Fixes#3420
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflows): treat falsey non-strings as type errors, not missing fields
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflows): only accept schema_version 1.0 so the error message is accurate
The check also accepted "1" while the error said Expected '1.0'.
Unquoted YAML 1.0 still works via str(); plain 1 is now rejected with
the message that matches.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(workflows): make shell step timeout configurable
The shell step hardcoded a 300s subprocess timeout, killing any
legitimate long-running QA command. Read an optional timeout field
(seconds, positive integer, default 300) and validate it.
Fixes#3327
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* style: multi-line timeout validation, assert status in default test
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Guard against unvalidated timeout in ShellStep.execute()
The engine does not auto-validate step config, so a string or null
timeout would reach subprocess.run() and crash the run with a
TypeError. Fall back to the 300s default for malformed values,
mirroring how the engine treats unvalidated continue_on_error.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflows): apply chained expression filters left-to-right
The pipe-filter parser in `_evaluate_simple_expression` split the
expression only at the *first* top-level `|` and treated the whole
remainder as a single filter. So a filter chain like
`{{ inputs.rows | map('name') | join(', ') }}` handed
`map('name') | join(', ')` to one filter, where the `(\w+)\((.+)\)`
regex mangled it and raised `ValueError`.
This broke the canonical use of `map`: it returns a list, and `join`
is the only filter that renders a list to a string, so the two are
meant to be chained. Chaining was impossible for every registered
filter.
Split the pipe segments at the top level (quote/bracket aware, so a
literal `|` inside a quoted argument like `join(' | ')` is preserved)
and fold each filter over the running value. The single-filter logic
is extracted verbatim into `_apply_filter`, so all existing strict
handling (`from_json` arity, unsupported-form vs unknown-filter
messages) is unchanged and now applies to every link in the chain.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(cli): exit cleanly on malformed IPv6 URLs in extension/preset/workflow add
extension add --from, preset add --from, and workflow add <url> parsed
the user-supplied URL with a bare urlparse before their HTTPS/host
validation, so an unclosed IPv6 bracket escaped as a raw ValueError
traceback. Wrap each parse and emit the surrounding validation's clean
error style + typer.Exit(1) instead.
Fixes#3368
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(cli): convert malformed redirect URLs to URLError in shared redirect handler
Parse the redirect target once in _StripAuthOnRedirect.redirect_request
before the validator and stdlib handler run, converting ValueError into
URLError which every download path already catches. Also escape from_url
in the preset install message so IPv6 brackets don't break Rich markup.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
FanInStep.validate() only checked wait_for, so a non-mapping 'output'
(a list or scalar) validated clean; execute() then silently coerces it
to {}, so the author's declared aggregation keys vanish with COMPLETED
status and no diagnostic. Reject a non-mapping output at validation,
mirroring the command-step (#3262) non-mapping fix. execute()'s
defensive coercion is left in place for unvalidated callers.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
ShellStep.validate() only checked that 'run' was present, so run: (null)
or a GitHub-Actions-style list validated clean; execute() then
str()-coerces the value and invokes it under shell=True, literally
running 'None' or "['echo', 'hi']" as a command. Add a type check after
the presence check, mirroring the command-step (#3262) and gate options
validation. Expression strings ('{{ ... }}') are strings, so they stay
valid.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
`validate` accepts a reject option case-insensitively
(`o.lower() in {"reject", "abort"}`), so a gate authored as
`options: [Approve, Reject]` passes validation. But `execute`
compared the echoed choice case-sensitively, so picking `Reject`
fell through to the approval path and silently ran downstream
steps instead of aborting.
Lower-case `choice` before the reject comparison so the runtime
agrees with the validation that let the option through.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_safe_compare coerced both operands to int/float unconditionally for <, >, <=,
>=. any non-numeric string (an iso date, a version tag, a name) failed that
coercion and the whole comparison silently returned False -- so
`{{ inputs.d < '2026-02-01' }}` was False even when the date was earlier.
only coerce when both operands look numeric; otherwise compare the original
values, so two strings order lexicographically the way python does and two
numeric strings still compare as numbers ("10" > "9"). a number vs a
non-numeric string stays incomparable and yields False.
add a regression test covering dates, plain strings, numeric strings, and the
number-vs-string case.
* fix(workflows): quote-aware interpolation so a literal }} in a filter arg doesn't break multi-expression templates
#3208/#3228 hardened the single-expression fast path (_is_single_expression)
so a literal {{ or }} inside a string argument like `| default('}}')` stays on
the typed path. the multi-expression interpolation path was left on the old
_EXPR_PATTERN regex, whose non-greedy `(.+?)}}` body stops at the first }}
regardless of quoting. so a multi-expression template with a literal }} in any
block captured a truncated body, hit the filter parser malformed, and raised
ValueError.
e.g. `{{ inputs.name }}: {{ inputs.missing | default('}}') }}` raised instead
of interpolating.
replace _EXPR_PATTERN.sub with _interpolate_expressions, which scans each block
for a }} outside string literals - the same quote handling _is_single_expression
already uses. plain-value passthrough (a literal }} in a resolved value, not an
expression) is unchanged.
add regression tests for a literal }} in the second block and in the first
block, plus a literal {{ guard.
* fix(workflows): surface malformed templates in interpolation instead of emitting verbatim
address copilot review on #3307: when the quote-aware scan finds no block-closing
`}}` (e.g. an unbalanced quote in a filter arg swallowed the delimiter), fall back
to the first raw `}}` in the tail and evaluate it, so the parser raises ValueError
just as the old _EXPR_PATTERN.sub path did. only when there is no `}}` at all is
the tail left verbatim (a genuinely unterminated `{{`, which the regex also could
not match). keeps a typo failing loudly rather than being silently hidden.
add a regression test for an unbalanced quote in a multi-expression template.
* fix: interpolate multi-expression templates instead of returning None (#3208)
`evaluate_expression` returned None for templates containing two or more
`{{ }}` blocks with no surrounding literal text, e.g.
`"{{ context.run_id }} {{ inputs.issue }}"`.
The single-expression fast path used `_EXPR_PATTERN.fullmatch()`, but
`fullmatch` defeats the pattern's non-greedy `(.+?)` body: for two adjacent
expressions it still matches, capturing everything between the first `{{`
and the last `}}` (`"context.run_id }} {{ inputs.issue"`) as the body. That
garbage failed dot-path resolution and returned None directly, bypassing the
`sub()` interpolation path that would have resolved each expression. Downstream
this surfaced as the literal string "None" reaching commands.
Guard the fast path on `stripped.count("{{") == 1` so only genuine
single-expression templates take the typed return; multi-expression templates
fall through to `sub()` and interpolate correctly.
Add regression tests for two expressions separated by a space and for adjacent
expressions with no separator.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(expressions): use match-span guard so single expressions with literal {{ keep their type
The previous `stripped.count("{{") == 1` guard misclassified a genuine
single expression whose string argument contains a literal `{{` (e.g.
`{{ inputs.text | contains('{{') }}`) as multi-expression, routing it
through `sub()` interpolation and coercing the typed (bool/int/list)
return value to a string -- breaking the type-preservation the docstring
promises (Copilot review on #3228).
Anchor a single match at the start and require it to consume the whole
stripped string instead. The non-greedy body stops at the first `}}`, so
a two-block template fails the span check (falls through to interpolation,
fixing #3208) while a lone expression -- including one with a `{{` inside
a string literal -- matches to the end and keeps its typed value.
Add a regression test for the literal-brace single-expression case.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(expressions): detect single expression with quote-aware scan
The match-span guard using the non-greedy _EXPR_PATTERN stopped at the
first `}}`, so a lone expression whose string argument contains a literal
`}}` (e.g. `{{ inputs.text | contains('}}') }}`) was misclassified as
multi-expression and mis-parsed by the interpolation path, raising
ValueError and turning CI red (Copilot review on #3228).
Replace the span check with `_is_single_expression`, which scans the
`{{ ... }}` body for a block-closing `}}` outside string literals (mirrors
the quote handling already in `_split_top_level_commas`). A genuine
two-block template closes early and falls through to interpolation
(fixing #3208); a lone expression with a literal `{{` or `}}` inside a
string argument keeps its typed return value.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* refactor: move workflow command handlers to workflows/_commands.py (PR-8/8)
Final PR of the __init__.py split. Moves the workflow command group out of
__init__.py into the existing workflows/ package, completing the domain-dir
layout established in PR-5 (integrations), PR-6 (presets) and PR-7
(extensions).
- New workflows/_commands.py holds the four Typer apps (workflow / catalog /
step / step-catalog), all 25 command handlers, the six workflow-only
helpers (_parse_input_values, _workflow_run_payload, _emit_workflow_json,
_stdout_to_stderr_when, _validate_step_id_or_exit,
_resolve_steps_base_dir_or_exit), and a register(app) entry point.
- workflows is already a package, so no rename is needed; intra-package
imports change from `.workflows.x` to `.x`. The only root-helper dep
(_require_specify_project) is reached through a call-time shim so test
monkeypatching of specify_cli._require_specify_project keeps working.
- __init__.py drops ~1445 lines (2066 -> 621); the workflow group is
re-attached via register(app). Dead `contextlib` import removed.
- tests/test_workflows.py: import the now-relocated _stdout_to_stderr_when
helper from its new home (workflows._commands) instead of the package root.
No behavior change. Full suite green (3847 passed), ruff clean.
* Prevent workflow state writes through symlinked storage
Workflow commands persist run state under .specify/workflows/runs, so the command-local project shim now rejects symlinked workflow storage before any workflow command proceeds. The standalone YAML path uses the same guard because it intentionally bypasses the normal project requirement while still creating workflow state under the current directory.
Constraint: Local YAML workflow runs do not require an existing .specify project directory but still create .specify/workflows/runs state
Rejected: Guard only .specify in the file-source path | .specify/workflows and runs can independently redirect writes
Confidence: high
Scope-risk: narrow
Directive: Keep workflow storage symlink checks centralized before constructing WorkflowEngine
Tested: .venv/bin/python -m pytest tests/test_workflow_run_without_project.py tests/test_workflows.py::TestWorkflowAddSymlinkGuard -v
Tested: .venv/bin/python -m py_compile src/specify_cli/workflows/_commands.py tests/test_workflow_run_without_project.py tests/test_workflows.py
Not-tested: Ruff lint; ruff is not installed in the repo virtualenv
Assisted-by: OpenAI Codex (model: GPT-5, autonomous)
* fix(workflows): pass github_hosts allowlist to GHES release asset resolver
workflow add resolved GitHub release download URLs without forwarding the
github_provider_hosts() allowlist, so resolve_github_release_asset_api_url
never treated any host as GHES. This regressed GitHub Enterprise Server
release asset resolution and diverged from presets/extensions, which already
pass github_hosts. Forward github_provider_hosts() at both the direct-URL and
catalog install call sites. The allowlist remains the anti-SSRF gate.
* fix(workflows): reject symlinked/traversal <id> dir on workflow install
Local/URL and catalog installs wrote to .specify/workflows/<id>/workflow.yml
without guarding the <id> segment. A pre-planted symlink at <id> or
<id>/workflow.yml let mkdir+copy/download follow it and write outside the
project root; a non-directory <id> made mkdir raise unhandled.
Add _safe_workflow_id_dir() to reject path traversal, symlinked or
non-directory <id>, and a symlinked workflow.yml leaf before any write.
Fold the catalog branch's existing traversal check into the helper.
* fix(workflows): harden _safe_workflow_id_dir output and leaf checks
- Reorder symlink/non-directory check before resolve() so a symlinked
<id> reports as symlinked instead of misleading "Invalid workflow ID"
- Reject a pre-existing <id>/workflow.yml that is not a file, avoiding an
unhandled IsADirectoryError on later write/copy2
- Escape workflow_id in Rich output to prevent markup injection; escape
the repr (not the raw id) so repr-added backslashes cannot re-expose
brackets, matching extensions/_commands.py hardening
- Add tests for workflow.yml-as-directory and markup-escaped invalid id
* Avoid stale lint failures from config helper imports
Move PyYAML loading into the helpers that read and write agent-context configuration, and replace the broad Any annotation with object. The runtime behavior stays the same while the module no longer exposes top-level imports that can be flagged as unused when CI analyzes a narrower code shape.
* Prevent workflow commands from targeting reserved storage
Workflow install and removal paths are derived from workflow IDs before any catalog download, local copy, or directory deletion. Validate that IDs are single workflow-id path segments and reject names reserved for workflow runtime storage so commands cannot target .specify/workflows/runs or .specify/workflows/steps.
* fix(workflows): reject bool max_iterations in while/do-while validation
while/do-while validate() checked 'not isinstance(max_iter, int) or max_iter < 1'. Since bool is a subclass of int, isinstance(True, int) is True and True < 1 is False, so 'max_iterations: true' passed validation and then ran as a single iteration (range(True) == range(1)) instead of being reported as a type error. Reject bools explicitly, matching the fail-fast-on-bool handling already used for number inputs and gate options.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test: assert empty error list for the valid do-while max_iterations case
Address Copilot review: the accepted-config assertion only checked that no error mentioned 'max_iterations', which could let an unrelated validation error pass unnoticed. For a known-good config, assert the entire error list is empty (consistent with the other validate tests in this file).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(workflows): honor max_concurrency in fan-out via a bounded thread pool
* feat(workflows): address review — sliding-window fan-out, locked output, faithful halt
Address the reviewer feedback on the bounded fan-out concurrency:
- Sliding submission window: keep at most `workers` items in flight and stop
launching new items once the run is halting, instead of submitting all items
up front (which let the pool keep starting queued work after a halt).
- Faithful halt prefix: attribute a halt to the specific item whose own
recorded result halted the run (replaying the sequential break condition,
honoring continue_on_error/aborted), not the shared run status a later
concurrent item may have flipped. The returned prefix now includes the actual
halting item, matching the sequential path. An item that fails before
recording a result (e.g. an unknown step type) is attributed too, since every
item runs the same template.
- Lock the parent fan-out output mutation: route the post-fan-out
step_results[...]['output'] update through a new RunState.set_step_output()
under the run lock, so it cannot race a concurrent save().
- Docstring: describe int() coercion accurately (numeric strings / floats are
honored; only non-coercible or <= 1 runs sequentially).
Tests: add concurrent halt-includes-halting-item, continue_on_error-does-not-
truncate, and unknown-template-type-matches-sequential coverage; make the
timing test use a monotonic clock with a looser threshold to avoid CI flakiness.
* feat(workflows): address second review pass — concurrency hardening
- append_log: serialize the log_entries append + log.jsonl write under a
dedicated RunState._log_lock so concurrent fan-out workers can't interleave
or corrupt log lines (kept separate from the state lock; never nested).
- _run_fan_out.run_item: read the item output back through the item_ctx it
executed against rather than the outer context closure — clearer and robust
if StepContext ever stops sharing the steps dict by reference.
- StepBase: document the thread-safety contract — STEP_REGISTRY holds one shared
instance per type, so concurrent fan-out invokes execute() on the same object;
implementations must be stateless/thread-safe (the built-ins already are).
- test_concurrency_is_real: prove parallelism deterministically with a
threading.Barrier (sequential execution can't clear it) instead of a
wall-clock timing assertion.
* feat(workflows): address review — stamp updated_at under lock, clarify cancel semantics
- RunState.save(): move the updated_at timestamp assignment inside the run lock
so the timestamp matches the snapshot the thread serializes and concurrent
savers don't race on it.
- _run_fan_out docstring: clarify that on a halt only not-yet-started items are
cancelled; items already running finish but their outputs are ignored
(Future.cancel() can't stop running work, and the pool joins on exit).
* feat(workflows): serialize on_step_start callback under a lock
The concurrent fan-out path invokes _execute_steps from worker threads, which
calls the engine's on_step_start callback (the CLI sets it to a console.print
lambda). Concurrent invocation could interleave/garble progress output. Guard
the call with a WorkflowEngine._callback_lock so callbacks are serialized;
the lock is uncontended for sequential runs.
* feat(workflows): re-raise worker exceptions in-place to preserve traceback
In _run_fan_out's concurrent path, a worker exception was stashed in first_exc
and re-raised after the loop. Re-raise it from within the except block with a
bare `raise` (after cancelling outstanding futures) so the original traceback is
preserved, and drop the now-unneeded first_exc variable. The ThreadPoolExecutor
__exit__ still joins any already-running workers before the exception escapes.
* feat(workflows): lock final fan-out status, drop redundant output write, bound workers
Address third review pass:
- Remove the unlocked `context.steps[step_id]["output"] = …` writes in the
fan-out parent update. context.steps[step_id] is the same dict object that
set_step_output() updates under the run lock, so the direct (unsynchronized)
mutation was redundant.
- Preserve sequential halt semantics under concurrency: a later in-flight item
could overwrite state.status after the halting item was identified. _run_fan_out
now derives the halting item's run status (item_halt_status, replacing the bool
item_halted) and restores it after the pool joins, so the final status is the
first halting item's outcome.
- Bound the pool: workers = min(max_concurrency, len(items)) and early-return for
empty items, so a user-controlled max_concurrency can't over-allocate threads.
Add coverage that an earlier PAUSED item's status wins over a later concurrent
FAILED item.
* feat(workflows): avoid unlocked context.steps writes when it aliases step_results
On a resume run, StepContext is built with steps=state.step_results, so the two
direct `context.steps[...] = ...` writes mutated the shared dict outside the run
lock and could race save(). Route both through a new _record_result helper that
mirrors into context.steps only when it is a distinct object (a fresh run) and
otherwise relies solely on record_step_result's locked write.
GateStep.validate() reports non-string options as an error, but then -- when on_reject is 'abort'/'retry' -- still runs the reject-choice check 'any(o.lower() in ... for o in options)'. For a non-string option (e.g. options: [123]) o.lower() raised AttributeError, which escaped validate() and broke validate_workflow's documented 'return a list of errors, never raise' contract. Guard the check so it only runs when every option is a string (the non-string case is already reported above).
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_evaluate_simple_expression used 'if "|" in expr' / expr.split("|", 1) to detect a filter pipe, so a literal '|' inside a quoted operand (e.g. inputs.x == 'a|b') was mistaken for a filter separator and raised a spurious ValueError ('unknown filter') instead of comparing the string. Use the existing quote/bracket-aware _find_top_level helper (added for the operator-splitting fix) so only a top-level pipe is treated as a filter separator.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(workflows): reject a fan-in wait_for that names an unknown step at validation
* fix(workflows): reject fan-in wait_for self-reference and non-string entries
Address review feedback on the fan-in wait_for validator:
- A fan-in's own id is added to seen_ids before the wait_for check, so
`wait_for: [<self>]` passed validation while producing a silent empty
join at runtime. Reject self-references explicitly.
- Non-string entries (e.g. YAML `wait_for: [123]`) were skipped by the
isinstance(str) guard and validated even though they can never match a
real step id. Flag them as wiring errors.
Add coverage for both cases.
WorkflowEngine._coerce_input normalizes a whole-valued number to int via int(value). For an infinite float (e.g. a 'type: number' input with YAML 'default: .inf') int(inf) raises OverflowError, which is not in the except (ValueError, TypeError) tuple. validate_workflow eager-coerces declared defaults and is documented to RETURN a list of errors, but it only catches ValueError -- so the OverflowError escaped and validate_workflow raised instead of reporting, breaking its contract. (NaN already surfaced cleanly because int(nan) raises ValueError.)
Add OverflowError to the except tuple so an infinite default surfaces as the same clean 'expected a number' ValueError as NaN, consistent with the function's existing fail-fast-on-authoring-mistakes design. Finite values (5.0 -> 5, 3.5 -> 3.5) are unaffected.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
_evaluate_simple_expression split on operator keywords using naive str.find/split, so a keyword INSIDE a quoted operand was treated as an operator: `inputs.mode == 'read and write'` split on the inner ' and ' and evaluated as `(inputs.mode == 'read) and (write')`. The literal short-circuit was also too greedy -- `'a' == 'b'` matched startswith("'")/endswith("'") and was stripped to the garbage truthy string `a' == 'b`, so `'done' == 'failed'` evaluated truthy and gated the wrong branch.
Add a quote/bracket-aware _find_top_level helper (mirroring the existing _split_top_level_commas) and use it for the and/or/comparison/in/not-in splits; tighten the literal short-circuit to fire only when the opening quote's match is the final char. The docstring already lists comparisons + and/or/not + in/not-in + string literals as supported, so this restores the documented contract.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(auth): add github_provider_hosts() to enumerate GHES hosts from auth.json
Assisted-by: Claude Code (model: claude-sonnet-4-6, autonomous)
* fix(extensions): resolve GHES release assets via /api/v3
Generalizes resolve_github_release_asset_api_url to GitHub Enterprise
Server hosts (gated by auth.json github hosts), fixing private GHES
extension/preset downloads. github/spec-kit#3147
Assisted-by: Claude Code (model: claude-sonnet-4-6, autonomous)
* fix(extensions,presets): pass auth.json github hosts into release resolver
Assisted-by: Claude Code (model: claude-sonnet-4-6, autonomous)
* docs(auth): document GHES private catalog + release-asset auth
Assisted-by: Claude Code (model: claude-sonnet-4-6, autonomous)
* fix(presets,workflows): pass auth.json github hosts into remaining release resolvers
Wires preset add --from and workflow add through github_provider_hosts()
so private GHES release assets resolve via /api/v3 there too. github/spec-kit#3147
Assisted-by: Claude Code (model: claude-sonnet-4-6, autonomous)
* test(presets): use module-level io.BytesIO in GHES preset test
Addresses Copilot review on PR #3157: drop unnecessary __import__("io")
in test_preset_add_from_ghes_release_url_resolves_via_api_v3 since io is
already imported at module level.
* fix(github-http): pass through GHES asset API URLs by path shape
Addresses Copilot review on PR #3157. A direct GHES /api/v3 release asset
URL was only returned as already-resolved when its host was in the
allowlist; otherwise the resolver returned None and the caller downloaded
the same URL without 'Accept: application/octet-stream', fetching JSON
metadata instead of the binary.
Gate the passthrough on path shape alone, mirroring the github.com case.
This is safe: passthrough returns the input URL unchanged and the caller
fetches it either way, so no new request to an arbitrary host is induced;
the token stays independently gated by auth.json in open_url. The
allowlist remains the anti-SSRF gate on the tag-lookup resolving path.
Add test_passthrough_for_unlisted_ghes_api_asset_url.
* fix(workflows): preserve commas inside quoted list-literal elements
The simple-expression evaluator parsed a list literal with a naive
`inner.split(",")`, which splits on commas inside quoted strings (and
nested brackets). So `{{ ["a, b", "c"] }}` evaluated to three items
(`["a", "b", "c"]`) instead of two, silently corrupting `fan-out` `items:`
and any list expression that contains a comma inside a quoted element.
Split list-literal elements on top-level commas only, ignoring commas
inside quotes or nested brackets, via a small `_split_top_level_commas`
helper. Plain and empty lists are unchanged.
Add tests covering quoted commas, nested lists, and the existing
plain/empty cases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(workflows): cover single-quoted and nested list literals
Address review: extend the list-literal regression test to assert single-quoted elements with commas and nested lists parse correctly, alongside the existing double-quoted cases.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(workflows): validate requires keys and reject phantom permissions gate
A workflow's `requires` block was parsed but its keys were never
validated, so a typo or an unsupported key was silently ignored. Most
importantly, authors could write `requires.permissions.shell: true`
expecting a runtime capability gate — but no such gate exists: a `shell`
step always runs with the user's privileges. The declaration gave a
false sense of sandboxing.
`validate_workflow` now accepts only the recognised keys
(`speckit_version`, `integrations`, `tools`, `mcp`) and rejects anything
else, with an explicit error for `requires.permissions` pointing authors
to `gate` steps for approval. Docs and the model comment are updated to
state that `requires` is advisory, not a security boundary.
- Reject non-mapping `requires`, unknown keys, and `requires.permissions`
- Clarify workflows reference + PUBLISHING.md shell-step guidance
- Tests for valid keys, non-mapping, unknown key, and permissions
Signed-off-by: Zied Jlassi <6190550+zied-jlassi@users.noreply.github.com>
Assisted-by: AI
* fix(workflows): address review feedback on requires validation
Follow-up to the review on #3079:
- Guard `requires` validation on `is not None` instead of truthiness so a
falsy non-mapping value (e.g. `requires: []` or `requires: ''`) is
reported as an error instead of being silently skipped; `requires:`
(YAML null) is still treated as an omitted block. Add a regression test.
- Reword the workflows security note so `requires.permissions` is shown
as rejected/unsupported rather than as a valid example of `requires`.
- Standardize on US spelling (`_RECOGNIZED_REQUIRES_KEYS`, "recognized")
to match the surrounding code and ease searching.
- Tighten the permissions-rejection test to assert on specific message
markers (`requires.permissions` and the `gate` guidance) so it fails if
the validation path or wording drifts.
Assisted-by: AI
Signed-off-by: Zied Jlassi (Architect AI) <6190550+zied-jlassi@users.noreply.github.com>
* fix(workflows): scope requires validation to workflow keys (drop tools/mcp)
tools and mcp belong to the bundle manifest requires schema (bundler/models/manifest.py, resolved in bundler/services/resolver.py), not the workflow requires validated here. Drop them from _RECOGNIZED_REQUIRES_KEYS and revert the PUBLISHING.md claim that this PR had introduced, so workflow requires only recognizes speckit_version and integrations.
This keeps the existing docs accurate and resolves the inline doc-consistency review comments.
Signed-off-by: Zied Jlassi <6190550+zied-jlassi@users.noreply.github.com>
* refactor(workflows): type WorkflowDefinition.requires as Any pre-validation
self.requires holds the raw parsed value, which before validate_workflow()
runs may be a non-mapping (None for a bare 'requires:', a list for
'requires: []', etc.). Annotating it dict[str, Any] was misleading for
editors/type-checkers; use Any and document that validate_workflow() enforces
the mapping shape.
Addresses Copilot review feedback on engine.py.
Signed-off-by: Zied Jlassi <6190550+zied-jlassi@users.noreply.github.com>
* fix(workflows): reject YAML-null requires: as a non-mapping
Address Copilot review: validate requires the same way as inputs. A
bare requires: parses as YAML null and was previously treated as an
omitted block, which is inconsistent with inputs and lets a stray
requires: line be silently ignored.
Drop the is-not-None guard and check isinstance(..., dict) directly: an
omitted block still defaults to {} (valid), but a present-but-non-mapping
value -- YAML null, [] or '' -- is now an authoring error that surfaces.
Tests: add YAML-null rejection + an omitted-is-still-valid guard test.
Signed-off-by: Zied Jlassi <6190550+zied-jlassi@users.noreply.github.com>
---------
Signed-off-by: Zied Jlassi <6190550+zied-jlassi@users.noreply.github.com>
Signed-off-by: Zied Jlassi (Architect AI) <6190550+zied-jlassi@users.noreply.github.com>
* fix: fail loudly on an unknown workflow expression filter
The expression evaluator's filter dispatch fell through to `return value`
for any unregistered filter, so a typo'd or unsupported filter such as
`{{ items | length }}` rendered the value unchanged with no error and the
run completed — a silent wrong result.
Raise a clear ValueError instead, naming the offending filter and the valid
ones, mirroring the strict handling already used for `from_json`. The five
registered filters (default/join/map/contains/from_json) are unchanged; the
`name(arg)` form of an unknown filter is now caught too.
* fix: distinguish a misused registered filter from an unknown one; cover map
Address the review feedback on the unknown-filter fail-loud path:
- A *registered* filter used in an unsupported form (e.g. `| join` or
`| map` with no argument) raised the misleading "unknown filter
'<name>'" — the filter is registered, the syntax isn't. It now raises
a message naming it as a known filter misused. A new
`_REGISTERED_FILTERS` constant drives the distinction.
- `test_registered_filters_unaffected` now also exercises `map('attr')`,
which it previously claimed to cover but didn't. Add
`test_registered_filter_unsupported_form_raises` to pin the new path.
* fix: include the no-arg default form in the filter-error hint
Copilot review: the hint listed default('x') but omitted the valid
no-argument default form (| default), which this module supports.
* feat: surface gate detail in the workflow run/resume --json payload
A paused run was indistinguishable from any other pause in the
machine-readable outcome, and the gate's prompt/options/choice never
left the human-facing stream. Record each step's type in the run
state's step results (one engine line) and, when the run sits at a
gate, add a gate block (step_id/message/options/choice) to the payload
so orchestrators can drive review gates without parsing stdout.
Reference implementation for the proposal in #2964.
Addresses #2964
* fix(workflow): only surface gate detail in --json when the run is paused
Address review (#2965): _gate_outcome() emitted a gate block whenever current_step_id pointed at a gate step. Since RunState.current_step_id is never cleared on completion, a completed/failed run whose last step was a gate leaked stale gate detail in run/resume/status --json. Guard on status == paused. Also assert CLI success in the _run_json test helper before JSON-parsing, and add direct coverage for the suppression guard.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(workflows): surface gate block on aborted runs; stabilize message
Address Copilot review:
- `_gate_outcome` now also surfaces the gate block when a run is `aborted`
by a gate rejection (`on_reject: abort`), not only when `paused`. Abort
is the only path that sets ABORTED and it leaves current_step_id on the
gate, so an orchestrator can read the recorded `choice` for the stop.
- Coerce `message` to a string (it may be a non-string YAML literal that
GateStep only coerces for interpolation) so the JSON schema stays stable.
- Tests: add a CLI-level aborted-path test, a message-coercion test, and
extend the suppression test to allow `aborted`; share the run helper via
`_invoke_json` to avoid duplicating the invoke boilerplate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(workflows): assert clean exit in gate-abort JSON test
Address Copilot review: the gate-abort test parsed stdout without first
asserting the CLI exited cleanly, so an invoke failure would surface as an
opaque JSON decode error. Route it through `_run_json` (which asserts
exit_code == 0 before parsing) and drop the now-redundant `_invoke_json`
helper — a gate abort emits the payload and returns, so the run exits 0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix: use result.output in run-helper assert; document step_data shape
Address Copilot review:
- `_run_json` asserted with `result.stdout` in the message, but under
`--json` step output is redirected off stdout — the useful diagnostics
live on `result.output`. Switch the assertion message to `result.output`
(the JSON parse still reads stdout), matching the other CLI tests.
- `StepContext.steps` documented a 5-key entry shape; the engine now also
persists `type` and `status`. Update the docstring to the canonical
7-key shape so step authors/debuggers see the real record.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* test(workflows): align gate-abort JSON test with aborted→exit-1
After rebasing onto main, a gate abort now emits the --json payload and
then exits non-zero (`_run_outcome_exit_code` maps aborted → 1, from the
merged exit-code work). Give `_run_json` an `expected_exit` parameter
(default 0) so the abort case asserts exit 1 while the paused/completed
cases stay at 0 — keeping a single shared helper rather than duplicating
the invoke boilerplate.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(workflows): backward-compat gate detection + normalize gate options
Address Copilot review:
- A run paused by an older version has no persisted step `type`, so
`_gate_outcome` would never surface its gate block on resume. Add
`_is_gate_step`: prefer the `type` field, but when it is absent fall back
to the gate's unique output signature (`on_reject`, written only by
GateStep). A record with a different known `type` is still not a gate.
- Normalize `options` to a list of strings (mirroring the `message`
coercion) so an unvalidated workflow with non-string options can't
destabilize the JSON schema.
- Tests: options coercion, type-less gate detection, and a type-less
non-gate negative case.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(workflows): normalize non-list gate options to a stable list[str]
Address Copilot review: the prior options normalization only mapped a
`list`, returning the raw value for any other shape (scalar/tuple), which
contradicted the "stable list[str]" intent. Extract `_normalize_gate_options`:
None stays None; list/tuple maps each element through str; any other scalar
becomes a single-element list (a bare string is one option, never iterated
character-by-character). The emitted schema is now always list[str] | None.
Extend the options test to cover list, tuple, bare string, numeric scalar,
and None.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(workflows): normalize gate choice to str; portable plain-gate test
Address Copilot review:
- `_gate_outcome` normalized `message` and `options` but passed `choice`
through as-is; an unvalidated gate can record a non-string `choice`,
which contradicts the stable-schema rationale. Coerce `choice` to
`str | None` (None still means "no decision yet"), consistent with the
other two fields. Adds a focused choice-coercion test.
- The plain (no-gate) test workflow used `run: "true"`, which fails under
cmd.exe on Windows (ShellStep uses shell=True). Use the cross-platform
`run: "exit 0"` (matching the exit-code suite's workflows).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(workflows): add from_json expression filter
Step outputs captured as strings could never become typed values in
templates - the filter set was default/join/map/contains only, so e.g.
a fan-out items: could never consume a step's JSON stdout. Add an
arg-less from_json pipe filter with parse-or-raise semantics: invalid
JSON or non-string input raises a clear ValueError rather than passing
through silently.
Fixes#2960
* fix(expressions): make from_json strict — reject any arguments
Address review (#2961): from_json('x') and from_json() previously fell through to a silent passthrough of the unparsed value. Reject any parenthesized form with a clear error so mis-wired templates fail loudly. Rename test to ...parses_object (JSON under test is an object) and add coverage for the strict no-arguments behavior.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* docs(workflows): document the from_json expression filter
Address Copilot review: the user-facing filter references omitted the
newly added `from_json` filter. Add it to the ARCHITECTURE.md filter table
(with the `{{ steps.emit.output.stdout | from_json }}` example) and to the
filter enumerations in workflows/README.md and docs/reference/workflows.md
so the docs match the evaluator's capabilities.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(workflows): make from_json strictness reject trailing tokens; fix docstring
Address Copilot review:
- Strictness only rejected parenthesized forms, so typos like
`| from_json)` or `| from_json extra` still fell through to the
unknown-filter path and silently returned the unparsed value. Match on
the leading filter token and require the whole filter to be exactly
`from_json`, so every mis-wired form raises. Extend the rejection test to
cover the trailing-token cases.
- The module docstring claimed "no imports", which is misleading now that
the module imports `json`. Reword to state the actual sandbox guarantee:
templates cannot do file I/O, import modules, or run arbitrary code.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Initial plan
* Add init workflow step to bootstrap projects like `specify init`
* Address review: simplify stderr capture and extract VALID_SCRIPT_TYPES
* Address review: fail fast on non-empty dir, stdout fallback, README force fix
* Populate exit_code/stdout/stderr in non-empty-dir fast-fail
* fix: address three unresolved review comments in InitStep
- Use `with os.scandir(...)` context manager so the iterator is always
closed even when `any()` short-circuits, preventing file-descriptor
leaks in long-running workflow runs.
- Guard `os.chdir(prev_cwd)` in the `finally` block with a try/except
so an `OSError` (e.g. directory deleted) doesn't bypass returning
the captured `StepResult`.
- Reject non-string `script` values in `validate()` with a clear error
message, rather than silently passing them through to become
`--script True` at runtime.
* Potential fix for pull request finding 'Empty except'
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
* fix: remove no_git and branch_numbering options removed upstream
The --no-git and --branch-numbering flags were removed from `specify init`
on main. Update InitStep to drop these unsupported config fields and fix
tests accordingly.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address review — integration defaults, integration_options, engine-owned dirs
- Apply DEFAULT_INIT_INTEGRATION fallback when neither step config nor
workflow context provides an integration, so output.integration always
reflects the actual integration used.
- Add integration_options config field to support --integration-options
passthrough (required for generic integration and --skills mode).
- Exclude .specify/ from the non-empty directory fast-fail check so that
here: true works when the engine has already created its run-state
directory before steps execute.
- Note: mix_stderr=False is not needed — Click 8.2+ captures stderr
separately by default and the existing try/except handles access.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: implicitly add --force when only engine-owned dirs exist
When the workflow engine creates .specify/workflows/runs/ before steps
execute, the directory is technically non-empty. Previously, specify init
would prompt for confirmation (hanging in unattended mode) unless the
user explicitly set force: true. Now the step detects that only
engine-owned directories (.specify/) are present and implicitly adds
--force so init proceeds without user interaction.
Also fixes the test to exercise the implicit-force path rather than
passing force: True explicitly (which bypassed the check entirely).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: derive VALID_SCRIPT_TYPES from shared constant, fail fast on OSError, include all resolved fields in output
- Derive VALID_SCRIPT_TYPES from SCRIPT_TYPE_CHOICES in _agent_config
so the valid set cannot drift from the specify init CLI.
- Fail fast with a clear error when os.scandir() raises OSError (e.g.
permission denied) instead of silently treating the directory as empty.
- Include preset, force, and ignore_agent_tools in all output dicts
(both fast-fail and normal paths) for consistent interpolation and
debugging downstream.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: populate stderr from stdout on older Click, fix force comment wording
- When Click does not expose result.stderr (older versions where stderr
is mixed into stdout), use stdout as stderr on non-zero exit so
workflows can consistently read steps.<id>.output.stderr for errors.
- Update README inline comment for force: wording to say 'when target
directory already exists' rather than 'non-empty directory', matching
the actual specify init behavior for the project: form.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: build argv flags before early returns, use any() for dir scan
- Move argv flag-building (--integration, --script, --preset,
--ignore-agent-tools) before the non-empty-dir and OSError early
returns so output['argv'] always reflects the complete command.
- --force is appended after the check since it may be set implicitly.
- Replace list comprehension with any() generator expression to
short-circuit without allocating a full list of DirEntry objects.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: only treat .specify as engine-owned when it is a real directory
A file or symlink named .specify should not be excluded from the
non-empty check. Use entry.is_dir(follow_symlinks=False) to ensure
only an actual directory is considered engine-owned content.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: guard implicit force for engine dirs only, fix integration fallback order
- Only set implicit --force when engine-owned directories (.specify/)
are actually present. A completely empty directory no longer gets
--force added unnecessarily.
- Fix integration resolution precedence: resolve step config expression
first, then fall back to workflow default (also resolved), then to
DEFAULT_INIT_INTEGRATION. Previously, a step expression resolving to
falsy would bypass the workflow default entirely.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Manfred Riem <15701806+mnriem@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Manfred Riem <mnriem@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(workflows): opt-in output_format: json exposes parsed shell stdout as output.data
No step that runs external code could hand a typed value to a later
step, so e.g. a fan-out could never consume a runtime-computed
collection. With output_format: json declared, stdout is parsed and
exposed under output.data (raw keys unchanged); a parse failure fails
the step with a clear error. Without the key, behavior is unchanged.
Reference implementation for the proposal in #2962.
Addresses #2962
* test(shell): emit JSON via sys.executable for cross-platform output_format tests
Address review (#2963): replace non-portable echo '{...}' (Windows cmd.exe keeps the single quotes, breaking JSON) with the established '"{py}" "{script}"' pattern using sys.executable + a temp script, so the output_format tests pass on the Windows CI matrix. Also make the validate test's run inert (exit 0).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* fix: non-zero exit code when a workflow run ends failed or aborted
workflow run and workflow resume printed Status: failed (or emitted the
--json payload) and exited 0, so scripts and CI could not rely on the
process exit code. Map terminal outcomes: failed|aborted -> 1,
completed|paused -> 0, on both the text and --json paths.
The previous exit-0-on-failed behavior was pinned by
test_workflow_run_failing_yaml_without_project; the pin is updated to
the new contract.
Fixes#2958
* test: portable exit-code step commands + cover resume failed->exit-1
Address review (#2959): replace non-portable run: 'true'/'false' with 'exit 0'/'exit 1' (Windows cmd.exe has no true/false builtins under shell=True), and add an end-to-end 'workflow resume' test asserting a resumed failed run exits non-zero.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Initial plan
* Add workflow step catalog: StepRegistry, StepCatalog, CLI commands, and tests
Agent-Logs-Url: https://github.com/github/spec-kit/sessions/2885e646-477d-4df8-b9a3-06d8cb29e748
Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
* Potential fix for pull request finding 'An assert statement has a side-effect'
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
* Address PR review: path traversal, cache robustness, collision check, failed-to-load display
- Add resolve()+relative_to() path traversal guards in workflow_step_add and
workflow_step_remove to prevent directory escape via step_id
- Harden _is_url_cache_valid in both StepCatalog and WorkflowCatalog to
coerce fetched_at to float and catch TypeError/ValueError
- Check STEP_REGISTRY and StepRegistry before installing to prevent
collisions with built-in step types or already-installed steps
- Show 'Custom (installed, failed to load)' section in workflow step list
for steps in the registry that failed to load into STEP_REGISTRY
* Fix StepRegistry shape validation and StepCatalog empty-YAML handling
Agent-Logs-Url: https://github.com/github/spec-kit/sessions/0dca6393-f5a9-40de-bb5c-77ba6af033d2
Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
* Polish: rename _default to default_registry, strengthen unreadable-file test
Agent-Logs-Url: https://github.com/github/spec-kit/sessions/0dca6393-f5a9-40de-bb5c-77ba6af033d2
Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
* Address PR review: atomic install, hostname validation, cache resilience, no dynamic imports in list/info
Agent-Logs-Url: https://github.com/github/spec-kit/sessions/3e18fef0-e2e6-4b3e-9e8d-9adb1e5e464e
Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
* Fix shutil.move with existing step_dir: remove before move to avoid subdirectory nesting
Agent-Logs-Url: https://github.com/github/spec-kit/sessions/3e18fef0-e2e6-4b3e-9e8d-9adb1e5e464e
Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
* Call load_custom_steps at execution time; enforce hostname in _safe_fetch and _validate_url
Agent-Logs-Url: https://github.com/github/spec-kit/sessions/73865880-fb25-4061-a43e-4e4b4d1c4de6
Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
* Wrap YAML parsing in try/except; atomic step install via os.rename() under same fs
Agent-Logs-Url: https://github.com/github/spec-kit/sessions/ff915bc5-ec7e-4e6a-b505-35f5795250df
Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
* Validate YAML root is a dict in _load_catalog_config and workflow_step_add; fix WorkflowCatalog hostname validation
Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
* Fix load_custom_steps() package imports and add reserved step ID validation
* Move _re/_sys imports out of loop and _RESERVED_STEP_IDS to module level
* Address review: collision-resistant module names, extra_files support, remove orphan dir
* Harden extra_files: warn on non-dict, resolve symlinks in path traversal check
* Switch _safe_fetch and StepCatalog._fetch_single_catalog to use open_url for auth consistency
* Harden step_id validation against path-segment tricks; raise on StepRegistry.save() OSError
* Clean up sys.modules on broken step packages; handle StepValidationError in step add/remove
* Address review thread: int-coerce priorities, sys.modules cleanup, _require_specify_project, registry-first remove
* fix: normalize workflow step catalog metadata fallbacks
* fix: address latest workflow step and catalog review findings
* Handle non-string extra_files keys in workflow step add
* Harden StepRegistry symlink reads and extra_files path/URL validation
* Harden custom step loader and step remove against symlinks and OSError
* Fix StepCatalog.search() to coerce non-string fields before joining
* Fix WorkflowCatalog YAML parsing error handling and isinstance checks
* Harden step registry save and custom step/catalog ID handling
* Harden cache validation and staging OSError handling
* Address review: reorder symlink guard and split mixed test
- Move symlink-parent check before is_dir() in load_custom_steps() so
we never stat an external target through a symlink
- Split test_get_merged_steps_normalizes_list_ids_to_strings into two
focused tests: one for list-id normalization, one for get_step_info
return values
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review: symlink-before-stat in loader, restore registry on rmtree failure
- load_custom_steps(): check is_symlink() before is_dir() on step
directories so symlinked entries are skipped without statting external
targets
- workflow_step_remove: restore the registry entry when shutil.rmtree()
fails so filesystem and registry state stay consistent and a future
'step add' isn't blocked
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Harden step_id validation and file-write error handling
- _validate_step_id_or_exit: reject whitespace-only/padded IDs,
Windows-invalid characters (<>:"|?*), control characters, trailing
dots/spaces, and Windows reserved device names (con, nul, etc.)
- Wrap step.yml/__init__.py staging writes in OSError handler
- Wrap extra_files disk writes (mkdir + write_bytes) in OSError handler
that names the failing relative path
- Registry rollback on rmtree failure: restore verbatim metadata and
emit a warning if the restore itself fails
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review: cache symlink guard, verbatim registry rollback, Windows test fix
- StepCatalog: add _is_cache_path_safe() guard that checks for symlinks
in .specify/workflows/steps/.cache path; skip cache reads and writes
when any component is symlinked to prevent writes outside project root
- Registry rollback: write metadata directly to registry.data['steps']
and call save() instead of using add() which overwrites timestamps
- temp_dir fixture: use ignore_errors=True on Windows to avoid flaky
teardown from locked file handles (WinError 32)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Simplify exec_module call by removing redundant nested try/except
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix empty YAML tolerance in WorkflowCatalog.add_catalog, scope ignore_errors to Windows
- WorkflowCatalog.add_catalog(): treat None from yaml.safe_load() (empty
file) as an empty mapping instead of raising 'corrupted'
- temp_dir fixture: limit ignore_errors to sys.platform == 'win32' so
real cleanup issues surface on non-Windows platforms
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Chain exceptions in _load_catalog_config for both catalog classes
Add 'from exc' to preserve root cause in tracebacks while keeping
clean user-facing messages.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Make default catalog tests hermetic by isolating HOME
Monkeypatch Path.home() to project_dir and clear catalog env vars so
tests don't break on machines with a real ~/.specify/step-catalogs.yml
or ~/.specify/workflow-catalogs.yml.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix falsy ID handling in _get_merged_steps for list-based catalogs
Check for None explicitly instead of using 'or' which drops valid
falsy IDs like 0.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Compare reserved step IDs case-insensitively for filesystem safety
On case-insensitive filesystems (Windows, common macOS), variants like
STEP-REGISTRY.JSON would collide with the actual registry file.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add explanatory comments to intentional empty except blocks
Document why cache-read failures are silently ignored in both
WorkflowCatalog and StepCatalog _fetch_single_catalog methods.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
Co-authored-by: Manfred Riem <mnriem@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
