Two more current Copilot review findings, both in workflow_add/update:
- `workflow add --from`'s `finally: tmp_path.unlink(missing_ok=True)` ran
unguarded after `_validate_and_install_local` had already committed the
file and registry entry (success) or already raised its own clean
`typer.Exit` (failure). An OSError from that cleanup unlink would
surface as an unhandled failure even though the install itself
succeeded. It is now wrapped in try/except OSError, printing a neutral
warning that doesn't claim success or failure (the finally runs on both
outcomes) instead of propagating.
- `workflow_update`'s per-item loop performed its own outer backup
(`wf_file.read_bytes()`) and restore (`wf_file.write_bytes(backup)`)
around `_install_workflow_from_catalog`, which is itself fully
transactional (staged download, atomic rename-based commit, its own
rollback on registry failure) and never leaves a raw OSError or a
partially-written workflow.yml. The outer restore was therefore dead
weight for its stated purpose, and — being an unguarded byte-level
write — was itself an unnecessary place a second failure could truncate
an already-safely-preserved file. Removed; the loop now only records
success/failure.
Also marks 3 registry-save file-mode tests
(`test_registry_save_preserves_existing_file_mode`,
`test_registry_save_on_new_registry_uses_secure_default_mode`,
`test_registry_save_failure_preserves_file_on_disk`) as POSIX-only via
the repo's existing `skipif(sys.platform == "win32", ...)` pattern, since
they assert exact POSIX permission bits that don't hold on Windows.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Address 3 current Copilot review findings on the disabled-workflow guard
in `workflow run`/`workflow resume`:
- The lexical `.specify/workflows/<id>` ownership scan stopped at the
first match scanning from the start of the path. A nested project
living beneath an outer installed workflow's own directory tree (reusing
the same segment names) was attributed to the wrong (outer) workflow
and ID, gating the run on an unrelated workflow's disabled state.
`_scan_for_workflow_owner` now scans from the end so the nearest
(innermost) owner always wins.
- A path with no `.specify/workflows` segments of its own (e.g.
`/tmp/alias.yml`) that is itself a symlink resolving *into* installed
storage bypassed the disabled check entirely, since only the raw
lexical path was inspected. `_resolve_installed_workflow_ownership` now
additionally resolves the real path when the lexical scan finds no
owner and re-runs the same scan against it, so an outward-pointing
alias into a disabled workflow is caught too. Genuinely standalone
external files (no symlink anywhere on the path) are unaffected.
- `workflow resume` bypassed the disabled check altogether: engine.resume()
replays a persisted run directly from disk with no registry awareness.
RunState now optionally persists `installed_workflow_id` and
`installed_registry_root` at run start (set by workflow_run when the
source resolved to an installed ID); `workflow_resume` pre-loads the
run state and re-checks the registry's *current* disabled state before
calling engine.resume(), mirroring workflow_run's own guard. Both new
fields default to None via RunState.load()'s `.get()`, so runs from a
direct/non-installed source, and any run persisted before this schema
addition, resume exactly as before.
The ownership-mapping logic (previously inlined in workflow_run) is
extracted into `_resolve_installed_workflow_ownership` /
`_scan_for_workflow_owner` so both the lexical and resolved-path cases
share the same scan and the existing inward-symlink-component refusal.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
_stage_workflow_file() does dest_dir.mkdir(parents=True, exist_ok=True)
then tempfile.mkstemp(dir=dest_dir, ...). For a fresh install (no prior
directory), if mkdir succeeds but mkstemp then raises (disk
full/EMFILE/quota), the exception previously propagated straight past
both the local-install and catalog-install call sites without any
cleanup, leaving the newly-created empty workflow directory orphaned
on disk with no error indicating why.
Fix at the shared _stage_workflow_file() boundary instead of duplicating
cleanup at each call site: track whether this call created dest_dir: on
a mkstemp failure, remove that directory via a guarded rmdir (never a
broad rmtree, so any concurrently written content would be left
untouched) before re-raising the original OSError unchanged. A
pre-existing (reinstall) dest_dir is never touched by this cleanup,
and a cleanup failure is reported as its own warning without masking
the original error.
Add red-first regression tests proving: a fresh local install (--dev,
plain local path, --from) and a fresh catalog install both clean up the
orphaned directory on a simulated mkstemp failure, and a reinstall over
a pre-existing directory is left untouched by the same failure.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
_commit_workflow_file() renames a prior workflow.yml aside to
workflow.yml.bak so it can be restored if registry.add() subsequently
fails. Neither the local install/reinstall path nor the catalog
install/reinstall path ever cleaned up that backup after a successful
registry.add() -- every successful reinstall permanently left a
workflow.yml.bak sibling, which later reinstalls would silently
overwrite/re-orphan.
Add a shared _discard_committed_backup_file() helper, called from both
success paths right after registry.add() durably succeeds (and before
the final "installed" message, preserving output ordering). A fresh
install (backup_file is None) is a no-op. A cleanup failure is reported
as a warning (exit 0), not a failure, since the install itself already
succeeded -- consistent with workflow_remove's post-commit cleanup
warning semantics.
Add red-first regression tests proving: (1) successful local reinstall
leaves no workflow.yml.bak sibling, (2) successful catalog reinstall
leaves no workflow.yml.bak sibling, (3) a cleanup failure on the backup
file after a successful reinstall reports a warning and still exits 0
with the registry correctly reflecting the new install.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Addresses 5 Copilot review findings on HEAD b8269c8, all centered on
transaction integrity around workflow install/remove/registry writes,
following the atomic_write_json pattern already used in _utils.py:
1. WorkflowRegistry.save() now preserves the existing registry file's
mode (e.g. 0640/0644) across a save instead of silently downgrading
it to mkstemp's 0600 default; a brand-new registry still gets the
secure 0600 default.
2. workflow_remove now stages the install directory out of the way via
an atomic rename *before* the registry write, rather than deleting
it directly with shutil.rmtree after the registry already claims it
removed. This closes a real data-integrity gap: a partially-failed
rmtree could no longer leave a damaged directory re-marked
"installed" by the old manual restore-after-rmtree-failure code
(now deleted -- it's structurally impossible to need it). A
registry-write failure renames the staged directory back
(guarded, with an explicit warning if the restore-back rename
itself fails); a registry-write success is durable, so a later
failure to delete the staged directory is now a warning (exit 0),
not a contradictory "Error: Failed to remove" (exit 1) that used to
claim failure while the registry already recorded success.
3. Local (--dev/--from/plain path) and catalog install/reinstall now
write new content to a same-directory staging file and commit it
onto the destination workflow.yml via a single atomic swap, instead
of writing/downloading directly into the destination file. A prior
file (reinstall) is renamed aside rather than overwritten in place,
so it can be restored via rename -- never a content rewrite -- if
registry.add() subsequently fails; a rollback failure is now
explicitly reported as a warning instead of escaping unguarded and
masking the original clean error. This also removes the need to
read the prior file's bytes into memory before installing (that
read-before-write step and its failure mode are now unreachable),
and both local and catalog installs share the same four small
helpers (_stage_workflow_file / _commit_workflow_file /
_discard_staged_workflow_file / _rollback_committed_workflow_file,
plus guarded wrappers) rather than duplicating the logic.
4. Updated a stale comment (workflow_run's ownership-guard rationale)
that still described WorkflowRegistry._load() as silently
substituting an empty registry; it now fails closed by raising
OSError, which the comment now states plainly.
Tests: rewrote the two workflow_remove tests whose assertions encoded
the old (incoherent) rmtree-then-restore contract to instead prove the
new stage-then-commit contract (post-registry-success cleanup failure
is a warning+exit 0; pre-registry-success stage-restore failure is
guarded and escapes markup correctly). Rewrote the local/catalog
"backup read failure" tests, which tested a step the new design no
longer performs, into "restore-rename failure" tests proving the new
guarded rollback boundary. Added registry file-mode preservation tests.
All other existing install/remove/reinstall tests (save-failure
cleanup, pre-existing-empty-dir handling, early-failure-during-
reinstall parametrized cases, Rich markup escaping) continue to pass
unmodified against the new implementation.
Verified via GraphQL that all 5 threads are current (not outdated/
resolved) before fixing. Full suite: 3996 passed, 110 skipped. Ruff
clean on all touched files.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
workflow_add's --from download path opened a NamedTemporaryFile(delete=False)
-- which creates the file on disk immediately -- then wrote the size-limited
response body before assigning `tmp_path`. If `_read_response_within_limit`
raised (oversized declared Content-Length, or an over-cap streamed body with
no/understated Content-Length), the exception propagated out of the `with`
block before `tmp_path` was ever set, so the outer except handler had no
path to clean up: a 0-byte `.yml` temp file was left behind permanently on
every rejected/failed --from download. Fixed by assigning `tmp_path`
immediately after the file is opened (before the size-limited read/write),
and unlinking it in the except branch when set. Normal post-download cleanup
in the existing `finally: tmp_path.unlink(missing_ok=True)` is unchanged.
Verified (not assumed) the catalog install path has no equivalent leak: it
writes the response bytes directly to `workflow_file` inside `workflow_dir`
(no separate temp file), and any read/size-limit failure is already caught
by the existing `except Exception: _cleanup_failed_install()` handler, which
correctly restores a reinstalled file or removes a freshly-created directory.
While investigating, found the previous round's 4 size-limit tests were
false positives: `_read_response_within_limit`'s `max_bytes` parameter had
its default bound to `_MAX_WORKFLOW_YAML_BYTES` at function-definition time,
so monkeypatching the module attribute in tests had no effect on the
function's actual behavior -- the tests were passing because the oversized
mock bodies failed downstream YAML/id validation instead of the size check.
Fixed by resolving `max_bytes` from the module attribute at call time
(default `None`, resolved inside the function body) so tests can actually
override the effective limit, and strengthened all 4 tests' assertions to
match the specific size-limit error text (whitespace-collapsed to tolerate
Rich's line-wrapping), so they now prove the real code path fires.
Tests: added 2 red-first regression tests (oversized-streamed-body and
oversized-Content-Length --from downloads leave no leftover temp file,
verified against a scratch tempfile.tempdir), confirmed red (real 0-byte
file found) before the fix and green after. Strengthened the pre-existing
4 --from/catalog size-limit tests to assert on the actual error message
instead of generic exit-code/non-empty-output checks.
tests/test_workflows.py: 487 passed
tests -k bundler: 186 passed
tests -q: 3994 passed, 110 skipped
ruff check: clean on all touched files
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1. bundle remove: BundlerError raised by the primitive installer itself
(e.g. from a kind manager) bypassed the partial-removal bookkeeping
message added previously via a bare `except BundlerError: raise`. Now
routes through the same detail-construction logic as generic
exceptions, so a mid-loop BundlerError after an earlier successful
removal still reports that the project may be partially uninstalled,
while a zero-removal BundlerError still reports "No components were
removed." Both preserve the original exception message and chain
`from exc`.
2/3. workflow add --from and catalog install/update downloads used
unbounded `response.read()`, buffering the entire server-controlled
body into memory before any size check, and trusted Content-Length
alone where checked at all. Added a single shared
`_read_response_within_limit()` helper reused by both call sites: it
fails fast on an oversized declared Content-Length, and separately
enforces the same cap while streaming in 64KiB chunks so a chunked or
Content-Length-less response cannot bypass the limit by lying about or
omitting its size. Chose 5 MiB as the cap: workflow YAML definitions
are small step/metadata text, not binaries, so this is generous
headroom against a malicious/misbehaving server without affecting any
legitimate workflow definition. Both call sites already route any
raised exception through their existing clean-error and rollback
(`_cleanup_failed_install`) paths, so no additional error-handling
plumbing was needed.
Tests: extended the shared `_FakeResponse` test helper (and 5 duplicate
per-test FakeResponse classes) to support `.read(amt)` chunked reads with
an internal cursor (backward compatible with existing bare `.read()`
callers) plus header simulation. Added red-first tests for: BundlerError
after partial removal reporting partial state, BundlerError with zero
removals reporting no changes, --from oversized-Content-Length rejection,
--from oversized-streamed-body-without-Content-Length rejection, and the
same two cases for the catalog install path (asserting no orphan
directory/registry mutation on rejection).
tests/integration/test_bundler_install_flow.py: 17 passed
tests/test_workflows.py: 485 passed
tests -q: 3992 passed, 110 skipped
ruff check: clean on all touched files
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1. WorkflowRegistry._load(): a symlinked .specify/.specify/workflows
parent (or a symlinked registry file) silently returned an empty
registry instead of raising, unlike an unreadable-file read failure.
A read-only caller (notably the bundler's remove path) querying
is_installed() before ever writing could conclude an installed
workflow is absent, skip removing it, then delete the bundle
record -- leaving the workflow untracked but still on disk. Now
raises OSError immediately, matching the existing unreadable-file
fail-closed behavior.
2/8. _validate_and_install_local and _install_workflow_from_catalog:
when the destination directory already existed but had no prior
workflow.yml (e.g. a leftover empty dir), existed_before was True
but there were no backup bytes to restore, so the rollback closure
did nothing on a later failure -- leaving the newly copied/
downloaded file behind. Both now unlink the newly created file in
this case, restoring the pre-existing directory to its prior
(empty) state.
3/4. Both install paths read the prior workflow.yml bytes (to seed
the reinstall rollback) *before* any try/except boundary: a read
failure on the existing file (e.g. a transient permission/FS
issue) leaked a raw, unescaped OSError instead of the same clean
CLI error used by every other failure branch in these functions.
Both reads are now guarded by their own try/except OSError, with
no writes attempted before the read succeeds (so there is nothing
to roll back on this specific failure).
5. remove_bundle's exception-conversion message unconditionally
claimed "No changes were recorded," even though a failure can
occur after earlier components in the same bundle have already
been removed from disk (save_records never runs on this path, so
the record is left claiming the bundle fully installed). The
message now reports how many components were already removed
when that happened, instead of asserting no changes occurred.
6/7. workflow_remove's new post-registry-removal directory-failure
error and its restore-failure warning interpolated workflow_dir
and the exception values into Rich markup unescaped. A project
path or OS error message containing Rich-markup-like brackets
could be parsed as markup and hide/corrupt the displayed text.
Both now use the existing _escape_markup helper, consistent with
every other error path in this file.
Tests (tests/test_workflows.py unless noted):
- TestWorkflowRegistry::test_load_symlinked_workflows_dir_fails_closed_not_silently_empty (1)
- TestWorkflowCliAlignment::test_add_dev_fresh_install_into_preexisting_empty_dir_cleans_new_file (2)
- TestWorkflowCliAlignment::test_add_catalog_fresh_install_into_preexisting_empty_dir_cleans_new_file (8)
- TestWorkflowCliAlignment::test_add_dev_reinstall_backup_read_failure_gives_clean_error (3)
- TestWorkflowCliAlignment::test_add_catalog_reinstall_backup_read_failure_gives_clean_error (4)
- tests/integration/test_bundler_install_flow.py::test_remove_partial_failure_message_reflects_partial_state (5)
- TestWorkflowRemoveGuard::test_remove_directory_and_restore_failure_escapes_rich_markup (6/7)
All seven were confirmed red beforehand, matching each thread's
described failure mode exactly (silent empty registry instead of a
raise; orphaned new file left behind; raw unescaped OSError leaking;
a misleading "no changes were recorded" claim; Rich markup consuming
bracketed path/exception text). Also updated
test_registry_save_refuses_symlinked_parent, a pre-existing test that
asserted the symlinked-parent raise at add()/save() time -- it now
raises at construction instead, per fix#1, so the test was adjusted
to match without weakening its guarantee (still asserts no writes
occur under the symlinked target).
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
remove_bundle() had no exception handling around its component
removal loop, unlike install_bundle() which converts any raw
exception into a clean BundlerError. Since WorkflowRegistry now
fails closed (raises OSError) on an unreadable registry file,
and _WorkflowKindManager.__init__ constructs WorkflowRegistry
with no try/except, an unreadable workflow registry surfaced as
a raw OSError through remove_bundle(). The bundle_remove CLI
command only catches BundlerError, so the raw OSError propagated
uncaught, producing exit_code=1 with empty output instead of a
clean, actionable message.
Wrap remove_bundle()'s component loop in the same
try/except BundlerError: raise / except Exception: raise
BundlerError(...) from exc pattern already used by
install_bundle(), converting any raw exception at this shared
boundary. save_records() remains outside the try block, so a
failure still leaves the bundle's record untouched (no removal
side effects recorded).
Tests:
- tests/integration/test_bundler_install_flow.py::test_remove_converts_raw_installer_exception_to_bundler_error
(function-level regression: a raw OSError from installer.is_installed
must become a clean BundlerError, and the bundle record must survive)
- tests/contract/test_bundle_cli.py::test_remove_reports_clean_error_when_primitive_raises_raw_exception
(CLI-level regression: `specify bundle remove` must print a clean
actionable message and exit non-zero instead of raw/empty output)
Both tests were confirmed red beforehand: the raw OSError propagated
uncaught out of remove_bundle(), and the CLI-level CliRunner result
showed exit_code=1 with empty output.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
workflow run's ownership check derived registry_root/registered_id from
the lexical path, then checked the id directory and workflow.yml leaf for
symlinks -- but never checked .specify or .specify/workflows themselves
for that derived root. _reject_unsafe_workflow_storage only guards the
cwd's project_root, which can differ from the path-derived registry_root
(a direct path into an unrelated project, or that project's own .specify
being a symlink to an attacker-controlled tree). WorkflowRegistry's own
symlinked-parent handling silently substitutes an empty registry instead
of raising, so a query against it (is_installed/get returning "not
found") is not a safety signal a caller can rely on: with a symlinked
.specify, the disabled check saw no registry entry and let a disabled
workflow run anyway.
Fix: reject an unsafe .specify/.specify-workflows for the actual derived
registry_root before ever consulting the registry, reusing the existing
_reject_unsafe_dir helper already used by _reject_unsafe_workflow_storage.
Red-first end-to-end repro: victim project's .specify symlinked to an
attacker-controlled tree containing a disabled workflow entry, run
invoked with a direct path from an unrelated cwd -- confirmed the
disabled workflow executed (exit 0) before the fix, now refused cleanly.
Tests: tests/test_workflows.py 475 passed. Full suite: 3977 passed, 110
skipped. ruff check: all checks passed.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1. workflow run ownership check followed symlinks via Path.resolve()
before mapping a direct YAML path back to its installed workflow ID.
A symlinked .specify/workflows/<id>/workflow.yml resolved outside the
tree, missed the ownership match entirely, and let the disabled-workflow
guard be silently skipped while engine.load_workflow still followed the
symlink. Now maps ownership from a lexically-normalized path (os.path.
normpath, no symlink following) and explicitly refuses to run if the
installed <id> directory or workflow.yml leaf is itself a symlink.
Direct external workflow paths that don't match .specify/workflows/...
are unaffected.
2. WorkflowRegistry._load() caught a read OSError and silently fell back
to an empty in-memory registry, only blocking a later save(). Callers
that only query is_installed()/get()/list() before writing a file
(e.g. commands/init.py's bundled speckit install, which overwrites
workflow.yml once is_installed() reports false) could act on that
false-empty state and destroy real data before ever reaching save().
_load() now raises OSError immediately so an unreadable registry fails
closed at construction, before any query or side effect is possible.
Added _open_workflow_registry() to give every CLI command a consistent
clean-error boundary around registry construction.
3. _validate_and_install_local's mkdir/copy2 ran before the try/except
that protected registry.add(); a copy2 failure (e.g. a truncating
partial write on a reinstall) was not caught at all, so the existing
backup-restore cleanup never ran and the prior working workflow.yml
was corrupted with a raw traceback surfaced to the user. mkdir/copy2
now run inside the same rollback-protected section as registry.add(),
sharing one _cleanup_failed_install() helper.
4. workflow update's skip message claimed any non-catalog source was
installed "from a local path or URL", which is wrong for the bundled
speckit workflow (source: "bundled"). Message is now source-neutral.
Verified all 4 threads are current (not outdated) via GraphQL review
thread query on PR #3419, HEAD 812050a.
Tests: strict TDD per fix (red test proving each bug, minimal production
change, green). tests/test_workflows.py: 474 passed. Full suite: 3976
passed, 110 skipped. ruff check: all checks passed on touched files and
full src tree.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
workflow_remove now persists registry.remove() before deleting any
files (fixed previously), but if the registry write succeeds and the
subsequent shutil.rmtree(workflow_dir) then fails, the registry was
left claiming the workflow uninstalled while its directory remained on
disk -- an orphaned install with no path back to a clean state.
workflow_step_remove already handles this exact sequencing by capturing
the registry entry before removal and restoring it directly into
registry.data plus save() (bypassing add(), which would stamp a new
updated_at) if the directory removal fails afterwards.
Applied the same pattern to workflow_remove: capture registry_metadata
via registry.get() before registry.remove(), and on an rmtree OSError,
write it straight back into registry.data["workflows"][workflow_id] and
save(), matching workflow_step_remove's restore-failure handling (a
yellow warning, not a hard failure, since the primary error is already
about to be reported). Existing error message and exit behavior for the
rmtree failure are unchanged.
Added a failing-first regression: install a workflow, monkeypatch
shutil.rmtree to raise OSError, and assert a clean existing error
message, the directory remaining (rmtree never actually deleted
anything), and the registry entry restored byte-for-byte identical
(including installed_at/updated_at) -- proving the fix bypasses add()
and doesn't re-stamp timestamps. Confirmed red (registry entry stayed
None) before the fix, green after.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
_install_workflow_from_catalog is new in this PR and has seven failure
branches after the mkdir/download step, each independently rmtree'ing
workflow_dir: redirect-to-non-HTTPS rejection, a generic download
exception, invalid downloaded YAML, a validate_workflow failure, a
workflow-id/catalog-key mismatch, a version mismatch, and (fixed in the
prior commit) a registry.add() OSError. Only the last one had been
special-cased to spare a prior working install on reinstall; the other
six still unconditionally deleted the whole directory, so re-adding an
already-installed catalog workflow and hitting any of those six earlier
failures destroyed the working install even though nothing about it had
actually changed.
Replaced all seven ad hoc rmtree call sites with a single local
_cleanup_failed_install() helper that closes over the existed_before /
prior_workflow_bytes captured once at the top of the function: restore
the prior workflow.yml for a reinstall, or rmtree only a directory that
this attempt itself created. Every failure branch now calls this one
helper, so the fix is structural rather than duplicated, and every
existing error message/exit code is unchanged -- only the cleanup
performed before each message is different.
Added a parametrized regression test covering the four early-failure
trigger points reachable from plain workflow add (redirect rejection,
download exception, invalid YAML, ID mismatch): each installs a catalog
workflow, re-adds it while forcing that specific failure, and asserts a
clean error plus the original workflow.yml surviving byte-for-byte.
Confirmed red against the unfixed code (all four raised FileNotFoundError
reading the deleted file) before applying the helper, green after.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
_install_workflow_from_catalog's final registry.add() failure handler
unconditionally rmtree'd workflow_dir. That's safe for a brand-new
install, but plain `workflow add <catalog-id>` also allows re-adding an
already-installed workflow, downloading the new version over the
existing directory first. If registry.add() then failed to save, the
unconditional rmtree deleted the prior working install while the
registry (after its own rollback) still reported it installed -- data
loss with no way back. workflow_update already avoids this via an outer
backup/restore around this function, but plain add has no such caller.
Fix mirrors _validate_and_install_local's existed-before/backup-aware
handling: capture whether workflow_dir existed and back up its
workflow.yml bytes before any download write, then on a registry.add()
OSError, restore those bytes for a reinstall or rmtree only a
brand-new directory. Only one file (workflow.yml) is ever written by
this path, so no further per-file bookkeeping is needed.
Added a failing-first regression: install a catalog workflow, re-add it
with a simulated registry save OSError, and assert a clean error, the
original workflow.yml restored byte-for-byte, and the registry still
reporting the original version installed. Confirmed red (prior file
deleted) before the fix, green after.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Critical: WorkflowRegistry.remove() deleted the in-memory entry then
called save() with no rollback, unlike add(). Combined with
workflow_remove deleting the workflow directory before calling
registry.remove(), a save failure permanently destroyed the workflow's
files, left the on-disk registry still claiming it installed, and
surfaced a raw unhandled OSError with no CLI message.
- WorkflowRegistry.remove() now rolls back the in-memory entry on a
save() OSError, mirroring add()'s existing rollback pattern.
- workflow_remove persists the registry removal (registry.remove(),
wrapped in try/except OSError -> clean escaped message) before
deleting any files, so a save failure never touches the workflow
directory.
Important sibling paths: workflow add (local/--dev/--from and catalog),
enable, and disable all called registry.add() without catching its
deliberate OSError, so a save failure surfaced either an orphaned
install directory (fresh local/catalog installs) or a raw/unhandled
exception with no clean CLI output.
- _validate_and_install_local (backs local/--dev/--from) now removes
the freshly created directory on a fresh install, or restores the
prior workflow.yml bytes on a reinstall-over-existing-local install,
before raising a clean escaped error.
- _install_workflow_from_catalog wraps the final registry.add() using
the function's own established convention (rmtree the just-downloaded
workflow_dir, then a clean escaped error) -- workflow_update's
existing backup/restore around this function is unaffected.
- workflow_enable/workflow_disable catch registry.add()'s OSError and
print a clean escaped message instead of leaking the exception.
Added failing-first tests proving each behavior (registry-unit rollback
test, CLI-level remove/add/enable/disable save-failure tests
parametrized where they share one root cause), all confirmed red before
the fix and green after.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- workflow add's plain local-path fallback (no --dev) checked wf_file.exists()
before installing, so a directory literally named workflow.yml passed the
guard and _validate_and_install_local() leaked an uncaught
IsADirectoryError instead of the documented CLI error. Use is_file(),
matching the --dev branch's existing guard.
- WorkflowRegistry._load() treated any OSError while reading an existing
registry the same as corrupted JSON, resetting to an empty in-memory
registry. A later save() would then silently persist that empty state via
os.replace, discarding every previously installed workflow entry. Track a
_load_error flag on OSError-during-read and have save() refuse to write
when it is set, so a transient I/O failure can no longer overwrite intact
data on disk.
- docs/reference/workflows.md: document `--from <url>` with its value
placeholder, matching extensions.md and presets.md.
Assisted-by: GitHub Copilot (model: Claude Sonnet 5, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
All three workflow download sites (add --from, catalog install, step
install) passed no redirect_validator to open_url, so an HTTPS URL
redirecting to cleartext HTTP issued the insecure request before the
post-hoc geturl() check reported it. Shared validator now rejects
non-HTTPS redirects (loopback HTTP allowed) pre-follow, matching the
preset download path.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- WorkflowRegistry now mirrors StepRegistry: _load refuses symlinked
parents/registry file and normalizes a non-dict workflows field;
save() rejects symlinked paths before writing.
- workflow add --dev requires workflow.yml to be a regular file so a
directory named workflow.yml gets the documented CLI error instead of
an uncaught IsADirectoryError.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- save() now uses tempfile.mkstemp in the workflows dir (matching the
engine's atomic writer), so a pre-created symlink at a predictable
.tmp path cannot redirect the write and concurrent processes cannot
collide.
- The direct-path disabled guard derives the owning project from the
resolved file path instead of the caller's cwd, so running an
installed workflow's YAML from outside the project still refuses when
disabled.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- save() wrote the registry with open('w'), so a failed dump truncated
the file and the next load reset every entry. Write to a sibling temp
file and os.replace into place.
- workflow update no longer claims all workflows are up to date when
some targets were skipped; it reports checked-only status with a
skipped count.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A corrupted-but-parseable registry entry (e.g. a string value) crashed
WorkflowRegistry.add with AttributeError on existing.get. Guard the
non-dict case while still restoring the original raw value on rollback.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- A truthy non-string catalog url (e.g. 123) reached urlparse and raised
AttributeError, escaping the clean error path; validate it is a string.
- enable/disable mutated the live registry entry before add(), so add's
rollback snapshot captured the already-toggled object; pass a fresh
mapping instead.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- WorkflowRegistry.add now rolls back its in-memory mutation when save()
raises, so a later successful save cannot persist metadata for a
failed update alongside the restored YAML backup.
- workflow run uses the same truthiness check for 'enabled' as list and
disable, so malformed values like 0 or null refuse to run.
- workflow update reports 'No workflows were eligible for update' when
every target was skipped instead of claiming all are up to date.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
'workflow add foo --from ""' fell through 'from_url or ...' to a
catalog install. Distinguish None from empty string so explicit values
stay on the URL-validation path and fail closed.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Running the installed copy's YAML directly (specify workflow run
.specify/workflows/align-wf/workflow.yml) skipped the registry check.
File sources resolving inside .specify/workflows/<id>/ now map back to
the workflow ID and refuse to run while disabled.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Path-equivalent spellings like "align-wf/" previously bypassed the
registry disabled check because the engine normalizes the path while the
registry matches the raw string. workflow run now validates non-file
sources against the workflow ID pattern before lookup.
Also updates docs/reference/workflows.md with --dev/--from install
options, update/enable/disable commands, and the search --author flag.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The update path never verified the downloaded workflow carries the catalog
version that triggered the update, so a stale or misconfigured URL could
report success while leaving the old version installed or downgrading it.
Pass the expected version into the install helper and fail the update when
the downloaded definition does not match.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
urlparse raises ValueError on e.g. an unbalanced IPv6 literal before the
invalid-URL branch is reached; on workflow update that also bypassed the
per-workflow handler and aborted the whole command. Convert the parse
failure into a clean error so add fails cleanly and update skips just the
affected workflow.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Covers the last few review threads not yet addressed:
- Escape yaml.YAMLError text in the local workflow add install path
(matches the already-escaped download/catalog paths).
- Escape the non---dev local directory fallback's "No workflow.yml
found in <path>" message (the --dev branch already escaped it).
- Escape the redirected final_url in the --from non-HTTPS redirect
error (IPv6 literals like http://[::1]/... are legal and contain
brackets).
- Escape the "Downloaded workflow is invalid" exception message in
_install_workflow_from_catalog, matching the sibling catalog-install
exception handler a few lines above it.
Adds regression tests for each in TestWorkflowCliAlignment, following
the existing escaping-test pattern in this class.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Transient FS errors (perms, disk full) from backup read or write no
longer abort the whole update run. The restore is wrapped in its own
try/except so a failed write only warns, and the offending workflow
is reported via 'Failed to update' like other per-workflow failures.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
workflow list now escapes id/name/version/description before printing,
matching how extensions render user-editable fields. The catalog install
helper computes safe_wf_id once and uses it for every early error path
plus the final failure message.
workflow update wraps _safe_workflow_id_dir and the backup read inside
the try/except typer.Exit block so an unsafe id in a corrupted registry
fails that one workflow and the rest continue.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The two id-mismatch error paths interpolated repr() into Rich markup, so
a stray bracket in a user typo could be parsed as markup. Route both
through rich.markup.escape.
`workflow add <source> --from <url>` also validated the source only
after downloading. Validate it up front so a URL/path/typo fails
without a network fetch.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
workflow list now skips non-dict registry entries with a warning instead
of crashing, matching update/enable/disable. The broad except in
_install_workflow_from_catalog no longer swallows typer.Exit, so precise
errors like the non-HTTPS redirect message are not duplicated.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Adds the missing workflow commands and flags so the workflow CLI
matches the extension/preset pattern: add --dev and --from, search
--author, update, enable and disable. Disabled workflows are blocked
from running and marked in list output.
Fixes#2342
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(agent-context): discover nested plan.md in scoped layouts (#3024)
The agent-context updater only looked for plan.md one level deep
(specs/*/plan.md), so scoped layouts created via SPECIFY_FEATURE_DIRECTORY
(specs/<scope>/<feature>/plan.md) were never picked up and no plan
reference was written into the context file.
Recurse into specs/ in both the bash (rglob) and PowerShell (-Recurse)
scripts. In the PowerShell script, also replace
[System.IO.Path]::GetRelativePath, which is .NET Core 2.1+ only and throws
under Windows PowerShell 5.1 (.NET Framework); the exception was swallowed
by the surrounding try/catch, leaving the plan path empty on 5.1 even when
a plan was found. Compute the project-relative path by stripping the root
prefix instead.
Add regression tests for both scripts covering nested discovery.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(agent-context): guard mtime plan discovery against symlink escape
Address Copilot review feedback on #3301:
- bash updater: the mtime fallback filtered candidates lexically via
relative_to() on the *unresolved* path, so a plan reached through a
specs/ symlink pointing outside the project could be selected and emit
an in-project-looking path. Resolve each candidate and keep only those
whose resolved path stays under root before picking the newest.
- test: the nested-plan PowerShell regression targets a Windows
PowerShell 5.1 (.NET Framework) failure mode, but ran whatever
POWERSHELL resolved to (prefers pwsh). Prefer powershell.exe on Windows
so the 5.1-only compat fix is actually exercised.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs(agent-context): note recursive plan.md discovery in update command
Auto-detection now recurses (`specs/**/plan.md`) to support nested scoped layouts created via SPECIFY_FEATURE_DIRECTORY (#3024). The update command doc still described the old one-level `specs/*/plan.md` glob, which could mislead users troubleshooting plan detection. Addresses Copilot review feedback on PR #3301.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
find_entries_for_url did (urlparse(url).hostname or "").lower() unguarded. a
malformed authority (e.g. an unterminated ipv6 bracket "https://[::1") makes
urlparse/hostname raise ValueError, so instead of the empty list the function
already returns for a host-less url, a raw ValueError leaked out of the shared
http client (build_request / open_url call this before any url validation).
no auth entry can match such a url, so treat it like the host-less case and
return no matches. added a regression test over an unterminated bracket and a
bracketed non-ip host; confirmed it fails on the pre-fix code.
CatalogStackBase._validate_catalog_url read parsed.hostname, which raises
ValueError on a malformed authority (e.g. an unclosed ipv6 bracket
"https://[::1"). every other reject path raises the class catalog error, so
the raw ValueError leaked to the caller. wrap the parse + hostname access and
convert ValueError to the normal error via cls._error.
twin of the bundler fix in adapters._validate_remote_url.
adapters._validate_remote_url reads parsed.hostname, which raises ValueError
on a malformed authority (e.g. an unclosed ipv6 bracket https://[::1). the
function's contract is to raise BundlerError for any bad url - every other
reject path does - so the raw ValueError leaked to the caller and crashed the
fetch instead of failing cleanly. wrap the parse and convert to BundlerError.
bundler sibling of #3369, which fixed the cli extension/preset/workflow add
paths but not this validator. added a regression test that fails pre-fix.
* fix(workflows): validate scalar types before string operations in workflow validation
YAML parses unquoted scalars like version: 1.0 and id: 123 as
float/int, which crashed validate_workflow and workflow add with raw
tracebacks. Type-check id, name, version and step ids before regex
and string operations so these surface as validation errors. Accept
an unquoted schema_version: 1.0 instead of printing a self-identical
rejection message.
Fixes#3420
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflows): treat falsey non-strings as type errors, not missing fields
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflows): only accept schema_version 1.0 so the error message is accurate
The check also accepted "1" while the error said Expected '1.0'.
Unquoted YAML 1.0 still works via str(); plain 1 is now rejected with
the message that matches.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
`-Number` defaults to 0, so the previous `-eq 0` / `-ne 0` checks could
not distinguish an unset flag from an explicit `-Number 0`: a user
requesting branch `000-...` was silently routed into auto-detection.
Switch both checks to `$PSBoundParameters.ContainsKey('Number')`, which
tests whether the flag was actually supplied — mirroring the bash twin's
empty-string sentinel (`[ -z "$BRANCH_NUMBER" ]` / `[ -n ... ]`).
Add a parity regression test to both TestCreateFeatureBash and
TestCreateFeaturePowerShell asserting `--number 0` / `-Number 0` yields
`000-zero`.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
test_template_renders_python_invocation monkeypatches shutil.which to
return /usr/bin/python3, but on Windows resolve_python_interpreter guards
the which() result with a real _interpreter_runs subprocess probe (#3304).
The mocked /usr/bin/python3 path does not exist on a Windows runner, so the
probe fails, the resolver falls back to sys.executable (a ...python.exe
path), and the python3-anchored regex assertion fails.
Patch _interpreter_runs to return True in the _pin_interpreter fixture so
the resolved interpreter token stays python3 across all platforms, keeping
the #3304 production guard intact while making the assertion deterministic.
Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
* feat(workflows): make shell step timeout configurable
The shell step hardcoded a 300s subprocess timeout, killing any
legitimate long-running QA command. Read an optional timeout field
(seconds, positive integer, default 300) and validate it.
Fixes#3327
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* style: multi-line timeout validation, assert status in default test
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Guard against unvalidated timeout in ShellStep.execute()
The engine does not auto-validate step config, so a string or null
timeout would reach subprocess.run() and crash the run with a
TypeError. Fall back to the 300s default for malformed values,
mirroring how the engine treats unvalidated continue_on_error.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: find plans in nested spec directories
The agent-context mtime fallback used a one-level specs/*/plan.md
glob, so scoped layouts (specs/<scope>/<feature>/plan.md via
SPECIFY_FEATURE_DIRECTORY) never matched and the SPECKIT block was
written without a plan path. Recurse in both script variants and
update the command doc wording.
Fixes#3024
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* refactor: pick newest plan with max(), align doc wording
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(templates): add py: lines to command templates' scripts frontmatter
Every templates/commands/*.md with a scripts: block now declares a py:
variant so --script py renders a Python invocation via the existing
interpreter-prefixing in process_template.
Fixes#3283
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: install scripts/python for the py script type
install_shared_infra mapped every non-sh script type to powershell, so
--script py rendered invocations pointing at files that were never
installed. Map py to the python variant dir and skip __pycache__
artifacts during the copy.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test: read templates with explicit utf-8 encoding
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: keep py: lines standalone, enforce script existence in tests
Drop the plan/tasks py: lines that referenced scripts shipping in the
core port (#3280); they move to that PR. Tests now assert every py:
line points at a script the repo ships, so a dangling reference can
never merge green.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(integrations): skip Windows Store python3 alias stub in resolve_python_interpreter
On stock Windows, python3 on PATH is the Microsoft Store App Execution
Alias stub: it exists but only prints an installer hint and exits
non-zero, so generated {SCRIPT} invocations for the py script type were
broken. Verify the found interpreter actually runs before accepting it,
on Windows only, mirroring the parse-success-not-availability approach
of #3312/#3320 for the sh scripts. POSIX keeps the plain existence
check. sys.executable remains the fallback and is always live.
Fixes#3383
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(integrations): probe interpreter isolated and without site, discard I/O
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* test: pin POSIX platform in PATH-resolution tests
The tests fake shutil.which with POSIX paths; on Windows CI the real
sys.platform made the stub probe run against those fake paths and
fall through to sys.executable.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
