_traverse_and_apply's insert_after loop iterated reversed(edits) over the
flat per-anchor edit list. The reversal is only meant to place a
higher-priority OVERLAY closer to the anchor (mirroring insert_before's
winner-closest behaviour), but reversing the flat list also flipped the
declared order of multiple insert_after edits authored within a SINGLE
overlay: [insert_after a->x, insert_after a->y] produced [a, y, x, b]
instead of [a, x, y, b]. insert_before (a forward loop) already preserves
order, so the two operations were asymmetric.
Group contiguous same-layer edits and reverse the GROUP order only, keeping
each overlay's own inserts in declared order. Cross-overlay priority is
unchanged (higher-priority overlay still lands closest to the anchor).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat(workflows): add standalone WorkflowResolver and overlay subsystem
Implement PR 1 of the workflow-overlays plan: a concrete, standalone
WorkflowResolver for downstream workflow extensibility without touching the
Preset subsystem.
- Add overlay manifest schema (Overlay, OverlayEdit, validate_overlay_yaml)
- Add pure-function merge engine (find_step, apply_edit, merge_steps,
validate_edits) with recursive anchor search and higher-wins semantics
- Add StepListComposer and tiered layer sources (project, installed, base)
- Add WorkflowResolver facade with inline HIGHER_WINS priority sorting
- Add CLI verbs: workflow overlay add/set-priority/enable/disable/remove/list
and workflow resolve <id>
- Wire WorkflowEngine.load_workflow through WorkflowResolver
- Extend workflow add to copy optional overlays/ subdirectory from local
workflow directories
- Add comprehensive unit, integration, and security tests
Refs: discussion #3473 (https://github.com/github/spec-kit/discussions/3473)
Assisted-by: Kimi (model: opencode-go/kimi-k2.7-code, autonomous)
* fix(workflows): reject symlinked overlay directories in layer sources
Address PR #3557 review comments r3594064534 and r3594064563:
- ProjectOverlaySource.collect now rejects symlinked per-workflow overlay
directories (.specify/workflows/overlays/<id>) before iterating
- InstalledOverlaySource.collect now rejects symlinked installed overlay
directories (.specify/workflows/<id>/overlays) before iterating
- workflow_overlay_list catches ValueError from resolver and exits with
code 1 instead of crashing on unhandled exceptions
- Added .specify/workflows/overlays to _reject_unsafe_workflow_storage
chokepoint for defense-in-depth
These guards prevent symlinked overlay directories from redirecting
auto-loaded overlay YAML to attacker-controlled content outside the
project, which could inject executable shell steps into trusted workflows.
Refs: PR #3557 review comments r3594064534, r3594064563
Assisted-by: opencode-go/qwen3.7-max (autonomous)
* fix(workflows): address Copilot review findings in merge engine
- Apply inserts before winning replace to prevent anchor-not-found errors
when replace changes step ID (r3594064604)
- Track attribution recursively for nested steps in composite inserts/replaces
so workflow resolve attributes all child steps correctly (r3594064638)
- Add regression tests for both fixes
Refs: PR #3557 review discussion
Assisted-by: GitHub Copilot (model: qwen3.7-plus, autonomous)
* refactor(workflows): simplify overlay architecture to 2-tier
Remove installed overlays tier to enforce clean separation of concerns:
- workflow add installs workflows only (no overlay copying)
- workflow overlay add installs overlays only (project-local)
Changes:
- Remove InstalledOverlaySource class and all references
- Remove overlay-copying logic from _validate_and_install_local()
- Update WorkflowResolver to 2-tier: project overlays + base workflow
- Fix --priority override timing: apply before validation, not after
- Remove tests for installed overlays (no longer applicable)
Rationale: If upstream controls both base workflow and shipped overlays,
and both get overwritten on bundle update, there's no reason to ship
overlays separately. Overlays only make sense when someone other than
the base author adds them.
Resolves all three review findings from PR #3557:
- r3594064677: workflow add no longer copies overlays from all call sites
- r3594064705: --priority override now applied before validation
- r3594064726: no stale installed overlays (tier removed entirely)
Assisted-by: Claude (model: claude-opus-4-7, autonomous)
* fix(workflows): harden overlay symlink handling
Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs(workflows): remove stale installed-overlay references from workflows.md
The 2-tier refactor (cc28185) removed the installed-overlay tier entirely,
but docs/reference/workflows.md was not updated. This commit addresses all
four Cluster 2 findings from the PR review:
- workflow add: remove sentence about copying overlays/ subdirectory
- How Overlays Work: drop installed-overlay table row and precedence prose;
rewrite to 2-tier model (project overlays only, source-order tie-break)
- overlay remove: drop trailing sentence about installed overlays
- Interaction with Bundles: rewrite to say workflow add installs only
workflow.yml; remove installed-overlay discovery language
Fixes: r3596368791, r3596368831, r3596368873, r3596368919
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(overlays): detect ancestor-conflict anchors in merge_steps
When two overlay edits target anchors that share a parent/descendant
relationship (e.g. remove an if-step + insert_after a nested child),
merge_steps processed them independently and in dict-insertion order,
making the outcome non-deterministic.
Add two private helpers to merge.py:
- _descendant_ids(step): returns all step IDs nested inside a step dict
by delegating to the existing _all_base_step_ids helper on children.
- _check_anchor_conflicts(anchors, base_steps): for each targeted anchor
finds its descendants and checks whether any other targeted anchor is
among them; returns human-readable error strings.
Wire _check_anchor_conflicts into merge_steps immediately after
edits_by_anchor is built, before any tree mutation occurs. Raises
ValueError listing the conflicting anchor pair(s) so overlay authors
know exactly what to fix.
Add TestMergeStepsAncestorConflicts (6 cases):
- remove parent + insert_after child raises ValueError
- replace parent + remove child raises ValueError
- conflict across multiple overlays raises ValueError
- sibling anchors (not ancestor/descendant) pass
- single anchor passes
- parent targeted but child not targeted passes
Closes review comment r3596368746 (PR #3557, round 2, cluster 3).
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(overlays): fix over-broad conflict detection and non-deterministic ID collision
Finding 1.1 — _check_anchor_conflicts was rejecting any ancestor/descendant
anchor pair, including insert-only edits that are perfectly safe. Only
replace/remove on an ancestor can destroy its subtree and make a descendant
anchor unresolvable. Change the signature to accept a dict[str, str]
(anchor → winning operation) and skip the check for insert_after/insert_before.
Finding 1.2 — merge_steps was calling find_step on the already-mutated tree,
so a replacement step that reused a base step ID could be accidentally targeted
by a later edit group (non-deterministic result depending on dict iteration
order). Replace the anchor-group loop with a single-pass _traverse_and_apply
that walks the original tree structure and applies edits as each step is
encountered. Anchors are never re-looked up in a mutated tree.
Design invariant enforced: overlays always apply to the original base tree and
cannot target steps introduced by other overlays. Non-remove edits on non-base
anchors now raise ValueError early.
Also removes apply_edit (no production callers, only tested in isolation) and
its test class — the new traversal inlines the same mechanics without the
find_step round-trip.
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(overlays): reject ID trailing newlines and reuse existing .yaml path
Fix two input validation bugs in the overlay layer (Group 2 of copilot
review PR #3557):
1. _validate_safe_id in schema.py used re.match() which anchors only at
the start of the string, so IDs like 'overlay\n' passed validation
and could produce newline-containing file paths. Changed to fullmatch()
so the entire string must satisfy the pattern.
2. workflow_overlay_add always wrote <id>.yml without checking whether
<id>.yaml already existed. Since the resolver loads both extensions,
this created two active layers whose edits applied twice. Now uses
the existing _find_overlay_file() to detect a pre-existing file and
reuse its path, falling back to .yml only for new overlays.
Tests added for both fixes.
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(overlays): fix display order inversion and wrap file-read errors
Finding group 3 from copilot-review-v2.md:
3.1 — Precedence display inverted (overlays/__init__.py)
collect_all_layers used a single-pass sort by (-priority, source_asc),
which placed the *losing* equal-priority source first in the display
while claiming "highest first". Fix: two-pass stable sort — source
descending then priority descending — so the actual winner (last applied
by the composer) rises to the top of the display.
3.2 — Unwrapped file-read errors (overlays/layer_sources.py)
Only yaml.YAMLError was caught around path.read_text(), so an
unreadable or non-UTF-8 overlay produced a raw traceback. Fix: widen
the except clause to (yaml.YAMLError, OSError, UnicodeDecodeError),
matching the pattern used throughout catalog.py.
Tests:
- test_workflow_resolve_equal_priority_winner_shown_first: verifies
project:zzz (the winner) appears before project:aaa in workflow resolve
output when both overlays share the same priority.
- tests/workflows/test_overlay_layer_sources.py (new): OSError and
non-UTF-8 bytes both produce OverlayLoadError, not raw tracebacks.
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: rename misleading overlay test
Assisted-by: GitHub Copilot (model: gpt-5.3-codex, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: remove EOF blank line in overlay resolver
Assisted-by: GitHub Copilot (model: gpt-5.3-codex, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix: handle overlay read and enumeration errors
Assisted-by: GitHub Copilot (model: gpt-5.3-codex, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(overlays): validate resolver workflow IDs
Reject unsafe and reserved workflow IDs before overlay or base sources construct paths, preventing traversal through resolver and engine fallback paths.
Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(overlays): drop _remove_sources_recursively from remove branch
In _traverse_and_apply, the remove branch called _remove_sources_recursively
to clean up attribution entries for the deleted step. This was inherited from
the old apply_edit loop (c70a5d6) where it was needed because the sources dict
was queried exhaustively.
In the current single-pass design, _build_attribution only traverses the result
list, so stale sources entries for removed steps are never read. The cleanup
call is therefore unnecessary — and actively harmful when another overlay has
replaced a different step with a new step that reuses the same ID: the pop
clobbers the replacement's attribution entry, causing workflow resolve to report
the surviving step as 'unknown'.
Fix: simply remove the _remove_sources_recursively call from the remove branch.
Add an attribution assertion to the existing reused-ID regression test to catch
this case.
Fixes: r3604242050 (Copilot review finding)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
* Fix CLI overlay ID validation anchoring
Use fullmatch for CLI workflow/overlay ID validation so trailing newlines are rejected consistently with manifest validation.
Add regression coverage for newline-suffixed workflow and overlay IDs in overlay set-priority.
Assisted-by: GitHub Copilot (model: gpt-5.3-codex, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflows): validate workflow_id in layer sources before path construction
ProjectOverlaySource.collect() and BaseWorkflowSource.collect() joined
workflow_id directly onto storage paths without validation, enabling
path traversal (e.g. '../../outside') when called outside the
WorkflowResolver.
Add _validate_workflow_id() to layer_sources.py — mirrors the same
_SAFE_ID_PATTERN / _RESERVED_WORKFLOW_IDS check used by WorkflowResolver
in overlays/__init__.py — and call it at the top of both collect()
methods before any path is constructed.
Adds parametrised tests covering unsafe IDs and verifying no filesystem
access occurs for an invalid ID.
Closes review finding r3604772700.
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(workflows): align layer source validation with _safe_workflow_id_dir
Plan §4.1 requires that Workflow-ID-Validierung, Symlink-/Containment-
Prüfungen and Fehlerübersetzung must not diverge between workflow
management and the overlay resolver.
My previous fix added ID pattern + reserved-name validation to both
collect() methods but was missing the containment step and the
BaseWorkflowSource directory/file checks that _safe_workflow_id_dir
performs.
Changes:
- Add _ensure_contained_dir(path, root) to layer_sources.py — pure
domain mirror of overlays/_commands.py::_ensure_contained_dir that
raises OverlayLoadError instead of typer.Exit
- ProjectOverlaySource.collect(): replace two inline symlink/dir checks
with _ensure_contained_dir(workflow_overlay_dir, self.overlays_dir),
adding the missing resolve().relative_to() containment step
- BaseWorkflowSource.collect(): add _ensure_contained_dir on the
workflow directory, and add workflow.yml symlink check before is_file()
The same logic now lives in three places (workflow CLI, overlay CLI,
layer sources). The DRY extraction to workflows/_validation.py is
deferred to PR 3 per plan §4.1.
Tests: add containment and symlink tests for both sources.
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(overlays): resolve identity from manifest field, not filename
Align overlay identity resolution with the project-wide convention:
presets use preset.id, extensions use extension.id, workflows use
workflow.id, and workflow steps use step.type_key. Overlays must
derive identity from the manifest id field, not the filename.
Rewrite _find_overlay_file() to scan all YAML files in the overlay
directory and match on the manifest id field, fixing the bug where
enable/disable/remove/set-priority failed when filename != manifest id.
Closes: PR #3557 discussion r3605010197
Assisted-by: opencode-go/qwen3.7-max (autonomous)
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* fix(workflows): make validation behavior consistent across YAML loading paths
Address PR #3557 review finding r3607632921:
- Wrap yaml.YAMLError → ValueError in from_yaml() and from_string() so
malformed YAML matches the documented exception contract
- Add except ValueError to workflow_info to handle composition errors
cleanly instead of crashing with a raw traceback
- Remove validate_workflow() from compose() so the resolver path is
parse-only like all other YAML loading mechanisms; callers validate
explicitly via engine.validate()
- Update test to reflect new behavior: resolve() returns composed
definition, caller validates separately
Assisted-by: opencode-go/qwen3.7-max (autonomous)
* fix(overlays): list disabled overlays in management view
Keep disabled overlays visible in workflow overlay list while leaving resolution behavior unchanged.
- add an include_disabled opt-in to overlay source/resolver collection
- use include_disabled=True for workflow overlay list
- add regression tests for list visibility and default filtering
Assisted-by: GitHub Copilot (model: GPT-5.4, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Potential fix for pull request finding
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* docs: align overlay extends and resolver contract
Assisted-by: GitHub Copilot (model: GPT-5.3-Codex, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: use atomic write for overlay file updates to prevent hard-link attack
Replace in-place write_text() calls in workflow_overlay_add() and
_update_overlay_field() with the same mkstemp → write → os.replace()
pattern used by the workflow installer (_stage_workflow_file /
_commit_workflow_file / _discard_staged_workflow_file).
The prior code rejected symlinks and validated path containment, but a
hard-linked destination file passes both checks while sharing an inode
with an external file. write_text() would then truncate and overwrite
that external inode. The atomic staging approach never opens the
existing destination for writing, eliminating the hard-link vector.
Fixes findings r3608669512 and r3608669517 on PR #3557.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, supervised)
* fix(composer): preserve invalid base definition instead of coercing steps to []
When 'steps' is not a list, returning early with the unmodified
WorkflowDefinition lets validate_workflow surface the proper error
("'steps' must be a list.") to the caller. The previous silent
coercion to [] masked the validation error entirely.
Fixes: https://github.com/github/spec-kit/pull/3557#discussion_r3608669506
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, supervised)
* fix: align workflow overlay priority semantics
Assisted-by: GitHub Copilot (model: GPT-5.6 Terra, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: validate overlay priority presentation
Assisted-by: GitHub Copilot (model: GPT-5.6 Terra, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: catch OverflowError in normalize_priority for float infinity values
YAML values like `priority: .inf` parse to float('inf'), causing
int() to raise OverflowError. This broke validate_overlay_yaml()'s
'validation never raises' contract. Adding OverflowError to the
except clause makes it fall back to the default priority (10),
consistent with other invalid value handling.
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Markus <markus@example.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Ali jawwad