feat(presets): Composition strategies (prepend, append, wrap) for templates, commands, and scripts (#2133)

* fix: rebase onto upstream/main, resolve conflicts with PR #2189

upstream/main merged PR #2189 (wrap-only strategy) which overlaps with
our comprehensive composition strategies (prepend/append/wrap). Resolved
conflicts keeping our implementation as source of truth:

- README: keep our future considerations (composition is now fully
  implemented, not a future item)
- presets.py: keep our composition architecture (_reconcile_composed_commands,
  collect_all_layers, resolve_content) while preserving #2189's
  _substitute_core_template which is used by agents.py for skill
  generation
- tests: keep both test sets (our composition tests + #2189's wrap
  tests), removed TestReplayWrapsForCommand and
  TestInstallRemoveWrapLifecycle which test the superseded
  _replay_wraps_for_command API; our composition tests cover equivalent
  scenarios
- Restored missing _unregister_commands call in remove() that was lost
  during #2189 merge

* fix: re-create skill directory in _reconcile_skills after removal

After _unregister_skills removes a skill directory, _register_skills
skips writing because the dir no longer passes the is_dir() check.
Fix by ensuring the skill subdirectory exists before calling
_register_skills so the next winning preset's content gets registered.

Fixes the Claude E2E failure where removing a top-priority override
preset left skill-based agents without any SKILL.md file.

* fix: address twenty-third round of Copilot PR review feedback

- Protect reconciliation in remove(): wrap _reconcile_composed_commands
  and _reconcile_skills in try/except so failures emit a warning instead
  of leaving the project in an inconsistent state
- Protect reconciliation in install(): same pattern for post-install
  reconciliation so partial installs don't lack cleanup
- Inherit scripts/agent_scripts from base frontmatter: when composing
  commands, merge scripts and agent_scripts keys from the base command's
  frontmatter into the top layer's frontmatter if missing, preventing
  composed commands from losing required script references
- Add tier-5 bundled core fallback to collect_all_layers(): check the
  bundled core_pack (wheel) or repo-root templates (source checkout) when
  .specify/templates/ doesn't contain the core file, matching resolve()'s
  tier-5 fallback so composition can always find a base layer

* fix: address twenty-fourth round of Copilot PR review feedback

- Use yaml.safe_load for frontmatter parsing in resolve_content instead
  of CommandRegistrar.parse_frontmatter which uses naive find('---',3);
  strip strategy key from final frontmatter to prevent leaking internal
  composition directives into rendered agent command files
- Filter _reconcile_skills to specific commands: use _FilteredManifest
  wrapper so only the commands being reconciled get their skills updated,
  preventing accidental overwrites of other commands' skills that may be
  owned by higher-priority presets

* fix: address twenty-fifth round of Copilot PR review feedback

- Support legacy command-frontmatter strategy: when preset.yml doesn't
  declare a strategy, check the command file's YAML frontmatter for
  strategy: wrap as a fallback so legacy wrap presets participate in
  composition and multi-preset chaining
- Guard skill dir creation in _reconcile_skills: only re-create the
  skill directory if the skill was previously managed (listed in some
  preset's registered_skills), avoiding creation of new skill dirs
  that _register_skills would normally skip

* fix: add explanatory comment to empty except in legacy frontmatter parsing

* fix: address twenty-sixth round of Copilot PR review feedback

- Unregister stale commands when composition fails: when resolve_content
  returns None during reconciliation (base layer removed), unregister
  the command from non-skill agents and emit a warning
- Load extension aliases during reconciliation: _register_command_from_path
  now checks extension.yml for aliases when the winning layer is an
  extension, so alias files are restored after preset removal
- Use line-based fence detection for legacy frontmatter strategy fallback:
  scan for --- on its own line instead of split('---',2) to avoid
  mis-parsing YAML values containing ---

* fix: address twenty-seventh round of Copilot PR review feedback

- Handle non-preset winners in _reconcile_skills: when the winning
  layer is core/extension/project-override, restore skills via
  _unregister_skills so skill-based agents stay consistent with the
  priority stack
- Update base_frontmatter_text on replace layers: when a higher-priority
  replace layer occurs during composition, update both top and base
  frontmatter so scripts/agent_scripts inheritance reflects the
  effective base beneath the top composed layer

* fix: address twenty-eighth round of Copilot PR review feedback

- Parse only interior lines in _parse_fm_yaml: use lines[1:-1] instead
  of filtering all --- lines, preventing corruption when YAML values
  contain a line that is exactly ---
- Omit empty frontmatter: skip re-rendering when top_fm is empty dict
  to avoid emitting ---/{}/--- for intentionally empty frontmatter
- Update scaffold wrap comment: mention both {CORE_TEMPLATE} and
  $CORE_SCRIPT placeholders for templates/commands vs scripts
- Clarify shell composition scope in ARCHITECTURE.md: note that bash/PS1
  resolve_template_content only handles templates; command/script
  composition is handled by the Python resolver

* fix: address twenty-ninth round of Copilot PR review feedback

- Fix TestCollectAllLayers docstring: reference collect_all_layers()
- Add default/unknown strategy handling in bash/PS1 composition: error
  on unrecognized strategy values instead of silently skipping
- Fix comment: .composed/ is a persistent dir, not temporary
- Fix comment: legacy fallback checks all valid strategies, not just wrap
- Cache PresetRegistry in _reconcile_skills: build presets_by_priority
  once instead of constructing registry per-command

* fix: address thirtieth round of Copilot PR review feedback

- Guard legacy frontmatter fallback: only check command file frontmatter
  for strategy when the manifest entry doesn't explicitly include the
  strategy key, preventing override of manifest-declared strategies
- Document rollback limitation: note that mid-registration failures may
  leave orphaned agent command files since partial progress isn't
  captured by the local vars

* fix: handle project override skills and extension context in reconciliation

* fix: add comment to empty except in extension registration fallback

* fix: filter extension commands in reconciliation and fix type annotation

* fix: filter extension commands from post-install reconciliation

Apply the same extension-installed check used in _register_commands to
the reconciliation command list, preventing reconciliation from
registering commands for extensions that are not installed.

* fix: skip convention fallback for explicit file paths and add stem fallback to tier-5

When a preset manifest provides an explicit file path that does not
exist, skip the convention-based fallback to avoid masking typos.
Also add speckit.<stem> to <stem>.md fallback in tier-5 bundled/source
core lookup for consistency with tier-4.

* fix: scan past non-replace layers to find base in resolve_content

The base-finding scan now skips non-replace layers below a replace
layer instead of stopping at the first non-replace. This fixes the
case where a low-priority append/prepend layer sits below a replace
that should serve as the base for composition.

* fix: add context_note to non-skill agent registration for extensions

Add context_note parameter to register_commands_for_non_skill_agents
and pass extension name/id during reconciliation so rendered command
files preserve the extension-specific context markers.

* fix: Optional type, rollback safety, and override skill restoration

- Fix context_note type to Optional[str]
- Wrap shutil.rmtree in try/except during install rollback
- Separate override-backed skills from core/extension in _reconcile_skills

* fix: align bash/PS1 base-finding with Python resolver

Rewrite bash and PowerShell composition loops to find the effective
base replace layer first (scanning bottom-up, skipping non-replace
layers below it), then compose only from the base upward. This
prevents evaluation of irrelevant lower layers (e.g. a wrap with
no placeholder below a replace) and matches resolve_content behavior.

* fix: PS1 no-python warning, integration hook for override skills, alias cleanup

- Warn when no Python 3 found in PS1 and presets use composition strategies
- Apply post_process_skill_content integration hook when restoring
  override-backed skills so agent-specific flags are preserved
- Unregister command aliases alongside primary name when composition
  fails to prevent orphaned alias files

* fix: include aliases in removed_cmd_names during preset removal

Read aliases from preset manifest before deleting pack_dir so alias
command files are included in unregistration and reconciliation.

* fix: add comment to empty except in alias extraction during removal

* fix: scan top-down for effective base in all resolvers

Change base-finding to scan from highest priority downward to find the
nearest replace layer, then compose only layers above it. Prevents
evaluation of irrelevant lower layers (e.g. a wrap without placeholder
below a higher-priority replace) across Python, bash, and PowerShell.

* fix: align CLI composition chain display with top-down base-finding

Show only contributing layers (base and above) in preset resolve
output, matching resolve_content top-down semantics. Layers below
the effective base are omitted since they do not contribute.

* fix: guard corrupted registry entries and make manifest authoritative

- Add isinstance(meta, dict) guard in bash registry parsing so corrupted
  entries are skipped instead of breaking priority ordering
- Only use convention-based file lookup when the manifest does not list
  the requested template, making preset.yml authoritative and preventing
  stray on-disk files from creating unintended layers

* fix: align resolve() with manifest file paths and match extension context_note

- Update resolve() preset tier to consult manifest file paths before
  convention-based lookup, matching collect_all_layers behavior
- Use exact extension context_note format matching extensions.CommandRegistrar
- Update test to declare template in manifest (authoritative manifest)

* revert: restore resolve() convention-based behavior for backwards compatibility

resolve() is the existing public API used by shell scripts and other
callers. Changing it to manifest-authoritative breaks backward compat
for presets that rely on convention-based file lookup. Only the new
collect_all_layers/resolve_content path uses manifest-authoritative
logic.

* fix: only pre-compose when this preset is the top composing layer

Skip composition in _register_commands when a higher-priority replace
layer already wins for the command. Register the raw file instead and
let reconciliation write the correct final content.

* fix: deduplicate PyYAML warnings and use self.registry in reconciliation

- Emit PyYAML-missing warning once per function call in bash/PS1 instead
  of per-preset to avoid spamming stderr
- Use self.registry.list_by_priority() in reconciliation methods instead
  of constructing new PresetRegistry instances to avoid redundant I/O
  and potential consistency issues

* fix: document strategy handling consistency between layers and registrar

Composed output already strips strategy from frontmatter (resolve_content
pops it). Raw file registration preserves legacy frontmatter strategy
for backward compat; reconciliation corrects the final state.

* fix: correct stale comments for alias tracking and base-finding algorithm

* security: validate manifest file paths in bash/PowerShell resolvers

Reject absolute paths and parent directory traversal (..) in the
manifest-declared file field before joining with the preset directory.
Matches the Python-side validation in PresetManifest._validate().

---------

Co-authored-by: Manfred Riem <15701806+mnriem@users.noreply.github.com>

presets/ARCHITECTURE.md

@@ -41,6 +41,24 @@ The resolution is implemented three times to ensure consistency:
 - **Bash**: `resolve_template()` in `scripts/bash/common.sh`
 - **PowerShell**: `Resolve-Template` in `scripts/powershell/common.ps1`
 
+### Composition Strategies
+
+Templates, commands, and scripts support a `strategy` field that controls how a preset's content is combined with lower-priority content instead of fully replacing it:
+
+| Strategy | Description | Templates | Commands | Scripts |
+|----------|-------------|-----------|----------|---------|
+| `replace` (default) | Fully replaces lower-priority content | ✓ | ✓ | ✓ |
+| `prepend` | Places content before lower-priority content (separated by a blank line) | ✓ | ✓ | — |
+| `append` | Places content after lower-priority content (separated by a blank line) | ✓ | ✓ | — |
+| `wrap` | Content contains `{CORE_TEMPLATE}` (templates/commands) or `$CORE_SCRIPT` (scripts) placeholder replaced with lower-priority content | ✓ | ✓ | ✓ |
+
+Composition is recursive — multiple composing presets chain. The `PresetResolver.resolve_content()` method walks the full priority stack bottom-up and applies each layer's strategy.
+
+Content resolution functions for composition:
+- **Python**: `PresetResolver.resolve_content()` in `src/specify_cli/presets.py` (templates, commands, and scripts)
+- **Bash**: `resolve_template_content()` in `scripts/bash/common.sh` (templates only; command/script composition is handled by the Python resolver)
+- **PowerShell**: `Resolve-TemplateContent` in `scripts/powershell/common.ps1` (templates only; command/script composition is handled by the Python resolver)
+
 ## Command Registration
 
 When a preset is installed with `type: "command"` entries, the `PresetManager` registers them into all detected agent directories using the shared `CommandRegistrar` from `src/specify_cli/agents.py`.

presets/README.md

@@ -61,7 +61,37 @@ specify preset add healthcare-compliance --priority 5  # overrides enterprise-sa
 specify preset add pm-workflow --priority 1            # overrides everything
 ```
 
-Presets **override**, they don't merge. If two presets both provide `spec-template`, the one with the lowest priority number wins entirely.
+Presets **override by default**, they don't merge. If two presets both provide `spec-template` with the default `replace` strategy, the one with the lowest priority number wins entirely. However, presets can use **composition strategies** to augment rather than replace content.
+
+### Composition Strategies
+
+Presets can declare a `strategy` per template to control how content is combined. The `name` field identifies which template to compose with in the priority stack, while `file` points to the actual content file (which can differ from the convention path `templates/<name>.md`):
+
+```yaml
+provides:
+  templates:
+    - type: "template"
+      name: "spec-template"
+      file: "templates/spec-addendum.md"
+      strategy: "append"        # adds content after the core template
+```
+
+| Strategy | Description |
+|----------|-------------|
+| `replace` (default) | Fully replaces the lower-priority template |
+| `prepend` | Places content **before** the resolved lower-priority template, separated by a blank line |
+| `append` | Places content **after** the resolved lower-priority template, separated by a blank line |
+| `wrap` | Content contains `{CORE_TEMPLATE}` placeholder (or `$CORE_SCRIPT` for scripts) replaced with the lower-priority content |
+
+**Supported combinations:**
+
+| Type | `replace` | `prepend` | `append` | `wrap` |
+|------|-----------|-----------|----------|--------|
+| **template** | ✓ (default) | ✓ | ✓ | ✓ |
+| **command** | ✓ (default) | ✓ | ✓ | ✓ |
+| **script** | ✓ (default) | — | — | ✓ |
+
+Multiple composing presets chain recursively. For example, a security preset with `prepend` and a compliance preset with `append` will produce: security header + core content + compliance footer.
 
 ## Catalog Management
 
@@ -108,13 +138,5 @@ See [scaffold/](scaffold/) for a scaffold you can copy to create your own preset
 
 The following enhancements are under consideration for future releases:
 
-- **Composition strategies** — Allow presets to declare a `strategy` per template instead of the default `replace`:
-
-  | Type | `replace` | `prepend` | `append` | `wrap` |
-  |------|-----------|-----------|----------|--------|
-  | **template** | ✓ (default) | ✓ | ✓ | ✓ |
-  | **command** | ✓ (default) | ✓ | ✓ | ✓ |
-  | **script** | ✓ (default) | — | — | ✓ |
-
-  For artifacts and commands (which are LLM directives), `wrap` injects preset content before and after the core template using a `{CORE_TEMPLATE}` placeholder (implemented). For scripts, `wrap` would run custom logic before/after the core script via a `$CORE_SCRIPT` variable (not yet implemented).
-- **Script overrides** — Enable presets to provide alternative versions of core scripts (e.g. `create-new-feature.sh`) for workflow customization. A `strategy: "wrap"` option could allow presets to run custom logic before/after the core script without fully replacing it.
+- **Structural merge strategies** — Parsing Markdown sections for per-section granularity (e.g., "replace only ## Security").
+- **Conflict detection** — `specify preset lint` / `specify preset doctor` for detecting composition conflicts.

presets/scaffold/preset.yml

@@ -32,6 +32,15 @@ provides:
   templates:
     # CUSTOMIZE: Define your template overrides
     # Templates are document scaffolds (spec-template.md, plan-template.md, etc.)
+    #
+    # Strategy options (optional, defaults to "replace"):
+    #   replace  - Fully replaces the lower-priority template (default)
+    #   prepend  - Places this content BEFORE the lower-priority template
+    #   append   - Places this content AFTER the lower-priority template
+    #   wrap     - Uses {CORE_TEMPLATE} placeholder (templates/commands) or
+    #              $CORE_SCRIPT placeholder (scripts), replaced with lower-priority content
+    #
+    # Note: Scripts only support "replace" and "wrap" strategies.
     - type: "template"
       name: "spec-template"
       file: "templates/spec-template.md"
@@ -45,6 +54,26 @@ provides:
     #   description: "Custom plan template"
     #   replaces: "plan-template"
 
+    # COMPOSITION EXAMPLES:
+    # The `file` field points to the content file (can differ from the
+    # convention path `templates/<name>.md`). The `name` field identifies
+    # which template to compose with in the priority stack.
+    #
+    # Append additional sections to an existing template:
+    # - type: "template"
+    #   name: "spec-template"
+    #   file: "templates/spec-addendum.md"
+    #   description: "Add compliance section to spec template"
+    #   strategy: "append"
+    #
+    # Wrap a command with preamble/sign-off:
+    # - type: "command"
+    #   name: "speckit.specify"
+    #   file: "commands/specify-wrapper.md"
+    #   description: "Wrap specify command with compliance checks"
+    #   strategy: "wrap"
+    #   # In the wrapper file, use {CORE_TEMPLATE} where the original content goes
+
     # OVERRIDE EXTENSION TEMPLATES:
     # Presets sit above extensions in the resolution stack, so you can
     # override templates provided by any installed extension.