mirror of
https://github.com/github/spec-kit.git
synced 2026-07-03 12:28:06 +08:00
feat(presets): Composition strategies (prepend, append, wrap) for templates, commands, and scripts (#2133)
* fix: rebase onto upstream/main, resolve conflicts with PR #2189 upstream/main merged PR #2189 (wrap-only strategy) which overlaps with our comprehensive composition strategies (prepend/append/wrap). Resolved conflicts keeping our implementation as source of truth: - README: keep our future considerations (composition is now fully implemented, not a future item) - presets.py: keep our composition architecture (_reconcile_composed_commands, collect_all_layers, resolve_content) while preserving #2189's _substitute_core_template which is used by agents.py for skill generation - tests: keep both test sets (our composition tests + #2189's wrap tests), removed TestReplayWrapsForCommand and TestInstallRemoveWrapLifecycle which test the superseded _replay_wraps_for_command API; our composition tests cover equivalent scenarios - Restored missing _unregister_commands call in remove() that was lost during #2189 merge * fix: re-create skill directory in _reconcile_skills after removal After _unregister_skills removes a skill directory, _register_skills skips writing because the dir no longer passes the is_dir() check. Fix by ensuring the skill subdirectory exists before calling _register_skills so the next winning preset's content gets registered. Fixes the Claude E2E failure where removing a top-priority override preset left skill-based agents without any SKILL.md file. * fix: address twenty-third round of Copilot PR review feedback - Protect reconciliation in remove(): wrap _reconcile_composed_commands and _reconcile_skills in try/except so failures emit a warning instead of leaving the project in an inconsistent state - Protect reconciliation in install(): same pattern for post-install reconciliation so partial installs don't lack cleanup - Inherit scripts/agent_scripts from base frontmatter: when composing commands, merge scripts and agent_scripts keys from the base command's frontmatter into the top layer's frontmatter if missing, preventing composed commands from losing required script references - Add tier-5 bundled core fallback to collect_all_layers(): check the bundled core_pack (wheel) or repo-root templates (source checkout) when .specify/templates/ doesn't contain the core file, matching resolve()'s tier-5 fallback so composition can always find a base layer * fix: address twenty-fourth round of Copilot PR review feedback - Use yaml.safe_load for frontmatter parsing in resolve_content instead of CommandRegistrar.parse_frontmatter which uses naive find('---',3); strip strategy key from final frontmatter to prevent leaking internal composition directives into rendered agent command files - Filter _reconcile_skills to specific commands: use _FilteredManifest wrapper so only the commands being reconciled get their skills updated, preventing accidental overwrites of other commands' skills that may be owned by higher-priority presets * fix: address twenty-fifth round of Copilot PR review feedback - Support legacy command-frontmatter strategy: when preset.yml doesn't declare a strategy, check the command file's YAML frontmatter for strategy: wrap as a fallback so legacy wrap presets participate in composition and multi-preset chaining - Guard skill dir creation in _reconcile_skills: only re-create the skill directory if the skill was previously managed (listed in some preset's registered_skills), avoiding creation of new skill dirs that _register_skills would normally skip * fix: add explanatory comment to empty except in legacy frontmatter parsing * fix: address twenty-sixth round of Copilot PR review feedback - Unregister stale commands when composition fails: when resolve_content returns None during reconciliation (base layer removed), unregister the command from non-skill agents and emit a warning - Load extension aliases during reconciliation: _register_command_from_path now checks extension.yml for aliases when the winning layer is an extension, so alias files are restored after preset removal - Use line-based fence detection for legacy frontmatter strategy fallback: scan for --- on its own line instead of split('---',2) to avoid mis-parsing YAML values containing --- * fix: address twenty-seventh round of Copilot PR review feedback - Handle non-preset winners in _reconcile_skills: when the winning layer is core/extension/project-override, restore skills via _unregister_skills so skill-based agents stay consistent with the priority stack - Update base_frontmatter_text on replace layers: when a higher-priority replace layer occurs during composition, update both top and base frontmatter so scripts/agent_scripts inheritance reflects the effective base beneath the top composed layer * fix: address twenty-eighth round of Copilot PR review feedback - Parse only interior lines in _parse_fm_yaml: use lines[1:-1] instead of filtering all --- lines, preventing corruption when YAML values contain a line that is exactly --- - Omit empty frontmatter: skip re-rendering when top_fm is empty dict to avoid emitting ---/{}/--- for intentionally empty frontmatter - Update scaffold wrap comment: mention both {CORE_TEMPLATE} and $CORE_SCRIPT placeholders for templates/commands vs scripts - Clarify shell composition scope in ARCHITECTURE.md: note that bash/PS1 resolve_template_content only handles templates; command/script composition is handled by the Python resolver * fix: address twenty-ninth round of Copilot PR review feedback - Fix TestCollectAllLayers docstring: reference collect_all_layers() - Add default/unknown strategy handling in bash/PS1 composition: error on unrecognized strategy values instead of silently skipping - Fix comment: .composed/ is a persistent dir, not temporary - Fix comment: legacy fallback checks all valid strategies, not just wrap - Cache PresetRegistry in _reconcile_skills: build presets_by_priority once instead of constructing registry per-command * fix: address thirtieth round of Copilot PR review feedback - Guard legacy frontmatter fallback: only check command file frontmatter for strategy when the manifest entry doesn't explicitly include the strategy key, preventing override of manifest-declared strategies - Document rollback limitation: note that mid-registration failures may leave orphaned agent command files since partial progress isn't captured by the local vars * fix: handle project override skills and extension context in reconciliation * fix: add comment to empty except in extension registration fallback * fix: filter extension commands in reconciliation and fix type annotation * fix: filter extension commands from post-install reconciliation Apply the same extension-installed check used in _register_commands to the reconciliation command list, preventing reconciliation from registering commands for extensions that are not installed. * fix: skip convention fallback for explicit file paths and add stem fallback to tier-5 When a preset manifest provides an explicit file path that does not exist, skip the convention-based fallback to avoid masking typos. Also add speckit.<stem> to <stem>.md fallback in tier-5 bundled/source core lookup for consistency with tier-4. * fix: scan past non-replace layers to find base in resolve_content The base-finding scan now skips non-replace layers below a replace layer instead of stopping at the first non-replace. This fixes the case where a low-priority append/prepend layer sits below a replace that should serve as the base for composition. * fix: add context_note to non-skill agent registration for extensions Add context_note parameter to register_commands_for_non_skill_agents and pass extension name/id during reconciliation so rendered command files preserve the extension-specific context markers. * fix: Optional type, rollback safety, and override skill restoration - Fix context_note type to Optional[str] - Wrap shutil.rmtree in try/except during install rollback - Separate override-backed skills from core/extension in _reconcile_skills * fix: align bash/PS1 base-finding with Python resolver Rewrite bash and PowerShell composition loops to find the effective base replace layer first (scanning bottom-up, skipping non-replace layers below it), then compose only from the base upward. This prevents evaluation of irrelevant lower layers (e.g. a wrap with no placeholder below a replace) and matches resolve_content behavior. * fix: PS1 no-python warning, integration hook for override skills, alias cleanup - Warn when no Python 3 found in PS1 and presets use composition strategies - Apply post_process_skill_content integration hook when restoring override-backed skills so agent-specific flags are preserved - Unregister command aliases alongside primary name when composition fails to prevent orphaned alias files * fix: include aliases in removed_cmd_names during preset removal Read aliases from preset manifest before deleting pack_dir so alias command files are included in unregistration and reconciliation. * fix: add comment to empty except in alias extraction during removal * fix: scan top-down for effective base in all resolvers Change base-finding to scan from highest priority downward to find the nearest replace layer, then compose only layers above it. Prevents evaluation of irrelevant lower layers (e.g. a wrap without placeholder below a higher-priority replace) across Python, bash, and PowerShell. * fix: align CLI composition chain display with top-down base-finding Show only contributing layers (base and above) in preset resolve output, matching resolve_content top-down semantics. Layers below the effective base are omitted since they do not contribute. * fix: guard corrupted registry entries and make manifest authoritative - Add isinstance(meta, dict) guard in bash registry parsing so corrupted entries are skipped instead of breaking priority ordering - Only use convention-based file lookup when the manifest does not list the requested template, making preset.yml authoritative and preventing stray on-disk files from creating unintended layers * fix: align resolve() with manifest file paths and match extension context_note - Update resolve() preset tier to consult manifest file paths before convention-based lookup, matching collect_all_layers behavior - Use exact extension context_note format matching extensions.CommandRegistrar - Update test to declare template in manifest (authoritative manifest) * revert: restore resolve() convention-based behavior for backwards compatibility resolve() is the existing public API used by shell scripts and other callers. Changing it to manifest-authoritative breaks backward compat for presets that rely on convention-based file lookup. Only the new collect_all_layers/resolve_content path uses manifest-authoritative logic. * fix: only pre-compose when this preset is the top composing layer Skip composition in _register_commands when a higher-priority replace layer already wins for the command. Register the raw file instead and let reconciliation write the correct final content. * fix: deduplicate PyYAML warnings and use self.registry in reconciliation - Emit PyYAML-missing warning once per function call in bash/PS1 instead of per-preset to avoid spamming stderr - Use self.registry.list_by_priority() in reconciliation methods instead of constructing new PresetRegistry instances to avoid redundant I/O and potential consistency issues * fix: document strategy handling consistency between layers and registrar Composed output already strips strategy from frontmatter (resolve_content pops it). Raw file registration preserves legacy frontmatter strategy for backward compat; reconciliation corrects the final state. * fix: correct stale comments for alias tracking and base-finding algorithm * security: validate manifest file paths in bash/PowerShell resolvers Reject absolute paths and parent directory traversal (..) in the manifest-declared file field before joining with the preset directory. Matches the Python-side validation in PresetManifest._validate(). --------- Co-authored-by: Manfred Riem <15701806+mnriem@users.noreply.github.com>
This commit is contained in:
Copilot
@@ -320,8 +320,9 @@ try:
|
||||
with open(os.environ['SPECKIT_REGISTRY']) as f:
|
||||
data = json.load(f)
|
||||
presets = data.get('presets', {})
|
||||
for pid, meta in sorted(presets.items(), key=lambda x: x[1].get('priority', 10)):
|
||||
print(pid)
|
||||
for pid, meta in sorted(presets.items(), key=lambda x: x[1].get('priority', 10) if isinstance(x[1], dict) else 10):
|
||||
if isinstance(meta, dict) and meta.get('enabled', True) is not False:
|
||||
print(pid)
|
||||
except Exception:
|
||||
sys.exit(1)
|
||||
" 2>/dev/null); then
|
||||
@@ -373,3 +374,225 @@ except Exception:
|
||||
return 1
|
||||
}
|
||||
|
||||
# Resolve a template name to composed content using composition strategies.
|
||||
# Reads strategy metadata from preset manifests and composes content
|
||||
# from multiple layers using prepend, append, or wrap strategies.
|
||||
#
|
||||
# Usage: CONTENT=$(resolve_template_content "template-name" "$REPO_ROOT")
|
||||
# Returns composed content string on stdout; exit code 1 if not found.
|
||||
resolve_template_content() {
|
||||
local template_name="$1"
|
||||
local repo_root="$2"
|
||||
local base="$repo_root/.specify/templates"
|
||||
|
||||
# Collect all layers (highest priority first)
|
||||
local -a layer_paths=()
|
||||
local -a layer_strategies=()
|
||||
|
||||
# Priority 1: Project overrides (always "replace")
|
||||
local override="$base/overrides/${template_name}.md"
|
||||
if [ -f "$override" ]; then
|
||||
layer_paths+=("$override")
|
||||
layer_strategies+=("replace")
|
||||
fi
|
||||
|
||||
# Priority 2: Installed presets (sorted by priority from .registry)
|
||||
local presets_dir="$repo_root/.specify/presets"
|
||||
if [ -d "$presets_dir" ]; then
|
||||
local registry_file="$presets_dir/.registry"
|
||||
local sorted_presets=""
|
||||
if [ -f "$registry_file" ] && command -v python3 >/dev/null 2>&1; then
|
||||
if sorted_presets=$(SPECKIT_REGISTRY="$registry_file" python3 -c "
|
||||
import json, sys, os
|
||||
try:
|
||||
with open(os.environ['SPECKIT_REGISTRY']) as f:
|
||||
data = json.load(f)
|
||||
presets = data.get('presets', {})
|
||||
for pid, meta in sorted(presets.items(), key=lambda x: x[1].get('priority', 10) if isinstance(x[1], dict) else 10):
|
||||
if isinstance(meta, dict) and meta.get('enabled', True) is not False:
|
||||
print(pid)
|
||||
except Exception:
|
||||
sys.exit(1)
|
||||
" 2>/dev/null); then
|
||||
if [ -n "$sorted_presets" ]; then
|
||||
local yaml_warned=false
|
||||
while IFS= read -r preset_id; do
|
||||
# Read strategy and file path from preset manifest
|
||||
local strategy="replace"
|
||||
local manifest_file=""
|
||||
local manifest="$presets_dir/$preset_id/preset.yml"
|
||||
if [ -f "$manifest" ] && command -v python3 >/dev/null 2>&1; then
|
||||
# Requires PyYAML; falls back to replace/convention if unavailable
|
||||
local result
|
||||
local py_stderr
|
||||
py_stderr=$(mktemp)
|
||||
result=$(SPECKIT_MANIFEST="$manifest" SPECKIT_TMPL="$template_name" python3 -c "
|
||||
import sys, os
|
||||
try:
|
||||
import yaml
|
||||
except ImportError:
|
||||
print('yaml_missing', file=sys.stderr)
|
||||
print('replace\t')
|
||||
sys.exit(0)
|
||||
try:
|
||||
with open(os.environ['SPECKIT_MANIFEST']) as f:
|
||||
data = yaml.safe_load(f)
|
||||
for t in data.get('provides', {}).get('templates', []):
|
||||
if t.get('name') == os.environ['SPECKIT_TMPL'] and t.get('type', 'template') == 'template':
|
||||
print(t.get('strategy', 'replace') + '\t' + t.get('file', ''))
|
||||
sys.exit(0)
|
||||
print('replace\t')
|
||||
except Exception:
|
||||
print('replace\t')
|
||||
" 2>"$py_stderr")
|
||||
local parse_status=$?
|
||||
if [ $parse_status -eq 0 ] && [ -n "$result" ]; then
|
||||
IFS=$'\t' read -r strategy manifest_file <<< "$result"
|
||||
strategy=$(printf '%s' "$strategy" | tr '[:upper:]' '[:lower:]')
|
||||
fi
|
||||
if [ "$yaml_warned" = false ] && grep -q 'yaml_missing' "$py_stderr" 2>/dev/null; then
|
||||
echo "Warning: PyYAML not available; composition strategies may be ignored" >&2
|
||||
yaml_warned=true
|
||||
fi
|
||||
rm -f "$py_stderr"
|
||||
fi
|
||||
# Try manifest file path first, then convention path
|
||||
local candidate=""
|
||||
if [ -n "$manifest_file" ]; then
|
||||
# Reject absolute paths and parent traversal
|
||||
case "$manifest_file" in
|
||||
/*|*../*|../*) manifest_file="" ;;
|
||||
esac
|
||||
fi
|
||||
if [ -n "$manifest_file" ]; then
|
||||
local mf="$presets_dir/$preset_id/$manifest_file"
|
||||
[ -f "$mf" ] && candidate="$mf"
|
||||
fi
|
||||
if [ -z "$candidate" ]; then
|
||||
local cf="$presets_dir/$preset_id/templates/${template_name}.md"
|
||||
[ -f "$cf" ] && candidate="$cf"
|
||||
fi
|
||||
if [ -n "$candidate" ]; then
|
||||
layer_paths+=("$candidate")
|
||||
layer_strategies+=("$strategy")
|
||||
fi
|
||||
done <<< "$sorted_presets"
|
||||
fi
|
||||
else
|
||||
# python3 failed — fall back to unordered directory scan (replace only)
|
||||
for preset in "$presets_dir"/*/; do
|
||||
[ -d "$preset" ] || continue
|
||||
local candidate="$preset/templates/${template_name}.md"
|
||||
if [ -f "$candidate" ]; then
|
||||
layer_paths+=("$candidate")
|
||||
layer_strategies+=("replace")
|
||||
fi
|
||||
done
|
||||
fi
|
||||
else
|
||||
# No python3 or registry — fall back to unordered directory scan (replace only)
|
||||
for preset in "$presets_dir"/*/; do
|
||||
[ -d "$preset" ] || continue
|
||||
local candidate="$preset/templates/${template_name}.md"
|
||||
if [ -f "$candidate" ]; then
|
||||
layer_paths+=("$candidate")
|
||||
layer_strategies+=("replace")
|
||||
fi
|
||||
done
|
||||
fi
|
||||
fi
|
||||
|
||||
# Priority 3: Extension-provided templates (always "replace")
|
||||
local ext_dir="$repo_root/.specify/extensions"
|
||||
if [ -d "$ext_dir" ]; then
|
||||
for ext in "$ext_dir"/*/; do
|
||||
[ -d "$ext" ] || continue
|
||||
case "$(basename "$ext")" in .*) continue;; esac
|
||||
local candidate="$ext/templates/${template_name}.md"
|
||||
if [ -f "$candidate" ]; then
|
||||
layer_paths+=("$candidate")
|
||||
layer_strategies+=("replace")
|
||||
fi
|
||||
done
|
||||
fi
|
||||
|
||||
# Priority 4: Core templates (always "replace")
|
||||
local core="$base/${template_name}.md"
|
||||
if [ -f "$core" ]; then
|
||||
layer_paths+=("$core")
|
||||
layer_strategies+=("replace")
|
||||
fi
|
||||
|
||||
local count=${#layer_paths[@]}
|
||||
[ "$count" -eq 0 ] && return 1
|
||||
|
||||
# Check if any layer uses a non-replace strategy
|
||||
local has_composition=false
|
||||
for s in "${layer_strategies[@]}"; do
|
||||
[ "$s" != "replace" ] && has_composition=true && break
|
||||
done
|
||||
|
||||
# If the top (highest-priority) layer is replace, it wins entirely —
|
||||
# lower layers are irrelevant regardless of their strategies.
|
||||
if [ "${layer_strategies[0]}" = "replace" ]; then
|
||||
cat "${layer_paths[0]}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [ "$has_composition" = false ]; then
|
||||
cat "${layer_paths[0]}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Find the effective base: scan from highest priority (index 0) downward
|
||||
# to find the nearest replace layer. Only compose layers above that base.
|
||||
local base_idx=-1
|
||||
local i
|
||||
for (( i=0; i<count; i++ )); do
|
||||
if [ "${layer_strategies[$i]}" = "replace" ]; then
|
||||
base_idx=$i
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
if [ $base_idx -lt 0 ]; then
|
||||
return 1 # no base layer found
|
||||
fi
|
||||
|
||||
# Read the base content; compose layers above the base (higher priority)
|
||||
local content
|
||||
content=$(cat "${layer_paths[$base_idx]}"; printf x)
|
||||
content="${content%x}"
|
||||
|
||||
for (( i=base_idx-1; i>=0; i-- )); do
|
||||
local path="${layer_paths[$i]}"
|
||||
local strat="${layer_strategies[$i]}"
|
||||
local layer_content
|
||||
# Preserve trailing newlines
|
||||
layer_content=$(cat "$path"; printf x)
|
||||
layer_content="${layer_content%x}"
|
||||
|
||||
case "$strat" in
|
||||
replace) content="$layer_content" ;;
|
||||
prepend) content="$(printf '%s\n\n%s' "$layer_content" "$content")" ;;
|
||||
append) content="$(printf '%s\n\n%s' "$content" "$layer_content")" ;;
|
||||
wrap)
|
||||
case "$layer_content" in
|
||||
*'{CORE_TEMPLATE}'*) ;;
|
||||
*) echo "Error: wrap strategy missing {CORE_TEMPLATE} placeholder" >&2; return 1 ;;
|
||||
esac
|
||||
while [[ "$layer_content" == *'{CORE_TEMPLATE}'* ]]; do
|
||||
local before="${layer_content%%\{CORE_TEMPLATE\}*}"
|
||||
local after="${layer_content#*\{CORE_TEMPLATE\}}"
|
||||
layer_content="${before}${content}${after}"
|
||||
done
|
||||
content="$layer_content"
|
||||
;;
|
||||
*) echo "Error: unknown strategy '$strat'" >&2; return 1 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
printf '%s' "$content"
|
||||
return 0
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user