mirror of
https://github.com/github/spec-kit.git
synced 2026-07-03 12:28:06 +08:00
feat(presets): Composition strategies (prepend, append, wrap) for templates, commands, and scripts (#2133)
* fix: rebase onto upstream/main, resolve conflicts with PR #2189 upstream/main merged PR #2189 (wrap-only strategy) which overlaps with our comprehensive composition strategies (prepend/append/wrap). Resolved conflicts keeping our implementation as source of truth: - README: keep our future considerations (composition is now fully implemented, not a future item) - presets.py: keep our composition architecture (_reconcile_composed_commands, collect_all_layers, resolve_content) while preserving #2189's _substitute_core_template which is used by agents.py for skill generation - tests: keep both test sets (our composition tests + #2189's wrap tests), removed TestReplayWrapsForCommand and TestInstallRemoveWrapLifecycle which test the superseded _replay_wraps_for_command API; our composition tests cover equivalent scenarios - Restored missing _unregister_commands call in remove() that was lost during #2189 merge * fix: re-create skill directory in _reconcile_skills after removal After _unregister_skills removes a skill directory, _register_skills skips writing because the dir no longer passes the is_dir() check. Fix by ensuring the skill subdirectory exists before calling _register_skills so the next winning preset's content gets registered. Fixes the Claude E2E failure where removing a top-priority override preset left skill-based agents without any SKILL.md file. * fix: address twenty-third round of Copilot PR review feedback - Protect reconciliation in remove(): wrap _reconcile_composed_commands and _reconcile_skills in try/except so failures emit a warning instead of leaving the project in an inconsistent state - Protect reconciliation in install(): same pattern for post-install reconciliation so partial installs don't lack cleanup - Inherit scripts/agent_scripts from base frontmatter: when composing commands, merge scripts and agent_scripts keys from the base command's frontmatter into the top layer's frontmatter if missing, preventing composed commands from losing required script references - Add tier-5 bundled core fallback to collect_all_layers(): check the bundled core_pack (wheel) or repo-root templates (source checkout) when .specify/templates/ doesn't contain the core file, matching resolve()'s tier-5 fallback so composition can always find a base layer * fix: address twenty-fourth round of Copilot PR review feedback - Use yaml.safe_load for frontmatter parsing in resolve_content instead of CommandRegistrar.parse_frontmatter which uses naive find('---',3); strip strategy key from final frontmatter to prevent leaking internal composition directives into rendered agent command files - Filter _reconcile_skills to specific commands: use _FilteredManifest wrapper so only the commands being reconciled get their skills updated, preventing accidental overwrites of other commands' skills that may be owned by higher-priority presets * fix: address twenty-fifth round of Copilot PR review feedback - Support legacy command-frontmatter strategy: when preset.yml doesn't declare a strategy, check the command file's YAML frontmatter for strategy: wrap as a fallback so legacy wrap presets participate in composition and multi-preset chaining - Guard skill dir creation in _reconcile_skills: only re-create the skill directory if the skill was previously managed (listed in some preset's registered_skills), avoiding creation of new skill dirs that _register_skills would normally skip * fix: add explanatory comment to empty except in legacy frontmatter parsing * fix: address twenty-sixth round of Copilot PR review feedback - Unregister stale commands when composition fails: when resolve_content returns None during reconciliation (base layer removed), unregister the command from non-skill agents and emit a warning - Load extension aliases during reconciliation: _register_command_from_path now checks extension.yml for aliases when the winning layer is an extension, so alias files are restored after preset removal - Use line-based fence detection for legacy frontmatter strategy fallback: scan for --- on its own line instead of split('---',2) to avoid mis-parsing YAML values containing --- * fix: address twenty-seventh round of Copilot PR review feedback - Handle non-preset winners in _reconcile_skills: when the winning layer is core/extension/project-override, restore skills via _unregister_skills so skill-based agents stay consistent with the priority stack - Update base_frontmatter_text on replace layers: when a higher-priority replace layer occurs during composition, update both top and base frontmatter so scripts/agent_scripts inheritance reflects the effective base beneath the top composed layer * fix: address twenty-eighth round of Copilot PR review feedback - Parse only interior lines in _parse_fm_yaml: use lines[1:-1] instead of filtering all --- lines, preventing corruption when YAML values contain a line that is exactly --- - Omit empty frontmatter: skip re-rendering when top_fm is empty dict to avoid emitting ---/{}/--- for intentionally empty frontmatter - Update scaffold wrap comment: mention both {CORE_TEMPLATE} and $CORE_SCRIPT placeholders for templates/commands vs scripts - Clarify shell composition scope in ARCHITECTURE.md: note that bash/PS1 resolve_template_content only handles templates; command/script composition is handled by the Python resolver * fix: address twenty-ninth round of Copilot PR review feedback - Fix TestCollectAllLayers docstring: reference collect_all_layers() - Add default/unknown strategy handling in bash/PS1 composition: error on unrecognized strategy values instead of silently skipping - Fix comment: .composed/ is a persistent dir, not temporary - Fix comment: legacy fallback checks all valid strategies, not just wrap - Cache PresetRegistry in _reconcile_skills: build presets_by_priority once instead of constructing registry per-command * fix: address thirtieth round of Copilot PR review feedback - Guard legacy frontmatter fallback: only check command file frontmatter for strategy when the manifest entry doesn't explicitly include the strategy key, preventing override of manifest-declared strategies - Document rollback limitation: note that mid-registration failures may leave orphaned agent command files since partial progress isn't captured by the local vars * fix: handle project override skills and extension context in reconciliation * fix: add comment to empty except in extension registration fallback * fix: filter extension commands in reconciliation and fix type annotation * fix: filter extension commands from post-install reconciliation Apply the same extension-installed check used in _register_commands to the reconciliation command list, preventing reconciliation from registering commands for extensions that are not installed. * fix: skip convention fallback for explicit file paths and add stem fallback to tier-5 When a preset manifest provides an explicit file path that does not exist, skip the convention-based fallback to avoid masking typos. Also add speckit.<stem> to <stem>.md fallback in tier-5 bundled/source core lookup for consistency with tier-4. * fix: scan past non-replace layers to find base in resolve_content The base-finding scan now skips non-replace layers below a replace layer instead of stopping at the first non-replace. This fixes the case where a low-priority append/prepend layer sits below a replace that should serve as the base for composition. * fix: add context_note to non-skill agent registration for extensions Add context_note parameter to register_commands_for_non_skill_agents and pass extension name/id during reconciliation so rendered command files preserve the extension-specific context markers. * fix: Optional type, rollback safety, and override skill restoration - Fix context_note type to Optional[str] - Wrap shutil.rmtree in try/except during install rollback - Separate override-backed skills from core/extension in _reconcile_skills * fix: align bash/PS1 base-finding with Python resolver Rewrite bash and PowerShell composition loops to find the effective base replace layer first (scanning bottom-up, skipping non-replace layers below it), then compose only from the base upward. This prevents evaluation of irrelevant lower layers (e.g. a wrap with no placeholder below a replace) and matches resolve_content behavior. * fix: PS1 no-python warning, integration hook for override skills, alias cleanup - Warn when no Python 3 found in PS1 and presets use composition strategies - Apply post_process_skill_content integration hook when restoring override-backed skills so agent-specific flags are preserved - Unregister command aliases alongside primary name when composition fails to prevent orphaned alias files * fix: include aliases in removed_cmd_names during preset removal Read aliases from preset manifest before deleting pack_dir so alias command files are included in unregistration and reconciliation. * fix: add comment to empty except in alias extraction during removal * fix: scan top-down for effective base in all resolvers Change base-finding to scan from highest priority downward to find the nearest replace layer, then compose only layers above it. Prevents evaluation of irrelevant lower layers (e.g. a wrap without placeholder below a higher-priority replace) across Python, bash, and PowerShell. * fix: align CLI composition chain display with top-down base-finding Show only contributing layers (base and above) in preset resolve output, matching resolve_content top-down semantics. Layers below the effective base are omitted since they do not contribute. * fix: guard corrupted registry entries and make manifest authoritative - Add isinstance(meta, dict) guard in bash registry parsing so corrupted entries are skipped instead of breaking priority ordering - Only use convention-based file lookup when the manifest does not list the requested template, making preset.yml authoritative and preventing stray on-disk files from creating unintended layers * fix: align resolve() with manifest file paths and match extension context_note - Update resolve() preset tier to consult manifest file paths before convention-based lookup, matching collect_all_layers behavior - Use exact extension context_note format matching extensions.CommandRegistrar - Update test to declare template in manifest (authoritative manifest) * revert: restore resolve() convention-based behavior for backwards compatibility resolve() is the existing public API used by shell scripts and other callers. Changing it to manifest-authoritative breaks backward compat for presets that rely on convention-based file lookup. Only the new collect_all_layers/resolve_content path uses manifest-authoritative logic. * fix: only pre-compose when this preset is the top composing layer Skip composition in _register_commands when a higher-priority replace layer already wins for the command. Register the raw file instead and let reconciliation write the correct final content. * fix: deduplicate PyYAML warnings and use self.registry in reconciliation - Emit PyYAML-missing warning once per function call in bash/PS1 instead of per-preset to avoid spamming stderr - Use self.registry.list_by_priority() in reconciliation methods instead of constructing new PresetRegistry instances to avoid redundant I/O and potential consistency issues * fix: document strategy handling consistency between layers and registrar Composed output already strips strategy from frontmatter (resolve_content pops it). Raw file registration preserves legacy frontmatter strategy for backward compat; reconciliation corrects the final state. * fix: correct stale comments for alias tracking and base-finding algorithm * security: validate manifest file paths in bash/PowerShell resolvers Reject absolute paths and parent directory traversal (..) in the manifest-declared file field before joining with the preset directory. Matches the Python-side validation in PresetManifest._validate(). --------- Co-authored-by: Manfred Riem <15701806+mnriem@users.noreply.github.com>
This commit is contained in:
Copilot
@@ -287,6 +287,21 @@ function Test-DirHasFiles {
|
||||
}
|
||||
}
|
||||
|
||||
# Find a usable Python 3 executable (python3, python, or py -3).
|
||||
# Returns the command/arguments as an array, or $null if none found.
|
||||
function Get-Python3Command {
|
||||
if (Get-Command python3 -ErrorAction SilentlyContinue) { return @('python3') }
|
||||
if (Get-Command python -ErrorAction SilentlyContinue) {
|
||||
$ver = & python --version 2>&1
|
||||
if ($ver -match 'Python 3') { return @('python') }
|
||||
}
|
||||
if (Get-Command py -ErrorAction SilentlyContinue) {
|
||||
$ver = & py -3 --version 2>&1
|
||||
if ($ver -match 'Python 3') { return @('py', '-3') }
|
||||
}
|
||||
return $null
|
||||
}
|
||||
|
||||
# Resolve a template name to a file path using the priority stack:
|
||||
# 1. .specify/templates/overrides/
|
||||
# 2. .specify/presets/<preset-id>/templates/ (sorted by priority from .registry)
|
||||
@@ -315,6 +330,7 @@ function Resolve-Template {
|
||||
$presets = $registryData.presets
|
||||
if ($presets) {
|
||||
$sortedPresets = $presets.PSObject.Properties |
|
||||
Where-Object { $null -eq $_.Value.enabled -or $_.Value.enabled -ne $false } |
|
||||
Sort-Object { if ($null -ne $_.Value.priority) { $_.Value.priority } else { 10 } } |
|
||||
ForEach-Object { $_.Name }
|
||||
}
|
||||
@@ -354,3 +370,206 @@ function Resolve-Template {
|
||||
return $null
|
||||
}
|
||||
|
||||
# Resolve a template name to composed content using composition strategies.
|
||||
# Reads strategy metadata from preset manifests and composes content
|
||||
# from multiple layers using prepend, append, or wrap strategies.
|
||||
function Resolve-TemplateContent {
|
||||
param(
|
||||
[Parameter(Mandatory=$true)][string]$TemplateName,
|
||||
[Parameter(Mandatory=$true)][string]$RepoRoot
|
||||
)
|
||||
|
||||
$base = Join-Path $RepoRoot '.specify/templates'
|
||||
|
||||
# Collect all layers (highest priority first)
|
||||
$layerPaths = @()
|
||||
$layerStrategies = @()
|
||||
|
||||
# Priority 1: Project overrides (always "replace")
|
||||
$override = Join-Path $base "overrides/$TemplateName.md"
|
||||
if (Test-Path $override) {
|
||||
$layerPaths += $override
|
||||
$layerStrategies += 'replace'
|
||||
}
|
||||
|
||||
# Priority 2: Installed presets (sorted by priority from .registry)
|
||||
$presetsDir = Join-Path $RepoRoot '.specify/presets'
|
||||
if (Test-Path $presetsDir) {
|
||||
$registryFile = Join-Path $presetsDir '.registry'
|
||||
$sortedPresets = @()
|
||||
if (Test-Path $registryFile) {
|
||||
try {
|
||||
$registryData = Get-Content $registryFile -Raw | ConvertFrom-Json
|
||||
$presets = $registryData.presets
|
||||
if ($presets) {
|
||||
$sortedPresets = $presets.PSObject.Properties |
|
||||
Where-Object { $null -eq $_.Value.enabled -or $_.Value.enabled -ne $false } |
|
||||
Sort-Object { if ($null -ne $_.Value.priority) { $_.Value.priority } else { 10 } } |
|
||||
ForEach-Object { $_.Name }
|
||||
}
|
||||
} catch {
|
||||
$sortedPresets = @()
|
||||
}
|
||||
}
|
||||
|
||||
if ($sortedPresets.Count -gt 0) {
|
||||
$pyCmd = Get-Python3Command
|
||||
if (-not $pyCmd) {
|
||||
# Check if any preset has strategy fields that would be ignored
|
||||
foreach ($pid in $sortedPresets) {
|
||||
$mf = Join-Path $presetsDir "$pid/preset.yml"
|
||||
if ((Test-Path $mf) -and (Select-String -Path $mf -Pattern 'strategy:' -Quiet -ErrorAction SilentlyContinue)) {
|
||||
Write-Warning "No Python 3 found; preset composition strategies will be ignored"
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
$yamlWarned = $false
|
||||
foreach ($presetId in $sortedPresets) {
|
||||
# Read strategy and file path from preset manifest
|
||||
$strategy = 'replace'
|
||||
$manifestFilePath = ''
|
||||
$manifest = Join-Path $presetsDir "$presetId/preset.yml"
|
||||
if ((Test-Path $manifest) -and $pyCmd) {
|
||||
try {
|
||||
# Use Python to parse YAML manifest for strategy and file path
|
||||
$pyArgs = if ($pyCmd.Count -gt 1) { $pyCmd[1..($pyCmd.Count-1)] } else { @() }
|
||||
$pyStderrFile = [System.IO.Path]::GetTempFileName()
|
||||
$stratResult = & $pyCmd[0] @pyArgs -c @"
|
||||
import sys
|
||||
try:
|
||||
import yaml
|
||||
except ImportError:
|
||||
print('yaml_missing', file=sys.stderr)
|
||||
print('replace\t')
|
||||
sys.exit(0)
|
||||
try:
|
||||
with open(sys.argv[1]) as f:
|
||||
data = yaml.safe_load(f)
|
||||
for t in data.get('provides', {}).get('templates', []):
|
||||
if t.get('name') == sys.argv[2] and t.get('type', 'template') == 'template':
|
||||
print(t.get('strategy', 'replace') + '\t' + t.get('file', ''))
|
||||
sys.exit(0)
|
||||
print('replace\t')
|
||||
except Exception:
|
||||
print('replace\t')
|
||||
"@ $manifest $TemplateName 2>$pyStderrFile
|
||||
if ($stratResult) {
|
||||
$parts = $stratResult.Trim() -split "`t", 2
|
||||
$strategy = $parts[0].ToLowerInvariant()
|
||||
if ($parts.Count -gt 1 -and $parts[1]) { $manifestFilePath = $parts[1] }
|
||||
}
|
||||
if (-not $yamlWarned -and (Test-Path $pyStderrFile) -and (Get-Content $pyStderrFile -Raw -ErrorAction SilentlyContinue) -match 'yaml_missing') {
|
||||
Write-Warning "PyYAML not available; composition strategies may be ignored"
|
||||
$yamlWarned = $true
|
||||
}
|
||||
Remove-Item $pyStderrFile -Force -ErrorAction SilentlyContinue
|
||||
} catch {
|
||||
$strategy = 'replace'
|
||||
if ($pyStderrFile) { Remove-Item $pyStderrFile -Force -ErrorAction SilentlyContinue }
|
||||
}
|
||||
}
|
||||
# Try manifest file path first, then convention path
|
||||
$candidate = $null
|
||||
if ($manifestFilePath) {
|
||||
# Reject absolute paths and parent traversal
|
||||
if ([System.IO.Path]::IsPathRooted($manifestFilePath) -or $manifestFilePath -match '\.\.[\\/]') {
|
||||
$manifestFilePath = ''
|
||||
}
|
||||
}
|
||||
if ($manifestFilePath) {
|
||||
$mf = Join-Path $presetsDir "$presetId/$manifestFilePath"
|
||||
if (Test-Path $mf) { $candidate = $mf }
|
||||
}
|
||||
if (-not $candidate) {
|
||||
$cf = Join-Path $presetsDir "$presetId/templates/$TemplateName.md"
|
||||
if (Test-Path $cf) { $candidate = $cf }
|
||||
}
|
||||
if ($candidate) {
|
||||
$layerPaths += $candidate
|
||||
$layerStrategies += $strategy
|
||||
}
|
||||
}
|
||||
} else {
|
||||
# Fallback: alphabetical directory order (no registry or parse failure)
|
||||
foreach ($preset in Get-ChildItem -Path $presetsDir -Directory -ErrorAction SilentlyContinue | Where-Object { $_.Name -notlike '.*' }) {
|
||||
$candidate = Join-Path $preset.FullName "templates/$TemplateName.md"
|
||||
if (Test-Path $candidate) {
|
||||
$layerPaths += $candidate
|
||||
$layerStrategies += 'replace'
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Priority 3: Extension-provided templates (always "replace")
|
||||
$extDir = Join-Path $RepoRoot '.specify/extensions'
|
||||
if (Test-Path $extDir) {
|
||||
foreach ($ext in Get-ChildItem -Path $extDir -Directory -ErrorAction SilentlyContinue | Where-Object { $_.Name -notlike '.*' } | Sort-Object Name) {
|
||||
$candidate = Join-Path $ext.FullName "templates/$TemplateName.md"
|
||||
if (Test-Path $candidate) {
|
||||
$layerPaths += $candidate
|
||||
$layerStrategies += 'replace'
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
# Priority 4: Core templates (always "replace")
|
||||
$core = Join-Path $base "$TemplateName.md"
|
||||
if (Test-Path $core) {
|
||||
$layerPaths += $core
|
||||
$layerStrategies += 'replace'
|
||||
}
|
||||
|
||||
if ($layerPaths.Count -eq 0) { return $null }
|
||||
|
||||
# If the top (highest-priority) layer is replace, it wins entirely —
|
||||
# lower layers are irrelevant regardless of their strategies.
|
||||
if ($layerStrategies[0] -eq 'replace') {
|
||||
return (Get-Content $layerPaths[0] -Raw)
|
||||
}
|
||||
|
||||
# Check if any layer uses a non-replace strategy
|
||||
$hasComposition = $false
|
||||
foreach ($s in $layerStrategies) {
|
||||
if ($s -ne 'replace') { $hasComposition = $true; break }
|
||||
}
|
||||
|
||||
if (-not $hasComposition) {
|
||||
return (Get-Content $layerPaths[0] -Raw)
|
||||
}
|
||||
|
||||
# Find the effective base: scan from highest priority (index 0) downward
|
||||
# to find the nearest replace layer. Only compose layers above that base.
|
||||
$baseIdx = -1
|
||||
for ($i = 0; $i -lt $layerPaths.Count; $i++) {
|
||||
if ($layerStrategies[$i] -eq 'replace') {
|
||||
$baseIdx = $i
|
||||
break
|
||||
}
|
||||
}
|
||||
if ($baseIdx -lt 0) { return $null }
|
||||
|
||||
$content = Get-Content $layerPaths[$baseIdx] -Raw
|
||||
|
||||
for ($i = $baseIdx - 1; $i -ge 0; $i--) {
|
||||
$path = $layerPaths[$i]
|
||||
$strat = $layerStrategies[$i]
|
||||
$layerContent = Get-Content $path -Raw
|
||||
|
||||
switch ($strat) {
|
||||
'replace' { $content = $layerContent }
|
||||
'prepend' { $content = "$layerContent`n`n$content" }
|
||||
'append' { $content = "$content`n`n$layerContent" }
|
||||
'wrap' {
|
||||
if (-not $layerContent.Contains('{CORE_TEMPLATE}')) {
|
||||
throw "Wrap strategy missing {CORE_TEMPLATE} placeholder"
|
||||
}
|
||||
$content = $layerContent.Replace('{CORE_TEMPLATE}', $content)
|
||||
}
|
||||
default { throw "Unknown strategy: $strat" }
|
||||
}
|
||||
}
|
||||
|
||||
return $content
|
||||
}
|
||||
Reference in New Issue
Block a user