2 Commits

Author	SHA1	Message	Date
Copilot	a067d4c2e3	feat(presets): Composition strategies (prepend, append, wrap) for templates, commands, and scripts (#2133) ... * fix: rebase onto upstream/main, resolve conflicts with PR #2189 upstream/main merged PR #2189 (wrap-only strategy) which overlaps with our comprehensive composition strategies (prepend/append/wrap). Resolved conflicts keeping our implementation as source of truth: - README: keep our future considerations (composition is now fully implemented, not a future item) - presets.py: keep our composition architecture (_reconcile_composed_commands, collect_all_layers, resolve_content) while preserving #2189's _substitute_core_template which is used by agents.py for skill generation - tests: keep both test sets (our composition tests + #2189's wrap tests), removed TestReplayWrapsForCommand and TestInstallRemoveWrapLifecycle which test the superseded _replay_wraps_for_command API; our composition tests cover equivalent scenarios - Restored missing _unregister_commands call in remove() that was lost during #2189 merge * fix: re-create skill directory in _reconcile_skills after removal After _unregister_skills removes a skill directory, _register_skills skips writing because the dir no longer passes the is_dir() check. Fix by ensuring the skill subdirectory exists before calling _register_skills so the next winning preset's content gets registered. Fixes the Claude E2E failure where removing a top-priority override preset left skill-based agents without any SKILL.md file. * fix: address twenty-third round of Copilot PR review feedback - Protect reconciliation in remove(): wrap _reconcile_composed_commands and _reconcile_skills in try/except so failures emit a warning instead of leaving the project in an inconsistent state - Protect reconciliation in install(): same pattern for post-install reconciliation so partial installs don't lack cleanup - Inherit scripts/agent_scripts from base frontmatter: when composing commands, merge scripts and agent_scripts keys from the base command's frontmatter into the top layer's frontmatter if missing, preventing composed commands from losing required script references - Add tier-5 bundled core fallback to collect_all_layers(): check the bundled core_pack (wheel) or repo-root templates (source checkout) when .specify/templates/ doesn't contain the core file, matching resolve()'s tier-5 fallback so composition can always find a base layer * fix: address twenty-fourth round of Copilot PR review feedback - Use yaml.safe_load for frontmatter parsing in resolve_content instead of CommandRegistrar.parse_frontmatter which uses naive find('---',3); strip strategy key from final frontmatter to prevent leaking internal composition directives into rendered agent command files - Filter _reconcile_skills to specific commands: use _FilteredManifest wrapper so only the commands being reconciled get their skills updated, preventing accidental overwrites of other commands' skills that may be owned by higher-priority presets * fix: address twenty-fifth round of Copilot PR review feedback - Support legacy command-frontmatter strategy: when preset.yml doesn't declare a strategy, check the command file's YAML frontmatter for strategy: wrap as a fallback so legacy wrap presets participate in composition and multi-preset chaining - Guard skill dir creation in _reconcile_skills: only re-create the skill directory if the skill was previously managed (listed in some preset's registered_skills), avoiding creation of new skill dirs that _register_skills would normally skip * fix: add explanatory comment to empty except in legacy frontmatter parsing * fix: address twenty-sixth round of Copilot PR review feedback - Unregister stale commands when composition fails: when resolve_content returns None during reconciliation (base layer removed), unregister the command from non-skill agents and emit a warning - Load extension aliases during reconciliation: _register_command_from_path now checks extension.yml for aliases when the winning layer is an extension, so alias files are restored after preset removal - Use line-based fence detection for legacy frontmatter strategy fallback: scan for --- on its own line instead of split('---',2) to avoid mis-parsing YAML values containing --- * fix: address twenty-seventh round of Copilot PR review feedback - Handle non-preset winners in _reconcile_skills: when the winning layer is core/extension/project-override, restore skills via _unregister_skills so skill-based agents stay consistent with the priority stack - Update base_frontmatter_text on replace layers: when a higher-priority replace layer occurs during composition, update both top and base frontmatter so scripts/agent_scripts inheritance reflects the effective base beneath the top composed layer * fix: address twenty-eighth round of Copilot PR review feedback - Parse only interior lines in _parse_fm_yaml: use lines[1:-1] instead of filtering all --- lines, preventing corruption when YAML values contain a line that is exactly --- - Omit empty frontmatter: skip re-rendering when top_fm is empty dict to avoid emitting ---/{}/--- for intentionally empty frontmatter - Update scaffold wrap comment: mention both {CORE_TEMPLATE} and $CORE_SCRIPT placeholders for templates/commands vs scripts - Clarify shell composition scope in ARCHITECTURE.md: note that bash/PS1 resolve_template_content only handles templates; command/script composition is handled by the Python resolver * fix: address twenty-ninth round of Copilot PR review feedback - Fix TestCollectAllLayers docstring: reference collect_all_layers() - Add default/unknown strategy handling in bash/PS1 composition: error on unrecognized strategy values instead of silently skipping - Fix comment: .composed/ is a persistent dir, not temporary - Fix comment: legacy fallback checks all valid strategies, not just wrap - Cache PresetRegistry in _reconcile_skills: build presets_by_priority once instead of constructing registry per-command * fix: address thirtieth round of Copilot PR review feedback - Guard legacy frontmatter fallback: only check command file frontmatter for strategy when the manifest entry doesn't explicitly include the strategy key, preventing override of manifest-declared strategies - Document rollback limitation: note that mid-registration failures may leave orphaned agent command files since partial progress isn't captured by the local vars * fix: handle project override skills and extension context in reconciliation * fix: add comment to empty except in extension registration fallback * fix: filter extension commands in reconciliation and fix type annotation * fix: filter extension commands from post-install reconciliation Apply the same extension-installed check used in _register_commands to the reconciliation command list, preventing reconciliation from registering commands for extensions that are not installed. * fix: skip convention fallback for explicit file paths and add stem fallback to tier-5 When a preset manifest provides an explicit file path that does not exist, skip the convention-based fallback to avoid masking typos. Also add speckit.<stem> to <stem>.md fallback in tier-5 bundled/source core lookup for consistency with tier-4. * fix: scan past non-replace layers to find base in resolve_content The base-finding scan now skips non-replace layers below a replace layer instead of stopping at the first non-replace. This fixes the case where a low-priority append/prepend layer sits below a replace that should serve as the base for composition. * fix: add context_note to non-skill agent registration for extensions Add context_note parameter to register_commands_for_non_skill_agents and pass extension name/id during reconciliation so rendered command files preserve the extension-specific context markers. * fix: Optional type, rollback safety, and override skill restoration - Fix context_note type to Optional[str] - Wrap shutil.rmtree in try/except during install rollback - Separate override-backed skills from core/extension in _reconcile_skills * fix: align bash/PS1 base-finding with Python resolver Rewrite bash and PowerShell composition loops to find the effective base replace layer first (scanning bottom-up, skipping non-replace layers below it), then compose only from the base upward. This prevents evaluation of irrelevant lower layers (e.g. a wrap with no placeholder below a replace) and matches resolve_content behavior. * fix: PS1 no-python warning, integration hook for override skills, alias cleanup - Warn when no Python 3 found in PS1 and presets use composition strategies - Apply post_process_skill_content integration hook when restoring override-backed skills so agent-specific flags are preserved - Unregister command aliases alongside primary name when composition fails to prevent orphaned alias files * fix: include aliases in removed_cmd_names during preset removal Read aliases from preset manifest before deleting pack_dir so alias command files are included in unregistration and reconciliation. * fix: add comment to empty except in alias extraction during removal * fix: scan top-down for effective base in all resolvers Change base-finding to scan from highest priority downward to find the nearest replace layer, then compose only layers above it. Prevents evaluation of irrelevant lower layers (e.g. a wrap without placeholder below a higher-priority replace) across Python, bash, and PowerShell. * fix: align CLI composition chain display with top-down base-finding Show only contributing layers (base and above) in preset resolve output, matching resolve_content top-down semantics. Layers below the effective base are omitted since they do not contribute. * fix: guard corrupted registry entries and make manifest authoritative - Add isinstance(meta, dict) guard in bash registry parsing so corrupted entries are skipped instead of breaking priority ordering - Only use convention-based file lookup when the manifest does not list the requested template, making preset.yml authoritative and preventing stray on-disk files from creating unintended layers * fix: align resolve() with manifest file paths and match extension context_note - Update resolve() preset tier to consult manifest file paths before convention-based lookup, matching collect_all_layers behavior - Use exact extension context_note format matching extensions.CommandRegistrar - Update test to declare template in manifest (authoritative manifest) * revert: restore resolve() convention-based behavior for backwards compatibility resolve() is the existing public API used by shell scripts and other callers. Changing it to manifest-authoritative breaks backward compat for presets that rely on convention-based file lookup. Only the new collect_all_layers/resolve_content path uses manifest-authoritative logic. * fix: only pre-compose when this preset is the top composing layer Skip composition in _register_commands when a higher-priority replace layer already wins for the command. Register the raw file instead and let reconciliation write the correct final content. * fix: deduplicate PyYAML warnings and use self.registry in reconciliation - Emit PyYAML-missing warning once per function call in bash/PS1 instead of per-preset to avoid spamming stderr - Use self.registry.list_by_priority() in reconciliation methods instead of constructing new PresetRegistry instances to avoid redundant I/O and potential consistency issues * fix: document strategy handling consistency between layers and registrar Composed output already strips strategy from frontmatter (resolve_content pops it). Raw file registration preserves legacy frontmatter strategy for backward compat; reconciliation corrects the final state. * fix: correct stale comments for alias tracking and base-finding algorithm * security: validate manifest file paths in bash/PowerShell resolvers Reject absolute paths and parent directory traversal (..) in the manifest-declared file field before joining with the preset directory. Matches the Python-side validation in PresetManifest._validate(). --------- Co-authored-by: Manfred Riem <15701806+mnriem@users.noreply.github.com>	2026-04-23 10:07:52 -05:00
Copilot	69ee7a836e	feat(presets): Pluggable preset system with catalog, resolver, and skills propagation (#1787) ... * Initial plan * feat(templates): add pluggable template system with packs, catalog, resolver, and CLI commands Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com> * test(templates): add comprehensive unit tests for template pack system Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com> * feat(presets): pluggable preset system with template/command overrides, catalog, and resolver - Rename 'template packs' to 'presets' to avoid naming collision with core templates - PresetManifest, PresetRegistry, PresetManager, PresetCatalog, PresetResolver in presets.py - Extract CommandRegistrar to agents.py as shared infrastructure - CLI: specify preset list/add/remove/search/resolve/info - CLI: specify preset catalog list/add/remove - --preset option on specify init - Priority-based preset stacking (--priority, lower = higher precedence) - Command overrides registered into all detected agent directories (17+ agents) - Extension command safety: skip registration if target extension not installed - Multi-catalog support: env var, project config, user config, built-in defaults - resolve_template() / Resolve-Template in bash/PowerShell scripts - Self-test preset: overrides all 6 core templates + 1 command - Scaffold with 4 examples: core/extension template and command overrides - Preset catalog (catalog.json, catalog.community.json) - Documentation: README.md, ARCHITECTURE.md, PUBLISHING.md - 110 preset tests, 253 total tests passing * feat(presets): propagate command overrides to skills via init-options - Add save_init_options() / load_init_options() helpers that persist CLI flags from 'specify init' to .specify/init-options.json - PresetManager._register_skills() overwrites SKILL.md files when --ai-skills was used during init and corresponding skill dirs exist - PresetManager._unregister_skills() restores core template content on preset removal - registered_skills stored in preset registry metadata - 8 new tests covering skill override, skip conditions, and restore * fix: address PR check failures (ruff F541, CodeQL URL substring) - Remove extraneous f-prefix from two f-strings without placeholders - Replace substring URL check in test with startswith/endswith assertions to satisfy CodeQL incomplete URL substring sanitization rule * fix: address Copilot PR review comments - Move save_init_options() before preset install so skills propagation works during 'specify init --preset --ai-skills' - Clean up downloaded ZIP after successful preset install during init - Validate --from URL scheme (require HTTPS, HTTP only for localhost) - Expose unregister_commands() on extensions.py CommandRegistrar wrapper instead of reaching into private _registrar field - Use _get_merged_packs() for search() and get_pack_info() so all active catalogs are searched, not just the highest-priority one - Fix fetch_catalog() cache to verify cached URL matches current URL - Fix PresetResolver: script resolution uses .sh extension, consistent file extensions throughout resolve(), and resolve_with_source() delegates to resolve() to honor template_type parameter - Fix bash common.sh: fall through to directory scan when python3 returns empty preset list - Fix PowerShell Resolve-Template: filter out dot-folders and sort extensions deterministically * fix: narrow empty except blocks and add explanatory comments * fix: address Copilot PR review comments (round 2) - Fix init --preset error masking: distinguish "not found" from real errors - Fix bash resolve_template: skip hidden dirs in extensions (match Python/PS) - Fix temp dir leaks in tests: use temp_dir fixture instead of mkdtemp - Fix self-test catalog entry: add note that it's local-only (no download_url) - Fix Windows path issue in resolve_with_source: use Path.relative_to() - Fix skill restore path: use project's .specify/templates/commands/ not source tree - Add encoding="utf-8" to all file read/write in agents.py - Update test to set up core command templates for skill restoration * fix: remove self-test from catalog.json (local-only preset) * fix: address Copilot PR review comments (round 3) - Fix PS Resolve-Template fallback to skip dot-prefixed dirs (.cache) - Rename _catalog to _catalog_name for consistency with extension system - Enforce install_allowed policy in CLI preset add and download_pack() - Fix shell injection: pass registry path via env var instead of string interpolation * fix: correct PresetError docstring from template to preset * Removed CHANGELOG requirement * Applying review recommendations * Applying review recommendations * Applying review recommendations * Applying review recommendations --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: mnriem <15701806+mnriem@users.noreply.github.com>	2026-03-13 15:09:14 -05:00