feat(scripts): port create-new-feature, setup-plan and setup-tasks to Python (#3386)

* feat(scripts): port create-new-feature, setup-plan and setup-tasks to Python

Ports the three core workflow scripts to Python as part of #3280,
following the check-prerequisites PoC pattern from #3302. Adds
resolve_template() to the shared common.py module and parity tests
that run bash and Python side by side.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(tests): treat only None env as unset in parity run helper

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): fall back to directory scan on any registry error, skip hidden preset dirs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* feat(templates): add py: lines for setup_plan and setup_tasks

Ships with the scripts they reference; the remaining templates got
their py: lines in #3403.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: support py variant in skills placeholder resolver

resolve_skill_placeholders only accepted sh/ps, so a py init option
fell into the fallback path and {SCRIPT} rendered without an
interpreter prefix. Accept py and prefix the resolved interpreter,
matching process_template. Also guard ps_cmd against a missing
PowerShell with a clear assert.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: pin clean-error behavior for invalid --number

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(scripts): reword unused-arg comment to match implementation

The loop accepts and silently ignores extra positional args (it doesn't
build a collected list); match the wording to what the code and
setup-plan.sh actually do.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: fall back when configured script variant is missing from frontmatter

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): reject signed/whitespace --number values to match bash 10# parity

The bash twin uses $((10#$BRANCH_NUMBER)), which rejects signed and
whitespace-padded values. Python's int() accepted them (e.g. -1),
producing a malformed -01-... prefix that sequential scans ignore.
Restrict --number to unsigned decimal digits before conversion, and
pin the parity with a bash-comparison test.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): complete Python port installation

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(integrations): fall back for missing script variants

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: make Python script checks platform-aware

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix Windows Python command invocation parity

Use PowerShell's call operator for spaced Python interpreter paths and align setup-tasks missing-template errors across script variants.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): preserve cross-platform Python parity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: reject signed PowerShell feature numbers

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): align feature number range

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): reject exhausted feature numbers

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): complete create feature parity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): align create feature outputs

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): harden cross-platform parity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): keep truncation JSON clean

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): align setup failure parity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): close parity edge cases

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): propagate PowerShell setup errors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): harden fallback resolution

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): stabilize PowerShell fallbacks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): complete setup-plan parity

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(cli): require runnable script fallbacks

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(cli): preserve shell fallback without preference

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): restore help and symlink parity

- setup-tasks.ps1: check -Help before unknown-argument validation so
  '-Help --bogus' exits 0 like the Bash/Python variants
- common.py: strip the repo root prefix lexically in persist_feature_json
  instead of resolve(), so a symlinked specs/ still persists the relative
  'specs/NNN-name' path the Bash/PowerShell helpers store

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(scripts): align persist-hint quoting with shlex.quote

- create-new-feature.sh: replace printf %q with a shell_quote helper that
  emits shlex.quote-identical output, so the persistence hints stay
  byte-identical between the Bash and Python variants (printf %q output
  also varies between bash versions)
- promote the negative --number test to an all-variants parity test now
  that Bash and PowerShell reject signed values consistently
- add a spaced-repo-path parity test for the persistence hints

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Marsel Safin
2026-07-21 18:40:15 +02:00
committed by GitHub
parent 7873c447bd
commit 2a0ada9a6a
28 changed files with 2883 additions and 167 deletions

View File

@@ -140,10 +140,9 @@ def _install_shared_infra(
"""Install shared infrastructure files into *project_path*.
Copies ``.specify/scripts/<variant>/`` and ``.specify/templates/`` from
the bundled core_pack or source checkout, where ``<variant>`` is
``bash`` when *script_type* is ``"sh"``, ``python`` when it is ``"py"``,
and ``powershell`` when it is ``"ps"``. Tracks all installed files in
``speckit.manifest.json``.
the bundled core_pack or source checkout. ``sh`` installs Bash, ``ps``
installs PowerShell, and ``py`` installs Python plus the platform shell
fallback. Tracks all installed files in ``speckit.manifest.json``.
Shared scripts and page templates are processed to resolve
``__SPECKIT_COMMAND_<NAME>__`` placeholders using *invoke_separator*

View File

@@ -7,7 +7,6 @@ command files into agent-specific directories in the correct format.
"""
import os
import platform
import re
from copy import deepcopy
from pathlib import Path
@@ -486,26 +485,19 @@ class CommandRegistrar:
init_opts = {}
script_variant = init_opts.get("script")
if script_variant not in {"sh", "ps"}:
fallback_order = []
default_variant = (
"ps" if platform.system().lower().startswith("win") else "sh"
if scripts:
from specify_cli.integrations.base import IntegrationBase
script_variant = IntegrationBase.select_script_variant(
script_variant, scripts
)
secondary_variant = "sh" if default_variant == "ps" else "ps"
if default_variant in scripts:
fallback_order.append(default_variant)
if secondary_variant in scripts:
fallback_order.append(secondary_variant)
for key in scripts:
if key not in fallback_order:
fallback_order.append(key)
script_variant = fallback_order[0] if fallback_order else None
script_command = scripts.get(script_variant) if script_variant else None
if script_command:
if script_variant == "py":
script_command = IntegrationBase.build_python_invocation(
script_command, project_root
)
script_command = script_command.replace("{ARGS}", "$ARGUMENTS")
body = body.replace("{SCRIPT}", script_command)

View File

@@ -86,7 +86,7 @@ def register(app: typer.Typer) -> None:
help="Name for your new project directory (optional if using --here, or use '.' for current directory)",
),
script_type: str = typer.Option(
None, "--script", help="Script type to use: sh or ps"
None, "--script", help="Script type to use: sh, ps, or py"
),
ignore_agent_tools: bool = typer.Option(
False,

View File

@@ -38,7 +38,7 @@ from ._helpers import (
@integration_app.command("install")
def integration_install(
key: str = typer.Argument(help="Integration key to install (e.g. claude, copilot)"),
script: str | None = typer.Option(None, "--script", help="Script type: sh or ps (default: from init-options.json or platform default)"),
script: str | None = typer.Option(None, "--script", help="Script type: sh, ps, or py (default: from init-options.json or platform default)"),
force: bool = typer.Option(False, "--force", help="Allow multi-install when integrations are not declared safe"),
integration_options: str | None = typer.Option(None, "--integration-options", help='Options for the integration (e.g. --integration-options="--commands-dir .myagent/cmds")'),
):

View File

@@ -131,7 +131,7 @@ def _installed_presets_affecting_agent(project_root, agent_key: str) -> list[str
@integration_app.command("switch")
def integration_switch(
target: str = typer.Argument(help="Integration key to switch to"),
script: str | None = typer.Option(None, "--script", help="Script type: sh or ps (default: from init-options.json or platform default)"),
script: str | None = typer.Option(None, "--script", help="Script type: sh, ps, or py (default: from init-options.json or platform default)"),
force: bool = typer.Option(False, "--force", help="Force removal of modified files during uninstall of the previous integration"),
refresh_shared_infra: bool = typer.Option(False, "--refresh-shared-infra", help="Also overwrite shared infrastructure files even if you customized them (otherwise customizations are preserved)"),
integration_options: str | None = typer.Option(None, "--integration-options", help='Options for the target integration'),
@@ -425,7 +425,7 @@ def integration_switch(
def integration_upgrade(
key: str | None = typer.Argument(None, help="Integration key to upgrade (default: current integration)"),
force: bool = typer.Option(False, "--force", help="Force upgrade even if files are modified"),
script: str | None = typer.Option(None, "--script", help="Script type: sh or ps (default: from init-options.json or platform default)"),
script: str | None = typer.Option(None, "--script", help="Script type: sh, ps, or py (default: from init-options.json or platform default)"),
integration_options: str | None = typer.Option(None, "--integration-options", help="Options for the integration"),
):
"""Upgrade an integration by reinstalling with diff-aware file handling.

View File

@@ -14,6 +14,7 @@ Provides:
from __future__ import annotations
import os
import platform
import re
import shlex
import shutil
@@ -668,6 +669,46 @@ class IntegrationBase(ABC):
return name
return sys.executable or "python3"
@staticmethod
def build_python_invocation(
script_command: str, project_root: Path | None = None
) -> str:
"""Build a Python script command for the current platform shell."""
interpreter = IntegrationBase.resolve_python_interpreter(project_root)
if os.name == "nt" and not re.fullmatch(r"[A-Za-z0-9_./:\\-]+", interpreter):
quoted_interpreter = interpreter.replace("'", "''")
interpreter = f"& '{quoted_interpreter}'"
elif os.name != "nt":
interpreter = shlex.quote(interpreter)
return f"{interpreter} {script_command}"
@staticmethod
def select_script_variant(
requested: object, script_commands: dict[str, str]
) -> str:
"""Select the requested variant or a runnable platform fallback."""
if isinstance(requested, str) and requested in script_commands:
return requested
platform_variant = (
"ps" if platform.system().lower().startswith("win") else "sh"
)
secondary_variant = "sh" if platform_variant == "ps" else "ps"
fallbacks = (
(platform_variant, "py")
if requested == "py"
else (platform_variant, secondary_variant, "py")
)
for candidate in fallbacks:
if candidate in script_commands:
return candidate
available = ", ".join(sorted(script_commands)) or "none"
raise ValueError(
"No runnable script variant for this platform: "
f"requested {requested!r}; available: {available}"
)
@staticmethod
def _interpreter_runs(path: str) -> bool:
"""Return True when *path* executes as a Python interpreter.
@@ -702,7 +743,8 @@ class IntegrationBase(ABC):
"""Process a raw command template into agent-ready content.
Performs the same transformations as the release script:
1. Extract ``scripts.<script_type>`` value from YAML frontmatter
1. Select ``scripts.<script_type>`` from YAML frontmatter, falling
back to a runnable platform shell or Python variant when unavailable
2. Replace ``{SCRIPT}`` with the extracted script command
3. Strip ``scripts:`` section from frontmatter
4. Replace ``{ARGS}`` and ``$ARGUMENTS`` with *arg_placeholder*
@@ -711,37 +753,46 @@ class IntegrationBase(ABC):
7. Replace ``__SPECKIT_COMMAND_<NAME>__`` with invocation strings
"""
# 1. Extract script command from frontmatter
script_command = ""
script_pattern = re.compile(
rf"^\s*{re.escape(script_type)}:\s*(.+)$", re.MULTILINE
)
script_commands: dict[str, str] = {}
script_pattern = re.compile(r"^\s*([A-Za-z0-9_-]+):\s*(.+)$")
# Find the scripts: block
in_frontmatter = False
in_scripts = False
for line in content.splitlines():
if line.strip() == "scripts:":
if line == "---":
if in_frontmatter:
break
in_frontmatter = True
continue
if not in_frontmatter:
continue
if line == "scripts:":
in_scripts = True
continue
if in_scripts and line and not line[0].isspace():
in_scripts = False
break
if in_scripts:
m = script_pattern.match(line)
if m:
script_command = m.group(1).strip()
break
script_commands[m.group(1)] = m.group(2).strip()
selected_script_type = (
IntegrationBase.select_script_variant(script_type, script_commands)
if script_commands
else ""
)
script_command = script_commands.get(selected_script_type, "")
# 2. Replace {SCRIPT}
if script_command:
# For the Python script type, prefix the resolved interpreter so
# the command is portable (``.py`` files are not directly
# executable on Windows).
if script_type == "py":
interpreter = IntegrationBase.resolve_python_interpreter(project_root)
# Quote the interpreter if it contains whitespace (e.g. an
# absolute ``sys.executable`` path under Windows
# ``Program Files``) so it isn't split into multiple args.
if any(ch.isspace() for ch in interpreter):
interpreter = f'"{interpreter}"'
script_command = f"{interpreter} {script_command}"
if selected_script_type == "py":
script_command = IntegrationBase.build_python_invocation(
script_command, project_root
)
content = content.replace("{SCRIPT}", script_command)
# 3. Strip scripts: section from frontmatter

View File

@@ -402,8 +402,13 @@ def install_shared_infra(
# Track every shared path the current bundle produces so we can detect
# manifest entries the core no longer ships (stale-script cleanup, #3076).
seen_rels: set[str] = set()
scripts_scanned = False
variant_dir = {"sh": "bash", "py": "python"}.get(script_type, "powershell")
scanned_variant_dirs: set[str] = set()
shell_variant = "powershell" if os.name == "nt" else "bash"
variant_dirs = (
("python", shell_variant)
if script_type == "py"
else ("bash" if script_type == "sh" else "powershell",)
)
def _decide_overwrite(rel: str, dst: Path) -> tuple[bool, str | None]:
"""Return (write, bucket) where bucket is 'skip', 'preserved', or None."""
@@ -458,69 +463,69 @@ def install_shared_infra(
if scripts_src.is_dir():
dest_scripts = project_path / ".specify" / "scripts"
if _ensure_or_bucket_dir(dest_scripts):
variant_src = scripts_src / variant_dir
if variant_src.is_dir():
for variant_dir in variant_dirs:
variant_src = scripts_src / variant_dir
if not variant_src.is_dir():
continue
dest_variant = dest_scripts / variant_dir
if _ensure_or_bucket_dir(dest_variant):
for src_path in variant_src.rglob("*"):
if not src_path.is_file():
continue
# Python bytecode caches are local artifacts, not
# workflow scripts — never install them.
if "__pycache__" in src_path.parts:
continue
# Mark scanned only once a real source file is seen. An
# empty (or symlink-skipped) variant keeps this False, so
# stale-cleanup is skipped — otherwise it would treat every
# tracked script as obsolete and delete it. (The safety
# hinge is this flag, not ``seen_rels``, which also holds
# template paths populated later.)
scripts_scanned = True
if not _ensure_or_bucket_dir(dest_variant):
continue
for src_path in variant_src.rglob("*"):
if not src_path.is_file():
continue
# Python bytecode caches are local artifacts, not
# workflow scripts — never install them.
if "__pycache__" in src_path.parts:
continue
# Mark scanned only once a real source file is seen. An
# empty (or symlink-skipped) variant stays untracked, so
# stale-cleanup cannot treat its managed scripts as obsolete.
scanned_variant_dirs.add(variant_dir)
rel_path = src_path.relative_to(variant_src)
dst_path = dest_variant / rel_path
rel = dst_path.relative_to(project_path).as_posix()
seen_rels.add(rel)
if not _safe_dest_or_bucket(dst_path, rel, parent_must_exist=False):
continue
write, bucket = _decide_overwrite(rel, dst_path)
if not write:
if bucket == "preserved":
preserved_user_files.append(rel)
else:
skipped_files.append(rel)
# Record the existing-on-disk file in the manifest so a
# fresh manifest run against an already-populated
# ``.specify/`` tree does not silently drop it (#2107).
# ``prior_hashes`` is the function-scope snapshot taken
# at entry, so this membership check is O(1) and avoids
# the repeated ``dict(self._files)`` copy that
# ``manifest.files`` performs on every access.
if dst_path.is_file() and rel not in prior_hashes:
try:
manifest.record_existing(rel, recovered=True)
except (OSError, ValueError) as exc:
# Tolerate races / permission issues / non-file
# collisions so one weird path does not abort
# the whole install.
console.print(
f"[yellow]⚠[/yellow] could not record {rel} in manifest: {exc}"
)
continue
rel_path = src_path.relative_to(variant_src)
dst_path = dest_variant / rel_path
rel = dst_path.relative_to(project_path).as_posix()
seen_rels.add(rel)
if not _safe_dest_or_bucket(dst_path, rel, parent_must_exist=False):
continue
write, bucket = _decide_overwrite(rel, dst_path)
if not write:
if bucket == "preserved":
preserved_user_files.append(rel)
else:
skipped_files.append(rel)
# Record the existing-on-disk file in the manifest so a
# fresh manifest run against an already-populated
# ``.specify/`` tree does not silently drop it (#2107).
# ``prior_hashes`` is the function-scope snapshot taken
# at entry, so this membership check is O(1) and avoids
# the repeated ``dict(self._files)`` copy that
# ``manifest.files`` performs on every access.
if dst_path.is_file() and rel not in prior_hashes:
try:
manifest.record_existing(rel, recovered=True)
except (OSError, ValueError) as exc:
# Tolerate races / permission issues / non-file
# collisions so one weird path does not abort
# the whole install.
console.print(
f"[yellow]⚠[/yellow] could not record {rel} in manifest: {exc}"
)
continue
if not _ensure_or_bucket_dir(dst_path.parent):
continue
content = src_path.read_text(encoding="utf-8")
content = IntegrationBase.resolve_command_refs(content, invoke_separator)
content = _resolve_dynamic_command_refs(content, invoke_separator)
planned_copies.append(
(
dst_path,
rel,
content.encode("utf-8"),
src_path.stat().st_mode & 0o777,
)
if not _ensure_or_bucket_dir(dst_path.parent):
continue
content = src_path.read_text(encoding="utf-8")
content = IntegrationBase.resolve_command_refs(content, invoke_separator)
content = _resolve_dynamic_command_refs(content, invoke_separator)
planned_copies.append(
(
dst_path,
rel,
content.encode("utf-8"),
src_path.stat().st_mode & 0o777,
)
)
templates_src = shared_templates_source(core_pack=core_pack, repo_root=repo_root)
if templates_src.is_dir():
@@ -618,14 +623,16 @@ def install_shared_infra(
# agent-context extension. Left behind, such an orphan can crash when it
# sources a refreshed ``common.sh`` (#3076). Only run when the script source
# was actually scanned (so a missing/empty source never triggers mass
# deletion), scoped to the active variant, and only for *managed* copies —
# deletion), scoped to the selected variants, and only for *managed* copies —
# a user-customized file (hash diverges), a symlink, or a recovered entry is
# preserved by ``_is_managed``.
if scripts_scanned:
if scanned_variant_dirs:
stale_removed: list[str] = []
script_prefix = f".specify/scripts/{variant_dir}/"
script_prefixes = tuple(
f".specify/scripts/{variant_dir}/" for variant_dir in scanned_variant_dirs
)
for rel in list(prior_hashes):
if rel in seen_rels or not rel.startswith(script_prefix):
if rel in seen_rels or not rel.startswith(script_prefixes):
continue
# Guard corrupted/hand-edited manifest keys BEFORE any filesystem
# access: absolute, ``..``, or (on Windows) drive-relative keys such