Compare commits

..

30 Commits

Author SHA1 Message Date
Ali jawwad
d1e86f6382 fix(workflows): fail a gate whose on_reject is not abort/skip/retry (#3888)
execute() reads `on_reject = config.get("on_reject", "abort")` and, in the
reject branch, handles only "abort" and "retry" before falling through to
its `# on_reject == "skip"` case. So any other value makes a REJECTED gate
report COMPLETED and the run walks straight past the review the gate
exists to enforce:

  on_reject='abort'  -> failed     "Gate rejected by user at step 'g'"
  on_reject='retry'  -> paused
  on_reject='skip'   -> completed  (by design)
  on_reject='Abort'  -> completed  <-- rejection silently discarded
  on_reject='fail'   -> completed  <-- same
  on_reject='stop'   -> completed  <-- same
  on_reject=None     -> completed  <-- same
  on_reject=5        -> completed  <-- same

Reachable by a capitalisation slip, a guessed verb, a non-string, or a
bare `on_reject:` — note `config.get(k, default)` does NOT substitute the
default for an explicit YAML null.

`validate` already rejects anything outside abort/skip/retry, but the
engine does not auto-validate before execute(). Fail loudly instead,
mirroring the `options` and `verdict_input` guards in the same method, and
placed before the non-TTY short-circuit so it surfaces in CI too.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 12:45:15 -05:00
Marsel Safin
400ad01f12 fix(presets): validate required manifest mappings (#3898)
Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-31 12:15:18 -05:00
Quratulain-bilal
642fa56c0a fix: eliminate TOCTOU race in zip packaging (#3855)
* fix: eliminate TOCTOU race in zip packaging

Open file once and derive both stat info and content from the same file
descriptor to prevent race conditions where the file is modified between
stat() and read_bytes() calls.

* test: add regression test for TOCTOU stat/read consistency in packager

The old implementation called file_path.stat() then file_path.read_bytes()
as separate syscalls. The fix opens the file once and uses os.fstat() +
fh.read() on the same handle. This test verifies the archived bytes and
mode are consistent with the opened file descriptor.
2026-07-31 12:05:47 -05:00
Ali jawwad
521020bc3a fix(workflows): fail a fan-in step whose output is not a mapping (#3887)
execute() did:

    output_config = config.get("output") or {}
    if not isinstance(output_config, dict):
        output_config = {}

so every non-mapping `output` was silently discarded and the step still
returned COMPLETED — every declared aggregation key vanished, and
downstream `{{ steps.<id>.output.<key> }}` resolved to None and
interpolated as an empty string:

  output=[]       -> completed, error=None
  output=False    -> completed, error=None
  output=0        -> completed, error=None
  output=''       -> completed, error=None
  output=['a']    -> completed, error=None
  output='oops'   -> completed, error=None
  output=5        -> completed, error=None

`validate` already rejects this and its comment names the flaw exactly:
"execute() silently coerces a non-mapping output to {}, so the author's
declared aggregation keys would vanish with no error." The engine does not
auto-validate before execute(), so on an unvalidated run that is what
happened — and `x or {}` masked the falsy shapes before the isinstance
check even ran.

Fail loudly with validate()'s own message, mirroring the `wait_for` guard
in the same method. An explicit `output:` (YAML null) stays valid.

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-31 12:04:02 -05:00
Marsel Safin
14e82353cb fix(workflows): refetch non-UTF-8 catalog caches (#3901)
Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-31 12:02:49 -05:00
Marsel Safin
1831fffde6 fix(bundler): wrap local catalog decode failures (#3902)
Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-31 11:52:29 -05:00
Manfred Riem
ba7ae79c66 Add --extension flag to specify init for opting into extensions at init time (#3914)
* Add --extension flag to specify init for installing extensions at init time

Adds a repeatable --extension flag to `specify init` so users can opt into
extensions (bundled name, local path, or HTTPS URL) during initialization,
without a separate `specify extension add` step.

- New `_install_extension_during_init` helper in commands/init.py that
  auto-detects source type (URL / local path / bundled name / catalog) and
  installs via ExtensionManager. Failures are non-fatal and recorded in the
  tracker without aborting init.
- Extension tracker steps are pre-registered before the Live context and run
  after preset install, before finalize.
- Five new tests in TestExtensionFlag covering bundled name, multiple
  extensions, local absolute path, unknown extension (graceful error), and
  combination with --preset.

Rebased onto upstream/main and adapted to the refactored init command
(moved to src/specify_cli/commands/init.py) from stale PR #2396.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Address review: reuse hardened downloader, refresh events, escape labels, fix bundler call

Responds to review feedback on #3914 and fixes CI (pytest bundler failure).

- Extract shared `install_extension_from_url` helper in extensions/_commands.py
  that reuses the authenticated, redirect-guarded, bounded (50 MiB) download
  and TOCTOU-safe transient archive used by `extension add --from`. Both
  `extension add --from` and `specify init --extension <url>` now go through
  this single downloader instead of a second raw urlopen path.
- Refresh native event configuration once after successful extension installs
  during init (mirrors `_refresh_events_and_warn` in the add path) so an
  extension declaring `events:` has its hooks activated.
- Escape user-controlled extension specs and error text before interpolating
  them into StepTracker labels (Rich markup injection).
- Pass `extensions=None` from bundler's `_run_init` so the init callback no
  longer receives the typer OptionInfo sentinel ('OptionInfo' object is not
  iterable), which broke `test_install_initializes_uninitialized_project`.
- Add init URL coverage in TestExtensionFlag: non-HTTPS rejection and a
  successful HTTPS ZIP install with download-cache cleanup assertion.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Add default-deny trust confirmation for URL extension installs at init

URL-based --extension installs now require explicit trust, matching the
`extension add --from` posture. Interactive sessions show an "Untrusted
Source" panel and prompt (default no); non-interactive sessions deny by
default unless --trust-extension-urls is passed. Trust is resolved before
the Live display since the prompt can't be answered under the spinner.

- Add --trust-extension-urls option and _ext_spec_is_url /
  _confirm_extension_url_trust helpers
- Skip (not abort) unconfirmed URL extensions, consistent with other
  non-fatal extension failures
- Pass trust_extension_urls=False from the bundler init callback
- Add tests for deny-by-default, interactive confirm, and trusted install

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8bc6802d-81b8-48f4-8f60-cba3aebc3bb3

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8bc6802d-81b8-48f4-8f60-cba3aebc3bb3
2026-07-31 11:15:25 -05:00
Quratulain-bilal
36cb7e3c11 fix: bound response reads in extension catalog and download (#3775)
* fix: bound response reads in extension catalog and download

Replace unbounded 
esponse.read() calls with 
ead_response_limited()
from _download_security in extensions/__init__.py to prevent denial-
of-service via oversized catalog or extension archive responses.

Three call sites fixed:
- _fetch_single_catalog JSON read (catalog metadata)
- _fetch_catalog JSON read (legacy path)
- download_extension ZIP read (binary download)

All existing mock tests updated to use side_effect with BytesIO.read
instead of 
eturn_value, ensuring compatibility with the chunked read
loop in 
ead_response_limited.

Two regression tests added:
- test_oversized_catalog_response_rejected
- test_oversized_extension_download_rejected

* fix: remove .decode utf-8 to preserve bytes for json.loads

json.loads accepts bytes directly. Removing .decode maintains
compatibility with BOM-bearing or UTF-16/32 catalogs.
2026-07-31 10:02:25 -05:00
Noor ul ain
cf71d00dfe fix(workflows): reject a retry gate whose verdict enum forbids the reset value (#3912)
A gate with `on_reject: retry` consumes a bound reject verdict before
pausing by resetting the named input to `""` (documented behaviour, so a
later resume prompts again). Every `resume()` re-resolves the persisted
inputs through `_coerce_input`.

Those two rules collide when the bound input declares an `enum` that does
not list `""`. The reset writes a value the input's own enum forbids, and
the run wedges:

    inputs:
      spec_verdict:
        type: string
        enum: [approve, reject]
    steps:
      - id: review
        type: gate
        options: [approve, reject]
        on_reject: retry
        verdict_input: spec_verdict

    $ specify workflow run wf --input spec_verdict=reject
    Status: paused
    $ specify workflow resume <run_id> --input note=b
    Error: Input 'spec_verdict' value '' not in allowed values:
           ['approve', 'reject'].

The workflow validates clean and the first run looks fine, so the failure
only appears at the second resume. It is also unrecoverable in practice:
`_resolve_inputs` re-coerces the whole persisted map, so *any* resume that
supplies an input dies on the stored `""`. Only a resume with no inputs at
all still works -- and that is precisely the call that cannot deliver a new
verdict, which is the one thing the retry cycle exists to allow.

Extend the existing `verdict_input` cross-check (which already confirms the
name is declared) to also require that a retry-bound input's `enum` admits
the reset sentinel, and report it with a fix hint. To do that, thread the
input *definitions* through `_validate_steps` instead of just their names.

Rejected the alternative of popping the key instead of writing `""`: that
lets the input's `default` flow back in on the next resume, so a gate the
user just rejected would silently auto-approve.

Docs: note the `enum` requirement next to the reset behaviour it follows
from.

Adds 4 validation tests for the new guard plus a characterization test that
drives the engine directly to pin the wedge it prevents.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Assisted-by: Claude Opus 5 (1M context)
2026-07-31 09:53:25 -05:00
Manfred Riem
7f40c82945 chore: release 0.15.1, begin 0.15.2.dev0 development (#3913)
* chore: bump version to 0.15.1

* chore: begin 0.15.2.dev0 development

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-31 08:38:33 -05:00
Noor ul ain
184de79749 fix: escape Rich markup in workflow resolve output (#3879)
`workflow resolve` printed two lines through `console.print`, which has
Rich markup enabled, without escaping:

1. The layer tier was wrapped in literal brackets:
   `f"  • [{layer.tier}] {layer.source} ..."`. Rich parsed `[base]`
   and `[project-overlay]` as style tags, so the tier label was swallowed
   on *every* invocation -- no untrusted input required. The column has
   never rendered.

2. Step attribution interpolated `composed.step_id` raw. Step IDs come
   from base-workflow / overlay YAML and are only validated against `:`
   (see `_parse_edit`), so brackets pass validation. A balanced
   `[stuff]` is silently swallowed; an unbalanced `[/red]` raises
   `rich.errors.MarkupError`, producing an uncaught traceback and exit 1
   -- the workflow cannot be inspected at all.

Route the interpolated fields through `rich.markup.escape` and escape
the literal tier bracket as `\[`, matching the existing pattern in
`workflow info`'s step graph and `workflow_list`'s `\[disabled]`.

Only display is affected; the returned payload was already unescaped and
is unchanged.

Adds 3 regression tests, all of which fail without the fix: the tier
label renders, and a step ID survives both the swallowing and the
crashing markup cases.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Assisted-by: Claude Opus 5 (1M context)
2026-07-31 08:36:44 -05:00
dependabot[bot]
d82c915f9f chore(deps): bump actions/stale from 10.4.0 to 11.0.0 (#3877)
Bumps [actions/stale](https://github.com/actions/stale) from 10.4.0 to 11.0.0.
- [Release notes](https://github.com/actions/stale/releases)
- [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md)
- [Commits](1e223db275...4391f3da66)

---
updated-dependencies:
- dependency-name: actions/stale
  dependency-version: 11.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-31 07:27:08 -05:00
dependabot[bot]
acd8b801fd chore(deps): bump actions/setup-python from 6.3.0 to 7.0.0 (#3876)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 6.3.0 to 7.0.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](ece7cb06ca...5fda3b95a4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-version: 7.0.0
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-07-31 07:25:00 -05:00
Manfred Riem
a3e183d069 feat: support tar archives for installs (#3874)
* feat: support tar archives for installs

Add secure .tar.gz and .tgz parity with ZIP installation for extensions, presets, and workflows, including full workflow package preservation.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd

* chore: clean rebased archive imports

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd

* fix: preserve hardened archive install behavior

Keep malformed ZIP diagnostics, filesystem-independent manifest selection, and reserved workflow overlays consistent after adding generic archive support.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd

* fix: extract staged workflow archives by descriptor

Avoid reopening a held staging path on Windows while retaining authoritative-inode archive validation.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd

* fix: extract catalog archives from verified bytes

Use the already bounded and SHA-verified response bytes directly so Windows file-sharing semantics cannot affect archive detection.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd

* fix: address archive install review feedback

Preserve forced preset reinstalls, sniff suffixless workflow archives without weakening YAML limits, and restore prior workflow packages before failed-install cleanup.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd

---------

Copilot-Session: bd07c6b3-f1f9-484c-869a-94d8fef970dd
2026-07-31 07:23:00 -05:00
Quratulain-bilal
6bf51e728a fix: eliminate TOCTOU race in file unlink calls (#3819) 2026-07-31 07:19:25 -05:00
Ali jawwad
5e2f9bcd9b fix(scripts): tolerate an unusable integration.json in the Python helper (#3785)
* fix(scripts): tolerate an unusable integration.json in the Python helper

`get_invoke_separator()` in scripts/python/common.py indexed the parsed JSON
directly, so two shapes escaped its `except (OSError, json.JSONDecodeError)`
while BOTH of its twins fall back to "." for them:

  * A non-mapping top level is valid JSON, so JSONDecodeError never fires and
    `state.get(...)` raised AttributeError.
  * A non-UTF-8 file raises UnicodeDecodeError -- a ValueError, not an OSError.
    Realistic on Windows, where PowerShell 5.1's Out-File/`>` default to UTF-16.

Measured on main -- 6 of 7 inputs crashed the Python helper while bash and
PowerShell 5.1 returned "." for every one:

    input                             python        bash   pwsh 5.1
    {"default_integration":"forge"}   '.'           .      .
    []                                AttributeError .      .
    "forge"                           AttributeError .      .
    42                                AttributeError .      .
    null                              AttributeError .      .
    UTF-16 file                       UnicodeDecodeError .  .

Split the parse out of the lookup, complete the exception tuple, and guard the
top-level shape -- matching `read_feature_json_feature_directory` in this same
module, which already does exactly this. The hyphen-separator feature is
unchanged (regression test included).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(scripts): point the parity comment at the sibling above, not below

read_feature_json_feature_directory is defined at line 81, above
get_invoke_separator, so "below" sent maintainers the wrong way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 14:22:38 -05:00
Ali jawwad
e4318a3d1a fix(catalogs): validate the port in the shared catalog-URL validator, like its mirrors do (#3804)
* fix(catalogs): validate the port in the shared catalog-URL validator

`CatalogStackBase._validate_catalog_url()` reads `parsed.hostname` inside
its `try/except ValueError` but never reads `parsed.port`. `urlparse()` and
`.hostname` do not perform port validation — only `.port` does — so a
catalog URL with a non-numeric or out-of-range port passes validation.

Every implementation that documents itself as mirroring this function
already reads `.port` inside the same try: workflows/catalog.py (4 sites),
bundler/services/adapters.py (2), bundler/commands_impl/catalog_config.py,
and commands/bundle/__init__.py. The shared base — inherited by
ExtensionCatalog and IntegrationCatalog — is the only one without it.

The accepted URL then escapes as a raw `http.client.InvalidURL`, which is
neither `urllib.error.URLError` nor `json.JSONDecodeError` (the only two
the fetcher converts), so it surfaces as an unhandled traceback rather
than the validator's normal error.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(catalogs): describe both bad-port failure modes accurately

The comment attributed both malformed-port cases to
http.client.InvalidURL. Only a non-numeric port raises that (when the
connection object is built); an out-of-range port constructs fine and
fails later in the socket layer. Measured:

  example.invalid:notaport -> http.client.InvalidURL: nonnumeric port
  example.invalid:65536    -> HTTPSConnection() OK, connect() fails

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 13:41:17 -05:00
Manfred Riem
43a54bf2d6 feat(presets): add opt-in constitution-sync preset (#3873)
* feat(presets): add opt-in constitution-sync preset

Follow-up to #3790, which removed the consistency-propagation pass from the
core /constitution command in favor of runtime resolution. Teams that treat
materialized plan/spec/tasks templates as reviewed, committed artifacts lost
the auto-sync of amended constitutional guidance on a non-forced upgrade.

Add a bundled, opt-in `constitution-sync` preset that restores that behavior
via a wrap-strategy override of speckit.constitution (composes on {CORE_TEMPLATE}
so it stays forward-compatible). It only writes into the project's own
.specify/templates scaffolds and installed command files, never into
stack-owned template layers.

- presets/constitution-sync/: preset.yml (requires >=0.14.4), wrap command,
  README documenting the tension between auto-propagation and the resolution stack
- presets/catalog.json: bundled entry
- docs/upgrade.md: document the 0.14.4 behavior change and the opt-in
- tests/test_presets.py: structural + composition coverage (TestConstitutionSyncPreset)

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: afa7c1d2-147b-4f62-a6fc-a2cc824cfa3e

* fix(presets): ship constitution-sync in wheel, clarify scope guard, assert composition

Address review feedback on #3873:

- pyproject.toml: force-include presets/constitution-sync into the wheel's
  core_pack so `_locate_bundled_preset` resolves it in a released install; the
  bundled advertisement was otherwise unshippable.
- tests/contract/test_wheel_bundled_presets.py: new contract test asserting
  every bundled preset in presets/catalog.json is force-included (guards lean too).
- commands/speckit.constitution.md: explicitly state the propagation section
  supersedes the core Scope Guard, which otherwise says dependent templates are
  not modified here.
- tests/test_presets.py: assert resolve_content substitutes {CORE_TEMPLATE} and
  the effective command embeds both the core body and the sync pass.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: afa7c1d2-147b-4f62-a6fc-a2cc824cfa3e

* test(presets): parse frontmatter as YAML in constitution-sync wrapper test

Address review feedback on #3873: assert `strategy: wrap` structurally by
parsing the Markdown frontmatter as YAML (instead of a substring match that
could false-positive on body text), and assert {CORE_TEMPLATE} in the body
section only.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: afa7c1d2-147b-4f62-a6fc-a2cc824cfa3e

* docs(presets): reframe constitution-sync README around behavior and caveats

Rework the preset's user-facing docs to describe what it does, what it does
not do, and the caveats you take on — rather than leading with version/origin
history. The preset stack is the project's forward direction, so the README no
longer positions this as "restoring pre-0.14.4 behavior."

Also make the edit-in-place vs. composition conflict explicit and consistent
across the wrapper command and docs: propagation into command files/templates
that are provided or wrapped by a preset/extension is clobbered on stack
reconciliation (integration use/upgrade, preset/extension install/remove), so
the wrapper restricts propagation to project-local artifacts the team owns.

- README: forward-looking "What it does / does not do / When to use / Caveats"
- speckit.constitution.md: step 4 no longer hand-edits composed command files;
  closing caveat covers command files too
- docs/upgrade.md: note the composition-model conflict in the opt-in section
- tests: assert the updated closing-caveat wording

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: afa7c1d2-147b-4f62-a6fc-a2cc824cfa3e

* docs(presets): tweak constitution-sync README default-behavior wording

Phrase the default-behavior note as "the current version of Spec Kit" and
rewrap the opening paragraph.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: afa7c1d2-147b-4f62-a6fc-a2cc824cfa3e

* docs(presets): keep emphasis spans on one line in constitution-sync README

Avoid **bold** spans broken across soft line breaks (runtime resolution,
reviewed committed artifacts) so they render consistently.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: afa7c1d2-147b-4f62-a6fc-a2cc824cfa3e

* docs(presets): refocus constitution-sync README on what it restores

Reframe the intro around what the preset restores and what the user opts
into, rather than describing current Spec Kit default behavior. Be honest
that propagation was removed deliberately (duplicates the source of truth,
fights composition) and this preset knowingly reintroduces it and its
tradeoffs. Minor flow fixes (comma splice, terse bullet).

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: afa7c1d2-147b-4f62-a6fc-a2cc824cfa3e

* docs(upgrade): de-pin version from /constitution behavior-change section

The upgrade guide always describes the latest version, so hard-pinning
"0.14.4" in the heading and "Starting in 0.14.4" in the body added no
value. Keep the #3790 provenance link and the "no longer propagates"
framing; the machine-readable version gate stays in preset.yml.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: afa7c1d2-147b-4f62-a6fc-a2cc824cfa3e

* docs(upgrade): clarify non-breaking nuance and presets direction

Note that the /constitution scope change is only noticeable if you relied
on the old edit-in-place behavior, and add the forward-looking framing:
presets and extensions — not in-place file edits — are how Spec Kit now
governs, versions, and audits shared assets across repositories.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, supervised)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: afa7c1d2-147b-4f62-a6fc-a2cc824cfa3e

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: afa7c1d2-147b-4f62-a6fc-a2cc824cfa3e
2026-07-30 13:36:13 -05:00
Marsel Safin
515d2810fb fix: reject non-object workflow caches (#3860)
* fix: reject non-object workflow caches

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* test: cover non-object stale workflow cache

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-30 13:25:21 -05:00
Manfred Riem
6577ffc92b Harden extension URL download cache against symlink and junction races (#3869)
* fix(extensions): harden URL download cache

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8f71e02a-bc64-4593-b305-2554debe96f6

* fix(extensions): retain secure archive descriptor

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 8f71e02a-bc64-4593-b305-2554debe96f6

* Harden extension URL cache anchor opens

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8f71e02a-bc64-4593-b305-2554debe96f6

* Use descriptor-safe mkdir for cache components

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8f71e02a-bc64-4593-b305-2554debe96f6

* Harden extension URL download cache

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8f71e02a-bc64-4593-b305-2554debe96f6

* Align extension manifest regression expectation

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8f71e02a-bc64-4593-b305-2554debe96f6

* Make download cache leaf anonymous to remove cleanup TOCTOU

Address review: the best-effort cleanup walk re-derived the downloads
directory by path, so a cache ancestor swapped after the archive was
opened could redirect os.unlink to a replacement leaf, and it silently
no-op'd (failing open) on platforms without descriptor-relative unlink.

_safe_open_download_zip now unlinks the exclusively-created leaf
immediately via the same directory descriptor, returning an fd backed by
an anonymous inode. Installation already consumes that descriptor through
archive_file, so the on-disk pathname is never reopened and no cleanup
walk is needed. The capability gate additionally requires os.unlink in
os.supports_dir_fd, so unsupported platforms fail closed. Removed the
now-unused _safe_unlink_download_zip helper and its cleanup finally, and
updated the tests accordingly.

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8f71e02a-bc64-4593-b305-2554debe96f6

* Fix Windows test matrix for cache hardening tests

The hardened cache primitives fail closed on platforms without dir_fd/
O_NOFOLLOW, so on the windows-latest matrix several tests errored instead
of exercising POSIX behavior:

- test_symlinked_cache_ancestor_is_refused and
  test_cache_ancestor_resolving_outside_project_is_refused called
  _validate_safe_cache_dir directly and expected typer.Exit, but on
  Windows it raises NotImplementedError first. Guard both with
  _require_secure_dir_fd() so they skip where the primitive is unavailable.
- test_safe_open_fails_closed_without_atomic_platform_support built its
  download dir via _validate_safe_cache_dir, which itself fails closed on
  Windows; construct the directory directly so the assertion targets
  _safe_open_download_zip's platform gate in isolation.
- The _open_test_download_zip stand-in unlinked a still-open file, which
  raises PermissionError on Windows. Use O_TEMPORARY there (auto-delete on
  close) and keep immediate unlink on POSIX.

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8f71e02a-bc64-4593-b305-2554debe96f6

* Support Windows in extension URL download-cache hardening

Replace the fail-closed NotImplementedError on platforms lacking dir_fd
with a portable, still-hardened download path so `specify extension add
--from <url>` works on Windows instead of rejecting the install.

- `_validate_safe_cache_dir` now dispatches to a POSIX dir_fd + O_NOFOLLOW
  walk when available, and otherwise a portable path-wise walk that rejects
  symlink/junction components before and after each mkdir and requires every
  component to resolve back under the project root.
- `_safe_open_download_zip` keeps the POSIX anonymous-inode create/unlink and
  adds a portable leaf create using O_EXCL + O_TEMPORARY (auto-delete on
  close) plus a post-open fstat/lstat inode-identity check to detect a leaf
  swapped underneath us. Installation still consumes only the open
  descriptor, so the cache pathname is never reopened.
- Detect the symlink-refusal case via errno (ELOOP/ENOTDIR/EMLINK) instead of
  FileExistsError, and add O_CLOEXEC to the descriptor-walk opens.
- Drop the now-unreachable NotImplementedError handling in the --from branch.
- Tests: cover the portable path (success, symlinked-leaf refusal, symlinked
  ancestor refusal, full --from install) and keep the POSIX-only cases guarded.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8f71e02a-bc64-4593-b305-2554debe96f6

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 8f71e02a-bc64-4593-b305-2554debe96f6
2026-07-30 11:59:50 -05:00
Marsel Safin
0f3f2aa45e fix: escape workflow step metadata (#3863)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-30 11:10:22 -05:00
github-actions[bot]
81bf741b92 [bug-fix] Fix bundle-update-force-mislead: add refresh() to DefaultPrimitiveInstaller (#3452)
* Fix bundle-update-force-mislead: add refresh() to DefaultPrimitiveInstaller

Apply the remediation from the bug assessment on issue #3424.

DefaultPrimitiveInstaller lacked a refresh() method, causing
_refresh_component() to fall back to install(), which calls
ExtensionManager.install_from_directory() with force=False. This
raised ExtensionError with a leaked --force hint that bundle update
does not support, leaving users with no valid recovery path.

Fix: add refresh() to each kind manager (ExtensionKindManager and
PresetKindManager delegate to _do_install(force=True); WorkflowKindManager
and StepKindManager delegate to install() as their callables are
idempotent). DefaultPrimitiveInstaller.refresh() dispatches to the kind
manager's refresh(). PresetManager.install_from_directory() and
install_from_zip() gain a force parameter that removes the existing
preset before reinstalling, mirroring ExtensionManager's force semantics.

Refs #3424

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address review feedback on primitives.py and test_bundler_primitives.py

- Replace ... with pass in _KindManager Protocol method stubs
- Conditionally pass force= keyword only when force=True in _PresetKindManager
- Fix _StepKindManager.refresh() to remove step before re-installing
- Rename test to reflect actual assertion (refresh succeeds + force=True)
- Remove duplicate install_bundle import

Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)

* fix: add missing role/effective_integration to InstallPlan in _plan() and remove redundant import

- Remove duplicate `DefaultPrimitiveInstaller` import inside test body
  (already imported at module scope on line 15)
- Add required `role` and `effective_integration` fields to `InstallPlan`
  constructor in `_plan()` helper to prevent TypeError at runtime

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

* fix: address latest PR review comments

Assisted-by: GitHub Copilot (model: gpt-5.6-terra, autonomous)

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-07-30 10:59:53 -05:00
Quratulain-bilal
4803a22b33 fix: use chunked read for extension manifest hash (#3841)
Replace unbounded f.read() with chunked iteration to prevent excessive
memory allocation on large or corrupted manifest files. Matches the
pattern used in integrations/manifest.py _sha256().
2026-07-30 09:22:35 -05:00
Marsel Safin
e916fd1b3b fix: preserve unreadable event config files (#3861)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-30 09:08:27 -05:00
Noor ul ain
296fdf2ee7 fix(scripts): use a .NET Framework-safe trim in the PowerShell init-dir resolver (#3872)
`Resolve-SpecifyInitDir` normalized the resolved path with
`[System.IO.Path]::TrimEndingDirectorySeparator`, which is .NET Core only.
Windows PowerShell 5.1 runs on .NET Framework, so on every 5.1 host the call
throws at that line and root resolution fails before the requested command runs:

    $ $env:SPECIFY_INIT_DIR = "C:\repo\web"
    $ .specify\scripts\powershell\check-prerequisites.ps1 -Json
    check-prerequisites.ps1 : Method invocation failed because
    [System.IO.Path] does not contain a method named
    'TrimEndingDirectorySeparator'.

The same file already documents this exact incompatibility and avoids it
correctly in `Get-FeaturePathsEnv` (~150 lines below), which uses `TrimEnd`
with a comment naming `TrimEndingDirectorySeparator` as .NET Core only.

Worse than a clean failure when the resolver is called directly: the throw is
non-terminating, so `$initRoot` stays `$null` and the very next `Join-Path`
throws too, `Get-RepoRoot` returns `$null`, and the shell exits **0**. A caller
that checks the exit code sees success with an empty root.

Switched to the `TrimEnd('/', '\')` the file already endorses. Note the obvious
swap is not quite enough on its own: a bare `TrimEnd` turns `C:\` into `C:`,
which is not the drive root but a drive-relative reference that later path APIs
re-resolve against the *current directory* — so validation would probe the wrong
tree and, from a cwd that happens to contain `.specify/`, could silently accept
`C:` as the project root. A `GetPathRoot` length check keeps a path that is its
own root intact. Both `GetPathRoot` and `TrimEnd` exist on .NET Framework.

Trailing-separator trimming (the reason the call was there — bash's `cd && pwd`
never yields one, so the two resolvers must agree) is unchanged, as are all
error paths and messages.

Tests in `tests/test_init_dir.py`:

- A static check that no shipped `.ps1` calls a .NET Core-only
  `[System.IO.Path]` member (`TrimEndingDirectorySeparator`,
  `EndsInDirectorySeparator`, `GetRelativePath`, `Join`). This one runs on all
  platforms and is what actually guards CI: the matrix runs the PowerShell
  tests under `pwsh`, which is .NET Core, so a .NET Framework-only regression
  is otherwise invisible to it. Anchored to the `Path` type so `[string]::Join`
  is not flagged.
- Two runtime tests under `powershell.exe` specifically (never `pwsh`),
  covering resolution and trailing-separator parity.
- A drive-root test asserting the reported root survives the trim intact.

Test-the-test: reverting the source change fails all four (the runtime pair
with the `does not contain a method named` throw, the static check by locating
the call). Applying only the naive `TrimEnd` fails the drive-root test, which
reports `C:` instead of `C:\`. Verified on Windows PowerShell 5.1.19041.6456,
including the previously-crashing `check-prerequisites.ps1 -Json` end to end.

Also fixes six pre-existing `test_ps_*` failures on 5.1-only hosts, which were
this bug rather than test-harness issues.

Fixes #3749

Assisted-by: Claude Code (model: claude-opus-5, under direct human supervision)

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-30 07:51:38 -05:00
github-actions[bot]
fdfc5ae330 Add ContextForge MCP extension to community catalog (#3487)
Add contextforge-mcp extension submitted by @capatinore to:
- extensions/catalog.community.json (alphabetical order)
- docs/community/extensions.md community extensions table

Closes #3456

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-30 07:49:40 -05:00
Marsel Safin
227b4f5e11 fix: normalize non-UTF-8 integration manifests (#3862)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-07-30 07:46:23 -05:00
Markus Wondrak
675143591d feat: bind gate verdict to workflow input via verdict_input (#3725)
* feat: bind gate verdict to workflow input via verdict_input

Add an optional `verdict_input` field to gate steps that lets an
external system supply a verdict through a declared workflow input
instead of an interactive TTY prompt.

When the referenced input carries a non-empty string value that
matches one of the gate's `options` (case-insensitive), the gate
auto-decides, records the matched spelling in `output.choice`, and
applies the existing `on_reject` / abort / skip / retry semantics.
If the value is present but does not match an option, or is a
non-string, the gate fails immediately with a clear error message.
When the input is absent, null, or empty, the gate falls back to
today's TTY-prompt-or-pause behaviour unchanged.

The engine now persists `result.error` alongside each step's status
and output so that failed-step error messages survive across runs.
The CLI (`workflow run` and `workflow resume`) surfaces these
persisted errors after a failed or aborted run. `validate_workflow`
cross-references `verdict_input` against the workflow's declared
inputs block and reports an error for undeclared names, consistent
with the existing `wait_for` id cross-check.

Closes discussion: https://github.com/github/spec-kit/discussions/3717

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* Fix workflow JSON error payloads

Include persisted step errors in _workflow_run_payload so workflow run/resume/status --json all surface failure reasons consistently. Add JSON-path tests for failed and successful runs.

Assisted-by: GitHub Copilot (model: gpt-5.3-codex, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Potential fix for pull request finding

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

* fix(workflows): reject verdict inputs in fan-out

Fan-out items share workflow inputs and cannot safely consume a bound gate verdict. Reject verdict_input bindings during validation and at runtime while preserving unbound gates.

Assisted-by: GitHub Copilot (model: GPT-5.6 Sol, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: update workflow command handling

Assisted-by: GitHub Copilot (model: gpt-5.3-codex, autonomous)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Markus <markus@example.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-07-30 07:39:49 -05:00
Manfred Riem
99cd5e21b1 docs: use absolute image URLs in README for PyPI rendering (#3867)
Relative image paths do not render on the PyPI project page. Convert the
remaining logo and video-header image references to absolute
raw.githubusercontent.com URLs so they display correctly on
https://pypi.org/project/specify-cli/ while continuing to render on GitHub.

Addresses the rendering gap noted in github/spec-kit#2908.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e8a8563a-328e-43a4-8eb7-ff381f912161
2026-07-30 07:08:41 -05:00
Manfred Riem
edc1699481 chore: release 0.15.0, begin 0.15.1.dev0 development (#3871)
* chore: bump version to 0.15.0

* chore: begin 0.15.1.dev0 development

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-07-30 07:01:41 -05:00
53 changed files with 6112 additions and 366 deletions

View File

@@ -35,7 +35,7 @@ jobs:
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.13"

View File

@@ -27,7 +27,7 @@ jobs:
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
@@ -58,7 +58,7 @@ jobs:
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}

View File

@@ -14,7 +14,7 @@ jobs:
stale:
runs-on: ubuntu-latest
steps:
- uses: actions/stale@1e223db275d687790206a7acac4d1a11bd6fe629 # v10
- uses: actions/stale@4391f3da665fdf50b6810c1a66712fb9ba21aa93 # v11.0.0
with:
# Days of inactivity before an issue or PR becomes stale
days-before-stale: 150

View File

@@ -19,7 +19,7 @@ jobs:
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- name: Set up Python
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: "3.14"
@@ -40,7 +40,7 @@ jobs:
uses: astral-sh/setup-uv@c771a70e6277c0a99b617c7a806ffedaca235ff9 # v9.0.0
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
with:
python-version: ${{ matrix.python-version }}

View File

@@ -2,6 +2,31 @@
<!-- insert new changelog below this comment -->
## [0.15.1] - 2026-07-31
### Changed
- fix: escape Rich markup in `workflow resolve` output (#3879)
- chore(deps): bump actions/stale from 10.4.0 to 11.0.0 (#3877)
- chore(deps): bump actions/setup-python from 6.3.0 to 7.0.0 (#3876)
- feat: support tar archives for installs (#3874)
- fix: eliminate TOCTOU race in file unlink calls (#3819)
- fix(scripts): tolerate an unusable integration.json in the Python helper (#3785)
- fix(catalogs): validate the port in the shared catalog-URL validator, like its mirrors do (#3804)
- feat(presets): add opt-in constitution-sync preset (#3873)
- fix: reject non-object workflow caches (#3860)
- Harden extension URL download cache against symlink and junction races (#3869)
- fix: escape workflow step metadata (#3863)
- [bug-fix] Fix bundle-update-force-mislead: add refresh() to DefaultPrimitiveInstaller (#3452)
- fix: use chunked read for extension manifest hash (#3841)
- fix: preserve unreadable event config files (#3861)
- fix(scripts): use a .NET Framework-safe trim in the PowerShell init-dir resolver (#3872)
- Add ContextForge MCP extension to community catalog (#3487)
- fix: normalize non-UTF-8 integration manifests (#3862)
- feat: bind gate verdict to workflow input via verdict_input (#3725)
- docs: use absolute image URLs in README for PyPI rendering (#3867)
- chore: release 0.15.0, begin 0.15.1.dev0 development (#3871)
## [0.15.0] - 2026-07-30
### Changed

View File

@@ -1,5 +1,5 @@
<div align="center">
<img src="./media/logo_large.webp" alt="Spec Kit Logo" width="200" height="200"/>
<img src="https://raw.githubusercontent.com/github/spec-kit/main/media/logo_large.webp" alt="Spec Kit Logo" width="200" height="200"/>
<h1>🌱 Spec Kit</h1>
<h3><em>Define what to build before building it — with any AI coding agent.</em></h3>
</div>
@@ -136,7 +136,7 @@ For detailed step-by-step instructions, see our [comprehensive guide](./spec-dri
Want to see Spec Kit in action? Watch our [video overview](https://www.youtube.com/watch?v=a9eR1xsfvHg&pp=0gcJCckJAYcqIYzv)!
[![Spec Kit video header](/media/spec-kit-video-header.jpg)](https://www.youtube.com/watch?v=a9eR1xsfvHg&pp=0gcJCckJAYcqIYzv)
[![Spec Kit video header](https://raw.githubusercontent.com/github/spec-kit/main/media/spec-kit-video-header.jpg)](https://www.youtube.com/watch?v=a9eR1xsfvHg&pp=0gcJCckJAYcqIYzv)
## 🌍 Community

View File

@@ -50,6 +50,7 @@ The following community-contributed extensions are available in [`catalog.commun
| Coding Standards Drift Control | Generate coding-standards drift reports and remediation tasks for active Spec Kit features | `code` | Read+Write | [spec-kit-coding-standards-drift-control](https://github.com/benizzio/spec-kit-coding-standards-drift-control) |
| Conduct Extension | Orchestrates spec-kit phases via sub-agent delegation to reduce context pollution. | `process` | Read+Write | [spec-kit-conduct-ext](https://github.com/twbrandon7/spec-kit-conduct-ext) |
| Confluence Extension | Create a doc in Confluence summarizing the specifications and planning files | `integration` | Read+Write | [spec-kit-confluence](https://github.com/aaronrsun/spec-kit-confluence) |
| ContextForge MCP | Integrates codebase-memory-mcp + headroom into Spec Kit — graph-based code intelligence and context compression for the implement phase | `code` | Read+Write | [contextforge-mcp](https://github.com/capatinore/contextforge-mcp) |
| Cost Tracker | Track real LLM dollar cost across SDD workflows — per-feature budgets, per-integration comparison, and finance-ready exports | `visibility` | Read+Write | [spec-kit-cost](https://github.com/Quratulain-bilal/spec-kit-cost) |
| Data Model Diagram | Generates Mermaid ER diagrams from Spec Kit data models after planning | `docs` | Read+Write | [spec-kit-data-model-diagram](https://github.com/benizzio/spec-kit-data-model-diagram) |
| DocGuard — CDD Enforcement | The only doc-integrity engine with an MCP server, SARIF/JUnit output, and a deterministic zero-LLM core. Validates, scores, and traces documentation against code — 27 validators, stable finding codes, adoption baseline for legacy repos, compliance-evidence reports, GitHub Action with PR annotations, spec-kit hooks. Pure Node.js, one pinned dep. | `docs` | Read+Write | [spec-kit-docguard](https://github.com/raccioly/docguard) |

View File

@@ -39,6 +39,21 @@ specify workflow run my-pipeline.yml --json
`workflow_id` is the `workflow.id` declared inside the YAML, not the file name. The object is printed exactly as shown — pretty-printed with two-space indentation, on plain stdout with no Rich markup — so it always parses. While the workflow runs under `--json`, any progress a step would print (for example a gate prompt, or output from a prompt step's CLI subprocess) is redirected to stderr, so stdout carries only the JSON object. Read the object from stdout; leave stderr attached to the terminal or capture it separately.
For `failed` and `aborted` runs, the payload includes an `error` field carrying the terminal step's error message:
```json
{
"run_id": "662bf791",
"workflow_id": "build-and-review",
"status": "failed",
"current_step_id": "boom",
"current_step_index": 0,
"error": "Command exited with code 3"
}
```
`completed` and `paused` runs omit the `error` field. The error is persisted in the run's `state.json`, so `specify workflow status <run_id> --json` surfaces the same message after the fact.
> **Note:** Most workflow commands require a project already initialized with `specify init`. The exception is `specify workflow run <local-file.{yml,yaml}>`, which can run outside a project; in that case, run state is stored under the current directory's `.specify/workflows/runs/<run_id>/`.
## Resume a Workflow
@@ -88,10 +103,17 @@ specify workflow add <source>
| Option | Description |
| --------------- | ------------------------------------------------------ |
| `--dev` | Install from a local workflow YAML file or directory |
| `--dev` | Install from a local YAML file, package directory, or archive |
| `--from <url>` | Install from a custom URL (`<source>` names the expected workflow ID) |
Installs a workflow from the catalog, a URL (HTTPS required), a local YAML file, or a local directory containing `workflow.yml`.
Installs a workflow from the catalog, an HTTPS URL, a local YAML file, a
directory containing `workflow.yml`, or a `.zip`, `.tar.gz`, or `.tgz`
archive. Archives may contain `workflow.yml` at the root or inside one
top-level directory.
Directory and archive installs preserve the complete workflow package,
including scripts and other companion files. ZIP, `.tar.gz`, and `.tgz`
archives follow the same validation and installation behavior.
## Workflow Overlays
@@ -266,7 +288,9 @@ Lower priority values have higher precedence. Change this overlay to `priority:
### Interaction with Bundles and Updates
`specify workflow add <local-directory>` installs `workflow.yml` from the local directory into `.specify/workflows/<id>/`.
`specify workflow add <local-directory>` installs the complete local workflow
package into `.specify/workflows/<id>/`. Archive installs preserve the same
package contents.
When an installed workflow is refreshed or reinstalled, project overlays in `.specify/workflows/overlays/<id>/` are preserved because they live outside the installed workflow directory.
@@ -554,6 +578,64 @@ Each workflow run persists its state at `.specify/workflows/runs/<run_id>/`:
This enables `specify workflow resume` to continue from the exact step where a run was paused (e.g., at a gate) or failed.
### Gate Verdict Inputs
`verdict_input` binds a gate's verdict to a named workflow input. The input must be declared in the workflow's `inputs` block; `specify workflow validate` reports an undeclared reference.
`verdict_input` is not supported inside a `fan-out` template. Fan-out items
share workflow inputs, while workflow state can represent only one paused
gate. Place a gate before the fan-out to approve the whole batch, or after a
fan-in to review the aggregated results.
**Input value semantics:**
| Value | Behavior |
|---|---|
| Non-empty string, matches an option (case-insensitive) | Gate auto-decides; `output.choice` is set to the configured option spelling |
| Non-empty string, no match | Gate fails immediately |
| Non-string | Gate fails immediately |
| Missing or empty | Gate prompts on a TTY; pauses otherwise |
**Default value semantics:** A non-empty `default` is consumed as a verdict on the first run — matching an option auto-decides the gate, not matching fails it immediately.
```yaml
inputs:
spec_verdict:
type: string
default: ""
steps:
- id: review-spec
type: gate
message: "Approve the specification?"
options: [approve, reject]
on_reject: retry
verdict_input: spec_verdict
```
Supply a verdict when resuming:
```bash
specify workflow resume <run_id> --input spec_verdict=approve
```
For `on_reject: retry`, a bound reject verdict is consumed before the gate
pauses: the named stored input is reset to `""`. A later resume therefore
prompts or pauses again until another verdict is supplied. Approve, abort, and
skip outcomes leave the input unchanged.
Because of that reset, a verdict input used with `on_reject: retry` must accept
`""`. If it declares an `enum`, include the empty string — otherwise the reset
value violates the input's own `enum` and the run can no longer be resumed with
any input. `specify workflow add` reports this as a validation error.
```yaml
inputs:
spec_verdict:
type: string
enum: ["", approve, reject]
default: ""
```
## FAQ
### What happens when a workflow hits a gate step?

View File

@@ -208,6 +208,74 @@ Restart your IDE to refresh the command list.
---
## Behavior change: `/constitution` no longer propagates into templates
The `/constitution` command ([#3790](https://github.com/github/spec-kit/pull/3790)) is scoped to
its own artifact. It updates
`.specify/memory/constitution.md` and writes a Sync Impact Report, and **no longer edits**
`plan-template.md`, `spec-template.md`, `tasks-template.md`, installed command files, or
guidance docs.
### Why
Spec Kit uses **runtime resolution**: `plan`, `tasks`, and `analyze` read
`.specify/memory/constitution.md` live on every run, and `analyze` is the dedicated drift
checker. The governed templates carry a pointer, not a copy — `plan-template.md` ships
`[Gates determined based on constitution file]`, and `/plan` fills that section from the live
constitution each run. Propagation duplicated the single source of truth and fought the
preset/override composition system (a `replace` preset shadows an edited core template).
More broadly, presets and extensions — not in-place file edits — are how Spec Kit now governs
shared assets. Composing policy through the resolution stack keeps it centrally owned, versioned,
and auditable across repositories, instead of frozen into per-repo copies no core team can see.
### Is this a breaking change for existing projects?
**No — your workflow keeps working.** You would only notice a difference if you relied on
`/constitution` editing those files in place. The templates are scaffolds, not authorities. When you
run `/plan`, it copies the template into a per-feature `plan.md` and re-derives the Constitution
Check from the live constitution; `/analyze` validates against it. Even if a previous
`/constitution` run materialized concrete gate text into `.specify/templates/plan-template.md`,
the live constitution remains the source of truth at runtime.
On a **non-forced upgrade**, a materialized template is *preserved* (its hash diverges from the
recorded managed copy, so the refresh treats it as a customization and does not overwrite it).
Nothing regresses.
### Optional cleanup — return to the runtime pointer
A frozen, pre-filled Constitution Check is a slightly misleading scaffold and can bias the first
`/plan` pass. To move fully back to runtime resolution, reset the section body in
`.specify/templates/plan-template.md` to the pointer:
```text
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
[Gates determined based on constitution file]
```
Leave the rest of the file untouched. This is cleanup, not a required migration.
### Keeping the old behavior (opt-in)
If your team treats the materialized templates as **reviewed, committed artifacts** and wants
`/constitution` to keep propagating, install the bundled **`constitution-sync`** preset:
```bash
specify preset add constitution-sync
```
It wraps the core `/constitution` command and re-adds the propagation pass. It does **not** edit
versioned preset- or extension-provided templates or command files (those are owned by their
packages and are recomposed on reconciliation). Note that this edit-in-place propagation model
conflicts with the composition model used by the rest of the SDD commands when they are
preset/extension-managed — see the "Interaction with the resolution stack" section in
`presets/constitution-sync/README.md` for the tradeoffs and when to prefer the default instead.
---
## Common Scenarios
### Scenario 1: "I just want new slash commands"

View File

@@ -1070,6 +1070,41 @@
"created_at": "2026-03-29T00:00:00Z",
"updated_at": "2026-03-29T00:00:00Z"
},
"contextforge-mcp": {
"name": "ContextForge MCP",
"id": "contextforge-mcp",
"description": "Integrates codebase-memory-mcp + headroom into Spec Kit — graph-based code intelligence and context compression for the implement phase.",
"author": "capatinore",
"version": "0.1.0",
"download_url": "https://github.com/capatinore/contextforge-mcp/releases/download/ext-v0.1.0/contextforge-mcp-speckit-extension.zip",
"repository": "https://github.com/capatinore/contextforge-mcp",
"homepage": "https://github.com/capatinore/contextforge-mcp",
"documentation": "https://github.com/capatinore/contextforge-mcp/blob/main/README.md",
"changelog": "",
"license": "MIT",
"category": "code",
"effect": "read-write",
"requires": {
"speckit_version": ">=0.10.0"
},
"provides": {
"commands": 4,
"hooks": 0
},
"tags": [
"mcp",
"code-intelligence",
"context-compression",
"tokens",
"claude",
"spec-driven-development"
],
"verified": false,
"downloads": 0,
"stars": 0,
"created_at": "2026-07-13T00:00:00Z",
"updated_at": "2026-07-13T00:00:00Z"
},
"cost": {
"name": "Cost Tracker",
"id": "cost",

View File

@@ -25,6 +25,29 @@
"workflow",
"core"
]
},
"constitution-sync": {
"name": "Constitution Template Sync",
"id": "constitution-sync",
"version": "1.0.0",
"description": "Opt-in: restores /constitution propagation of amended guidance into plan/spec/tasks templates and installed command files, for teams that treat materialized templates as reviewed artifacts.",
"author": "github",
"repository": "https://github.com/github/spec-kit",
"license": "MIT",
"bundled": true,
"requires": {
"speckit_version": ">=0.14.4"
},
"provides": {
"commands": 1,
"templates": 0
},
"tags": [
"constitution",
"governance",
"templates",
"compatibility"
]
}
}
}

View File

@@ -0,0 +1,126 @@
# Constitution Template Sync
An **opt-in** preset that restores `/constitution`'s ability to propagate amended guidance into your
project's own templates and command files. After you update the constitution, it aligns
`plan-template.md`, `spec-template.md`, `tasks-template.md`, project-local command files, and
guidance docs so they reflect the current principles.
This propagation used to be built into `/constitution`; it was dropped when the command moved to the
preset model. Installing this preset opts you back into it: you get the guidance materialized into
reviewed, committed artifacts instead of relying on runtime resolution alone.
> **What you're opting into.** Propagation was removed deliberately — it duplicates the constitution
> as the source of truth and can fight the composition stack (materialized edits get shadowed or
> clobbered on the next recompose). This preset knowingly **reintroduces** that behavior, and those
> tradeoffs, for teams that want it. Read the [caveats](#caveats-you-take-on) before installing.
For most projects the default composable stack is the **recommended** approach, and at organization
scale it is usually the stronger governance model. Runtime resolution keeps the live constitution as
the single source of truth (nothing to re-sync, so nothing drifts), and the stack composes the
**entire** Spec Kit ecosystem — not just the SDD commands, but every command, template, script and
extension — with explicit priority levels, strategies, and independent versioning. It is a
capability, not automatic governance: a core team authors its own organizational presets and
extensions, then owns, versions, and audits that policy in one place and rolls it across many
repositories, instead of scattering frozen, per-repo copies no central team can see. This preset is
a supported escape hatch for teams whose workflow depends on reviewing materialized artifacts
directly — useful as a bridge, though for org-wide policy the better long-term path is usually a
versioned preset a core team maintains.
## What it does
Ships a single `wrap`-strategy override of `speckit.constitution`. It composes on top of the
current core command (via `{CORE_TEMPLATE}`), so it stays forward-compatible with core changes, and
appends a propagation pass that, after the constitution is written:
- Aligns `plan/spec/tasks-template.md` in `.specify/templates/` with the updated principles.
- Updates **project-local** command files and guidance docs to correct stale references.
- Extends the Sync Impact Report in `.specify/memory/constitution.md` with the files it touched.
## What it does not do
- It does **not** change behavior for anyone who does not install it — the default runtime
resolution model is untouched.
- It does **not** disable runtime resolution. `plan`, `tasks`, and `analyze` still read the live
constitution every run; this preset adds materialized copies on top — it does not replace the
source of truth.
- It does **not** edit versioned, package-owned files — templates or command files provided or
wrapped by another preset or extension. Those are recomposed from the resolution stack, so it
only ever writes into your project's own `.specify/templates/` scaffolds and command files that
are not managed by a preset/extension.
## When to use it
Install it **only** if your team treats the materialized templates and commands as
**reviewed, committed artifacts** — for example, if `plan-template.md`'s Constitution Check is
read in PRs as "here are our current gates" and is expected to track the constitution.
If you rely on the default runtime-resolution model, you do **not** need this preset: the live
constitution is already the single source of truth and there is nothing to sync.
## Caveats you take on
The preset resolution stack is how Spec Kit composes templates and commands going forward: they are
**layered, package-owned artifacts recomposed on demand**, not frozen files you edit in place.
Propagation is the opposite idea — it **materializes** guidance into files and freezes it. That
tension is the main thing to understand before installing:
- **Materialized copies can drift.** Anything propagated is a snapshot; if you amend the
constitution and do not re-run `/constitution`, the copies fall out of sync. The default runtime
model has no drift because it reads the live constitution every run.
- **Edits to composed files do not survive reconciliation.** If the rest of your SDD flow is
preset/extension-managed, the commands it materializes (`speckit.plan`, `speckit.specify`,
`speckit.tasks`, `speckit.analyze`, `speckit.implement`, …) are recomputed from the stack. Any
guidance propagated into them is clobbered the next time the stack reconciles — on
`specify integration use <key>` / `switch`, `specify integration upgrade`, or any preset/extension
install or remove. The same applies to templates owned by another preset/extension. This is why
the preset restricts itself to project-local files; propagation is reliable **only** for
artifacts you own outright.
- **A pre-filled Constitution Check can bias `/plan`.** Materializing concrete gates into
`plan-template.md` replaces the runtime pointer, so the first `/plan` pass may anchor on the
frozen text. Keep the pointer unless you specifically want committed gates.
**Bottom line:** this preset fits projects whose governed templates and commands are project-local
artifacts they review, with the rest of the SDD flow on the plain bundled core. If your
`plan`/`specify`/`tasks`/`analyze` commands or templates come from other presets or extensions,
prefer the default runtime-resolution model.
## Installation
```bash
# constitution-sync is a bundled preset — no download needed
specify preset add constitution-sync
```
## Development
```bash
# Test from local directory
specify preset add --dev ./presets/constitution-sync
# Verify the wrapped command resolves
specify preset resolve speckit.constitution
# Remove when done
specify preset remove constitution-sync
```
## Migrating back to the default
To move back to runtime resolution, reset each materialized `## Constitution Check` section in
`.specify/templates/plan-template.md` to the pointer:
```text
## Constitution Check
*GATE: Must pass before Phase 0 research. Re-check after Phase 1 design.*
[Gates determined based on constitution file]
```
Then remove this preset. See `docs/upgrade.md` for details.
## License
MIT

View File

@@ -0,0 +1,54 @@
---
description: Create or update the project constitution, then propagate the amended guidance into dependent templates and installed command files (opt-in template sync).
strategy: wrap
handoffs:
- label: Build Specification
agent: speckit.specify
prompt: Implement the feature specification based on the updated constitution. I want to build...
---
{CORE_TEMPLATE}
## Constitution Template Sync
> **This section supersedes the "Scope Guard" above for template and command propagation.**
> The core Scope Guard states that dependent templates and commands "are not modified here" —
> that restriction is intentionally lifted by this preset. Propagating amended guidance into the
> project's own dependent scaffolds and installed command files (as described below) **is in
> scope** whenever this preset is installed. The Scope Guard's other constraints still apply:
> do not implement features, generate application code, or touch artifacts unrelated to the
> constitution/template workflow.
After you have written the updated constitution above, perform a consistency propagation pass
so the dependent artifacts reflect the amended principles:
1. Read `.specify/templates/plan-template.md` and ensure any "Constitution Check" or rules align
with the updated principles. Only materialize concrete gate text here if your team intends to
review it as committed content; otherwise leave the runtime pointer
`[Gates determined based on constitution file]` in place so `/plan` fills it from the live
constitution.
2. Read `.specify/templates/spec-template.md` for scope/requirements alignment — update if the
constitution adds/removes mandatory sections or constraints.
3. Read `.specify/templates/tasks-template.md` and ensure task categorization reflects new or
removed principle-driven task types (e.g., observability, versioning, testing discipline).
4. Read each installed Spec Kit command file for your agent (including this one) — named
`speckit.*` or `speckit-*` (dot or hyphen depending on the agent), or laid out as
`speckit-<name>/SKILL.md` for skills-based integrations, e.g. in `.github/agents/`,
`.github/skills/`, `.claude/skills/`, or your agent's equivalent commands directory — to verify
no outdated references (CLAUDE-only or other agent-specific names) remain when generic guidance
is required. **Only hand-edit a command file if it is a project-local file not managed by a
preset or extension.** Command files that are composed from the resolution stack (anything
provided or wrapped by a preset/extension) must be regenerated through the stack — do **not**
edit them in place, because reconciliation (`specify integration use`, `specify integration
upgrade`, or any preset/extension install/remove) will clobber the edits.
5. Read any runtime guidance docs (e.g., `README.md`, `docs/quickstart.md`, or agent-specific
guidance files if present) and update references to principles that changed.
Then extend the Sync Impact Report at the top of `.specify/memory/constitution.md` with:
- Templates requiring updates (✅ updated / ⚠ pending) with file paths.
**Do not edit versioned preset- or extension-provided template or command files directly.** Those
artifacts are owned by their packages and are recomposed on the package's next update or on stack
reconciliation — hand edits are clobbered. Limit propagation to the project's own
`.specify/templates/` scaffolds and to command files that are not managed by a preset or extension.

View File

@@ -0,0 +1,30 @@
schema_version: "1.0"
preset:
id: "constitution-sync"
name: "Constitution Template Sync"
version: "1.0.0"
description: "Opt-in: restores /constitution propagation of amended guidance into plan/spec/tasks templates and installed command files, for teams that treat materialized templates as reviewed artifacts."
author: "github"
repository: "https://github.com/github/spec-kit"
license: "MIT"
requires:
# Requires the runtime-resolution baseline (#3790, shipped in 0.14.4) where the
# core /constitution command no longer propagates. Installing this preset on an
# older core would double-apply propagation.
speckit_version: ">=0.14.4"
provides:
templates:
- type: "command"
name: "speckit.constitution"
file: "commands/speckit.constitution.md"
description: "Wrap /constitution to also propagate guidance into dependent templates and command files"
strategy: "wrap"
tags:
- "constitution"
- "governance"
- "templates"
- "compatibility"

View File

@@ -1,6 +1,6 @@
[project]
name = "specify-cli"
version = "0.15.0"
version = "0.15.2.dev0"
description = "Specify CLI, part of GitHub Spec Kit. A tool to bootstrap your projects for Spec-Driven Development (SDD)."
readme = "README.md"
requires-python = ">=3.11"
@@ -49,6 +49,7 @@ packages = ["src/specify_cli"]
"workflows/speckit" = "specify_cli/core_pack/workflows/speckit"
# Bundled presets (installable via `specify preset add <name>` or `specify init --preset <name>`)
"presets/lean" = "specify_cli/core_pack/presets/lean"
"presets/constitution-sync" = "specify_cli/core_pack/presets/constitution-sync"
# Community bundle catalog snapshot (used for offline discovery)
"bundles/catalog.community.json" = "specify_cli/core_pack/bundles/catalog.community.json"

View File

@@ -55,9 +55,17 @@ function Resolve-SpecifyInitDir {
}
# Resolve-Path echoes back any trailing separator from the input; trim it so
# the returned root matches the bash resolver, whose `cd && pwd` never yields
# one. TrimEndingDirectorySeparator is a no-op on a bare root and on a path
# that already has no trailing separator.
$initRoot = [System.IO.Path]::TrimEndingDirectorySeparator($resolved.Path)
# one. TrimEnd (not [Path]::TrimEndingDirectorySeparator, which is .NET Core
# only) keeps this working on Windows PowerShell 5.1 / .NET Framework, as
# Get-FeaturePathsEnv already does below. Unlike a bare TrimEnd, the
# GetPathRoot check preserves a path that *is* its own root ('C:\' must not
# become 'C:', which every later API re-resolves against the current
# directory instead of the drive root). No-op on a path with no trailing
# separator.
$initRoot = $resolved.Path.TrimEnd('/', '\')
if ($initRoot.Length -lt [System.IO.Path]::GetPathRoot($resolved.Path).Length) {
$initRoot = $resolved.Path
}
if (-not (Test-Path -LiteralPath (Join-Path $initRoot '.specify') -PathType Container)) {
[Console]::Error.WriteLine("ERROR: SPECIFY_INIT_DIR is not a Spec Kit project (no .specify/ directory): $initRoot")
if ($ReturnNullOnError) { return $null }

View File

@@ -258,16 +258,27 @@ def get_invoke_separator(repo_root: Path) -> str:
integration_json = repo_root / ".specify" / "integration.json"
if not integration_json.is_file():
return "."
# Split the parse out of the lookup and guard the top-level shape, matching
# read_feature_json_feature_directory above and the bash/PowerShell twins,
# which both fall back to "." for any unusable integration.json:
# * a non-mapping top level ([], "forge", 42, null) is valid JSON, so
# json.JSONDecodeError never fires and state.get(...) raised
# AttributeError;
# * a non-UTF-8 file raises UnicodeDecodeError, which is a ValueError --
# not an OSError -- so it escaped the except tuple. Realistic on
# Windows, where PowerShell 5.1's Out-File/`>` default to UTF-16.
try:
state = json.loads(integration_json.read_text(encoding="utf-8"))
key = state.get("default_integration") or state.get("integration") or ""
settings = state.get("integration_settings")
if isinstance(key, str) and isinstance(settings, dict):
entry = settings.get(key)
if isinstance(entry, dict) and entry.get("invoke_separator") in {".", "-"}:
return entry["invoke_separator"]
except (OSError, json.JSONDecodeError):
pass
except (OSError, UnicodeError, json.JSONDecodeError):
return "."
if not isinstance(state, dict):
return "."
key = state.get("default_integration") or state.get("integration") or ""
settings = state.get("integration_settings")
if isinstance(key, str) and isinstance(settings, dict):
entry = settings.get(key)
if isinstance(entry, dict) and entry.get("invoke_separator") in {".", "-"}:
return entry["invoke_separator"]
return "."

View File

@@ -7,6 +7,7 @@ import re
import socket
import stat
import struct
import tarfile
import unicodedata
import zipfile
from collections.abc import Iterator
@@ -14,11 +15,12 @@ from contextlib import ExitStack, contextmanager
from ipaddress import IPv4Address, IPv6Address, ip_address
from itertools import pairwise
from pathlib import Path, PurePosixPath, PureWindowsPath
from typing import NoReturn, TypeVar
from typing import BinaryIO, Literal, NoReturn, TypeVar
from urllib.parse import ParseResult, urlparse
ErrorT = TypeVar("ErrorT", bound=Exception)
ArchiveFormat = Literal["zip", "tar.gz"]
MAX_DOWNLOAD_BYTES = 50 * 1024 * 1024
MAX_ZIP_ENTRIES = 512
@@ -67,6 +69,130 @@ _ZIP_MAX_COMMENT_BYTES = (1 << 16) - 1
_BOUNDED_ZIP_COMPRESSION_METHODS = frozenset(
(zipfile.ZIP_STORED, zipfile.ZIP_DEFLATED)
)
_ARCHIVE_CONTENT_TYPES: dict[str, ArchiveFormat] = {
"application/gzip": "tar.gz",
"application/x-gzip": "tar.gz",
"application/x-tar+gzip": "tar.gz",
"application/zip": "zip",
"application/x-zip-compressed": "zip",
}
def archive_format_from_name(name: str) -> ArchiveFormat | None:
"""Return the supported archive format declared by a path or URL."""
try:
path = urlparse(name).path.lower()
except (TypeError, ValueError):
return None
if path.endswith(".tar.gz") or path.endswith(".tgz"):
return "tar.gz"
if path.endswith(".zip"):
return "zip"
return None
def archive_format_from_content_type(content_type: str | None) -> ArchiveFormat | None:
"""Return the supported archive format declared by an HTTP Content-Type."""
if not isinstance(content_type, str):
return None
media_type = content_type.partition(";")[0].strip().lower()
return _ARCHIVE_CONTENT_TYPES.get(media_type)
def archive_suffix(archive_format: ArchiveFormat) -> str:
"""Return the canonical filename suffix for *archive_format*."""
if archive_format == "zip":
return ".zip"
if archive_format == "tar.gz":
return ".tar.gz"
raise ValueError(f"Unsupported archive format: {archive_format!r}")
def detect_archive_format(
archive_path: Path,
*,
archive_file: BinaryIO | None = None,
source_name: str | None = None,
content_type: str | None = None,
error_type: type[ErrorT] = ValueError,
) -> ArchiveFormat:
"""Validate the declared archive format against the file contents.
A recognized path/URL suffix is authoritative. For remote responses whose
final URL has no archive suffix, a recognized Content-Type may declare the
format instead. When both declarations are recognized they must agree, and
the resulting declaration must match the archive bytes.
"""
archive_path = Path(archive_path)
name_format = archive_format_from_name(
source_name if source_name is not None else str(archive_path)
)
content_format = archive_format_from_content_type(content_type)
if (
name_format is not None
and content_format is not None
and name_format != content_format
):
_raise(
error_type,
f"Archive format mismatch: filename declares {name_format} but "
f"Content-Type declares {content_format}",
)
declared_format = name_format or content_format
with ExitStack() as stack:
if archive_file is None:
try:
archive_file = stack.enter_context(archive_path.open("rb"))
except OSError as exc:
_raise_from(error_type, f"Invalid archive: {archive_path}", exc)
try:
archive_file.seek(0)
is_zip = zipfile.is_zipfile(archive_file)
archive_file.seek(0)
signature = archive_file.read(4)
# Let the bounded ZIP preflight report structural errors such as
# impossible entry counts. ``is_zipfile`` rejects those before the
# extractor can produce the established security diagnostic.
is_zip = is_zip or signature in {
b"PK\x03\x04",
b"PK\x05\x06",
b"PK\x07\x08",
}
is_gzip = signature[:2] == b"\x1f\x8b"
archive_file.seek(0)
is_tar_gz = False
if is_gzip:
try:
with tarfile.open(fileobj=archive_file, mode="r:gz"):
is_tar_gz = True
except tarfile.TarError:
pass
archive_file.seek(0)
except OSError as exc:
_raise_from(error_type, f"Invalid archive: {archive_path}", exc)
actual_format: ArchiveFormat | None
if is_zip and not is_tar_gz:
actual_format = "zip"
elif is_tar_gz and not is_zip:
actual_format = "tar.gz"
else:
actual_format = None
if declared_format is None:
if actual_format is None:
_raise(
error_type,
"Unsupported archive format; expected .zip, .tar.gz, or .tgz",
)
declared_format = actual_format
if actual_format != declared_format:
actual_label = actual_format or "invalid/unsupported data"
_raise(
error_type,
f"Archive format mismatch: expected {declared_format}, got {actual_label}",
)
return declared_format
def _ip_address_without_scope(
@@ -292,6 +418,7 @@ def build_safe_download_path(
*,
error_type: type[ErrorT] = ValueError,
label: str = "archive",
suffix: str = ".zip",
) -> Path:
"""Build a portable single-component archive path inside *target_dir*."""
if not isinstance(identifier, str) or not isinstance(version, str):
@@ -301,7 +428,9 @@ def build_safe_download_path(
f"{identifier!r} and {version!r}",
)
filename = f"{identifier}-{version}.zip"
if suffix not in {".zip", ".tar.gz", ".tgz"}:
_raise(error_type, f"Unsupported archive download suffix: {suffix!r}")
filename = f"{identifier}-{version}{suffix}"
try:
filename_too_long = (
len(filename.encode("utf-8")) > MAX_ZIP_COMPONENT_BYTES
@@ -378,24 +507,25 @@ def read_zip_member_limited(
)
def normalize_zip_member_name(
def normalize_archive_member_name(
name: str,
*,
archive_label: str = "archive",
error_type: type[ErrorT] = ValueError,
) -> str:
"""Return a normalized, portable ZIP member name or raise if unsafe."""
"""Return a normalized, portable archive member name or raise if unsafe."""
if "\x00" in name:
_raise(error_type, f"Unsafe path in ZIP archive: {name!r}")
_raise(error_type, f"Unsafe path in {archive_label} archive: {name!r}")
normalized = name.replace("\\", "/")
try:
encoded_name = normalized.encode("utf-8")
except UnicodeEncodeError:
_raise(error_type, f"Unsafe path in ZIP archive: {name!r}")
_raise(error_type, f"Unsafe path in {archive_label} archive: {name!r}")
if len(encoded_name) > MAX_ZIP_PATH_BYTES:
_raise(
error_type,
f"Unsafe path in ZIP archive: {name!r} "
f"Unsafe path in {archive_label} archive: {name!r} "
"(not portable across supported filesystems)",
)
path = PurePosixPath(normalized)
@@ -415,7 +545,8 @@ def normalize_zip_member_name(
):
_raise(
error_type,
f"Unsafe path in ZIP archive: {name!r} (potential path traversal)",
f"Unsafe path in {archive_label} archive: {name!r} "
"(potential path traversal)",
)
for part in raw_parts:
reserved_stem = part.partition(".")[0].partition(":")[0].rstrip(" ")
@@ -432,13 +563,26 @@ def normalize_zip_member_name(
):
_raise(
error_type,
f"Unsafe path in ZIP archive: {name!r} "
f"Unsafe path in {archive_label} archive: {name!r} "
"(not portable across supported filesystems)",
)
return normalized
def portable_zip_path_key(name: str) -> tuple[str, ...]:
def normalize_zip_member_name(
name: str,
*,
error_type: type[ErrorT] = ValueError,
) -> str:
"""Return a normalized, portable ZIP member name or raise if unsafe."""
return normalize_archive_member_name(
name,
archive_label="ZIP",
error_type=error_type,
)
def portable_archive_path_key(name: str) -> tuple[str, ...]:
"""Return a comparison key for filesystems with case/Unicode folding."""
normalized_name = name.replace("\\", "/")
return tuple(
@@ -447,6 +591,11 @@ def portable_zip_path_key(name: str) -> tuple[str, ...]:
)
def portable_zip_path_key(name: str) -> tuple[str, ...]:
"""Backward-compatible ZIP-specific alias for portable archive keys."""
return portable_archive_path_key(name)
def _raise_zip64(error_type: type[ErrorT]) -> NoReturn:
_raise(
error_type,
@@ -705,6 +854,7 @@ def _preflight_zip_central_directory(
def open_zip_bounded(
zip_path: Path,
*,
archive_file: BinaryIO | None = None,
error_type: type[ErrorT] = ValueError,
max_entries: int = MAX_ZIP_ENTRIES,
) -> Iterator[zipfile.ZipFile]:
@@ -712,10 +862,11 @@ def open_zip_bounded(
_validate_non_negative_int(max_entries, "max_entries")
zip_path = Path(zip_path)
with ExitStack() as stack:
try:
archive_file = stack.enter_context(zip_path.open("rb"))
except OSError as exc:
_raise_from(error_type, f"Invalid ZIP archive: {zip_path}", exc)
if archive_file is None:
try:
archive_file = stack.enter_context(zip_path.open("rb"))
except OSError as exc:
_raise_from(error_type, f"Invalid ZIP archive: {zip_path}", exc)
try:
_preflight_zip_central_directory(
archive_file,
@@ -737,6 +888,7 @@ def safe_extract_zip(
zip_path: Path,
target_dir: Path,
*,
archive_file: BinaryIO | None = None,
error_type: type[ErrorT] = ValueError,
max_entries: int = MAX_ZIP_ENTRIES,
max_member_bytes: int = MAX_ZIP_MEMBER_BYTES,
@@ -752,6 +904,7 @@ def safe_extract_zip(
with open_zip_bounded(
zip_path,
archive_file=archive_file,
error_type=error_type,
max_entries=max_entries,
) as zf:
@@ -774,7 +927,7 @@ def safe_extract_zip(
error_type=error_type,
)
is_dir = member.is_dir() or normalized_name.endswith("/")
path_key = portable_zip_path_key(normalized_name)
path_key = portable_archive_path_key(normalized_name)
existing = validated_paths.get(path_key)
if existing is not None:
@@ -894,3 +1047,211 @@ def safe_extract_zip(
)
if limit_error is not None:
_raise(error_type, limit_error)
def safe_extract_tar(
archive_path: Path,
target_dir: Path,
*,
archive_file: BinaryIO | None = None,
error_type: type[ErrorT] = ValueError,
max_entries: int = MAX_ZIP_ENTRIES,
max_member_bytes: int = MAX_ZIP_MEMBER_BYTES,
max_total_bytes: int = MAX_ZIP_TOTAL_BYTES,
) -> None:
"""Extract a gzip-compressed tar after ZIP-equivalent safety validation."""
_validate_non_negative_int(max_entries, "max_entries")
_validate_non_negative_int(max_member_bytes, "max_member_bytes")
_validate_non_negative_int(max_total_bytes, "max_total_bytes")
archive_path = Path(archive_path)
try:
target_root = target_dir.resolve()
except OSError as exc:
_raise_from(error_type, f"Invalid tar extraction target: {target_dir}", exc)
try:
if archive_file is not None:
archive_file.seek(0)
archive = tarfile.open(
archive_path if archive_file is None else None,
mode="r:gz",
fileobj=archive_file,
)
except (tarfile.TarError, OSError) as exc:
_raise_from(error_type, f"Invalid tar.gz archive: {archive_path}", exc)
with archive:
validated: list[tuple[tarfile.TarInfo, str, bool]] = []
validated_paths: dict[tuple[str, ...], tuple[str, bool]] = {}
total_size = 0
try:
for index, member in enumerate(archive, start=1):
if index > max_entries:
_raise(
error_type,
f"tar.gz archive contains too many entries "
f"({index} > {max_entries})",
)
normalized_name = normalize_archive_member_name(
member.name,
archive_label="tar.gz",
error_type=error_type,
)
is_dir = member.isdir()
if member.issym():
_raise(
error_type,
f"Unsafe symlink in tar.gz archive: {member.name}",
)
if member.islnk():
_raise(
error_type,
f"Unsafe hard link in tar.gz archive: {member.name}",
)
if not is_dir and not member.isreg():
_raise(
error_type,
f"Unsafe member type in tar.gz archive: {member.name}",
)
path_key = portable_archive_path_key(normalized_name)
existing = validated_paths.get(path_key)
if existing is not None:
_raise(
error_type,
f"Conflicting path in tar.gz archive: {member.name} "
f"conflicts with {existing[0]}",
)
validated_paths[path_key] = (member.name, is_dir)
member_path = (target_dir / normalized_name).resolve()
try:
member_path.relative_to(target_root)
except ValueError:
_raise(
error_type,
f"Unsafe path in tar.gz archive: {member.name} "
"(potential path traversal)",
)
if not is_dir:
if member.size > max_member_bytes:
_raise(
error_type,
f"tar.gz member {member.name} exceeds maximum size "
f"of {max_member_bytes} bytes",
)
total_size += member.size
if total_size > max_total_bytes:
_raise(
error_type,
f"tar.gz archive exceeds maximum uncompressed size "
f"of {max_total_bytes} bytes",
)
validated.append((member, normalized_name, is_dir))
except (tarfile.TarError, OSError) as exc:
_raise_from(
error_type,
f"Invalid tar.gz archive: {archive_path}",
exc,
)
for (
(path_key, (original, is_dir)),
(next_key, (next_original, _next_is_dir)),
) in pairwise(sorted(validated_paths.items())):
if (
not is_dir
and len(next_key) > len(path_key)
and next_key[: len(path_key)] == path_key
):
_raise(
error_type,
f"Conflicting path in tar.gz archive: {original} conflicts "
f"with {next_original}",
)
total_written = 0
for member, normalized_name, is_dir in validated:
member_path = target_dir / normalized_name
if is_dir:
try:
member_path.mkdir(parents=True, exist_ok=True)
except OSError as exc:
_raise_from(
error_type,
f"Failed to create tar.gz directory {member.name}: {exc}",
exc,
)
continue
try:
member_path.parent.mkdir(parents=True, exist_ok=True)
source = archive.extractfile(member)
if source is None:
_raise(
error_type,
f"Failed to read tar.gz member {member.name}",
)
written = 0
limit_error: str | None = None
with source, member_path.open("wb") as dest:
while True:
chunk = source.read(READ_CHUNK_SIZE)
if not chunk:
break
written += len(chunk)
if written > max_member_bytes:
limit_error = (
f"tar.gz member {member.name} exceeds maximum size "
f"of {max_member_bytes} bytes"
)
break
total_written += len(chunk)
if total_written > max_total_bytes:
limit_error = (
f"tar.gz archive exceeds maximum uncompressed size "
f"of {max_total_bytes} bytes"
)
break
dest.write(chunk)
except Exception as exc:
_raise_from(
error_type,
f"Failed to extract tar.gz member {member.name}: {exc}",
exc,
)
if limit_error is not None:
_raise(error_type, limit_error)
def safe_extract_archive(
archive_path: Path,
target_dir: Path,
*,
archive_file: BinaryIO | None = None,
source_name: str | None = None,
content_type: str | None = None,
error_type: type[ErrorT] = ValueError,
max_entries: int = MAX_ZIP_ENTRIES,
max_member_bytes: int = MAX_ZIP_MEMBER_BYTES,
max_total_bytes: int = MAX_ZIP_TOTAL_BYTES,
) -> ArchiveFormat:
"""Detect and securely extract a supported archive."""
archive_format = detect_archive_format(
archive_path,
archive_file=archive_file,
source_name=source_name,
content_type=content_type,
error_type=error_type,
)
extractor = safe_extract_zip if archive_format == "zip" else safe_extract_tar
extractor(
archive_path,
target_dir,
archive_file=archive_file,
error_type=error_type,
max_entries=max_entries,
max_member_bytes=max_member_bytes,
max_total_bytes=max_total_bytes,
)
return archive_format

View File

@@ -18,7 +18,7 @@ from urllib.request import url2pathname
from ..._assets import _locate_core_pack, _repo_root
from ..._download_security import MAX_JSON_CATALOG_BYTES, read_response_limited
from .. import BundlerError
from ..lib.yamlio import loads_json
from ..lib.yamlio import load_json, loads_json
from ..models.catalog import CatalogSource
from ..models.manifest import ComponentRef
@@ -145,13 +145,13 @@ def make_catalog_fetcher(*, allow_network: bool = True):
path = _file_url_to_path(parsed)
if not path.exists():
raise BundlerError(f"Catalog file not found: {path}")
return loads_json(path.read_text(encoding="utf-8"), origin=str(path))
return load_json(path)
if scheme == "" or _is_windows_drive_path(url):
path = Path(url)
if not path.exists():
raise BundlerError(f"Catalog file not found: {path}")
return loads_json(path.read_text(encoding="utf-8"), origin=str(path))
return load_json(path)
if scheme in ("http", "https"):
if not allow_network:
@@ -227,6 +227,10 @@ class DefaultPrimitiveInstaller:
manager = self._manager_for(component, project_root)
manager.install(component)
def refresh(self, project_root: Path, component: ComponentRef) -> None:
manager = self._manager_for(component, project_root)
manager.refresh(component)
def remove(self, project_root: Path, component: ComponentRef) -> None:
manager = self._manager_for(component, project_root)
manager.remove(component)

View File

@@ -93,9 +93,11 @@ def build_bundle(
# extraction, but collapse to two canonical modes (0755 when any
# execute bit is set on the source, otherwise 0644) so identical
# inputs yield a byte-for-byte identical artifact.
mode = 0o755 if file_path.stat().st_mode & 0o111 else 0o644
info.external_attr = mode << 16
archive.writestr(info, file_path.read_bytes())
with file_path.open("rb") as fh:
st = os.fstat(fh.fileno())
mode = 0o755 if st.st_mode & 0o111 else 0o644
info.external_attr = mode << 16
archive.writestr(info, fh.read())
return BuildResult(artifact_path=artifact_path, file_count=len(files))

View File

@@ -85,11 +85,17 @@ def _bundled_manifest_version(manifest_path: Path, root_key: str) -> str | None:
class _KindManager(Protocol):
def is_installed(self, component: ComponentRef) -> bool: ...
def is_installed(self, component: ComponentRef) -> bool:
pass
def install(self, component: ComponentRef) -> None: ...
def install(self, component: ComponentRef) -> None:
pass
def remove(self, component: ComponentRef) -> None: ...
def refresh(self, component: ComponentRef) -> None:
pass
def remove(self, component: ComponentRef) -> None:
pass
def primitive_manager(
@@ -151,6 +157,12 @@ class _PresetKindManager:
return False
def install(self, component: ComponentRef) -> None:
self._do_install(component, force=False)
def refresh(self, component: ComponentRef) -> None:
self._do_install(component, force=True)
def _do_install(self, component: ComponentRef, *, force: bool) -> None:
from ... import get_speckit_version
from ..._assets import _locate_bundled_preset
@@ -168,7 +180,9 @@ class _PresetKindManager:
component.version,
_bundled_manifest_version(bundled / "preset.yml", "preset"),
)
self._manager.install_from_directory(bundled, speckit_version, priority)
self._manager.install_from_directory(
bundled, speckit_version, priority, **({"force": True} if force else {})
)
return
if not self._allow_network:
@@ -194,7 +208,9 @@ class _PresetKindManager:
)
zip_path = catalog.download_pack(component.id)
try:
self._manager.install_from_zip(zip_path, speckit_version, priority)
self._manager.install_from_zip(
zip_path, speckit_version, priority, **({"force": True} if force else {})
)
finally:
with contextlib.suppress(Exception):
if zip_path.exists():
@@ -224,6 +240,12 @@ class _ExtensionKindManager:
return False
def install(self, component: ComponentRef) -> None:
self._do_install(component, force=False)
def refresh(self, component: ComponentRef) -> None:
self._do_install(component, force=True)
def _do_install(self, component: ComponentRef, *, force: bool) -> None:
from ... import get_speckit_version
from ..._assets import _locate_bundled_extension
@@ -242,7 +264,7 @@ class _ExtensionKindManager:
_bundled_manifest_version(bundled / "extension.yml", "extension"),
)
self._manager.install_from_directory(
bundled, speckit_version, priority=priority
bundled, speckit_version, priority=priority, force=force
)
return
@@ -272,7 +294,7 @@ class _ExtensionKindManager:
zip_path = catalog.download_extension(component.id)
try:
self._manager.install_from_zip(
zip_path, speckit_version, priority=priority
zip_path, speckit_version, priority=priority, force=force
)
finally:
with contextlib.suppress(Exception):
@@ -318,6 +340,11 @@ class _WorkflowKindManager:
lambda: workflow_add(component.id),
)
def refresh(self, component: ComponentRef) -> None:
# workflow_add is idempotent for already-installed workflows; delegate
# to the standard install path which handles version refresh correctly.
self.install(component)
def _assert_pinned_version(self, component: ComponentRef) -> None:
if not component.version:
return
@@ -378,6 +405,35 @@ class _StepKindManager:
lambda: workflow_step_add(component.id),
)
def refresh(self, component: ComponentRef) -> None:
# Preserve an existing step until we've validated we can perform refresh.
# For already-installed steps, keep a backup and restore it if the
# remove+reinstall path fails.
if not (self._allow_network and self.is_installed(component)):
self.install(component)
return
import shutil
import tempfile
step_dir = self._registry.steps_dir / component.id
metadata = self._registry.get(component.id)
backup_dir = Path(tempfile.mkdtemp(prefix="speckit-step-refresh-")) / component.id
try:
if step_dir.exists():
shutil.copytree(step_dir, backup_dir)
self.remove(component)
try:
self.install(component)
except BundlerError:
if backup_dir.exists():
shutil.copytree(backup_dir, step_dir, dirs_exist_ok=True)
if metadata is not None and not self._registry.is_installed(component.id):
self._registry.add(component.id, metadata)
raise
finally:
shutil.rmtree(backup_dir.parent, ignore_errors=True)
def remove(self, component: ComponentRef) -> None:
from ... import workflow_step_remove

View File

@@ -74,6 +74,13 @@ class CatalogStackBase:
try:
parsed = urlparse(url)
hostname = parsed.hostname
# Accessing ``port`` performs urllib's syntax/range validation;
# ``hostname`` alone does not, so a non-numeric or out-of-range
# port would otherwise pass validation here and only fail later,
# at fetch time, as an error this module does not translate --
# a raw http.client.InvalidURL for a non-numeric port, and a
# socket-layer failure for one that is merely out of range.
_ = parsed.port
except ValueError:
raise cls._error(f"Catalog URL is malformed: {url}") from None
is_localhost = hostname in ("localhost", "127.0.0.1", "::1")

View File

@@ -119,6 +119,8 @@ def _run_init(integration: str, *, script_type: str, offline: bool = False) -> N
preset=None,
integration=integration,
integration_options=None,
extensions=None,
trust_extension_urls=False,
)
except typer.Exit as exc:
if exc.exit_code:

View File

@@ -30,6 +30,145 @@ def _stdin_is_interactive() -> bool:
return sys.stdin.isatty()
def _ext_spec_is_url(ext_spec: str) -> bool:
"""Return True when *ext_spec* is an http(s) URL rather than a name/path."""
from urllib.parse import urlparse
try:
return urlparse(ext_spec).scheme in ("http", "https")
except ValueError:
return False
def _confirm_extension_url_trust(
url_specs: list[str], *, trust_override: bool
) -> dict[str, bool]:
"""Resolve trust for each URL-based extension before the Live display.
URL installs pull an arbitrary external extension, so they get the same
default-deny confirmation as ``extension add --from``. Returns a mapping of
``url_spec -> approved``. With *trust_override* every URL is pre-approved.
In a non-interactive session without the override, every URL is denied
(the prompt cannot be answered), mirroring the default-deny posture.
"""
from rich.markup import escape as _escape_markup
from rich.panel import Panel
approvals: dict[str, bool] = {}
interactive = _stdin_is_interactive()
for spec in url_specs:
if trust_override:
approvals[spec] = True
continue
if not interactive:
approvals[spec] = False
continue
console.print()
console.print(
Panel(
"[bold]You are installing an extension from an external URL that is not\n"
"listed in any of your configured extension catalogs.[/bold]\n\n"
f"URL: {_escape_markup(spec)}\n\n"
"Only install extensions from sources you trust.",
title="[bold yellow]⚠ Untrusted Source[/bold yellow]",
border_style="yellow",
padding=(1, 2),
)
)
console.print()
approvals[spec] = typer.confirm(
f"Install extension from {spec}?", default=False
)
return approvals
def _install_extension_during_init(project_path: Path, ext_spec: str, speckit_version: str) -> str:
"""Install a single extension during ``specify init``.
Handles bundled extension names, local directory paths, and HTTPS URLs.
Returns a short status message on success.
Raises ``ValueError`` on failure so the caller can convert it to a
tracker error without aborting the entire init.
"""
from urllib.parse import urlparse
from .._assets import _locate_bundled_extension
from ..extensions import ExtensionCatalog, ExtensionError, ExtensionManager
from ..extensions._commands import (
_resolve_catalog_extension,
install_extension_from_url,
)
manager = ExtensionManager(project_path)
# --- URL ---
parsed = urlparse(ext_spec)
if parsed.scheme in ("http", "https"):
try:
manifest = install_extension_from_url(
manager, project_path, ext_spec, speckit_version
)
except ExtensionError as exc:
raise ValueError(str(exc)) from exc
return f"{manifest.name} v{manifest.version} installed"
# --- Local path ---
if ext_spec.startswith(("./", "../", "/", "~/", ".\\", "..\\")) or Path(ext_spec).is_absolute():
source_path = Path(ext_spec).expanduser().resolve()
if not source_path.exists():
raise ValueError(f"Directory not found: {source_path}")
if not (source_path / "extension.yml").exists():
raise ValueError(f"No extension.yml found in {source_path}")
manifest = manager.install_from_directory(source_path, speckit_version)
return f"{manifest.name} v{manifest.version} installed"
# --- Bundled extension name or catalog ID ---
bundled_path = _locate_bundled_extension(ext_spec)
if bundled_path is not None:
if manager.registry.is_installed(ext_spec):
return "already installed"
manifest = manager.install_from_directory(bundled_path, speckit_version)
return f"{manifest.name} v{manifest.version} installed"
# Fall back to catalog
catalog = ExtensionCatalog(project_path)
ext_info, catalog_error = _resolve_catalog_extension(ext_spec, catalog, "add")
if catalog_error:
raise ValueError(f"Could not query extension catalog: {catalog_error}")
if not ext_info:
raise ValueError(f"Extension '{ext_spec}' not found in bundled extensions or catalog")
resolved_id = ext_info["id"]
if resolved_id != ext_spec:
bundled_path = _locate_bundled_extension(resolved_id)
if bundled_path is not None:
if manager.registry.is_installed(resolved_id):
return "already installed"
manifest = manager.install_from_directory(bundled_path, speckit_version)
return f"{manifest.name} v{manifest.version} installed"
if ext_info.get("bundled") and not ext_info.get("download_url"):
from ..extensions import REINSTALL_COMMAND
raise ValueError(
f"Extension '{resolved_id}' is bundled with spec-kit but not found in the installed package. "
f"Try reinstalling spec-kit: {REINSTALL_COMMAND}"
)
if not ext_info.get("_install_allowed", True):
catalog_name = ext_info.get("_catalog_name", "community")
raise ValueError(
f"Extension '{ext_spec}' is in the '{catalog_name}' catalog but installation is not allowed from that catalog"
)
zip_path = catalog.download_extension(resolved_id)
try:
manifest = manager.install_from_zip(zip_path, speckit_version)
finally:
zip_path.unlink(missing_ok=True)
return f"{manifest.name} v{manifest.version} installed"
def ensure_constitution_from_template(
project_path: Path, tracker: StepTracker | None = None
) -> None:
@@ -142,6 +281,16 @@ def register(app: typer.Typer) -> None:
"--integration-options",
help='Options for the integration (e.g. --integration-options="--commands-dir .myagent/cmds")',
),
extensions: list[str] | None = typer.Option(
None,
"--extension",
help="Install an extension during initialization (bundled name, local path, or HTTPS URL). Repeatable.",
),
trust_extension_urls: bool = typer.Option(
False,
"--trust-extension-urls",
help="Pre-authorize installing extensions from external URLs without the interactive trust prompt (required for non-interactive URL installs).",
),
):
"""
Initialize a new Specify project.
@@ -174,6 +323,10 @@ def register(app: typer.Typer) -> None:
specify init --here --integration gemini
specify init my-project --integration generic --integration-options="--commands-dir .myagent/commands/" # Bring your own agent; requires --commands-dir
specify init my-project --integration claude --preset healthcare-compliance # With preset
specify init my-project --integration copilot --extension git # With bundled extension
specify init my-project --extension git --extension selftest # Multiple extensions
specify init my-project --extension ./my-extensions/custom-ext # Local path extension
specify init my-project --extension https://example.com/extensions/my-ext.zip --trust-extension-urls # URL extension (non-interactive)
"""
# Lazy imports to avoid circular dependency — __init__.py imports this module
from .. import (
@@ -413,10 +566,31 @@ def register(app: typer.Typer) -> None:
("chmod", "Ensure scripts executable"),
("constitution", "Constitution setup"),
("workflow", "Install bundled workflow"),
("final", "Finalize"),
]:
tracker.add(key, label)
if extensions:
from rich.markup import escape as _escape_markup
for i, ext_spec in enumerate(extensions):
tracker.add(
f"extension-{i}", f"Install extension: {_escape_markup(ext_spec)}"
)
tracker.add("final", "Finalize")
# Resolve trust for URL-based extensions BEFORE entering the Live
# display: the confirmation prompt cannot be shown/answered underneath
# the Rich Live spinner. URL installs are default-deny unless the user
# confirms interactively or passes --trust-extension-urls.
extension_url_approvals: dict[str, bool] = {}
if extensions:
url_specs = [e for e in extensions if _ext_spec_is_url(e)]
if url_specs:
extension_url_approvals = _confirm_extension_url_trust(
url_specs, trust_override=trust_extension_urls
)
# Disable transient mode on Windows: PowerShell 5.1's legacy console
# hangs when Rich tries to restore cursor state via VT escape sequences.
_transient = sys.platform != "win32"
@@ -626,6 +800,46 @@ def register(app: typer.Typer) -> None:
continuing="Continuing without the optional preset.",
)
# Install extensions specified via --extension
if extensions:
from rich.markup import escape as _escape_markup
from ..extensions._commands import _refresh_events_and_warn
speckit_ver = get_speckit_version()
any_extension_installed = False
for i, ext_spec in enumerate(extensions):
tracker.start(f"extension-{i}")
# Skip URL extensions the user did not confirm as trusted
# (default-deny; resolved before the Live display).
if _ext_spec_is_url(ext_spec) and not extension_url_approvals.get(
ext_spec, False
):
tracker.error(
f"extension-{i}",
"skipped: untrusted URL not confirmed "
"(use --trust-extension-urls)",
)
continue
try:
status_msg = _install_extension_during_init(
project_path, ext_spec, speckit_ver
)
tracker.complete(f"extension-{i}", status_msg)
any_extension_installed = True
except Exception as ext_err:
sanitized_ext = str(ext_err).replace("\n", " ").strip()
tracker.error(
f"extension-{i}",
f"failed: {_escape_markup(sanitized_ext[:120])}",
)
# Refresh native event configuration once after the batch so
# that an extension declaring ``events:`` has its hooks
# activated, mirroring the ``extension add`` path.
if any_extension_installed:
_refresh_events_and_warn(project_path)
# Seed the constitution AFTER preset installation so that a
# preset-provided constitution-template (resolved via the
# priority stack) wins over the core template.

View File

@@ -1983,23 +1983,25 @@ def _drop_marked_entries(entries: list) -> list:
def _load_user_json(path: Path) -> dict | None:
"""Load a user-owned JSON file, aborting (None) on parse failure (#22/#23).
"""Load a user-owned JSON file, aborting (None) on read/parse failure (#22/#23).
Returns the parsed dict, or ``None`` when the file is missing or cannot be
parsed (e.g. JSONC with comments, or temporarily malformed JSON). Callers
must skip the merge rather than resetting user content to ``{}``.
read or parsed (e.g. JSONC with comments, a temporarily malformed JSON
document, or an unreadable path). Callers must skip the merge rather than
resetting user content to ``{}``.
"""
if not path.exists():
return {}
try:
data = json.loads(path.read_text(encoding="utf-8"))
except (json.JSONDecodeError, ValueError) as exc:
except (json.JSONDecodeError, OSError, ValueError) as exc:
logger.warning(
"Could not parse %s (may contain JSONC comments or be malformed); "
"Could not read or parse %s (it may be unreadable, contain JSONC "
"comments, or be malformed); "
"skipping event-config merge to preserve user content.",
path,
)
logger.debug("Parse error detail: %s", exc)
logger.debug("Read/parse error detail: %s", exc)
return None
if not isinstance(data, dict):
logger.warning("%s is not a JSON object; skipping event-config merge.", path)

View File

@@ -20,7 +20,7 @@ import tempfile
from dataclasses import dataclass
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Callable, Dict, List, Optional, Set
from typing import Any, BinaryIO, Callable, Dict, List, Optional, Set
import pathspec
import yaml
@@ -29,11 +29,14 @@ from packaging.specifiers import InvalidSpecifier, SpecifierSet
from .._assets import _locate_core_pack, _repo_root
from .._download_security import (
archive_format_from_name,
archive_suffix,
MAX_JSON_CATALOG_BYTES,
build_safe_download_path,
detect_archive_format,
is_https_or_localhost_http,
read_response_limited,
safe_extract_zip,
safe_extract_archive,
)
from .._init_options import is_ai_skills_enabled
from .._invocation_style import is_dollar_skills_agent, is_slash_skills_agent
@@ -566,8 +569,11 @@ class ExtensionManifest:
def get_hash(self) -> str:
"""Calculate SHA256 hash of manifest file."""
h = hashlib.sha256()
with open(self.path, "rb") as f:
return f"sha256:{hashlib.sha256(f.read()).hexdigest()}"
for chunk in iter(lambda: f.read(8192), b""):
h.update(chunk)
return f"sha256:{h.hexdigest()}"
class ExtensionRegistry:
@@ -2400,7 +2406,7 @@ class ExtensionManager:
pass # Best-effort; install already committed to the registry.
# Restore execute bits on shipped POSIX scripts. copytree here (and the
# zipfile.extractall in install_from_zip, which delegates to this method) does
# archive extraction in install_from_archive, which delegates here, does
# not restore a stripped Unix mode, so a bundled *.sh would land non-executable
# and a documented `.specify/extensions/<id>/scripts/...` invocation would fail
# with "Permission denied". This is the single sink every install route funnels
@@ -2419,21 +2425,27 @@ class ExtensionManager:
return manifest
def install_from_zip(
def install_from_archive(
self,
zip_path: Path,
archive_path: Path,
speckit_version: str,
priority: int = 10,
force: bool = False,
*,
archive_file: BinaryIO | None = None,
source_name: str | None = None,
content_type: str | None = None,
) -> ExtensionManifest:
"""Install extension from ZIP file.
"""Install an extension from a supported archive.
Args:
zip_path: Path to extension ZIP file
archive_path: Path to a .zip, .tar.gz, or .tgz archive
speckit_version: Current spec-kit version
priority: Resolution priority (lower = higher precedence, default 10)
force: If True and extension is already installed, remove it first
before proceeding with installation
archive_file: Already-open archive stream to consume instead of
reopening ``zip_path``
Returns:
Installed extension manifest
@@ -2449,7 +2461,14 @@ class ExtensionManager:
with tempfile.TemporaryDirectory() as tmpdir:
temp_path = Path(tmpdir)
safe_extract_zip(zip_path, temp_path, error_type=ValidationError)
safe_extract_archive(
archive_path,
temp_path,
archive_file=archive_file,
source_name=source_name,
content_type=content_type,
error_type=ValidationError,
)
# Find extension directory (may be nested)
extension_dir = temp_path
@@ -2463,13 +2482,35 @@ class ExtensionManager:
manifest_path = extension_dir / "extension.yml"
if not manifest_path.exists():
raise ValidationError("No extension.yml found in ZIP file")
raise ValidationError("No extension.yml found in archive")
# Install from extracted directory
return self.install_from_directory(
extension_dir, speckit_version, priority=priority, force=force
)
def install_from_zip(
self,
zip_path: Path,
speckit_version: str,
priority: int = 10,
force: bool = False,
*,
archive_file: BinaryIO | None = None,
source_name: str | None = None,
content_type: str | None = None,
) -> ExtensionManifest:
"""Backward-compatible wrapper for archive installation."""
return self.install_from_archive(
zip_path,
speckit_version,
priority=priority,
force=force,
archive_file=archive_file,
source_name=source_name,
content_type=content_type,
)
def remove(self, extension_id: str, keep_config: bool = False) -> bool:
"""Remove an installed extension.
@@ -3787,14 +3828,14 @@ class ExtensionCatalog(CatalogStackBase):
def download_extension(
self, extension_id: str, target_dir: Optional[Path] = None
) -> Path:
"""Download extension ZIP from catalog.
"""Download an extension archive from a catalog.
Args:
extension_id: ID of the extension to download
target_dir: Directory to save ZIP file (defaults to temp directory)
target_dir: Directory to save the archive
Returns:
Path to downloaded ZIP file
Path to the downloaded archive
Raises:
ExtensionError: If extension not found or download fails
@@ -3853,45 +3894,88 @@ class ExtensionCatalog(CatalogStackBase):
target_dir = self.cache_dir / "downloads"
target_dir = Path(target_dir)
version = ext_info.get("version", "unknown")
zip_path = build_safe_download_path(
declared_format = archive_format_from_name(download_url)
build_safe_download_path(
target_dir,
extension_id,
version,
error_type=ExtensionError,
label="extension",
suffix=archive_suffix(declared_format or "tar.gz"),
)
target_dir.mkdir(parents=True, exist_ok=True)
original_download_url = download_url
extra_headers = None
resolved_download_url = self._resolve_github_release_asset_api_url(download_url)
if resolved_download_url:
download_url = resolved_download_url
extra_headers = {"Accept": "application/octet-stream"}
# Download the ZIP file
staging_path: Path | None = None
try:
with self._open_url(
download_url, timeout=60, extra_headers=extra_headers
) as response:
zip_data = read_response_limited(
archive_data = read_response_limited(
response,
error_type=ExtensionError,
label=f"extension '{extension_id}' download",
)
final_url = (
response.geturl()
if hasattr(response, "geturl")
else download_url
)
content_type = (
response.getheader("Content-Type")
if hasattr(response, "getheader")
else None
)
verify_archive_sha256(
zip_data, ext_info.get("sha256"), extension_id, ExtensionError
archive_data, ext_info.get("sha256"), extension_id, ExtensionError
)
zip_path.write_bytes(zip_data)
return zip_path
with tempfile.NamedTemporaryFile(
prefix="extension-download-",
suffix=".archive",
dir=target_dir,
delete=False,
) as staging_file:
staging_path = Path(staging_file.name)
staging_file.write(archive_data)
archive_format = detect_archive_format(
staging_path,
source_name=(
final_url
if archive_format_from_name(final_url) is not None
else original_download_url
),
content_type=content_type,
error_type=ExtensionError,
)
archive_path = build_safe_download_path(
target_dir,
extension_id,
version,
error_type=ExtensionError,
label="extension",
suffix=archive_suffix(archive_format),
)
os.replace(staging_path, archive_path)
staging_path = None
return archive_path
except urllib.error.URLError as e:
raise ExtensionError(
f"Failed to download extension from {download_url}: {e}"
)
except IOError as e:
raise ExtensionError(f"Failed to save extension ZIP: {e}")
raise ExtensionError(f"Failed to save extension archive: {e}")
finally:
if staging_path is not None:
staging_path.unlink(missing_ok=True)
def clear_cache(self):
"""Clear the catalog cache (both legacy and URL-hash-based files)."""

View File

@@ -8,11 +8,12 @@ which re-fetch from the parent package at call time so test monkeypatching of
"""
from __future__ import annotations
import errno
import hashlib
import os
import shutil
import stat
import tempfile
import zipfile
from pathlib import Path
from typing import Optional
from uuid import uuid4
@@ -26,12 +27,11 @@ from rich.table import Table
from .._console import console
from .._assets import get_speckit_version
from .._download_security import (
archive_format_from_name,
detect_archive_format,
is_https_or_localhost_http,
normalize_zip_member_name,
open_zip_bounded,
portable_zip_path_key,
read_response_limited,
read_zip_member_limited,
safe_extract_archive,
)
from .._init_options import is_ai_skills_enabled
@@ -95,6 +95,141 @@ def _refresh_events_and_warn(project_root: Path) -> None:
console.print(f" {key}: {_escape_markup(detail)}")
def install_extension_from_url(
manager,
project_root: Path,
url: str,
speckit_version: str,
*,
priority: int = 10,
force: bool = False,
):
"""Download an archive from *url* and install it, reusing the hardened path.
Shares the same download hardening as ``extension add --from``:
HTTPS enforcement, the catalog's authenticated + redirect-guarded
``_open_url`` fetch, a bounded (50 MiB) response read, archive-format
detection (ZIP or tar.gz/tgz), and a TOCTOU-safe transient download file
consumed directly by ``install_from_zip``.
Returns the installed manifest. Raises ``ExtensionError`` on any failure so
callers can present a uniform message without a second downloader.
"""
import urllib.error
from . import ExtensionCatalog, ExtensionError
if not is_https_or_localhost_http(url):
raise ExtensionError(
"URL must use HTTPS (HTTP is only allowed for localhost)"
)
download_dir = _validate_safe_cache_dir(project_root)
archive_filename = f"extension-url-download-{uuid4().hex}.archive"
# Only used for diagnostic messages: the real archive is a transient inode
# (unlinked on POSIX, O_TEMPORARY on Windows) consumed via ``archive_file``
# below, so this path is never opened again.
archive_path = download_dir / archive_filename
try:
dl_catalog = ExtensionCatalog(project_root)
download_url = url
extra_headers = None
resolved_url = dl_catalog._resolve_github_release_asset_api_url(download_url)
if resolved_url:
download_url = resolved_url
extra_headers = {"Accept": "application/octet-stream"}
with dl_catalog._open_url(
download_url, timeout=60, extra_headers=extra_headers
) as response:
archive_data = read_response_limited(
response,
error_type=ExtensionError,
label=f"extension {url}",
)
final_url = (
response.geturl() if hasattr(response, "geturl") else download_url
)
content_type = (
response.getheader("Content-Type")
if hasattr(response, "getheader")
else None
)
except urllib.error.URLError as exc:
raise ExtensionError(f"Failed to download from {url}: {exc}") from exc
download_fd = -1
download_file = None
try:
try:
download_fd = _safe_open_download_zip(
project_root, download_dir, archive_filename
)
except OSError as exc:
raise ExtensionError(
f"Could not safely create download file: {exc}"
) from exc
try:
download_file = os.fdopen(download_fd, "w+b")
download_fd = -1
download_file.write(archive_data)
download_file.flush()
download_file.seek(0)
except OSError as exc:
raise ExtensionError(
f"Could not safely write download file: {exc}"
) from exc
format_source = (
final_url
if archive_format_from_name(final_url) is not None
else url
)
try:
detect_archive_format(
archive_path,
archive_file=download_file,
source_name=format_source,
content_type=content_type,
error_type=ExtensionError,
)
except ExtensionError as exc:
raise ExtensionError(
f"{url} did not return a ZIP archive or tar.gz/tgz archive "
f"(got {len(archive_data)} bytes). This usually means the request "
"was not authenticated and a login/HTML page was returned. "
"Verify the URL and configured credentials."
) from exc
# Consume the transient inode reserved above rather than reopening the
# cache pathname during extraction.
try:
return manager.install_from_zip(
archive_path,
speckit_version,
priority=priority,
force=force,
archive_file=download_file,
)
except OSError as exc:
raise ExtensionError(
f"Could not install extension from downloaded archive: {exc}"
) from exc
finally:
if download_file is not None:
try:
download_file.close()
except OSError:
pass
elif download_fd >= 0:
try:
os.close(download_fd)
except OSError:
pass
def _load_catalog_command_config(project_root: Path, config_path: Path) -> dict:
"""Load extension catalog CLI config with user-facing shape errors."""
try:
@@ -437,6 +572,278 @@ def catalog_remove(
console.print("\n[dim]No catalogs remain in config. Built-in defaults will be used.[/dim]")
# Relative path, below the project root, of the extension URL download cache.
_CACHE_REL_PARTS = (".specify", "extensions", ".cache", "downloads")
def _has_secure_dir_fd() -> bool:
"""Whether this platform supports the strongest (POSIX) hardening path.
The descriptor-anchored walk needs ``O_NOFOLLOW`` plus ``dir_fd`` support
for ``os.open``/``os.mkdir``/``os.unlink``. When any of those is missing
(notably on Windows) the caller falls back to the portable path-wise walk,
which reproduces the same guarantees using symlink/reparse-point rejection,
resolve-under-root containment checks, and post-open inode-identity
verification instead of file descriptors.
"""
return bool(
getattr(os, "O_NOFOLLOW", 0)
and os.open in os.supports_dir_fd
and os.mkdir in os.supports_dir_fd
and os.unlink in os.supports_dir_fd
)
def _is_symlink_refusal_errno(exc: OSError) -> bool:
"""Whether an ``os.open``/``os.mkdir`` error means a component is a symlink.
Opening an ``O_NOFOLLOW`` path whose final component is a symlink raises
``ELOOP`` on Linux and ``EMLINK`` on some BSDs, while a symlinked component
that no longer resolves to a directory surfaces as ``ENOTDIR``.
"""
return exc.errno in (errno.ELOOP, errno.ENOTDIR, getattr(errno, "EMLINK", -1))
def _verify_leaf_identity(fd: int, path: Path) -> None:
"""Confirm ``fd`` still refers to the regular file at ``path``.
Mirrors the workflow installer's staged-file check: comparing the open
descriptor's ``fstat`` against a ``lstat`` of the pathname detects a leaf
that was swapped for a symlink/reparse point between creation and use, so
the portable (dir_fd-less) path is not vulnerable to an ancestor swap race.
"""
path_stat = path.stat(follow_symlinks=False)
open_stat = os.fstat(fd)
if (
not stat.S_ISREG(path_stat.st_mode)
or path_stat.st_dev != open_stat.st_dev
or path_stat.st_ino != open_stat.st_ino
):
raise OSError(
errno.ENOTDIR, "Download file changed between creation and open"
)
def _validate_safe_cache_dir(project_root: Path) -> Path:
"""Create and validate the extension URL download cache one component at a
time, refusing symlinked/junctioned components on every supported platform."""
download_dir = project_root.joinpath(*_CACHE_REL_PARTS)
try:
if _has_secure_dir_fd():
_validate_cache_dir_via_dir_fd(project_root, download_dir)
else:
_validate_cache_dir_via_paths(project_root, download_dir)
except typer.Exit:
raise
except FileExistsError:
console.print(
"[red]Error:[/red] Refusing to use symlinked download cache directory"
)
raise typer.Exit(1)
except OSError as exc:
if _is_symlink_refusal_errno(exc):
console.print(
"[red]Error:[/red] Refusing to use symlinked download cache directory"
)
raise typer.Exit(1)
console.print(
"[red]Error:[/red] Could not prepare download cache directory: "
f"{_escape_markup(str(exc))}"
)
raise typer.Exit(1)
return download_dir
def _validate_cache_dir_via_dir_fd(project_root: Path, download_dir: Path) -> None:
"""POSIX cache-dir walk anchored on ``dir_fd`` + ``O_NOFOLLOW`` descriptors."""
o_nofollow = getattr(os, "O_NOFOLLOW", 0)
o_directory = getattr(os, "O_DIRECTORY", 0)
o_cloexec = getattr(os, "O_CLOEXEC", 0)
walk_flags = os.O_RDONLY | o_directory | o_nofollow | o_cloexec
project_root_resolved = project_root.resolve()
parent_fd = os.open(project_root, walk_flags)
current_path = project_root
try:
for part in _CACHE_REL_PARTS:
current_path = current_path / part
try:
child_fd = os.open(part, walk_flags, dir_fd=parent_fd)
except FileNotFoundError:
try:
os.mkdir(part, dir_fd=parent_fd)
except FileExistsError:
pass
child_fd = os.open(part, walk_flags, dir_fd=parent_fd)
try:
current_path.resolve().relative_to(project_root_resolved)
except (OSError, ValueError):
try:
os.close(child_fd)
except OSError:
pass
console.print(
"[red]Error:[/red] Download cache directory escapes project root"
)
raise typer.Exit(1)
os.close(parent_fd)
parent_fd = child_fd
finally:
if parent_fd >= 0:
try:
os.close(parent_fd)
except OSError:
pass
def _validate_cache_dir_via_paths(project_root: Path, download_dir: Path) -> None:
"""Portable cache-dir walk for platforms without ``dir_fd`` (e.g. Windows).
Each component is created individually while a symlink/junction is rejected
both before and after creation, and every component is required to resolve
back under the project root so a mount-point alias or reparse point cannot
redirect the cache outside the project.
"""
project_root_resolved = project_root.resolve()
current_path = project_root
for part in _CACHE_REL_PARTS:
current_path = current_path / part
if current_path.is_symlink():
console.print(
"[red]Error:[/red] Refusing to use symlinked download cache directory"
)
raise typer.Exit(1)
try:
current_path.mkdir()
except FileExistsError:
pass
# Re-check after creation: a component swapped for a symlink/junction
# (or an existing non-directory) between the check and mkdir is caught
# here before the walk descends into it.
if current_path.is_symlink() or not current_path.is_dir():
console.print(
"[red]Error:[/red] Refusing to use symlinked download cache directory"
)
raise typer.Exit(1)
try:
current_path.resolve().relative_to(project_root_resolved)
except (OSError, ValueError):
console.print(
"[red]Error:[/red] Download cache directory escapes project root"
)
raise typer.Exit(1)
def _safe_open_download_zip(
project_root: Path, download_dir: Path, zip_filename: str
) -> int:
"""Exclusively create a download ZIP and return an owned descriptor.
The archive never persists as a nameable on-disk file: the POSIX path
unlinks the leaf immediately after exclusive creation (anonymous inode),
while the portable path opens it with ``O_TEMPORARY`` so the OS deletes it
when the last handle closes. Installation proceeds entirely through the
returned descriptor, removing the pathname-reopen and cleanup-walk TOCTOU
classes on every supported platform.
"""
if _has_secure_dir_fd():
return _open_download_zip_via_dir_fd(
project_root, download_dir, zip_filename
)
return _open_download_zip_via_paths(project_root, download_dir, zip_filename)
def _open_download_zip_via_dir_fd(
project_root: Path, download_dir: Path, zip_filename: str
) -> int:
"""POSIX leaf create: descriptor walk, ``O_EXCL`` create, immediate unlink."""
o_nofollow = getattr(os, "O_NOFOLLOW", 0)
o_directory = getattr(os, "O_DIRECTORY", 0)
o_cloexec = getattr(os, "O_CLOEXEC", 0)
walk_flags = os.O_RDONLY | o_directory | o_nofollow | o_cloexec
rel_parts = download_dir.relative_to(project_root).parts
parent_fd = os.open(project_root, walk_flags)
try:
for part in rel_parts:
new_fd = os.open(part, walk_flags, dir_fd=parent_fd)
os.close(parent_fd)
parent_fd = new_fd
download_fd = os.open(
zip_filename,
os.O_RDWR | os.O_CREAT | os.O_EXCL | o_nofollow | o_cloexec,
0o600,
dir_fd=parent_fd,
)
try:
os.unlink(zip_filename, dir_fd=parent_fd)
except OSError:
os.close(download_fd)
raise
return download_fd
finally:
os.close(parent_fd)
def _open_download_zip_via_paths(
project_root: Path, download_dir: Path, zip_filename: str
) -> int:
"""Portable leaf create for platforms without ``dir_fd`` (e.g. Windows).
The cache directory is re-validated (real directory, under the project
root) immediately before an exclusive create. ``O_EXCL`` guarantees an
attacker cannot pre-stage the leaf as a symlink/junction, ``O_TEMPORARY``
makes the OS delete it on close, and a post-open inode-identity check
detects a leaf swapped underneath us. The returned descriptor is the only
handle installation ever uses, so the cache pathname is never reopened.
"""
zip_path = download_dir / zip_filename
project_root_resolved = project_root.resolve()
if download_dir.is_symlink() or not download_dir.is_dir():
raise OSError(
errno.ENOTDIR, "Download cache directory is not a real directory"
)
try:
download_dir.resolve().relative_to(project_root_resolved)
except (OSError, ValueError):
raise OSError(errno.ENOTDIR, "Download cache directory escapes project root")
if zip_path.is_symlink():
raise OSError(errno.ELOOP, "Refusing to write through a symlinked download file")
flags = os.O_RDWR | os.O_CREAT | os.O_EXCL
flags |= getattr(os, "O_NOFOLLOW", 0)
flags |= getattr(os, "O_CLOEXEC", 0)
flags |= getattr(os, "O_BINARY", 0)
o_temporary = getattr(os, "O_TEMPORARY", 0)
flags |= o_temporary
download_fd = os.open(zip_path, flags, 0o600)
try:
_verify_leaf_identity(download_fd, zip_path)
except OSError:
os.close(download_fd)
# Without O_TEMPORARY the leaf is not auto-deleted, so remove the file
# we just exclusively created (best effort, never through a symlink).
if not o_temporary:
try:
if not zip_path.is_symlink():
zip_path.unlink()
except OSError:
pass
raise
return download_fd
@extension_app.command("add")
def extension_add(
extension: str = typer.Argument(help="Extension name or path"),
@@ -538,69 +945,19 @@ def extension_add(
)
elif from_url:
# Install from URL (ZIP file)
import io
import urllib.error
# Install from URL archive via the shared hardened downloader
# (HTTPS enforcement, authenticated redirect-guarded fetch,
# bounded read, archive-format detection, TOCTOU-safe transient
# archive). Same path used by ``specify init --extension <url>``.
console.print(f"Downloading from {safe_url}...")
# Download ZIP to temp location
download_dir = project_root / ".specify" / "extensions" / ".cache" / "downloads"
download_dir.mkdir(parents=True, exist_ok=True)
with tempfile.NamedTemporaryFile(
prefix="extension-url-download-",
suffix=".zip",
dir=download_dir,
delete=False,
) as download_file:
zip_path = Path(download_file.name)
try:
# Use the catalog's authenticated fetch so configured
# credentials (incl. GitHub Enterprise Server) are applied
# and GHES release-asset URLs resolve via /api/v3 — keeping
# --from consistent with catalog-based installs.
dl_catalog = ExtensionCatalog(project_root)
download_url = from_url
extra_headers = None
resolved_url = dl_catalog._resolve_github_release_asset_api_url(download_url)
if resolved_url:
download_url = resolved_url
extra_headers = {"Accept": "application/octet-stream"}
with dl_catalog._open_url(
download_url, timeout=60, extra_headers=extra_headers
) as response:
zip_data = read_response_limited(
response,
error_type=ExtensionError,
label=f"extension {from_url}",
)
if not zipfile.is_zipfile(io.BytesIO(zip_data)):
console.print(
f"[red]Error:[/red] {safe_url} did not return a ZIP archive "
f"(got {len(zip_data)} bytes). This usually means the request "
f"was not authenticated and a login/HTML page was returned. "
f"Verify the URL is correct and that credentials for its host "
f"are configured in ~/.specify/auth.json."
)
raise typer.Exit(1)
zip_path.write_bytes(zip_data)
# Install from downloaded ZIP
manifest = manager.install_from_zip(zip_path, speckit_version, priority=priority, force=force)
except urllib.error.URLError as e:
console.print(
f"[red]Error:[/red] Failed to download from {safe_url}: "
f"{_escape_markup(str(e))}"
)
raise typer.Exit(1)
finally:
# Clean up downloaded ZIP
if zip_path.exists():
zip_path.unlink()
manifest = install_extension_from_url(
manager,
project_root,
from_url,
speckit_version,
priority=priority,
force=force,
)
else:
# Try bundled extensions first (shipped with spec-kit)
@@ -660,18 +1017,21 @@ def extension_add(
)
raise typer.Exit(1)
# Download extension ZIP (use resolved ID, not original argument which may be display name)
# Download extension archive (use the resolved catalog ID).
extension_id = ext_info['id']
console.print(f"Downloading {_escape_markup(str(ext_info['name']))} v{_escape_markup(str(ext_info.get('version', 'unknown')))}...")
zip_path = catalog.download_extension(extension_id)
archive_path = catalog.download_extension(extension_id)
try:
# Install from downloaded ZIP
manifest = manager.install_from_zip(zip_path, speckit_version, priority=priority, force=force)
manifest = manager.install_from_zip(
archive_path,
speckit_version,
priority=priority,
force=force,
)
finally:
# Clean up downloaded ZIP
if zip_path.exists():
zip_path.unlink()
if archive_path.exists():
archive_path.unlink()
console.print("\n[green]✓[/green] Extension installed successfully!")
console.print(f"\n[bold]{_escape_markup(str(manifest.name))}[/bold] (v{_escape_markup(str(manifest.version))})")
@@ -882,7 +1242,7 @@ def extension_search(
console.print(f"\n [yellow]⚠[/yellow] Not directly installable from '{catalog_name}'.")
console.print(
f" Add to an approved catalog with install_allowed: true, "
f"or install from a ZIP URL: specify extension add {safe_id} --from <zip-url>"
f"or install from an archive URL: specify extension add {safe_id} --from <archive-url>"
)
console.print()
@@ -1510,131 +1870,105 @@ def extension_update(
backup_hooks[hook_name] = ext_hooks
# 5. Download new version
zip_path = catalog.download_extension(extension_id)
archive_path = catalog.download_extension(extension_id)
try:
# 6. Validate extension ID from ZIP BEFORE modifying installation
# Handle both root-level and nested extension.yml (GitHub auto-generated ZIPs)
with open_zip_bounded(zip_path) as zf:
import yaml
manifest_data = None
manifest_bytes = None
namelist = zf.namelist()
# Read the manifest under a hard size cap: this happens
# before install_from_zip()'s safe_extract_zip(), so a
# raw zf.open().read() here would bypass that bound and
# let a zip-bomb extension.yml exhaust memory.
# Normalize separators before choosing the manifest so
# this pre-scan cannot approve one entry while extraction
# later overwrites it with a backslash alias.
manifest_candidates = []
archive_entries = []
for name in namelist:
normalized_name = normalize_zip_member_name(name)
parts = normalized_name.removesuffix("/").split(
"/"
)
path_key = portable_zip_path_key(normalized_name)
archive_entries.append(
(normalized_name, parts)
)
# 6. Validate the archive and extension ID before modifying
# the existing installation. The shared extractor applies
# the same bounded security checks to ZIP and tar archives.
with tempfile.TemporaryDirectory(
prefix="speckit-update-archive-"
) as archive_tmpdir:
extracted_root = Path(archive_tmpdir)
try:
safe_extract_archive(archive_path, extracted_root)
except ValueError as exc:
if (
len(parts) in {1, 2}
and path_key[-1] == "extension.yml"
"Conflicting path" in str(exc)
and "extension.yml" in str(exc).casefold()
):
manifest_candidates.append(
(name, normalized_name, path_key)
)
seen_manifest_keys = {}
for name, _normalized_name, path_key in manifest_candidates:
previous = seen_manifest_keys.get(path_key)
if previous is not None:
raise ValueError(
"Downloaded extension archive contains multiple "
"extension.yml manifests"
)
seen_manifest_keys[path_key] = name
for _name, normalized_name, _path_key in manifest_candidates:
if normalized_name.split("/")[-1] != "extension.yml":
raise ValueError(
"Downloaded extension archive manifest "
"filenames must use canonical "
"'extension.yml' casing"
)
root_manifest = next(
) from exc
raise
manifest_root = extracted_root
top_level = list(extracted_root.iterdir())
root_manifest_entries = [
entry
for entry in top_level
if entry.name.casefold() == "extension.yml"
]
if any(
entry.name != "extension.yml"
for entry in root_manifest_entries
):
raise ValueError(
"Archive must use canonical 'extension.yml' casing"
)
canonical_root_manifest = next(
(
name
for name, _normalized_name, path_key
in manifest_candidates
if path_key == ("extension.yml",)
entry
for entry in root_manifest_entries
if entry.name == "extension.yml"
),
None,
)
nested_manifests = [
(name, normalized_name)
for name, normalized_name, path_key
in manifest_candidates
if len(path_key) == 2
and path_key[-1] == "extension.yml"
]
manifest_path = root_manifest
if manifest_path is None and len(nested_manifests) == 1:
manifest_path, normalized_manifest_path = (
nested_manifests[0]
)
manifest_root = normalized_manifest_path.split(
"/", 1
)[0]
top_level_dirs = {
parts[0]
for normalized_name, parts in archive_entries
if (
len(parts) > 1
or normalized_name.endswith("/")
)
}
if top_level_dirs != {manifest_root}:
if canonical_root_manifest is not None:
manifest_path = canonical_root_manifest
else:
top_level_dirs = [
entry for entry in top_level if entry.is_dir()
]
if len(top_level_dirs) != 1:
raise ValueError(
"Downloaded extension archive with a "
"nested extension.yml must contain exactly "
"Downloaded extension archive must contain exactly "
"one top-level directory"
)
if manifest_path is not None:
manifest_bytes = read_zip_member_limited(
zf, manifest_path
manifest_root = top_level_dirs[0]
nested_manifest_entries = [
entry
for entry in manifest_root.iterdir()
if entry.name.casefold() == "extension.yml"
]
if any(
entry.name != "extension.yml"
for entry in nested_manifest_entries
):
raise ValueError(
"Archive must use canonical 'extension.yml' casing"
)
manifest_path = next(
(
entry
for entry in nested_manifest_entries
if entry.name == "extension.yml"
),
manifest_root / "extension.yml",
)
parsed_manifest = yaml.safe_load(
manifest_bytes
if not manifest_path.is_file():
raise ValueError(
"Downloaded extension archive is missing 'extension.yml'"
)
manifest_data = (
parsed_manifest
if parsed_manifest is not None
else {}
)
if manifest_data is None:
raise ValueError("Downloaded extension archive is missing 'extension.yml'")
manifest_bytes = manifest_path.read_bytes()
parsed_manifest = yaml.safe_load(manifest_bytes)
manifest_data = (
parsed_manifest if parsed_manifest is not None else {}
)
if not isinstance(manifest_data, dict):
raise ValueError(
"Invalid extension manifest in downloaded archive: expected YAML mapping"
"Invalid extension manifest in downloaded archive: "
"expected YAML mapping"
)
extension_data = manifest_data.get("extension", {})
if not isinstance(extension_data, dict):
raise ValueError(
"Invalid extension manifest in downloaded archive: expected 'extension' mapping"
"Invalid extension manifest in downloaded archive: "
"expected 'extension' mapping"
)
# Run the same manifest and compatibility validation as a
# normal install while the existing extension is still
# untouched. Reuse the exact bounded bytes selected above.
if manifest_bytes is None:
raise ValueError(
"Downloaded extension archive is missing 'extension.yml'"
)
with tempfile.TemporaryDirectory(
prefix="speckit-update-manifest-"
) as manifest_tmpdir:
@@ -1830,7 +2164,7 @@ def extension_update(
manager.remove(extension_id, keep_config=True)
# 8. Install new version
_ = manager.install_from_zip(zip_path, speckit_version)
_ = manager.install_from_zip(archive_path, speckit_version)
# Restore user config files from backup after successful install.
new_extension_dir = manager.extensions_dir / extension_id
@@ -1876,12 +2210,12 @@ def extension_update(
hook["enabled"] = False
hook_executor.save_project_config(config)
finally:
# ZIP cleanup is housekeeping: never replace an install
# Archive cleanup is housekeeping: never replace an install
# error or roll back an already committed update because a
# scanner temporarily locks the download on Windows.
if zip_path.exists():
if archive_path.exists():
try:
zip_path.unlink()
archive_path.unlink()
except OSError as error:
zip_cleanup_error = error

View File

@@ -471,6 +471,10 @@ class IntegrationManifest:
path = inst.manifest_path
try:
data = json.loads(path.read_text(encoding="utf-8"))
except UnicodeDecodeError as exc:
raise ValueError(
f"Integration manifest at {path} is not valid UTF-8"
) from exc
except json.JSONDecodeError as exc:
raise ValueError(
f"Integration manifest at {path} contains invalid JSON"

View File

@@ -27,11 +27,14 @@ from packaging import version as pkg_version
from packaging.specifiers import SpecifierSet, InvalidSpecifier
from .._download_security import (
archive_format_from_name,
archive_suffix,
MAX_JSON_CATALOG_BYTES,
build_safe_download_path,
detect_archive_format,
is_https_or_localhost_http,
read_response_limited,
safe_extract_zip,
safe_extract_archive,
)
from ..extensions import REINSTALL_COMMAND, ExtensionRegistry, normalize_priority
from .._init_options import (
@@ -296,6 +299,12 @@ class PresetManifest:
f"(expected {self.SCHEMA_VERSION})"
)
for section in ("preset", "requires", "provides"):
if not isinstance(self.data[section], dict):
raise PresetValidationError(
f"Invalid {section}: expected a mapping"
)
# Validate preset metadata
pack = self.data["preset"]
for field in ["id", "name", "version", "description"]:
@@ -3359,6 +3368,7 @@ class PresetManager:
source_dir: Path,
speckit_version: str,
priority: int = 10,
force: bool = False,
) -> PresetManifest:
"""Install preset from a local directory.
@@ -3366,6 +3376,7 @@ class PresetManager:
source_dir: Path to preset directory
speckit_version: Current spec-kit version
priority: Resolution priority (lower = higher precedence, default 10)
force: If True and the preset is already installed, remove it first
Returns:
Installed preset manifest
@@ -3384,10 +3395,12 @@ class PresetManager:
self.check_compatibility(manifest, speckit_version)
if self.registry.is_installed(manifest.id):
raise PresetError(
f"Preset '{manifest.id}' is already installed. "
f"Use 'specify preset remove {manifest.id}' first."
)
if not force:
raise PresetError(
f"Preset '{manifest.id}' is already installed. "
f"Use 'specify preset remove {manifest.id}' first."
)
self.remove(manifest.id)
dest_dir = self.presets_dir / manifest.id
if dest_dir.exists():
@@ -3530,18 +3543,20 @@ class PresetManager:
return
_materialize_constitution_template(self.project_root, memory_constitution)
def install_from_zip(
def install_from_archive(
self,
zip_path: Path,
archive_path: Path,
speckit_version: str,
priority: int = 10,
force: bool = False,
) -> PresetManifest:
"""Install preset from ZIP file.
"""Install a preset from a supported archive.
Args:
zip_path: Path to preset ZIP file
archive_path: Path to a .zip, .tar.gz, or .tgz archive
speckit_version: Current spec-kit version
priority: Resolution priority (lower = higher precedence, default 10)
force: If True and the preset is already installed, remove it first
Returns:
Installed preset manifest
@@ -3557,7 +3572,11 @@ class PresetManager:
with tempfile.TemporaryDirectory() as tmpdir:
temp_path = Path(tmpdir)
safe_extract_zip(zip_path, temp_path, error_type=PresetValidationError)
safe_extract_archive(
archive_path,
temp_path,
error_type=PresetValidationError,
)
pack_dir = temp_path
manifest_path = pack_dir / "preset.yml"
@@ -3570,10 +3589,25 @@ class PresetManager:
if not manifest_path.exists():
raise PresetValidationError(
"No preset.yml found in ZIP file"
"No preset.yml found in archive"
)
return self.install_from_directory(pack_dir, speckit_version, priority)
return self.install_from_directory(pack_dir, speckit_version, priority, force=force)
def install_from_zip(
self,
zip_path: Path,
speckit_version: str,
priority: int = 10,
force: bool = False,
) -> PresetManifest:
"""Backward-compatible wrapper for archive installation."""
return self.install_from_archive(
zip_path,
speckit_version,
priority,
force=force,
)
def remove(self, pack_id: str) -> bool:
"""Remove an installed preset.
@@ -4599,14 +4633,14 @@ class PresetCatalog:
def download_pack(
self, pack_id: str, target_dir: Optional[Path] = None
) -> Path:
"""Download preset ZIP from catalog.
"""Download a preset archive from a catalog.
Args:
pack_id: ID of the preset to download
target_dir: Directory to save ZIP file (defaults to cache directory)
target_dir: Directory to save the archive
Returns:
Path to downloaded ZIP file
Path to the downloaded archive
Raises:
PresetError: If pack not found or download fails
@@ -4675,42 +4709,86 @@ class PresetCatalog:
target_dir = self.cache_dir / "downloads"
target_dir = Path(target_dir)
version = pack_info.get("version", "unknown")
zip_path = build_safe_download_path(
declared_format = archive_format_from_name(download_url)
build_safe_download_path(
target_dir,
pack_id,
version,
error_type=PresetError,
label="preset",
suffix=archive_suffix(declared_format or "tar.gz"),
)
target_dir.mkdir(parents=True, exist_ok=True)
original_download_url = download_url
extra_headers = None
resolved_download_url = self._resolve_github_release_asset_api_url(download_url)
if resolved_download_url:
download_url = resolved_download_url
extra_headers = {"Accept": "application/octet-stream"}
staging_path: Path | None = None
try:
with self._open_url(download_url, timeout=60, extra_headers=extra_headers) as response:
zip_data = read_response_limited(
archive_data = read_response_limited(
response,
error_type=PresetError,
label=f"preset '{pack_id}' download",
)
final_url = (
response.geturl()
if hasattr(response, "geturl")
else download_url
)
content_type = (
response.getheader("Content-Type")
if hasattr(response, "getheader")
else None
)
verify_archive_sha256(
zip_data, pack_info.get("sha256"), pack_id, PresetError
archive_data, pack_info.get("sha256"), pack_id, PresetError
)
zip_path.write_bytes(zip_data)
return zip_path
with tempfile.NamedTemporaryFile(
prefix="preset-download-",
suffix=".archive",
dir=target_dir,
delete=False,
) as staging_file:
staging_path = Path(staging_file.name)
staging_file.write(archive_data)
archive_format = detect_archive_format(
staging_path,
source_name=(
final_url
if archive_format_from_name(final_url) is not None
else original_download_url
),
content_type=content_type,
error_type=PresetError,
)
archive_path = build_safe_download_path(
target_dir,
pack_id,
version,
error_type=PresetError,
label="preset",
suffix=archive_suffix(archive_format),
)
os.replace(staging_path, archive_path)
staging_path = None
return archive_path
except urllib.error.URLError as e:
raise PresetError(
f"Failed to download preset from {download_url}: {e}"
)
except IOError as e:
raise PresetError(f"Failed to save preset ZIP: {e}")
raise PresetError(f"Failed to save preset archive: {e}")
finally:
if staging_path is not None:
staging_path.unlink(missing_ok=True)
def clear_cache(self):
"""Clear all catalog cache files, including per-URL hashed caches."""

View File

@@ -17,6 +17,9 @@ from rich.markup import escape as _escape_markup
from .._console import console
from .._download_security import (
archive_format_from_name,
archive_suffix,
detect_archive_format,
is_https_or_localhost_http,
is_safe_download_redirect,
read_response_limited,
@@ -75,7 +78,11 @@ def preset_list():
@preset_app.command("add")
def preset_add(
preset_id: str = typer.Argument(None, help="Preset ID to install from catalog"),
from_url: str = typer.Option(None, "--from", help="Install from a URL (ZIP file)"),
from_url: str = typer.Option(
None,
"--from",
help="Install from a .zip, .tar.gz, or .tgz URL",
),
dev: str = typer.Option(None, "--dev", help="Install from local directory (development mode)"),
priority: int = typer.Option(10, "--priority", help="Resolution priority (lower = higher precedence, default 10)"),
):
@@ -142,7 +149,7 @@ def preset_add(
import tempfile
with tempfile.TemporaryDirectory() as tmpdir:
zip_path = Path(tmpdir) / "preset.zip"
archive_path = Path(tmpdir) / "preset.archive"
try:
from specify_cli.authentication.http import open_url as _open_url
from specify_cli.authentication.http import github_provider_hosts
@@ -170,13 +177,33 @@ def preset_add(
"or HTTP for localhost (127.0.0.1, ::1)."
)
raise typer.Exit(1)
zip_path.write_bytes(
read_response_limited(
response,
error_type=PresetError,
label=f"preset {from_url}",
)
archive_data = read_response_limited(
response,
error_type=PresetError,
label=f"preset {from_url}",
)
content_type = (
response.getheader("Content-Type")
if hasattr(response, "getheader")
else None
)
archive_path.write_bytes(archive_data)
format_source = (
final_url
if archive_format_from_name(final_url) is not None
else from_url
)
archive_format = detect_archive_format(
archive_path,
source_name=format_source,
content_type=content_type,
error_type=PresetError,
)
detected_path = archive_path.with_suffix(
archive_suffix(archive_format)
)
os.replace(archive_path, detected_path)
archive_path = detected_path
except (urllib.error.URLError, PresetError) as e:
console.print(
f"[red]Error:[/red] Failed to download: "
@@ -184,7 +211,11 @@ def preset_add(
)
raise typer.Exit(1)
manifest = manager.install_from_zip(zip_path, speckit_version, priority)
manifest = manager.install_from_zip(
archive_path,
speckit_version,
priority,
)
console.print(f"[green]✓[/green] Preset '{manifest.name}' v{manifest.version} installed (priority {priority})")
@@ -227,12 +258,16 @@ def preset_add(
console.print(f"Installing preset [cyan]{pack_info.get('name', preset_id)}[/cyan]...")
try:
zip_path = catalog.download_pack(preset_id)
manifest = manager.install_from_zip(zip_path, speckit_version, priority)
archive_path = catalog.download_pack(preset_id)
manifest = manager.install_from_zip(
archive_path,
speckit_version,
priority,
)
console.print(f"[green]✓[/green] Preset '{manifest.name}' v{manifest.version} installed (priority {priority})")
finally:
if 'zip_path' in locals() and zip_path.exists():
zip_path.unlink(missing_ok=True)
if 'archive_path' in locals() and archive_path.exists():
archive_path.unlink(missing_ok=True)
else:
console.print("[red]Error:[/red] Specify a preset ID, --from URL, or --dev path")
raise typer.Exit(1)

View File

@@ -21,10 +21,17 @@ from rich.markup import escape as _escape_markup
from .._console import console, err_console
from .._download_security import (
archive_format_from_content_type,
archive_format_from_name,
archive_suffix,
detect_archive_format,
is_https_or_localhost_http,
is_safe_download_redirect,
read_response_limited,
safe_extract_archive,
)
from .._project import _resolve_init_dir_override
from ..shared_infra import verify_archive_sha256
workflow_app = typer.Typer(
name="workflow",
@@ -455,6 +462,42 @@ def _read_response_within_limit(response, max_bytes: int | None = None) -> bytes
return b"".join(chunks)
def _workflow_yaml_is_declared(
source_name: str, content_type: str | None
) -> bool:
"""Return whether response metadata explicitly identifies workflow YAML."""
from urllib.parse import urlparse
path = urlparse(source_name).path.casefold()
media_type = (content_type or "").split(";", 1)[0].strip().casefold()
return path.endswith((".yml", ".yaml")) or media_type in {
"application/yaml",
"application/x-yaml",
"text/yaml",
"text/x-yaml",
}
def _sniff_workflow_archive_format(data: bytes):
"""Return a supported archive format when suffixless response bytes match."""
from io import BytesIO
try:
return detect_archive_format(
Path("workflow-download"),
archive_file=BytesIO(data),
)
except ValueError:
return None
def _enforce_workflow_yaml_size(data: bytes) -> None:
if len(data) > _MAX_WORKFLOW_YAML_BYTES:
raise ValueError(
f"response exceeds the {_MAX_WORKFLOW_YAML_BYTES}-byte workflow size limit"
)
def _validate_workflow_id_or_exit(workflow_id: str) -> None:
"""Validate that ``workflow_id`` is a safe installed-workflow directory name."""
if (
@@ -879,6 +922,231 @@ def _discard_committed_backup_file(backup_file: Path | None) -> None:
)
def _workflow_package_root(extracted_root: Path) -> Path:
"""Resolve a root-level or single-nested workflow package."""
if (extracted_root / "workflow.yml").is_file():
return extracted_root
entries = list(extracted_root.iterdir())
if (
len(entries) == 1
and entries[0].is_dir()
and not entries[0].is_symlink()
and (entries[0] / "workflow.yml").is_file()
):
return entries[0]
raise ValueError(
"Archive must contain workflow.yml at its root or in exactly one "
"top-level directory"
)
def _validate_local_workflow_package(package_dir: Path) -> None:
"""Reject links and special files before copying a local package."""
import stat
for root, dirnames, filenames in os.walk(package_dir, followlinks=False):
root_path = Path(root)
for name in [*dirnames, *filenames]:
path = root_path / name
mode = path.lstat().st_mode
if stat.S_ISLNK(mode):
raise ValueError(f"Workflow package contains symlink: {path}")
if not stat.S_ISDIR(mode) and not stat.S_ISREG(mode):
raise ValueError(f"Workflow package contains unsupported file: {path}")
def _workflow_package_has_companions(package_dir: Path) -> bool:
"""Return whether a directory contains anything beyond workflow.yml."""
return any(path.name != "workflow.yml" for path in package_dir.iterdir())
def _install_workflow_package(
project_root: Path,
workflows_dir: Path,
package_dir: Path,
source_label: str,
*,
expected_id: str | None = None,
expected_version: str | None = None,
expected_installed_version: str | None = None,
catalog_info: dict[str, Any] | None = None,
) -> None:
"""Validate and atomically install a complete workflow package directory."""
import shutil
import tempfile
from .engine import WorkflowDefinition, validate_workflow
workflow_file = package_dir / "workflow.yml"
try:
_validate_local_workflow_package(package_dir)
workflow_bytes = workflow_file.read_bytes()
definition = WorkflowDefinition.from_string(workflow_bytes.decode("utf-8"))
except (OSError, UnicodeDecodeError, ValueError, yaml.YAMLError) as exc:
console.print(
f"[red]Error:[/red] Invalid workflow package: "
f"{_escape_markup(str(exc))}"
)
raise typer.Exit(1)
errors = validate_workflow(definition)
if errors:
console.print("[red]Error:[/red] Workflow validation failed:")
for error in errors:
console.print(f"{_escape_markup(str(error))}")
raise typer.Exit(1)
if not isinstance(definition.id, str) or not definition.id.strip():
console.print("[red]Error:[/red] Workflow definition has an empty or missing 'id'")
raise typer.Exit(1)
if expected_id is not None and definition.id != expected_id:
console.print(
f"[red]Error:[/red] Workflow ID in YAML "
f"({_escape_markup(repr(definition.id))}) does not match the requested "
f"workflow ID ({_escape_markup(repr(expected_id))})."
)
raise typer.Exit(1)
if expected_version is not None and str(definition.version) != expected_version:
console.print(
f"[red]Error:[/red] Downloaded workflow version "
f"({_escape_markup(str(definition.version))}) does not match the catalog "
f"version ({_escape_markup(expected_version)})."
)
raise typer.Exit(1)
dest_dir = _safe_workflow_id_dir(workflows_dir, definition.id)
staged_dir = Path(
tempfile.mkdtemp(prefix=f".{definition.id}.installing-", dir=workflows_dir)
)
try:
package_root = package_dir.resolve()
def ignore_reserved_package_entries(
source: str, names: list[str]
) -> set[str]:
if Path(source).resolve() == package_root and "overlays" in names:
return {"overlays"}
return set()
shutil.copytree(
package_dir,
staged_dir,
dirs_exist_ok=True,
ignore=ignore_reserved_package_entries,
)
except OSError as exc:
shutil.rmtree(staged_dir, ignore_errors=True)
console.print(
f"[red]Error:[/red] Failed to stage workflow package: "
f"{_escape_markup(str(exc))}"
)
raise typer.Exit(1)
backup_dir: Path | None = None
try:
with _workflow_install_transaction(project_root):
registry = _open_workflow_registry(project_root)
existing = registry.get(definition.id)
if expected_installed_version is not None and (
not isinstance(existing, dict)
or existing.get("source") != "catalog"
or str(existing.get("version")) != expected_installed_version
):
console.print(
f"[yellow]Warning:[/yellow] Workflow "
f"'{_escape_markup(definition.id)}' changed during update; "
"rerun the command."
)
raise typer.Exit(1)
if dest_dir.exists():
backup_dir = Path(
tempfile.mkdtemp(
prefix=f".{definition.id}.backup-",
dir=workflows_dir,
)
)
backup_dir.rmdir()
os.replace(dest_dir, backup_dir)
try:
os.replace(staged_dir, dest_dir)
except BaseException:
if backup_dir is not None:
os.replace(backup_dir, dest_dir)
backup_dir = None
raise
entry = {
"name": definition.name,
"version": definition.version,
"description": definition.description,
"source": source_label,
}
if catalog_info is not None:
entry.update(
{
"source": "catalog",
"catalog_name": catalog_info.get("_catalog_name", ""),
"url": catalog_info.get("url", ""),
}
)
if isinstance(existing, dict) and not existing.get("enabled", True):
entry["enabled"] = False
try:
registry.add(definition.id, entry)
except (OSError, TypeError, ValueError):
failed_dir: Path | None = None
try:
failed_dir = Path(
tempfile.mkdtemp(
prefix=f".{definition.id}.failed-",
dir=workflows_dir,
)
)
failed_dir.rmdir()
os.replace(dest_dir, failed_dir)
if backup_dir is not None:
os.replace(backup_dir, dest_dir)
backup_dir = None
except OSError as rollback_exc:
console.print(
"[yellow]Warning:[/yellow] Failed to fully restore the prior "
f"workflow package: {_escape_markup(str(rollback_exc))}"
)
finally:
if failed_dir is not None and failed_dir.exists():
try:
shutil.rmtree(failed_dir)
except OSError as cleanup_exc:
console.print(
"[yellow]Warning:[/yellow] Could not remove failed "
f"workflow package: {_escape_markup(str(cleanup_exc))}"
)
raise
except typer.Exit:
raise
except (OSError, TypeError, ValueError) as exc:
console.print(
f"[red]Error:[/red] Failed to install workflow package: "
f"{_escape_markup(str(exc))}"
)
raise typer.Exit(1)
finally:
if staged_dir.exists():
shutil.rmtree(staged_dir, ignore_errors=True)
if backup_dir is not None:
try:
shutil.rmtree(backup_dir)
except OSError as exc:
console.print(
"[yellow]Warning:[/yellow] Workflow installed, but its backup "
f"directory could not be removed: {_escape_markup(str(exc))}"
)
console.print(
f"[green]✓[/green] Workflow '{_escape_markup(definition.name)}' "
f"({_escape_markup(definition.id)}) installed"
)
# Root helper re-fetched at call time so test monkeypatching of
# `specify_cli._require_specify_project` keeps working after the move.
def _require_specify_project(*args, **kwargs):
@@ -889,6 +1157,18 @@ def _require_specify_project(*args, **kwargs):
return project_root
def _failed_step_error(state: Any) -> str | None:
"""Terminal error for a failed/aborted run, if any.
Returns the run-level error persisted by the engine at the moment
the run terminated. Returns ``None`` for non-terminal statuses so
the caller can print unconditionally.
"""
if getattr(state.status, "value", state.status) not in ("failed", "aborted"):
return None
return getattr(state, "error", None)
def _workflow_run_payload(state: Any) -> dict[str, Any]:
"""Machine-readable summary of a run/resume outcome."""
payload = {
@@ -901,6 +1181,9 @@ def _workflow_run_payload(state: Any) -> dict[str, Any]:
gate = _gate_outcome(state)
if gate is not None:
payload["gate"] = gate
error = _failed_step_error(state)
if error is not None:
payload["error"] = error
return payload
@@ -1161,6 +1444,10 @@ def workflow_run(
console.print(f"\n[{color}]Status: {state.status.value}[/{color}]")
console.print(f"[dim]Run ID: {state.run_id}[/dim]")
err_msg = _failed_step_error(state)
if err_msg:
console.print(f"[red]Error:[/red] {_escape_markup(err_msg)}")
if state.status.value == "paused":
console.print(f"\nResume with: [cyan]specify workflow resume {state.run_id}[/cyan]")
@@ -1271,6 +1558,10 @@ def workflow_resume(
color = status_colors.get(state.status.value, "white")
console.print(f"\n[{color}]Status: {state.status.value}[/{color}]")
err_msg = _failed_step_error(state)
if err_msg:
console.print(f"[red]Error:[/red] {_escape_markup(err_msg)}")
raise typer.Exit(_run_outcome_exit_code(state.status.value))
@@ -1338,6 +1629,10 @@ def workflow_status(
if state.current_step_id:
console.print(f" Current: {state.current_step_id}")
err_msg = _failed_step_error(state)
if err_msg:
console.print(f" [red]Error: {_escape_markup(err_msg)}[/red]")
if state.step_results:
console.print(f"\n [bold]Steps ({len(state.step_results)}):[/bold]")
for step_id, step_data in state.step_results.items():
@@ -1575,16 +1870,48 @@ def workflow_add(
if dev_path.is_file() and dev_path.suffix.lower() in (".yml", ".yaml"):
_validate_and_install_local(dev_path, str(dev_path))
return
if dev_path.is_file() and archive_format_from_name(str(dev_path)) is not None:
import tempfile
with tempfile.TemporaryDirectory(
prefix="speckit-workflow-archive-"
) as tmpdir:
extracted_root = Path(tmpdir)
try:
safe_extract_archive(dev_path, extracted_root)
package_root = _workflow_package_root(extracted_root)
except ValueError as exc:
console.print(
f"[red]Error:[/red] Invalid workflow archive: "
f"{_escape_markup(str(exc))}"
)
raise typer.Exit(1)
_install_workflow_package(
project_root,
workflows_dir,
package_root,
str(dev_path),
)
return
if dev_path.is_dir():
dev_wf_file = dev_path / "workflow.yml"
if not dev_wf_file.is_file():
console.print(f"[red]Error:[/red] No workflow.yml found in {_escape_markup(source)}")
raise typer.Exit(1)
_validate_and_install_local(dev_wf_file, str(dev_path))
if _workflow_package_has_companions(dev_path):
_install_workflow_package(
project_root,
workflows_dir,
dev_path,
str(dev_path),
)
else:
_validate_and_install_local(dev_wf_file, str(dev_path))
return
console.print(
"[red]Error:[/red] --dev source must be a workflow YAML file or a "
f"directory containing workflow.yml: {_escape_markup(source)}"
"[red]Error:[/red] --dev source must be a workflow YAML file, "
"supported archive, or directory containing workflow.yml: "
f"{_escape_markup(source)}"
)
raise typer.Exit(1)
@@ -1647,6 +1974,7 @@ def workflow_add(
import tempfile
tmp_path: Path | None = None
downloaded_archive_format = None
try:
with _open_url(
download_url,
@@ -1660,13 +1988,48 @@ def workflow_add(
f"[red]Error:[/red] URL redirected to non-HTTPS: {_escape_markup(final_url)}"
)
raise typer.Exit(1)
with tempfile.NamedTemporaryFile(suffix=".yml", delete=False) as tmp:
content_type = (
resp.getheader("Content-Type")
if hasattr(resp, "getheader")
else None
)
downloaded_archive_format = (
archive_format_from_name(final_url)
or archive_format_from_name(download_url)
or archive_format_from_content_type(content_type)
)
declared_yaml = _workflow_yaml_is_declared(final_url, content_type)
suffix = (
archive_suffix(downloaded_archive_format)
if downloaded_archive_format is not None
else ".yml" if declared_yaml else ".download"
)
with tempfile.NamedTemporaryFile(suffix=suffix, delete=False) as tmp:
# Assign tmp_path immediately: NamedTemporaryFile(delete=False)
# creates the file on disk right away, before any bytes are
# written, so a failure in the size-limited read below must
# still be able to find and remove it.
tmp_path = Path(tmp.name)
tmp.write(_read_response_within_limit(resp))
if downloaded_archive_format is not None:
downloaded_content = read_response_limited(
resp,
error_type=ValueError,
label="workflow archive download",
)
elif declared_yaml:
downloaded_content = _read_response_within_limit(resp)
else:
downloaded_content = read_response_limited(
resp,
error_type=ValueError,
label="workflow download",
)
downloaded_archive_format = (
_sniff_workflow_archive_format(downloaded_content)
)
if downloaded_archive_format is None:
_enforce_workflow_yaml_size(downloaded_content)
tmp.write(downloaded_content)
except typer.Exit:
raise
except Exception as exc:
@@ -1686,13 +2049,38 @@ def workflow_add(
console.print(f"[red]Error:[/red] Failed to download workflow: {_escape_markup(str(exc))}")
raise typer.Exit(1)
try:
# When installed via --from, the positional argument names the
# workflow the user expects — enforce it like the catalog branch.
_validate_and_install_local(
tmp_path,
download_url,
expected_id=source if from_url else None,
)
if downloaded_archive_format is None:
_validate_and_install_local(
tmp_path,
download_url,
expected_id=source if from_url else None,
)
else:
with tempfile.TemporaryDirectory(
prefix="speckit-workflow-archive-"
) as extract_dir:
extracted_root = Path(extract_dir)
try:
safe_extract_archive(
tmp_path,
extracted_root,
source_name=final_url,
content_type=content_type,
)
package_root = _workflow_package_root(extracted_root)
except ValueError as exc:
console.print(
f"[red]Error:[/red] Invalid workflow archive: "
f"{_escape_markup(str(exc))}"
)
raise typer.Exit(1)
_install_workflow_package(
project_root,
workflows_dir,
package_root,
download_url,
expected_id=source if from_url else None,
)
finally:
# Best-effort: _validate_and_install_local may already have
# committed the file + registry entry (success) or already
@@ -1716,12 +2104,46 @@ def workflow_add(
if source_path.is_file() and source_path.suffix.lower() in (".yml", ".yaml"):
_validate_and_install_local(source_path, str(source_path))
return
elif (
source_path.is_file()
and archive_format_from_name(str(source_path)) is not None
):
import tempfile
with tempfile.TemporaryDirectory(
prefix="speckit-workflow-archive-"
) as tmpdir:
extracted_root = Path(tmpdir)
try:
safe_extract_archive(source_path, extracted_root)
package_root = _workflow_package_root(extracted_root)
except ValueError as exc:
console.print(
f"[red]Error:[/red] Invalid workflow archive: "
f"{_escape_markup(str(exc))}"
)
raise typer.Exit(1)
_install_workflow_package(
project_root,
workflows_dir,
package_root,
str(source_path),
)
return
elif source_path.is_dir():
wf_file = source_path / "workflow.yml"
if not wf_file.is_file():
console.print(f"[red]Error:[/red] No workflow.yml found in {_escape_markup(source)}")
raise typer.Exit(1)
_validate_and_install_local(wf_file, str(source_path))
if _workflow_package_has_companions(source_path):
_install_workflow_package(
project_root,
workflows_dir,
source_path,
str(source_path),
)
else:
_validate_and_install_local(wf_file, str(source_path))
return
# Try from catalog
@@ -1826,6 +2248,9 @@ def _install_workflow_from_catalog(
)
raise typer.Exit(1)
original_workflow_url = workflow_url
downloaded_archive_format = None
archive_content_type = None
try:
from specify_cli.authentication.http import open_url as _open_url
from specify_cli.authentication.http import github_provider_hosts as _github_provider_hosts
@@ -1857,10 +2282,38 @@ def _install_workflow_from_catalog(
f"[red]Error:[/red] Workflow '{safe_wf_id}' redirected to non-HTTPS URL: {_escape_markup(final_url)}"
)
raise typer.Exit(1)
archive_content_type = (
response.getheader("Content-Type")
if hasattr(response, "getheader")
else None
)
downloaded_archive_format = (
archive_format_from_name(final_url)
or archive_format_from_name(original_workflow_url)
or archive_format_from_content_type(archive_content_type)
)
# Written to the staging file, never workflow_file directly, so a
# reinstall's prior working copy is never touched until the
# atomic commit below runs.
downloaded_content = _read_response_within_limit(response)
if downloaded_archive_format is not None:
downloaded_content = read_response_limited(
response,
error_type=ValueError,
label=f"workflow '{workflow_id}' archive download",
)
elif _workflow_yaml_is_declared(final_url, archive_content_type):
downloaded_content = _read_response_within_limit(response)
else:
downloaded_content = read_response_limited(
response,
error_type=ValueError,
label=f"workflow '{workflow_id}' download",
)
downloaded_archive_format = _sniff_workflow_archive_format(
downloaded_content
)
if downloaded_archive_format is None:
_enforce_workflow_yaml_size(downloaded_content)
staged_file.write_bytes(downloaded_content)
except typer.Exit:
raise
@@ -1869,6 +2322,59 @@ def _install_workflow_from_catalog(
console.print(f"[red]Error:[/red] Failed to install workflow '{safe_wf_id}' from catalog: {_escape_markup(str(exc))}")
raise typer.Exit(1)
if downloaded_archive_format is not None:
try:
verify_archive_sha256(
downloaded_content,
info.get("sha256"),
workflow_id,
ValueError,
)
import tempfile
from io import BytesIO
with tempfile.TemporaryDirectory(
prefix="speckit-workflow-archive-"
) as extract_dir:
extracted_root = Path(extract_dir)
safe_extract_archive(
staged_file.path,
extracted_root,
archive_file=BytesIO(downloaded_content),
source_name=original_workflow_url,
content_type=archive_content_type,
)
package_root = _workflow_package_root(extracted_root)
_safe_discard_staged_workflow_file(
staged_file,
workflow_dir,
existed_before,
)
_install_workflow_package(
project_root,
workflows_dir,
package_root,
workflow_url,
expected_id=workflow_id,
expected_version=expected_version,
expected_installed_version=expected_installed_version,
catalog_info={**info, "url": workflow_url},
)
except typer.Exit:
raise
except (OSError, ValueError) as exc:
_safe_discard_staged_workflow_file(
staged_file,
workflow_dir,
existed_before,
)
console.print(
f"[red]Error:[/red] Invalid workflow archive: "
f"{_escape_markup(str(exc))}"
)
raise typer.Exit(1)
return
# Validate the downloaded workflow (still staged, not yet committed)
# before registering.
try:
@@ -2549,9 +3055,10 @@ def workflow_step_list():
console.print(" [bold]Custom (installed):[/bold]")
for key in sorted(installed):
meta = installed[key] or {}
name = meta.get("name", key)
version = meta.get("version", "?")
console.print(f" • [bold]{name}[/bold] ({key}) v{version}")
name = _escape_markup(str(meta.get("name", key)))
safe_key = _escape_markup(str(key))
version = _escape_markup(str(meta.get("version", "?")))
console.print(f" • [bold]{name}[/bold] ({safe_key}) v{version}")
console.print()
if not built_in and not installed:
@@ -3095,13 +3602,15 @@ def workflow_step_search(
install_note = (
"" if step.get("_install_allowed", True) else " [dim](discovery only)[/dim]"
)
name = _escape_markup(str(step.get("name", step.get("id", "?"))))
step_id = _escape_markup(str(step.get("id", "?")))
version = _escape_markup(str(step.get("version", "?")))
console.print(
f" [bold]{step.get('name', step.get('id', '?'))}[/bold]"
f" ({step.get('id', '?')}) v{step.get('version', '?')}{install_note}"
f" [bold]{name}[/bold] ({step_id}) v{version}{install_note}"
)
desc = step.get("description", "")
if desc:
console.print(f" {desc}")
console.print(f" {_escape_markup(str(desc))}")
console.print()
@@ -3114,6 +3623,7 @@ def workflow_step_info(
from .catalog import StepCatalog, StepCatalogError, StepRegistry
project_root = _require_specify_project()
safe_step_id = _escape_markup(str(step_id))
registry = StepRegistry(project_root)
installed_meta = registry.get(step_id)
@@ -3123,20 +3633,27 @@ def workflow_step_info(
is_builtin = builtin_step is not None and not installed_meta
if is_builtin:
console.print(f"\n[bold cyan]{step_id}[/bold cyan] [dim](built-in)[/dim]")
console.print(f" Type key: {step_id}")
console.print(f"\n[bold cyan]{safe_step_id}[/bold cyan] [dim](built-in)[/dim]")
console.print(f" Type key: {safe_step_id}")
console.print(" [green]Built-in step type[/green]")
return
if installed_meta:
name = _escape_markup(str(installed_meta.get("name", step_id)))
version = _escape_markup(str(installed_meta.get("version", "?")))
console.print(
f"\n[bold cyan]{installed_meta.get('name', step_id)}[/bold cyan] ({step_id})"
f"\n[bold cyan]{name}[/bold cyan] ({safe_step_id})"
)
console.print(f" Version: {installed_meta.get('version', '?')}")
console.print(f" Version: {version}")
if installed_meta.get("author"):
console.print(f" Author: {installed_meta['author']}")
console.print(
f" Author: {_escape_markup(str(installed_meta['author']))}"
)
if installed_meta.get("description"):
console.print(f" Description: {installed_meta['description']}")
console.print(
f" Description: "
f"{_escape_markup(str(installed_meta['description']))}"
)
console.print(" [green]Installed[/green]")
return
@@ -3148,20 +3665,24 @@ def workflow_step_info(
info = None
if info:
name = _escape_markup(str(info.get("name", step_id)))
version = _escape_markup(str(info.get("version", "?")))
console.print(
f"\n[bold cyan]{info.get('name', step_id)}[/bold cyan] ({step_id})"
f"\n[bold cyan]{name}[/bold cyan] ({safe_step_id})"
)
console.print(f" Version: {info.get('version', '?')}")
console.print(f" Version: {version}")
if info.get("author"):
console.print(f" Author: {info['author']}")
console.print(f" Author: {_escape_markup(str(info['author']))}")
if info.get("description"):
console.print(f" Description: {info['description']}")
console.print(
f" Description: {_escape_markup(str(info['description']))}"
)
console.print(" [yellow]Not installed[/yellow]")
console.print(
f"\n Install with: [cyan]specify workflow step add {step_id}[/cyan]"
f"\n Install with: [cyan]specify workflow step add {safe_step_id}[/cyan]"
)
else:
console.print(f"[red]Error:[/red] Step type '{step_id}' not found")
console.print(f"[red]Error:[/red] Step type '{safe_step_id}' not found")
raise typer.Exit(1)

View File

@@ -56,6 +56,9 @@ class StepContext:
#: Current fan-out item (set only inside fan-out iterations).
item: Any = None
#: Whether the current step is executing inside a fan-out template.
inside_fan_out: bool = False
#: Fan-in aggregated results (set only for fan-in steps).
fan_in: dict[str, Any] = field(default_factory=dict)

View File

@@ -495,6 +495,8 @@ class WorkflowCatalog:
try:
with open(meta_file, encoding="utf-8") as f:
meta = json.load(f)
if not isinstance(meta, dict):
return False
fetched_at = float(meta.get("fetched_at", 0))
return (time.time() - fetched_at) < self.CACHE_DURATION
except (json.JSONDecodeError, OSError, TypeError, ValueError):
@@ -509,8 +511,10 @@ class WorkflowCatalog:
if not force_refresh and self._is_url_cache_valid(entry.url):
try:
with open(cache_file, encoding="utf-8") as f:
return json.load(f)
except (json.JSONDecodeError, OSError):
cached = json.load(f)
if isinstance(cached, dict):
return cached
except (UnicodeDecodeError, json.JSONDecodeError, OSError):
# Ignore invalid/unreadable cache and fall back to fetching from source.
pass
@@ -574,7 +578,9 @@ class WorkflowCatalog:
if cache_file.exists():
try:
with open(cache_file, encoding="utf-8") as f:
return json.load(f)
cached = json.load(f)
if isinstance(cached, dict):
return cached
except (json.JSONDecodeError, ValueError, OSError):
# Stale-cache read failed; let the original fetch error propagate.
pass
@@ -1184,6 +1190,8 @@ class StepCatalog:
try:
with open(meta_file, encoding="utf-8") as f:
meta = json.load(f)
if not isinstance(meta, dict):
return False
fetched_at = float(meta.get("fetched_at", 0))
return (time.time() - fetched_at) < self.CACHE_DURATION
except (json.JSONDecodeError, OSError, TypeError, ValueError):
@@ -1202,7 +1210,7 @@ class StepCatalog:
cached = json.load(f)
if isinstance(cached, dict):
return cached
except (json.JSONDecodeError, OSError):
except (UnicodeDecodeError, json.JSONDecodeError, OSError):
# Ignore invalid/unreadable cache and fall back to fetching from source.
pass

View File

@@ -308,7 +308,16 @@ def validate_workflow(definition: WorkflowDefinition) -> list[str]:
errors.append("Workflow has no steps defined.")
seen_ids: set[str] = set()
_validate_steps(definition.steps, seen_ids, errors)
# ``input_defs`` maps declared workflow input names to their definitions —
# used by ``_validate_steps`` to cross-reference gate ``verdict_input``
# bindings (both that the name exists and that its ``enum`` permits the
# reset sentinel). ``None`` means the inputs block itself is malformed
# (already reported above); the cross-check is then disabled so one
# authoring mistake does not cascade into N spurious "undeclared" errors.
input_defs: dict[str, Any] | None = (
dict(definition.inputs) if isinstance(definition.inputs, dict) else None
)
_validate_steps(definition.steps, seen_ids, errors, input_defs)
return errors
@@ -317,8 +326,16 @@ def _validate_steps(
steps: list[dict[str, Any]],
seen_ids: set[str],
errors: list[str],
input_defs: dict[str, Any] | None = None,
inside_fan_out: bool = False,
) -> None:
"""Recursively validate a list of steps."""
"""Recursively validate a list of steps.
``input_defs`` maps declared workflow input names to their definitions (or
is ``None`` when the inputs block is malformed). ``inside_fan_out`` is
threaded through nested control-flow steps so gate verdict bindings can be
rejected anywhere inside a fan-out template.
"""
from . import STEP_REGISTRY
for step_config in steps:
@@ -411,30 +428,101 @@ def _validate_steps(
f"unknown or not-yet-declared step id {wid!r}."
)
# Gate verdict_input: fan-out items cannot bind shared workflow inputs
# as per-item verdicts. Outside fan-out, the binding must reference a
# declared workflow input because ``_resolve_inputs`` drops undeclared
# names at both initial run and resume. Only check a non-empty string;
# malformed shapes are already reported by ``GateStep.validate()``.
if step_type == "gate":
verdict_input = step_config.get("verdict_input")
if isinstance(verdict_input, str) and verdict_input:
if inside_fan_out:
errors.append(
f"Gate step {step_id!r}: 'verdict_input' is not "
"supported inside fan-out templates."
)
elif input_defs is not None and verdict_input not in input_defs:
errors.append(
f"Gate step {step_id!r}: 'verdict_input' references "
f"undeclared input {verdict_input!r}."
)
elif input_defs is not None:
# ``on_reject: retry`` resets the bound input to "" before
# pausing, and every later resume re-resolves the persisted
# inputs through ``_coerce_input``. If the input declares an
# ``enum`` that omits "", that reset value is instantly
# illegal: the run pauses fine, but the next resume that
# supplies any input raises "value '' not in allowed
# values", and no verdict can be routed through the gate
# again. Require the enum to admit the sentinel so the
# retry cycle the field advertises is actually reachable.
verdict_def = input_defs.get(verdict_input)
enum_values = (
verdict_def.get("enum")
if isinstance(verdict_def, dict)
else None
)
if (
step_config.get("on_reject") == "retry"
and isinstance(enum_values, list)
and "" not in enum_values
):
errors.append(
f"Gate step {step_id!r}: on_reject='retry' resets "
f"verdict input {verdict_input!r} to '' when the "
f"gate is rejected, but that input's 'enum' does "
f"not allow ''. Add '' to the enum or use "
f"on_reject='abort'/'skip'."
)
# Recursively validate nested steps
for nested_key in ("then", "else", "steps"):
nested = step_config.get(nested_key)
if isinstance(nested, list):
_validate_steps(nested, seen_ids, errors)
_validate_steps(
nested,
seen_ids,
errors,
input_defs,
inside_fan_out=inside_fan_out,
)
# Validate switch cases
cases = step_config.get("cases")
if isinstance(cases, dict):
for _case_key, case_steps in cases.items():
if isinstance(case_steps, list):
_validate_steps(case_steps, seen_ids, errors)
_validate_steps(
case_steps,
seen_ids,
errors,
input_defs,
inside_fan_out=inside_fan_out,
)
# Validate switch default
default = step_config.get("default")
if isinstance(default, list):
_validate_steps(default, seen_ids, errors)
_validate_steps(
default,
seen_ids,
errors,
input_defs,
inside_fan_out=inside_fan_out,
)
# Validate fan-out nested step (template — not added to seen_ids
# since the engine generates parentId:templateId:index at runtime)
fan_step = step_config.get("step")
if isinstance(fan_step, dict):
fan_errors: list[str] = []
_validate_steps([fan_step], set(), fan_errors)
_validate_steps(
[fan_step],
set(),
fan_errors,
input_defs,
inside_fan_out=True,
)
errors.extend(fan_errors)
@@ -560,6 +648,7 @@ class RunState:
self.created_at = datetime.now(timezone.utc).isoformat()
self.updated_at = self.created_at
self.log_entries: list[dict[str, Any]] = []
self.error: str | None = None
@property
def runs_dir(self) -> Path:
@@ -614,6 +703,7 @@ class RunState:
"workflow_dir": self.workflow_dir,
"created_at": self.created_at,
"updated_at": self.updated_at,
"error": self.error,
}
self._atomic_write_json(runs_dir / "state.json", state_data)
self._atomic_write_json(runs_dir / "inputs.json", {"inputs": self.inputs})
@@ -707,6 +797,7 @@ class RunState:
state.workflow_dir = state_data.get("workflow_dir")
state.created_at = state_data.get("created_at", "")
state.updated_at = state_data.get("updated_at", "")
state.error = state_data.get("error")
inputs_path = runs_dir / "inputs.json"
if inputs_path.exists():
@@ -901,6 +992,7 @@ class WorkflowEngine:
return state
except Exception as exc:
state.status = RunStatus.FAILED
state.error = str(exc)
state.append_log({"event": "workflow_failed", "error": str(exc)})
state.save()
raise
@@ -959,6 +1051,7 @@ class WorkflowEngine:
from . import STEP_REGISTRY
state.error = None
state.status = RunStatus.RUNNING
state.save()
@@ -979,6 +1072,7 @@ class WorkflowEngine:
return state
except Exception as exc:
state.status = RunStatus.FAILED
state.error = str(exc)
state.append_log({"event": "resume_failed", "error": str(exc)})
state.save()
raise
@@ -1038,6 +1132,7 @@ class WorkflowEngine:
step_impl = registry.get(step_type)
if not step_impl:
state.status = RunStatus.FAILED
state.error = f"Unknown step type: {step_type!r}"
state.append_log(
{
"event": "step_failed",
@@ -1065,6 +1160,7 @@ class WorkflowEngine:
or step_config.get("input", {}),
"output": result.output,
"status": result.status.value,
"error": result.error,
}
self._record_result(context, state, step_id, step_data)
@@ -1090,6 +1186,7 @@ class WorkflowEngine:
# is for transient/expected step failures only.
if result.output.get("aborted"):
state.status = RunStatus.ABORTED
state.error = result.error
state.append_log(
{
"event": "workflow_aborted",
@@ -1132,6 +1229,7 @@ class WorkflowEngine:
continue
state.status = RunStatus.FAILED
state.error = result.error
state.append_log(
{
"event": "step_failed",
@@ -1305,11 +1403,18 @@ class WorkflowEngine:
# Sequential path — identical to the historical behavior.
if workers <= 1:
results: list[Any] = []
for item_idx, item_val in enumerate(items):
context.item = item_val
results.append(run_item(item_idx, context))
if state.status in halting:
break
previous_item = context.item
previous_inside_fan_out = context.inside_fan_out
context.inside_fan_out = True
try:
for item_idx, item_val in enumerate(items):
context.item = item_val
results.append(run_item(item_idx, context))
if state.status in halting:
break
finally:
context.item = previous_item
context.inside_fan_out = previous_inside_fan_out
return results
# Concurrent path — bounded sliding window; results assembled in item order.
@@ -1320,7 +1425,14 @@ class WorkflowEngine:
# Each item runs against its own context copy so context.item is not
# clobbered across threads; the shared steps dict is written only on the
# disjoint parentId:templateId:index key (GIL-safe on distinct keys).
return run_item(idx, dataclasses.replace(context, item=items[idx]))
return run_item(
idx,
dataclasses.replace(
context,
item=items[idx],
inside_fan_out=True,
),
)
def item_halt_status(idx: int) -> RunStatus | None:
# If THIS item's own execution halted the run, return the resulting run
@@ -1401,6 +1513,16 @@ class WorkflowEngine:
# pool joined; restore the halting item's own outcome so the final run
# status matches the sequential semantics.
state.status = halted_status
# Restore the halting item's error so it matches the terminal
# status — a concurrent item may have overwritten state.error
# before the pool joined. Assign unconditionally when a record
# exists (even when the halting item's own error is falsy) so a
# third-party step returning FAILED with no message never inherits
# an unrelated concurrent item's error; this mirrors the sequential
# path, which sets state.error = result.error verbatim.
halt_rec = context.steps.get(item_id(halted_at))
if isinstance(halt_rec, dict):
state.error = halt_rec.get("error")
return slots[: halted_at + 1]
return slots[:collected]

View File

@@ -7,6 +7,7 @@ from typing import Any
import typer
import yaml
from rich.markup import escape as _escape_markup
from ..._console import console, err_console
from ...extensions import normalize_priority
@@ -412,14 +413,23 @@ def workflow_resolve(project_root: Path, workflow_id: str) -> dict[str, Any] | N
priority = (
"n/a" if layer.tier == "base" else str(normalize_priority(layer.priority))
)
# ``\[`` keeps the literal bracket: unescaped, Rich parses ``[base]`` /
# ``[project-overlay]`` as a style tag and swallows the tier label whole.
console.print(
f" \u2022 [{layer.tier}] {layer.source} "
f" \u2022 \\[{_escape_markup(layer.tier)}] "
f"{_escape_markup(layer.source)} "
f"(priority={priority})"
)
console.print("Step attribution:")
for composed in attribution:
console.print(f" \u2022 {composed.step_id}: {composed.source}")
# Step IDs come from base-workflow / overlay YAML, which only bans ``:``
# \u2014 brackets pass validation, so they reach Rich as markup. A balanced
# ``[stuff]`` is swallowed; an unbalanced ``[/red]`` raises MarkupError.
console.print(
f" \u2022 {_escape_markup(composed.step_id)}: "
f"{_escape_markup(composed.source)}"
)
return {
"workflow_id": workflow_id,

View File

@@ -20,9 +20,31 @@ class FanInStep(StepBase):
def execute(self, config: dict[str, Any], context: StepContext) -> StepResult:
wait_for = config.get("wait_for", [])
output_config = config.get("output") or {}
if not isinstance(output_config, dict):
output_config = config.get("output")
if output_config is None:
output_config = {}
elif not isinstance(output_config, dict):
# ``validate`` rejects a non-mapping ``output`` and its comment says
# why: "execute() silently coerces a non-mapping output to {}, so the
# author's declared aggregation keys would vanish with no error."
# The engine does not auto-validate before ``execute``, so on an
# unvalidated run that is exactly what happened -- and ``x or {}``
# masked the falsy shapes ([], false, 0, '') before the isinstance
# check even ran. Every declared key vanished while the step still
# reported COMPLETED, so downstream ``steps.<id>.output.<key>``
# resolved to None and interpolated as "": the same "silent empty
# result + COMPLETED" wiring bug the ``wait_for`` guard below
# rejects. Fail loudly with validate()'s own message instead. An
# explicit ``output:`` (YAML null) stays valid, matching validate.
return StepResult(
status=StepStatus.FAILED,
error=(
f"Fan-in step {config.get('id', '?')!r}: 'output' must be a "
f"mapping of key -> expression, got "
f"{type(output_config).__name__}."
),
output={"results": []},
)
# The engine does not auto-validate step config, so an unvalidated run
# with a non-list ``wait_for`` reaches here raw. Iterating it then

View File

@@ -26,7 +26,8 @@ class GateStep(StepBase):
later with ``specify workflow resume``.
The user's choice is stored in ``output.choice``. ``on_reject``
controls abort / skip / retry behaviour.
controls abort / skip / retry behaviour. ``verdict_input`` can name a
workflow input to use as the choice when resuming non-interactively.
"""
type_key = "gate"
@@ -42,6 +43,8 @@ class GateStep(StepBase):
options = config.get("options", ["approve", "reject"])
on_reject = config.get("on_reject", "abort")
has_verdict_input = "verdict_input" in config
verdict_input = config.get("verdict_input")
# ``validate`` rejects a non-list (or empty) ``options``, and requires
# every option to be a string, but the engine does not auto-validate
@@ -72,6 +75,51 @@ class GateStep(StepBase):
},
)
# ``validate`` rejects an ``on_reject`` outside abort/skip/retry, but the
# engine does not auto-validate before ``execute``. The reject branch
# below handles only "abort" and "retry" and then falls through to its
# ``on_reject == "skip"`` case, so on an unvalidated run any other value
# makes a REJECTED gate report COMPLETED and the run walks straight past
# the review the gate exists to enforce. Reachable by a capitalisation
# slip ("Abort"), a guessed verb ("fail", "stop"), a non-string, or the
# ``None`` that a bare ``on_reject:`` yields -- note ``config.get(k,
# default)`` does NOT substitute the default for an explicit null. Fail
# loudly instead, mirroring the ``options``/``verdict_input`` guards here.
if on_reject not in ("abort", "skip", "retry"):
return StepResult(
status=StepStatus.FAILED,
error=(
f"Gate step {config.get('id', '?')!r}: 'on_reject' must be "
f"'abort', 'skip', or 'retry', got {on_reject!r}."
),
output={
"message": message,
"options": options,
"on_reject": on_reject,
"choice": None,
},
)
if has_verdict_input and (
not isinstance(verdict_input, str) or not verdict_input
):
return StepResult(
status=StepStatus.FAILED,
error=(
f"Gate step {config.get('id', '?')!r}: 'verdict_input' must be "
"a non-empty string."
),
)
if has_verdict_input and context.inside_fan_out:
return StepResult(
status=StepStatus.FAILED,
error=(
f"Gate step {config.get('id', '?')!r}: 'verdict_input' is "
"not supported inside fan-out templates."
),
)
show_file = config.get("show_file")
if isinstance(show_file, str) and "{{" in show_file:
show_file = evaluate_expression(show_file, context)
@@ -90,16 +138,48 @@ class GateStep(StepBase):
"choice": None,
}
# Non-interactive: pause for later resume (the file is not read here)
if not sys.stdin.isatty():
return StepResult(status=StepStatus.PAUSED, output=output)
choice: str | None = None
bound_verdict_input: str | None = None
if verdict_input is not None:
value = context.inputs.get(verdict_input)
if value is not None and value != "":
if not isinstance(value, str):
return StepResult(
status=StepStatus.FAILED,
output=output,
error=(
f"Gate step {config.get('id', '?')!r}: verdict input "
f"{verdict_input!r} must be a string, got "
f"{type(value).__name__}."
),
)
choice = next(
(option for option in options if option.lower() == value.lower()),
None,
)
if choice is None:
return StepResult(
status=StepStatus.FAILED,
output=output,
error=(
f"Gate step {config.get('id', '?')!r}: verdict input "
f"{verdict_input!r} value {value!r} does not match any "
"configured option."
),
)
bound_verdict_input = verdict_input
# Interactive: prompt the user. ``show_file`` contents are folded
# into the displayed message so the operator can review the
# referenced material before choosing. Composing the prompt text
# here keeps ``_prompt`` to its ``(message, options)`` contract, so
# adding review material never widens the interactive seam.
choice = self._prompt(self._compose_prompt(message, show_file), options)
if choice is None:
# Non-interactive: pause for later resume (the file is not read here)
if not sys.stdin.isatty():
return StepResult(status=StepStatus.PAUSED, output=output)
# Interactive: prompt the user. ``show_file`` contents are folded
# into the displayed message so the operator can review the
# referenced material before choosing. Composing the prompt text
# here keeps ``_prompt`` to its ``(message, options)`` contract, so
# adding review material never widens the interactive seam.
choice = self._prompt(self._compose_prompt(message, show_file), options)
output["choice"] = choice
# Match rejection case-insensitively. ``_prompt`` echoes the option's
@@ -119,6 +199,8 @@ class GateStep(StepBase):
)
if on_reject == "retry":
# Pause so the next resume re-executes this gate
if bound_verdict_input is not None:
context.inputs[bound_verdict_input] = ""
return StepResult(status=StepStatus.PAUSED, output=output)
# on_reject == "skip" → completed, downstream steps decide
return StepResult(status=StepStatus.COMPLETED, output=output)
@@ -234,6 +316,13 @@ class GateStep(StepBase):
f"Gate step {config.get('id', '?')!r}: 'on_reject' must be "
f"'abort', 'skip', or 'retry'."
)
if "verdict_input" in config and (
not isinstance(config["verdict_input"], str) or not config["verdict_input"]
):
errors.append(
f"Gate step {config.get('id', '?')!r}: 'verdict_input' must be "
"a non-empty string."
)
# Only inspect option text when every option is a string; otherwise the
# `o.lower()` below would raise AttributeError on a non-string option
# (already reported above) and break validate_workflow's never-raise contract.

View File

@@ -0,0 +1,53 @@
"""Contract tests: every bundled preset must ship inside the wheel's core_pack.
``specify preset add <id>`` resolves a bundled preset via
``specify_cli._assets._locate_bundled_preset``, which checks the wheel's
``specify_cli/core_pack/presets/<id>/`` directory first. Any preset marked
``bundled: true`` in ``presets/catalog.json`` must therefore be force-included
at build time; otherwise the released wheel advertises a bundled preset it does
not actually ship, and ``specify preset add <id>`` falls through and reports the
preset as missing.
"""
from __future__ import annotations
import json
import tomllib
from pathlib import Path
REPO_ROOT = Path(__file__).parents[2]
def _force_include() -> dict[str, str]:
with (REPO_ROOT / "pyproject.toml").open("rb") as pyproject_file:
pyproject = tomllib.load(pyproject_file)
return pyproject["tool"]["hatch"]["build"]["targets"]["wheel"]["force-include"]
def _bundled_preset_ids() -> list[str]:
catalog = json.loads((REPO_ROOT / "presets" / "catalog.json").read_text())
return sorted(
preset_id
for preset_id, entry in catalog["presets"].items()
if entry.get("bundled")
)
def test_every_bundled_preset_is_force_included():
force_include = _force_include()
bundled = _bundled_preset_ids()
assert bundled, "expected at least one bundled preset in presets/catalog.json"
for preset_id in bundled:
assert force_include.get(f"presets/{preset_id}") == (
f"specify_cli/core_pack/presets/{preset_id}"
), f"bundled preset '{preset_id}' is missing from the wheel force-include list"
def test_constitution_sync_is_bundled_and_shipped():
# Explicit regression guard: constitution-sync was advertised as bundled
# before it was added to the wheel force-include list.
assert "constitution-sync" in _bundled_preset_ids()
assert _force_include()["presets/constitution-sync"] == (
"specify_cli/core_pack/presets/constitution-sync"
)

View File

@@ -2372,3 +2372,279 @@ def test_refresh_shared_templates_preserves_recovered_user_file(tmp_path):
# Recovered user content must survive (fail-before: replaced by bundled body).
assert user_file.read_text(encoding="utf-8") == "# USER CUSTOM CONTENT\n"
class TestExtensionFlag:
"""Tests for the --extension flag on specify init."""
def _run_init(self, tmp_path, args, project_name="ext-test"):
from unittest.mock import patch
from typer.testing import CliRunner
from specify_cli import app
project = tmp_path / project_name
project.mkdir(exist_ok=True)
old_cwd = os.getcwd()
try:
os.chdir(project)
runner = CliRunner()
# Patch get_speckit_version to return a stable (non-dev) version so that
# the extension compatibility check (SpecifierSet(">=0.2.0")) passes.
with patch(
"specify_cli.commands.init.get_speckit_version",
return_value="0.8.2",
):
result = runner.invoke(app, [
"init", "--here",
"--integration", "copilot",
"--script", "sh",
"--ignore-agent-tools",
] + args, catch_exceptions=False)
finally:
os.chdir(old_cwd)
return project, result
def test_bundled_extension_installed(self, tmp_path):
"""--extension git installs the bundled git extension."""
project, result = self._run_init(tmp_path, ["--extension", "git"], project_name="ext-bundled")
assert result.exit_code == 0, f"init failed:\n{result.output}"
ext_dir = project / ".specify" / "extensions" / "git"
assert ext_dir.exists(), "git extension directory not found"
assert (ext_dir / "extension.yml").exists(), "extension.yml not found"
# Tracker should show extension step as done
normalized = _normalize_cli_output(result.output)
assert "Install extension: git" in normalized
def test_multiple_extensions_installed(self, tmp_path):
"""--extension can be specified multiple times."""
project, result = self._run_init(
tmp_path,
["--extension", "git", "--extension", "selftest"],
project_name="ext-multi",
)
assert result.exit_code == 0, f"init failed:\n{result.output}"
ext_dir_git = project / ".specify" / "extensions" / "git"
ext_dir_selftest = project / ".specify" / "extensions" / "selftest"
assert ext_dir_git.exists(), "git extension not installed"
assert ext_dir_selftest.exists(), "selftest extension not installed"
def test_local_path_extension_installed(self, tmp_path):
"""--extension /abs/path installs from a local absolute directory path."""
from specify_cli import _locate_bundled_extension
# Use the bundled git extension directory as our "local" extension source
bundled_git = _locate_bundled_extension("git")
assert bundled_git is not None, "bundled git extension not found; cannot run test"
# Pass the absolute path directly (starts with "/")
project, result = self._run_init(
tmp_path,
["--extension", str(bundled_git)],
project_name="ext-local",
)
assert result.exit_code == 0, f"init failed:\n{result.output}"
ext_dir = project / ".specify" / "extensions" / "git"
assert ext_dir.exists(), "extension from local path not installed"
def test_unknown_extension_shows_error_in_tracker(self, tmp_path):
"""An unknown extension name records a tracker error but does not abort init."""
project, result = self._run_init(
tmp_path,
["--extension", "nonexistent-xyz-ext"],
project_name="ext-unknown",
)
assert result.exit_code == 0, "init should not abort on unknown extension"
normalized = _normalize_cli_output(result.output)
assert "failed" in normalized.lower(), "expected 'failed' for unknown extension"
def test_extension_flag_works_with_preset(self, tmp_path):
"""--extension and --preset can be combined."""
project, result = self._run_init(
tmp_path,
["--extension", "git", "--preset", "lean"],
project_name="ext-preset",
)
assert result.exit_code == 0, f"init failed:\n{result.output}"
ext_dir = project / ".specify" / "extensions" / "git"
assert ext_dir.exists(), "git extension not installed alongside preset"
@staticmethod
def _zip_bytes_from_dir(source_dir):
"""Build in-memory ZIP bytes from an extension directory (yml at root)."""
import io
import zipfile
buf = io.BytesIO()
with zipfile.ZipFile(buf, "w", zipfile.ZIP_DEFLATED) as zf:
for path in sorted(source_dir.rglob("*")):
if path.is_file():
zf.write(path, arcname=str(path.relative_to(source_dir)))
return buf.getvalue()
def test_url_extension_rejects_non_https(self, tmp_path):
"""A non-HTTPS URL is rejected before any download; init is not aborted."""
project, result = self._run_init(
tmp_path,
["--extension", "http://example.com/ext.zip", "--trust-extension-urls"],
project_name="ext-http",
)
assert result.exit_code == 0, "init should not abort on a rejected URL"
normalized = _normalize_cli_output(result.output)
assert "failed" in normalized.lower()
# No extension directory should have been created for the bad URL.
assert not (project / ".specify" / "extensions" / "ext").exists()
def test_url_extension_skipped_without_trust(self, tmp_path):
"""Non-interactive URL install without --trust-extension-urls is denied."""
from unittest.mock import patch
with patch(
"specify_cli.commands.init._stdin_is_interactive", return_value=False
), patch("specify_cli.authentication.http.open_url") as mock_open:
project, result = self._run_init(
tmp_path,
["--extension", "https://example.com/git.zip"],
project_name="ext-url-denied",
)
assert result.exit_code == 0, f"init failed:\n{result.output}"
# Default-deny: no download attempted, nothing installed.
mock_open.assert_not_called()
normalized = _normalize_cli_output(result.output)
assert "untrusted url" in normalized.lower()
assert not (project / ".specify" / "extensions" / "git").exists()
def test_url_extension_interactive_confirm_installs(self, tmp_path):
"""An interactive 'yes' to the trust prompt allows the URL install."""
import io
from unittest.mock import patch
from specify_cli import _locate_bundled_extension
bundled_git = _locate_bundled_extension("git")
assert bundled_git is not None, "bundled git extension not found"
zip_bytes = self._zip_bytes_from_dir(bundled_git)
class FakeResponse(io.BytesIO):
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return False
def _cache_dir_stand_in(project_root):
d = project_root / ".specify" / "extensions" / ".cache" / "downloads"
d.mkdir(parents=True, exist_ok=True)
return d
def _open_download_zip(project_root, download_dir, zip_filename):
target = download_dir / zip_filename
o_temporary = getattr(os, "O_TEMPORARY", 0)
if o_temporary:
return os.open(
target, os.O_RDWR | os.O_CREAT | os.O_EXCL | o_temporary, 0o600
)
fd = os.open(target, os.O_RDWR | os.O_CREAT | os.O_EXCL, 0o600)
try:
os.unlink(target)
except OSError:
os.close(fd)
raise
return fd
with patch(
"specify_cli.commands.init._stdin_is_interactive", return_value=True
), patch("typer.confirm", return_value=True), patch(
"specify_cli.authentication.http.open_url",
return_value=FakeResponse(zip_bytes),
), patch(
"specify_cli.extensions._commands._validate_safe_cache_dir",
side_effect=_cache_dir_stand_in,
), patch(
"specify_cli.extensions._commands._safe_open_download_zip",
side_effect=_open_download_zip,
):
project, result = self._run_init(
tmp_path,
["--extension", "https://example.com/git.zip"],
project_name="ext-url-confirm",
)
assert result.exit_code == 0, f"init failed:\n{result.output}"
assert (project / ".specify" / "extensions" / "git").exists()
def test_url_extension_installs_zip(self, tmp_path):
"""A successful HTTPS ZIP download installs via the shared hardened path."""
import io
from unittest.mock import patch
from specify_cli import _locate_bundled_extension
bundled_git = _locate_bundled_extension("git")
assert bundled_git is not None, "bundled git extension not found"
zip_bytes = self._zip_bytes_from_dir(bundled_git)
class FakeResponse(io.BytesIO):
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return False
def _cache_dir_stand_in(project_root):
d = project_root / ".specify" / "extensions" / ".cache" / "downloads"
d.mkdir(parents=True, exist_ok=True)
return d
def _open_download_zip(project_root, download_dir, zip_filename):
target = download_dir / zip_filename
o_temporary = getattr(os, "O_TEMPORARY", 0)
if o_temporary:
return os.open(
target, os.O_RDWR | os.O_CREAT | os.O_EXCL | o_temporary, 0o600
)
fd = os.open(target, os.O_RDWR | os.O_CREAT | os.O_EXCL, 0o600)
try:
os.unlink(target)
except OSError:
os.close(fd)
raise
return fd
with patch(
"specify_cli.authentication.http.open_url",
return_value=FakeResponse(zip_bytes),
), patch(
"specify_cli.extensions._commands._validate_safe_cache_dir",
side_effect=_cache_dir_stand_in,
), patch(
"specify_cli.extensions._commands._safe_open_download_zip",
side_effect=_open_download_zip,
):
project, result = self._run_init(
tmp_path,
["--extension", "https://example.com/git.zip", "--trust-extension-urls"],
project_name="ext-url",
)
assert result.exit_code == 0, f"init failed:\n{result.output}"
ext_dir = project / ".specify" / "extensions" / "git"
assert ext_dir.exists(), "extension from URL not installed"
assert (ext_dir / "extension.yml").exists()
# Transient download archive must not linger in the cache.
cache_dir = project / ".specify" / "extensions" / ".cache" / "downloads"
leftover = list(cache_dir.glob("*.zip")) if cache_dir.exists() else []
assert not leftover, f"download cache not cleaned: {leftover}"

View File

@@ -1411,6 +1411,19 @@ class TestTeardownDataSafety:
# User content preserved verbatim — not reset to {}.
assert config_path.read_text() == jsonc
def test_unreadable_config_not_overwritten_on_merge(self, tmp_path):
"""An unreadable user config aborts the merge instead of crashing."""
integration = ClaudeIntegration()
config_path = tmp_path / ".claude/settings.json"
config_path.mkdir(parents=True)
install_integration_events(
integration, tmp_path, _claude_manifest(tmp_path),
{"pre_tool_use": [{"command": "speckit.tdd.validate"}]},
)
assert config_path.is_dir()
def test_jsonc_opencode_config_not_reset(self, tmp_path):
"""#23: a malformed opencode.json is preserved, not reset to {}."""
integration = OpencodeIntegration()

View File

@@ -116,12 +116,15 @@ class TestCatalogURLValidation:
[
"https://[::1", # unclosed ipv6 bracket
"https://[not-an-ip]/c.json", # bracketed non-ip host
"https://example.com:notaport/c.json", # non-numeric port
"https://example.com:65536/c.json", # out-of-range port
],
)
def test_malformed_url_rejected_cleanly(self, url):
# A malformed authority makes urlparse/hostname raise ValueError. The
# validator must turn that into its normal catalog error, not leak a
# raw ValueError to the caller.
# A malformed authority makes urlparse/hostname raise ValueError, and a
# bad port makes ``parsed.port`` raise it. The validator must turn that
# into its normal catalog error, not leak a raw ValueError to the caller
# (or, for a bad port, accept the URL and fail later at fetch time).
with pytest.raises(IntegrationCatalogError, match="malformed"):
IntegrationCatalog._validate_catalog_url(url)

View File

@@ -375,6 +375,14 @@ class TestManifestLoadValidation:
with pytest.raises(ValueError, match="invalid JSON"):
IntegrationManifest.load("bad", tmp_path)
def test_load_non_utf8_json_raises_value_error(self, tmp_path):
path = tmp_path / ".specify" / "integrations" / "bad.manifest.json"
path.parent.mkdir(parents=True)
path.write_bytes(b"\xff\xfe")
with pytest.raises(ValueError, match="valid UTF-8"):
IntegrationManifest.load("bad", tmp_path)
def test_load_filters_recovered_files_not_in_files(self, tmp_path):
# Finding B (Round-9): a recovered_files entry referencing a path
# not present in files indicates an internally-inconsistent manifest

View File

@@ -370,3 +370,69 @@ def test_python_branch_falls_back_to_feature_dir_basename(prereq_repo: Path) ->
assert py.returncode == 0, py.stderr
assert _json_stdout(py)["BRANCH"] == "001-my-feature"
class TestGetInvokeSeparatorTolerance:
"""`get_invoke_separator` must fall back to "." for an unusable
`integration.json`, matching its bash and PowerShell twins.
The bash twin tries jq -> python3 -> awk and keeps its `separator="."`
default on any parse failure; the PowerShell twin likewise returns ".".
The Python twin instead indexed the parsed value directly, so two shapes
escaped its `except (OSError, json.JSONDecodeError)`:
* a non-mapping top level (`[]`, `"forge"`, `42`, `null`) is valid JSON,
so JSONDecodeError never fires and `.get()` raised AttributeError;
* a non-UTF-8 file raises UnicodeDecodeError -- a ValueError, not an
OSError. Realistic on Windows, where PowerShell 5.1's `Out-File`/`>`
default to UTF-16.
The sibling `read_feature_json_feature_directory` in the same module
already guards both.
"""
@staticmethod
def _load_common():
import importlib.util
spec = importlib.util.spec_from_file_location("_speckit_common_py", COMMON_PY)
module = importlib.util.module_from_spec(spec)
# Register before exec: the module defines @dataclass types, and
# dataclasses resolves cls.__module__ through sys.modules.
sys.modules[spec.name] = module
try:
spec.loader.exec_module(module)
except Exception: # pragma: no cover - defensive cleanup
sys.modules.pop(spec.name, None)
raise
return module
def _repo(self, tmp_path: Path, body: str | bytes) -> Path:
(tmp_path / ".specify").mkdir(parents=True, exist_ok=True)
target = tmp_path / ".specify" / "integration.json"
if isinstance(body, bytes):
target.write_bytes(body)
else:
target.write_text(body, encoding="utf-8")
return tmp_path
@pytest.mark.parametrize(
"body", ["[]", '[{"a": 1}]', '"forge"', "42", "true", "null"]
)
def test_non_mapping_integration_json_falls_back(self, tmp_path: Path, body: str):
common = self._load_common()
assert common.get_invoke_separator(self._repo(tmp_path, body)) == "."
def test_non_utf8_integration_json_falls_back(self, tmp_path: Path):
common = self._load_common()
raw = '{"default_integration": "forge"}'.encode("utf-16")
assert common.get_invoke_separator(self._repo(tmp_path, raw)) == "."
def test_hyphen_separator_is_still_honoured(self, tmp_path: Path):
"""Regression guard: the real feature must keep working."""
common = self._load_common()
body = json.dumps({
"default_integration": "droid",
"integration_settings": {"droid": {"invoke_separator": "-"}},
})
assert common.get_invoke_separator(self._repo(tmp_path, body)) == "-"

View File

@@ -5,6 +5,7 @@ from __future__ import annotations
import io
import stat
import struct
import tarfile
import weakref
import zipfile
import zlib
@@ -13,11 +14,16 @@ import pytest
from specify_cli._download_security import (
MAX_ZIP_CENTRAL_DIRECTORY_BYTES,
archive_format_from_content_type,
archive_format_from_name,
build_safe_download_path,
detect_archive_format,
is_https_or_localhost_http,
is_loopback_url,
read_response_limited,
read_zip_member_limited,
safe_extract_archive,
safe_extract_tar,
safe_extract_zip,
)
@@ -314,6 +320,176 @@ def test_build_safe_download_path_rejects_nonportable_identifiers(
)
@pytest.mark.parametrize(
("name", "expected"),
[
("package.zip", "zip"),
("PACKAGE.TAR.GZ", "tar.gz"),
("https://example.com/package.tgz?download=1", "tar.gz"),
("package.tar", None),
],
)
def test_archive_format_from_name(name, expected):
assert archive_format_from_name(name) == expected
@pytest.mark.parametrize(
("content_type", "expected"),
[
("application/zip", "zip"),
("application/x-zip-compressed; charset=binary", "zip"),
("application/gzip", "tar.gz"),
("application/x-gzip", "tar.gz"),
("application/octet-stream", None),
],
)
def test_archive_format_from_content_type(content_type, expected):
assert archive_format_from_content_type(content_type) == expected
def _write_tar_gz(path, members):
with tarfile.open(path, "w:gz") as archive:
for name, content in members:
info = tarfile.TarInfo(name)
info.size = len(content)
archive.addfile(info, io.BytesIO(content))
@pytest.mark.parametrize("suffix", [".tar.gz", ".tgz"])
def test_detect_archive_format_accepts_tar_suffixes(tmp_path, suffix):
archive_path = tmp_path / f"package{suffix}"
_write_tar_gz(archive_path, [("file.txt", b"contents")])
assert detect_archive_format(archive_path) == "tar.gz"
def test_detect_archive_format_allows_content_type_fallback(tmp_path):
archive_path = tmp_path / "download"
_write_tar_gz(archive_path, [("file.txt", b"contents")])
assert (
detect_archive_format(
archive_path,
source_name="https://example.com/download",
content_type="application/gzip",
)
== "tar.gz"
)
def test_detect_archive_format_rejects_suffix_content_mismatch(tmp_path):
archive_path = tmp_path / "package.zip"
_write_tar_gz(archive_path, [("file.txt", b"contents")])
with pytest.raises(ValueError, match="format mismatch"):
detect_archive_format(archive_path)
def test_detect_archive_format_rejects_suffix_header_mismatch(tmp_path):
archive_path = tmp_path / "package.zip"
with zipfile.ZipFile(archive_path, "w") as archive:
archive.writestr("file.txt", "contents")
with pytest.raises(ValueError, match="Content-Type"):
detect_archive_format(
archive_path,
content_type="application/gzip",
)
def test_build_safe_download_path_uses_archive_suffix(tmp_path):
path = build_safe_download_path(tmp_path, "package", "1.0.0", suffix=".tar.gz")
assert path.name == "package-1.0.0.tar.gz"
@pytest.mark.parametrize(
"member_name",
["../evil.txt", "nested/../../evil.txt", "C:/Windows/evil.txt"],
)
def test_safe_extract_tar_rejects_traversal(tmp_path, member_name):
archive_path = tmp_path / "bad.tar.gz"
_write_tar_gz(archive_path, [(member_name, b"nope")])
with pytest.raises(ValueError, match="Unsafe path"):
safe_extract_tar(archive_path, tmp_path / "out")
@pytest.mark.parametrize(
("link_type", "message"),
[(tarfile.SYMTYPE, "symlink"), (tarfile.LNKTYPE, "hard link")],
)
def test_safe_extract_tar_rejects_links_without_partial_extraction(
tmp_path, link_type, message
):
archive_path = tmp_path / "bad.tar.gz"
with tarfile.open(archive_path, "w:gz") as archive:
safe = tarfile.TarInfo("safe.txt")
safe.size = 4
archive.addfile(safe, io.BytesIO(b"safe"))
link = tarfile.TarInfo("escape")
link.type = link_type
link.linkname = "../../outside"
archive.addfile(link)
out_dir = tmp_path / "out"
with pytest.raises(ValueError, match=message):
safe_extract_tar(archive_path, out_dir)
assert not out_dir.exists() or not any(out_dir.rglob("*"))
def test_safe_extract_tar_rejects_special_file(tmp_path):
archive_path = tmp_path / "bad.tar.gz"
with tarfile.open(archive_path, "w:gz") as archive:
fifo = tarfile.TarInfo("pipe")
fifo.type = tarfile.FIFOTYPE
archive.addfile(fifo)
with pytest.raises(ValueError, match="Unsafe member type"):
safe_extract_tar(archive_path, tmp_path / "out")
def test_safe_extract_tar_rejects_conflicting_paths(tmp_path):
archive_path = tmp_path / "bad.tar.gz"
_write_tar_gz(
archive_path,
[("Folder/file.txt", b"one"), ("folder/FILE.txt", b"two")],
)
with pytest.raises(ValueError, match="Conflicting path"):
safe_extract_tar(archive_path, tmp_path / "out")
def test_safe_extract_tar_enforces_entry_and_size_limits(tmp_path):
archive_path = tmp_path / "bad.tar.gz"
_write_tar_gz(
archive_path,
[("one.txt", b"1234"), ("two.txt", b"5678")],
)
with pytest.raises(ValueError, match="too many entries"):
safe_extract_tar(archive_path, tmp_path / "entries", max_entries=1)
with pytest.raises(ValueError, match="member.*maximum size"):
safe_extract_tar(archive_path, tmp_path / "member", max_member_bytes=3)
with pytest.raises(ValueError, match="uncompressed size"):
safe_extract_tar(archive_path, tmp_path / "total", max_total_bytes=7)
@pytest.mark.parametrize("suffix", [".zip", ".tar.gz", ".tgz"])
def test_safe_extract_archive_has_format_parity(tmp_path, suffix):
archive_path = tmp_path / f"package{suffix}"
if suffix == ".zip":
with zipfile.ZipFile(archive_path, "w") as archive:
archive.writestr("nested/file.txt", b"contents")
else:
_write_tar_gz(archive_path, [("nested/file.txt", b"contents")])
out_dir = tmp_path / f"out-{suffix.replace('.', '-')}"
safe_extract_archive(archive_path, out_dir)
assert (out_dir / "nested" / "file.txt").read_bytes() == b"contents"
@pytest.mark.parametrize(
"member_name",
[

View File

@@ -0,0 +1,401 @@
"""Security tests for the extension URL download cache."""
from __future__ import annotations
import io
import os
import shutil
from pathlib import Path
from types import SimpleNamespace
from unittest.mock import MagicMock
import pytest
import typer
from typer.testing import CliRunner
from specify_cli import app
from specify_cli.extensions import ExtensionCatalog, ExtensionManager
from specify_cli.extensions import _commands
_MINIMAL_ZIP_BYTES = b"PK\x05\x06" + b"\x00" * 18
runner = CliRunner()
def _require_secure_dir_fd() -> None:
if (
not getattr(os, "O_NOFOLLOW", 0)
or os.open not in os.supports_dir_fd
or os.mkdir not in os.supports_dir_fd
):
pytest.skip("requires dir_fd and O_NOFOLLOW support")
def _symlink_directory(link: Path, target: Path) -> None:
try:
link.symlink_to(target, target_is_directory=True)
except (OSError, NotImplementedError) as exc:
pytest.skip(f"directory symlinks are unavailable: {exc}")
@pytest.fixture
def project_dir(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Path:
project = tmp_path / "project"
project.mkdir()
(project / ".specify").mkdir()
monkeypatch.chdir(project)
return project
@pytest.mark.parametrize(
"ancestor_parts",
[
("extensions",),
("extensions", ".cache"),
("extensions", ".cache", "downloads"),
],
)
def test_symlinked_cache_ancestor_is_refused(
project_dir: Path, tmp_path: Path, ancestor_parts: tuple[str, ...]
) -> None:
_require_secure_dir_fd()
outside = tmp_path / "outside"
outside.mkdir()
parent = project_dir / ".specify"
for part in ancestor_parts[:-1]:
parent = parent / part
parent.mkdir()
_symlink_directory(parent / ancestor_parts[-1], outside)
with pytest.raises(typer.Exit):
_commands._validate_safe_cache_dir(project_dir)
assert list(outside.iterdir()) == []
@pytest.mark.parametrize(
"ancestor_parts",
[
("extensions",),
("extensions", ".cache"),
("extensions", ".cache", "downloads"),
],
)
def test_symlinked_cache_ancestor_is_refused_without_dir_fd(
project_dir: Path,
tmp_path: Path,
monkeypatch: pytest.MonkeyPatch,
ancestor_parts: tuple[str, ...],
) -> None:
"""The portable (Windows) validation path must also refuse a symlinked
cache ancestor and never create anything under the symlink target."""
monkeypatch.setattr(os, "supports_dir_fd", set())
outside = tmp_path / "outside"
outside.mkdir()
parent = project_dir / ".specify"
for part in ancestor_parts[:-1]:
parent = parent / part
parent.mkdir()
_symlink_directory(parent / ancestor_parts[-1], outside)
with pytest.raises(typer.Exit):
_commands._validate_safe_cache_dir(project_dir)
assert list(outside.iterdir()) == []
def test_cache_ancestor_resolving_outside_project_is_refused(
project_dir: Path, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_require_secure_dir_fd()
cache_root = project_dir / ".specify" / "extensions" / ".cache"
cache_root.mkdir(parents=True)
outside = tmp_path / "outside"
outside.mkdir()
real_resolve = Path.resolve
def fake_resolve(self: Path, *args, **kwargs) -> Path:
if self == cache_root:
return real_resolve(outside, *args, **kwargs)
return real_resolve(self, *args, **kwargs)
monkeypatch.setattr(Path, "resolve", fake_resolve)
with pytest.raises(typer.Exit):
_commands._validate_safe_cache_dir(project_dir)
assert list(outside.iterdir()) == []
def test_safe_open_refuses_exclusive_leaf_collision(project_dir: Path) -> None:
_require_secure_dir_fd()
download_dir = _commands._validate_safe_cache_dir(project_dir)
zip_filename = "extension-url-download-collision.zip"
collision = download_dir / zip_filename
collision.write_bytes(b"sentinel")
with pytest.raises(OSError):
_commands._safe_open_download_zip(
project_dir, download_dir, zip_filename
)
assert collision.read_bytes() == b"sentinel"
def test_safe_open_refuses_swapped_cache_ancestor(
project_dir: Path, tmp_path: Path
) -> None:
_require_secure_dir_fd()
download_dir = _commands._validate_safe_cache_dir(project_dir)
cache_root = project_dir / ".specify" / "extensions" / ".cache"
outside = tmp_path / "outside"
outside.mkdir()
shutil.rmtree(cache_root)
_symlink_directory(cache_root, outside)
with pytest.raises(OSError):
_commands._safe_open_download_zip(
project_dir,
download_dir,
"extension-url-download-swapped.zip",
)
assert list(outside.iterdir()) == []
def test_safe_open_refuses_symlinked_project_root(
project_dir: Path, tmp_path: Path
) -> None:
_require_secure_dir_fd()
project_link = tmp_path / "project-link"
_symlink_directory(project_link, project_dir)
download_dir = project_link / ".specify" / "extensions" / ".cache" / "downloads"
with pytest.raises(OSError):
_commands._safe_open_download_zip(
project_link,
download_dir,
"extension-url-download-project-link.zip",
)
def test_safe_open_succeeds_without_dir_fd_support(
project_dir: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""On a platform without dir_fd (e.g. Windows) the portable path must
still hand back a usable, exclusively-created descriptor rather than
failing closed."""
monkeypatch.setattr(os, "supports_dir_fd", set())
download_dir = _commands._validate_safe_cache_dir(project_dir)
assert download_dir == (
project_dir / ".specify" / "extensions" / ".cache" / "downloads"
)
fd = _commands._safe_open_download_zip(
project_dir, download_dir, "extension-url-download-portable.zip"
)
try:
os.write(fd, b"payload")
os.lseek(fd, 0, os.SEEK_SET)
assert os.read(fd, 7) == b"payload"
finally:
os.close(fd)
def test_safe_open_without_dir_fd_refuses_symlinked_leaf(
project_dir: Path, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""The portable path must refuse a leaf pre-staged as a symlink so an
attacker cannot redirect the exclusive create outside the project."""
monkeypatch.setattr(os, "supports_dir_fd", set())
download_dir = _commands._validate_safe_cache_dir(project_dir)
outside = tmp_path / "outside.zip"
zip_filename = "extension-url-download-symlink-leaf.zip"
try:
(download_dir / zip_filename).symlink_to(outside)
except (OSError, NotImplementedError) as exc:
pytest.skip(f"symlinks are unavailable: {exc}")
with pytest.raises(OSError):
_commands._safe_open_download_zip(project_dir, download_dir, zip_filename)
assert not outside.exists()
def test_url_install_succeeds_without_dir_fd_support(
project_dir: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""A full ``--from`` install must work on platforms without dir_fd rather
than failing closed, exercising the portable hardened download path."""
captured: dict[str, object] = {}
class FakeResponse(io.BytesIO):
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return False
def fake_install(
self,
zip_path: Path,
speckit_version: str,
priority: int = 10,
force: bool = False,
*,
archive_file=None,
):
captured["bytes"] = archive_file.read()
archive_file.seek(0)
return SimpleNamespace(
id="test-ext",
name="Test Extension",
version="1.0.0",
description="",
warnings=[],
commands=[],
)
monkeypatch.setattr(os, "supports_dir_fd", set())
monkeypatch.setattr(typer, "confirm", lambda *args, **kwargs: True)
monkeypatch.setattr(
ExtensionCatalog,
"_open_url",
lambda *args, **kwargs: FakeResponse(_MINIMAL_ZIP_BYTES),
)
monkeypatch.setattr(ExtensionManager, "install_from_zip", fake_install)
monkeypatch.setattr(_commands, "_refresh_events_and_warn", lambda root: None)
monkeypatch.setattr(_commands, "load_init_options", lambda root: {})
result = runner.invoke(
app,
[
"extension",
"add",
"test-ext",
"--from",
"https://example.com/test-ext.zip",
],
)
assert result.exit_code == 0, result.output
assert captured["bytes"] == _MINIMAL_ZIP_BYTES
def test_url_install_writes_and_cleans_up_secure_download(
project_dir: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
_require_secure_dir_fd()
captured: dict[str, object] = {}
class FakeResponse(io.BytesIO):
def __enter__(self):
return self
def __exit__(self, exc_type, exc, tb):
return False
def fake_install(
self,
zip_path: Path,
speckit_version: str,
priority: int = 10,
force: bool = False,
*,
archive_file=None,
):
captured["path"] = zip_path
captured["mode"] = os.fstat(archive_file.fileno()).st_mode & 0o777
captured["exists_during_install"] = zip_path.exists()
captured["bytes"] = archive_file.read()
archive_file.seek(0)
return SimpleNamespace(
id="test-ext",
name="Test Extension",
version="1.0.0",
description="",
warnings=[],
commands=[],
)
monkeypatch.setattr(typer, "confirm", lambda *args, **kwargs: True)
monkeypatch.setattr(
ExtensionCatalog,
"_open_url",
lambda *args, **kwargs: FakeResponse(_MINIMAL_ZIP_BYTES),
)
monkeypatch.setattr(ExtensionManager, "install_from_zip", fake_install)
monkeypatch.setattr(_commands, "_refresh_events_and_warn", lambda root: None)
monkeypatch.setattr(_commands, "load_init_options", lambda root: {})
result = runner.invoke(
app,
[
"extension",
"add",
"test-ext",
"--from",
"https://example.com/test-ext.zip",
],
)
assert result.exit_code == 0, result.output
assert captured["bytes"] == _MINIMAL_ZIP_BYTES
assert captured["mode"] == 0o600
# The archive is an anonymous inode: it is never visible on disk, even
# while installation consumes the open descriptor.
assert captured["exists_during_install"] is False
zip_path = captured["path"]
assert isinstance(zip_path, Path)
assert zip_path.parent == (
project_dir / ".specify" / "extensions" / ".cache" / "downloads"
)
assert zip_path.name.startswith("extension-url-download-")
assert not zip_path.exists()
def test_url_install_open_error_surfaces_as_controlled_exit(
project_dir: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
"""An ``OSError`` from the hardened create (e.g. an exclusive-leaf
collision or a swapped ancestor) must fail closed as ``typer.Exit(1)``
rather than escaping as an unhandled traceback, and installation must
not run."""
download_dir = project_dir / ".specify" / "extensions" / ".cache" / "downloads"
monkeypatch.setattr(typer, "confirm", lambda *args, **kwargs: True)
monkeypatch.setattr(
ExtensionCatalog,
"_open_url",
lambda *args, **kwargs: io.BytesIO(_MINIMAL_ZIP_BYTES),
)
monkeypatch.setattr(
_commands, "_validate_safe_cache_dir", lambda root: download_dir
)
download_dir.mkdir(parents=True, exist_ok=True)
def _raise_collision(project_root, dir_, zip_filename):
raise FileExistsError("leaf already exists")
monkeypatch.setattr(_commands, "_safe_open_download_zip", _raise_collision)
install_spy = MagicMock()
monkeypatch.setattr(ExtensionManager, "install_from_zip", install_spy)
result = runner.invoke(
app,
[
"extension",
"add",
"test-ext",
"--from",
"https://example.com/test-ext.zip",
],
)
assert result.exit_code == 1
assert "Could not safely create download file" in result.output
install_spy.assert_not_called()

View File

@@ -49,6 +49,38 @@ from specify_cli._utils import version_satisfies
_MINIMAL_ZIP_BYTES = b"PK\x05\x06" + b"\x00" * 18
def _open_test_download_zip(project_root, download_dir, zip_filename):
"""Cross-platform stand-in for the POSIX-only secure cache primitive.
Mirrors production behavior by making the leaf disappear from disk while
the descriptor stays open. On POSIX the file is unlinked immediately; on
Windows an in-use file cannot be unlinked, so it is opened with
``O_TEMPORARY`` and removed automatically when the descriptor closes.
"""
target = download_dir / zip_filename
o_temporary = getattr(os, "O_TEMPORARY", 0)
if o_temporary:
return os.open(
target,
os.O_RDWR | os.O_CREAT | os.O_EXCL | o_temporary,
0o600,
)
fd = os.open(target, os.O_RDWR | os.O_CREAT | os.O_EXCL, 0o600)
try:
os.unlink(target)
except OSError:
os.close(fd)
raise
return fd
def _validate_safe_cache_dir_test_stand_in(project_root):
"""Cross-platform stand-in for the secure cache validator."""
download_dir = project_root / ".specify" / "extensions" / ".cache" / "downloads"
download_dir.mkdir(parents=True, exist_ok=True)
return download_dir
def can_create_symlink(tmp_path: Path) -> bool:
"""Return True when the current platform/user can create file symlinks."""
target = tmp_path / "symlink-target.txt"
@@ -2229,6 +2261,79 @@ class TestExtensionManager:
assert not manager.registry.is_installed("test-ext")
@pytest.mark.skipif(os.name == "nt", reason="requires replacing an open file")
def test_install_from_zip_uses_open_archive_after_path_replacement(
self, extension_dir, project_dir, temp_dir
):
"""An authoritative archive stream must survive pathname replacement."""
import zipfile
zip_path = temp_dir / "original-extension.zip"
with zipfile.ZipFile(zip_path, "w") as archive:
for file_path in extension_dir.rglob("*"):
if file_path.is_file():
archive.write(file_path, file_path.relative_to(extension_dir))
manager = ExtensionManager(project_dir)
with zip_path.open("rb") as archive_file:
zip_path.unlink()
with zipfile.ZipFile(zip_path, "w"):
pass
manifest = manager.install_from_zip(
zip_path,
"0.1.0",
archive_file=archive_file,
)
assert manifest.id == "test-ext"
assert manager.registry.is_installed("test-ext")
@pytest.mark.parametrize("suffix", [".tar.gz", ".tgz"])
@pytest.mark.parametrize("nested", [False, True])
def test_install_from_tar_archive(
self, extension_dir, project_dir, temp_dir, suffix, nested
):
"""Tar archives install with the same flat/nested behavior as ZIP."""
import tarfile
archive_path = temp_dir / f"test-ext{suffix}"
with tarfile.open(archive_path, "w:gz") as archive:
for file_path in extension_dir.rglob("*"):
if file_path.is_file():
relative = file_path.relative_to(extension_dir)
arcname = Path("test-ext-v1") / relative if nested else relative
archive.add(file_path, arcname=arcname)
manager = ExtensionManager(project_dir)
manifest = manager.install_from_archive(archive_path, "0.1.0")
assert manifest.id == "test-ext"
assert manager.registry.is_installed("test-ext")
def test_install_from_tar_rejects_symlink_entry(
self, extension_dir, project_dir, temp_dir
):
import tarfile
archive_path = temp_dir / "symlink-extension.tar.gz"
with tarfile.open(archive_path, "w:gz") as archive:
for file_path in extension_dir.rglob("*"):
if file_path.is_file():
archive.add(
file_path,
arcname=file_path.relative_to(extension_dir),
)
link = tarfile.TarInfo("templates/escape")
link.type = tarfile.SYMTYPE
link.linkname = "../../outside"
archive.addfile(link)
manager = ExtensionManager(project_dir)
with pytest.raises(ValidationError, match="Unsafe symlink"):
manager.install_from_archive(archive_path, "0.1.0")
assert not manager.registry.is_installed("test-ext")
assert not manager.registry.is_installed("test-ext")
def test_install_duplicate_error_mentions_force(self, extension_dir, project_dir):
"""Test that duplicate install error message suggests --force."""
manager = ExtensionManager(project_dir)
@@ -4898,9 +5003,9 @@ class TestExtensionCatalog:
catalog = self._make_catalog(temp_dir)
mock_response = MagicMock()
mock_response.read.return_value = json.dumps(
mock_response.read.side_effect = io.BytesIO(json.dumps(
{"schema_version": "1.0", "extensions": {}}
).encode()
).encode()).read
mock_response.__enter__ = lambda s: s
mock_response.__exit__ = MagicMock(return_value=False)
mock_response.geturl.return_value = "http://evil.test/catalog.json"
@@ -4946,9 +5051,9 @@ class TestExtensionCatalog:
catalog = self._make_catalog(temp_dir)
mock_response = MagicMock()
mock_response.read.return_value = json.dumps(
mock_response.read.side_effect = io.BytesIO(json.dumps(
{"schema_version": "1.0", "extensions": {}}
).encode()
).encode()).read
mock_response.__enter__ = lambda s: s
mock_response.__exit__ = MagicMock(return_value=False)
mock_response.geturl.return_value = "http://evil.test/catalog.json"
@@ -5706,6 +5811,35 @@ class TestExtensionCatalog:
assert captured[0].get_header("Authorization") == "Bearer ghp_testtoken"
assert captured[0].get_header("Accept") == "application/octet-stream"
@pytest.mark.parametrize("suffix", [".tar.gz", ".tgz"])
def test_download_extension_preserves_tar_archive_format(
self, temp_dir, suffix
):
import tarfile
from unittest.mock import patch
archive_buffer = io.BytesIO()
with tarfile.open(fileobj=archive_buffer, mode="w:gz") as archive:
content = b"extension:\n id: test-ext\n"
member = tarfile.TarInfo("extension.yml")
member.size = len(content)
archive.addfile(member, io.BytesIO(content))
archive_bytes = archive_buffer.getvalue()
catalog = self._make_catalog(temp_dir)
ext_info = {
"id": "test-ext",
"name": "Test Extension",
"version": "1.0.0",
"download_url": f"https://example.com/test-ext{suffix}",
}
with patch.object(catalog, "get_extension_info", return_value=ext_info), \
patch.object(catalog, "_open_url", return_value=self._mock_response(archive_bytes)):
archive_path = catalog.download_extension("test-ext", target_dir=temp_dir)
assert archive_path.name == "test-ext-1.0.0.tar.gz"
assert archive_path.read_bytes() == archive_bytes
# ===== CatalogEntry Tests =====
@@ -7391,7 +7525,15 @@ class TestExtensionAddCLI:
manifest_id = "[red]bad[/red]"
def fake_install_from_zip(self_obj, zip_path, speckit_version, priority=10, force=False):
def fake_install_from_zip(
self_obj,
zip_path,
speckit_version,
priority=10,
force=False,
*,
archive_file=None,
):
return SimpleNamespace(
id=manifest_id,
name="Bad Extension",
@@ -7405,7 +7547,9 @@ class TestExtensionAddCLI:
runner = CliRunner()
with patch.object(Path, "cwd", return_value=project_dir), \
patch("typer.confirm", return_value=True), \
patch("specify_cli.extensions._commands._validate_safe_cache_dir", side_effect=_validate_safe_cache_dir_test_stand_in), \
patch("specify_cli.authentication.http.open_url", return_value=FakeResponse(_MINIMAL_ZIP_BYTES)), \
patch("specify_cli.extensions._commands._safe_open_download_zip", side_effect=_open_test_download_zip), \
patch.object(ExtensionManager, "install_from_zip", fake_install_from_zip), \
patch.object(ExtensionRegistry, "get", return_value={}):
result = runner.invoke(
@@ -7453,6 +7597,7 @@ class TestExtensionAddCLI:
runner = CliRunner()
with patch.object(Path, "cwd", return_value=project_dir), \
patch("typer.confirm", return_value=True), \
patch("specify_cli.extensions._commands._validate_safe_cache_dir", side_effect=_validate_safe_cache_dir_test_stand_in), \
patch(
"specify_cli.authentication.http.open_url",
side_effect=urllib.error.URLError("bad [red]download[/red]"),
@@ -7494,6 +7639,7 @@ class TestExtensionAddCLI:
runner = CliRunner()
with patch.object(Path, "cwd", return_value=project_dir), \
patch("typer.confirm", return_value=True), \
patch("specify_cli.extensions._commands._validate_safe_cache_dir", side_effect=_validate_safe_cache_dir_test_stand_in), \
patch(
"specify_cli.authentication.http.open_url",
return_value=FakeResponse(b"<!DOCTYPE html><html>Sign in</html>"),
@@ -7544,6 +7690,7 @@ class TestExtensionAddCLI:
runner = CliRunner()
with patch.object(Path, "cwd", return_value=project_dir), \
patch("typer.confirm", return_value=True), \
patch("specify_cli.extensions._commands._validate_safe_cache_dir", side_effect=_validate_safe_cache_dir_test_stand_in), \
patch(
"specify_cli.authentication.http.open_url",
return_value=FakeResponse(_MINIMAL_ZIP_BYTES),
@@ -7599,7 +7746,15 @@ class TestExtensionAddCLI:
seen["headers"] = extra_headers
return FakeResponse(_MINIMAL_ZIP_BYTES)
def fake_install(self_obj, zip_path, speckit_version, priority=10, force=False):
def fake_install(
self_obj,
zip_path,
speckit_version,
priority=10,
force=False,
*,
archive_file=None,
):
return SimpleNamespace(
id="x", name="X", version="1.0.0", description="", warnings=[], commands=[], hooks=[]
)
@@ -7607,8 +7762,10 @@ class TestExtensionAddCLI:
runner = CliRunner()
with patch.object(Path, "cwd", return_value=project_dir), \
patch("typer.confirm", return_value=True), \
patch("specify_cli.extensions._commands._validate_safe_cache_dir", side_effect=_validate_safe_cache_dir_test_stand_in), \
patch("specify_cli.authentication.http.github_provider_hosts", return_value=("ghes.example",)), \
patch("specify_cli.authentication.http.open_url", side_effect=fake_open_url), \
patch("specify_cli.extensions._commands._safe_open_download_zip", side_effect=_open_test_download_zip), \
patch.object(ExtensionManager, "install_from_zip", fake_install):
result = runner.invoke(
app,
@@ -7681,10 +7838,19 @@ class TestExtensionAddCLI:
downloads_dir = project_dir / ".specify" / "extensions" / ".cache" / "downloads"
installed = {}
def fake_install_from_zip(self_obj, zip_path, speckit_version, priority=10, force=False):
def fake_install_from_zip(
self_obj,
zip_path,
speckit_version,
priority=10,
force=False,
*,
archive_file=None,
):
captured_path = Path(zip_path)
installed["zip_path"] = captured_path
installed["zip_bytes"] = captured_path.read_bytes()
installed["zip_bytes"] = archive_file.read()
archive_file.seek(0)
return SimpleNamespace(
id="escape",
name="Escape Test",
@@ -7698,7 +7864,9 @@ class TestExtensionAddCLI:
runner = CliRunner()
with patch.object(Path, "cwd", return_value=project_dir), \
patch("typer.confirm", return_value=True), \
patch("specify_cli.extensions._commands._validate_safe_cache_dir", side_effect=_validate_safe_cache_dir_test_stand_in), \
patch("specify_cli.authentication.http.open_url", return_value=FakeResponse(_MINIMAL_ZIP_BYTES)), \
patch("specify_cli.extensions._commands._safe_open_download_zip", side_effect=_open_test_download_zip), \
patch.object(ExtensionManager, "install_from_zip", fake_install_from_zip):
result = runner.invoke(
app,
@@ -7759,7 +7927,7 @@ class TestDownloadExtensionBundled:
}
mock_response = MagicMock()
mock_response.read.side_effect = io.BytesIO(b"fake zip data").read
mock_response.read.side_effect = io.BytesIO(_MINIMAL_ZIP_BYTES).read
mock_response.__enter__ = lambda s: s
mock_response.__exit__ = MagicMock(return_value=False)
mock_response.geturl.return_value = "https://example.com/catalog.json"

View File

@@ -11,6 +11,7 @@ See proposals/monorepo-support and github/spec-kit discussion #2834.
import json
import os
import re
import shutil
import subprocess
from pathlib import Path
@@ -30,6 +31,11 @@ HAS_PWSH = shutil.which("pwsh") is not None
_POWERSHELL = shutil.which("powershell.exe") or shutil.which("powershell")
_PS_EXE = "pwsh" if HAS_PWSH else _POWERSHELL
# Windows PowerShell 5.1 (.NET Framework) specifically, never pwsh: the only
# host that lacks the .NET Core-only [System.IO.Path] members, so it is the only
# one that can pin a 5.1 compatibility regression (issue #3749).
_WINDOWS_POWERSHELL = _POWERSHELL if os.name == "nt" else None
def _clean_env() -> dict[str, str]:
"""Inherited env minus all SPECIFY_* vars, so a developer/CI override
@@ -101,6 +107,10 @@ requires_pwsh = pytest.mark.skipif(
not (HAS_PWSH or _POWERSHELL), reason="no PowerShell available"
)
requires_windows_powershell = pytest.mark.skipif(
_WINDOWS_POWERSHELL is None, reason="Windows PowerShell 5.1 not available"
)
# ── Bash: positive cases ────────────────────────────────────────────────────
@@ -465,3 +475,131 @@ def test_ps_file_path_errors_no_fallback(tmp_path: Path) -> None:
result = _ps("Get-RepoRoot", cwd=web, env=env)
assert result.returncode != 0
assert "does not point to an existing directory" in result.stderr
# ── Windows PowerShell 5.1 compatibility (issue #3749) ──────────────────────
#
# The CI matrix runs these PowerShell tests under `pwsh` on every OS, and pwsh
# is .NET Core, so a .NET Framework-only regression is invisible to it. The
# static test below therefore runs everywhere and is the one that actually
# guards CI; the runtime tests pin the real behavior where a 5.1 host exists.
# .NET Core-only [System.IO.Path] members. Absent on .NET Framework, so calling
# one throws "does not contain a method named ..." on Windows PowerShell 5.1.
_DOTNET_CORE_ONLY_PATH_MEMBERS = (
"TrimEndingDirectorySeparator",
"EndsInDirectorySeparator",
"GetRelativePath",
"Join",
)
@pytest.mark.parametrize("member", _DOTNET_CORE_ONLY_PATH_MEMBERS)
def test_shipped_ps1_avoids_dotnet_core_only_path_members(member: str) -> None:
"""No shipped .ps1 may call a .NET Core-only [System.IO.Path] member.
Windows PowerShell 5.1 ships on every Windows box and runs on .NET
Framework, where these members do not exist. A call is not a graceful
degradation but a hard "Method invocation failed" at the call site, which
for a root resolver aborts the command before it starts.
Runs on all platforms because the CI matrix only has pwsh (.NET Core),
where such a call works fine -- so this static check is what keeps CI able
to catch the regression at all.
"""
# Anchored to the Path type: [string]::Join and other same-named members on
# .NET Framework types are unaffected and must not be flagged.
pattern = re.compile(
r"\[(?:System\.IO\.)?Path\]::" + re.escape(member) + r"\s*\(",
re.IGNORECASE,
)
offenders = []
for ps1 in sorted(PROJECT_ROOT.glob("scripts/powershell/*.ps1")) + sorted(
PROJECT_ROOT.glob("extensions/*/scripts/powershell/*.ps1")
):
for lineno, line in enumerate(
ps1.read_text(encoding="utf-8").splitlines(), start=1
):
code = line.split("#", 1)[0]
if pattern.search(code):
offenders.append(f"{ps1.relative_to(PROJECT_ROOT)}:{lineno}")
assert not offenders, (
f"[System.IO.Path]::{member}() is .NET Core only and throws on Windows "
f"PowerShell 5.1; found at {offenders}. Use a .NET Framework-safe "
f"equivalent (e.g. TrimEnd('/', '\\') for a trailing separator)."
)
@requires_windows_powershell
def test_ps51_init_dir_resolves(tmp_path: Path) -> None:
"""SPECIFY_INIT_DIR must resolve under Windows PowerShell 5.1 (issue #3749).
Before the fix, Resolve-SpecifyInitDir called the .NET Core-only
[System.IO.Path]::TrimEndingDirectorySeparator, so every 5.1 invocation
threw at that line -- root resolution failed before the requested command
ran, and $initRoot stayed $null so the very next Join-Path threw too.
"""
web = _make_project(tmp_path, "web")
env = {**_clean_env(), "SPECIFY_INIT_DIR": str(web)}
result = subprocess.run(
[_WINDOWS_POWERSHELL, "-NoProfile", "-Command", f'. "{COMMON_PS}"; Get-RepoRoot'],
cwd=tmp_path,
capture_output=True,
text=True,
check=False,
env=env,
)
assert "does not contain a method named" not in result.stderr, result.stderr
assert result.returncode == 0, result.stderr
assert result.stdout.strip() == str(web)
@requires_windows_powershell
def test_ps51_init_dir_trailing_separator_trimmed(tmp_path: Path) -> None:
"""The 5.1-safe trim must still strip a trailing separator, for bash parity.
Resolve-Path echoes back the input's trailing separator; the bash resolver's
`cd && pwd` never yields one, so the two must agree.
"""
web = _make_project(tmp_path, "web")
for suffix in ("/", "\\"):
env = {**_clean_env(), "SPECIFY_INIT_DIR": f"{web}{suffix}"}
result = subprocess.run(
[
_WINDOWS_POWERSHELL,
"-NoProfile",
"-Command",
f'. "{COMMON_PS}"; Get-RepoRoot',
],
cwd=tmp_path,
capture_output=True,
text=True,
check=False,
env=env,
)
assert result.returncode == 0, result.stderr
assert result.stdout.strip() == str(web)
@requires_pwsh
def test_ps_drive_root_reports_root_not_bare_drive(tmp_path: Path) -> None:
"""A path that IS its own root must survive the trim intact.
A bare TrimEnd('/', '\\') -- the obvious 5.1-safe swap -- turns 'C:\\' into
'C:', which is not the drive root but a drive-relative reference that every
later path API re-resolves against the *current directory*. Validation would
then probe the wrong tree entirely and, on a cwd that happens to contain
.specify/, silently accept 'C:' as the project root. The drive root
normally has no .specify/, so assert on the error naming the intact root.
"""
root = Path(tmp_path.anchor or "/")
if (root / ".specify").exists():
pytest.skip("filesystem root is itself a Spec Kit project")
env = {**_clean_env(), "SPECIFY_INIT_DIR": str(root)}
result = _ps("Get-RepoRoot", cwd=tmp_path, env=env)
assert result.returncode != 0
assert "not a Spec Kit project" in result.stderr
# The error echoes the resolved root, so it pins what the trim produced:
# 'C:\' (or '/') intact, never the bare 'C:' (or '') a naive TrimEnd leaves.
reported = result.stderr.replace("\r", "").rstrip("\n").split("directory): ", 1)[-1]
assert reported == str(root)

View File

@@ -14,6 +14,7 @@ import pytest
import io
import json
import tempfile
import tarfile
import shutil
import warnings
import zipfile
@@ -197,6 +198,25 @@ class TestPresetManifest:
with pytest.raises(PresetValidationError, match="YAML mapping"):
PresetManifest(manifest_path)
@pytest.mark.parametrize("section", ["preset", "requires", "provides"])
@pytest.mark.parametrize("bad_value", [None, [], "text"])
def test_required_section_not_mapping_raises_validation_error(
self, temp_dir, valid_pack_data, section, bad_value
):
"""Required manifest sections reject null, list, and scalar values."""
valid_pack_data[section] = bad_value
manifest_path = temp_dir / "preset.yml"
manifest_path.write_text(
yaml.safe_dump(valid_pack_data),
encoding="utf-8",
)
with pytest.raises(
PresetValidationError,
match=rf"Invalid {section}: expected a mapping",
):
PresetManifest(manifest_path)
@pytest.mark.parametrize(
"bad",
[
@@ -670,6 +690,27 @@ class TestPresetManager:
assert manifest.id == "test-pack"
assert manager.registry.is_installed("test-pack")
def test_install_from_zip_forwards_force(
self, project_dir, pack_dir, temp_dir
):
"""The compatibility wrapper must retain forced reinstall behavior."""
zip_path = temp_dir / "test-pack.zip"
with zipfile.ZipFile(zip_path, "w") as zf:
for file_path in pack_dir.rglob("*"):
if file_path.is_file():
zf.write(file_path, file_path.relative_to(pack_dir))
manager = PresetManager(project_dir)
manager.install_from_directory(pack_dir, "0.1.5")
manifest = manager.install_from_zip(
zip_path,
"0.1.5",
force=True,
)
assert manifest.id == "test-pack"
assert manager.registry.is_installed("test-pack")
def test_install_from_zip_nested(self, project_dir, pack_dir, temp_dir):
"""Test installing from ZIP with nested directory."""
zip_path = temp_dir / "test-pack.zip"
@@ -715,6 +756,45 @@ class TestPresetManager:
assert not manager.registry.is_installed("test-pack")
@pytest.mark.parametrize("suffix", [".tar.gz", ".tgz"])
@pytest.mark.parametrize("nested", [False, True])
def test_install_from_tar_archive(
self, project_dir, pack_dir, temp_dir, suffix, nested
):
"""Tar archives install with the same flat/nested behavior as ZIP."""
archive_path = temp_dir / f"test-pack{suffix}"
with tarfile.open(archive_path, "w:gz") as archive:
for file_path in pack_dir.rglob("*"):
if file_path.is_file():
relative = file_path.relative_to(pack_dir)
arcname = Path("test-pack-v1") / relative if nested else relative
archive.add(file_path, arcname=arcname)
manager = PresetManager(project_dir)
manifest = manager.install_from_archive(archive_path, "0.1.5")
assert manifest.id == "test-pack"
assert manager.registry.is_installed("test-pack")
def test_install_from_tar_rejects_symlink_entry(
self, project_dir, pack_dir, temp_dir
):
archive_path = temp_dir / "symlink-preset.tar.gz"
with tarfile.open(archive_path, "w:gz") as archive:
for file_path in pack_dir.rglob("*"):
if file_path.is_file():
archive.add(file_path, arcname=file_path.relative_to(pack_dir))
link = tarfile.TarInfo("templates/escape")
link.type = tarfile.SYMTYPE
link.linkname = "../../outside"
archive.addfile(link)
manager = PresetManager(project_dir)
with pytest.raises(PresetValidationError, match="Unsafe symlink"):
manager.install_from_archive(archive_path, "0.1.5")
assert not manager.registry.is_installed("test-pack")
def test_remove(self, project_dir, pack_dir):
"""Test removing a preset."""
manager = PresetManager(project_dir)
@@ -2668,6 +2748,39 @@ class TestPresetCatalog:
assert captured[0].get_header("Authorization") == "Bearer ghp_testtoken"
assert captured[0].get_header("Accept") == "application/octet-stream"
@pytest.mark.parametrize("suffix", [".tar.gz", ".tgz"])
def test_download_pack_preserves_tar_archive_format(
self, project_dir, suffix
):
from unittest.mock import patch, MagicMock
archive_buffer = io.BytesIO()
with tarfile.open(fileobj=archive_buffer, mode="w:gz") as archive:
content = b"preset:\n id: test-pack\n"
member = tarfile.TarInfo("preset.yml")
member.size = len(content)
archive.addfile(member, io.BytesIO(content))
archive_bytes = archive_buffer.getvalue()
response = MagicMock()
response.read.side_effect = io.BytesIO(archive_bytes).read
response.__enter__.return_value = response
response.__exit__.return_value = False
catalog = PresetCatalog(project_dir)
pack_info = {
"id": "test-pack",
"name": "Test Pack",
"version": "1.0.0",
"download_url": f"https://example.com/test-pack{suffix}",
"_install_allowed": True,
}
with patch.object(catalog, "get_pack_info", return_value=pack_info), \
patch.object(catalog, "_open_url", return_value=response):
archive_path = catalog.download_pack("test-pack", target_dir=project_dir)
assert archive_path.name == "test-pack-1.0.0.tar.gz"
assert archive_path.read_bytes() == archive_bytes
# ===== Integration Tests =====
@@ -10084,7 +10197,7 @@ class TestBundledPresetLocator:
self.read_sizes.append(size)
return super().read(size)
response = FakeResponse(b"zip-bytes")
response = FakeResponse(b"PK\x05\x06" + b"\x00" * 18)
installed = {}
def fake_install_from_zip(self, zip_path, speckit_version, priority=10):
@@ -10105,7 +10218,7 @@ class TestBundledPresetLocator:
assert response.read_sizes
assert installed == {
"zip_bytes": b"zip-bytes",
"zip_bytes": b"PK\x05\x06" + b"\x00" * 18,
"speckit_version": "0.6.0",
"priority": 7,
}
@@ -12493,3 +12606,86 @@ class TestInstalledPresetRichMarkup:
assert "Composition chain" in output, output
assert "[base]" in output, output
assert "[append]" in output, output
class TestConstitutionSyncPreset:
"""The bundled opt-in ``constitution-sync`` preset re-adds propagation.
Follow-up to #3790: core ``/constitution`` no longer propagates guidance
into templates. This preset restores that behavior for teams that treat
materialized templates as reviewed artifacts, delivered as a ``wrap`` of
the core command so it stays forward-compatible with core changes.
"""
PRESET_DIR = Path(__file__).parent.parent / "presets" / "constitution-sync"
def test_manifest_provides_wrap_of_constitution(self):
manifest = yaml.safe_load((self.PRESET_DIR / "preset.yml").read_text())
assert manifest["preset"]["id"] == "constitution-sync"
entries = manifest["provides"]["templates"]
assert len(entries) == 1
entry = entries[0]
assert entry["type"] == "command"
assert entry["name"] == "speckit.constitution"
assert entry["strategy"] == "wrap"
# Must target the post-#3790 baseline so propagation is not double-applied.
assert manifest["requires"]["speckit_version"] == ">=0.14.4"
def test_wrapper_uses_core_template_and_propagates(self):
text = (self.PRESET_DIR / "commands" / "speckit.constitution.md").read_text()
# Parse the Markdown frontmatter as YAML rather than substring-matching,
# so `strategy: wrap` is asserted structurally (not as text that could
# appear in the body) and {CORE_TEMPLATE} is asserted in the body only.
assert text.startswith("---\n")
_, frontmatter_block, body = text.split("---", 2)
frontmatter = yaml.safe_load(frontmatter_block)
assert frontmatter["strategy"] == "wrap"
assert "{CORE_TEMPLATE}" in body
assert "strategy: wrap" not in body # only in frontmatter
# The three governed scaffolds the old checklist propagated into.
assert "plan-template.md" in body
assert "spec-template.md" in body
assert "tasks-template.md" in body
# Must not mutate versioned preset/extension artifacts.
assert "Do not edit versioned preset- or extension-provided template or command files" in body
def test_catalog_lists_bundled_preset(self):
manifest = yaml.safe_load((self.PRESET_DIR / "preset.yml").read_text())
catalog = json.loads((self.PRESET_DIR.parent / "catalog.json").read_text())
entry = catalog["presets"]["constitution-sync"]
assert entry["bundled"] is True
assert entry["version"] == manifest["preset"]["version"]
assert entry["provides"]["commands"] == 1
assert entry["provides"]["templates"] == 0
def test_wrap_composes_over_core_constitution(self, project_dir):
"""Installing the preset yields a wrap layer atop the bundled core."""
manager = PresetManager(project_dir)
manager.install_from_directory(self.PRESET_DIR, "0.15.0")
resolver = PresetResolver(project_dir)
layers = resolver.collect_all_layers("speckit.constitution", "command")
assert len(layers) >= 2, "expected preset wrap layer plus a core base"
assert layers[0]["strategy"] == "wrap"
assert any("constitution-sync" in str(layer["path"]) for layer in layers)
assert layers[-1]["source"] == "core (bundled)"
def test_resolved_content_embeds_core_and_sync_pass(self, project_dir):
"""resolve_content substitutes {CORE_TEMPLATE} so the effective command
contains both the bundled core body and the propagation pass."""
manager = PresetManager(project_dir)
manager.install_from_directory(self.PRESET_DIR, "0.15.0")
resolver = PresetResolver(project_dir)
content = resolver.resolve_content("speckit.constitution", "command")
assert content is not None
# {CORE_TEMPLATE} must be replaced, not left literal.
assert "{CORE_TEMPLATE}" not in content
# Core body is present (distinctive core-only heading).
assert "## Scope Guard" in content
# The wrapper's propagation pass is present and supersedes the guard.
assert "## Constitution Template Sync" in content
assert "supersedes the \"Scope Guard\" above" in content
assert "plan-template.md" in content

File diff suppressed because it is too large Load Diff

View File

@@ -104,6 +104,18 @@ def test_fetch_rejects_malformed_source_url_cleanly(url):
fetcher(_source(url))
@pytest.mark.parametrize("use_file_url", [False, True], ids=["path", "file-url"])
def test_local_catalog_decode_errors_are_wrapped(tmp_path, use_file_url):
catalog_path = tmp_path / "catalog.json"
catalog_path.write_bytes(b"\xff\xfe")
url = catalog_path.as_uri() if use_file_url else str(catalog_path)
fetcher = adapters.make_catalog_fetcher(allow_network=False)
with pytest.raises(BundlerError, match="Could not read"):
fetcher(_source(url))
def test_builtin_community_catalog_fetches_repository_catalog_online(monkeypatch):
captured: dict = {}

View File

@@ -207,4 +207,30 @@ def test_executable_bit_preserved_in_artifact(tmp_path: Path):
}
# Executable source -> 0755; plain text files -> 0644.
assert modes["scripts/hook.sh"] == 0o755
def test_toctou_stat_read_consistency(tmp_path: Path):
"""Regression: stat() and read() must use the same file descriptor.
The old implementation called file_path.stat() then file_path.read_bytes()
as separate syscalls. Between the two, another process could replace the
file. The fix opens the file once and uses os.fstat() + fh.read() on the
same handle. This test verifies the archived bytes and mode are consistent.
"""
bundle = _make_bundle(tmp_path / "b")
target = bundle / "assets" / "data.bin"
target.parent.mkdir(parents=True, exist_ok=True)
target.write_bytes(b"\x00\x01\x02\x03")
target.chmod(0o644)
result = build_bundle(bundle, output_dir=tmp_path / "out")
with zipfile.ZipFile(result.artifact_path) as archive:
content = archive.read("assets/data.bin")
modes = {
info.filename: (info.external_attr >> 16) & 0o777
for info in archive.infolist()
}
assert content == b"\x00\x01\x02\x03"
assert modes["assets/data.bin"] == 0o644
assert modes["README.md"] == 0o644

View File

@@ -20,6 +20,7 @@ from specify_cli.bundler.services.primitives import (
_WorkflowKindManager,
primitive_manager,
)
from tests.bundler_helpers import valid_manifest_dict
def _component(kind: str, cid: str = "x") -> ComponentRef:
@@ -215,3 +216,121 @@ def test_bundled_preset_pin_match_installs(tmp_path: Path, monkeypatch):
manager.install(ComponentRef(kind="presets", id="my-preset", version="1.0.0"))
manager.install(ComponentRef(kind="presets", id="my-preset", version=None))
assert len(called) == 2
def test_extension_refresh_calls_install_with_force(tmp_path: Path, monkeypatch):
"""_ExtensionKindManager.refresh() must pass force=True to install_from_directory
so an already-installed extension is overwritten instead of raising an error."""
import specify_cli._assets as assets
from specify_cli.extensions import ExtensionManager
bundled = _write_manifest(tmp_path / "ext", "extension", "1.0.0")
monkeypatch.setattr(assets, "_locate_bundled_extension", lambda cid: bundled)
force_values: list = []
monkeypatch.setattr(
ExtensionManager, "install_from_directory",
lambda self, *a, **k: force_values.append(k.get("force", False)),
)
manager = primitive_manager("extensions", tmp_path, allow_network=False)
manager.refresh(ComponentRef(kind="extensions", id="my-ext"))
assert force_values == [True], "refresh() must pass force=True"
def test_preset_refresh_calls_install_with_force(tmp_path: Path, monkeypatch):
"""_PresetKindManager.refresh() must pass force=True to install_from_directory
so an already-installed preset is overwritten instead of raising an error."""
import specify_cli._assets as assets
from specify_cli.presets import PresetManager
bundled = _write_manifest(tmp_path / "preset", "preset", "1.0.0")
monkeypatch.setattr(assets, "_locate_bundled_preset", lambda cid: bundled)
force_values: list = []
monkeypatch.setattr(
PresetManager, "install_from_directory",
lambda self, *a, **k: force_values.append(k.get("force", False)),
)
manager = primitive_manager("presets", tmp_path, allow_network=False)
manager.refresh(ComponentRef(kind="presets", id="my-preset"))
assert force_values == [True], "refresh() must pass force=True"
def test_default_installer_refresh_dispatches_to_kind_manager(tmp_path: Path, monkeypatch):
"""DefaultPrimitiveInstaller.refresh() must call the kind manager's refresh(),
which is the hook _refresh_component() will find — fixing the --force leak."""
import specify_cli._assets as assets
from specify_cli.extensions import ExtensionManager
bundled = _write_manifest(tmp_path / "ext", "extension", "1.0.0")
monkeypatch.setattr(assets, "_locate_bundled_extension", lambda cid: bundled)
force_values: list = []
monkeypatch.setattr(
ExtensionManager, "install_from_directory",
lambda self, *a, **k: force_values.append(k.get("force", False)),
)
installer = DefaultPrimitiveInstaller(allow_network=False)
installer.refresh(tmp_path, _component("extensions", "my-ext"))
assert force_values == [True], "DefaultPrimitiveInstaller.refresh() must use force=True"
def test_refresh_succeeds_and_passes_force_true(tmp_path: Path, monkeypatch):
"""Regression: bundle update (refresh=True) of an already-installed extension
must succeed and pass force=True to install_from_directory."""
from specify_cli.bundler.services.installer import install_bundle
from specify_cli.bundler.models.manifest import BundleManifest
import specify_cli._assets as assets
from specify_cli.extensions import ExtensionManager
bundled = _write_manifest(tmp_path / "ext", "extension", "1.0.0")
monkeypatch.setattr(assets, "_locate_bundled_extension", lambda cid: bundled)
# Simulate refresh succeeding (force=True removes the duplicate-install guard)
force_seen: list = []
def _fake_install_from_directory(self, *a, **k):
force_seen.append(k.get("force", False))
self.registry.add("my-ext", {"version": "1.0.0"})
monkeypatch.setattr(
ExtensionManager, "install_from_directory", _fake_install_from_directory
)
raw = valid_manifest_dict(
bundle={
"id": "test-bundle",
"name": "Test",
"version": "1.0.0",
"role": "developer",
"description": "Test bundle",
"author": "Spec Kit",
"license": "MIT",
},
provides={
"extensions": [{"id": "my-ext", "version": "1.0.0"}],
"presets": [],
"steps": [],
"workflows": [],
},
)
manifest = BundleManifest.from_dict(raw)
installer = DefaultPrimitiveInstaller(allow_network=False)
# First install
install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest)
# Refresh (bundle update) — must not raise with --force hint
install_bundle(tmp_path, _plan(manifest), installer, manifest=manifest, refresh=True)
# force=True must have been passed during the refresh call
assert True in force_seen, "refresh path should have called install_from_directory with force=True"
def _plan(manifest):
from specify_cli.bundler.services.installer import InstallPlan
from specify_cli.bundler.models.manifest import ComponentRef as CR
components = [CR(kind=c.kind, id=c.id) for c in manifest.components]
return InstallPlan(
bundle_id=manifest.bundle.id,
version=manifest.bundle.version,
role=manifest.bundle.role,
effective_integration=None,
components=components,
)

View File

@@ -509,6 +509,86 @@ class TestOverlayCli:
assert payload["layers"][-1]["tier"] == "base"
assert payload["layers"][-1]["priority"] is None
def test_workflow_resolve_prints_tier_labels(self, project_dir, monkeypatch):
"""Layer tiers render literally; an unescaped ``[base]`` is eaten as markup."""
monkeypatch.setattr("specify_cli._require_specify_project", lambda: project_dir)
_write_workflow(
project_dir,
"wf",
{
"schema_version": "1.0",
"workflow": {"id": "wf", "name": "WF", "version": "1.0.0"},
"steps": [{"id": "a", "type": "command", "command": "echo"}],
},
)
_write_overlay(
project_dir,
"wf",
"ov1",
{
"id": "ov1",
"extends": "wf",
"priority": 10,
"edits": [
{
"operation": "insert_after",
"anchor": "a",
"step": {"id": "new", "type": "command", "command": "echo"},
}
],
},
)
result = runner.invoke(app, ["workflow", "resolve", "wf"])
assert result.exit_code == 0, result.output
assert "[base]" in result.output
assert "[project-overlay]" in result.output
@pytest.mark.parametrize(
"step_id",
[
# Balanced tag: silently swallowed, so the step vanishes from output.
"new[stuff]",
# Unbalanced closer: raises MarkupError -> traceback and exit 1.
"new[/red]",
],
)
def test_workflow_resolve_escapes_rich_markup_in_step_id(
self, project_dir, monkeypatch, step_id
):
"""Step IDs are unvalidated for brackets, so they must be escaped."""
monkeypatch.setattr("specify_cli._require_specify_project", lambda: project_dir)
_write_workflow(
project_dir,
"wf",
{
"schema_version": "1.0",
"workflow": {"id": "wf", "name": "WF", "version": "1.0.0"},
"steps": [{"id": "a", "type": "command", "command": "echo"}],
},
)
_write_overlay(
project_dir,
"wf",
"ov1",
{
"id": "ov1",
"extends": "wf",
"priority": 10,
"edits": [
{
"operation": "insert_after",
"anchor": "a",
"step": {"id": step_id, "type": "command", "command": "echo"},
}
],
},
)
result = runner.invoke(app, ["workflow", "resolve", "wf"])
assert result.exit_code == 0, result.output
assert step_id in result.output
def test_workflow_resolve_equal_priority_layers_sort_by_source(self, project_dir, monkeypatch):
"""Equal-priority overlays are listed alphabetically by source."""
monkeypatch.setattr("specify_cli._require_specify_project", lambda: project_dir)