[bug-fix] Fix reinstall-overwrites-kept-config: preserve config on plain reinstall after --keep-config (#3449)

* Fix reinstall-overwrites-kept-config: preserve config on plain reinstall after --keep-config

Apply the remediation from the bug assessment on issue #3427.

Before the unconditional shutil.rmtree(dest_dir), scan dest_dir for any
*-config.yml and *-config.local.yml files and hold their contents in memory.
After shutil.copytree succeeds, write them back so user-customized values
always win over the packaged defaults.

This mirrors the existing backup/restore logic for the --force reinstall path
but handles the case where remove --keep-config left config files behind in
an unregistered extension directory.

Refs #3427

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix: restore method decl, move config restore before registration, preserve file mode

- Restore missing `test_install_force_without_existing` method declaration in
  tests/test_extensions.py so pytest collects it as a separate test.
- Move stranded-config restoration to immediately after `copytree`, before
  command/skill/hook registration, so a failed registration step can't leave
  preserved configs permanently lost.
- Store `(bytes, mode)` tuples instead of bare bytes when rescuing stranded
  configs, and reapply the original file mode after writing so permission bits
  (e.g. 0600 for credential files) are faithfully restored.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)

* fix: mask setuid/setgid bits when restoring stranded config file mode

Only preserve user/group read-write bits (mode & 0o660) to avoid
restoring setuid, setgid, or world-writable permissions from a
user-modified config file.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)

* fix: add copytree rollback path and strengthen regression test with packaged default config

- Wrap shutil.copytree in a try/except BaseException so stranded configs
  rescued before rmtree are written back even if copytree fails mid-way
  (addresses review comment: configs were permanently lost on copy failure)
- Add a packaged default config to extension_dir in the regression test so
  a naive 'restore only when absent' implementation would fail; assert the
  user's customized values beat the packaged defaults after reinstall

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

* fix: restore configs with secure atomic writes

Assisted-by: GitHub Copilot (model: gpt-5, autonomous)

* fix: write secure temp file then chmod to preserved_mode; add copytree-failure test

- _restore_stranded_config_file: write content while temp file is at its
  secure OS-default mode (typically 0600 on POSIX), then apply the
  original preserved_mode after the file is fully written and before the
  atomic os.replace. Removes the & 0o660 mask that was silently stripping
  world-read and executable bits (e.g. 0644 → 0640).

- Add test_copytree_failure_restores_stranded_config: patches
  shutil.copytree to create a partial destination then raise OSError,
  then asserts that the preserved config bytes and file mode are restored
  by the rollback path and that the extension remains unregistered.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

* Potential fix for pull request finding 'Unused local variable'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* Potential fix for pull request finding 'Module is imported with 'import' and 'import from''

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* fix: durable staging for stranded configs and import style fix

- Stage stranded config files to a durable rescue_staging_dir
  (extensions_dir/.rescue-staging-<id>) before rmtree so original bytes
  survive partial rmtree, copytree failure, or partial restore on retry.
  On retry the staging dir is detected and its content reused instead of
  whatever mix of packaged defaults and partial restores remains on disk.
  The staging dir is cleaned up only after every restore succeeds.
- Fix CodeQL: change `import specify_cli.extensions as _ext_module` to
  `from specify_cli import extensions as _ext_module` in test file.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

* fix: harden rescue staging dir - symlink checks, secure writes, cleanup errors

- Thread 14: Change except BaseException to except Exception in the staging
  fallback block so KeyboardInterrupt/SystemExit propagate correctly
- Thread 15: Add explanatory comment to the bare pass in the chmod except
  block to satisfy static analysis
- Thread 16: Reject a symlinked staging directory and only reload
  non-symlinked files whose names match the two recognised config suffixes
- Thread 17: Create each staging file via os.open with mode 0600 and
  O_CREAT|O_EXCL before writing so preserved bytes are never transiently
  exposed to other local users
- Thread 18: Remove ignore_errors=True from the final staging-dir cleanup
  so a failed rmtree propagates rather than silently leaving a stale
  backup that could be misread on the next retry

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

* fix: mask file-type bits from chmod, harden staging dir symlink check

- Add `import stat` to imports
- Use `stat.S_IMODE(mode)` before chmod in staging write (thread 20, line 1464)
- Use `stat.S_IMODE(preserved_mode)` and make chmod best-effort in
  `_restore_stranded_config_file` (thread 18, line 1492)
- Add `not rescue_staging_dir.is_symlink()` guard to cleanup (thread 19, line 1522)

Assisted-by: GitHub Copilot (model: claude-sonnet-4.6, autonomous)

* fix: use completion marker for rescue staging, abort on staging failure, full os.write

Assisted-by: GitHub Copilot (model: GPT-5.3-Codex, autonomous)

* fix(extensions): make rescue staging durable

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* test(extensions): fix flaky copytree regression test

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* test(extensions): fix module import alias for review feedback

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* fix(workflows): keep cleanup warnings single-line and remove dead helper

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Potential fix for pull request finding 'Module is imported with 'import' and 'import from''

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* Preserve rescued extension config across retry

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Clarify ignored directory fsync cleanup errors

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Fix reinstall durability and workflow cleanup warnings

Assisted-by: GitHub Copilot (model: MAI-Code-1-Flash, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Open rescue staging file in binary mode to fix Windows CRLF corruption

On Windows os.open() defaults to text mode, so os.write() of preserved
config bytes containing \r\n was translated to \r\r\n, corrupting the
staged backup and failing the retry-restore regression test. Add
O_BINARY (0 on POSIX) to the staging file open flags.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Load .extensionignore before deleting dest_dir on reinstall

The .extensionignore loader can raise ValidationError (invalid UTF-8) or
OSError. Previously it ran after dest_dir was removed, so such a failure
left the kept config only in the hidden staging directory rather than its
documented location. Load/validate it before the rmtree so every
post-deletion failure path restores the config. Adds a regression test.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Validate .extensionignore before publishing rescue staging

Loading .extensionignore after the rescue staging directory was published
meant a validation failure left a complete staging copy behind. A later
retry (after the user fixed the ignore file and edited the kept config)
would reload the stale staged bytes and silently overwrite the newer
config. Move the loader ahead of reading/creating rescue staging so a
failure aborts while the kept config is still authoritative on disk, and
extend the regression test to prove no staging is published and a retry
adopts the newer bytes.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Harden preserved-config rescue against divergence and long names

Address three review findings on the reinstall config-rescue path:

- A complete .rescue-complete marker proves only that staging finished,
  not that dest_dir was modified. A crash after staging sync but before
  the rmtree leaves the live kept config intact; if the user edits it
  before retrying, preferring the staged bytes silently overwrote the
  newer config. The two copies are indistinguishable in provenance from
  disk, so detect divergence between a complete staging copy and the live
  config and abort (preserving both) instead of unconditionally choosing
  staging.
- The staging directory embedded the full extension ID in one path
  component. Extension IDs are length-unbounded, so a valid long ID could
  install at dest_dir yet fail every reinstall-after-keep-config with
  ENAMETOOLONG. Derive the staging component from a fixed-length hash via
  a new _rescue_staging_dir() helper.
- The stranded-config restore used the full config filename as a
  NamedTemporaryFile prefix; a name already near the component limit plus
  the random suffix raised ENAMETOOLONG. Use a short fixed prefix.

Updates the retry regression test to the new divergence semantics and
adds conflict-abort, long-ID, and fixed-prefix coverage.

Assisted-by: GitHub Copilot (model: Claude Opus 4.8, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

* Harden preserved-config rescue divergence check and fix test path

Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

* fix: reject/flag symlinked preserved configs on reinstall

Assisted-by: GitHub Copilot (model: GPT-5.6-Sol, autonomous)

* fix: include symlinks in live-dir config enumeration and address review feedback

- _recognized_config_names() now accepts follow_symlinks=False for live dir
  so symlinked *-config.yml entries are detected and treated as conflicts
  rather than being silently deleted by rmtree.
- Add explanatory comment to bare 'except OSError: pass' in
  _restore_stranded_config_file's finally block.
- Resolve CodeQL dual-import style: use 'from specify_cli import extensions
  as _ext_module' instead of 'import specify_cli.extensions as _ext_module'.

Assisted-by: GitHub Copilot (model: claude-sonnet-4, autonomous)

* test: add staging-failure fault-injection test for rescue staging block

Add test_staging_failure_aborts_before_dest_dir_removal covering three
failure modes (mkdir, os.open/O_CREAT, fsync with EIO) in the rescue
staging block. Each parametrized case verifies:
- the install aborts before dest_dir is removed
- the preserved config bytes remain authoritative
- any partial staging is cleaned up and not left as complete
- the extension stays unregistered

Addresses review feedback on PRRT_kwDOPiFCnc6R351t.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)

* test: add test_retry_restores_config_from_staging_when_live_absent

Exercises the retry-from-staging branch (if staging_is_complete at
line 1505 of extensions/__init__.py) in a scenario where the live
config is absent — simulating a power loss that interrupted the
rollback before it could write the config back.

When the live copy is gone, the live-dir fallback (elif dest_dir.exists())
finds no stranded configs and the packaged default would be kept. Only the
staging-complete branch can restore the original bytes and mode. This proves
staging (not the fallback) is used on retry.

Addresses review feedback on PRRT_kwDOPiFCnc6SAL3L.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)

* fix: keep staging files writable; record modes in .rescue-modes.json; fix live-only conflict message

Thread 64: Remove os.fchmod/chmod from staged files to avoid Windows
read-only attribute that prevents shutil.rmtree from cleaning up.
Original permission bits are now written to a .rescue-modes.json sidecar
in the staging dir and reloaded during retry, with a fall-back to the
staged file's own mode for backwards-compat with pre-sidecar staging dirs.

Thread 65: Split the ValidationError message for staging-vs-live conflicts
into two accurate cases: files that diverged between both locations
("Both copies have been preserved") and live-only files that have no
backup counterpart, which previously incorrectly claimed "Both copies
have been preserved" and offered a restore instruction that was impossible.

Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)

* Potential fix for pull request finding 'Empty except'

Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>

* fix: add .keep-config provenance marker to guard rescue path against partially-failed installs

When `remove --keep-config` strands config files, write a `.keep-config`
marker into the extension directory.  `install_from_directory` now only
enters the rescue path when that marker is present, preventing a partially-
failed install (which also leaves dest_dir with no registry entry but no
marker) from having its packaged default configs treated as user-preserved
data on a retry from an updated package.

Refs: https://github.com/github/spec-kit/pull/3449#discussion_r3606283457

Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)

* refactor: extract _has_keep_config_marker helper and document empty-content choice

Assisted-by: GitHub Copilot (model: claude-sonnet-4.5, autonomous)

* fix: defer rescue-backup cleanup until registry commit; validate modes sidecar shape

Assisted-by: GitHub Copilot (model: GPT-5.6, autonomous)

* fix legacy keep-config rescue and retry baseline handling

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: Manfred Riem <15701806+mnriem@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>
This commit is contained in:
github-actions[bot]
2026-07-21 10:22:07 -05:00
committed by GitHub
parent 74662cffad
commit eabfabb490
3 changed files with 1315 additions and 6 deletions

View File

@@ -22,6 +22,7 @@ from datetime import datetime, timezone
from unittest.mock import MagicMock
from tests.conftest import strip_ansi
from specify_cli import extensions as _ext_module
from specify_cli.extensions import (
CatalogEntry,
CORE_COMMAND_NAMES,
@@ -1226,6 +1227,785 @@ class TestExtensionManager:
assert new_config.exists()
assert new_config.read_text() == "test: config"
def test_reinstall_after_keep_config_preserves_config(
self, extension_dir, project_dir
):
"""Reinstalling after `remove --keep-config` must not overwrite preserved config."""
manager = ExtensionManager(project_dir)
# Add a packaged default config so the reinstall has a file to overwrite.
# Without the fix, the packaged default would silently win on reinstall.
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\nmax_iterations: 1\n")
# Install once (packaged default is copied into the installed directory)
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
# Overwrite the installed config with user-customized values
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
config_file = ext_dir / "test-ext-config.yml"
config_file.write_text("model: custom-model\nmax_iterations: 99\n")
# Remove while preserving config
manager.remove("test-ext", keep_config=True)
assert not manager.registry.is_installed("test-ext")
assert config_file.exists()
assert "custom-model" in config_file.read_text()
# Plain reinstall (no --force) — packaged default is still present in
# extension_dir, so a naive implementation would overwrite the custom values.
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
# Preserved config must survive the reinstall and beat the packaged default
assert config_file.exists()
assert "custom-model" in config_file.read_text()
assert "99" in config_file.read_text()
assert "default-model" not in config_file.read_text()
def test_reinstall_after_keep_config_preserves_local_config(
self, extension_dir, project_dir
):
"""Local config override files (*-config.local.yml) are also rescued on reinstall."""
manager = ExtensionManager(project_dir)
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
local_cfg = ext_dir / "test-ext-config.local.yml"
local_cfg.write_text("local_override: true\n")
manager.remove("test-ext", keep_config=True)
assert local_cfg.exists()
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
assert local_cfg.exists()
assert "local_override: true" in local_cfg.read_text()
def test_reinstall_after_legacy_keep_config_preserves_config(
self, extension_dir, project_dir
):
"""Pre-marker keep-config leftovers are still rescued on reinstall."""
manager = ExtensionManager(project_dir)
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\nmax_iterations: 1\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
config_file = ext_dir / "test-ext-config.yml"
config_file.write_text("model: legacy-custom-model\nmax_iterations: 99\n")
manager.remove("test-ext", keep_config=True)
(ext_dir / ".keep-config").unlink()
assert not manager.registry.is_installed("test-ext")
assert config_file.exists()
assert "legacy-custom-model" in config_file.read_text()
packaged_config.write_text("model: upgraded-default-model\nmax_iterations: 2\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
assert config_file.exists()
assert "legacy-custom-model" in config_file.read_text()
assert "99" in config_file.read_text()
assert "upgraded-default-model" not in config_file.read_text()
def test_reinstall_with_symlinked_config_rejects_install(
self, extension_dir, project_dir
):
"""A preserved symlinked config must abort reinstall, not be deleted."""
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
if not can_create_symlink(ext_dir.parent if ext_dir.parent.exists() else project_dir):
pytest.skip("Current platform/user cannot create symlinks")
manager = ExtensionManager(project_dir)
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
# Replace the installed config with a symlink to a file outside dest_dir.
config_file = ext_dir / "test-ext-config.yml"
external_target = project_dir / "external-config.yml"
external_target.write_text("model: linked-model\n")
config_file.unlink()
os.symlink(external_target, config_file)
assert config_file.is_symlink()
# `remove --keep-config` follows the symlink via is_file() and keeps it.
manager.remove("test-ext", keep_config=True)
assert not manager.registry.is_installed("test-ext")
assert config_file.is_symlink()
# Plain reinstall must reject rather than silently delete the link.
with pytest.raises(ValidationError, match="is a symlink"):
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
# The symlink and its target survive; nothing was silently discarded.
assert config_file.is_symlink()
assert external_target.read_text() == "model: linked-model\n"
assert not manager.registry.is_installed("test-ext")
def test_retry_with_symlinked_live_config_aborts_and_preserves_both(
self, extension_dir, project_dir, monkeypatch
):
"""A live config replaced by a symlink on retry is a conflict, not overwritten."""
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
if not can_create_symlink(project_dir):
pytest.skip("Current platform/user cannot create symlinks")
manager = ExtensionManager(project_dir)
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
config_file = ext_dir / "test-ext-config.yml"
config_file.write_text("model: custom-model\nmax_iterations: 99\n")
staged_bytes = config_file.read_bytes()
manager.remove("test-ext", keep_config=True)
assert not manager.registry.is_installed("test-ext")
staging_dir = manager._rescue_staging_dir("test-ext")
original_copytree = shutil.copytree
copytree_calls = 0
def flaky_copytree(*args, **kwargs):
nonlocal copytree_calls
copytree_calls += 1
if copytree_calls == 1:
dst = args[1]
Path(dst).mkdir(parents=True, exist_ok=True)
(Path(dst) / "_partial.txt").write_text("partial")
raise OSError("simulated disk full")
return original_copytree(*args, **kwargs)
monkeypatch.setattr(_ext_module.shutil, "copytree", flaky_copytree)
with pytest.raises(OSError, match="simulated disk full"):
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
assert staging_dir.exists()
assert (staging_dir / ".rescue-complete").exists()
# Simulate the user replacing the live config with a symlink before retry.
external_target = project_dir / "external-config.yml"
external_target.write_text("model: newer-linked-model\n")
config_file.unlink()
os.symlink(external_target, config_file)
assert config_file.is_symlink()
with pytest.raises(ValidationError, match="Preserved extension config conflict"):
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
# Both copies survive: the live symlink choice and the staged backup.
assert config_file.is_symlink()
assert external_target.read_text() == "model: newer-linked-model\n"
assert staging_dir.exists()
assert (staging_dir / "test-ext-config.yml").read_bytes() == staged_bytes
assert not manager.registry.is_installed("test-ext")
def test_copytree_failure_restores_stranded_config(
self, extension_dir, project_dir, monkeypatch
):
"""A copytree failure must not permanently lose a preserved config.
When copytree raises after the existing directory has been removed, the
rollback path must write the rescued bytes back to dest_dir and restore
the original file mode, while leaving the extension unregistered.
"""
import stat
manager = ExtensionManager(project_dir)
# Add a packaged default config so copytree would overwrite it on success.
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\n")
# Install once so the extension is on disk.
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
config_file = ext_dir / "test-ext-config.yml"
config_file.write_text("model: custom-model\nmax_iterations: 99\n")
# Set a known, non-default mode so we can assert it survives the rollback.
if platform.system() != "Windows":
config_file.chmod(0o640)
original_bytes = config_file.read_bytes()
original_mode = config_file.stat().st_mode
# Remove while preserving the config — it is now a stranded file.
manager.remove("test-ext", keep_config=True)
assert not manager.registry.is_installed("test-ext")
assert config_file.exists()
# Make copytree create a partial destination then raise so the rollback
# path is exercised.
def failing_copytree(src, dst, **kwargs):
Path(dst).mkdir(parents=True, exist_ok=True)
(Path(dst) / "_partial.txt").write_text("partial")
raise OSError("simulated disk full")
monkeypatch.setattr(_ext_module.shutil, "copytree", failing_copytree)
with pytest.raises(OSError, match="simulated disk full"):
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
# The preserved config must have been written back by the rollback path.
assert config_file.exists(), "rollback must recreate the config file"
assert config_file.read_bytes() == original_bytes
# On POSIX, the original file mode must be faithfully restored.
if platform.system() != "Windows":
restored_mode = config_file.stat().st_mode
assert stat.S_IMODE(restored_mode) == stat.S_IMODE(original_mode)
# The extension must remain unregistered after the failed install.
assert not manager.registry.is_installed("test-ext")
def test_extensionignore_load_failure_preserves_kept_config(
self, extension_dir, project_dir
):
"""An .extensionignore load failure must not lose a preserved config.
`.extensionignore` is loaded/validated before the rescue staging
directory is read or created (and thus before dest_dir is deleted), so a
ValidationError raised for invalid UTF-8 must abort the reinstall while
leaving the kept config authoritative in its documented location. It must
NOT publish a stale staging copy that a later retry — after the user
fixes the ignore file and edits the kept config — would reload and use to
silently overwrite the newer bytes.
"""
manager = ExtensionManager(project_dir)
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
config_file = ext_dir / "test-ext-config.yml"
config_file.write_text("model: custom-model\nmax_iterations: 99\n")
original_bytes = config_file.read_bytes()
manager.remove("test-ext", keep_config=True)
assert not manager.registry.is_installed("test-ext")
assert config_file.exists()
# Author an .extensionignore that is not valid UTF-8 so the loader
# raises before rescue staging is read or created.
(extension_dir / ".extensionignore").write_bytes(b"\xff\xfe invalid\n")
with pytest.raises(ValidationError, match="not valid UTF-8"):
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
# The kept config must remain in its documented location, untouched.
assert config_file.exists(), "config must survive the ignore-load failure"
assert config_file.read_bytes() == original_bytes
assert not manager.registry.is_installed("test-ext")
# No rescue staging may have been published, so a later retry reads the
# live (possibly newly edited) config rather than stale staged bytes.
staging_dir = manager._rescue_staging_dir("test-ext")
assert not staging_dir.exists()
# Simulate the user fixing the ignore file and editing the kept config,
# then retrying: the retry must adopt the newer bytes, never a stale
# staged copy.
(extension_dir / ".extensionignore").write_text("*.log\n")
config_file.write_text("model: newer-model\n")
newer_bytes = config_file.read_bytes()
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
assert manager.registry.is_installed("test-ext")
assert config_file.read_bytes() == newer_bytes
def test_retry_after_staging_backup_restores_stranded_config(
self, extension_dir, project_dir, monkeypatch
):
"""A retry after an interrupted install restores the rescued config.
When the live config is unchanged since the interrupted attempt (it
still matches the staged backup), the retry proceeds and yields the
preserved bytes, and the staging directory is cleaned up on success.
"""
import stat
manager = ExtensionManager(project_dir)
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
config_file = ext_dir / "test-ext-config.yml"
config_file.write_text("model: custom-model\nmax_iterations: 99\n")
if platform.system() != "Windows":
config_file.chmod(0o640)
original_bytes = config_file.read_bytes()
original_mode = config_file.stat().st_mode
manager.remove("test-ext", keep_config=True)
assert not manager.registry.is_installed("test-ext")
assert config_file.exists()
staging_dir = manager._rescue_staging_dir("test-ext")
assert not staging_dir.exists()
original_copytree = shutil.copytree
copytree_calls = 0
def flaky_copytree(*args, **kwargs):
nonlocal copytree_calls
copytree_calls += 1
if copytree_calls == 1:
dst = args[1]
Path(dst).mkdir(parents=True, exist_ok=True)
(Path(dst) / "_partial.txt").write_text("partial")
raise OSError("simulated disk full")
return original_copytree(*args, **kwargs)
monkeypatch.setattr(_ext_module.shutil, "copytree", flaky_copytree)
with pytest.raises(OSError, match="simulated disk full"):
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
assert staging_dir.exists()
assert (staging_dir / ".rescue-complete").exists()
assert (staging_dir / "test-ext-config.yml").exists()
# The rollback restored the preserved bytes to the live config, so it
# still agrees with the staged backup — the retry proceeds normally.
assert config_file.read_bytes() == original_bytes
manifest = manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
assert manifest.id == "test-ext"
assert manager.registry.is_installed("test-ext")
assert config_file.read_bytes() == original_bytes
assert not (ext_dir / "_partial.txt").exists()
assert not staging_dir.exists()
if platform.system() != "Windows":
restored_mode = config_file.stat().st_mode
assert stat.S_IMODE(restored_mode) == stat.S_IMODE(original_mode)
def test_retry_restores_config_from_staging_when_live_absent(
self, extension_dir, project_dir, monkeypatch
):
"""Retry succeeds using only staging when the live config is absent.
After a copytree failure, staging is complete and the rollback writes
the config back to dest_dir. If a power loss interrupts that rollback
the config may be absent on the next attempt. The retry-from-staging
branch (``if staging_is_complete``) must restore the config from staging
alone so the original bytes and mode are recovered even when no live copy
is present. This is the critical path that distinguishes the staging
branch from the live-dir fallback.
"""
import stat
manager = ExtensionManager(project_dir)
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
config_file = ext_dir / "test-ext-config.yml"
config_file.write_text("model: custom-model\nmax_iterations: 99\n")
if platform.system() != "Windows":
config_file.chmod(0o640)
original_bytes = config_file.read_bytes()
original_mode = config_file.stat().st_mode
manager.remove("test-ext", keep_config=True)
assert not manager.registry.is_installed("test-ext")
assert config_file.exists()
staging_dir = manager._rescue_staging_dir("test-ext")
assert not staging_dir.exists()
original_copytree = shutil.copytree
copytree_calls = 0
def flaky_copytree(*args, **kwargs):
nonlocal copytree_calls
copytree_calls += 1
if copytree_calls == 1:
dst = args[1]
Path(dst).mkdir(parents=True, exist_ok=True)
(Path(dst) / "_partial.txt").write_text("partial")
raise OSError("simulated disk full")
return original_copytree(*args, **kwargs)
monkeypatch.setattr(_ext_module.shutil, "copytree", flaky_copytree)
with pytest.raises(OSError, match="simulated disk full"):
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
# Staging is complete after the first failure.
assert staging_dir.exists()
assert (staging_dir / ".rescue-complete").exists()
assert (staging_dir / "test-ext-config.yml").exists()
# Simulate a power loss that prevented the rollback from writing the
# config back: delete the live copy so the retry must rely on staging.
config_file.unlink()
assert not config_file.exists()
manifest = manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
# The staging branch must restore the original bytes even though no live
# copy was present — proving staging (not the live-dir fallback) was used.
assert manifest.id == "test-ext"
assert manager.registry.is_installed("test-ext")
assert config_file.read_bytes() == original_bytes
assert not (ext_dir / "_partial.txt").exists()
assert not staging_dir.exists()
if platform.system() != "Windows":
restored_mode = config_file.stat().st_mode
assert stat.S_IMODE(restored_mode) == stat.S_IMODE(original_mode)
def test_retry_ignores_live_only_packaged_config_after_registry_failure(
self, extension_dir, project_dir, monkeypatch
):
"""Retry should keep packaged live-only configs that still match source."""
manager = ExtensionManager(project_dir)
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
config_file = ext_dir / "test-ext-config.yml"
config_file.write_text("model: custom-model\nmax_iterations: 99\n")
original_bytes = config_file.read_bytes()
manager.remove("test-ext", keep_config=True)
assert not manager.registry.is_installed("test-ext")
live_only_packaged = extension_dir / "test-ext-config.local.yml"
live_only_packaged.write_text("new_default: true\n")
original_add = manager.registry.add
add_calls = 0
def flaky_add(*args, **kwargs):
nonlocal add_calls
add_calls += 1
if add_calls == 1:
raise OSError("simulated registry failure")
return original_add(*args, **kwargs)
monkeypatch.setattr(manager.registry, "add", flaky_add)
with pytest.raises(OSError, match="simulated registry failure"):
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
staging_dir = manager._rescue_staging_dir("test-ext")
assert staging_dir.exists()
assert (staging_dir / ".rescue-complete").exists()
assert config_file.read_bytes() == original_bytes
assert (
ext_dir / "test-ext-config.local.yml"
).read_text() == "new_default: true\n"
manifest = manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
assert manifest.id == "test-ext"
assert manager.registry.is_installed("test-ext")
assert config_file.read_bytes() == original_bytes
assert (
ext_dir / "test-ext-config.local.yml"
).read_text() == "new_default: true\n"
assert not staging_dir.exists()
def test_retry_with_edited_live_config_aborts_and_preserves_both(
self, extension_dir, project_dir, monkeypatch
):
"""A retry must not silently overwrite a config edited after a crash.
A complete staging directory proves only that staging finished, not
that dest_dir was modified. If the user edits the live kept config
before retrying, the retry must detect the divergence, preserve both
copies, and abort rather than blindly restoring the older staged bytes.
"""
manager = ExtensionManager(project_dir)
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
config_file = ext_dir / "test-ext-config.yml"
config_file.write_text("model: custom-model\nmax_iterations: 99\n")
staged_bytes = config_file.read_bytes()
manager.remove("test-ext", keep_config=True)
assert not manager.registry.is_installed("test-ext")
staging_dir = manager._rescue_staging_dir("test-ext")
original_copytree = shutil.copytree
copytree_calls = 0
def flaky_copytree(*args, **kwargs):
nonlocal copytree_calls
copytree_calls += 1
if copytree_calls == 1:
dst = args[1]
Path(dst).mkdir(parents=True, exist_ok=True)
(Path(dst) / "_partial.txt").write_text("partial")
raise OSError("simulated disk full")
return original_copytree(*args, **kwargs)
monkeypatch.setattr(_ext_module.shutil, "copytree", flaky_copytree)
with pytest.raises(OSError, match="simulated disk full"):
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
assert staging_dir.exists()
assert (staging_dir / ".rescue-complete").exists()
# Simulate the user editing the live config before retrying so it now
# diverges from the staged backup.
config_file.write_text("model: newer-edited-model\n")
edited_bytes = config_file.read_bytes()
assert edited_bytes != staged_bytes
with pytest.raises(ValidationError, match="Preserved extension config conflict"):
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
# Both copies must survive: the edited live config and the staged backup.
assert config_file.read_bytes() == edited_bytes
assert staging_dir.exists()
assert (staging_dir / "test-ext-config.yml").read_bytes() == staged_bytes
assert not manager.registry.is_installed("test-ext")
@pytest.mark.parametrize(
"failure_mode",
[
pytest.param("mkdir", id="mkdir"),
pytest.param("os_open", id="os_open"),
pytest.param("fsync", id="fsync"),
],
)
def test_staging_failure_aborts_before_dest_dir_removal(
self, extension_dir, project_dir, monkeypatch, failure_mode
):
"""Staging failures abort the install before dest_dir is removed.
When mkdir, os.open, or fsync fails while publishing rescue staging,
the install must abort before removing dest_dir so the preserved
config bytes remain authoritative, any partial staging is cleaned up
rather than trusted on retry, and the extension stays unregistered.
"""
import errno as _errno
manager = ExtensionManager(project_dir)
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
config_file = ext_dir / "test-ext-config.yml"
config_file.write_text("model: custom-model\nmax_iterations: 99\n")
original_bytes = config_file.read_bytes()
manager.remove("test-ext", keep_config=True)
assert not manager.registry.is_installed("test-ext")
assert config_file.exists()
staging_dir = manager._rescue_staging_dir("test-ext")
assert not staging_dir.exists()
if failure_mode == "mkdir":
original_mkdir = Path.mkdir
def failing_mkdir(self_path, *args, **kwargs):
if self_path == staging_dir:
raise OSError("staging mkdir failed")
return original_mkdir(self_path, *args, **kwargs)
monkeypatch.setattr(Path, "mkdir", failing_mkdir)
elif failure_mode == "os_open":
original_os_open = _ext_module.os.open
def failing_os_open(path, flags, mode=0o777, *args, **kwargs):
# Fail only for file creation (O_CREAT) inside the staging
# directory so other os.open calls (e.g. directory fsync) are
# unaffected.
if str(staging_dir) in str(path) and (flags & os.O_CREAT):
raise OSError(_errno.ENOSPC, "No space left on device")
return original_os_open(path, flags, mode, *args, **kwargs)
monkeypatch.setattr(_ext_module.os, "open", failing_os_open)
else: # "fsync"
def failing_fsync_fd(fd: int) -> None:
# Simulate a real storage error (EIO) that the helper propagates.
raise OSError(_errno.EIO, "Input/output error")
monkeypatch.setattr(_ext_module, "_fsync_fd", failing_fsync_fd)
with pytest.raises(OSError):
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
# dest_dir must still exist — the install aborted before rmtree.
assert ext_dir.exists(), "dest_dir must survive a staging failure"
assert config_file.read_bytes() == original_bytes, (
"preserved config must remain authoritative"
)
# Partial staging must have been cleaned up and not left as complete.
assert not staging_dir.exists() or not (
staging_dir / ".rescue-complete"
).exists(), "incomplete staging must not be trusted"
assert not manager.registry.is_installed("test-ext")
def test_rescue_staging_dir_is_fixed_length_for_long_ids(self, project_dir):
"""The rescue staging component length must not grow with the ID length.
Manifest validation caps the ID character set but not its length, so a
very long (but valid) ID must not lengthen the single staging path
component past a filesystem's per-component byte limit.
"""
manager = ExtensionManager(project_dir)
short_dir = manager._rescue_staging_dir("a")
long_id = "a" * 250
long_dir = manager._rescue_staging_dir(long_id)
# Same fixed component length regardless of ID length.
assert len(short_dir.name) == len(long_dir.name)
# Comfortably within the common 255-byte component limit.
assert len(long_dir.name.encode("utf-8")) <= 255
# Distinct IDs still map to distinct staging directories.
assert manager._rescue_staging_dir("b") != short_dir
def test_failed_install_without_keep_config_does_not_rescue_defaults(
self, extension_dir, project_dir, monkeypatch
):
"""A dir left by a partially-failed install must not trigger the rescue path.
Any install that copies files but then fails during command, skill, or
hook registration also leaves a complete dest_dir with no registry entry.
On a later retry from an updated package this branch must not treat the
previous package's default config as user-preserved data and restore it
over the new defaults. Only directories explicitly left by
``remove --keep-config`` (which writes a ``.keep-config`` marker) should
trigger the rescue path.
"""
manager = ExtensionManager(project_dir)
packaged_config = extension_dir / "test-ext-config.yml"
packaged_config.write_text("model: default-model\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
ext_dir = project_dir / ".specify" / "extensions" / "test-ext"
config_file = ext_dir / "test-ext-config.yml"
# Simulate a partially-failed install: the extension directory is present
# with the packaged default config but there is no .keep-config marker and
# the extension is not in the registry. This matches what happens when
# copytree succeeds but command/hook registration raises afterwards.
manager.registry.remove("test-ext")
assert not manager.registry.is_installed("test-ext")
assert ext_dir.exists()
assert not (ext_dir / ".keep-config").exists()
# Update the packaged config so a retry with the new package would use
# different defaults — the old defaults must NOT be rescued.
packaged_config.write_text("model: updated-default-model\n")
manager.install_from_directory(
extension_dir, "0.1.0", register_commands=False
)
assert manager.registry.is_installed("test-ext")
# The new packaged default must win; the old default was not user data.
assert config_file.read_text() == "model: updated-default-model\n"
def test_install_force_without_existing(self, extension_dir, project_dir):
"""Test force-install when extension is NOT already installed (works normally)."""
manager = ExtensionManager(project_dir)