mirror of
https://github.com/github/spec-kit.git
synced 2026-07-08 23:59:55 +08:00
fix(cli): pin UTF-8 encoding on init-options and .extensionignore I/O (#2686)
* fix(cli): pin UTF-8 encoding on init-options and .extensionignore I/O
``Path.read_text`` / ``Path.write_text`` default to the system locale
codec, which is cp1252 / gb2312 / cp932 on Windows. Two user-facing
file paths in spec-kit were calling them without an explicit
``encoding=`` argument:
- ``src/specify_cli/__init__.py:400,412`` —
``save_init_options`` / ``load_init_options`` for
``.specify/init-options.json``. A peer machine with a different
default locale (or a UTF-8 Unix CI runner reading a file written on
a cp1252 Windows host) cannot decode the file, raising
``UnicodeDecodeError``. ``UnicodeDecodeError`` is a subclass of
``ValueError`` — not ``OSError`` / ``json.JSONDecodeError`` — so
the existing fall-back ``except`` tuple in ``load_init_options``
also misses it and the error propagates raw to the CLI.
- ``src/specify_cli/extensions.py:764`` — ``.extensionignore``
pattern reader. The very next line already normalises
backslashes "so Windows-authored files work", proving the codebase
expects Windows authors to write this file. Multibyte UTF-8
patterns (Chinese filenames, accented directory names) silently
mojibake when the host locale is not UTF-8, so the patterns fail
to match and unintended files are shipped with the extension.
The sibling integration-catalog reader at
``src/specify_cli/integrations/catalog.py:150,156,193,202,374``
already pins ``encoding="utf-8"`` everywhere. PR #2280 fixed the
symmetric PowerShell-template BOM bug. This change brings the two
remaining drifted paths in line with that precedent.
Regression tests:
- ``tests/test_presets.py::TestInitOptions`` — parametrized non-ASCII
round-trip (CJK, Latin-1, Greek, emoji) plus a corrupted-file case
that asserts the existing "fall back to {}" contract still holds
when a peer file contains bytes invalid as UTF-8.
- ``tests/test_extensions.py::TestExtensionIgnore`` — Japanese
(``ドキュメント/``) and Latin-1 (``café/``) ignore patterns
correctly exclude their directories during install.
* fix(cli): wrap .extensionignore decode error and tighten UTF-8 contract
Addresses Copilot review feedback on this PR.
Three issues, three fixes:
1. ``save_init_options`` now writes JSON with ``ensure_ascii=False``.
Without that flag, ``json.dumps`` emits ASCII-only ``\uXXXX``
escapes, which means the ``encoding="utf-8"`` pin on the
surrounding ``Path.write_text`` makes no observable difference for
any value we currently write. Flipping ``ensure_ascii`` makes the
non-ASCII bytes hit the file directly, so the encoding pin becomes
the thing that decides between cp1252 garbage and clean UTF-8 on
Windows. The comment above the call now describes the real reason
instead of the previously-misleading rationale Copilot flagged.
2. ``test_save_load_round_trip_preserves_non_ascii`` was a no-op under
the old ``ensure_ascii=True`` writer (Copilot's second comment).
Added ``test_save_writes_real_utf8_bytes`` that asserts the on-disk
bytes contain the UTF-8 encoding of ``café`` (``0xC3 0xA9``), not
its JSON escape form ``é``. Removing either
``ensure_ascii=False`` or ``encoding="utf-8"`` from the writer now
breaks this test — the contract is pinned.
3. ``.extensionignore`` reader wraps ``UnicodeDecodeError`` as
``ValidationError`` with a pointer to the offending byte
(Copilot's third comment). Mirrors
``ExtensionManifest._load_yaml``'s existing handler for
``extension.yml``. Adds
``test_extensionignore_invalid_utf8_raises_validation_error``
asserting installation aborts with the wrapped error instead of a
raw Python traceback.
This commit is contained in:
@@ -761,7 +761,28 @@ class ExtensionManager:
|
||||
if not ignore_file.exists():
|
||||
return None
|
||||
|
||||
lines: List[str] = ignore_file.read_text().splitlines()
|
||||
# Pin UTF-8 explicitly: ``Path.read_text`` defaults to the system
|
||||
# locale codec on Windows (cp1252 / gb2312 / cp932), which silently
|
||||
# corrupts multibyte patterns when the file is shared across
|
||||
# machines with different locales. The next line already
|
||||
# normalises backslashes "so Windows-authored files work" — the
|
||||
# codebase already expects Windows authors to write this file.
|
||||
#
|
||||
# A file that is not valid UTF-8 is a user-authoring mistake, so
|
||||
# surface it as ``ValidationError`` with a pointer to the offending
|
||||
# byte — the same pattern ``ExtensionManifest._load_yaml`` uses
|
||||
# for ``extension.yml`` (see ``UnicodeDecodeError`` handler in
|
||||
# this module). Without the wrap, the raw ``UnicodeDecodeError``
|
||||
# would abort installation with a Python traceback instead of a
|
||||
# clear message naming the file.
|
||||
try:
|
||||
raw = ignore_file.read_text(encoding="utf-8")
|
||||
except UnicodeDecodeError as e:
|
||||
raise ValidationError(
|
||||
f".extensionignore is not valid UTF-8: {ignore_file} "
|
||||
f"({e.reason} at byte {e.start})"
|
||||
)
|
||||
lines: List[str] = raw.splitlines()
|
||||
|
||||
# Normalise backslashes in patterns so Windows-authored files work
|
||||
normalised: List[str] = []
|
||||
|
||||
Reference in New Issue
Block a user