fix(extensions): use explicit UTF-8 encoding when reading manifest YAML (#2370)

* fix(extensions): use explicit UTF-8 encoding when reading manifest YAML

On Windows, Python's open() defaults to the system locale encoding
(e.g., GBK on Chinese Windows), which causes UnicodeDecodeError when
extension.yml or preset.yml contains non-ASCII content such as Chinese
characters in description fields.

Add encoding='utf-8' to ExtensionManifest._load_yaml and
PresetManifest._load_yaml so manifests are read consistently across
platforms.

Fixes #2325

* test(extensions,presets): add UTF-8 manifest regression tests for #2325

Positive: extension.yml/preset.yml with non-ASCII (Chinese + emoji)
descriptions load correctly when written as UTF-8 bytes — fails on
Windows without explicit encoding='utf-8'.

Negative: files containing invalid UTF-8 bytes raise a clean error
(ValidationError or UnicodeDecodeError), not a silent crash.

* fix(extensions,presets): wrap I/O and decode errors as ValidationError

Address remaining Copilot concerns on #2370:

- Catch UnicodeDecodeError and OSError in both manifest loaders and
  re-raise as ValidationError / PresetValidationError so callers see a
  consistent error type, not a bare decode/IO traceback.
- Validate that PresetManifest YAML root is a mapping (extensions.py
  already had this; presets.py was missing it). Treat None as {} for
  empty-file compatibility.
- Tighten the negative regression tests to assert the specific message,
  and add a non-mapping-root test for PresetManifest matching the
  existing one for ExtensionManifest.
This commit is contained in:
Quratulain-bilal
2026-04-28 18:47:22 +05:00
committed by GitHub
parent 171b65ac33
commit 3a7f64c8a5
4 changed files with 83 additions and 3 deletions

View File

@@ -139,12 +139,18 @@ class ExtensionManifest:
def _load_yaml(self, path: Path) -> dict:
"""Load YAML file safely."""
try:
with open(path, 'r') as f:
with open(path, 'r', encoding='utf-8') as f:
data = yaml.safe_load(f)
except yaml.YAMLError as e:
raise ValidationError(f"Invalid YAML in {path}: {e}")
except FileNotFoundError:
raise ValidationError(f"Manifest not found: {path}")
except UnicodeDecodeError as e:
raise ValidationError(
f"Manifest is not valid UTF-8: {path} ({e.reason} at byte {e.start})"
)
except OSError as e:
raise ValidationError(f"Could not read manifest {path}: {e}")
if not isinstance(data, dict):
raise ValidationError(
f"Manifest must be a YAML mapping, got {type(data).__name__}: {path}"

View File

@@ -136,12 +136,25 @@ class PresetManifest:
def _load_yaml(self, path: Path) -> dict:
"""Load YAML file safely."""
try:
with open(path, 'r') as f:
return yaml.safe_load(f) or {}
with open(path, 'r', encoding='utf-8') as f:
data = yaml.safe_load(f)
except yaml.YAMLError as e:
raise PresetValidationError(f"Invalid YAML in {path}: {e}")
except FileNotFoundError:
raise PresetValidationError(f"Manifest not found: {path}")
except UnicodeDecodeError as e:
raise PresetValidationError(
f"Manifest is not valid UTF-8: {path} ({e.reason} at byte {e.start})"
)
except OSError as e:
raise PresetValidationError(f"Could not read manifest {path}: {e}")
if data is None:
return {}
if not isinstance(data, dict):
raise PresetValidationError(
f"Manifest must be a YAML mapping, got {type(data).__name__}: {path}"
)
return data
def _validate(self):
"""Validate manifest structure and required fields."""