fix: use bounded read for integration catalog HTTP responses (#3763)

* fix(skills): match closing frontmatter delimiter on its own line

SkillsIntegration.setup parsed each command template's frontmatter with
raw.split("---", 2). A bare substring split stops at the first `---`
*anywhere*, so a template whose description embeds `---` (e.g.
"Separate sections with --- markers") truncated the parsed frontmatter:
later keys were dropped, the description fell back to the generic default,
and the leftover frontmatter spilled into the skill body.

Scan for the closing `---` on its own line instead, for both the
description parse and the body strip. The frontmatter block is parsed
unstripped so trailing newlines in literal (|) block scalars still survive,
and the body slice keeps the newline after the marker so output stays
byte-for-byte identical to the old split for well-formed templates.

Adds regression tests covering the dashed-description truncation and the
frontmatter-spilled-into-body cases.

* fix: use bounded read for integration catalog HTTP responses

The integration catalog fetch used unbounded resp.read() to read
HTTP responses into memory. A malicious or misconfigured catalog
server could return an arbitrarily large response causing OOM.

Replace with read_response_limited() capped at MAX_JSON_METADATA_BYTES
(1 MiB), consistent with how other JSON fetch paths in the codebase
(_version.py, _github_http.py, authentication/azure_devops.py) already
enforce bounded reads.

Pass error_type=IntegrationCatalogError so oversized catalogs are
caught by the existing per-entry recovery path in
_get_merged_integrations() rather than aborting the entire merge.

Add regression test verifying oversized responses are rejected as
IntegrationCatalogError and that healthy catalogs remain usable.
This commit is contained in:
Quratulain-bilal
2026-07-29 00:32:31 +05:00
committed by GitHub
parent 88e997306f
commit 89e204ca3c
4 changed files with 245 additions and 13 deletions

View File

@@ -1652,13 +1652,27 @@ class SkillsIntegration(IntegrationBase):
command_name = src_file.stem # e.g. "plan"
skill_name = f"speckit-{command_name.replace('.', '-')}"
# Parse frontmatter for description
# Parse frontmatter for description. Locate the closing ``---`` on
# its own line rather than with ``raw.split("---", 2)`` — a bare
# substring split stops at the first ``---`` *anywhere*, including
# one inside a value such as ``description: Separate sections
# with ---``, which truncates the frontmatter and drops later keys.
# The block between the delimiters is parsed unstripped so trailing
# newlines in literal (``|``) block scalars survive.
frontmatter: dict[str, Any] = {}
if raw.startswith("---"):
parts = raw.split("---", 2)
if len(parts) >= 3:
fm_lines = raw.splitlines(keepends=True)
fm_close = next(
(
i
for i in range(1, len(fm_lines))
if fm_lines[i].rstrip() == "---"
),
None,
)
if fm_close is not None:
try:
fm = yaml.safe_load(parts[1])
fm = yaml.safe_load("".join(fm_lines[1:fm_close]))
if isinstance(fm, dict):
frontmatter = fm
except yaml.YAMLError:
@@ -1673,11 +1687,27 @@ class SkillsIntegration(IntegrationBase):
# Strip the processed frontmatter — we rebuild it for skills.
# Preserve leading whitespace in the body to match release ZIP
# output byte-for-byte (the template body starts with \n after
# the closing ---).
# the closing ---). Scan for the closing ``---`` on its own line
# rather than ``split("---", 2)`` so a ``---`` embedded in a value
# does not truncate the frontmatter and spill it into the body.
if processed_body.startswith("---"):
parts = processed_body.split("---", 2)
if len(parts) >= 3:
processed_body = parts[2]
body_lines = processed_body.splitlines(keepends=True)
close_idx = next(
(
i
for i in range(1, len(body_lines))
if body_lines[i].rstrip() == "---"
),
None,
)
if close_idx is not None:
# Keep whatever trails the ``---`` marker on the closing
# line (normally just the newline) so the body stays
# byte-for-byte identical to ``split("---", 2)[2]``. The
# line-anchored check guarantees ``---`` sits at index 0.
processed_body = body_lines[close_idx][3:] + "".join(
body_lines[close_idx + 1 :]
)
# Select description — use the original template description
# to stay byte-for-byte identical with release ZIP output.