fix(integrations): escape control characters in goose recipe YAML renderer (#3384)

* fix(integrations): escape control characters in goose recipe YAML renderer

YAML forbids C0 control characters (except tab and newline) and DEL in
every scalar form, and a bare CR acts as a line break inside a block
scalar. _render_yaml wrote the body verbatim into a |2 literal block
scalar, so such bodies produced recipes the YAML parser rejects. Detect
block-scalar-unsafe characters and fall back to an escaped double-quoted
scalar via yaml.safe_dump, mirroring the TOML renderer's fallback
strategy from #3341.

Fixes #3382

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(integrations): use sys.maxsize instead of float inf for yaml width

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(integrations): extend block-scalar guard to C1 controls and Unicode line breaks

YAML's printable set excludes C1 controls (U+0080-U+009F except NEL), and
YAML 1.1 treats NEL/LS/PS as line breaks inside a literal block scalar, so
bodies carrying any of these still produced unparseable recipes. Widen the
fallback guard to the full class and cover it in the regression loop.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* fix(integrations): also treat surrogates and U+FFFE/U+FFFF as block-scalar unsafe

YAML's printable set also excludes lone UTF-16 surrogates and the
non-characters U+FFFE/U+FFFF; bodies carrying them still hit the literal
block path and produced unparseable recipes. Extend the guard and the
regression loop.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* docs(integrations): clarify YAML prompt serialization

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This commit is contained in:
Marsel Safin
2026-07-14 14:15:04 +02:00
committed by GitHub
parent d956ab722b
commit 6ab0c1dac1
2 changed files with 61 additions and 3 deletions

View File

@@ -1122,6 +1122,17 @@ class TomlIntegration(IntegrationBase):
# YamlIntegration — YAML-format agents (Goose)
# ---------------------------------------------------------------------------
# Characters a YAML literal block scalar cannot carry: C0 controls other
# than tab/LF (a bare CR acts as a line break inside the scalar), DEL, the
# C1 range, lone UTF-16 surrogates, and the non-characters U+FFFE/U+FFFF.
# NEL (U+0085) is YAML-printable but, like LS/PS (U+2028/U+2029), YAML 1.1
# treats it as a line break, which corrupts the block scalar's structure
# just the same, so all three are included.
_YAML_BLOCK_SCALAR_UNSAFE = re.compile(
r"[\x00-\x08\x0b-\x1f\x7f-\x9f\u2028\u2029\ud800-\udfff\ufffe\uffff]"
)
class YamlIntegration(IntegrationBase):
"""Concrete base for integrations that use YAML recipe format.
@@ -1227,9 +1238,9 @@ class YamlIntegration(IntegrationBase):
def _render_yaml(cls, title: str, description: str, body: str, source_id: str) -> str:
"""Render a YAML recipe file from title, description, and body.
Produces a Goose-compatible recipe with a literal block scalar
for the prompt content. Uses ``yaml.safe_dump()`` for the
header fields to ensure proper escaping.
Produces a Goose-compatible recipe with a literal block scalar for
normal prompt content, or an escaped quoted scalar when control
characters require it. Uses ``yaml.safe_dump()`` for the header fields.
"""
header = cls._build_yaml_header(title, description)
@@ -1240,6 +1251,23 @@ class YamlIntegration(IntegrationBase):
default_flow_style=False,
).strip()
# YAML forbids C0 control characters (except tab and newline) and
# DEL in every scalar form, and a bare CR acts as a line break
# inside a block scalar. A literal block scalar emits such bytes
# verbatim, producing a recipe the YAML parser rejects, so fall
# back to an escaped double-quoted scalar for those bodies.
if _YAML_BLOCK_SCALAR_UNSAFE.search(body):
prompt_yaml = yaml.safe_dump(
{"prompt": body}, allow_unicode=True, default_style='"', width=sys.maxsize
).strip()
lines = [
header_yaml,
prompt_yaml,
"",
f"# Source: {source_id}",
]
return "\n".join(lines) + "\n"
# Indent the body for YAML block scalar. Use an explicit indentation
# indicator ("|2") rather than a bare "|": YAML infers a plain block
# scalar's indentation from its first non-empty line, so a body whose

View File

@@ -201,6 +201,36 @@ class YamlIntegrationTests:
parsed = yaml.safe_load("\n".join(yaml_lines))
assert parsed["prompt"].rstrip("\n") == body
def test_yaml_prompt_with_control_characters_stays_valid(self):
"""A body containing control characters must still produce parseable YAML.
YAML forbids C0 control characters (except tab and newline), DEL,
C1 controls, lone surrogates and U+FFFE/U+FFFF in every scalar form,
and YAML 1.1 treats NEL (U+0085), LS (U+2028) and PS (U+2029) as
line breaks that corrupt a literal block scalar's structure. The
renderer falls back to an escaped double-quoted scalar for such
bodies."""
for ch in (
"\x08", "\x0c", "\x1b", "\x7f",
"\x80", "\x84", "\x85", "\x86", "\x9f",
"\u2028", "\u2029",
"\ud800", "\udfff", "\ufffe", "\uffff",
):
body = f"before{ch}after\nsecond line"
rendered = YamlIntegration._render_yaml("Title", "Desc", body, "src")
parsed = yaml.safe_load(rendered)
assert parsed["prompt"].rstrip("\n") == body, f"char {ch!r} round-trip"
def test_yaml_prompt_with_bare_carriage_return_stays_valid(self):
"""A bare CR (not part of CRLF) must not break the generated YAML.
Inside a block scalar a lone \r acts as a line break, corrupting
the document structure."""
body = "line1\rstill line1\nline2"
rendered = YamlIntegration._render_yaml("Title", "Desc", body, "src")
parsed = yaml.safe_load(rendered)
assert parsed["prompt"].rstrip("\n") == body
def test_plan_command_has_no_context_placeholder(self, tmp_path):
"""The generated plan command must not carry a context-file placeholder.