Files
nexu-io-open-design/apps/web/tests/artifacts/parser.test.ts
Dongsen 12ce5ad38b fix(web): ignore <artifact> tags inside markdown code spans and fences (#1132)
* test(web): add failing parser cases for <artifact> recitation in markdown code

Cover the three real-world prose contexts where the model legitimately
quotes the artifact tag without intending to emit one:

- inside an inline backtick span
- inside a fenced code block
- spread across streaming chunks crossing the fence boundary

Establishes the RED baseline before parser code-fence awareness lands.

* fix(web): ignore <artifact> tags inside markdown code spans and fences

The streaming artifact parser scanned the buffer with a raw indexOf,
guarded only by 'next char must be whitespace'. That meant any literal
<artifact ...> the model recited while documenting the protocol — even
inside backticks or a ```html fence — flipped the parser into artifact
mode, swallowed the rest of the reply from the chat UI, and (when a
matching </artifact> appeared in the recitation) silently wrote a
spurious file to disk via persistArtifact.

Replace findOpenTag with a linear scan that tracks fenced code blocks
(```) and inline code spans (`), skipping any <artifact prefix found
inside either. If the buffer ends mid-fence, return a partial match
anchored at the fence start so the next streaming chunk can resolve
the boundary without losing fence context.

Closes #1130.

* fix(web): match renderer fence/inline-code rules in artifact parser

Codex review on PR #1132 caught that the previous fix toggled inFence on
any triple-backtick run anywhere in the buffer, including mid-line, while
the chat renderer (apps/web/src/runtime/markdown.tsx) only treats ``` as
a fence when it occupies a whole line matching /^[ ]{0,3}```(\w[\w+-]*)?\s*$/.
That asymmetry would suppress a real <artifact> tag emitted after a prose
sentence like "the opening marker is ```html and the response then writes:".

Rework findOpenTag in three passes that mirror the renderer:

  1. Walk \n-terminated lines; only a line that matches FENCE_LINE_RE
     toggles fence state. Open fences without a close (or with an
     unterminated tail line) return partial so the next chunk can resolve.
  2. Collect inline code spans with /`[^`]+`/g — the same regex used by
     renderInline — so what the parser skips matches what the user sees as
     code. Unmatched trailing backticks after the last \n hold back.
  3. Find the first <artifact …> outside any skip range; preserve the
     existing partial-prefix tail handling.

Adds a regression test covering the exact case Codex reported.

* test(web): pin parser behavior on double-backtick and in-fence string literal recitation

Two cases raised in PR #1132 review:

- a real artifact tag wrapped in '``<artifact …>``' (double-backtick
  inline code span) should not be treated as a real artifact
- a fenced JS example whose body contains a string literal like
  'const fence = "```";' should not pop fence state early and let a
  later literal <artifact> be parsed as real

Both already pass on 96e88ca because the line-anchored fence regex and
the renderer-aligned inline regex handle them correctly. Pinning the
behavior so future regressions surface as test failures.

* fix(web): make stripArtifact markdown-aware to stop truncating literal recitations

The streaming artifact parser was hardened in 96e88ca to skip <artifact>
recitations inside backticks and fences, but the post-stream stripper at
AssistantMessage.tsx still ran a naive 'content.indexOf("<artifact")' over
the same text events. As reported by lefarcen on PR #1132, that meant
chat replies with literal protocol recitations could still get silently
truncated mid-explanation — even though the parser preserved them in the
text stream and the file panel was no longer polluted with ghost files.

Extract the renderer-aligned classification (FENCE_LINE_RE, INLINE_CODE_RE,
computeSkipRanges, rangeContains) into a single source of truth at
apps/web/src/artifacts/markdown-context.ts so the parser and the stripper
agree on what counts as code. Add apps/web/src/artifacts/strip.ts with a
markdown-aware stripArtifact that:

- ignores any <artifact open inside a fenced block or inline code span
- looks for </artifact> with the same skip-range filter, so a real open
  paired with a literal close inside backticks does not strip a literal
  body that is meant to render
- returns content unchanged when an open exists with no matching real
  close (the previous implementation sliced to end-of-string, which would
  nuke trailing prose on a malformed or still-streaming tag)

Refactor parser.ts to import the shared helpers; behavior preserved (all
seven existing parser tests still pass). New strip.test.ts covers six
cases including the empirically-verified inline-backtick regression.

* fix(web): align artifact stripper/parser fence rules with renderer exactly

Two gaps surfaced in review at a0bf05f:

- markdown-context.ts used a single FENCE_LINE_RE that allowed 0-3 leading
  spaces and reused the same pattern for opening and closing fences. The
  chat renderer (runtime/markdown.tsx:44 and :49) is asymmetric — opens
  with /^```(\w[\w+-]*)?\s*$/, closes with /^```\s*$/, and rejects any
  leading indentation on either side. Indented "   ```html" was being
  treated as a code fence even though the renderer keeps it as a paragraph,
  and a literal "```html" line inside an open fenced example was closing
  the skip range early — both could expose a real or literal <artifact …>
  to the wrong handler.
- stripArtifact discarded computeSkipRanges' unclosedFenceStart, so a
  fenced literal that ends at EOF without a trailing newline (very common
  for chat output) leaked the inner <artifact …> recitation to the
  stripper, reproducing the original #1130 truncation symptom on a
  narrower input shape.

Split FENCE_LINE_RE into FENCE_OPEN_RE / FENCE_CLOSE_RE with no leading
indentation, gate the fence state machine on the right side of the toggle,
and have stripArtifact extend skip ranges to end-of-content when a fence
is left open. Also tightened the parser's tail-line hold-back regex to
match the renderer's no-leading-space rule. Added regression tests for the
EOF-unclosed-fence case, the indented pseudo-fence (renderer treats as
paragraph, stripper must strip the real artifact), and a "```html" line
inside an open fence.

Refs nexu-io/open-design#1130

* refactor(web): align streaming tail-line fence guard with FENCE_OPEN_RE

The streaming parser's tail-line hold-back used a stricter local regex
(/^```\w*$/) than the renderer's FENCE_OPEN_RE (/^```(\w[\w+-]*)?\s*$/),
missing valid opener tails like ```c++, ```ts-, or ``` (trailing space).

In practice these tails are still held back by the unmatched-backtick
parity scan that runs immediately after — three backticks in a tail line
are odd, so firstUnmatched stays set and the parser holds from that
position. So this wasn't a runtime correctness bug, just a regex
divergence that future readers could trip on.

Drop the local regex and reuse FENCE_OPEN_RE so the tail check matches
the same shape the rest of the pipeline already uses. Pinned the
behavior with three new parser tests (`+`/`-` info-string suffix and
trailing-space tails arriving as the first chunk) — they pass at HEAD,
proving the parity scan was already covering these cases.

Refs nexu-io/open-design#1132 (lefarcen polish P2)

* fix(web): scope inline-code skip ranges per block and reject <artifact prefix-shared opens

INLINE_CODE_RE previously ran over the whole buffer, so an unmatched
backtick in one paragraph could pair with a backtick in a later
paragraph and create a phantom inline span that swallowed any real
<artifact …> between them. Mirror runtime/markdown.tsx by splitting the
buffer on fence / blank / heading / list / hr boundaries and running
INLINE_CODE_RE per block region instead.

stripArtifact accepted any unskipped `<artifact` substring as a real
open, while the streaming parser already required a following whitespace
character — so prose like `<artifactual>demo</artifact>` was being
truncated to `prefix  suffix`. Extract the parser's real-open guard into
isRealArtifactOpenAt and reuse it from both sides.

While reordering findOpenTag for the shared guard, also fix the related
hold-back ordering issue tracked at #1141: a stray tail-line backtick or
fence-opener prefix used to suppress an artifact already complete
earlier in the buffer. Scan for the earliest complete real open first,
then pick the earliest hold-back position only when no complete tag was
found.

Regressions pinned in parser.test.ts and strip.test.ts for both new
finding shapes.

* fix(web): keep HR-shaped lines inside paragraph regions for inline-code scanning

The previous walker closed inline-scan regions on lines matching the HR
regex, but `parseBlocks()` in runtime/markdown.tsx does not break a
paragraph on HR — its inner accumulation loop only breaks on blank /
fence / heading / ul / ol (runtime/markdown.tsx:95-104). HR is only an
HR block in the outer loop's first-look, never mid-paragraph.

So inputs like `intro \`\n---\n<artifact …>…</artifact>\n---\nclosing \``
are one paragraph in the renderer, whose two stray backticks pair to
cover the literal artifact recitation — but the walker was splitting on
the `---` lines, leaving the recitation outside skip ranges, and the
parser/stripper would treat it as a real tag.

Drop HR from the paragraph-break list (HR-shaped lines carry no
backticks of their own, so keeping them inside the surrounding region
is benign either way) and document the renderer-mirror rationale.

Regressions pinned on both sides.
2026-05-11 19:29:22 +08:00

195 lines
8.3 KiB
TypeScript

import { describe, expect, it } from 'vitest';
import { type ArtifactEvent, createArtifactParser } from '../../src/artifacts/parser';
function collect(input: string): ArtifactEvent[] {
const parser = createArtifactParser();
const events: ArtifactEvent[] = [];
for (const e of parser.feed(input)) events.push(e);
for (const e of parser.flush()) events.push(e);
return events;
}
describe('createArtifactParser', () => {
it('parses a real artifact tag in prose', () => {
const events = collect(
'Here is a page:\n<artifact identifier="hello" type="text/html" title="Hi">\n<h1>Hi</h1>\n</artifact>\nDone.',
);
const start = events.find((e) => e.type === 'artifact:start');
const end = events.find((e) => e.type === 'artifact:end');
expect(start).toMatchObject({ identifier: 'hello', artifactType: 'text/html', title: 'Hi' });
expect(end).toMatchObject({ identifier: 'hello' });
const trailing = events
.filter((e): e is Extract<ArtifactEvent, { type: 'text' }> => e.type === 'text')
.map((e) => e.delta)
.join('');
expect(trailing).toContain('Done.');
});
it('does not enter artifact mode for a tag inside inline backticks', () => {
const events = collect(
'To emit an artifact, wrap output in `<artifact identifier="x" type="text/html" title="X">` and continue writing normal prose afterwards.',
);
expect(events.find((e) => e.type === 'artifact:start')).toBeUndefined();
const text = events
.filter((e): e is Extract<ArtifactEvent, { type: 'text' }> => e.type === 'text')
.map((e) => e.delta)
.join('');
expect(text).toContain('continue writing normal prose afterwards.');
});
it('does not enter artifact mode for a tag inside a fenced code block', () => {
const events = collect(
[
'Example:',
'```html',
'<artifact identifier="demo" type="text/html" title="Demo">',
'<h1>Demo</h1>',
'</artifact>',
'```',
'After the fence, more prose.',
].join('\n'),
);
expect(events.find((e) => e.type === 'artifact:start')).toBeUndefined();
const text = events
.filter((e): e is Extract<ArtifactEvent, { type: 'text' }> => e.type === 'text')
.map((e) => e.delta)
.join('');
expect(text).toContain('After the fence, more prose.');
});
it('still parses a real artifact tag when prose contains an inline triple-backtick that is not a fence', () => {
// The chat markdown renderer (apps/web/src/runtime/markdown.tsx) only treats
// ``` as a fence when it appears alone on a line. A mid-line ```html that
// is not a fence per the renderer must not suppress a real artifact that
// follows. (Reported by Codex review on PR #1132.)
const events = collect(
'The opening marker is ```html and the response then writes:\n<artifact identifier="real" type="text/html" title="Real">\n<h1>real</h1>\n</artifact>',
);
expect(events.find((e) => e.type === 'artifact:start')).toMatchObject({
identifier: 'real',
artifactType: 'text/html',
title: 'Real',
});
});
it('does not enter artifact mode for a tag wrapped in double backticks', () => {
// lefarcen P2: double-backtick code spans (``…``) are valid Markdown.
const events = collect(
'You can quote it as ``<artifact identifier="x" type="text/html" title="X">`` in prose.',
);
expect(events.find((e) => e.type === 'artifact:start')).toBeUndefined();
});
it('does not enter artifact mode on a triple-backtick string literal inside a fenced block', () => {
// lefarcen P2: fenced JS example whose body contains a string with literal ``` should
// not pop fence state early and expose a later <artifact> as real.
const events = collect(
[
'```js',
'const fence = "```";',
'const tag = "<artifact identifier=\\"x\\" type=\\"text/html\\" title=\\"X\\">";',
'```',
'After.',
].join('\n'),
);
expect(events.find((e) => e.type === 'artifact:start')).toBeUndefined();
});
it('holds back when a chunk ends mid-line on a renderer-valid fence opener prefix', () => {
// lefarcen polish P2: the streaming tail-line guard must mirror the
// renderer's FENCE_OPEN_RE shape, not a stricter \w-only subset.
// Opener tails like "```c++" (info string with `+`/`-`) or "``` "
// (trailing whitespace) are valid renderer openers waiting for a `\n`.
// If the parser flushes them as text and then sees a literal `<artifact>`
// on the next chunk, that artifact would incorrectly enter artifact mode.
const cases: Array<{ name: string; chunks: [string, string] }> = [
{
name: 'plus suffix',
chunks: ['Header.\n```c++', '\n<artifact identifier="x" type="text/plain" title="X">demo</artifact>\n```\n'],
},
{
name: 'dash suffix',
chunks: ['Header.\n```ts-', '\n<artifact identifier="x" type="text/plain" title="X">demo</artifact>\n```\n'],
},
{
name: 'trailing space',
chunks: ['Header.\n``` ', '\n<artifact identifier="x" type="text/plain" title="X">demo</artifact>\n```\n'],
},
];
for (const { name, chunks } of cases) {
const parser = createArtifactParser();
const events: ArtifactEvent[] = [];
for (const c of chunks) for (const e of parser.feed(c)) events.push(e);
for (const e of parser.flush()) events.push(e);
expect(events.find((e) => e.type === 'artifact:start'), name).toBeUndefined();
}
});
it('parses a real artifact between paragraphs that each carry a stray backtick', () => {
// Inline code is paragraph-local in the renderer; an unbalanced backtick
// in one paragraph must not bridge across a blank line to pair with a
// backtick in a later paragraph and swallow a real <artifact …> in
// between (mrcfps's 2026-05-11 repro).
const events = collect(
[
'intro `',
'',
'<artifact identifier="x" type="text/plain" title="X">demo</artifact>',
'',
'closing `',
].join('\n'),
);
expect(events.find((e) => e.type === 'artifact:start')).toBeDefined();
expect(events.find((e) => e.type === 'artifact:end')).toBeDefined();
});
it('does not enter artifact mode when bridged by stray backticks across HR-shaped lines', () => {
// Renderer's paragraph loop does not break on HR (runtime/markdown.tsx:95-104),
// so `intro \`\n---\n<artifact …>…</artifact>\n---\nclosing \`` is one paragraph
// and the backticks pair to cover the literal recitation. mrcfps's
// 2026-05-11 05:46 follow-up — the skip-range walker must not split on HR.
const events = collect(
[
'intro `',
'---',
'<artifact identifier="x" type="text/plain" title="X">demo</artifact>',
'---',
'closing `',
].join('\n'),
);
expect(events.find((e) => e.type === 'artifact:start')).toBeUndefined();
});
it('does not enter artifact mode for <artifactual> or other prefix-shared identifiers', () => {
// `<artifact` must be followed by whitespace to count as a real open;
// strings like `<artifactual>` are not protocol tags and must survive as
// literal text on both the parser and the stripper sides.
const events = collect('prefix <artifactual>demo</artifact> suffix');
expect(events.find((e) => e.type === 'artifact:start')).toBeUndefined();
const text = events
.filter((e): e is Extract<ArtifactEvent, { type: 'text' }> => e.type === 'text')
.map((e) => e.delta)
.join('');
expect(text).toBe('prefix <artifactual>demo</artifact> suffix');
});
it('does not enter artifact mode when a fenced tag arrives across multiple chunks', () => {
const parser = createArtifactParser();
const chunks = [
'Example:\n```html\n<artifact identifier="demo"',
' type="text/html" title="Demo">\n<h1>Demo</h1>\n</artif',
'act>\n```\nAfter the fence, more prose.',
];
const events: ArtifactEvent[] = [];
for (const c of chunks) for (const e of parser.feed(c)) events.push(e);
for (const e of parser.flush()) events.push(e);
expect(events.find((e) => e.type === 'artifact:start')).toBeUndefined();
const text = events
.filter((e): e is Extract<ArtifactEvent, { type: 'text' }> => e.type === 'text')
.map((e) => e.delta)
.join('');
expect(text).toContain('After the fence, more prose.');
});
});