mirror of
https://github.com/CherryHQ/cherry-studio.git
synced 2026-07-07 15:12:18 +08:00
hotfix(image-generation): fix gpt-image-2 / gpt-image-1.5 failures and stuck pending placeholder (#14578)
<!-- Template from https://github.com/kubevirt/kubevirt/blob/main/.github/PULL_REQUEST_TEMPLATE.md?--> <!-- Thanks for sending a pull request! Here are some tips for you: 1. Consider creating this PR as draft: https://github.com/CherryHQ/cherry-studio/blob/main/CONTRIBUTING.md --> <!-- 🚨 Branch Strategy Change (Effective April 3, 2026) 🚨 The `main` branch is now under CODE FREEZE. - main branch: Only accepts critical bug fixes via `hotfix/*` branches. Fix PRs must be minimal in scope and must not include any refactoring code. - v2 branch: All new features, refactoring, and optimizations should be submitted to the `v2` branch. If you are submitting a bug fix to main, please ensure your PR is from a `hotfix/*` branch. --> ### What this PR does Before this PR: - `gpt-image-2` (and any image-edit request routed through AiHubMix / NewAPI / CherryIN) failed with `common.UnmarshalBodyReusable failed: bind request body failed: invalid character '-' in numeric literal`. The providers hard-coded `Content-Type: application/json` in `authHeaders()`, but `OpenAICompatibleImageModel.doGenerate` uses `postFormDataToApi` for `/images/edits` and relies on `fetch` to auto-set `multipart/form-data; boundary=...`. Forcing JSON made the server receive a multipart body under a JSON content-type and choke on the leading `--boundary`. - Even when the request reached the server, image uploads failed with `unsupported mimetype ('image')` because `collectImagesFromMessages` built data URIs using `block.file.type` — which is the `FileType` enum (`"image"`), not a MIME type. - Some saved/pasted images produced `ENOENT: ... <uuid>jpg` because `saveBase64Image` / `savePastedImage` return `ext` without the leading dot, and the caller was concatenating `file.id + file.ext`. - After a successful image generation, a `BeatLoader` placeholder kept spinning next to the finished image. `fetchImageGeneration` only emitted `LLM_RESPONSE_COMPLETE`; the trailing loader in `Blocks/index.tsx` is driven by `isMessageProcessing(message)`, which is only cleared when `onComplete` runs on `BLOCK_COMPLETE`. - If a `file` chunk arrived without a preceding `IMAGE_CREATED`, the initial UNKNOWN placeholder block was never claimed, leaving an orphan spinner below the image. - `gpt-image-2` / `gpt-image-1.5` / `gpt-image-1*` / `chatgpt-image-*` generations through any OpenAI-compatible provider still failed with `400 Unknown parameter: 'response_format'` (see #14485, #14540, #14579). The earlier `@ai-sdk/openai@3.0.53` patch (#14488) only covered `OpenAIImageModel` (direct OpenAI + Azure via `@ai-sdk/azure`); `OpenAICompatibleImageModel.doGenerate` unconditionally added `response_format: "b64_json"` to the `/images/generations` body, so every provider whose image model is wired to `OpenAICompatibleImageModel` kept hitting the 400 — including `type: 'openai-compatible'` and the AiHubMix / NewAPI / CherryIN gateway image models in this repo. After this PR: - `aihubmix-provider`, `newapi-provider`, and `cherryin-provider` no longer hard-code `Content-Type` in auth headers. `postJsonToApi` still defaults JSON for other endpoints, and `postFormDataToApi` is free to let `fetch` set `multipart/form-data; boundary=...` for `/images/edits`. - `collectImagesFromMessages` reads via `window.api.file.base64Image(block.file.name)`, which returns a correct `data:image/<ext>;base64,...` URI (with `jpg → jpeg` normalization) and uses the consistent on-disk filename regardless of whether the stored `ext` has a leading dot. - `fetchImageGeneration` now emits `ChunkType.BLOCK_COMPLETE` before `LLM_RESPONSE_COMPLETE`, so `onComplete` runs and the assistant message transitions out of `PROCESSING`. - `onImageGenerated` reuses the initial UNKNOWN placeholder when one exists, so a `file` chunk without a prior `IMAGE_CREATED` no longer leaves an orphan spinner. - `gpt-image-2` is registered in `IMAGE_ENHANCEMENT_MODELS`. - `patches/@ai-sdk__openai-compatible@2.0.37.patch` is extended with the same `hasDefaultResponseFormat` guard that `@ai-sdk/openai` already uses. `OpenAICompatibleImageModel.doGenerate` now spreads `response_format: "b64_json"` only when the model ID does **not** start with `chatgpt-image-`, `gpt-image-1-mini`, `gpt-image-1.5`, `gpt-image-1`, or `gpt-image-2`. This fixes #14485, #14540, #14579 for `openai-compatible` typed providers and the AiHubMix / NewAPI / CherryIN gateways. Fixes #14485 Fixes #14540 Fixes #14579 ### Why we need it and why it was done in this way The following tradeoffs were made: - The `ext`-with-or-without-dot inconsistency exists in historical DB records, so patching only `FileStorage.ts` would not help already-saved rows. The fix uses `file.name` at the consumer side — always the real on-disk filename across every save path — without touching the storage layer. - The provider `authHeaders()` change could theoretically affect non-JSON, non-multipart requests, but all SDK models in use go through `postJsonToApi`, which already defaults `Content-Type: application/json` internally, so behavior is preserved. - Extending the `@ai-sdk/openai-compatible` patch rather than overriding `response_format` per-provider keeps the fix in one place and automatically covers every caller of `OpenAICompatibleImageModel` (current and future). The guard list mirrors the upstream `@ai-sdk/openai` list, so the two stay in sync. The following alternatives were considered: - Normalizing `ext` in `FileStorage.saveBase64Image` / `savePastedImage`: rejected because it would not repair existing records and could regress other callers that have compensated for the dotless form. - Overriding `Content-Type` only inside the image model factory: rejected as more surface area than needed; the global `authHeaders()` already defers to `postJsonToApi` defaults. - Stripping `response_format` in each custom provider (AiHubMix / NewAPI / CherryIN) instead of patching the SDK: rejected — it would leave `type: 'openai-compatible'` users still broken and duplicate the model-prefix list in three places. Links to places where the discussion took place: #14485, #14540, #14579, #14488 ### Breaking changes <!-- optional --> None. ### Special notes for your reviewer <!-- optional --> Scope is intentionally minimal for the main branch freeze: no refactoring, no new abstractions. Surface changes: - `authHeaders()` in three custom providers (drops one hard-coded header). - `collectImagesFromMessages` (one helper, swaps input building). - `fetchImageGeneration` (one extra chunk emission). - `onImageGenerated` (placeholder-claim fallback). - `IMAGE_ENHANCEMENT_MODELS` (one new entry). - `patches/@ai-sdk__openai-compatible@2.0.37.patch` (adds `hasDefaultResponseFormat` helper + conditional spread of `response_format`; mirrors `@ai-sdk/openai@3.0.53` patch). Drop the openai-compatible patch addition once `@ai-sdk/openai-compatible` ships the equivalent check upstream. ### Checklist This checklist is not enforcing, but it's a reminder of items that could be relevant to every PR. Approvers are expected to review this list. - [x] PR: The PR description is expressive enough and will help future contributors - [x] Code: [Write code that humans can understand](https://en.wikiquote.org/wiki/Martin_Fowler#code-for-humans) and [Keep it simple](https://en.wikipedia.org/wiki/KISS_principle) - [x] Refactor: You have [left the code cleaner than you found it (Boy Scout Rule)](https://learning.oreilly.com/library/view/97-things-every/9780596809515/ch08.html) - [x] Upgrade: Impact of this change on upgrade flows was considered and addressed if required - [ ] Documentation: A [user-guide update](https://docs.cherry-ai.com) was considered and is present (link) or not required. Check this only when the PR introduces or changes a user-facing feature or behavior. - [x] Self-review: I have reviewed my own code (e.g., via [`/gh-pr-review`](/.claude/skills/gh-pr-review/SKILL.md), `gh pr diff`, or GitHub UI) before requesting review from others ### Release note <!-- Write your release note: 1. Enter your extended release note in the below block. If the PR requires additional action from users switching to the new release, include the string "action required". 2. If no release note is required, just write "NONE". 3. Only include user-facing changes (new features, bug fixes visible to users, UI changes, behavior changes). For CI, maintenance, internal refactoring, build tooling, or other non-user-facing work, write "NONE". --> ```release-note Fix image generation and editing for gpt-image-2 / gpt-image-1.5 / gpt-image-1* / chatgpt-image-* across AiHubMix, NewAPI, CherryIN, Azure OpenAI, and other OpenAI-compatible providers: requests no longer fail with `Unknown parameter: 'response_format'`, multipart content-type, or mimetype errors; generated/pasted images are resolved correctly; and the loading placeholder disappears once the image is rendered. ``` --------- Signed-off-by: suyao <sy20010504@gmail.com>
This commit is contained in:
7
.changeset/cherryin-image-edit-content-type.md
Normal file
7
.changeset/cherryin-image-edit-content-type.md
Normal file
@@ -0,0 +1,7 @@
|
||||
---
|
||||
'@cherrystudio/ai-sdk-provider': patch
|
||||
---
|
||||
|
||||
Fix CherryIN image-edit requests failing with `invalid character '-' in numeric literal`.
|
||||
|
||||
The JSON headers getter hard-coded `Content-Type: application/json`, which leaked into `OpenAICompatibleImageModel`'s `/images/edits` call. That path uses `postFormDataToApi` and relies on `fetch` to auto-set `multipart/form-data; boundary=...`; forcing JSON made the server try to parse the multipart body as JSON and choke on the leading `--boundary`. Removed the explicit `Content-Type` — `postJsonToApi` still defaults it for JSON endpoints.
|
||||
8
.changeset/openai-compatible-image-response-format.md
Normal file
8
.changeset/openai-compatible-image-response-format.md
Normal file
@@ -0,0 +1,8 @@
|
||||
---
|
||||
'@cherrystudio/ai-core': patch
|
||||
'@cherrystudio/ai-sdk-provider': patch
|
||||
---
|
||||
|
||||
Fix `Unknown parameter: 'response_format'` when generating images with `gpt-image-2`, `gpt-image-1.5`, `gpt-image-1`, `gpt-image-1-mini`, or `chatgpt-image-*` through any provider that routes the image model via `OpenAICompatibleImageModel` (e.g. `openai-compatible` typed providers and the AiHubMix / NewAPI / CherryIN gateways in this repo).
|
||||
|
||||
`OpenAICompatibleImageModel.doGenerate` unconditionally added `response_format: "b64_json"` to `/images/generations` bodies. The previous `@ai-sdk/openai@3.0.53` patch only covered `OpenAIImageModel` (direct OpenAI + Azure via `@ai-sdk/azure`), so users on the compatible route (AiHubMix, NewAPI, CherryIN, generic openai-compatible) kept hitting this 400 on newer gpt-image models. Extended `patches/@ai-sdk__openai-compatible@2.0.37.patch` with the same `hasDefaultResponseFormat` guard used upstream by `@ai-sdk/openai`. Drop the addition once `@ai-sdk/openai-compatible` ships the equivalent check.
|
||||
@@ -137,7 +137,6 @@ const toBearerToken = (authorization?: string) => (authorization ? authorization
|
||||
const createJsonHeadersGetter = (options: CherryInProviderSettings): (() => Record<string, HeaderValue>) => {
|
||||
return () => ({
|
||||
Authorization: `Bearer ${resolveApiKey(options)}`,
|
||||
'Content-Type': 'application/json',
|
||||
...resolveConfiguredHeaders(options.headers)
|
||||
})
|
||||
}
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
diff --git a/dist/index.js b/dist/index.js
|
||||
index d73573b013e61903991cad2074187a3ade8921ab..919cbb2d14c0019716080c1bf041dde96c2b6cab 100644
|
||||
--- a/dist/index.js
|
||||
+++ b/dist/index.js
|
||||
@@ -572,6 +572,17 @@ var OpenAICompatibleChatLanguageModel = class {
|
||||
@@ -571,6 +571,17 @@
|
||||
type: "reasoning",
|
||||
text: reasoning
|
||||
});
|
||||
}
|
||||
+ }
|
||||
+ if (choice.message.images) {
|
||||
+ for (const image of choice.message.images) {
|
||||
+ const match1 = image.image_url.url.match(/^data:([^;]+)/);
|
||||
@@ -16,11 +16,10 @@ index d73573b013e61903991cad2074187a3ade8921ab..919cbb2d14c0019716080c1bf041dde9
|
||||
+ data: match2 ? match2[1] : image.image_url.url,
|
||||
+ });
|
||||
+ }
|
||||
+ }
|
||||
}
|
||||
if (choice.message.tool_calls != null) {
|
||||
for (const toolCall of choice.message.tool_calls) {
|
||||
const thoughtSignature = (_c = (_b = toolCall.extra_content) == null ? void 0 : _b.google) == null ? void 0 : _c.thought_signature;
|
||||
@@ -732,6 +743,17 @@ var OpenAICompatibleChatLanguageModel = class {
|
||||
@@ -732,6 +743,17 @@
|
||||
delta: delta.content
|
||||
});
|
||||
}
|
||||
@@ -38,10 +37,11 @@ index d73573b013e61903991cad2074187a3ade8921ab..919cbb2d14c0019716080c1bf041dde9
|
||||
if (delta.tool_calls != null) {
|
||||
if (isActiveReasoning) {
|
||||
controller.enqueue({
|
||||
@@ -928,6 +950,14 @@ var OpenAICompatibleChatResponseSchema = import_v43.z.looseObject({
|
||||
@@ -927,6 +949,14 @@
|
||||
thought_signature: import_v43.z.string().nullish()
|
||||
}).nullish()
|
||||
}).nullish()
|
||||
})
|
||||
+ })
|
||||
+ ).nullish(),
|
||||
+ images: import_v43.z.array(
|
||||
+ import_v43.z.object({
|
||||
@@ -49,14 +49,14 @@ index d73573b013e61903991cad2074187a3ade8921ab..919cbb2d14c0019716080c1bf041dde9
|
||||
+ image_url: import_v43.z.object({
|
||||
+ url: import_v43.z.string(),
|
||||
+ })
|
||||
+ })
|
||||
})
|
||||
).nullish()
|
||||
}),
|
||||
finish_reason: import_v43.z.string().nullish()
|
||||
@@ -964,6 +994,14 @@ var chunkBaseSchema = import_v43.z.looseObject({
|
||||
@@ -963,6 +993,14 @@
|
||||
thought_signature: import_v43.z.string().nullish()
|
||||
}).nullish()
|
||||
}).nullish()
|
||||
})
|
||||
+ })
|
||||
+ ).nullish(),
|
||||
+ images: import_v43.z.array(
|
||||
+ import_v43.z.object({
|
||||
@@ -64,18 +64,43 @@ index d73573b013e61903991cad2074187a3ade8921ab..919cbb2d14c0019716080c1bf041dde9
|
||||
+ image_url: import_v43.z.object({
|
||||
+ url: import_v43.z.string(),
|
||||
+ })
|
||||
+ })
|
||||
})
|
||||
).nullish()
|
||||
}).nullish(),
|
||||
finish_reason: import_v43.z.string().nullish()
|
||||
@@ -1529,6 +1567,16 @@
|
||||
// src/image/openai-compatible-image-model.ts
|
||||
var import_provider_utils5 = require("@ai-sdk/provider-utils");
|
||||
var import_v48 = require("zod/v4");
|
||||
+var defaultResponseFormatPrefixes = [
|
||||
+ "chatgpt-image-",
|
||||
+ "gpt-image-1-mini",
|
||||
+ "gpt-image-1.5",
|
||||
+ "gpt-image-1",
|
||||
+ "gpt-image-2"
|
||||
+];
|
||||
+function hasDefaultResponseFormat(modelId) {
|
||||
+ return defaultResponseFormatPrefixes.some((prefix) => modelId.startsWith(prefix));
|
||||
+}
|
||||
var OpenAICompatibleImageModel = class {
|
||||
constructor(modelId, config) {
|
||||
this.modelId = modelId;
|
||||
@@ -1625,7 +1673,7 @@
|
||||
n,
|
||||
size,
|
||||
...args,
|
||||
- response_format: "b64_json"
|
||||
+ ...(!hasDefaultResponseFormat(this.modelId) ? { response_format: "b64_json" } : {})
|
||||
},
|
||||
failedResponseHandler: (0, import_provider_utils5.createJsonErrorResponseHandler)(
|
||||
(_e = this.config.errorStructure) != null ? _e : defaultOpenAICompatibleErrorStructure
|
||||
diff --git a/dist/index.mjs b/dist/index.mjs
|
||||
index 237c9e209da9da2d4d557eb309e1adc2157c1426..2e140f053bb9d7e88ed6b54238d355e7f5aae49b 100644
|
||||
--- a/dist/index.mjs
|
||||
+++ b/dist/index.mjs
|
||||
@@ -559,6 +559,17 @@ var OpenAICompatibleChatLanguageModel = class {
|
||||
@@ -558,6 +558,17 @@
|
||||
type: "reasoning",
|
||||
text: reasoning
|
||||
});
|
||||
}
|
||||
+ }
|
||||
+ if (choice.message.images) {
|
||||
+ for (const image of choice.message.images) {
|
||||
+ const match1 = image.image_url.url.match(/^data:([^;]+)/);
|
||||
@@ -86,11 +111,10 @@ index 237c9e209da9da2d4d557eb309e1adc2157c1426..2e140f053bb9d7e88ed6b54238d355e7
|
||||
+ data: match2 ? match2[1] : image.image_url.url,
|
||||
+ });
|
||||
+ }
|
||||
+ }
|
||||
}
|
||||
if (choice.message.tool_calls != null) {
|
||||
for (const toolCall of choice.message.tool_calls) {
|
||||
const thoughtSignature = (_c = (_b = toolCall.extra_content) == null ? void 0 : _b.google) == null ? void 0 : _c.thought_signature;
|
||||
@@ -719,6 +730,17 @@ var OpenAICompatibleChatLanguageModel = class {
|
||||
@@ -719,6 +730,17 @@
|
||||
delta: delta.content
|
||||
});
|
||||
}
|
||||
@@ -108,10 +132,11 @@ index 237c9e209da9da2d4d557eb309e1adc2157c1426..2e140f053bb9d7e88ed6b54238d355e7
|
||||
if (delta.tool_calls != null) {
|
||||
if (isActiveReasoning) {
|
||||
controller.enqueue({
|
||||
@@ -915,6 +937,14 @@ var OpenAICompatibleChatResponseSchema = z3.looseObject({
|
||||
@@ -914,6 +936,14 @@
|
||||
thought_signature: z3.string().nullish()
|
||||
}).nullish()
|
||||
}).nullish()
|
||||
})
|
||||
+ })
|
||||
+ ).nullish(),
|
||||
+ images: z3.array(
|
||||
+ z3.object({
|
||||
@@ -119,14 +144,14 @@ index 237c9e209da9da2d4d557eb309e1adc2157c1426..2e140f053bb9d7e88ed6b54238d355e7
|
||||
+ image_url: z3.object({
|
||||
+ url: z3.string(),
|
||||
+ })
|
||||
+ })
|
||||
})
|
||||
).nullish()
|
||||
}),
|
||||
finish_reason: z3.string().nullish()
|
||||
@@ -951,6 +981,14 @@ var chunkBaseSchema = z3.looseObject({
|
||||
@@ -950,6 +980,14 @@
|
||||
thought_signature: z3.string().nullish()
|
||||
}).nullish()
|
||||
}).nullish()
|
||||
})
|
||||
+ })
|
||||
+ ).nullish(),
|
||||
+ images: z3.array(
|
||||
+ z3.object({
|
||||
@@ -134,7 +159,32 @@ index 237c9e209da9da2d4d557eb309e1adc2157c1426..2e140f053bb9d7e88ed6b54238d355e7
|
||||
+ image_url: z3.object({
|
||||
+ url: z3.string(),
|
||||
+ })
|
||||
+ })
|
||||
})
|
||||
).nullish()
|
||||
}).nullish(),
|
||||
finish_reason: z3.string().nullish()
|
||||
@@ -1543,6 +1581,16 @@
|
||||
postJsonToApi as postJsonToApi4
|
||||
} from "@ai-sdk/provider-utils";
|
||||
import { z as z8 } from "zod/v4";
|
||||
+var defaultResponseFormatPrefixes = [
|
||||
+ "chatgpt-image-",
|
||||
+ "gpt-image-1-mini",
|
||||
+ "gpt-image-1.5",
|
||||
+ "gpt-image-1",
|
||||
+ "gpt-image-2"
|
||||
+];
|
||||
+function hasDefaultResponseFormat(modelId) {
|
||||
+ return defaultResponseFormatPrefixes.some((prefix) => modelId.startsWith(prefix));
|
||||
+}
|
||||
var OpenAICompatibleImageModel = class {
|
||||
constructor(modelId, config) {
|
||||
this.modelId = modelId;
|
||||
@@ -1639,7 +1687,7 @@
|
||||
n,
|
||||
size,
|
||||
...args,
|
||||
- response_format: "b64_json"
|
||||
+ ...(!hasDefaultResponseFormat(this.modelId) ? { response_format: "b64_json" } : {})
|
||||
},
|
||||
failedResponseHandler: createJsonErrorResponseHandler4(
|
||||
(_e = this.config.errorStructure) != null ? _e : defaultOpenAICompatibleErrorStructure
|
||||
|
||||
14
pnpm-lock.yaml
generated
14
pnpm-lock.yaml
generated
@@ -44,7 +44,7 @@ patchedDependencies:
|
||||
hash: 3dbdd2fbbe5a44bc23a04b1195329dd8d78043447e925342ad5ed53c8c3b96ec
|
||||
path: patches/@ai-sdk__google@3.0.64.patch
|
||||
'@ai-sdk/openai-compatible@2.0.37':
|
||||
hash: d24ef2c45b84eab205e51894a90c5c36e93e6e0b17f56533aa37145129539a1a
|
||||
hash: 50ffd57b3f7d2de99f0315f0b3bca05cde1dc3d9f760d4f1be4143d8c6f260c7
|
||||
path: patches/@ai-sdk__openai-compatible@2.0.37.patch
|
||||
'@ai-sdk/openai@3.0.53':
|
||||
hash: 83a4d486601eff47ad8b5e22024878be2ced05b5b86e09884199a3f47fd53530
|
||||
@@ -216,7 +216,7 @@ importers:
|
||||
version: 3.0.53(patch_hash=83a4d486601eff47ad8b5e22024878be2ced05b5b86e09884199a3f47fd53530)(zod@4.3.4)
|
||||
'@ai-sdk/openai-compatible':
|
||||
specifier: ^2.0.37
|
||||
version: 2.0.37(patch_hash=d24ef2c45b84eab205e51894a90c5c36e93e6e0b17f56533aa37145129539a1a)(zod@4.3.4)
|
||||
version: 2.0.37(patch_hash=50ffd57b3f7d2de99f0315f0b3bca05cde1dc3d9f760d4f1be4143d8c6f260c7)(zod@4.3.4)
|
||||
'@ai-sdk/perplexity':
|
||||
specifier: ^3.0.29
|
||||
version: 3.0.29(zod@4.3.4)
|
||||
@@ -1306,7 +1306,7 @@ importers:
|
||||
version: 3.0.53(patch_hash=83a4d486601eff47ad8b5e22024878be2ced05b5b86e09884199a3f47fd53530)(zod@4.3.6)
|
||||
'@ai-sdk/openai-compatible':
|
||||
specifier: ^2.0.37
|
||||
version: 2.0.37(patch_hash=d24ef2c45b84eab205e51894a90c5c36e93e6e0b17f56533aa37145129539a1a)(zod@4.3.6)
|
||||
version: 2.0.37(patch_hash=50ffd57b3f7d2de99f0315f0b3bca05cde1dc3d9f760d4f1be4143d8c6f260c7)(zod@4.3.6)
|
||||
'@ai-sdk/provider':
|
||||
specifier: ^3.0.8
|
||||
version: 3.0.8
|
||||
@@ -1343,7 +1343,7 @@ importers:
|
||||
version: 3.0.53(patch_hash=83a4d486601eff47ad8b5e22024878be2ced05b5b86e09884199a3f47fd53530)(zod@4.3.4)
|
||||
'@ai-sdk/openai-compatible':
|
||||
specifier: ^2.0.37
|
||||
version: 2.0.37(patch_hash=d24ef2c45b84eab205e51894a90c5c36e93e6e0b17f56533aa37145129539a1a)(zod@4.3.4)
|
||||
version: 2.0.37(patch_hash=50ffd57b3f7d2de99f0315f0b3bca05cde1dc3d9f760d4f1be4143d8c6f260c7)(zod@4.3.4)
|
||||
'@ai-sdk/provider':
|
||||
specifier: ^3.0.8
|
||||
version: 3.0.8
|
||||
@@ -13036,13 +13036,13 @@ snapshots:
|
||||
'@ai-sdk/provider-utils': 4.0.23(zod@4.3.4)
|
||||
zod: 4.3.4
|
||||
|
||||
'@ai-sdk/openai-compatible@2.0.37(patch_hash=d24ef2c45b84eab205e51894a90c5c36e93e6e0b17f56533aa37145129539a1a)(zod@4.3.4)':
|
||||
'@ai-sdk/openai-compatible@2.0.37(patch_hash=50ffd57b3f7d2de99f0315f0b3bca05cde1dc3d9f760d4f1be4143d8c6f260c7)(zod@4.3.4)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 3.0.8
|
||||
'@ai-sdk/provider-utils': 4.0.23(zod@4.3.4)
|
||||
zod: 4.3.4
|
||||
|
||||
'@ai-sdk/openai-compatible@2.0.37(patch_hash=d24ef2c45b84eab205e51894a90c5c36e93e6e0b17f56533aa37145129539a1a)(zod@4.3.6)':
|
||||
'@ai-sdk/openai-compatible@2.0.37(patch_hash=50ffd57b3f7d2de99f0315f0b3bca05cde1dc3d9f760d4f1be4143d8c6f260c7)(zod@4.3.6)':
|
||||
dependencies:
|
||||
'@ai-sdk/provider': 3.0.8
|
||||
'@ai-sdk/provider-utils': 4.0.23(zod@4.3.6)
|
||||
@@ -15977,7 +15977,7 @@ snapshots:
|
||||
|
||||
'@opeoginni/github-copilot-openai-compatible@1.0.0(patch_hash=4bea0e526136ed32d0be4849ec5cbb41bd5de7b7418dd6920598357438cfef25)(zod@4.3.4)':
|
||||
dependencies:
|
||||
'@ai-sdk/openai-compatible': 2.0.37(patch_hash=d24ef2c45b84eab205e51894a90c5c36e93e6e0b17f56533aa37145129539a1a)(zod@4.3.4)
|
||||
'@ai-sdk/openai-compatible': 2.0.37(patch_hash=50ffd57b3f7d2de99f0315f0b3bca05cde1dc3d9f760d4f1be4143d8c6f260c7)(zod@4.3.4)
|
||||
'@ai-sdk/provider': 3.0.8
|
||||
'@ai-sdk/provider-utils': 4.0.23(zod@4.3.4)
|
||||
transitivePeerDependencies:
|
||||
|
||||
@@ -45,9 +45,13 @@ export function createAihubmix(options: AihubmixProviderSettings = {}): Aihubmix
|
||||
const resolveApiKey = () =>
|
||||
loadApiKey({ apiKey: options.apiKey, environmentVariableName: 'AIHUBMIX_API_KEY', description: 'AiHubMix' })
|
||||
|
||||
// Note: Do not hard-code `Content-Type: application/json` here. `postJsonToApi`
|
||||
// already defaults it for JSON endpoints, while `postFormDataToApi` (used by
|
||||
// `OpenAICompatibleImageModel` for `/images/edits`) relies on fetch to set
|
||||
// `multipart/form-data; boundary=...` automatically — forcing JSON here breaks
|
||||
// image edits with "invalid character '-' in numeric literal" on the server.
|
||||
const authHeaders = (): Record<string, string> => ({
|
||||
Authorization: `Bearer ${resolveApiKey()}`,
|
||||
'Content-Type': 'application/json',
|
||||
...APP_CODE_HEADER,
|
||||
...options.headers
|
||||
})
|
||||
|
||||
@@ -53,9 +53,13 @@ export function createNewApi(options: NewApiProviderSettings = {}): NewApiProvid
|
||||
const resolveApiKey = () =>
|
||||
loadApiKey({ apiKey: options.apiKey, environmentVariableName: 'NEWAPI_API_KEY', description: 'NewAPI' })
|
||||
|
||||
// Note: Do not hard-code `Content-Type: application/json` here. `postJsonToApi`
|
||||
// already defaults it for JSON endpoints, while `postFormDataToApi` (used by
|
||||
// `OpenAICompatibleImageModel` for `/images/edits`) relies on fetch to set
|
||||
// `multipart/form-data; boundary=...` automatically — forcing JSON here breaks
|
||||
// image edits with "invalid character '-' in numeric literal" on the server.
|
||||
const authHeaders = (): Record<string, string> => ({
|
||||
Authorization: `Bearer ${resolveApiKey()}`,
|
||||
'Content-Type': 'application/json',
|
||||
...options.headers
|
||||
})
|
||||
|
||||
|
||||
@@ -118,6 +118,7 @@ const IMAGE_ENHANCEMENT_MODELS = [
|
||||
'grok-2-image(?:-[\\w-]+)?',
|
||||
'qwen-image-edit',
|
||||
'gpt-image-1',
|
||||
'gpt-image-2',
|
||||
'gemini-2.5-flash-image(?:-[\\w-]+)?',
|
||||
'gemini-2.0-flash-preview-image-generation',
|
||||
'gemini-3(?:\\.\\d+)?-(?:flash|pro)-image(?:-[\\w-]+)?'
|
||||
|
||||
@@ -38,7 +38,6 @@ import {
|
||||
getQuickModel
|
||||
} from './AssistantService'
|
||||
import { ConversationService } from './ConversationService'
|
||||
import FileManager from './FileManager'
|
||||
import { injectUserMessageWithKnowledgeSearchPrompt } from './KnowledgeService'
|
||||
import type { BlockManager } from './messageStreaming'
|
||||
import type { StreamProcessorCallbacks } from './StreamProcessingService'
|
||||
@@ -312,12 +311,18 @@ async function collectImagesFromMessages(userMessage: Message, assistantMessage?
|
||||
const images: string[] = []
|
||||
|
||||
// 收集用户消息中的图像
|
||||
// NOTE: Use `block.file.name` (always the on-disk filename) rather than
|
||||
// `block.file.id + block.file.ext` — some save paths (saveBase64Image,
|
||||
// savePastedImage) store `ext` without the leading dot, so concatenation
|
||||
// produces broken paths like `<uuid>jpg` → ENOENT.
|
||||
// Also note: `block.file.type` is a FileType enum (e.g. "image"), NOT a MIME
|
||||
// type. `base64Image` derives the real MIME from the extension internally
|
||||
// (and normalizes jpg → image/jpeg).
|
||||
const userImageBlocks = findImageBlocks(userMessage)
|
||||
for (const block of userImageBlocks) {
|
||||
if (block.file) {
|
||||
const base64 = await FileManager.readBase64File(block.file)
|
||||
const mimeType = block.file.type || 'image/png'
|
||||
images.push(`data:${mimeType};base64,${base64}`)
|
||||
const { data } = await window.api.file.base64Image(block.file.name)
|
||||
images.push(data)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -401,17 +406,20 @@ export async function fetchImageGeneration({
|
||||
image: { type: imageType, images }
|
||||
})
|
||||
|
||||
onChunkReceived({
|
||||
type: ChunkType.LLM_RESPONSE_COMPLETE,
|
||||
response: {
|
||||
usage: { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 },
|
||||
metrics: {
|
||||
completion_tokens: 0,
|
||||
time_first_token_millsec: 0,
|
||||
time_completion_millsec: Date.now() - startTime
|
||||
}
|
||||
// Emit BLOCK_COMPLETE so the stream processor's onComplete runs and the
|
||||
// assistant message transitions out of "processing". Without this, the
|
||||
// trailing PlaceholderBlock in Blocks/index.tsx stays visible next to the
|
||||
// finished image because `isMessageProcessing(message)` remains true.
|
||||
const imageResponse = {
|
||||
usage: { prompt_tokens: 0, completion_tokens: 0, total_tokens: 0 },
|
||||
metrics: {
|
||||
completion_tokens: 0,
|
||||
time_first_token_millsec: 0,
|
||||
time_completion_millsec: Date.now() - startTime
|
||||
}
|
||||
})
|
||||
}
|
||||
onChunkReceived({ type: ChunkType.BLOCK_COMPLETE, response: imageResponse })
|
||||
onChunkReceived({ type: ChunkType.LLM_RESPONSE_COMPLETE, response: imageResponse })
|
||||
} catch (error) {
|
||||
onChunkReceived({ type: ChunkType.ERROR, error: error as Error })
|
||||
throw error
|
||||
|
||||
@@ -71,14 +71,21 @@ export const createImageCallbacks = (deps: ImageCallbacksDependencies) => {
|
||||
}
|
||||
}
|
||||
|
||||
if (!imageBlockId && blockManager.hasInitialPlaceholder) {
|
||||
imageBlockId = blockManager.initialPlaceholderBlockId
|
||||
}
|
||||
|
||||
if (imageBlockId) {
|
||||
if (!imageData) {
|
||||
const changes: Partial<ImageMessageBlock> = {
|
||||
status: MessageBlockStatus.SUCCESS
|
||||
}
|
||||
blockManager.smartBlockUpdate(imageBlockId, changes, MessageBlockType.IMAGE)
|
||||
blockManager.smartBlockUpdate(imageBlockId, changes, MessageBlockType.IMAGE, true)
|
||||
} else {
|
||||
const changes = await buildImageBlockFields(imageData)
|
||||
const changes = {
|
||||
type: MessageBlockType.IMAGE,
|
||||
...(await buildImageBlockFields(imageData))
|
||||
}
|
||||
blockManager.smartBlockUpdate(imageBlockId, changes, MessageBlockType.IMAGE, true)
|
||||
}
|
||||
imageBlockId = null
|
||||
|
||||
Reference in New Issue
Block a user