useCache, useSharedCache and usePersistCache setters now accept a React-style functional updater `(prev) => next` in addition to a concrete value. The updater resolves `prev` from the latest stored value at write time rather than the render-time snapshot, making read-modify-write correct across an `await` — the root cause of the keep-alive overwrite race behind #16460.
`prev` is typed shallow-readonly (`ReadonlyValue<T>`), so mutating it in place and returning the same reference — which the CacheService `isEqual` short-circuit would otherwise swallow silently — is a compile error. Concrete-value calls are unchanged, so existing consumers keep compiling.
The renderer useCache mock mirrors the functional branch with the same default fallback; docs and hook tests updated. Consumer call-site adoption lands separately.
### What this PR does
Before this PR:
- Knowledge embeddings and reranking ran through the legacy
embedjs-based
knowledgeV1 stack with their own provider clients, independent of the
app's
AI service.
- File-processing intake accepted several heterogeneous input shapes,
and
knowledge file items were tracked by FileEntry ids, coupling file
content to
the file-manager entry/cache.
After this PR:
- Embeddings and reranking are routed through the unified `AiService`
(with
cherryin rerank support) and guarded by strict embedding-dimension
validation
that rejects stale/mismatched vectors.
- File-processing intake is collapsed to a single path-based model;
knowledge
file items are stored by base-relative path under the knowledge-base
directory, and v1 uploads are copied into the v2 base dir during
migration so
migrated items stay reindexable/restorable.
- Legacy `knowledgeV1` is removed; the orchestration services were
renamed to
`KnowledgeService` / `FileProcessingService`.
- Chat -> knowledge attach is temporarily disconnected (tracked TODO)
while the
v2 file-manager bridge is rebuilt.
Fixes #N/A (no linked issue)
### Why we need it and why it was done in this way
Routing embeddings/rerank through `AiService` unifies provider handling
and
credentials and removes the parallel embedjs client stack and its v1
coupling.
Storing knowledge files by base-relative path (instead of FileEntry ids)
makes
each knowledge base self-contained and portable.
The following tradeoffs were made:
- A large, coordinated refactor plus a migration step that physically
copies v1
uploads into the v2 base dir, in exchange for removing the parallel
client
stack and making bases self-contained.
- Base-relative path storage required a fail-fast/dedup strategy for
same-named
files and a guard for blank legacy filenames.
The following alternatives were considered:
- Keeping the embedjs stack behind an adapter — rejected; perpetuates
the
parallel client and v1 coupling.
- Keeping FileEntry-id storage — rejected; couples knowledge files to
the
file-manager cache and blocks portability.
### Breaking changes
- `knowledgeV1` is removed. Legacy v1 knowledge data reaches v2 only
through the
v2 migrators; there is no v1 fallback.
- The v2 knowledge HTTP API (API gateway) now returns v2-native
per-entry fields
(`embeddingModelId`, `createdAt` on base entries; `chunkId`,
`scoreKind`,
`rank` on search results). The response envelope (`knowledge_bases`,
`searched_bases`, `total`) is unchanged. See
`v2-refactor-temp/docs/breaking-changes/2026-06-05-knowledge-api-v2.md`.
### Special notes for your reviewer
- This branch went through several rounds of multi-agent code review.
The most
recent 6 commits address review findings: directory-import path
collisions,
migrated-file source copying + blank `relativePath` guard, addItems
rollback
error preservation, eager `document_to_markdown` output-target
validation, a
`CompletedKnowledgeBase` type guard, and breaking-changes doc
corrections.
- Chat -> knowledge attach is intentionally disconnected for now
(tracked in
`v2-refactor-temp/docs/knowledge/knowledge-todo.md`).
- Local full `pnpm lint`/`pnpm test` was not run per the project's
review
conventions; please rely on CI / `pnpm build:check`.
### Checklist
- [x] Branch: This PR targets the correct branch — `main` for active
development, `v1` for v1 maintenance fixes
- [x] PR: The PR description is expressive enough and will help future
contributors
- [x] Code: Write code that humans can understand and Keep it simple
- [x] Refactor: You have left the code cleaner than you found it (Boy
Scout Rule)
- [x] Upgrade: Impact of this change on upgrade flows was considered and
addressed if required
- [ ] Documentation: A user-guide update was considered and is present
(link) or not required.
- [x] Self-review: I have reviewed my own code before requesting review
from others
### Release note
```release-note
NONE
```
---------
Signed-off-by: eeee0717 <chentao020717Work@outlook.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
libsql client-ts upstream issue #288 makes PRAGMA busy_timeout ineffective
for async transactions, so concurrent db.transaction() calls reliably surface
SQLITE_BUSY. Introduce DbService.withWriteTx as a serialized write helper:
- Process-wide FIFO mutex (async-mutex) serializes write transactions.
- libsql client's default BEGIN IMMEDIATE protects against read-then-write
tx upgrade failures (no override needed at the drizzle layer).
- Single 50ms BUSY retry guards against transient external locks.
Reads do NOT need this — WAL gives readers snapshot isolation that is never
blocked by writers.
Includes unit tests (FIFO ordering, finally release on throw, single BUSY
retry, persistent BUSY rethrow, non-BUSY passthrough) plus a real-libsql
integration test. Updates the DbService test mock with a passthrough
withWriteTx so dependent services do not throw "is not a function" in
tests. Documents the API in database-patterns.md and points
CLAUDE.md / data-api-overview.md at the new pattern.
### What this PR does
Before this PR:
File processing on the `v2` branch was still described and wired around
split OCR / markdown APIs, legacy feature names, and feature-first
provider structure. OCR-like image text extraction and
document-to-markdown conversion did not share one task contract, and
provider task ids / polling details were harder to keep behind the
Main-process boundary.
After this PR:
`v2` file processing follows
`v2-refactor-temp/docs/fileProcessing/file-processing-service.md` as the
design baseline:
- exposes one Main-side task API: `startTask`, `getTask`, and
`cancelTask`
- replaces split file-processing IPC with `file-processing:start-task`,
`file-processing:get-task`, and `file-processing:cancel-task`
- renames features and preference keys to `image_to_text` and
`document_to_markdown`
- adds `FileProcessingTaskService` as the in-memory source of truth for
task ids, task state, progress, cancellation, TTL pruning, remote-poll
dedupe, and task change events
- keeps provider task ids, remote context, query context, abort
controllers, and in-flight polling inside Main-process task records
- maps completed results to artifacts: inline `text/plain` for
`image_to_text`, and persisted markdown file artifacts for
`document_to_markdown`
- reorganizes providers into processor-first handlers under
`src/main/services/fileProcessing/processors`
- moves Tesseract worker ownership under `processors/tesseract/runtime`
- removes the new file-processing module's old `ocr/` and `markdown/`
split directories after migrating their logic
- updates shared schemas, presets, preference generation, migration
mappings, and tests for the renamed feature model
The public file-processing contract is now:
```ts
await window.api.fileProcessing.startTask({
feature: 'image_to_text',
file,
processorId: 'tesseract'
})
await window.api.fileProcessing.getTask({ taskId })
await window.api.fileProcessing.cancelTask({ taskId })
```
Architecture overview:
```text
Renderer / upper-layer caller
|
| startTask / getTask / cancelTask
v
FileProcessingOrchestrationService
|
| Zod validation + delegation
v
FileProcessingTaskService
|
| taskId, task store, TTL, cancellation,
| background execution, remote polling, artifacts
v
processorRegistry[processorId].capabilities[feature]
|
+--> image-to-text handlers
| -> text/plain artifact
|
+--> document-to-markdown handlers
-> feature.files.data/fileId/file-processing/taskId/output.md
```
Notes:
- The new file-processing API does not keep facades for
`file-processing:extract-text`,
`file-processing:start-markdown-conversion-task`, or
`file-processing:get-markdown-conversion-task-result`.
- `FileProcessingOrchestrationService` is intentionally only the IPC
validation and delegation layer.
- Task state is Main-process runtime coordination state, not DataApi or
Cache state.
- Renderer task subscriptions, a global UI task center, and full
renderer business-flow migration are intentionally out of scope for this
PR.
- The legacy standalone OCR path outside the new file-processing module
can coexist during the v2 transition, but the new file-processing
interface is not polluted by those split-API types.
Fixes #N/A
### Why we need it and why it was done in this way
This PR makes OCR-style image text extraction and document-to-markdown
conversion use the same Main-process task model before renderer-side
adoption. The unified contract gives upper layers one way to start work,
query progress, handle failure, cancel work, and consume completed
artifacts without learning provider-specific polling details.
The following tradeoffs were made:
- Fast OCR now also goes through a task API, so callers need start/query
behavior instead of a direct `extractText -> text` call.
- Task state remains session-scoped in memory; completed artifacts are
persisted, but task snapshots are not restored after app restart.
- Remote-provider cancellation is best-effort: local polling and state
transition stop immediately, but third-party provider-side cancellation
is not guaranteed.
- Renderer integration is intentionally compile-safe and minimal in this
PR; full UX migration should happen in follow-up changes.
- Tesseract keeps a processor-owned runtime service, while other
processors stay as handlers/utilities until they need lifecycle-managed
resources.
The following alternatives were considered:
- Keeping separate OCR and markdown conversion APIs, which would
preserve the current split but continue duplicating task, progress,
cancellation, and result semantics.
- Adding a DataApi task table or Cache mirror for file-processing task
state, which would create a second source of truth for runtime
coordination state.
- Adding renderer push subscriptions in this PR, which would expand the
scope beyond the Main-side task contract.
- Introducing a generic process manager for all processors, which is
premature while only Tesseract currently owns reusable lifecycle
resources.
Links to places where the discussion took place:
`v2-refactor-temp/docs/fileProcessing/file-processing-service.md`
### Breaking changes
None for released user-facing behavior.
If this PR introduces breaking changes, please describe the changes and
the impact on users.
For the in-progress `v2` file-processing integration, this replaces the
split file-processing IPC/preload shape with the unified
`startTask/getTask/cancelTask` contract. It also renames file-processing
feature and preference keys from the old `text_extraction` /
`markdown_conversion` model to `image_to_text` / `document_to_markdown`.
### Special notes for your reviewer
- This PR targets `v2`, not `main`.
- Review this against
`v2-refactor-temp/docs/fileProcessing/file-processing-service.md`; that
document is the source of truth for the module boundary.
- Main review points: unified task API, artifact model, cancellation
semantics, processor-first registry/handlers, hidden provider runtime
state, and no DataApi/Cache task storage.
- `document_to_markdown` artifacts are persisted under
`application.getPath('feature.files.data')/fileId/file-processing/taskId/output.md`.
- `image_to_text` artifacts are returned inline as plain text artifacts
and are not persisted.
- Current local verification status:
- `pnpm format`: passed
- `pnpm build:check`: passed
- Vitest inside `build:check`: `432` test files passed, `7171` tests
passed, `72` skipped
### Checklist
This checklist is not enforcing, but it's a reminder of items that could
be relevant to every PR.
Approvers are expected to review this list.
- [x] PR: The PR description is expressive enough and will help future
contributors
- [x] Code: [Write code that humans can
understand](https://en.wikiquote.org/wiki/Martin_Fowler#code-for-humans)
and [Keep it simple](https://en.wikipedia.org/wiki/KISS_principle)
- [x] Refactor: You have [left the code cleaner than you found it (Boy
Scout
Rule)](https://learning.oreilly.com/library/view/97-things-every/9780596809515/ch08.html)
- [x] Upgrade: Impact of this change on upgrade flows was considered and
addressed if required
- [ ] Documentation: A [user-guide update](https://docs.cherry-ai.com)
was considered and is present (link) or not required. Check this only
when the PR introduces or changes a user-facing feature or behavior.
- [x] Self-review: I have reviewed my own code (e.g., via
[`/gh-pr-review`](/.claude/skills/gh-pr-review/SKILL.md), `gh pr diff`,
or GitHub UI) before requesting review from others
### Release note
```release-note
NONE
```
---------
Signed-off-by: eeee0717 <chentao020717Work@outlook.com>
Co-authored-by: fullex <106392080+0xfullex@users.noreply.github.com>
Enables main-process services to react to cache changes without writer-side
wiring — unblocks the web-search/OCR provider rotation use case where new
providers would otherwise require manual hook-ups at every write site.
Also unifies equality across main and renderer on lodash.isEqual (fixing
redundant cross-window broadcasts when Record/Array values are rebuilt on
every write) and extracts template utilities to the shared package so both
processes use one implementation.
Align SharedCache type system with Memory (UseCache) template support and
remove the sharedCasual escape hatch that existed only because SharedCache
could not type-check dynamic keys:
- Introduce InferSharedCacheValue and expand SharedCacheKey through
ProcessKey so template schema entries like
'web_search.provider.last_used_key.${providerId}' match concrete keys
with precise value types on both Main and Renderer.
- Extend useSharedCache hook with findMatchingSharedCacheSchemaKey /
getSharedCacheDefaultValue, mirroring useCache's template-aware default
resolution.
- Remove getSharedCasual / setSharedCasual / hasSharedCasual /
deleteSharedCasual / hasSharedTTLCasual from the Renderer CacheService
and all test mocks; Main CacheService never had them.
- Migrate BaseWebSearchProvider and OcrBaseApiClient rotation from
sharedCasual to type-safe getShared/setShared. Rename keys to conform
to schema naming rules (ESLint data-schema-key/valid-key):
web-search-provider:${id}:last_used_key
-> web_search.provider.last_used_key.${providerId}
ocr_provider:${id}:last_used_key
-> ocr.provider.last_used_key.${providerId}
Old values under legacy key names become orphans after rollout; this
is acceptable because rotation state is transient and consumers
reinitialize from keys[0] on a miss.
- Add type-level assertions for SharedCacheKey and InferSharedCacheValue
in useCache.types.test.ts to lock the contract in CI.
- Update cache-overview.md, cache-usage.md, and tests/__mocks__/README.md
to describe SharedCache's template support and reflect that casual
methods now exist only on the Memory tier.
The hooks shipped in 87d1a947a without test coverage or team-facing
mocks. Consumer tests would otherwise need to hand-roll an SWRConfig
wrapper or skip asserting cache-control semantics entirely. This adds:
- Direct unit tests that exercise the real hooks inside a self-provided
SWRConfig (covering key shape, empty-query folding, no-revalidate
writes, non-reactive reads, and read/write round-trip).
- Mock factories in tests/__mocks__/renderer/useDataApi.ts backed by a
shared in-memory Map, plus seedCache/getCachedValue/clearCache
helpers on MockUseDataApiUtils. Registered in MockUseDataApi so the
global renderer setup picks them up without further wiring.
- Docs update in tests/__mocks__/README.md with signatures and usage.
The mocks README described what was mocked but never stated that
tests/__mocks__/main/ is intentionally limited to cross-cutting
infrastructure. Without that boundary in writing, feature-specific
lifecycle services were getting registered into defaultServiceInstances,
bloating the global default surface without serving any other test.
Adds an explicit Scope section with a pattern-selection table, documents
the canonical local-stub pattern for feature-specific lifecycle services
(vi.mock on @application + MockBaseService stand-in), and a Common
Assertions table covering phase, dependencies, IPC handlers, and
disposables. Adds the missing Main PreferenceService section to match the
other three infrastructure service docs. Links the lifecycle README to
the new testing guidance.
Align with the existing @logger alias convention by introducing
@application as a short alias for src/main/core/application. This
reduces import verbosity across ~130 main-process files while keeping
4 intentional sub-path imports (@main/core/application/Application)
unchanged to preserve the vi.mock bypass mechanism in tests.
Configured in tsconfig.node.json and electron.vite.config.ts; Vitest
inherits the alias automatically.
Signed-off-by: fullex <0xfullex@gmail.com>
Replace ad-hoc vi.mock('@main/core/application') calls across test files
with a centralized mockApplicationFactory(overrides?) utility that provides
all registered services by default and supports per-service overrides.
- Add tests/__mocks__/main/DbService.ts (standalone DbService mock)
- Add tests/__mocks__/main/application.ts (unified factory)
- Simplify tests/main.setup.ts global mock
- Migrate 4 test files to use mockApplicationFactory
- Document unified mock system in CLAUDE.md and README.md
Signed-off-by: fullex <0xfullex@gmail.com>
- Introduced a new path alias '@test-mocks' in TypeScript configuration files to simplify the import of mock utilities across the project.
- Updated relevant files to utilize the new alias, enhancing code readability and maintainability.
- Enhanced the README for test mocks to document the new import path, providing clearer guidance for developers on using mock utilities.
- Added a new section in the data management documentation for testing, including a link to unified test mocks for Cache, Preference, and DataApi.
- Refined the README for test mocks, providing a clearer overview and detailed descriptions of available mocks and their organization.
- Improved the structure and clarity of mock usage examples, ensuring better guidance for developers on utilizing the testing utilities effectively.
- Deleted batch and transaction related schemas, handlers, and IPC channels to streamline the Data API.
- Updated related type definitions and import paths to reflect the removal of batch and transaction functionalities.
- Simplified the API server and adapter logic by eliminating unused methods and handlers.
- Refactored tests in MainTextBlock and ThinkingBlock to utilize the usePreference hook for managing user settings.
- Updated snapshots in DraggableVirtualList test to reflect changes in class names.
- Enhanced export tests to ensure proper handling of markdown formatting and citation footnotes.
- Mocked additional dependencies globally for improved test reliability.
