Pagination docs were scattered across api-types.md (types + cursor
semantics), data-api-in-renderer.md (hooks), data-api-in-main.md (offset
example + keyset note), api-design-guidelines.md (query params), and
data-ordering-guide.md (cache shapes + determinism), with no single
discoverable home for the offset-vs-cursor model.
Add docs/references/data/data-pagination-guide.md as the canonical hub
(mirrors data-ordering-guide.md): two modes, four-layer quickstart, wire
contract, server impl (offset + keyset cursor + multi-band caveat),
renderer consumption, FTS pagination, gotchas, and a see-also map. Other
docs keep their authoritative slice and link to the guide; the migrated
conceptual prose is removed from api-types.md to avoid duplication.
Also fix two pre-existing broken anchors found while verifying links
(database-patterns withWriteTx; ordering guide section number).
Consolidate the per-service <key>:<id> cursor codec and the keyset
WHERE/ORDER BY tuple into services/utils/keysetCursor.ts. keysetOrdering
derives the WHERE predicate and the matching ORDER BY from one direction
spec, so the two cannot drift apart and silently skip/repeat rows.
Migrate TranslateHistory, AgentSession, AgentSessionMessage (list), and
Painting to the shared util; delegate ftsSearch's codec to parseCursor /
encodeCursor while keeping its 422-throw policy. Harden Painting's cursor
from a single key to a defensive (orderKey, id) tuple.
Replace the `await import(...)` cycle-breaking hack with a lightweight, type-safe service locator. Participating services self-register and resolve siblings lazily through `getDataService(...)`; the registry imports services only as `import type`, so it stays a sink in the import graph and no value cycle can form. Only the services in a real cycle join the registry.
- add dataServiceRegistry (register/get) + symmetric migration of the two cycles: Message<->Topic, Provider<->ProviderRegistry
- tests load the sibling via a side-effect import so it self-registers
- document the pattern in data-api-in-main.md and a new services/README.md
DataApi handlers and their services may perform only SQLite reads/writes via Drizzle; fs/network/process/external-service side effects are prohibited regardless of nesting depth or an accompanying DB write. Add a Hard Rule section to api-design-guidelines, scope-limit service domain workflows to DB I/O, and echo the boundary in the overview and README.
Apply the naming-conventions §6.1 acronym-casing rule (MCP -> Mcp) to the
MCP* PascalCase identifier family across the codebase (McpServer, McpTool,
McpToolResponse, BuiltinMcpServerNames, McpService, etc.) plus the local
identifiers boundMcp/enableMcp/disableMcp and the didiMcp registry key.
Regenerate the OpenAPI spec from the renamed schemas.
Deliberately left unchanged (not naming-convention identifiers): persisted
field keys read by migrators (enabledMCPs), v1 Redux selectors (selectMCP),
string values (ExaMCP, logger labels), and UPPER_SNAKE constants (MCP_*).
Also fix naming issues in the data reference docs that prompted this:
- JSONStreamReader -> JsonStreamReader (match the real class name)
- rowToMCPServer -> rowToMcpServer (match the real function name)
- replace the TopicService getInstance() skeleton with a direct singleton
- sync stale MCPServer/MCPTool/McpService references in affected docs
packages/shared was never a real pnpm workspace package (no package.json); it was referenced only through the @shared TypeScript path alias. Relocate it under src/ via git mv (143 files, detected as pure renames).
Repoint the @shared alias and include globs to src/shared across electron.vite.config.ts, tsconfig.{json,node,web}.json and vitest.config.ts; update scripts/check-custom-exts.ts, scripts/update-languages.ts, the eslint.config.mjs generated-file globs, the data-classify generator output targets, .github/CODEOWNERS path rules, and CLAUDE.md/docs/source-comment references.
The @shared alias name is unchanged, so all 1403 @shared/* import sites resolve without modification. Verified with typecheck:node, typecheck:web and the full test suite (700 files, 9739 tests passing).
- data-api-in-main: add Registry Sub-Resource Endpoints section -
GET-only for stateless reads, AIP-136 colon notation for derived
views, registry packages are main-only (bundle waste + merge
already in main)
- best-practice-layered-preset-pattern: preset-only static fields
must merge in rowToEntity rather than via parallel endpoint;
document acceptable exceptions for catalog and specialised
surfaces
- data-ordering-guide section 2: drop user_provider.isEnabled from
the Live partition list - the table is whole-table ordered
(already correct in section 7)
- database-patterns: flag boolean columns without .notNull() as a
common R3 offender, with concrete wrong/right example
All service methods accepting a Drizzle transaction now follow:
- tx is the first parameter
- method name ends with Tx
Renamed:
- pinService.purgeForEntity / purgeForEntities -> *Tx
- tagService.purgeForEntity / purgeForEntities -> *Tx
- TagService#assertTagsExist (private) -> assertTagsExistTx
- PromptService#assertPromptsExist (private) -> assertPromptsExistTx
- AssistantService#syncRelations (private) -> syncRelationsTx
- AgentService#syncSettingsToSessions (private) -> syncSettingsToSessionsTx
The convention is documented in docs/references/data/data-api-in-main.md
under the new "Transaction Method Naming" section. Behavior is unchanged.
Establish team standards for placing default values across the data stack (DB DEFAULT / Drizzle $defaultFn / Zod .default() / service ??) and judging column nullability.
Originating context: PR #14689 fixed a PATCH leakage bug rooted in defaults living in three places at once (DB, Zod Create schema, row mapper) for the assistant entity. The follow-up discussion recovered general principles that other entities (agent, message) also violate; this doc captures them as a reference for future schema/service work.
- New: best-practice-default-values-and-nullability.md — Five rules (R1-R5), decision matrices for nullability and default placement, standard layered design example, anti-patterns table, case studies (assistant, modelId, agent.accessiblePaths)
- api-design-guidelines.md: refine Rule C Update derivation guidance; add Rule E discouraging Zod .default() on entity / Create / Update schemas
- data-api-in-main.md: upgrade row-mapper ?? fallbacks from "needs hand-write" tolerance to anti-pattern; add Write-path defaults section codifying R4
- database-patterns.md: add Column Nullability and Defaults section; add R3 no-fabricated-fallbacks bullet to Row → Entity Mapping
- README.md: index entry under Reference Guides
No code changes. Implementation follow-up will land in separate PRs that apply these rules entity by entity.
Mirror the existing single-row purgeForEntity on PinService and TagService
with a purgeForEntities(tx, entityType, ids[]) variant. Empty input is a
no-op; non-empty input collapses N round trips into one and emits a single
aggregated log line with a count, removing the "loop and log per id" smell
when consumers later cascade deletes for many entities of the same type.
Document a Cross-Service Table Access rule in data-api-in-main.md:
- Writes to a foreign table must go through the owner (pass tx into its
method) — owners are the single source of truth for invariants and the
emitter of mutation logs; foreign writes split that knowledge.
- Reads that inline-JOIN a foreign table are allowed when JOIN is the
simpler path (e.g. AssistantService.list pulling per-row tags).
The new bulk methods are intentionally landed without a consumer so that
the "add a bulk method on the owner instead of writing raw SQL from the
caller" path described by the rule is available the moment a cascading
delete appears, rather than nudging the first author to bypass the owner.
All 14 per-module handler files used the same 7-line mapped type.
Extract it into a single HandlersFor<Schemas> helper in apiTypes and
migrate each file to a one-line annotation (net -119 lines).
The helper is defined as Pick<ApiImplementation, Extract<keyof Schemas,
keyof ApiImplementation>> rather than re-applying ApiHandler<Path, Method>
over generic parameters -- the latter triggers TS2590 (union too complex)
because ApiHandler nests several conditional types that must resolve
symbolically when Path is a free type variable. Indexing the
already-evaluated ApiImplementation sidesteps that.
Also elevate the annotation rule to a dedicated "Handler Type Annotation"
subsection in the data-api-in-main doc and add a type-level regression
matrix under handlers/__tests__.
- Add "Zod Schema & DTO Conventions" section to api-design-guidelines.md
covering the four rules applied across the prior refactor commits:
A. type (not interface) for XxxSchemas route tables
B. XxxSchema for Zod constants, XxxDto for TypeScript type names
C. EntitySchema.pick({...}) whitelist derivation with field atoms
and z.strictObject (guarded against overposting)
D. handwritten Zod everywhere; no drizzle-zod; no pure TS interface
DTOs (responses stay as interface — opposite direction of the
IPC trust boundary)
- Include a decision rule for when to extract an XXX_MUTABLE_FIELDS
constant: both Create/Update share the pick set AND fields >= 5
- Sync example code in api-types.md to the new conventions
- Flip the TopicSchemas example in data-api-in-main.md from interface
to type
Add `.notNull()` to `createdAt` / `updatedAt` in the shared
`createUpdateTimestamps` helper so Drizzle `$inferSelect` produces
`number` instead of the misleading `number | null`. `deletedAt` in
`createUpdateDeleteTimestamps` stays nullable (soft-delete semantics).
Generated migration 0013 rebuilds 26 affected tables via the standard
SQLite table-recreation pattern; FK / CHECK / INDEX constraints are
preserved across rebuild. No backfill is added (project is in the
development phase; null pre-existing rows are accepted as a "wipe DB"
signal rather than engineered around).
Fix upstream in `KnowledgeMappings.toTimestamp` so it returns a
`Date.now()` fallback instead of `undefined` — otherwise future Dexie
-> v2 migrator runs would try to insert undefined into NOT NULL
columns. Three test assertions updated from `undefined` to
`expect.any(Number)`.
Sweep 18 downstream call sites across 9 functions that were carrying a
dead `?? new Date().toISOString()` fallback:
- AssistantService, KnowledgeBaseService, KnowledgeItemService,
MessageService, TopicService, TranslateHistoryService,
TranslateLanguageService (the original Pattern A set)
- McpServerService and MiniAppService.rowToMiniApp (reclassified from
Pattern B: the domain types stay `optional` to accommodate builtin
literals in the renderer, but `string` assigns legally into
`string | undefined`, so the switch is safe)
Keep `MiniAppService.builtinToMiniApp` on `timestampToISOOrUndefined`
— its `dbRow?: MiniAppSelect` semantics ("the preference row may not
exist at all") is genuinely optional, not a disguised "nullable column".
Also remove a Pattern C that neither the plan nor the grep audit
caught: `TagService.ensureTagTimestamp` was a self-rolled defense
layer that threw INTERNAL_SERVER_ERROR on null timestamps. The DB
now refuses to produce such rows, so the defense — and the test
named "should surface timestamp anomalies instead of masking them" —
are dead code. Removed both.
Update three docs to reflect the new defaults:
- `services/utils/README.md` — drop the "DB still nullable" table row
and the predictive paragraph; reframe Pattern B around
"whole row may not exist"
- `services/utils/rowMappers.ts` JSDoc — same reframing
- `docs/references/data/data-api-in-main.md` — delete the fallback
code samples and simplify Convention §3
Introduce shared `services/utils/rowMappers.ts` exporting three helpers:
`nullsToUndefined` (renamed from the previous `stripNulls`, with a
corrected type signature that preserves `notNull()` columns unchanged),
`timestampToISO` for guaranteed-present timestamps, and
`timestampToISOOrUndefined` for nullable ones.
Migrate 9 services off hand-rolled null/date handling:
- Eliminate the `stripNulls` duplicate in MiniAppService
- Replace 18 repetitive createdAt/updatedAt ternaries with helper calls
- Fix McpServerService misuse where an optional domain field was being
forced to a synthesized "now" value; restore honest undefined semantics
- Simplify KnowledgeBaseService.rowToKnowledgeBase via the spread idiom,
bypassing `clean` for the `T | null` embeddingModelId field
Document the paradigm in docs/references/data/data-api-in-main.md with
standard and advanced skeletons, plus an explicit "when NOT to spread"
list covering services with field renaming, custom fallbacks, computed
fields, or sensitive-data sanitization. Per-helper design decisions
(shallow vs. recursive, rejected alternatives) live in
services/utils/README.md.
IpcAdapter is a transport adapter bridging Electron IPC to the
transport-agnostic ApiServer. Previously it managed its own lifecycle
via manual setupHandlers()/removeHandlers() calls, bypassing
BaseService's Disposable tracking.
Now IpcAdapter implements Disposable (setup/dispose), and
DataApiService registers it via registerDisposable() — eliminating
manual onStop() cleanup and gaining exception-safe teardown via
BaseService's finally block. This also paves the way for future
HttpAdapter to follow the same pattern.
Signed-off-by: fullex <0xfullex@gmail.com>
Align with the existing @logger alias convention by introducing
@application as a short alias for src/main/core/application. This
reduces import verbosity across ~130 main-process files while keeping
4 intentional sub-path imports (@main/core/application/Application)
unchanged to preserve the vi.mock bypass mechanism in tests.
Configured in tsconfig.node.json and electron.vite.config.ts; Vitest
inherits the alias automatically.
Signed-off-by: fullex <0xfullex@gmail.com>
Services now handle both business logic and data access directly via Drizzle ORM.
Repository pattern is strongly discouraged unless absolutely necessary.
Signed-off-by: fullex <0xfullex@gmail.com>
