Files
CherryHQ-cherry-studio/docs/references/data/data-api-in-main.md
Phantom d2c568e349 feat(file): Add schema and foundation for new file module (#13451)
### What this PR does

Adds the **Phase 1a contract surface** for the file module — types, DB
schema, DataApi + File IPC contracts, FileManager skeleton, and
architecture docs.

**Phase 1b.1 (Read Path & Repository), 1b.2 (Write Path & Lifecycle),
1b.3 (Watcher & DanglingCache), and 1b.4 (OrphanSweep &
FileRefCheckerRegistry) are now all landed on top of 1a in this same
PR.** This is the complete Phase 1b runtime — reviewers see the full
read + write + watcher + orphan-sweep picture in one place.

Design, contracts, and decision rationale live in the architecture docs:

-
[`docs/references/file/architecture.md`](docs/references/file/architecture.md)
— module boundaries, type system, IPC/DataApi contracts, layered
architecture, service lifecycle, mutation propagation
-
[`docs/references/file/file-manager-architecture.md`](docs/references/file/file-manager-architecture.md)
— FileManager internals (storage, version detection, atomic writes,
reference cleanup, DirectoryWatcher, orphan sweep, DanglingCache state
machine, key design decisions)

#### Phase 1a deliverables

- Types (`FileEntry` / `FileInfo` / `FileHandle` /
`CanonicalExternalPath` brand)
- DB schema (`file_entry` + `file_ref`) with per-origin CHECK
constraints
- DataApi schemas + stub handlers
- File IPC contract (polymorphic `FileHandle` dispatch;
`batchGetMetadata` included)
- FileManager skeleton + `internal/*` + `ops/*` + `watcher/` +
`DanglingCache` + `versionCache`
- Mutation propagation design (three typed events + prefix-based
queryKey invalidation)

#### Phase 1b.1 deliverables (read-path runtime)

- Shared utilities: `getFileTypeByExt`, `sanitizeFilename`,
`validateFileName` extracted to `@shared/file/types`
- Path utilities: `canonicalizeExternalPath` (NFC + null-byte guard +
trailing-sep strip), `isPathInside`, `isUnderInternalStorage`,
`canWrite`
- FS read primitives: `stat`, `exists`, `read` (text/base64/binary
overloads), `hash` (initial MD5; swapped to xxhash-h64 in 1b.2)
- Metadata utilities: `getFileType(path)`, `isTextFile`, `mimeToExt`
- Repositories: `FileEntryService` + `FileRefService` read methods
(Drizzle-backed, Zod-branded outputs)
- Pure-function modules: `internal/content/{read,hash}`,
`internal/dispatch.ts` (FileHandle dispatcher)
- `toFileInfo(entry)` projection
- `FileManager` class as `BaseService` (`@Injectable('FileManager')`
`@ServicePhase(WhenReady)`); read methods only
- `DanglingCache` + `VersionCache` minimal viable singletons (full impls
in 1b.3 / 1b.2)
- DataApi `/files/*` read handlers fully implemented (entries / single /
ref-counts / refs-by-source)
- 60+ TDD tests (unit + boundary + setupTestDatabase integration)

#### Phase 1b.2 deliverables (write-path runtime)

- **FS atomic primitives** (open to non-file-module consumers per
architecture §5.3): `atomicWriteFile` (tmp + fsync + rename +
fsync(dir)), `atomicWriteIfUnchanged` (re-stat OCC + content-hash
fallback for second-precision mtime), `createAtomicWriteStream`
(Writable wrapper, abort/destroy unlinks tmp)
- **FS general primitives**: `write` (delegates to atomicWriteFile),
`copy` (atomic dest), `move` (rename + EXDEV → copy+unlink fallback),
`remove` (idempotent ENOENT), `mkdir` / `ensureDir` / `removeDir`,
`download` (fetch → atomic stream)
- **Hash swap**: `hash()` migrated MD5 → `xxhash-wasm` h64 streaming;
legacy `md5` dep retained for KnowledgeService loaders
- **VersionCache LRU**: capacity-bounded (default 2000) with
re-insert-on-touch recency
- **Repository mutations**: `FileEntryService.create/update/delete`
(auto UUIDv7 default id, raw DB CHECK errors propagate);
`FileRefService.create/createMany/cleanupBySource/cleanupBySourceBatch`
(`onConflictDoNothing` for batch upsert)
- **internal/entry/**: `create.createInternal` (4 source variants: bytes
/ base64 / path / url) + `create.ensureExternal` (canonicalize + stat +
idempotent upsert + duplicate-suspect peer warn);
`lifecycle.trash/restore/permanentDelete` + batch variants (DB+FS
decoupled, internal best-effort unlink); `rename` (internal DB-only,
external fs.move + canonical externalPath); `copy` (pipes through
createInternal with rollback)
- **internal/content/write.ts**: `write` / `writeIfUnchanged`
(cache-not-trusted re-stat OCC, `StaleVersionError` rewrap from
`PathStaleVersionError`) / `createWriteStream` / `*ByPath` variants
- **internal/system/**: `shell.open` / `shell.showInFolder` (electron
`shell` wrappers); `tempCopy.withTempCopy` (isolated tmp dir; cleanup on
throw)
- **FileManager facade**: every IFileManager mutation method now
delegates to its `internal/*` counterpart (`createInternalEntry` /
`ensureExternalEntry` / `batchCreate*` / `batchEnsure*` / `write` /
`writeIfUnchanged` / `createWriteStream` / `createReadStream` / `trash`
/ `restore` / `permanentDelete` + batch / `rename` / `copy` /
`withTempCopy` / `open` / `showInFolder`); no method throws
notImplemented anymore
- ~60 new TDD tests (each behavior unit = one RED→GREEN→REFACTOR
commit); end-to-end integration scenarios via `setupTestDatabase` cover
atomic-rollback zero-residue, OCC second-precision-mtime
no-false-positive, trash-external CHECK enforcement, full
create→write→read→trash→restore→permanentDelete round-trip, and external
permanentDelete-leaves-user-file-untouched

#### Phase 1b.3 deliverables (watcher + DanglingCache observability)

- **DanglingCache class** (replaces 1a const-literal skeleton):
`byEntryId: Map<entryId, CachedState>` + `pathToEntryIds:
Map<canonicalPath, Set<entryId>>` reverse index, lazy TTL expiration
(default 30 min per architecture §11.2), `forceRecheck` escape hatch,
`Emitter<DanglingStateChangedEvent>` firing only on genuine state
transitions (same-state observations are silent). Injectable `now` /
`statProbe` / `ttlMs` / `fileEntryService` seams for deterministic
tests.
- **createDirectoryWatcher** chokidar v4 wrapper: `add` / `unlink` /
`change` / `ready` / `error` events; built-in OS-junk basename ignores
(`.DS_Store` / `.localized` / `Thumbs.db` / `desktop.ini`); idempotent
`close()`. Factory auto-wires `add` →
`danglingCache.onFsEvent(path,'present')` and `unlink` → `'missing'`.
(Architecture §8.2's richer `onAddDir`/`onUnlinkDir`/`onRename` events
deferred — no consumer needs them in scope.)
- **Reverse-index maintenance from mutation flows**: `ensureExternal`
calls `addEntry` + `onFsEvent('present','ops')` on insert (no-op on
reuse); `permanentDelete(external)` calls `removeEntry`;
`rename(external)` swaps `removeEntry(oldPath) + addEntry(newPath) +
onFsEvent(newPath,'present','ops')`.
- **FileManager surface**: `getDanglingState({id})` (internal →
'present', external → cache check, unknown id → 'unknown');
`batchGetDanglingStates({ids})` (parallel fan-out, unknown ids mapped to
'unknown'); `subscribeDangling({id}, listener)` (in-process per-entry
filter; renderer fan-out via `file-manager-event` IPC channel deferred
to Phase 2).
- **FileManager.onInit**: awaits `danglingCache.initFromDb()` (populates
reverse index from non-trashed external entries; no startup stat probe
per architecture §10.6); registers `File_GetDanglingState` /
`File_BatchGetDanglingStates` IPC handlers via `this.ipcHandle`
(auto-disposed on stop).
- New `IpcChannel` constants: `File_GetDanglingState`,
`File_BatchGetDanglingStates`.
- ~30 new TDD tests across DanglingCache (18 unit) + watcher (6 real-FS)
+ FileManager integration (INT-7..INT-10).

#### Phase 1b.4 deliverables (orphan sweep + FileRefCheckerRegistry)

- **FileRefCheckerRegistry**: `Record<FileRefSourceType,
SourceTypeChecker<...>>` typed registry forces exhaustive coverage at
compile time — adding a new variant to `FileRefSourceType` without a
checker triggers a TS build error. Phase 1 ships `FileRefSourceType =
'temp_session' | 'knowledge_item'`: real DB-backed checker for
`knowledge_item` (Drizzle `inArray` against `knowledge_item`);
`temp_session` checker treats every sourceId as gone (sessions are
in-memory only). `chat_message` / `painting` / `note` are **deliberately
not in the union yet** — each will be added in lockstep (tuple entry in
`allSourceTypes` + `createRefSchema` variant + `SourceTypeChecker`) by
the PR that migrates the owning domain's DB tables to v2. Stray writes
during the migration window fail fast at `FileRefSchema.parse` rather
than being silently persisted under a no-op stub.
- **OrphanRefScanner** (RFC §6.4): `scanOneType(sourceType)` enumerates
distinct `file_ref.sourceId` per type, asks the checker which are alive,
deletes the rest via `cleanupBySourceBatch`. `scanAll()` aggregates
across every registered sourceType. Backed by new
`FileRefService.listDistinctSourceIds` to keep all SQL inside the repo.
- **Report-only orphan-entry pass** (architecture §7.1 default policy is
"preserve"): `scanOrphanEntries` groups active entries with zero
`file_ref` rows by origin. **No deletion** — surfaced via
`getOrphanReport()` for the cleanup-UI consumer. Backed by new
`FileEntryService.findUnreferenced` LEFT JOIN-based query.
- **Startup file sweep** (architecture §10): `runStartupFileSweep`
snapshots `file_entry.id` (active + trashed) into a `Set` via new
`FileEntryService.listAllIds`, walks `{userData}/files/`, plans unlink
for (a) UUID-named files whose id is not in the snapshot and (b)
`*.tmp-<UUID>` atomic-write residue. Applies the `mtime > 5min`
freshness gate (§10.3) — files newer than that are presumed in-flight
and preserved. Plan-then-execute with the `50% / 20-count-floor /
10MB-floor` safety threshold (§10.4); aborts emit
`abortReason='count-fraction'|'byte-fraction'`. Single structured
`orphan-file-sweep` log per run (info / warn / error per outcome,
§10.5).
- **DB-sweep umbrella + observability** (`runDbSweep`): runs scanAll +
scanOrphanEntries, emits one `orphan-sweep` structured record
summarising both passes; failure path returns `outcome='failed'` +
`errorMessage` so callers don't throw on background fire-and-forget.
- **FileManager integration**: `onInit` schedules a fire-and-forget
`runStartupSweeps` that runs the FS-level + DB-level sweeps in parallel;
failures of either are logged but never block ready. `getOrphanReport()`
exposes the most recent `DbSweepReport` (orphan-ref counts already
cleaned + orphan-entry counts preserved) + `lastRunAt` for the cleanup
UI surface.
- ~30 new TDD tests across registry (14 unit) + orphan sweep (16 unit +
integration) + FileManager integration (INT-11/INT-12) + repo
(`findUnreferenced`, `listAllIds`, `listDistinctSourceIds`).

**Out of scope (deferred to Phase 2)**:
- Architecture §7.2 dangling-external auto-cleanup (external + missing +
0-ref + >30d retention) — narrow extension shipping with the cleanup UI.
- Adding `chat_message` / `painting` / `note` as `FileRefSourceType`
variants (tuple entry + schema + checker added together) — gated on each
domain's v2 batch migration.
- Cleanup-UI surface that consumes `getOrphanReport()` — Phase 2
renderer work.

Renderer-side File IPC bridge for write/dangling methods stays deferred
to Phase 2 alongside the consumer-batch migrations. The Phase 1b runtime
is consumable from main-side business services through
`application.get('FileManager')`.

### Why we need it and why it was done in this way

Contract-first concentrates design review in one place; Phase 1b.x then
becomes pure "honor the contracts". Each 1b.x phase keeps strict TDD
(RED → GREEN → REFACTOR per behavior, ~one commit per cycle); each phase
ends with a verification gate (push → CI green) before the next phase
begins.

Core decisions (origin two-state, `FileEntry`/`FileInfo` split, DataApi
SQL-only, external `permanentDelete` DB-only, TTL `DanglingCache`, OCC
trust boundary, atomic write fsync default, etc.) and their rationale
are recorded in `file-manager-architecture.md §12 Key Design Decisions`
— not duplicated here.

### Breaking changes

None — purely additive (read+write paths are new, no existing callers
replaced yet).

### Special notes for your reviewer

- Review focus: contracts (1a) + read-path runtime (1b.1) + write-path
runtime (1b.2) + watcher/DanglingCache (1b.3) + orphan sweep / registry
(1b.4). Phase 1b is now complete on this branch.
- **Phase 1a contract stability policy** (architecture.md top) is
binding — any 1b.x PR that finds a contract mismatch PRs the doc
revision first.
- Deferred (Phase 2): renderer-side File IPC bridge for
write/dangling/orphan methods (alongside consumer migration); cleanup-UI
surface consuming `getOrphanReport()`; architecture §7.2
dangling-external auto-cleanup (>30d retention); adding `chat_message` /
`painting` / `note` as `FileRefSourceType` variants (each adds tuple
entry + schema + checker in lockstep, gated on the owning domain's v2
batch migration); DanglingCache periodic snapshot logger (architecture
§11.8); `listDirectory` ripgrep wrapper; `compressImage`
(KnowledgeService consumer); FileUploadService + `file_upload` table
(Vercel AI SDK Files API).

### Checklist

- [x] PR: The PR description is expressive enough and will help future
contributors
- [x] Code: Write code that humans can understand and Keep it simple
- [x] Refactor: You have left the code cleaner than you found it (Boy
Scout Rule)
- [ ] Upgrade: N/A — purely additive
- [ ] Documentation: Internal architecture docs included
(`docs/references/file/`); no user-facing docs change
- [x] Self-review: I have reviewed my own code before requesting review
from others

### Release note

```release-note
NONE
```

---------

Signed-off-by: icarus <eurfelux@gmail.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: fullex <106392080+0xfullex@users.noreply.github.com>
2026-05-13 16:06:10 +08:00

19 KiB

DataApi in Main Process

This guide covers how to implement API handlers and services in the Main process.

Architecture Layers

Handlers → Services → Database
  • Handlers: Thin layer, extract params, call service, transform response
  • Services: Business logic, validation, transaction coordination, data access via Drizzle ORM
  • Database: Drizzle ORM + SQLite

Transport Adapters

ApiServer is transport-agnostic. Adapters in api/core/adapters/ bridge specific transports (IPC, HTTP) to ApiServer. Each adapter implements Disposable for automatic lifecycle cleanup. See IpcAdapter.ts JSDoc for design rationale and extension guide.

Implementing Handlers

Location

src/main/data/api/handlers/

Handler Responsibilities

  • Extract parameters from request
  • Delegate to business service
  • Transform response for IPC
  • NO business logic here

Handler Type Annotation

Every per-module handler record MUST be annotated with HandlersFor<XxxSchemas>. This is the canonical shape for all files in src/main/data/api/handlers/ — not a convention to choose among alternatives.

HandlersFor<XxxSchemas> enforces two invariants:

  • Paths are narrowed to the module's own schema. Path strings outside XxxSchemas (typos, cross-module leaks) produce a compile error.
  • Methods are exhaustive. Every path + method declared in XxxSchemas must have a handler; adding an endpoint to the schema without a matching handler is a compile error.

Example Handler

// handlers/topics.ts
import { topicService } from '@data/services/TopicService'
import type { HandlersFor } from '@shared/data/api/apiTypes'
import type { TopicSchemas } from '@shared/data/api/schemas/topics'

export const topicHandlers: HandlersFor<TopicSchemas> = {
  '/topics': {
    GET: async ({ query }) => {
      const { page = 1, limit = 20 } = query ?? {}
      return await topicService.list({ page, limit })
    },
    POST: async ({ body }) => {
      return await topicService.create(body)
    }
  },
  '/topics/:id': {
    GET: async ({ params }) => {
      return await topicService.getById(params.id)
    },
    PUT: async ({ params, body }) => {
      return await topicService.replace(params.id, body)
    },
    PATCH: async ({ params, body }) => {
      return await topicService.update(params.id, body)
    },
    DELETE: async ({ params }) => {
      await topicService.delete(params.id)
    }
  }
}

Register Handlers

// handlers/index.ts
import { topicHandlers } from './topic'
import { messageHandlers } from './message'

export const allHandlers: ApiImplementation = {
  ...topicHandlers,
  ...messageHandlers
}

Implementing Services

Location

src/main/data/services/

Service Responsibilities

  • Business validation
  • Transaction coordination
  • Domain workflows
  • Data access via Drizzle ORM

Cross-Service Table Access

Each table has exactly one owning service — the rule is split by access kind:

  • Writes (insert / update / delete) to a table you do not own: forbidden. Call the owner's method (pass tx for transactional writes — owners' mutation methods accept Pick<DbType, 'delete' | 'insert' | ...> as the first arg). If a needed shape is missing, add a method on the owner; bulk needs get a bulk method (e.g. purgeForEntitiesTx).
  • Reads from a table you do not own: allowed when inlining is the simpler path. A cross-table JOIN that combines the owner's table into your query in one round-trip is fine; reach for the owner's read API only when the read needs business logic the owner already encapsulates.

Why writes are strict: the owning service is the single source of truth for the table's invariants (unique indices, orderKey semantics, soft-delete, audit timestamps) and emits its mutation logs. Foreign writes split that knowledge across every caller and silence the log narrative.

ProviderService.deletepinService.purgeForEntitiesTx(tx, 'model', ids) AssistantService.list JOINs entity_tag + tag inline to load tags per assistant ProviderService.deletetx.delete(pinTable).where(...) directly

If you're tempted to write "going through XxxService would be over-engineering" — stop. A 5-line method on the owner is not over-engineering; a foreign service writing to its table is.

Example Service

// services/TopicService.ts
import { eq, desc, sql } from 'drizzle-orm'
import { application } from '@application'
import { topicTable } from '@data/db/schemas/topic'
import { DataApiErrorFactory } from '@shared/data/api'

export class TopicService {
  private static instance: TopicService

  static getInstance(): TopicService {
    if (!this.instance) {
      this.instance = new TopicService()
    }
    return this.instance
  }

  private get db() {
    return application.get('DbService').getDb()
  }

  async list(options: { page: number; limit: number }) {
    const { page, limit } = options
    const offset = (page - 1) * limit

    const [items, countResult] = await Promise.all([
      this.db.select().from(topicTable)
        .orderBy(desc(topicTable.updatedAt))
        .limit(limit).offset(offset),
      this.db.select({ count: sql<number>`count(*)` }).from(topicTable)
    ])

    return { items, total: countResult[0].count, page, limit }
  }

  async getById(id: string) {
    const [topic] = await this.db.select().from(topicTable)
      .where(eq(topicTable.id, id)).limit(1)
    if (!topic) {
      throw DataApiErrorFactory.notFound('Topic', id)
    }
    return topic
  }

  async create(data: CreateTopicDto) {
    this.validateTopicData(data)
    const [topic] = await this.db.insert(topicTable).values(data).returning()
    return topic
  }

  async update(id: string, data: Partial<UpdateTopicDto>) {
    await this.getById(id) // Throws if not found
    const [topic] = await this.db.update(topicTable)
      .set(data).where(eq(topicTable.id, id)).returning()
    return topic
  }

  async delete(id: string) {
    await this.getById(id) // Throws if not found
    await this.db.delete(topicTable).where(eq(topicTable.id, id))
  }

  private validateTopicData(data: CreateTopicDto) {
    if (!data.name?.trim()) {
      throw DataApiErrorFactory.validation({ name: ['Name is required'] })
    }
  }
}

export const topicService = TopicService.getInstance()

Write-path defaults

service.create() passes a value into db.insert(...).values({...}) only for columns that are NOT NULL, have neither a DB DEFAULT nor a $defaultFn, and are not already supplied by the DTO:

async create(dto: CreateXxxDto) {
  return await this.db.insert(xxxTable).values({
    ...dto,
    settings: dto.settings ?? DEFAULT_XXX_SETTINGS  // service-owned default for a tunable product value
  }).returning()
}

For everything else — fields with DB DEFAULTs, $defaultFn columns, or genuinely nullable columns — omit the field from values({...}). Drizzle leaves it out of the SQL; the DB applies its own default (or NULL for nullable columns). Restating the DB's knowledge in app code creates drift risk when defaults later change.

For the cross-layer placement decision tree, see Default Values & Nullability.

Row → Entity Mapping

Each Entity Service provides a rowToEntity function that bridges a Drizzle row to its domain entity. Use nullsToUndefined (from services/utils/rowMappers.ts) for the SQLite NULL → TypeScript undefined translation.

Standard skeleton:

import { nullsToUndefined, timestampToISO } from './utils/rowMappers'

function rowToMCPServer(row: typeof mcpServerTable.$inferSelect): MCPServer {
  const clean = nullsToUndefined(row)
  return {
    ...clean,
    type: clean.type as MCPServer['type'], // narrow enum
    installSource: clean.installSource as MCPServer['installSource'],
    createdAt: timestampToISO(row.createdAt),
    updatedAt: timestampToISO(row.updatedAt)
  }
}

Audit columns generated by createUpdateTimestamps are DB-level .notNull(), so row.createdAt / row.updatedAt narrow to number and timestampToISO is the default. timestampToISOOrUndefined is reserved for construction paths where the entire source row may be absent (e.g. MiniAppService.builtinToMiniApp merging a builtin definition with an optional dbRow).

Advanced skeleton — preserving T | null fields:

When the domain type declares a field as T | null (e.g. KnowledgeBaseSchema.embeddingModelId: z.string().nullable()), bypass clean for that field and reference row directly. nullsToUndefined narrows all top-level nulls to undefined and would break the T | null contract if the field came from clean.

function rowToKnowledgeBase(row: typeof knowledgeBaseTable.$inferSelect): KnowledgeBase {
  const clean = nullsToUndefined(row)
  return {
    ...clean,
    // Preserve `string | null` contract — bypass clean (which would narrow null → undefined)
    embeddingModelId: row.embeddingModelId,
    createdAt: timestampToISO(row.createdAt),
    updatedAt: timestampToISO(row.updatedAt)
  }
}

Rule of thumb: domain field typed T | null → use row.x; domain field typed T? or T → use clean.x (or ...clean).

When nullsToUndefined + spread is NOT a fit:

Some rowToEntity functions do too much to benefit from spread. Keep them hand-written when any of the following apply:

  • Field renaming: row.parameters → domain parameterSupport (ModelService)
  • Computed / merged fields: authType derivation, apiFeatures merging from defaults (ProviderService)
  • Sensitive data sanitization: apiKeys stripping — ...clean would leak unsanitized values
  • Discriminator-driven field stripping with brand validation: branded discriminated union where each variant declares only its own fields — nullsToUndefined + spread would emit absent fields as undefined and break the BO shape. Dispatch on the discriminator and call schema.parse per variant. Example: FileEntryService.rowToFileEntry for FileEntry (variants on origin); see packages/shared/data/types/file/fileEntry.ts header (§"DB row vs Business Object") for the full DB-CHECK / BO-narrow rationale.

Anti-pattern — ?? fallbacks for fabricated defaults:

row.x ?? '🌟' / row.x ?? [] inside rowToEntity is forbidden. The presence of such a fallback is reverse evidence that the column should be NOT NULL with a DB DEFAULT or $defaultFn — see Default Values & Nullability § R3. The legitimate exception is when the entity field is genuinely T | null (e.g. assistant.modelId); then bypass clean and reference row.x directly to preserve the NULL contract — that is the Advanced skeleton above, not a ?? fallback.

Conventions:

  1. DB NULL ↔ domain undefined boundary. Domain types under @shared/data/types/* use optional fields (?:) rather than T | null, aligning with the Google TypeScript Style Guide and keeping null from leaking to the renderer via IPC. nullsToUndefined(row) is the only place this translation happens.
  2. Batch vs single-field null handling. For processing an entire row, always use nullsToUndefined(row) + spread — do NOT hand-write per-field ?? undefined. For single values that are NOT from a row (DTO fields, computed values, function returns), inline value ?? undefined is enough — TypeScript narrows T | null to T | undefined automatically at the ?? expression. Do NOT wrap the single-field case in a helper.
  3. Date fields: two helpers, clear boundary. timestampToISO(value: number | Date): string is the default for rowToEntity — audit columns from createUpdateTimestamps are .notNull(), so the DB row hands back a real number. timestampToISOOrUndefined(value: number | Date | null | undefined): string | undefined is reserved for merge paths where the source row itself may be absent (e.g. a builtin/preset definition without a preference row). Do NOT use timestampToISOOrUndefined as a "safer default" — if your input is a DB row, it always has these fields.

For function signature details and design-decision history (e.g. why shallow-not-recursive, why not dnull), see services/utils/README.md.

Service with Transaction

async createTopicWithMessage(data: CreateTopicWithMessageDto) {
  const db = application.get('DbService').getDb()

  return await db.transaction(async (tx) => {
    const [topic] = await tx.insert(topicTable).values(data.topic).returning()

    const [message] = await tx.insert(messageTable).values({
      ...data.message,
      topicId: topic.id
    }).returning()

    return { topic, message }
  })
}

Transaction Method Naming

Service methods accepting a Drizzle transaction:

Rule
Parameter position tx is the first parameter
Method name ends with Tx
Parameter type Pick<DbType, '...'> with the minimum operations needed
Non-Tx wrapper optional; thin db.transaction(...) wrapper, only when a caller needs to own the transaction
// ✅
async purgeForEntityTx(tx: Pick<DbType, 'delete'>, entityType: EntityType, entityId: string): Promise<void>

// ❌ tx not first
async purgeForEntity(entityType: EntityType, entityId: string, tx: Pick<DbType, 'delete'>)
// ❌ missing Tx suffix
async purgeForEntity(tx: Pick<DbType, 'delete'>, entityType: EntityType, entityId: string)
// ❌ over-broad type
async purgeForEntityTx(tx: DbType, entityType: EntityType, entityId: string)

Optional non-Tx wrapper:

async purgeForEntity(entityType: EntityType, entityId: string): Promise<void> {
  await this.db.transaction((tx) => this.purgeForEntityTx(tx, entityType, entityId))
}

Repository Pattern (Strongly Discouraged)

⚠️ Do NOT create Repository files by default. Services handle both business logic and data access directly via Drizzle ORM. This is an intentional design decision.

Only create a separate Repository when you are 1000% certain it is absolutely necessary — e.g., extremely complex multi-table queries with joins/CTEs that would make the Service unreadable, AND the query logic is reused across multiple services.

If in doubt, keep it in the Service. The overhead of an extra architectural layer is not justified for this project's scale (Electron desktop app + SQLite).

Registry Services (Supplementary)

In rare cases where a handler needs to merge read-only preset data (shipped JSON/TS) with database data, a Registry Service may be introduced. This is uncommon — the vast majority of services are Entity Services.

Registry Services:

  • Do NOT own a database table and do NOT access the database directly
  • Obtain DB data by calling the owning Entity Service
  • Named {Domain}RegistryService (e.g., ProviderRegistryService)
  • Primary data source is static preset data (JSON files, TS constants)
  • All methods are read-only (no inserts, updates, or deletes)

See Layered Preset Pattern for the general architecture.

Registry Sub-Resource Endpoints

Registry data reaches the renderer through sub-resource endpoints on the owning entity. Three rules govern their shape.

GET only. Registry endpoints are stateless reads — preset merged with DB rows. POST is reserved for state changes; using it for reads breaks SWR caching, request dedup, and retry safety. For composite IDs containing /, use the greedy path form :id* (see Greedy Path Parameters). For batched lookups exceeding URL limits, split into multiple GETs — DataApi dedup makes burst reads cheap.

Colon-notation for derived views. When the sub-resource name is ambiguous, disambiguate with AIP-136 colon notation:

Shape Use for
GET /:parent/:id/:sub List the merged collection
GET /:parent/:id/:sub:action Compute a derived view
GET /:parent/:id/:sub/:childId Look up one merged item

Registry packages are main-only. Packages like @cherrystudio/provider-registry ship the preset data Registry Services merge against. Renderer code must not import them. Two reasons:

  • Bundle waste. Registry packages are large (preset catalogs, vendor metadata, icons). Importing them in the renderer ships the same payload twice — once in the main bundle, once in every renderer entry that touches it — for data the renderer already gets via DataApi.
  • Merge already lives in main. Registry Services merge preset + DB rows on the main side. Re-doing the merge in the renderer duplicates logic and re-introduces preset-version drift this layer was designed to remove.

The merged result reaches the renderer exclusively through these endpoints.

Error Handling

Using DataApiErrorFactory

import { DataApiErrorFactory } from '@shared/data/api'

// Not found
throw DataApiErrorFactory.notFound('Topic', id)

// Validation error
throw DataApiErrorFactory.validation({
  name: ['Name is required', 'Name must be at least 3 characters'],
  email: ['Invalid email format']
})

// Database error
try {
  await db.insert(table).values(data)
} catch (error) {
  throw DataApiErrorFactory.database(error, 'insert topic')
}

// Invalid operation
throw DataApiErrorFactory.invalidOperation(
  'delete root message',
  'cascade=true required'
)

// Conflict
throw DataApiErrorFactory.conflict('Topic name already exists')

// Timeout
throw DataApiErrorFactory.timeout('fetch topics', 3000)

Adding New Endpoints

Step-by-Step

  1. Define schema in packages/shared/data/api/schemas/
// schemas/topic.ts
export type TopicSchemas = {
  '/topics': {
    GET: { response: PaginatedResponse<Topic> }
    POST: { body: CreateTopicDto; response: Topic }
  }
}
  1. Register schema in schemas/index.ts
export type ApiSchemas = AssertValidSchemas<TopicSchemas & MessageSchemas>
  1. Create service in services/

  2. Implement handler in handlers/

  3. Register handler in handlers/index.ts

Best Practices

  1. Keep handlers thin: Only extract params and call services
  2. Put logic in services: All business rules and data access belong in services
  3. Do NOT create separate Repository files: Services own data access directly via Drizzle ORM
  4. Always use .returning(): Get inserted/updated data without re-querying
  5. Support transactions: Accept optional tx parameter in service methods
  6. Validate in services: Business validation belongs in the service layer
  7. Use error factory: Consistent error creation with DataApiErrorFactory
  8. Use nullsToUndefined in rowToEntity: Canonical SQLite NULL → undefined translation; shallow, not recursive (see Row → Entity Mapping)