CherryHQ-cherry-studio/docs/references/data/v2-migration-guide.md

Migration V2 (Main Process)

Architecture for the new one-shot migration from the legacy Dexie + Redux Persist stores into the SQLite schema. This module owns orchestration, data access helpers, migrator plugins, and IPC entry points used by the renderer migration window.

Version Upgrade Requirements

The v2 migration system enforces a linear upgrade path to ensure data integrity:

v1.old  →  v1.last (≥1.9.0)  →  v2.0.0  →  v2.x

Why a linear path?

v2.0.0 contains the one-shot data migration from Redux/Dexie to SQLite. Supporting migration from every v1 version would create an O(n²) test matrix. By requiring all users to be on the final v1 release first, the migration code only needs to handle a single source data format.

How it works

1. VersionService has been embedded since v1.7. It writes a version.log file to {userData}/ on every launch where the version changes.
2. On v2 first launch, v2MigrationGate.ts reads version.log via MigrationPaths.versionLogFile (using the resolved userData path that accounts for v1 custom directories).
3. If the previous version is too old, missing, or if the user skipped v2.0.0, the gate shows an error dialog and quits.

Blocking rules

Scenario	Block reason	User action
No version.log (v1 < 1.7 user)	no_version_log	Install v1.last, run once, then install v2.0.0
Previous version < 1.9.0	v1_too_old	Upgrade to v1.last first
Previous version is v1.x but current > v2.0.0	v2_gateway_skipped	Install v2.0.0 first

Pre-release versions

v2.0.0 pre-releases (alpha/beta/rc) are treated as before v2.0.0 per semver ordering. They are allowed as migration targets from v1.last (the gateway check coerces currentVersion, so 2.0.0-alpha → 2.0.0 passes). Pre-release to pre-release upgrades work because migration status is completed after the first successful run.

The gateway is strictly v2.0.0 — v2.0.x patches are blocked from being a first migration target. This may be relaxed in a future release.

Relationship with the auto-updater

The auto-updater (AppUpdaterService) controls which versions are offered via OTA using minCompatibleVersion in the remote config. The migration gate is a separate safety net for users who manually download and install a version. Both systems enforce compatible upgrade paths but operate independently.

Directory Layout

src/main/data/migration/v2/
├── core/              # Engine + shared context
├── migrators/         # Domain-specific migrators and mappings
├── utils/             # Data source readers (Redux, Dexie, streaming JSON)
├── window/            # IPC handlers + migration window manager
└── index.ts           # Public exports for main process

Core Contracts

• core/MigrationEngine.ts coordinates all migrators in order, surfaces progress to the UI, and marks status in app_state.key = 'migration_v2_status'. It will clear new-schema tables before running and abort on any validation failure.
• core/MigrationPaths.ts defines MigrationPaths (a frozen object of pre-computed paths) and resolveMigrationPaths() which detects v1 legacy userData directories from ~/.cherrystudio/config/config.json. Called once at the migration gate entry, before engine initialization. All migration code uses these paths instead of app.getPath() — see the Path safety convention below.
• core/MigrationContext.ts builds the shared context passed to every migrator:
  • sources: ConfigManager (ElectronStore), ReduxStateReader (parsed Redux Persist data), DexieFileReader (JSON exports), LegacyHomeConfigReader (v1 ~/.cherrystudio/config/config.json for the config-file migration path used by BootConfigMigrator)
  • db: current SQLite connection
  • paths: MigrationPaths — pre-computed filesystem paths; migrators that need file paths use ctx.paths instead of app.getPath()
  • sharedData: Map for passing cross-cutting info between migrators
  • logger: loggerService scoped to migration
• @shared/data/migration/v2/types defines stages, results, and validation stats used across main and renderer.

Migrators

• Base contract: extend migrators/BaseMigrator.ts and implement:
  • id, name, description, order (lower runs first)
  • prepare(ctx): dry-run checks, counts, and staging data; return PrepareResult
  • execute(ctx): perform inserts/updates; manage your own transactions; report progress via reportProgress; self-check FK integrity of owned tables via assertOwnedForeignKeys (see Conventions → Foreign keys)
  • validate(ctx): verify counts and integrity; return ValidateResult with stats (sourceCount, targetCount, skippedCount) and any errors
• Registration: list migrators (in order) in migrators/index.ts so the engine can sort and run them.
• Current migrators (see migrators/README-<name>.md for detailed documentation):
  • PreferencesMigrator (implemented): maps ElectronStore + Redux settings to the preference table using mappings/PreferencesMappings.ts.
  • ChatMigrator (implemented): migrates topics and messages from Dexie to SQLite. See README-ChatMigrator.md.
  • BootConfigMigrator (implemented, file-target): migrates early-boot settings into the file-based bootConfigService (~/.cherrystudio/boot-config.json) rather than a SQLite table. Reads from Redux (disableHardwareAcceleration) and from the v1 home config file (~/.cherrystudio/config/config.json's appDataPath → app.user_data_path) via a 'configfile' source kind. See README-BootConfigMigrator.md.
  • AssistantMigrator, KnowledgeMigrator (placeholders): scaffolding and TODO notes for future tables.
• Conventions:
  • All logging goes through loggerService with a migrator-specific context.
  • Use MigrationContext.sources instead of accessing raw files/stores directly.
  • Use sharedData to pass IDs or lookup tables between migrators (e.g., assistant -> chat references) instead of re-reading sources.
  • Stream large Dexie exports (JsonStreamReader) and batch inserts to avoid memory spikes.
  • Foreign keys are OFF for the whole migration — do NOT toggle them per-migrator: better-sqlite3 keeps a single persistent connection open for the whole process, so the engine sets PRAGMA foreign_keys = OFF once on that connection (in MigrationDbService) and it stays in effect for the entire migration — there is no per-transaction reconnection that could reset it. This lets bulk inserts carry not-yet-resolved references (self-referencing message.parentId, or cross-domain refs a later migrator resolves). Integrity is verified in two layers: (1) each migrator calls this.assertOwnedForeignKeys(ctx.db, [...]) at the end of execute() for the tables it owns, giving early, well-attributed failures; (2) the engine runs a whole-database PRAGMA foreign_key_check after all migrators complete (MigrationEngine.verifyForeignKeys) as the final backstop.
    • Self-check scope: pass only tables whose FKs are fully resolved when your migrator finishes. Exclude refs a later migrator resolves — e.g. assistant_knowledge_base.knowledgeBaseId is written by AssistantMigrator but only becomes valid after KnowledgeMigrator remaps/prunes it, so KnowledgeMigrator self-checks that table, not AssistantMigrator. Dedicated file association tables (for example chat_message_file_ref) may be self-checked by the migrator that owns both the source rows and ref rows.
  • Count validation is mandatory; engine will fail the run if targetCount < sourceCount - skippedCount or if ValidateResult.errors is non-empty.
  • Keep migrations idempotent per run—engine clears target tables before it starts, but each migrator should tolerate retries within the same run.
  • Path safety: All filesystem paths MUST come from ctx.paths (the MigrationPaths object). NEVER call app.getPath('userData') or construct paths with path.join from scratch. Doing so bypasses the v1 legacy userData detection and may cause data loss for users with custom appDataPath configurations. If you need a path not yet in MigrationPaths, add it to the interface — do not inline it.

Utilities

• utils/ReduxStateReader.ts: safe accessor for categorized Redux Persist data with dot-path lookup.
• utils/DexieFileReader.ts: reads exported Dexie JSON tables; can stream large tables.
• utils/JsonStreamReader.ts: streaming reader with batching, counting, and sampling helpers for very large arrays.
• utils/LegacyHomeConfigReader.ts: synchronously reads the v1 ~/.cherrystudio/config/config.json file and normalizes its appDataPath field (both the legacy string shape and the current { executablePath, dataPath }[] shape) into a Record<executablePath, dataPath> | null. Used exclusively by BootConfigMigrator's 'configfile' source.

Window & IPC Integration

• window/MigrationIpcHandler.ts exposes IPC channels for the migration UI:
  • Receives Redux data and Dexie export path, starts the engine, and streams progress back to renderer.
  • Manages backup flow (dialogs via BackupManager) and retry/cancel/restart actions.
• window/MigrationWindowManager.ts creates the frameless migration window, handles lifecycle, and relaunch instructions after completion in production.

Implementation Checklist for New Migrators

• Add mapping definitions (if needed) under migrators/mappings/.
• Implement prepare/execute/validate with explicit counts, batch inserts, and integrity checks.
• Wire progress updates through reportProgress so UI shows per-migrator progress.
• Register the migrator in migrators/index.ts with the correct order.
• Add any new target tables to MigrationEngine.verifyAndClearNewTables once those tables exist.
• Self-check FK integrity at the end of execute() via this.assertOwnedForeignKeys(ctx.db, [...ownedTables]), excluding cross-domain-deferred refs and shared polymorphic tables (see Conventions → Foreign keys). Do NOT toggle PRAGMA foreign_keys yourself — the engine keeps it OFF for the whole migration.
• Include detailed comments for maintainability (file-level, function-level, logic blocks).
• Create/update migrators/README-<MigratorName>.md with detailed documentation including:
  • Data sources and target tables
  • Key transformations
  • Field mappings (source → target)
  • Dropped fields and rationale
  • Code quality notes

Order-Key Stamping in Migrators

Legacy Redux/Dexie → SQLite migrators for sortable resources must produce order_key values for every row they insert. The v2 migrator layer owns a pair of pure functions under src/main/data/migration/v2/utils/orderKey.ts that handle this without touching the DB — they take a pre-flattened array and return the same rows with orderKey attached.

Helper	Shape	Use for
assignOrderKeysInSequence(rows)	Returns rows with one monotonically increasing orderKey per row.	Whole-table ordering (e.g. mcp_server, user_provider, miniapp).
assignOrderKeysByScope(rows, getScope)	Groups rows by the scope key, stamps each bucket independently (independent key spaces per bucket).	Partitioned tables (e.g. topic.groupId, user_model.providerId, group.entityType).

Pattern — flatten first, stamp last: keep transform* functions pure (no index parameter, no sortOrder argument); flatten the legacy source into an array, then stamp keys onto the whole array:

import { assignOrderKeysByScope, assignOrderKeysInSequence } from '@data/migration/v2/utils/orderKey'

// Before — each transform took an index and emitted a sortOrder
const rows = legacyServers.map((src, i) => transformMcpServer(src, i).row)

// After — transforms are pure; keys are assigned after the flatten
const rows = legacyServers.map((src) => transformMcpServerV2(src).row)
const stamped = assignOrderKeysInSequence(rows)
await tx.insert(mcpServerTable).values(stamped)

// Partitioned example — each providerId becomes its own independent key space
const stamped = assignOrderKeysByScope(userModels, (m) => m.providerId)

Import rule — never reach for fractional-indexing directly: the migrator helpers delegate to generateOrderKeySequence exported from src/main/data/services/utils/orderKey.ts, which is the single sanctioned integration point for the library. Migrator code, migration scripts, and drizzle custom-migration callbacks all re-import from that service-layer wrapper. This keeps the library boundary auditable and leaves a single place to change the character set or swap implementations.

For the runtime counterparts (insertWithOrderKey / insertManyWithOrderKey / applyMoves / resetOrder) used outside the migration window, see Reorder Guide — Server-Side Service Helpers.