CherryHQ-cherry-studio/docs/references/data/data-ordering-guide.md

Ordering Guide

Canonical spec for any sortable resource in the DataApi system. Uses a single fractional-indexing design (fractional-indexing, Rocicorp, ~2 KB gzip) — PATCH /{resource}/:id/order with an anchor body. Scales from tens to thousands of rows without background rebalancing; applies uniformly whether the view is paginated or not. Replaces the two incompatible predecessors (PATCH /miniapps absolute sortOrder integers and PATCH /mcp-servers full orderedIds list).

Every sortable resource stores its position as a string order_key column. A reorder is always relative against an anchor (another row's id, or a first / last sentinel), never an absolute index. The server computes a new key between neighbours in one transaction; the renderer optimistically reorders its local cache and revalidates on completion.

Quickstart — The Four Layers

A sortable resource touches four layers. The toolkit provides one import per layer — this section shows the end-to-end picture at a glance before the specification dives into specifics.

1. Database schema — orderKeyColumns + index helpers

File: src/main/data/db/schemas/_columnHelpers.ts. Spread ...orderKeyColumns into the table definition (the field name is locked to orderKey at the type level) and attach the right index helper.

import { sqliteTable, text } from 'drizzle-orm/sqlite-core'
import { orderKeyColumns, orderKeyIndex, scopedOrderKeyIndex } from './_columnHelpers'

// Whole-table ordering
export const mcpServer = sqliteTable(
  'mcp_server',
  {
    id: text().primaryKey(),
    ...orderKeyColumns,                     // TEXT NOT NULL `order_key`
  },
  (t) => [orderKeyIndex('mcp_server')(t)]   // CREATE INDEX mcp_server_order_key_idx
)

// Partitioned ordering (scope = providerId)
export const userModel = sqliteTable(
  'user_model',
  {
    id: text().primaryKey(),
    providerId: text('provider_id').notNull(),
    ...orderKeyColumns,
  },
  (t) => [scopedOrderKeyIndex('user_model', 'providerId')(t)]  // (provider_id, order_key)
)

2. API schema — OrderEndpoints<TRes>

File: packages/shared/data/api/schemas/_endpointHelpers.ts. Intersect the resource's schema type with OrderEndpoints<'/res'> to inject the two PATCH endpoints into ApiSchemas.

import type { OrderEndpoints } from './_endpointHelpers'

export type McpServerSchemas = {
  '/mcp-servers': { GET: { ... }; POST: { ... } }
  '/mcp-servers/:id': { GET: { ... }; PATCH: { ... }; DELETE: { ... } }
} & OrderEndpoints<'/mcp-servers'>
// Adds '/mcp-servers/:id/order' (PATCH) and '/mcp-servers/order:batch' (PATCH)
// with correct params / body (OrderRequest, OrderBatchRequest) / response.

Handlers validate with the matching Zod schemas (OrderRequestSchema, OrderBatchRequestSchema) exported from the same file.

3. Server-side service — insertWithOrderKey / applyMoves / resetOrder

File: src/main/data/services/utils/orderKey.ts. Use these helpers on any POST create path, reorder path, and /order:reset path — never hand-write tx.insert(table).values({ orderKey }) or touch fractional-indexing directly.

import { insertWithOrderKey, insertManyWithOrderKey, applyMoves, resetOrder } from './utils/orderKey'

await insertWithOrderKey(tx, mcpServerTable, values, { pkColumn: mcpServerTable.id })
await insertManyWithOrderKey(tx, mcpServerTable, valuesList, { pkColumn: mcpServerTable.id })
await applyMoves(tx, mcpServerTable, moves, { pkColumn: mcpServerTable.id })
await resetOrder(tx, mcpServerTable, orderedRows, { pkColumn: mcpServerTable.id })

// Scoped variants pass a `scope` SQL expression, e.g.:
//   scope: eq(userModelTable.providerId, providerId)

Migrators (Redux/Dexie → SQLite) use the pure-function counterparts assignOrderKeysInSequence / assignOrderKeysByScope from src/main/data/migration/v2/utils/orderKey.ts — see v2 Migration Guide — Order-Key Stamping in Migrators.

4. Renderer — useReorder hook

File: src/renderer/src/data/hooks/useReorder.ts. One hook on top of useMutation; drop its applyReorderedList straight into a drag-and-drop callback.

import { useQuery } from '@data/hooks/useDataApi'
import { useReorder } from '@data/hooks/useReorder'

function McpServerList() {
  const { data } = useQuery('/mcp-servers')
  const { applyReorderedList, isPending } = useReorder('/mcp-servers')
  return <DraggableList items={data?.items ?? []} onReorder={applyReorderedList} />
}

// Non-`id` primary key (e.g. miniapp.appId):
useReorder('/mini-apps', { idKey: 'appId' })

Optimistic writes / server revalidation / failure rollback are all handled internally through the DataApi cache hooks (useReadCache / useWriteCache / useInvalidateCache) — the component never tracks the list in local state and never calls SWR directly.

---

The sections below spell out the normative spec for each layer.

1. API Shape

PATCH /{resource}/:id/order — primary

Request body — exactly one of:

{ "before":   "row-abc" }            // move :id before row-abc
{ "after":    "row-xyz" }            // move :id after row-xyz
{ "position": "first" }              // (or "last") move :id to the head/tail

Response: 204 No Content.

Code	HTTP	When
VALIDATION_ERROR	422	Body does not match the three-way union
NOT_FOUND	404	:id or before/after anchor id does not exist

POST /{resource}/order:reset — auxiliary (opt-in per resource)

{ "preset": "alphabetical" }

Each resource declares its own preset enum inline. Server sorts the rows per preset, then rewrites every order_key in a single transaction. Not funneled through useReorder — call with a plain useMutation:

useMutation('POST', '/providers/order:reset', { refresh: ['/providers'] })

PATCH /{resource}/order:batch — auxiliary (used internally by useReorder)

{
  "moves": [
    { "id": "a", "anchor": { "after": "b" } },
    { "id": "c", "anchor": { "position": "first" } }
  ]
}

Moves apply sequentially in one transaction; each anchor resolves against the state produced by preceding moves. Duplicate ids fold (last wins, warn logged); no-op moves (newKey === currentKey) are skipped. useReorder.applyReorderedList auto-dispatches: zero changes → no-op, one change → primary endpoint, two+ → this endpoint. Same error codes as the primary.

---

2. Database Schema Rules

(Code examples are in Quickstart §1.)

• Column: order_key TEXT NOT NULL. Always injected via ...orderKeyColumns; the spread locks the TS field name to orderKey.
• Index: required. Use orderKeyIndex(tableName)(t) for whole-table or scopedOrderKeyIndex(tableName, scopeColumn)(t) for partitioned tables.
• Known partition dimensions in the codebase: topic.groupId, group.entityType, user_model.providerId, miniapp.status, user_provider.isEnabled.
• No secondary order axes. Secondary semantics like topic pinned are modelled as scope values (e.g. a reserved groupId = '__pinned__' bucket), not a second column. One axis, one API.

---

3. Server-Side Service Helpers

All runtime order_key reads and writes go through src/main/data/services/utils/orderKey.ts — the single place in the codebase allowed to import fractional-indexing.

Helper	Use for
insertWithOrderKey(tx, table, values, { pkColumn, position?, scope? })	Single-row POST create on a sortable table.
insertManyWithOrderKey(tx, table, valuesList, { pkColumn, position?, scope? })	Batch/seed create (≥2 rows). One boundary lookup + one bulk insert; insertWithOrderKey delegates to it internally.
applyMoves(tx, table, moves, { pkColumn, scope? })	Reorder path for both PATCH /:id/order (wrapped as a single move) and PATCH /order:batch.
resetOrder(tx, table, orderedRows, { pkColumn })	POST /order:reset — caller sorts by preset, helper rewrites every key.
computeNewOrderKey(...)	Internal to applyMoves; exported only for tests.
generateOrderKeySequence / generateOrderKeyBetween / generateOrderKeySequenceBetween	The ONLY sanctioned wrappers around fractional-indexing. Services, migrators, and custom-migration scripts all re-import from here.

Binding semantics:

• pkColumn is required. Primary-key column names vary (miniapp.appId, mcpServer.id, topic.id, ...); helpers make zero assumptions.
• Must run inside an outer transaction. Helpers take tx and never open their own.
• scope applies symmetrically to target, anchor, and neighbour lookups — anchoring across scopes throws.
• insertManyWithOrderKey preserves input order under ORDER BY orderKey ASC. For position: 'last' the batch lands after existing rows; for 'first' before; within the batch, relative order mirrors valuesList.

Scoped usage:

// Topic: groupId is nullable, both NULL and non-NULL are real partitions
await insertWithOrderKey(tx, topicTable, values, {
  pkColumn: topicTable.id,
  scope: values.groupId ? eq(topicTable.groupId, values.groupId) : isNull(topicTable.groupId),
})

// user_model: scope by providerId
await applyMoves(tx, userModelTable, moves, {
  pkColumn: userModelTable.id,
  scope: eq(userModelTable.providerId, providerId),
})

---

4. Renderer Integration

4.1 Sequence

User  Component          SWR cache         Main           SQLite
 |        |                  |               |              |
 | drop→  | applyReorderedList                              |
 |        |--- useWriteCache(url, next) ---> overwrite      |
 | [UI instantly updates from optimistic value]             |
 |        |--- PATCH /:res/:id/order --> IPC --> UPDATE --> |
 |        |<---------- 204 -----------------                |
 |        | useMutation.refresh → auto GET → overwrite      |
 | [UI settles with server truth]                           |
 | on error: useInvalidateCache(url) → GET → overwrite      |

Three observable steps: optimistic write → PATCH → revalidate (or invalidate on error). The optimistic value is derived by a pure reorderLocally(items, id, anchor, idKey) from the current cache — the hook never constructs order_key client-side. All cache side-effects go through the DataApi hooks; useReorder holds zero direct SWR imports.

4.2 Non-id primary keys — the idKey option

useReorder('/mini-apps', { idKey: 'appId' })

Flows into both the optimistic reducer and the new-list diff. The server-facing contract is unchanged — move(id, anchor) still takes a plain string id, PATCH body shape is untouched. idKey only affects how the client extracts ids from cached items.

Single field only — composite keys like ${providerId}:${modelName} are out of scope; pre-project a synthetic id field before passing items to the drag library.

4.3 Anti-pattern: don't shadow SWR with local state

// WRONG — fights SWR cache, flickers, goes stale
const [list, setList] = useState<Item[]>([])
useEffect(() => { setList(data?.items ?? []) }, [data])

// RIGHT — render straight from SWR; optimistic updates go to the cache
const { data } = useQuery('/mcp-servers')
<DraggableList items={data?.items ?? []} onReorder={applyReorderedList} />

The cached list is the source of truth. Every subscriber of the same key stays in sync automatically.

---

5. v2 Migrator Usage

Pure-function helpers assignOrderKeysInSequence / assignOrderKeysByScope stamp pre-flattened arrays; no DB access; delegate to generateOrderKeySequence from the service-layer wrapper.

→ See v2 Migration Guide — Order-Key Stamping in Migrators.

---

6. URL and Naming Conventions

• Sub-resource name: /order. Never /sort, /rank, /position.
• Custom methods: colon notation — /{resource}/order:reset, /{resource}/order:batch (Google AIP-136; see API Design Guidelines — Non-CRUD Operations).
• Body enum: position: 'first' | 'last' — distinct from the /order URL segment.
• DB column: order_key (SQL) / orderKey (TS), always TEXT NOT NULL. No nullable variants.
• Type names: every order-related export prefixed with Order (OrderRequest, OrderRequestSchema, OrderBatchRequest, OrderBatchRequestSchema, OrderEndpoints). No Sort* / Position* / Rank* aliases — the Order prefix is what keeps _endpointHelpers.ts classifiable as it grows.

Disallowed: POST /{res}:reorder, POST /{res}/reorder, PUT /{res}/order (rejected full-list design), collection-level PATCH /{res} for reordering, nested URLs like /groups/:gid/topics/:id/order (group dimension travels in the body, not the path).

---

7. Migration Checklist — New Sortable Resource

Complete in one PR:

1. Schema: ...orderKeyColumns + orderKeyIndex(tableName)(t) or scopedOrderKeyIndex(tableName, scopeColumn)(t).
2. Endpoints: & OrderEndpoints<'/{res}'> on the resource's schema type. Add POST /{res}/order:reset inline if needed. Handlers validate bodies with OrderRequestSchema / OrderBatchRequestSchema.
3. Service: insertWithOrderKey for create, applyMoves for reorder, resetOrder for reset. For partitioned tables, pass the right scope:
   • topic: topic.groupId ? eq(topicTable.groupId, groupId) : isNull(topicTable.groupId)
   • group: eq(groupTable.entityType, entityType)
   • user_model: eq(userModelTable.providerId, providerId)
   • miniapp: eq(miniappTable.status, status)
   • user_provider / mcp_server: whole-table (scope: undefined)
4. Migrator: replace legacy sortOrder = index with assignOrderKeysByScope (or assignOrderKeysInSequence for whole-table). Drop index / sortOrder parameters from transform* functions.
5. Renderer: useReorder(collectionUrl), or useReorder(collectionUrl, { idKey: 'appId' }) for non-id pk.
6. Drizzle custom migration (runs when the consuming resource's PR lands, not part of the base-infrastructure PR): add order_key nullable → backfill bucket-by-bucket via generateOrderKeySequence imported from @data/services/utils/orderKey (never from fractional-indexing directly) → promote to NOT NULL → drop the old sort_order column → create the index. Until this step runs, the production schema keeps the legacy sort_order INT column — the base infrastructure never touches existing tables.

---

8. FAQ

Paginated lists — what if the drag anchor is off-page? The server resolves before/after by id against the database, not against the client's loaded window. Any visible row's id is a valid anchor. That's what makes the scheme work for topics/messages that never fit on screen.

Is order:reset safe under concurrent calls? Yes. Reset is deterministic (same preset + row data → same keys via generateOrderKeySequence). SQLite's write lock serializes concurrent resets; the second overwrites the first and the end state is consistent.

Known boundary — fractional-indexing collisions. Two transactions reading the same anchor pair simultaneously both call generateKeyBetween and produce identical new keys. order_key is not UNIQUE, so both rows succeed — the effect is a tie in ORDER BY order_key (two rows alternate in the UI). No data loss; the next drag self-repairs. Single-user SQLite makes it extremely rare. Future fix: composite (order_key, pk) index + deterministic tiebreaker. Don't implement now.

Known boundary — multi-window drag flicker. Window A mid-drag receives window B's cache invalidation → revalidates with server state (not yet including A's in-flight reorder) → optimistic value overwritten → UI snaps back → A's PATCH returns → another revalidate brings new order in → UI jumps forward. ~150–300 ms, visual-only, no data loss. Future fix: suspend external revalidations while an in-flight PATCH holds the key.

Why no dual-mode sort (default-by-createdAt / switch-to-custom)? order_key is always present and maintained regardless of which sort mode the UI shows. Mode switching belongs at the query layer — a business-level Preference picks ORDER BY lastAccessedAt DESC or ORDER BY order_key ASC at read time. Keeping order_key unconditional gives a uniform write path and avoids "when do we first materialize keys" complexity.

---

9. Group Ordering (Future Extension)

Placeholder. When the first resource that needs group-level ordering adopts the spec (most likely topic), this section will cover:

• Extending OrderRequestSchema with an optional groupId field.
• Cross-group move semantics — a groupId change is a row update (regular PATCH /:id), not a reorder. The /order endpoint only moves rows within their current scope.
• scope interaction when a row changes groups in adjacent operations.

No speculative groupId field, helper variants, or UI code is added before a real consumer drives the design.