thedotmack-claude-mem/plans/2026-06-09-opt-in-posthog-telemetry.md

Opt-In PostHog Telemetry for claude-mem

Goal: One mergeable PR adding privacy-respecting, OPT-IN product analytics to claude-mem using the official posthog-node SDK, gated by a consent resolver and a whitelist property scrubber. Ships dark: default OFF, nothing leaves the machine without explicit consent.

Origin: Research on branch claude/analytics-platforms-comparison-o58erd (commit 256b3584f, claude-mem-analytics-research-transcript.txt) + the official PostHog skill (npx skills use posthog/ai-plugin@instrument-product-analytics).

Architecture decision (locked — do not re-litigate)

The research transcript proposed a SQLite telemetry_events spool table drained by a worker flush job. We are NOT building that. The spool solved "short-lived hook processes exit before the SDK's in-memory queue flushes" — but in claude-mem all the events worth tracking (compression, search, startup, errors) occur inside the long-running worker service, which is exactly where a posthog-node client is designed to live. So:

• posthog-node SDK initialized lazily, once, in the worker. SDK handles batching/retry/flush.
• NO new SQLite migration. NO custom /batch/ wire format. NO custom retry loop.
• Every capture goes through ONE wrapper: consent gate → whitelist scrub → debug-print or posthog.capture(). The wrapper never throws and never blocks.
• cli.command events from short-lived npx processes are scope-cut from v1 (future: POST to worker HTTP). Keeps the PR small.

What we keep from the research (the trust-critical layer):

• Opt-in, default OFF. Consent precedence: DO_NOT_TRACK (truthy → forced off) > CLAUDE_MEM_TELEMETRY env (0/false/off → off, 1/true/on → on) > telemetry.json config > default OFF.
• Random install UUID (crypto.randomUUID, first use) + consent stored in ~/.claude-mem/telemetry.json (data dir via existing helper; survives DB resets). UUID = PostHog distinctId.
• Whitelist scrubber. Allowed property keys ONLY: version, os, arch, runtime, runtime_version, duration_ms, outcome, error_category, locale, is_ci, plus the event name. Everything else dropped. NEVER: paths, project names, git remotes, prompts, source code, IPs, hardware IDs (even hashed), env values, emails.
• CLAUDE_MEM_TELEMETRY_DEBUG=1 → print would-be payloads to stderr, send nothing.
• claude-mem telemetry [status|enable|disable] CLI command.
• docs/public/telemetry.mdx enumerating every field collected and not-collected.
• Unit tests for the consent resolver and the scrubber.

Allowed APIs (verified)

posthog-node (https://posthog.com/docs/libraries/node — re-read before coding):

• new PostHog(apiKey, { host, flushAt, flushInterval }) — host default https://us.i.posthog.com; key/host via CLAUDE_MEM_TELEMETRY_KEY / CLAUDE_MEM_TELEMETRY_HOST env with a hardcoded publishable project token fallback constant (publishable tokens are safe to embed — verified at posthog.com/docs/api: capture endpoints are public POST-only, no rate limits, 20MB body cap).
• client.capture({ distinctId, event, properties }) — fire-and-forget, queues in memory.
• Set properties.$process_person_profile = false on every event (anonymous events — cheaper, privacy-aligned).
• await client.shutdown() — flush on worker graceful stop ONLY. Never on the capture path.

Codebase facts (verified 2026-06-09):

• Migration runner is at version 34 (src/services/sqlite/migrations/runner.ts) — we deliberately do not touch it.
• Tests: bun test, files at tests/**/*.test.ts, import from bun:test; copy style from tests/json-utils.test.ts.
• Worker code: src/services/worker/ (Express http/, SettingsManager.ts, providers). CLI commands: src/npx-cli/commands/ (e.g. doctor.ts).
• @clack/prompts is already a dependency (used by installer flows).
• Event naming: snake_case (per official PostHog skill).

Phase 1 — Discovery + consent & scrub core (pure logic, no network, no SDK)

1. Grep for existing posthog/telemetry references (expect only incidental SDK-flag mentions in ProviderObservationGenerator.ts, ClaudeProvider.ts, ChromaMcpManager.ts — read them to confirm they're unrelated, do not modify).
2. Find the existing data-dir helper that resolves CLAUDE_MEM_DATA_DIR / ~/.claude-mem (grep CLAUDE_MEM_DATA_DIR in src/), and the readJsonSafe util (src/utils/json-utils.ts). Reuse both.
3. Create src/services/telemetry/consent.ts:
   • resolveTelemetryConsent(env, config): boolean — pure, precedence as locked above.
   • loadTelemetryConfig() / saveTelemetryConfig() — telemetry.json in data dir: { enabled: boolean, installId: string, decidedAt: string }.
   • getOrCreateInstallId(): string.
4. Create src/services/telemetry/scrub.ts: scrubProperties(props): Record<string, string|number|boolean> — whitelist filter, primitive-only values, truncate strings >200 chars.
5. Tests: tests/telemetry/consent.test.ts, tests/telemetry/scrub.test.ts (style of tests/json-utils.test.ts). Cover: DO_NOT_TRACK=1 beats enabled config; env off beats config on; default off; unknown keys dropped; nested objects dropped; denylisted-looking keys (path, cwd, prompt) dropped even if someone whitelists by mistake — add an explicit denylist assertion test.

Verify: bun test tests/telemetry/ green. No imports from posthog-node anywhere yet.

Phase 2 — SDK wrapper + worker capture sites

1. npm install posthog-node (package manager command, do not hand-edit package.json). Check esbuild/build config: if worker bundle has an external array pattern (it did for bullmq/pg), determine whether posthog-node bundles cleanly; mirror the existing convention.
2. Create src/services/telemetry/telemetry.ts:
   • Lazy singleton getClient() — constructs PostHog only on first consented capture.
   • captureEvent(event: string, props?: Record<string, unknown>): void — consent gate → base props (version from package.json/version helper, os, arch, runtime: 'bun'|'node', runtime_version, is_ci, locale) → scrub → if CLAUDE_MEM_TELEMETRY_DEBUG=1 print JSON to stderr and return → else client.capture({...}). Entire body in try/catch that swallows (debug-log only). Synchronous, O(1).
   • shutdownTelemetry(): Promise<void> — flush with a 3s race timeout; never rejects.
3. Wire capture sites (find exact locations by reading worker code; keep each insertion to 1-3 lines):
   • worker_started — worker service startup completion (src/services/worker-service.ts or equivalent startup path).
   • session_compressed — where a session compression/summarization completes successfully (with duration_ms, outcome).
   • search_performed — SearchManager / search HTTP route entry (NO query text — only outcome).
   • error_occurred — central worker error handler if one exists (with error_category only — a coarse enum string, never error.message).
   • shutdownTelemetry() in the worker's graceful-stop path (find existing SIGTERM/stop handler).
4. Anti-pattern guards: no posthog import outside src/services/telemetry/; no capture( call site passing raw error messages, paths, or query strings; key only via env/constant.

Verify: npm run build (or the repo's build script) passes. bun test green. Grep checks: grep -rn "posthog" src/ | grep -v services/telemetry returns nothing; with consent absent, boot worker with CLAUDE_MEM_TELEMETRY_DEBUG=1 → zero telemetry stderr lines (gate is before debug print? NO — debug prints only when consented; with no consent, nothing prints. Confirm that ordering in code: consent gate FIRST, debug second).

Phase 3 — CLI command + docs

1. Read src/npx-cli/commands/doctor.ts and the command router that registers it; copy the registration pattern exactly.
2. Create src/npx-cli/commands/telemetry.ts:
   • status (default): prints enabled/disabled, which layer decided it (DO_NOT_TRACK / env / config / default), install ID, and the docs URL.
   • enable: @clack/prompts confirm showing the exact field list collected + "no prompts, paths, code, or project names — ever" + docs link; on yes, write config with installId.
   • disable: write enabled: false. No prompt.
3. Create docs/public/telemetry.mdx + add nav entry in docs/public/docs.json (check existing docs.json structure first): tables of every field collected / explicitly never collected, all four disable methods (DO_NOT_TRACK, env var, config, CLI command), debug mode, where telemetry.json lives, the fact it's opt-in and ships disabled.

Verify: run the CLI command locally (bun src/npx-cli/... telemetry status per repo's dev pattern) — status reflects env overrides correctly (test with DO_NOT_TRACK=1).

Phase 4 — Verification + PR

1. Full bun test and the build script. Fix anything broken by the new dep.
2. Anti-pattern sweep (all must pass):
   • grep -rn "from 'posthog-node'" src/ | grep -v services/telemetry/telemetry → empty.
   • grep -rn "captureEvent(" src/ → every call site's props are literal objects containing only whitelisted keys.
   • No personal API key (phx_) anywhere; only publishable (phc_) token constant/env.
   • telemetry.json never written without explicit user action (enable command) — grep saveTelemetryConfig call sites.
3. Branch feat/opt-in-telemetry from main, commit (conventional: feat(telemetry): opt-in anonymous usage analytics via PostHog), push, gh pr create with: summary, the architecture-decision paragraph (worker-resident SDK, no spool), the collected/never-collected table, and screenshots/output of telemetry status.
4. Hand off to claude-mem:babysit for the PR.

Done means: PR open, CI green, telemetry provably inert by default (fresh install sends nothing), and the privacy docs page renders.