4 Commits

Author SHA1 Message Date
Yifan Yang
9de9220214 docs(sleep): add cross-model scaling results (nano +11.9) and hyperparam ablation (#89)
Update RESULTS.md with:
- §2: GPT-5.4-nano target yields +11.9 pt (0.560→0.679) on SearchQA —
  2× the GPT-5.5 gain, demonstrating bigger benefit where headroom exists
- §4: Hyperparameter sweep confirms shipped defaults are optimal

Co-authored-by: Claude Opus 4 <noreply@anthropic.com>
2026-06-26 01:40:58 +08:00
Yifan Yang
46b3207b96 docs(sleep): trim RESULTS to the headline results (remove the full grid)
Remove the per-cell full deployment grid section; keep the gate-safety stress
test, experience-replay scaling + night-by-night climb, the dream-diversity
ablation, the gbrain end-to-end result, and the scope/limitations. Renumber
sections; update the README pointer accordingly.
2026-06-15 17:08:51 +00:00
Yifan Yang
d43e8dba1a docs(sleep): expand the grid into per-benchmark night-by-night tables
Replace the compact baseline->after grid with three grouped per-benchmark tables
(SearchQA / LiveMath / SpreadsheetBench), each showing all 3 targets x both modes
across every night (N0..N5) + Δ. Makes the trajectory visible — gains reach a
level and hold rather than being single lucky readings — and presents the full
18-cell evidence in a more solid, readable form. Footnotes LiveMath's 4-night run
(train split <50 tasks). Numbers unchanged; just richer presentation.
2026-06-15 16:54:01 +00:00
Yifan Yang
d02098ffc4 docs(sleep): add full Results & Analysis (RESULTS.md); link from README
Adds docs/sleep/RESULTS.md — the complete deployment-scale study behind
SkillOpt-Sleep, presented rigorously (named benchmarks, test sizes, metrics,
baseline->after, single shared protocol):
  1. Gate-safety stress test: ungated nano SearchQA collapses 0.554->0.026
     (-52.8); the gated twin holds 0.570 — the core argument for the design.
  2. Full 18-cell deployment grid (3 benchmarks x 3 targets x gate/free),
     shipped config: mean +0.5, range [-2.4, +5.1], nothing hidden.
  3. Experience-replay scaling (recall_k 10->20->full: +3.1->+4.5->+5.6) and
     the night-by-night climb (0.798->...->0.858, gate accepts as late as N5).
  4. Dream-diversity fix as defense-in-depth: 3-config grid comparison
     (-2.66/-52.8 -> +0.24/-4.0 -> +0.53/-2.4); the -52.8 cell becomes +2.7
     from the dream fix alone.
  5. gbrain end-to-end 0.00->1.00 on real Claude + Codex.
  6. Honest scope: where it helps vs flat-in-noise, single-seed caveat with a
     seed-robustness spot check, keep-the-gate-on.
README Results section now links prominently to it. Docs only; numbers are
self-contained with reproduce commands (no raw run dumps committed).
2026-06-15 16:49:13 +00:00