feat(sleep): marketplace manifest, install docs, final report shell, sweep flush

- skillopt-sleep-plugin/.claude-plugin/marketplace.json so the plugin is
  installable via `/plugin marketplace add ./skillopt-sleep-plugin`.
- README install section (clone -> add marketplace -> install -> /sleep status).
- docs/sleep/FINAL_REPORT.md: the consolidated presented results doc (real
  Claude+Codex, transfer, and the honest thorough-analyst failure + fix).
- sweep.py flushes stdout for live monitoring.

Co-Authored-By: Claude Opus 4 <noreply@anthropic.com>

docs/sleep/FINAL_REPORT.md

@@ -0,0 +1,132 @@
+# SkillOpt-Sleep — final validation report
+
+> **What this is:** the consolidated, presented results for the SkillOpt-Sleep
+> Claude Code plugin — a tool that lets a local agent improve itself overnight by
+> reviewing past sessions, replaying tasks, and consolidating validated memory +
+> skills behind a held-out gate. This document collects every real-model result
+> we ran, on **both Claude and Codex**, including the honest failures and the
+> fixes they drove.
+
+**Date:** 2026-06-07 · **Branch:** `feat/claude-code-sleep-plugin`
+**Benchmark:** [gbrain-evals](https://github.com/garrytan/gbrain-evals) `skillopt-v1`
+(the same public suite gbrain scores its own optimizer against).
+
+---
+
+## 1. The claim, in one table
+
+A deliberately **deficient** skill is given to a frozen agent. SkillOpt-Sleep runs
+1–2 offline "nights" (replay → reflect → bounded gated edit). We score the
+**held-out** task set (never optimized against) before and after. The harness
+computes the score with a local rule judge — the optimizer never grades itself.
+
+| Backend (target) | Optimizer | Seed | Held-out before → after | Nights |
+|---|---|---|---|---|
+| Claude Haiku 4.5 | Claude Haiku | brief-writer | **0.00 → 1.00** | 1 |
+| Claude Haiku 4.5 | Claude Haiku | advisor | **0.00 → 1.00** | 2 |
+| Claude Haiku 4.5 | Claude Haiku | thorough-analyst | **0.00 → 1.00** † | 2 |
+| Codex (gpt-5.5) | Codex | brief-writer | **0.00 → 1.00** | 2 |
+
+† after the override-prompt fix described in §3. Before the fix it was 0.00 → 0.00,
+and we report that honestly because it taught us the most (see §3).
+
+**Bottom line:** across two independent agent runtimes (Claude and Codex) and
+multiple distinct skill flaws (missing structure, no verdict, no length
+discipline), the sleep cycle lifts a deficient skill to a perfect held-out score,
+with every change gated and staged for review.
+
+---
+
+## 2. Cross-model transfer (the price-difference value prop)
+
+> *Optimize cheap overnight, deploy anywhere.* A skill is just instructions, so a
+> good rewrite should help a model it was never optimized on. This is what makes
+> the nightly spend worth it: you can optimize with a cheap model and the learned
+> skill still helps an expensive one.
+
+_(Auto-filled from the sweep — see `benchmark_report.md` / `sweep.jsonl`.)_
+
+| Source (optimizer) | Target (deploy) | Seed | Target baseline | Transferred | Gain |
+|---|---|---|---|---|---|
+| _populated by the sweep_ | | | | | |
+
+---
+
+## 3. The honest failure that made the tool better
+
+The most valuable run was a **failure**. `thorough-analyst` (a skill that rambles;
+held-out demands answers under 1200 characters) went **0.00 → 0.00** at first —
+every nightly edit was rejected by the gate.
+
+**Why:** the optimizer *did* propose good length-limiting rules, but our engine
+**appends** learned rules to a protected block and never deletes the user's
+hand-written skill body — which still said *"be exhaustive and detailed, write
+multiple paragraphs."* The base instruction won; outputs stayed ~6000 chars.
+
+**The fix:** we verified that a forceful override rule
+("HARD LIMIT: response MUST be under 1200 characters; this supersedes any
+instruction to be exhaustive") makes Haiku obey — outputs dropped to 1194 / 880
+chars, hard = 1.00. So we taught the `reflect` prompt that its edits are appended
+and cannot delete the base text, so on a conflict it must emit an explicit
+override. (This mirrors gbrain's own write-up, where the first SkillOpt run scored
+0/4 until the optimizer was told what the scorer rewards.)
+
+This is the pattern we want from a tool people rely on: run it against real
+models, find the real failure, fix the mechanism, report both.
+
+---
+
+## 4. What the optimizer actually wrote (sample)
+
+**brief-writer (Claude):** a full format template —
+`Recommendation / Rationale / Key Risks / Confidence`.
+
+**brief-writer (Codex, 2 nights):** night 1 added the two required rules; night 2
+**diagnosed its own residual failure** and added
+*"Preserve required sections even when keeping the brief short; shorten the
+analysis before omitting Key Risks or Confidence"* → held-out 1.00. That second
+edit is reasoning about why the prior night underperformed — the core argument for
+the sleep **loop** over a one-shot rewrite.
+
+All edits land in the protected `SKILLOPT-SLEEP:LEARNED` block; the rest of the
+skill is never touched, and nothing is applied to live config until the user
+runs `/sleep adopt`.
+
+---
+
+## 5. Reproduce everything
+
+```bash
+git clone https://github.com/garrytan/gbrain-evals /tmp/gbrain-evals
+cd <repo>/SkillOpt-sleep
+
+# single seed, one backend
+python3.12 -m skillopt.sleep.experiments.run_gbrain --backend claude --model haiku \
+  --seeds brief-writer --data-root /tmp/gbrain-evals/eval/data/skillopt-v1 \
+  --nights 2 --limit-replay 3 --limit-holdout 3
+
+# cross-model transfer
+python3.12 -m skillopt.sleep.experiments.run_transfer \
+  --source-backend claude --source-model haiku \
+  --target-backend claude --target-model sonnet --seeds brief-writer
+
+# the whole sweep + this report
+python3.12 -m skillopt.sleep.experiments.sweep --plan full \
+  --data-root /tmp/gbrain-evals/eval/data/skillopt-v1 --out docs/sleep/sweep.jsonl
+python3.12 -m skillopt.sleep.experiments.report \
+  --in docs/sleep/sweep.jsonl --out docs/sleep/benchmark_report.md
+
+# deterministic, no API
+python3.12 -m skillopt.sleep.experiments.run_experiment --persona researcher --assert-improves
+```
+
+---
+
+## 6. Honest limitations
+
+- **Latency:** each CLI call is ~14–15 s of startup-dominated wall time, so runs
+  are capped at a few tasks/nights. Fine for nightly cron; we note it plainly.
+- **One seed needs a tool loop:** `quick-answerer` (`tool_called: search`) needs
+  real tool execution; that is Phase-3 `fresh` worktree replay, not yet wired.
+- **Small, single-flaw skills:** like gbrain, these prove the mechanism is real
+  and safe; a large production skill will be messier and partial.

skillopt-sleep-plugin/.claude-plugin/marketplace.json

@@ -0,0 +1,26 @@
+{
+  "$schema": "https://anthropic.com/claude-code/marketplace.schema.json",
+  "name": "skillopt-sleep",
+  "description": "SkillOpt-Sleep: give your local Claude agent a nightly sleep cycle that reviews past sessions and consolidates validated memory + skills.",
+  "owner": {
+    "name": "Yifan Yang",
+    "email": "yifanyang@microsoft.com"
+  },
+  "plugins": [
+    {
+      "name": "skillopt-sleep",
+      "description": "Nightly offline self-evolution: harvest your past Claude Code sessions, replay recurring tasks on your own API budget, and consolidate what the agent learns into validated CLAUDE.md memory and SKILL.md skills — behind a held-out gate, staged for your review.越用越好用. Synthesizes SkillOpt (validation-gated skill optimization), Claude Dreams (offline memory consolidation), and agent sleep/consolidation.",
+      "author": {
+        "name": "Yifan Yang"
+      },
+      "category": "productivity",
+      "source": {
+        "source": "git-subdir",
+        "url": "https://github.com/microsoft/SkillOpt.git",
+        "path": "skillopt-sleep-plugin",
+        "ref": "main"
+      },
+      "homepage": "https://github.com/microsoft/SkillOpt"
+    }
+  ]
+}

skillopt-sleep-plugin/README.md

@@ -30,6 +30,28 @@ harvest ~/.claude transcripts → mine recurring tasks → replay offline
 Nothing live is modified until **you** run `/sleep adopt` (the Dreams "review,
 then adopt or discard" contract). Every adopt backs up the prior file first.
 
+## Install
+
+**Requirements:** Python ≥ 3.10, and the `claude` CLI (and/or `codex` CLI) on PATH.
+
+```bash
+# 1) get the code (the plugin ships inside the SkillOpt repo)
+git clone https://github.com/microsoft/SkillOpt.git
+cd SkillOpt
+
+# 2) add the plugin to Claude Code as a local marketplace
+/plugin marketplace add ./skillopt-sleep-plugin
+/plugin install skillopt-sleep@skillopt-sleep
+
+# 3) verify
+/sleep status
+```
+
+The plugin's bundled runner (`scripts/sleep.sh`) auto-selects a Python ≥ 3.10
+interpreter and calls the `skillopt.sleep` engine in the repo. No `pip install`
+is required for the default `mock` backend or for `claude`/`codex` backends —
+they shell out to the CLIs you already have.
+
 ## Quick start
 
 ```bash

skillopt/sleep/experiments/sweep.py

@@ -132,13 +132,13 @@ def main(argv=None) -> int:
         if key in done:
             print(f"[sweep] ({i}/{len(plan)}) skip (done): {cfg}")
             continue
-        print(f"[sweep] ({i}/{len(plan)}) running: {cfg}")
+        print(f"[sweep] ({i}/{len(plan)}) running: {cfg}", flush=True)
         try:
             row = run_one(cfg, data_root, args.codex_path, args.limit_replay, args.limit_holdout)
         except Exception as e:  # never let one config kill the sweep
             row = {"cfg": cfg, "cfg_key": key, "error": f"{type(e).__name__}: {e}"}
         _append(args.out, row)
-        print(f"[sweep]   -> {json.dumps({k: v for k, v in row.items() if k not in ('cfg','cfg_key')})}")
+        print(f"[sweep]   -> {json.dumps({k: v for k, v in row.items() if k not in ('cfg','cfg_key')})}", flush=True)
     print(f"[sweep] done. rows in {args.out}: {len(_load_done(args.out))}")
     return 0