Addresses remaining maintainer + Copilot review blockers on #134.
- Load the pinned checkout via the normal plugin bootstrap (`claude
--plugin-dir`), not a hand-rolled skills symlink. A per-run session marker is
injected into using-superpowers/SKILL.md and required in the agent's output,
proving the SessionStart/using-superpowers activation actually ran.
- Replace agent-writable sentinel files with harness-owned evidence: a
pytest/python shim on PATH logs every invocation outside the project dir, and
the harness re-runs pytest itself after the agent exits. Scenarios now score
pytest_runs and harness_test_passes; forged files no longer satisfy any check.
- Stop reusing host credentials by default. ~/.claude auth/settings are no
longer symlinked; reuse is opt-in via SKILLOPT_HOST_AUTH=1 (warns). Fail
closed (NO_AUTH) when neither a key nor host-auth is available.
- Add OS-level isolation, opt-in via SKILLOPT_SANDBOX=bwrap|docker.
- Prompt on stdin + --output-format text, matching backend.py CLI usage.
- Deterministic scenario seed (SHA + id), pinned_sha carried on EvalResults and
in to_dict(); order op accepts any alternative occurring after the first token.
- Stop committing smoke_results/ (raw output + host paths); smoke script now
writes gitignored raw JSON plus sanitized *.summary.txt excerpts to share.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- Fix not_contains to split on pipe (all alternatives must be absent)
- Add regression tests for false completion claim detection
- Scrub host env: only PATH/TERM/LANG/ANTHROPIC_API_KEY, no credentials
- Remove unconditional --dangerously-skip-permissions (opt-in via SKILLOPT_UNSAFE=1)
- Include raw output in JSON for smoke test evidence
- Fix smoke script: fail on errors, preserve raw output
Add missing configuration setup in scripts/eval_only.py to properly
support the minimax_chat backend, which was entirely omitted.
Fix the following coverage gaps in eval_only.py:
- Add minimax CLI arguments
- Include the minimax config mappings in _MAP
- Update the backend parsing logic
- Call configure_minimax_chat
The eval-only tool skipped configuring some of the backend types, that
the training did configure. Because of this, the eval is silently
fell back to a local endpoint that wasn't actually configured, and
all evaluations runs failed.
Replicate the backend setup based on the trainer's code, and eval-only
can run with the qwen_chat backends.
Co-authored-by: Qwen-Coder <noreply@qwen.ai>
Split failure reflections into SKILL_DEFECT (body edit) vs EXECUTION_LAPSE
(protected appendix note that re-emphasizes an existing rule, never edited
by step-level analysts). Toggle: optimizer.use_skill_aware_reflection
(default false; baseline byte-identical when off).
- optimizer/appendix.py: protected APPENDIX region (inject/extract/append
with dedup), mirrors the slow_update protected-field pattern
- optimizer/skill_aware.py: analyst prompt augmentation, appendix_notes
parsing, threshold-gated LLM consolidation, and a process-wide runtime
switch (configure_skill_aware_reflection) set once by the trainer
- gradient/reflect.py: augment error/success analyst prompts at runtime;
None-sentinel kwargs resolve from the global switch, so env adapters
need no per-benchmark wiring (works for all envs, present and future)
- optimizer/skill.py: generalize the protected-region check to
(slow_update, appendix); edits inside any protected region are skipped
- engine/trainer.py: inject appendix at init, flush per-step
EXECUTION_LAPSE notes after the gate settles, optional consolidation
- tests: regression suite incl. toggle-off byte-identical guarantee and
env-independent global-switch resolution (6/6 passing + live smoke)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PR #26 added a MiniMax chat backend but left three loose ends that
silently dropped any YAML / CLI configuration of minimax_* keys: only
the environment-variable path worked.
- skillopt/config.py: add 6 model.minimax_* entries to _FLATTEN_MAP so
the keys declared in configs/_base_/default.yaml actually survive
flatten_config() (mirroring the existing model.qwen_chat_* block).
- skillopt/engine/trainer.py: import configure_minimax_chat and call
it alongside configure_qwen_chat, so cfg-supplied credentials,
temperature, max_tokens, and enable_thinking reach the backend. Also
apply cfg["minimax_model"] via set_target_deployment when the active
target backend is minimax_chat.
- scripts/train.py: add 6 --minimax_* CLI flags + the corresponding
_CLI_TO_YAML entries, add 'minimax' / 'minimax_chat' to the --backend
choices, auto-route to target_backend=minimax_chat, and pick the
right default target_model for the new backend.
Default behavior on existing backends (openai, claude, qwen, codex,
claude_code_exec) is unchanged; all 8 shipped configs continue to load
with gate_metric falling back to 'hard' for paper reproduction.
Remove sealqa, babyvision, mathverse, mmrb, swebench envs and configs.
Remove deep_probe, deep_reflect, meta_reflect modules and prompts.
Remove download_babyvision script.
These are not part of the core released benchmarks.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Skill optimization framework with training loop analogy
- 11 benchmarks, 4 model backends (Azure OpenAI, Claude, Codex, Qwen)
- WebUI for browser-based training control
- Pluggable architecture for extending benchmarks and backends
