--- summary: "How to run tests locally (vitest) and when to use force/coverage modes" read_when: - Running or fixing tests title: "Tests" --- - Full testing kit (suites, live, Docker): [Testing](/help/testing) - Update and plugin package validation: [Testing updates and plugins](/help/testing-updates-plugins) ## Agent default Agent sessions run tests and computationally intensive validation remotely through Crabbox. Trusted maintainer code defaults to Blacksmith Testbox. The configured Testbox workflow hydrates credentials, so untrusted contributor or fork code must use secretless fork CI or sanitized direct AWS Crabbox instead. When a trusted code task is likely to need tests or heavy proof, pre-warm immediately in a background command session, keep working while it hydrates, reuse the returned `tbx_...` id, sync the current checkout on every run, and stop it before handoff: ```bash node scripts/crabbox-wrapper.mjs warmup --provider blacksmith-testbox --keep --timing-json ``` Local test commands below are for human workflows or an explicit agent fallback requested by the user. Remote-provider unavailability must be reported; it is not permission to silently run a broad local gate. For untrusted code, pre-warm with `--provider aws`. Every run must set `CRABBOX_ENV_ALLOW=CI`, pass `--provider aws --no-hydrate`, and use a fresh temporary remote `HOME` before installing dependencies or running tests. Use a newly warmed lease dedicated to that untrusted source; never reuse a trusted or previously hydrated lease. Launch an installed trusted Crabbox binary from a clean trusted `main` checkout and fetch only the remote PR with `--fresh-pr`; never execute the untrusted checkout's wrapper or config locally. Unset `CRABBOX_AWS_INSTANCE_PROFILE` and fail closed unless resolved `aws.instanceProfile` is empty. Before any install/test, use trusted absolute-path tools to require an IMDSv2 token, prove the IAM credentials endpoint returns 404, and verify remote `git rev-parse HEAD` equals the full reviewed PR head SHA. Bind the lease to that SHA and stop/rewarm when the head changes. Upload trusted `scripts/crabbox-untrusted-bootstrap.sh` from clean `main` alongside `--fresh-pr`; it installs pinned Node/pnpm, verifies the SHA and package-manager pin, isolates `HOME`, installs dependencies, then executes the requested test. If the broker cannot prove no role or no remote PR exists, use secretless fork CI. Do not use `hydrate-github`, `--no-sync`, or a credential-hydrated Testbox workflow. Unset all `CRABBOX_TAILSCALE*` overrides, force `--network public --tailscale=false`, clear exit-node/LAN flags, and require `crabbox inspect` to report public networking with no Tailscale state before uploading any script. ## Routine local order 1. `pnpm test:changed` for changed-scope Vitest proof. 2. `pnpm test ` for one file, directory, or explicit target. 3. `pnpm test` only when you intentionally need the full local Vitest suite. In a Codex worktree or linked/sparse checkout, agents avoid direct local `pnpm test*` / `pnpm check*` / `pnpm crabbox:run`: - Explicit user-requested local fallback for a tiny file: `node scripts/run-vitest.mjs `. - Changed gates or broad proof: `node scripts/crabbox-wrapper.mjs run --provider blacksmith-testbox ... -- env OPENCLAW_CHECK_CHANGED_REMOTE_CHILD=1 OPENCLAW_CHANGED_LANES_RAW_SYNC=1 corepack pnpm check:changed` so pnpm runs inside Testbox. - The wrapper's final `exitCode` and timing JSON are the command result. A delegated Blacksmith GitHub Actions run may show `cancelled` after a successful SSH command because the Testbox is stopped from outside the keepalive action; check the wrapper summary and command output before treating that as a failure. - `OPENCLAW_HEAVY_CHECK_LOCK_SCOPE=worktree `: keeps heavy-check serialization inside the current worktree instead of the Git common dir for commands such as `pnpm check:changed` and targeted `pnpm test ...`. Use it only on high-capacity local hosts when you intentionally run independent checks across linked worktrees. ## Core commands Test wrapper runs end with a short `[test] passed|failed|skipped ... in ...` summary; Vitest's own duration line stays the per-shard detail. | Command | What it does | | ------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `pnpm test` | Explicit file/directory targets route through scoped Vitest lanes. Untargeted runs are full-suite proof: fixed shard groups expand to leaf configs for local parallel execution, with the expected shard fanout printed before starting. The extension group always expands to per-extension shard configs instead of one giant root-project process. | | `pnpm test:changed` | Cheap smart changed-test run: precise targets from direct test edits, sibling `*.test.ts` files, explicit source mappings, and the local import graph. Broad/config/package changes are skipped unless they map to precise tests. | | `OPENCLAW_TEST_CHANGED_BROAD=1 pnpm test:changed` | Explicit broad changed-test run; use when a test harness/config/package edit should fall back to Vitest's broader changed-test behavior. | | `pnpm test:force` | Frees the configured OpenClaw gateway port (default `18789`), then runs the full suite with an isolated gateway port so server tests do not collide with a running instance. | | `pnpm test:coverage` | Unit suite with V8 coverage (`vitest.unit.config.ts`). Default-unit-lane gate, not whole-repo coverage: `coverage.all` is `false` and thresholds are lines/functions/statements 70%, branches 55%, scoped to non-fast unit tests with sibling source files. | | `pnpm test:coverage:changed` | Unit coverage only for files changed since `origin/main`. | | `pnpm changed:lanes` | Shows the architectural lanes triggered by the diff against `origin/main`. | | `pnpm check:changed` | Delegates to Crabbox/Testbox by default outside CI, then runs the smart changed check gate inside the remote child: typecheck, lint, and guard commands for affected lanes. Does not run Vitest; use `pnpm test:changed` or `pnpm test ` for test proof. | ## Shared test state and process helpers - `src/test-utils/openclaw-test-state.ts`: use from Vitest when a test needs an isolated `HOME`, `OPENCLAW_STATE_DIR`, `OPENCLAW_CONFIG_PATH`, config fixture, workspace, agent dir, or auth-profile store. - `pnpm test:env-mutations:report`: non-blocking report of tests/harnesses that mutate `HOME`, `OPENCLAW_STATE_DIR`, `OPENCLAW_CONFIG_PATH`, `OPENCLAW_WORKSPACE_DIR`, or related env keys directly. Use it to find migration candidates for the shared test-state helper. - `test/helpers/openclaw-test-instance.ts`: process-level E2E tests needing a running Gateway, CLI env, log capture, and cleanup in one place. - Docker/Bash E2E lanes that source `scripts/lib/docker-e2e-image.sh` can pass `docker_e2e_test_state_shell_b64