Testing

What to test here, which style to reach for, and how to write an assertion that means something.

Tests live beside the code and run in plain Node. That is not a convenience — it is the check that the core stays free of DOM, React and WebGL. Nothing registers a browser environment.

That now includes the client. vitest.config.ts covers apps/* as well as packages/*, and apps/game/src/engine/gameEngine.test.ts drives the real frame loop, lens resolution, observatory framing, save/load and derived-state invalidation under Node — because GameEngine takes its worker factory and save store as arguments instead of constructing a browser Worker and IndexedDB itself. The dedicated terrain-descent slow test runs through pnpm test:slow, alongside galaxy convergence and population checks; both regular and slow suites are in pnpm check.

The regular suite runs at most four files concurrently, or fewer when the host has fewer available cores. This bounds contention between procedural generation and CPU rendering tests while preserving the 20-second timeout. Use VITEST_MAX_WORKERS=2 pnpm check for a more constrained verification run; do not raise timeouts before investigating contention.

Verify the committed change

Commit each coherent step before lengthy verification, as described in the working guide. When another task's untracked files make a repository-wide check fail, preserve them. Verify the committed change in a temporary worktree instead of formatting, hiding or deleting someone else's work:

git worktree add --detach /tmp/inertialref-verify HEAD
cd /tmp/inertialref-verify
pnpm install --frozen-lockfile --prefer-offline
pnpm check

Use an unused temporary path and remove only your own verification worktree when finished. Report the shared checkout's failure and the isolated result separately. The isolated tree must contain the same changes being claimed as verified. Fresh worktrees still need the locked install; --prefer-offline can require network access when the local store lacks a package.

Choosing a style

flowchart TB
    Q{"what are you testing?"}
    Q -->|"a mathematical relationship"| PROP["<b>property test</b><br/>round trips, invariants, ordering"]
    Q -->|"a specific known value"| GOLD["<b>golden vector</b><br/>PRNG output, wire formats"]
    Q -->|"determinism"| HASH["<b>state hash equality</b>"]
    Q -->|"physics"| ANA["<b>compare to the analytic result</b>"]
    Q -->|"a boundary or a failure"| EX["<b>example test</b><br/>malformed input, refusals"]

    style PROP fill:#0369a1,stroke:#0c4a6e,color:#fff
    style HASH fill:#0369a1,stroke:#0c4a6e,color:#fff

Several real bugs in this repository were found by a property test and would not have been found by an example. Reach for fast-check whenever the thing under test is mathematical.

The five patterns

1. Property tests

it('round-trips translate/difference (property)', () => {
  fc.assert(
    fc.property(anyPosition, anyDisplacement, (uv, d) => {
      const back = UV.difference(UV.translate(uv, d), uv)
      // bounded by the representation's ULP, not by the absolute magnitude
      expect(Math.abs(back.x - d.x)).toBeLessThanOrEqual(
        UV.POSITION_RESOLUTION * 4,
      )
    }),
  )
})

Good candidates: coordinate round trips, frame re-expression, quaternion composition, orbital element recovery, address parse/format, wire codecs, cube-sphere direction mapping.

A flaky property test is usually the code telling you something. The depth- compression monotonicity test failed intermittently because the mapping really is only non-decreasing past 1e17 m. The fix was to state the true property in two tests, not to loosen the tolerance.

2. Golden vectors

expect(formatSeed(ROOT)).toBe('0df87e57180611d601f6e442eb5fc374')

Not testing that the value is right — any stream would do. Testing that it never changes, because a silent change regenerates every player's universe. Changing one is deliberate and comes with an algorithm version bump in the same commit.

3. State-hash equality

The canonical comparison for anything about determinism:

const jittery = build(),
  steady = build()
while (jittery.clock.tick < 512) jittery.advance(randomFrameTime())
steady.runTicks(jittery.clock.tick)
expect(jittery.stateHash()).toBe(steady.stateHash())

4. Compare to the analytic result

const fallen = radius - now
const predicted = 0.5 * gravity * seconds * seconds
expect(Math.abs(fallen - predicted) / predicted).toBeLessThan(0.02)

Not "the number went down". Capability check 5 once passed while reporting "fell from 57287 km to 57287 km" — the ship had fallen 19 m out of 57,000 km and the assertion was now <= start, which equality satisfies.

A test that cannot fail informatively converts an unknown into a false assurance.

5. Example tests for boundaries

Malformed JSON, a save from a newer schema, a frame that cannot be rebuilt, an unknown worker task, a version mismatch. These are the paths where behavior is a decision (refuse? default? migrate?) and the test documents the decision.

Writing an honest bound

When a tolerance is loose because of a real limit, name the limit:

// says where the number came from
expect(error).toBeLessThan(UV.POSITION_RESOLUTION * 2)

// says nothing; will be "fixed" by loosening when it fails
expect(value).toBeCloseTo(expected, 3)

The first survives a refactor with its meaning intact. The second is a magic number that the next person will adjust rather than investigate.

Related: prefer relative comparisons when magnitudes span orders of magnitude. expect(Math.abs(a / b - 1)).toBeLessThan(1e-9) means something at both 1e-3 and 1e16; an absolute epsilon does not.

A wall-clock bound has one honest job: catching a collapse. The catalog's search bound allows half a second for six queries that cost 1.9 ms, and the margin is the whole usable range rather than generosity — the same queries cost an order of magnitude more under vitest than under bare Node, and anything close enough to the real figure to be interesting is measuring how busy the machine is. It catches search decoding the catalog per query or going quadratic. It does not catch the scan it was written against, and no bound can: a naive scan of 7,123 stars is 2.9 ms against the index's 1.9, so the index buys a factor of one and a half. Say what a clock assertion cannot see, in the assertion's own comment.

Prove a regression test can fail

Before keeping a regression test, temporarily reintroduce the defect and watch that test fail for the intended reason. Then restore the fix and watch it pass. Three ways a test has failed that check here, each different:

• The assertion did not distinguish the cases. A terrain-normal regression asserted only that normals were unit length; a radial normal is also unit length, so it passed both before and after the bug it claimed to guard.
• The bound was looser than the claim beside it. A shape-generator property test asserted that a body could not exceed exp(0.6) — 1.82 times — its published extent, while its own comment said "30% larger would be inventing a size". The measured inflation topped out at 1.7, so the assertion could not fail for any input in its own range and the 37% volume error it was written to catch sailed through.
• The runtime was not the one with the bug. A module cycle threw ReferenceError: Cannot access 'X' before initialization under native Node ESM. Written as an import() inside a vitest test it passed with the defect deliberately reintroduced, because vitest evaluates modules through its own runner and does not enforce the temporal dead zone Node's linker does. The working version spawns a Node process per entry point from apps/headless. If a bug lives in a boundary — a loader, a bundler, a host — the test has to cross that boundary too.

A timeout is a guard against a hang, not a performance budget

testTimeout is 20 s in vitest.config.ts, not vitest's default 5, and the reason is written there. Several tests legitimately take a second or more of pure CPU — a 129-body Solar System stepped for thousands of ticks, a fast-check property over a quarter of a million noise samples — and the runner puts eighty-odd files across every core at once. At 5 s the suite failed intermittently, and the tests it killed were mostly not the ones that had grown: an Rng uniformity property, an atmosphere sweep, the catalog's own search bound, all pre-existing and green standalone.

The diagnosis is worth remembering because the symptom points somewhere else entirely: when unrelated tests start failing together and pass on a re-run, the timeout has stopped measuring the code and started measuring how busy the machine is. A test that spawns processes is the usual culprit — one here began as ten parallel node invocations under it.each and starved everything.

The other culprit is a browser you started yourself and left running. scripts/drive.mjs leaves Chrome up between invocations on purpose — boot is about five seconds and attaching is under one — so a session that has been verifying in the browser is a session running a GPU process, up to eight worker threads and a Vite dev server beside the suite. Measured here across two consecutive full runs with that rig up: one file timed out each time, and it was a different file each time — a terrain descent once, a crater-field property the next — with both passing clean the moment pnpm drive --down had run. Neither was a regression and both looked exactly like one. --down before the gate, and treat a single timeout that moves between runs as a machine reading rather than a code one.

If contention remains after closing your own rig, rerun the failing files with fewer workers before changing a test or its timeout:

pnpm vitest run packages/universe/src/geology.test.ts apps/game/src/engine/terrainStreamer.test.ts --maxWorkers 2
VITEST_MAX_WORKERS=2 pnpm check

Vitest reads VITEST_MAX_WORKERS for both test projects, so the second command keeps the full gate and every assertion while limiting concurrency. A green focused rerun supports a contention diagnosis; it does not replace the full run. Keep performance measurements separate from a contended gate.

A test that needs more says so at the call site. gameEngine.descent.slow.test.ts streams a landing on Proxima Centauri b through the CPU source. Its disk archive can skip repeated heightfield generation, but a cold run still pays the field serially through an inline worker. The five-minute timeout guards the cold path under contention; a warm run does not justify lowering it. A shared beforeAll performs the descent, and four assertions inspect its terrain, geometry and contact behavior.

The descent and galaxy count use the slow suite. *.slow.test.ts is a second vitest project — apps/game/vitest.slow.config.ts, behind pnpm test:slow — selected by suffix the way the GPU suite is and excluded from the root config for the opposite reason: it makes the same plain-Node claim the root suite does, while cold terrain generation can dominate that suite's wall time. The Stop gate runs the root suite after every turn; pnpm check and CI run both, which is where "the ship lands on the ground it drew" has to be proved before a merge. Anything else that streams a landing takes the suffix. A change under engine/terrainStreamer.ts or the terrain path is therefore proved by pnpm check and CI rather than by the Stop gate, and pnpm test:slow by hand is how to prove it sooner. Test speed carries the dated measurements and remaining cold-run work.

galaxy/field.slow.test.ts also takes the suffix. Its count-convergence check samples 9,953,280 positions across two quadratures: 8.4 s in isolation locally, but beyond the regular suite’s 20 s timeout on CI. It keeps both grids and all count assertions, with a two-minute timeout at the call site.

The figure moves with the field and cache state. Every level the detail floor gains requests another ring of heightfields. A cold CPU descent pays their generation serially; a warm descent validates and reads retained records. Report which path the measured run took.

Check a distribution when the claim is about one

irregularFigure claims its numbers come from twenty-five measured shape models. That is a claim about a population, and a test that generated one body and looked at it could not evaluate it. The test generates eighty across the parameter range and asserts the median and both tails — which is how the generator was caught delivering 0.03 when asked for 0.18, because an fBm's standard deviation is a sixth of its range and nothing divided it out.

The same shape appears wherever a generator has a statistical claim: the spin barrier (nothing may rotate faster than sqrt(3π/Gρ)), the retrograde fraction, the size ladder. Each is one assertion over a population and none of them is expressible about a single draw.

Compare derived quantities, not stored ones

apps/headless/src/solarSystem.test.ts checks the engine's Solar System against a JPL reference, and the strongest assertions in it are the ones about numbers the engine does not store. It has no orbital period field — it computes one from G(M+m) and the semi-major axis — and no surface gravity or escape velocity at all. Matching JPL's published period to four figures says the axis is right, the star's mass is right, the body's mass is right and orbitalPeriod is right, in one assertion, because there is no way for two of those to be wrong and still produce it.

It also converts units in only one direction: the reference keeps the kilometers and days JPL publishes, and the test does the arithmetic, so a check and the thing it checks cannot share a factor of 86,400 and agree with each other about it.

Testing across the boundary

Two mechanisms make otherwise-hard things testable in Node:

flowchart LR
    subgraph PORTS["ports"]
        WP["WorkerPort → in-process fake"]
        SP["SaveStore → in-memory store"]
    end
    subgraph RESULT["what it buys"]
        R1["the pool has real tests<br/>with no worker environment"]
        R2["persistence has real tests<br/>with no IndexedDB"]
    end
    WP --> R1
    SP --> R2

The inline worker is not a mock — it runs the real host loop through the real envelopes, so a value that is not structured-cloneable still fails.

Shader behavior runs on the real GPU, from Node

A TSL node graph is compiled and run on the physical GPU by pnpm test:gpu, without a browser. webgpu on npm is Dawn — the WebGPU implementation Chrome ships — built as a Node addon; apps/game/src/render/gpuSetup.ts installs the three globals three/webgpu reads at import time, and apps/game/src/render/gpuHarness.ts hands a test a WebGPURenderer and five verbs: compile, shader (the generated WGSL), drawGraph and draw (pixels back from a render target), and compute with readBuffer (a storage buffer back from a kernel). The suite is *.gpu.test.ts, its config is apps/game/vitest.gpu.config.ts. It covers material compilation, WGSL structure, pixel and bake readback, terrain agreement, star projection and dust, physical galaxy light, temporal history and sensor behavior.

Name a focused file when investigating one mechanism: pnpm vitest run --config apps/game/vitest.gpu.config.ts materials.gpu. Suite size and wall time change as coverage grows, and cost depends on the adapter and selected tests. The early eight-file, 41-test suite took about 18 seconds on an M5; that is a historical measurement, not the current suite's budget. The config explains why the physical-GPU suite sits outside pnpm check; the roadmap carries the open hosted-adapter question. Two limits travel with every answer the suite gives: Dawn tracks its own release while Chrome ships its own, so a graph that compiles under one and not the other is possible, which is why the browser rig stays the arbiter; and navigator.gpu is undefined on about:blank and data: URLs because isSecureContext is false there, so a browser-side probe has to be served from localhost or a real origin to mean anything.

It is a separate command rather than part of pnpm test because it makes a different portability claim: the rest of the suite runs on any Node, and this needs a Metal, Vulkan or D3D adapter. It is not in pnpm check for the same reason. Run it during shader work, and before shipping any change under render/.

A sandbox can prevent Dawn from finding the physical adapter even when this host has one. An adapter-initialization failure is not shader evidence. Run the same focused command with adapter access before changing rendering code; an unavailable or skipped GPU does not count as a passing shader check.

Lifecycle and pixel behavior need different observations. Deferred source jobs expose replacement, cancellation and recovery without drawing. A recording renderer exposes warm targets, state restoration and disposal. Keep a real GPU integration test alongside those checks to compile the actual graph and read its output; a fake renderer cannot detect a black bake or an invalid pipeline. Browser checks cover the host boundary, including R3F remounts and agreement between the actual canvas and output encoder.

Do not write a scalar mirror of a shader and test that instead. That rule matters more now, not less: the mirror can pass while the graph it claims to describe drifts, and the remedy is to test the graph itself. terrainKernel.gpu.test.ts is the whole of it: the GPU tile producer's kernel against generateHeightfield on every zoo body and three Sol bodies at seven levels each, held to a bound stated as arithmetic about the kernel's two limits, and terrainBands.gpu.test.ts holds each of the eight bands to its own figure so a red whole names its band. terrainKernels.gpu.test.ts is the seed those grew from — a TSL port of faceToDirection and of the pcg3d lattice hash, each run on the GPU and compared with the CPU function, the float one to a named f32 bound and the integer one bit for bit.

Four things the harness owns, because each cost a round trip to learn:

• A readback is padded. readRenderTargetPixelsAsync returns the mapped staging buffer whole, with every row aligned to 256 bytes — an 8-wide RGBA8 target reads back with its second row at element 256. Pixels unpacks it, and puts row 0 at the top, which is where WebGPU's texture origin is.
• A pipeline that will not build does not reject. A draw builds it inside the backend's own validation scope and reports through three's console sink; compileAsync builds it with createRenderPipelineAsync, whose rejection carries the failure, and the backend discards that rejection. The harness brackets every verb in a validation scope of its own and listens on the sink, so a shader Tint refuses is a red test with the compiler's message in it.
• A compute kernel must guard its own index. A compute node dispatches whole workgroups of 64, and WGSL clamps an out-of-range write onto the last element, so a 1,547-cell kernel runs 1,600 times and the last cell holds whichever excess invocation ran last. A count that happens to divide by 64 never shows it.
• A stand-in texture must be filtered like the map it stands in for. DataTexture defaults to nearest, the WGSL builder reads a nearest texture with textureLoad and no sampler, and the gradient sample has no such path — so the ground's stand-in referenced a sampler that was never declared, and every mapless body's ground was a black frame. materials.gpu.test.ts holds each stand-in and a real map to the same program.

A headless GPU check is still not a real one. Nothing here observes presentation: one renderer failure reproduced only at devicePixelRatio 2, terrain selection is measured in display pixels, and a strobe is a property of what the compositor presented. Rendering work still ends at the browser procedure in Driving. The two layers are ordered, not alternative — the fast one answers whether a graph is valid and correct, the browser answers whether a frame is right.

Reconstruction and persistent terrain

The reconstruction policy and preference migration run in Node. Actual shader behavior belongs to the physical-GPU gate:

pnpm vitest run apps/game/src/render/picture.test.ts apps/game/src/state/preferences.test.ts apps/game/src/hud/hud.test.ts
pnpm vitest run --config apps/game/vitest.gpu.config.ts upscale.gpu radianceReactivity.gpu

The GPU fixtures exercise constant HDR preservation, repeated dispatch, attachment warm-up, resize, jitter cleanup after a throw, reactive coverage, rigid motion across an origin rebase and distant reversed depth. They do not prove compositor behavior or the appearance of a moving camera. Browser acceptance uses generated worlds outside Sol, with native, spatial, bilinear and temporal pictures at DPR 1 and 2. Hold the camera, photographic instant, surface settings and display size fixed. Let temporal history converge and record its phase; compare two boots of the same build before interpreting a plate's error. Also verify that changing anti-aliasing rebuilds the sensor without announcing a new renderer.

Persistent terrain tests cover complete key separation, corrupt payloads, cancellation, storage failures, bounded eviction and reopen. The external-world replay test reopens a real SQLite store, serves the same Proxima Centauri b requests without another generator call, and holds the world's state hash unchanged.

pnpm vitest run packages/workers/src/heightfieldCache.test.ts apps/headless/src/heightfieldStore.test.ts apps/headless/src/terrainCacheBaseline.test.ts apps/game/src/engine/terrainStore.test.ts
pnpm sim --ticks 0 --terrain-cache-clear --terrain-cache-descent 'g:milky-way/s:HIP70890/b:0'
pnpm sim --ticks 0 --terrain-cache-descent 'g:milky-way/s:HIP70890/b:0'
pnpm sim --ticks 0 --terrain-cache --terrain-baseline
IR_TERRAIN_CACHE_REPORT=1 pnpm vitest run --config apps/game/vitest.slow.config.ts gameEngine.descent.slow

The first two simulation commands are a cold/warm pair over the same requested regions; the third measures the zoo through the source. The disk store is .data/heightfields/tiles.sqlite. Cache hits avoid field generation, so retain direct generator tests. A drawn-only arithmetic change spends a version of generateHeightfieldTask, which heightfieldFingerprint.test.ts enforces by failing until it is spent. A warm descent cannot prove a generator implementation it did not execute.

Browser retention needs a real IndexedDB check as well. After a generated-world visit, inspect await engine.terrainCache(), reload through the driver with --keep-storage, revisit the same stance and compare hits and pending writes. await engine.clearTerrainCache() clears the archive independently of saves. Run storage-denied and quota-failure fixtures to prove regeneration still completes. Measure cold and warm runs separately, with the browser closed during CPU benchmarks. ADR-0045 states the invalidation contract.

The pictures are a fixture

A preset is an address, a framing, a lens and a held instant, so it is the one frame the renderer can be asked for twice. The thirteen shipped pictures are therefore the regression fixture for anything visible, in two halves.

pnpm presets:compare is the browser half. It serves the tree on a port of its own, opens each picture's public URL with the chrome cleared and standard output, waits until the sky and the ground have stopped arriving, and differences the frame against the committed plate — the frame a reviewer last accepted — printing the pixels over a 3% per-pixel threshold, the peak, and a verdict against a floor measured from two captures of one build. By default it compares the lit pictures only: a frame that is mostly sky or shadow compares its noise, so a plate has to be a fifth lit and the ledger's verdict has to agree, and the dark ones are named as skipped (--all compares them). --base <ref> serves that commit from a worktree of its own and photographs it instead, for plates known to be behind or for a same-machine control; --twice photographs the tree a second time and is the rig against itself. Each result has a reference | tree | difference pair under .data/presets/compare/, which is what a pull request attaches. A move is accepted with pnpm presets:plates <id>.

apps/headless/src/pictureLedger.test.ts is the Node half, in pnpm test. It takes the thirteen against the real catalog and snapshots what the camera decided — position, lens, altitude, a digest of the body, how lit the view is (the sun's elevation over a stance, the phase angle from orbit), and for a stance on the ground the terrain the plate's 480×320 asks the streamer for — into pictureLedger.json. Five of the pictures stand on generated bodies outside Sol, so a change to the procedural terrain or the detail floor moves a number there before a plate is taken. It also holds the world's state hash across all thirteen, which is rule 37 executed rather than described. pnpm presets:ledger rewrites the snapshot after a deliberate move.

The capability checks

Twelve executable assertions about the architecture, in packages/devtools/src/capabilities.ts. They run in the test suite, in pnpm sim --self-test, and in the browser via ir.selfTest().

They differ from unit tests in two ways:

1. They run against a whole assembled world, not a unit.
2. They report measurements, so a passing run is still informative.

Add one when a new claim about the architecture becomes something you would otherwise assert in prose.

What is not covered yet

Running them

pnpm test                       # regular Node suite
pnpm test:slow                  # terrain descent and galaxy calibration/population
pnpm test:gpu                   # the shader suite, on the real GPU — not in check
pnpm test:coverage              # the same suite, plus coverage/coverage-final.json
pnpm vitest run world.test      # one file
# One GPU file. The root config excludes the suffix, so the plain form above
# answers "No test files found" for anything named *.gpu.test.ts.
pnpm vitest run --config apps/game/vitest.gpu.config.ts materials.gpu
pnpm vitest                     # watch
pnpm check                      # every stage as one graph; --only gate is the Stop hook's four

Related

• Observability — the structures tests assert on
• Determinism — what the golden vectors protect
• AGENTS.md — the rules a test is defending
• Agent handbook