Workers

The question: how does expensive generation stay off the main thread without scattering new Worker() through the codebase? The answer: one abstraction — typed, versioned tasks dispatched by a pool that owns job ids, cancellation and instrumentation — behind a port so the same code runs with no worker at all.

Code: packages/workers/

The port pattern

packages/* has no DOM lib, deliberately. So this package cannot mention Worker — and it does not need to.

flowchart TB
    POOL["<b>WorkerPool</b><br/>queue · dispatch · cancel · measure"]
    PORT["<b>WorkerPort</b> (interface)<br/>post · subscribe · terminate"]
    BROWSER["browser Worker adapter<br/><i>apps/game — the one place<br/>a Worker is constructed</i>"]
    INLINE["in-process fake<br/><i>runs the real host loop<br/>on a microtask</i>"]
    NODE["node:worker_threads<br/><i>if a server ever needs it</i>"]

    POOL --> PORT
    PORT --> BROWSER
    PORT --> INLINE
    PORT -.-> NODE

    INLINE -.- NOTE["this is why the pool has<br/>real tests, in Node,<br/>with no worker environment"]
    classDef note fill:none,stroke:none,color:#64748b,font-style:italic
    class NOTE note
    style PORT fill:#0369a1,stroke:#0c4a6e,color:#fff

The inline worker is not a mock. It runs the same host loop against the same registry, serializing through the same envelopes — so a bug that only appears when something is not structured-cloneable still shows up in a Node test.

It is not the browser's fallback, though it used to say so. A browser without module workers gets no pool at all: the starfield survey runs on the main thread and terrain streaming stops until a pool exists, because TerrainStreamer returns early without one. The inline worker's four callers are all Node — the headless runner and three test files. It does go through structuredClone and honor its transfer list, so a payload that a real Worker could not clone fails here too.

A task is a function that happens to run elsewhere

flowchart LR
    DEF["defineTask({<br/>  name, version,<br/>  run(payload, ctx),<br/>  transfers?(result)<br/>})"]
    DEF --> A["called directly<br/><i>runInline — tests</i>"]
    DEF --> B["dispatched to a pool<br/><i>production</i>"]
    A --> SAME["identical results —<br/>asserted by a test"]
    B --> SAME
    style SAME fill:#065f46,stroke:#064e3b,color:#fff

Both sides of the boundary import the same definition, so the request shape cannot drift from the handler shape.

Today's tasks:

The seven universe.* tasks are createTaskRegistry() in packages/workers, the set every host serves. render.bakeAtmosphere is the game's own: its bake lives in packages/rendering, which sits at the same layer as packages/workers and therefore cannot be imported by it, and the headless runner would never ask for it. So the game's worker entry serves createGameTaskRegistry() — apps/game/src/workers/registry.ts, the shared set plus what only a browser client needs — and the prefix on the name says which set a task belongs to. ADR-0028 is the record.

universe.prepareSurface is on the pool because it reads nothing a heightfield request does not already carry, and paid on the main thread it lands inside a frame a player is watching: the floor is 85% of a 40 ms spike on Earth as the ground appears, and the drainage graph a wet body's field reads is a 117 ms frame there and a 139 ms one on focusing Earth in the planetarium. The graph comes back with the floor and seeds the main thread's memo. The consequence a caller has to know is that a body whose preparation has not come back has no ceiling to select against, so the streamer holds the ground back for the frames it takes rather than guessing one, and the GPU producer's requests and the planetarium's site list wait on SurfacePreparation the same way for a body that drains — a dry one's graph is a null nothing builds, so they do not wait for it. A host with no pool prepares synchronously; none of those have a frame to drop.

The envelope, and why versions travel with it

sequenceDiagram
    participant P as pool (main thread)
    participant W as worker
    P->>W: {kind:'request', job, task, taskVersion, payload}
    alt unknown task
        W-->>P: {kind:'failure', job, error}
    else version mismatch
        W-->>P: {kind:'failure', error:'version mismatch'}
        Note right of W: the page has been open<br/>across a deploy
    else ok
        W->>W: run(payload, {canceled})
        W-->>P: {kind:'success', job, payload, durationMs}
        Note over P,W: transferables move, they do not copy
    end
    P->>W: {kind:'cancel', job}

The worker also posts {kind:'ready', tasks} when it starts, so the pool can log which tasks a worker actually serves.

The version check matters more than it looks. An offline-first app is designed to be left open across a deploy. Without the check, a page could generate half a planet with algorithm v1 and half with v2 and never notice. With it, the job fails loudly.

The host validates the envelope with decode(decodeWorkerRequest, message) and drops malformed messages with a log line. Discriminating on kind alone would leave job, task and taskVersion unchecked at the transport boundary.

Instrumentation: two numbers, not one

flowchart LR
    Q["<b>queue latency</b><br/>enqueued → dispatched"] --> QF["a deep queue wants<br/>more workers or fewer requests"]
    R["<b>execution time</b><br/>dispatched → returned"] --> RF["a slow task wants<br/>optimizing"]

    classDef note fill:none,stroke:none,color:#64748b,font-style:italic
    class QF,RF note

They fail differently, so the pool measures them separately, over a rolling 64-job window, and the debug overlay shows both:

workers  8w · 0 active · 0 queued · 25 done · q 9.2ms · run 11.3ms

The clock is injected, so the pool has no host API dependency and timing tests are exact rather than flaky.

Cancellation

A job can be canceled whether or not it has started:

• Queued — removed from the queue, promise rejects immediately.
• Running — a cancel envelope is posted; the task polls context.canceled() at a sensible granularity (per cell in a region survey, not per star — the check should not cost more than the work).

The terrain streamer is the caller that makes this pay. submit hands it a JobHandle rather than a bare promise, it holds one per in-flight heightfield, and clear() cancels the whole window on a retarget. Dropping the answer is not enough: at the 128-job cap all but poolSize() of those are still queued, where canceling is a splice and the work never happens, and leaving them there put up to 50 s of ground nobody will see ahead of everything the next view wants. The few actually running finish — generateHeightfield polls nothing and cannot be interrupted mid-field — and their answers are discarded by the streamer's epoch. A cancellation arriving back at its own caller is that caller's decision, not a failure, so the rejection is matched on and not logged.

A source is a port over the pool

The streamer does not ask the pool for heightfields. It asks a HeightfieldSource — kind, available, an optional maxLevel, submit(surface, request) → JobHandle, the arguments generateHeightfield takes — and poolHeightfieldSource(pool) is the pool wearing that interface. The other implementation is createTileProducer(renderer) in apps/game/src/render/terrainProducer.ts: a TSL compute kernel that produces sixteen tiles a dispatch on the GPU, installed by App once the renderer is ready and its pipeline has compiled behind the boot cover (ADR-0023).

The port is what makes the two interchangeable in the one place that matters. submit returns the same JobHandle either way, so clear() cancels a GPU batch's queue exactly as it cancels the pool's, and the epoch discards a late answer from either. A source that fails — a lost device, a pipeline that will not build — sets available = false, the streamer's next request goes to the pool, and the session continues on the canon. ?producer=cpu refuses the GPU source outright, which is how the pool's own figures are re-measured; ir.terrain().producer says which one answered.

The interface lives here, in packages/workers, rather than beside the renderer, because the pool implements it and this layer cannot see the one above. Nothing in it names a renderer, a device or a buffer: a surface and a request in, a handle out. The caller hands over the SurfaceParameters it holds rather than a flattening of it — that is the identity surfaceKernel memoizes on, and what lets the GPU source batch by it — and the pool's adapter is the one place the seed becomes a string, through encodeSurface, because the pool is the one source with a wire to cross.

What is deliberately not here

Related

• Streaming — the main consumer
• Determinism — why worker order cannot change the universe
• Persistence — the same codec discipline at the storage boundary