Deployment

Every process of a deployment loads the same configuration and differs only in the roles it runs. That leaves two decisions: how many processes of each role you run and where, and whether you run them with the better-compute serve host or embed the control plane in an application of your own.

Topologies

One process

better-compute serve with the default --roles all runs the API, every worker role, and the gateway when the configuration exports host.gateway. It suits a small installation, a staging environment or a single site. Nothing changes in the database when you later split it: leases and shard ownership already keep every job single-writer.

better-compute migrate
better-compute serve --port 3000 --no-migrate

Split roles

The production shape runs each role in its own processes, all from one configuration. This is what examples/postgres does:

better-compute migrate                                                    # once per deploy
better-compute serve --roles api --port 3000 --no-migrate                 # behind an L7 load balancer
better-compute serve --roles controllers,scheduler,sinks,iam --no-migrate # workers, no listener
better-compute serve --roles gateway --no-migrate                         # mTLS on :8443, behind an L4 balancer

Run as many of each as you need:

• api processes are stateless. Put them behind any HTTP load balancer.
• Worker processes register in the workers table and split the 256 shard:<n> leases by rendezvous hashing; singleton jobs (the CommandSweeper, outbox dispatch to each event sink, each IAM job) run under their own singleton:<name> lease on whichever worker holds it. Shard and singleton leases live 15 s and renew every 5 s.
• Gateway processes each hold a gateway:<id> lease (20 s, renewed every 5 s) and a row in gateways. A node's session row names the gateway that holds it, and dispatch joins on it, so any number of gateways can share a region.

With separate processes, give each gateway an internal control API (host.gateway.internal) and the api a notifier. Then the api wakes the owning gateway right after it enqueues a command, relays tunnels, and pushes revocations. Without them dispatch waits for the gateway's 1-second loop, and tunnels cannot open; doctor reports both (host.gateway.internal, host.gateway.notifier, host.gateway.tunnels).

A highly available region

Section 14 of PLAN.md sizes a highly available region as:

It also lists an object store, an OTLP collector with a time-series database, and KMS or Vault. None of those is required to start: the local CAs in the PKI bundle are the form the code ships, meant for development and small deployments.

Regions and sites

A region is a set of gateways with the same host.gateway.region (the value in sites.region). Daemons dial the gateways of their own region; sites need only outbound 443. Every region's gateways write to the same database as the api and workers.

An air-gapped installation (local object store, image mirror, bcd downloads) is designed in PLAN.md. Of it, the api's GET {basePath}/downloads/bcd/{version}/{os}-{arch} route is built: set downloads: { dir } or downloads: { blob } and the install script fetches bcd from the control plane itself. See installing bcd.

Load balancers

For networks whose TLS-inspecting proxies strip client certificates, the gateway can also serve a WebSocket fallback (v0.4) at wss://<publicHost>/v1/session (host.gateway.ws, default port 443), where the daemon proves its identity in-band.

The serve host

better-compute serve (packages/cli/src/host/**) is a thin Node.js host around compute. On start it:

1. loads the configuration and resolves the host export;
2. runs compute.initialize() (migrations, registry checks, module start) unless --no-migrate;
3. installs the IAM event handlers when host.iam is set (give it readTenants: iamTenantReader(iam, { credential }) so a tenant suspension or deletion reaches the whole subtree; see tenancy);
4. starts the worker roles with compute.start({ roles });
5. starts the gateway role: createGateway, the revocation feed, the mTLS listener, and the WebSocket and internal listeners when configured;
6. opens the HTTP listener for the api role.

Routes

The HTTP listener answers, in order:

Host routes must be absolute paths without a trailing slash, outside the base path, and must not reuse the built-in paths; doctor reports a clash as host.routes. GET /metrics is added only when host.metrics has a bearerToken of at least 16 characters; scrapers send Authorization: Bearer <token>. See observability.

Health checks

Use /healthz for liveness and /readyz for readiness of api processes. {basePath}/health also pings the store and answers { status, database, latencyMs, time }. Worker-only and gateway-only processes open no HTTP listener, so check them through the database instead: better-compute doctor reports overdue work and a stalled sweeper (workers.queue, workers.sweeper), gateways near their stream limit (fleet.gateway-headroom) and silent nodes (fleet.heartbeats), and the gateways row of a live gateway has a fresh heartbeat_at.

Shutdown

On SIGINT or SIGTERM serve stops in this order:

1. The gateway drains: it tells each daemon to reconnect elsewhere (Drain, with a random delay of up to 60 s), closes its listeners with a 2 s grace, and releases its lease.
2. The HTTP listener closes, so no new requests arrive.
3. Tunnel relays close.
4. The worker roles stop and release their shard and singleton leases, and the worker leaves the workers table.

A process killed without that still loses nothing: its leases expire after 15 s (20 s for a gateway) and another process takes them over, with a new epoch that fences out anything the old one still tries.

The gateway role

--roles gateway (or all with host.gateway set) needs:

• host.gateway.region;
• host.gateway.tls with key and cert from the gateway serving CA (pki issue-gateway) and ca, the node CA roots, or host.gateway.tlsFiles { key, cert, ca?, intervalMs? } naming those files;
• a way to authenticate daemons: pki.nodeCa and pki.join in the options (from the PKI bundle), or host.gateway.identity { roots, trustDomain, revocations }. The role refuses to start without roots and a trust domain.

The host fills in the gateway's store, services, work queue, events, report handlers and the tunnel JWKS. Anything else createGateway() takes goes in host.gateway.options, for example maxStreams (default 5,000) or vps's instanceStatusHandler in options.reports.status. With tlsFiles (and internal.tlsFiles for the control API), the gateway re-reads the files every 60 seconds and hot-reloads a renewed certificate for new handshakes while open sessions keep theirs; a certificate given inline in tls is read once at start, so after pki issue-gateway writes a new one, restart gateways one at a time. See PKI.

In this release the gateway process builds compute.gatewayServices from the shared configuration, so it loads the PKI bundle like every other process. See security.

Embedding in your application

betterCompute() returns a fetch handler and a Node handler, so you can serve the API from any framework:

import { getStack } from '@/lib/stack';

export const runtime = 'nodejs';
export const dynamic = 'force-dynamic';

async function handle(request: Request): Promise<Response> {
  return (await getStack()).compute.handler(request);
}

export { handle as GET, handle as POST };

apps/console is the reference: Better IAM and Better Compute in one Next.js process on one origin (/api/iam and /api/compute), with acceptIamCookie: true so the browser's IAM session cookie authenticates compute calls that carry X-Better-Compute: 1 from that origin. Next.js route handlers cannot take a WebSocket upgrade, so its server.mjs wraps Next and hands GET /api/compute/tunnels/{ticket} to handleTunnel from better-compute/server.

When you embed, you take over what serve does for you:

• call compute.initialize() before serving and compute.start({ roles }) in the processes that run workers;
• register retentionModule() from better-compute/postgres yourself; the CLI adds it when the configuration does not (see retention);
• serve your own health and readiness routes, and /.well-known/better-compute at the origin root if daemons use it (the server answers it under the base path);
• run the gateway with createGateway and serveGateway from better-compute/gateway, or run it with serve --roles gateway from the same configuration;
• call compute.stop() on shutdown so leases are released at once rather than after they expire.

The caller address

IAM policies can condition on request.sourceIp, so compute hands IAM the caller's address only as the host verified it (d269738, security review 14 L3). compute.nodeHandler (and so serve) takes the socket's peer address. With compute.handler, pass it yourself as the second argument, HandlerInfo:

// behind no proxy, the socket's peer is the caller
const response = await compute.handler(request, { peer: { address: socket.remoteAddress } });

The address becomes Credential.sourceIp, which remote IAM receives as x-forwarded-for; the client's own forwarding headers never reach IAM (e7092bf). Since 0eba0ad that holds for an embedded IAM too: the handler strips X-Forwarded-For, Forwarded, X-Real-IP and the other client-address headers and sets X-Forwarded-For to the verified address, and since 004579e the in-process API does the same with a credential's headers and sourceIp, so no path in lets a caller name its own address. The IAM caches key on the address as well, so the authentication cache (30 s) and allowed read decisions (10 s) of a token used from one address never serve it from another. Without an address, IAM sees none.

Behind a load balancer the socket's peer is the balancer, for every caller. Name your load balancers and reverse proxies in trustedProxies (0873d20), addresses or CIDR ranges, IPv4 or IPv6:

betterCompute({
  // …
  trustedProxies: ['10.0.0.0/8', 'fd00::/8'],
});

IPv4-mapped IPv6 peers (::ffff:10.0.0.1) are compared as IPv4. An entry that is not an address or CIDR is INVALID_CONFIG at startup, and the default trusts no proxy. serve takes the option from the configuration file like every other, and applies it to compute.nodeHandler; with compute.handler, pass the socket's peer as above and the option applies the same way.

Containers

The repository has no container image for the control plane (apps/docs/Dockerfile builds the documentation site). A container runs better-compute serve like any other process. It needs Node.js 22.12 or newer (22.18 or newer for a .ts configuration), the configuration file, and the environment below; serve handles SIGTERM.

Run better-compute migrate as a one-off job before rolling out new processes, and start them with --no-migrate. See configuration.

Upgrades

PLAN.md orders an upgrade as:

1. expand migrations (better-compute migrate);
2. workers and api;
3. gateways, one at a time, each drained first;
4. daemon waves within each pool's disruption budget (see rollouts);
5. contract migrations in the next release.

migrate refuses to run an older build against a database a newer build migrated, naming the migrations it does not know, so a rollback of the code needs the matching database. See database.

Examples

Each directory under examples/ is a workspace package with a better-compute.config.ts. All three run the same deployment from examples/shared/compose.ts: every module (provision and metal, vps, events, fleet, and rollout and audit when their keys exist) and a host export with GET /watch, GET /metrics and the gateway role when its certificate exists.

tests/examples-smoke.test.ts checks that each configuration loads, migrates and passes doctor, and that dev --sim adopts machines and serves /watch. The PostgreSQL and emulated-site walkthroughs are not run in CI beyond that.

The PostgreSQL walkthrough joins machines from a signed release (5792b2b): the api role serves goreleaser's dist through BETTER_COMPUTE_DOWNLOADS_DIR, daemon install-script takes --release-manifest and --release-pins so the script pins the binary's digest from the manifest, and the join token comes from join-tokens create --out. A site agent that drives IPMI BMCs adds --install-plugins bcx-ipmi, and --unprivileged runs bcd node beside a root bcd helper:

better-compute daemon install-script --pool pool_… --site site_… \
  --gateway-url https://gw.eu-1.compute.example.com:8443 --base-url https://compute.example.com \
  --bcd-version 0.1.0 --release-manifest dist/manifest.json --release-pins sha256/… --out install.sh
better-compute join-tokens create --pool pool_… --ttl 2h --max-uses 8 --out join-token

The shared composition reads these variables: