Gateway

The gateway is the process every daemon connects to. It terminates the daemons' mTLS, keeps their sessions, heartbeats and lane counters in memory, dispatches commands, ingests results, events, status and inventory, relays Join, Renew and secret requests, and relays interactive tunnels. It holds no CA and no signing key (principle P14), and PostgreSQL stays the only source of truth (P1): restarting a gateway loses open streams and nothing else.

@better-compute/gateway implements it in TypeScript on node:http2. It depends on core, protocol, inventory, pki and the server's types only; the server never imports it.

Running a gateway

better-compute serve runs the gateway role from the deployment's configuration; see deployment. Embedded in your own process:

import { createGateway, serveGateway, watchTlsFiles } from '@better-compute/gateway';

const gateway = createGateway({
  region: 'eu-1',
  database, // the compute store: postgres() or pglite()
  services: compute.gatewayServices, // JoinService, renewal, secret broker
  work: compute.context.work,
  events: compute.context.events,
  revocations, // pki RevocationSet: refused at open, pushed to live sessions
  reports: gatewayReportHandlers, // allocation transitions from @better-compute/server
});
await gateway.start(); // directory row and the gateway:<id> lease
const served = await serveGateway(gateway, {
  port: 443,
  tls: { key, cert, ca: nodeCaPem },
  nodeRoots: [nodeCa.certificate], // certificateAuthenticate; there is no weaker default
  revocations,
});
// A renewed serving certificate (pki issue-gateway) reaches new handshakes; open sessions keep theirs.
watchTlsFiles({ key: '/etc/better-compute/tls/gateway.key', cert: '/etc/better-compute/tls/gateway.crt' }, (tls) =>
  served.setTls(tls),
);

serveGateway, serveGatewayWs and serveInternal each return setTls({ key, cert, ca? }), a hot reload of the listener's certificate over its original TLS options; a key that does not match its certificate throws and changes nothing. watchTlsFiles (from @better-compute/gateway) polls the files every 60 seconds by default and reports a broken set to onError. serve does this for you with host.gateway.tlsFiles; see PKI.

Memory and PostgreSQL

Everything in the left column is rebuilt by the next Hello, which is why a gateway crash costs a reconnect and nothing more; see failure modes.

The gateways table is the directory: id, region, internal_url, software version, max_streams, current sessions, draining, started_at and heartbeat_at. start() upserts the row, and every lease renewal updates the session count and drain flag, which doctor reads for headroom.

The loops

gateway.status() reports each loop's consecutive failures, total failures, last clean run and last error, so a stuck loop can be alerted on. GET /v1/health on the control API returns the same object.

Dispatch by join

Routing is a join between commands and node_sessions at dispatch time, never a column copied onto commands:

SELECT c.*, s.session_epoch FROM commands c JOIN node_sessions s USING (node_id)
 WHERE s.gateway_id = $me AND c.state = 'pending'
   AND c.not_before <= compute_now() AND c.not_after > compute_now()
 ORDER BY c.node_id, c.seq LIMIT 1000
 FOR UPDATE OF c SKIP LOCKED FOR SHARE OF s

SKIP LOCKED lets several gateways and runs work side by side, and the FOR SHARE lock on the session row makes a dispatch and a concurrent Hello epoch bump serialize. A row is marked dispatched only when its session epoch matches the gateway's live session and the lane has credit; a session that closed after commit leaves the row dispatched, and the next Hello replays it. Because routing is a join, a reconnect needs no rewrite and a command enqueued during a reconnect cannot be stranded. Enqueues poke the gateway after commit; pokes are coalesced, so a burst costs a few dispatch passes, and a Hello pokes only when the node has pending commands, so a reconnect storm does not turn every Hello into a pass over every session.

Delivery rules (credits, cancel, refusals) are in delivery.

Group commit

A fleet sends many small frames. Writing each in its own transaction would serialize the gateway on the database, so command updates, event batches and inventory work are group-committed: frames from every session that arrive in the same I/O turn share one transaction, at most 500 entries each. A session still waits for its own entry before its next frame is handled, so per-session order holds, and a lone frame never waits for company.

Update transactions lock every command involved in id order, so two concurrent flushes never deadlock; event transactions lock session rows in node-id order. When a batch fails, each entry is retried alone, so one bad frame cannot fail the others. Inventory sections are not awaited by the frame queue at all: a section is an idempotent upsert, so the sections of every session in one flush (a node's sections sent back to back after Welcome, and other nodes' at the same time) share one transaction. They are ingested set-wise through the inventory port's ingestSections (inventory's ingestInventorySections): one INSERT … SELECT FROM unnest for the whole batch and one enqueueMany of the distinct nodes, where a corrupt section or an unknown node fails only its own slot. Heartbeat hash reports, and sections when the port has no ingestSections, still run each under its own savepoint. A section lost to an error is simply requested again at the next Hello.

The lease and fencing

Each gateway holds a leases row named gateway:<id>, renewed every 5 seconds with a 20-second TTL. The epoch of the lease rises whenever its owner changes. Every routing transaction (the Hello transaction, dispatch) starts by re-reading the lease FOR SHARE:

SELECT 1 FROM leases
 WHERE name = $name AND owner = $me AND epoch = $epoch AND expires_at > compute_now()
FOR SHARE

A zombie either still sees its lease as valid, or aborts; a takeover blocks until the zombie's transaction ends. If another owner holds the lease, or renewals have failed for half the TTL (10 seconds), the gateway fences itself: it forgets its lease and closes every stream UNAVAILABLE ("gateway fenced; reconnect"). Daemons redial, possibly to another gateway. Separately, the per-node session epoch fences routing and liveness writes; see epochs and fencing.

stop() closes every session UNAVAILABLE with a disconnect write, marks the gateway draining and releases the lease so a successor need not wait for the TTL. drain() sends every daemon Drain with a random delay up to 60 seconds and refuses new sessions, for rolling deploys: PLAN.md upgrades gateways one at a time with Drain.

Report and status hooks

The gateway imports no domain module. State transitions driven by what nodes report are injected by the host:

Each report hook runs in the transaction that stores the event and advances the ack, so a report is applied exactly once, and receives { nodeId, tenantId, journalId, seq, bootId, signed, attested, work }. signed says whether a node_sig came with it; attested is the signed envelope as received (the AttestedReport body bytes, kid and sig) for the worker to verify. Workers verify signatures; the gateway never treats one as proof. Status handlers may be a list, each run under its own savepoint. A failing hook rolls back to its savepoint and is published as node.report-failed; ingestion goes on. @better-compute/server exports gatewayReportHandlers with the capacity hooks, and better-compute serve always adds xidStatusHandler.

gatewayReportHandlers accepts a report only about an allocation placed on the reporting node: a report naming another node's allocation is ignored like an unknown allocation, so node A's session can neither bind, release nor fail node B's. AllocationBound passes the session's node id and the signed envelope to markAllocationBound, which stores the report as received (allocations.bound_report, bound_sig_kid, bound_sig); with attestedBindings the reservation worker verifies it before the reservation goes active. See attested bindings.

Certificate authentication

Join accepts clients without a certificate. Renew, Session and Tunnel need one that the authenticator accepts, and the node id always comes from the certificate. serveGateway requires either nodeRoots (for the production certificateAuthenticate) or an explicit authenticate hook; anything else is INVALID_CONFIG.

certificateAuthenticate verifies the chain to the node CA with the clientAuth usage, refuses revoked serials, optionally requires a SPIFFE trust domain, and reads the node id and site from the single SPIFFE URI SAN. It returns the serial, the role and the expiry, which drive node_certs.allowed_kinds, revocation, the Hello role check and the close at expiry, plus tenantControlled for a node-tenant certificate: such a session gets no secrets and no destructive platform commands. unsafeSanOnlyAuthenticate reads only the SAN and skips all of that; it is for tests, as is insecure: true (cleartext h2c).

Revocation

Revocations reach live sessions within about a second. In better-compute serve, pki's createRevocationFeed polls node_certs.revoked_at into the revocation set the gateway enforces, and the gateway subscribes to it: a session whose certificate serial is revoked receives RevocationNotice{reason, serials}, its tunnels close, and the stream ends UNAUTHENTICATED. New opens with that serial are refused at the handshake, and the Hello transaction re-checks node_certs itself, since the database is the revocation truth and the in-memory feed may lag a poll. The control API's revoke does the same for one node on demand. The gateway also revokes by itself in one case: a certificate that keeps alternating between two machines is quarantined with reason duplicate-identity (see duplicate identity).

Control API

Each gateway serves a small internal API on gateways.internal_url, HTTP/2 over the internal CA's mTLS, called by api and worker processes. Every call but tunnels is an accelerator: dispatch polls every second, specs are re-pushed by the sweep, revocations arrive through the feed and cancellations from the database, so correctness never depends on it.

Host wiring uses three helpers. gatewayControlClient(...) calls one gateway. gatewayNotifier(...) implements the core Notifier: publish('gw:<id>') pokes that gateway, and the runtime publishes it after an enqueue or a spec change commits. gatewayTunnelRelay(...) is the server's TunnelRelay: it opens a tunnel through the gateway named by node_sessions.gateway_id, caching gateway URLs for 30 seconds; an unregistered gateway is UNAVAILABLE.

Tunnels (v0.3)

The api authorizes the caller and issues a 60-second single-use exec ticket for one node, then asks the owning gateway to open a tunnel. The gateway verifies the ticket (signature against the api's JWKS, audience, its own jti set, which fails closed when full), sends OpenTunnel on the node's session, and waits up to 10 seconds for the node's Tunnel stream. The daemon verifies the ticket again before opening it (bcd and the FakeDaemon both do, each with its own jti set), and a daemon that refuses simply opens nothing. The gateway then only relays frames, with backpressure in both directions. It re-authorizes the caller every 60 seconds: a refusal closes the tunnel, and 2 minutes of IAM failures close it too. A tunnel lives at most 12 hours, and revocation or POST /v1/tunnels/close ends it at once. See tunnels.

Admission and scale

Three bounds protect a gateway from a reconnect storm: the handshake token bucket (200 opens and joins per second), maxStreams, and maxPendingHellos. The last one exists because the bucket can admit Hellos faster than the database commits them; an unbounded queue once starved the lease renewal until the gateway fenced itself and dropped every stream. With the bound, a refused Hello retries with jitter and the database queue stays short.

Each of these refusals (RESOURCE_EXHAUSTED) tells the daemon when to come back, so a herd spreads by the gateway's measure rather than by each daemon's doubling back-off. GrpcError carries retryAfterMs, sent as the standard grpc-retry-pushback-ms trailer:

The same hint reaches every transport: the HTTP/2 trailers-only or trailing status; the WebSocket upgrade refusal, as retry-after (seconds) and grpc-retry-pushback-ms headers, which connectGatewayWs turns into a RATE_LIMITED ComputeError with details.retryAfterMs (from grpc-retry-pushback-ms, else retry-after); and attachFrameConn, whose close(reason, { retryAfterMs }) hands it to simulated FakeDaemons. bcd and the FakeDaemon wait exactly that long; see opening a session.

PLAN.md's v0.1 gate is one Node gateway holding at least 5,000 mTLS sessions at 10-second heartbeats for 30 minutes, with dispatch p99 under 500 ms and RSS under 2 GiB; if it fails, the gateway is rewritten in Go on the same proto (decision D20). On a development workstation (PGlite, 60 seconds), the gateway in its own process held 5,000 mTLS sessions, one TLS 1.3 connection per daemon, with dispatch p99 26 ms, a gateway process RSS of 1,240 MiB (PGlite included) and 0 drops, and survived a reconnect storm with 5,000 of 5,000 back within 39 seconds. A 30-minute run of the same split setup (14ff010) held 5,000 sessions with dispatch p99 24 ms over 90,000 commands, 0 drops and every command of the storm succeeded, but the gateway process's RSS grew from 1,240 to 2,016 MiB while the JS heap stayed at 235 MiB; a 5-minute run with memory attributed traced the growth to PGlite's WebAssembly memory inside the gateway process, not the gateway's own. Every criterion is met on PGlite, and the gateway's own RSS stays near 1.1 GiB; the run with Postgres out of process, on the reference host, is still to confirm it. See scaling and bench/gateway-streams.

Several gateways

A region runs three or more gateways behind the L4 load balancer. Nothing is shared between them but PostgreSQL: each dispatches only the nodes routed to it (node_sessions.gateway_id), and a node that reconnects elsewhere simply moves its row with the next Hello. In the fleet simulator with the real control plane (bench/multi-gateway), killing one of three gateways re-homed its nodes with p99 of 1 second at 1,000 nodes and 13.25 seconds at 10,000, with no invariant violations. Peering between gateways is planned for v0.4.

Metrics

With a metrics registry, better-compute serve wraps the gateway services with meteredGatewayServices and registers gatewayCollector, which export (all prefixed better_compute_, no tenant labels):

The collector samples gateway.status() and gateway.sessions() at every scrape. See observability.

Testing without a network

gateway.open(peer, sink) drives a session without any transport, and attachFrameConn(gateway, peer, conn) serves one over the testing kit's in-memory frame stream, so the fleet simulator runs real FakeDaemons against a real gateway on simulated time. gateway.crash() simulates process death: loops stop and every session is forgotten without a database write, so the lease simply expires. See testing.

With a network, createDevStack({ externalTls }) serves the same gateway a second time over mTLS, with a dev serving CA and client certificates checked against the dev node CA, for daemons outside the test process. tests/e2e/bcd.test.ts runs the real Go bcd against it (see the Go daemon).