Requirements and sizing

This page collects what each part of a deployment needs to run, and what is known about how big it has to be. The short version: the control plane is Node.js processes and one PostgreSQL database; a node is a Linux server with systemd and cgroup v2, plus the NVIDIA driver for GPUs and KVM with an IOMMU for virtual machines. Sizing beyond that is not validated yet, and the page says where the numbers come from.

The control plane

Software

The control plane has no container image of its own; any host or container that runs Node.js runs better-compute serve (see containers).

PostgreSQL

PGlite (PostgreSQL in WebAssembly) is for development, tests and the simulators: one connection, transactions serialized, inside one process. It cannot be shared by separate processes, so it cannot run a split or highly available deployment. Details are on database.

Connections. Each process holds its own pool, 10 connections by default (postgres({ max }), 1 to 100), so the total is roughly the pool size times the number of processes. PLAN.md lists connections among the first limits at scale and puts PgBouncer in front of PostgreSQL.

Reference machines

bench/REFERENCE.md fixes the machines the PLAN.md §16 gates are quoted on:

Not fixed yet (decision D15): the instance type or CPU model, the network between hosts, the hosts for the api, workers and load generators, PgBouncer's placement and the PostgreSQL configuration.

What has been measured

Sizing each tier

Recommendations, derived from the defaults and the numbers above, not validated on production hardware:

PLAN.md's design point is 10,000 nodes and 100,000 GPUs per region on one PostgreSQL, and its estimate of the load at that size (liveness writes, reconnect storms, commands, events, placements) and the order in which limits are expected to bite are on what breaks first.

Storage. audit_events, inventory_snapshots and usage_windows grow with time and are partitioned by month; the retention module prunes and drops what every cursor has passed (see retention). No growth rate per node has been measured.

Nodes

Platform

No minimum kernel version is checked anywhere in bcd. The full list of bcd doctor checks is on the daemon overview; run it before the first join (the install script does, and only warns).

GPU hosts

bcd reads GPUs through nvidia-smi and needs no NVML library or cgo (cb987e5): a host build without nvidia-smi reports no GPUs, and bcd doctor's nvml check warns.

On WSL 2 (/dev/dxg without /dev/nvidiactl), doctor says the paravirtual GPU runs CUDA but supports no VFIO passthrough, gpu.reset, MIG or DCGM, and the nvidia-driver unit never installs a Linux driver there (2e71a60).

VPS and sandbox hosts

Site agents and Kubernetes

A bare-metal site agent needs its provisioning network, the iPXE builds and a ramdisk image you supply, and its own systemd unit for the privileged ports; see site agents and the ramdisk. bcd on an existing Kubernetes cluster runs as a DaemonSet with its own host mounts; see bcd on Kubernetes.

What has run on real hardware

Supported GPUs

The inventory normalizes each GPU's NVML name into a product and a family through a versioned table (gpuModels, version 2, in packages/inventory/src/gpu-models.ts). Pools, policies and meters use the family. A name the table does not know maps to the family other, which raises the GpuModelUnknown condition and keeps the node from scheduling until a release adds it. The normalization rules are on GPU naming.

"Supported" here means the inventory names the card and pools can hold it. What a card can do beyond whole-GPU holds depends on the card: MIG needs a MIG-capable part (see fractional GPUs), passthrough needs the IOMMU setup above. Only NVIDIA GPUs are inventoried.

Network and ports

Nodes only dial out. Everything the control plane sends a node travels down the session the node opened.