Roadmap

The version plan from P0 to v1.0 with each phase's deliverables and exit gates, and the honest state of every item in the repository today.

Better Compute is built in phases, each defined in PLAN.md §16 by its deliverables, its acceptance criteria and its exit tests. A phase closes only when its exit tests are green in CI; the benchmark gates are measured on the reference hardware in bench/REFERENCE.md (a gateway host with 8 vCPU and 32 GiB, PostgreSQL 17 on 16 vCPU and 64 GiB NVMe).

The phases were planned in order, but the work did not stay in order: the TypeScript control plane for every phase up to v0.5 exists in the repository, and the Go daemon covers the v0.1 node agent, while the daemon's later modes and the pieces that need a PostgreSQL server or real hardware do not exist yet. So no phase after P0 is formally closed yet. This page tracks each item as:

Done: in the code, with tests, and listed in the changelog.
In progress: partly there. The note says which part is missing.
Not started: nothing in the repository yet.

Three things hold every phase open

The Go daemon is not released, and only its node agent runs end to end. bcd join, bcd node and bcd status work and are tested end to end against a real gateway (tests/e2e/bcd.test.ts, the fake-hardware build in WSL), including a node console tunnel, and the node agent also converges break-glass SSH (access). bcd site and bcd ramdisk are written and covered by Go unit tests only. The instances converger, instance tunnels and the IMDS run end to end only on fakehw builds (tests/e2e/bcd-vps.test.ts); the libvirt-kvm and cloud-hypervisor runtimes of host builds (362317c, 2598f0a) are tested against fake tools only. The release pipeline exists but no release has been cut. GPUs are read through nvidia-smi (cb987e5) with no go-nvml backend. Self-update, the WebSocket fallback and bcd helper (self-update included) are done and exercised through the TypeScript FakeDaemon. bcd sim runs many real node agents in one process (fakehw builds only). One real host has run a host build of bcd (round 6, 2026-10-04): a WSL 2 workstation with a GeForce RTX 5090. tests/e2e/bcd-host-real.test.ts (BCD_E2E_REAL=1) joins it to a dev stack, adopts it, reads its real inventory and GPU, exports the GPU's live metrics over OTLP and answers bcd status and bcd doctor. That bcd ran as an ordinary user without [helper], so it binds no GPUs: no real /etc/cdi spec, hold slice, passthrough or MIG has run yet. See running bcd on real hardware and what bcd does today.
No PostgreSQL server has run the suites. Local runs use PGlite (PostgreSQL 18 in WebAssembly). Suites that need real concurrency skip unless BETTER_COMPUTE_POSTGRES_URL is set, and every benchmark number below is from PGlite on a development machine, which the gates do not count.
The hardware gates (PLAN.md D15) have not run. They need one NVIDIA box, a self-hosted KVM runner and the reference benchmark machines, and rented A100 or H100 capacity before the v0.5 exit.

At a glance

Phase	Theme	State
P0	Scaffold	Done
v0.1	Adopt and reserve	Control plane done; bcd's node agent done and tested end to end, not released (one real GPU host has joined, been adopted and reported its GPU; a real GPU hold, a data-center GPU server and the install-script run are open; GPUs come from `nvidia-smi`, cb987e5); PostgreSQL and the reference gate runs
v0.2	Bare metal	Control plane done and tested in simulation; `bcd site` and `bcd ramdisk` written (Go unit tests only, not released), never run against the control plane or on hardware
v0.3	VPS and access	Control plane done; tunnels and the `access` section done in bcd (not released); `libvirt-kvm` and `cloud-hypervisor` written in bcd and tested against fake tools only
v0.4	Scale and HA	Mostly done in simulation; `bcd helper` done, self-update through it included; the numbers need the reference PostgreSQL
v0.5	GPU depth and ecosystem	Control plane done ahead of schedule; sandboxes served in full on the control plane; hardware exits not run
v1.0	Freeze	Started: the API surface freeze check, bcd's SLSA3 release pipeline, the N-2 skew matrix and the upgrade-in-place scenario exist (FakeDaemons), and the destructive-batch quorum is built end to end, its producers included; the security review and npm provenance do not
v1.1	CPU, memory and Kubernetes	Mostly in the code (host configuration, host resources, slices, MIG and slot holds, Kubernetes workers, bcd as a DaemonSet), tested against fakes only

Every package is on npm at version 0.1.1 on the latest dist-tag (0.1.0 was the first release, on 2026-10-07) and accepts Better IAM >=0.2.0 <0.4.0 from npm. Provenance attestations come once CI publishes from the GitHub repository. bcd has no released binary yet.

P0: Scaffold

P0 was committed with pnpm check green, and its contracts are frozen: proto/bettercompute/agent/v1, the @better-compute/core contracts, the v0.1 schema and the v0.1 rows of PLAN.md §9.2 now change only by additive, reviewed edits.

Deliverable	State	Notes
git and `git init`	Done	With GitHub Desktop's bundled git; git is not on the machine's `PATH`
A pnpm workspace mirroring Better IAM's	Done
Scripts `check-deps`, `check-sql-now`, `umbrella`, `sync-better-iam`, `proto-gen`, `authz-matrix`	Done	Under `scripts/`; `pnpm lint:repo` runs the repository rules
Better IAM vendored	Done	0.1.0 as tarballs in `vendor/better-iam`, pinned through `overrides`, until Better IAM was on npm; the releases depend on Better IAM from npm (`>=0.2.0 <0.4.0` since 0.1.1)
Packages `core`, `protocol`, `postgres` (with `./pglite` and `compute_now()`), shells of `server`, `client`, `cli`, `testing`, stubs of `inventory`, `provision`, `metal`, `vps`, and the umbrella	Done	The stubs have since been implemented
`protocol` with TypeScript and Go generated code committed	Done	TypeScript by `scripts/proto-gen.mjs`; Go in `daemon/gen/` by `scripts/daemon-proto-gen.mjs` (protoc-gen-go v1.36.12, protoc-gen-connect-go v1.21.0), with `--check`
`daemon/` skeleton and goreleaser configuration	Done	The module builds and its tests pass (`node scripts/daemon-test.mjs`; cross-compiled and run in WSL on Windows). goreleaser is configured and checked in CI, but no release has been cut
CI: Ubuntu and Windows on Node 22 and 24, a PostgreSQL job, a Go job; the IAM contract test; `bench/REFERENCE.md`	Done	`.github/workflows/ci.yml` and `daemon.yml` are defined, and since then `nightly.yml` (the simulation matrix with the 10k cold start, the IAM-guarded suite in both IAM modes, the whole suite on Postgres, a green-night streak) and `release.yml` (bcd with SBOMs, cosign and SLSA3 provenance); see CI. This page does not track CI runs

Exit	State
`pnpm check` green, including `umbrella --check`	Done locally
`buf breaking` against `proto/baseline.binpb`	Done: the baseline is committed and CI runs it
Packed smoke importing `better-compute/{inventory,provision,metal,vps}`	Done: `test:packed`
Freeze of `agent/v1`, the core contracts, the v0.1 schema and the v0.1 §9.2 rows	Done

Of the day-one verifications, PGlite's btree_gist is verified. The go-nvml mock (go-nvml is not a dependency yet), the KVM runner and CDI on target hosts are not, for want of hardware.

v0.1: Adopt and reserve

v0.1 adopts servers you already run (a brownfield join with a token) and reserves their GPUs. It was split into five workstreams.

Workstream	State	Notes
W1, Go daemon: join, session, journal, Renew with proof of possession, NVML inventory, `devices` and `access` convergence, CDI holds, `diag.nvml`, `node.reboot`, release artifacts, install script	In progress	Done: `bcd join` (sealed token, verified `JoinResponse`), `bcd node` (session over pinned mTLS, bbolt journal, Renew with proof of possession at 60% of the lifetime, the `devices` converger with CDI holds and node-signed `AllocationBound`, the `access` converger for break-glass SSH (CA, node-bound principals, a native KRL), `inventory.refresh`, attested `diag.nvml`, `logs.collect`, `gpu.reset`, self-terminating `node.reboot`, signed-intent checks, the server time mark, the trust version in `Hello` and `StatusReport`), and beyond W1: tunnels with node-verified exec tickets (instance tickets never reach the host), `SecretRequest` and `WorkloadTokenRequest`, XIDs from the kernel log (kept across a bcd restart in the same boot) and host pressure, and the v0.2 `bcd site` and `bcd ramdisk` and the v0.3 `instances` converger (see below). Also `bcd status` over the unix socket, `bcd doctor`'s host checks, procfs/sysfs inventory, and the install script (`better-compute daemon install-script`); `tests/e2e/bcd.test.ts` runs it against a real gateway. The GPU inventory is read through `nvidia-smi` without cgo (cb987e5: MIG, NVLink, topology, ECC and row remapping, tested against captured output; no go-nvml backend and no NVML events). Open: release artifacts (the release workflow, with SBOMs, cosign signatures and SLSA provenance, exists, but no tag has been released), and a run on a real host
W2, gateway and PKI: mTLS sessions, the serving CA, epoch fencing, liveness batching, dispatch by join, resume rules, `BUSY` re-queue, local CAs, the `join-token` attestor, JoinService, the downloads route	Done	`@better-compute/gateway`, `@better-compute/pki`
W3, store, capacity and inventory: schema, `compute_now()`, work queue, outbox, leases, CommandSweeper, `InlinePlacer`, atoms, rollups, `gpu-models.json`, conformance and torture tests	Done	`@better-compute/postgres`, `capacity`, `inventory`
W4, IAM bridge: the catalog, plugin and inbox, projection and resolver, `embedded()`, IAM jobs, flag cache, usage sink, IamReconcile, contract test, authz matrix	Done	`@better-compute/iam`
W5, server, client, CLI, simulator: `operation()`, the method registry, the v0.1 groups, the typed client, the CLI (`dev`, `iam setup`, `daemon install-script`, `pki issue-gateway`), the deterministic simulator and the real-time harness	Done	Since then the CLI has commands for every API group (PLAN.md §9.3; 44915e5, e2fe6f8, bb89083, d1e83ca, 573b0b0, 171d9be), and a test ties every route they call to the method registry (fb6304f). See command groups

Acceptance	State	Notes
A real `bcd` installed by the install script, plus 3 FakeDaemons, join and match the golden inventory	In progress	The FakeDaemon half is done, and Go's `fakehw` matches the TypeScript section hashes. A real `bcd` (fakehw, in WSL) joins and is adopted in `tests/e2e/bcd.test.ts`, but not installed by the script (there is no release binary) and not beside FakeDaemons
An org user reserves 4 GPUs from an offered pool; the CDI spec lists exactly the booked UUIDs, which `reservations.get` returns	In progress	`tests/e2e/spine.test.ts` runs it with embedded Better IAM, the example policies, a real gateway and three FakeDaemons: the node's CDI spec, held devices and env file list exactly the booked UUIDs. `tests/e2e/bcd.test.ts` binds a 2-GPU reservation on the real bcd and checks its CDI spec file under the fake root. A CDI spec on a real GPU host has not been checked
`ReserveBoundedH100` allows 8 GPUs and denies 9; `BoundedExtend` denies 72 h + 1 h	Done	IAM bridge and contract tests
`gpu.h100.hours` reaches Better IAM billing without duplicates across worker restarts, even with a deleted identity in a batch	Done	Idempotency keys on every usage window; the sink re-attributes deleted identities
Place-to-active p99 under 2 s with 1k FakeDaemons in the real-time harness	Done, indicative	1.02 s at 5 placements/s on PGlite; the inline placer saturates one pool between 5 and 10/s
No upstream Better IAM change	Done	The supported topology of PLAN.md §12.1

Exit test	State
The spine end to end (`tests/e2e/spine.test.ts`)	Done with FakeDaemons: join with sealed tokens, `diag.nvml`, adopt, reserve 4 GPUs as a policy-bound org user, bind, release; 9 GPUs, 73 h, non-pool-users and other organizations denied. `tests/server-spine-v01.test.ts` covers the same path in-process
`tests/capacity-concurrency.test.ts`: 1,000 concurrent operations never double-book, and the torture test	Done on PGlite (interleaved); the truly concurrent PostgreSQL half skips without a server
Faults: gateway kill mid-command runs nothing twice, journal loss fails instead of replaying, a reboot completes on rejoin, invariants I1, I2, I5, I8	Done in `tests/sim/fleet-gateway.test.ts` and `fleet-stub.test.ts`
Security negatives: an IamPort spy records zero IAM calls on fleet paths over 10 simulated minutes	Done (`tests/sim/iam-negative.test.ts`)
Security negatives: authz matrix at 100%, resolver timeout denies, a failed `authorizeMany` fails the list, join tokens honour `maxUses` and TTL, node A cannot get node B's spec, Renew without proof of possession is rejected, cross-tenant requests leak nothing	Done

The v0.1 gates are where the phase is really held open:

Gate	State
Gateway: one Node gateway holds 5,000 mTLS sessions at 10 s heartbeats for 30 minutes, dispatch p99 under 500 ms, RSS under 2 GiB	Sessions, dispatch and RSS criteria met on a workstation; the reference run remains. On a development workstation (PGlite, 60 s, the gateway and its store in their own process): 5,000 mTLS sessions, one TLS 1.3 connection each, dispatch p99 26 ms, gateway process RSS 1,240 MiB, 0 drops, and a reconnect storm with 5,000 of 5,000 back in 39 s and every command settled once. A 30-minute run of the same setup on PGlite met every criterion (dispatch p99 24 ms over 90,000 commands, 0 drops), RSS only just: it grew to 2,016 MiB with the JS heap flat at 235 MiB, which a 5-minute attributed run traced to PGlite's WebAssembly memory in the gateway process. The 30-minute run with PostgreSQL 17 on the reference host, which is to confirm the RSS criterion, has not happened, so D20 (a Go gateway) stays formally undecided. See scaling
IAM lane: embedded Better IAM on adapter-postgres sustains 50 admissions/s with lock wait under 50%, and the ceiling is recorded	Not started: the suite exists (`tests/bench/iam-lane.bench.ts`) but needs PostgreSQL, so D18 (proposing R1 upstream) stays open
Placements/s recorded	Done on PGlite: 35.7/s for the inline placer at 1k nodes

v0.2: Metal

v0.2 takes servers from a BMC and network boot to a deployed OS and back, with verified wipes.

Deliverable	State	Notes
`bcd site` and `bcd ramdisk`	In progress	Written, with Go unit tests. `bcd site`: Redfish BMC operations, BIOS settings, RAID layouts and firmware updates from the verified image cache included (944221c), each only under its own `bmc.*` kind, other BMC drivers as hosted bcx plugins with `bcx-ipmi` for IPMI power, boot device and inspection (51806ab, 7818f19; tested against a fake `ipmitool`; no vendor driver), `site.httpProxy`, boot entries, and the boot server (proxy DHCP, TFTP, iPXE scripts with signed discovery assertions under the boot lease, the image cache). `bcd ramdisk`: the `discovery` join from its kernel command line, `disk.wipe`, `deploy.image`, `boot.kexec`, `diag.dcgm`. Open: a run against the control plane or on hardware, a release, and a ramdisk image and iPXE builds. The FakeDaemon plays both roles in the tests
`provision` and `metal()`	Done	The workflow engine, profiles, the `metal` group and reconciler
`siteTokens` and the SiteBoot lease; `joinRequests`	Done	A standby site agent is promoted within 30 s of the active one's loss
`discovery`, `provisioned` and `x509pop` attestors	Done
Signed destructive commands and the trust bundle; node-signed attested results	Done	Every command is worker-signed when `commandSigning` is configured; a gateway-forged wipe result is refused
`SecretRequest` sealed secrets, the `secrets` API and BMC rotation	Done	Two-phase BMC password rotation. bcd sends them for BMC credentials, `site.httpProxy` secret headers, discovery tokens and ramdisk user data, and signs `bmc.rotate` results
The image catalog and site caches; FleetConfig plan and apply	Done
XID and health classification, and `diag.dcgm` acceptance	In progress	Classification, quarantines and remediation are done, and bcd reports XIDs from the kernel log and runs `gpu.reset` and `diag.dcgm` (`dcgmi diag`, in the ramdisk and on nodes); no DCGM run on real GPUs yet
The revocation-push SLA	Done	A revoked node's session closes within 5 s (about 1 s in tests)
Emulation CI	In progress	`examples/emulated-site` describes metal on QEMU/KVM with sushy-tools and `bcd site` on a Linux host with KVM; nobody has run it (it also needs iPXE builds and a ramdisk image; BMC addresses may name each VM's system on one emulator since 4e4c1d7), and there is no KVM runner (D15)
Walkthrough 1, Ubuntu autoinstall	Done in simulation	`tests/extensions/ubuntu-autoinstall.test.ts`

Acceptance and exit	State
20 unattended enroll, ready, deployed, released, ready loops on UEFI and BIOS VMs	Done in simulation; not run on VMs
Killing the site agent or the worker mid-deploy resumes cleanly (`metal-chaos`)	Done (`tests/metal-chaos.test.ts`)
Killing the active site agent mid-PXE: the standby takes over boot within 30 s	Done in simulation (`tests/metal-site-failover.test.ts`): the standby is promoted when the dead agent's lease runs out, the machine boots from it, and the boot step's BMC commands ran once
A gateway-forged `AllocationBound` is rejected	Done (`tests/e2e/attested-bindings.test.ts`, through a real gateway with signing FakeDaemons); see attested bindings
Every `ready` follows a verified wipe; `metal-lifecycle`	Done
Site agents refuse other sites' steps	Done
A `SecretResponse` cannot be opened with another key or after its TTL	Done
Unsigned images are rejected; a ramdisk certificate cannot open a `node` session; unsigned destructive commands are rejected	Done

v0.3: VPS and access

Deliverable	State	Notes
`vps()`: instances, flavors, provider accounts, overcommit, IPAM	Done	The control plane and the instance reconciler
Host runtimes `libvirt-kvm` and `cloud-hypervisor`	In progress	They run inside bcd on each host, under its `instances` converger. `libvirt-kvm` is written (362317c): a host build with `virsh` runs instances through `virsh` and `qemu-img` (qcow2 overlays on sha256-verified base images, NoCloud seeds, VFIO GPUs, host-passthrough CPU, snapshots and restores), but it has been tested only against a fake `virsh` and `qemu-img`, never on a real KVM host. `cloud-hypervisor` is written (2598f0a): a host build with the `cloud-hypervisor` binary runs one VMM per instance as a confined `systemd-run` unit over its REST API, with the same overlays and seeds, VFIO GPUs, memory snapshots and restores; it has been tested only against a fake VMM API and fake tools, never on a real KVM host. Security review 12's two high-severity findings are fixed (933c51b): taps are persistent and bridged before the VMM starts, with an `inet bcd_taps` backstop for a tap off its bridge, and both runtimes refuse a tenant image or restored snapshot that names a backing file or an external data file; the VMM's unit sees only its own instance and base image and is bounded by its flavor. fakehw builds run a file-backed runtime, and the FakeDaemon's `FakeHypervisor` stands in for the control-plane tests; see delivery
`provider-hetzner` (Cloud, and Robot with `bootstrap`)	Done	Outside the umbrella package
Single-tenant containers	In progress	Pools declare `policy.runtime` and must set `policy.singleTenant` for container runtimes, and vps refuses a container on a pool that serves other tenants; the host runtime needs bcd
Tunnels with daemon-verified tickets, `nodes.issueSshCert`, the `access` section	Done	Including node-bound break-glass certificates and an SSH KRL for revocation. bcd (not released) re-verifies tickets against the pinned JWKS and its journaled `jti` set, runs node consoles on a real pseudo-terminal, dials instance `ssh` and `tcp` tunnels to the instance's own address, answers an instance `console` ticket with the instance's serial console (from its runtime: the pty `libvirt-kvm` gave the serial port, or `cloud-hypervisor`'s serial socket; f8273ef), which `tunnels.open` issues as op `console` with `instanceId`, decided as `compute:instances:exec` (798e710; `better-compute console --instance`, the console's `/instances/<id>/console` page, 191898b), advertises `tunnel.instance-dial` when it runs a runtime, and converges `access` into sshd; it refuses `exec` into a guest, which needs a guest agent. The FakeDaemon answers tunnels with an echo shell and has no `access` converger
Webhooks and SSE	Done	`@better-compute/events`
`mcp`	Done	`@better-compute/mcp`
Delegated confirmation	Done
The `cloud-iid` attestor	Done
Walkthrough 2 (a VPS provider plugin) and walkthrough 3 (Talos)	Done in tests	`tests/extensions/acme-cloud.test.ts` (against an emulator, live and from a recording), `tests/extensions/talos.test.ts`

Acceptance and exit	State
200-instance churn on 3 emulated hosts with no leaks	Done (`tests/vps-churn.test.ts`) against emulated hosts in the test process
Host loss recreates instances per policy	Done
An agent reserves over MCP with a person's confirmation, retrying with the same key	Done (`tests/mcp-e2e-confirmation.test.ts`)
A replayed exec ticket is rejected by the gateway and the daemon	Done: end to end with the FakeDaemon as the daemon, and in bcd's own unit and interop tests, which also check that the `jti` is journaled
`delegation:revoke` closes the agent's stream within 5 s	Done (`tests/e2e/access.test.ts`)
Multi-tenant container offers are rejected; no tenant VM runs bcd	Done
`tests/e2e/vps.test.ts`	Done with FakeDaemon hosts on the dev stack (real server runtime and gateway, fake hypervisors): cloud-config through the secret broker, 200-instance churn with no leaks, host loss and recreation, container offers refused, and a tenant VM cannot join as a node with anything it holds. Real KVM hosts need bcd's `libvirt-kvm` or `cloud-hypervisor`, neither of which has run on one yet

v0.4: Scale and HA

Deliverable	State	Notes
`PoolLeasePlacer`	Done	Opt-in: `capacityModule({ placer: 'pool-lease' })` (D22)
Shard rebalancing	Done	A silent worker's shards are taken at lease expiry
Several gateways	Done in simulation	Three real gateways behind the fleet simulator, with gateway kill and zombie fencing
WebSocket fallback, signed frames only	Done	Gateway and FakeDaemon sides, and bcd (6cd48be): `[websocket] url`, `proxy_ca_file` and `audience`, the gateway's proof checked against the pinned serving CA, renewal without a client certificate over `POST /v1/renew` on the WebSocket listener (06cb334), no tunnels, command-signing roots required; `daemon install-script --websocket-url` writes the section (0ea4ab4). `tests/e2e/bcd.test.ts` runs bcd over the real WebSocket gateway, renewal included (4b2d5aa). See the fallback
A/B self-update and Rollout	Done, not released	Signed releases and wave rollouts, and bcd's node half (de9f380): signed manifests under `[update] release_pins`, a release floor, A/B slots, `upgrade.json` with a 120 s deadline, confirmation on the first `Welcome`, a swap back without one, a crash-loop rollback unit, and `daemon.upgrade`; `tests/e2e/bcd.test.ts` updates a real bcd through a real exec (fc0ca6a). A confirmed release is never undone: bcd refuses every older binary, the other slot's included (c139b9c), `rollouts.abort` with `rollback` returns only unconfirmed upgrades (8347a3d), and a bad release is fixed by rolling forward to one with a higher sequence. No release has been cut. See self-update
`bcd helper`	Done	Privilege separation (28dc2ae): with `[helper] socket` and `user`, `bcd node` runs unprivileged and `bcd helper`, as root, serves it typed operations over a unix socket checked with `SO_PEERCRED` and the caller's executable: the files bcd owns by kind and id (CDI specs, allocation env files, the sshd drop-in, CA, KRL and principals), `firewall.apply` and `firewall.check` on `bridge bcd_instances` and `inet bcd_taps`, `gpu.reset` of the host's own GPUs, `reboot` and `ssh.reload`, each logged. `tests/e2e/bcd.test.ts` runs an unprivileged node beside a real helper through a GPU hold's CDI spec. `daemon install-script --unprivileged` sets it up (169c459): the `bcd` account, `bcd-helper.service`, a `User=bcd` drop-in, the socket in `/run/bcd-helper`. Self-update through the helper (2d67b25): the `update.` operations, root's `[update] slots_dir`, the helper re-verifying the manifest and hashing the staged binary as it copies it, its own deadline watchdog; the e2e delivers a signed build to an unprivileged node. Security review 11 (c139b9c): no downgrade through the other slot, the staged file opened beneath its directory without symlinks or FIFOs, the node's claimed version never trusted, at most 32 requests at once, and root-only directories on the way to the slots and socket; its follow-ups (9e71b1f) keep the entrypoint out of `data_dir`, adopt only a root-owned binary, and answer `busy` over the cap. The instance runtimes go through it too: `libvirt-kvm`'s checked `libvirt.` operations (ed3c7d2) and, since 7fe74a1, `cloud-hypervisor`'s `ch.` (taps, VFIO groups and VMM units root builds from the instance record it checks; tested against fakes only). Not helper operations: PLAN.md's `runtime.` and `mig.apply`; bcx plugins need a root node. See privilege separation
Signing of all commands and specs	Done	With `pki.commandSigning`, every command and every NodeSpec generation is signed by a worker key; the gateway relays stored signed specs verbatim, and the FakeDaemon verifies them (`intent.signedSpecs`). bcd verifies them too once its trust bundle holds command-signing roots: every spec and every command, whatever its kind (0d42de8; the FakeDaemon since 12ccfa1)
Remote IAM (`remoteIam`, `computeIamHost`, `replicaProjection`, `postgresInbox`)	Done	U0 is an upstream Better IAM item outside this repository. With `remoteIam({ refresh })`, an admission waits (at most 2 s) until the IAM host's replica has replayed compute's commit and re-reads the row, failing closed with `UNAVAILABLE`; `doctor` warns `iam.refresh` when it is missing
Remote-IAM MCP	Done	`remoteMcpIam()` gives `@better-iam/mcp`'s gate an IAM reached over HTTP; see MCP
Remote-IAM event intake over webhooks	Done	`host.iam.webhook`: Better IAM's signed subtree webhooks into `iam_inbox`, for deployments that cannot read the IAM database. See IAM events
Scheduler worker role	Done	`Reconciler.role` in core, the runtime's scheduler role with its own shard membership, and the pool-lease placer `capacity.placer` running in it
Partitioned history	Done	`audit_events`, `inventory_snapshots` and `usage_windows` by UTC month (`core/0018_partitioned_history`); `retentionModule()` keeps months created ahead, prunes and drops emptied months; audit months are archived by hand. See retention
Merkle audit checkpoints	Done	`@better-compute/audit`, with `better-compute audit verify`
`bcd sim`	In progress	Written (aa7501c): `bcd sim --nodes N --seed S` runs N real node agents on fakehw in one process against a gateway, each with its own identity, journal and session; `tests/e2e/bcd.test.ts` joins five with one five-use token. Open: the nightly `bcd sim --nodes 10000` against real gateways and PostgreSQL through Toxiproxy. `better-compute sim` remains the FakeDaemon fleet simulator; see bcd sim
R1, if the IAM-lane benchmark demands it	Not started	Waits for the v0.1 IAM-lane gate

The v0.4 acceptance is set at 10,000 nodes and 100,000 GPUs on the reference PostgreSQL. What exists is simulation:

Criterion	State
Gateway-kill recovery under 60 s	Met in simulation: at 10k nodes, the 3,334 re-homed nodes are back at a p99 of 13.25 s simulated time, with no invariant violated
0 false unreachable marks	Met in simulation (`tests/sim/v04-scenarios.test.ts`)
At least 200 placements/s	Indicative: with the pool-lease placer on PGlite, 1,092/s end to end at 1k nodes; at 10k nodes 464/s in the latest recorded run (`bench/placements/results`), and about 1,058/s in the measurement taken when the pool index started loading devices per node on demand (not yet a recorded result); see scaling. The benchmark's sustained mode (`BENCH_PLACE_SUSTAIN_SECONDS=1800` for the acceptance, gate `BENCH_PLACE_GATE`, default 200, judged on the slowest 10 s window, I1 checked) measured 699/s, slowest window 639/s, over 30 s at 1k nodes on PGlite
Cold start under 3 minutes	Met in simulation: `runColdStart` in `@better-compute/testing` runs the scenario on the real control plane and 3 real gateways in simulated time. The recorded 10k run (`bench/cold-start`) had every node connected, adopted and schedulable at 80 s and back 40 s after a power cycle, with 0 unreachable marks and 0 invariant violations; see scaling. The CI-sized run (24 nodes, `tests/sim/cold-start.test.ts`) checks the same in every build. The wall-clock run needs the reference PostgreSQL
Command p99 under 1.5 s with polling only, PostgreSQL CPU under 50%	Not measured: needs the reference PostgreSQL
Suites green in both IAM modes	Done: `pnpm check` runs `pnpm test:iam-remote`, which reruns the IAM-dependent suites against IAM over HTTP, the authz matrix's IAM side included (every §9.2 action and resource pair decided through `bridge.remote`)
Every §15 v0.4 scenario green on 3 consecutive nights	In progress: the scenarios run in the simulator, and the `nightly` workflow runs `sim:nightly`, the IAM-guarded suite and the Postgres suite every night and counts consecutive green nights; this page records no nightly history

v0.5: GPU depth and ecosystem

Most of v0.5 landed early, on the control plane. Sandboxes joined the phase on 2026-09-24 (D24): all 15 methods are served, with templates, TTL expiry, warm pools, snapshots and exec, and the exit test runs on emulated hosts. bcd enforces the template network, and a host build runs sandboxes on libvirt-kvm (362317c) or cloud-hypervisor (2598f0a), both tested against fake tools only: no sandbox has booted on a real host yet.

Deliverable	State	Notes
MIG, time-slice and MPS atoms	Done	See fractional GPUs; applying a MIG layout on a GPU is bcd's
Calendar reservations and BookingRepair	Done	See gang and calendar
Gang scheduling	Done
Spot	Done	See spot and preemption
`dedicated-root` metal and `binding: 'node'`	Done	See dedicated root
TPM attestor	Done in code, software TPM only	The verifier and a software TPM producer are done. `bcd join --attestor tpm` joins with the host's TPM (`/dev/tpmrm0`) and no token (8545688), and `daemon install-script --attestor tpm` writes a script for it (a006d9f). A fixture contract checks bcd's evidence and answer against `tpmAttestor` in both directions, and `tests/e2e/bcd-tpm.test.ts` joins a fakehw bcd with its software TPM through the real JoinService, reclaiming the same node after a reinstall (db9bd46); since 8a9af54 with an EK certificate from a test manufacturer root the attestor verifies (no `allowUncertifiedEk`) and the node key in that TPM. bcd keeps its node key in the host's TPM when it can open one (965c14d, 50e18f9, 0ba2a79). Pools can require it with `policy.joinAttestors` (1516728). Since cf24876 a `tpm` join also proves the node key: bcd makes it in the attestor's TPM, the AK certifies it over `sha256(csr)`, and `tpmAttestor` refuses a join without that proof by default (`nodeKey: 'certified'`), recording `nodeKey: 'tpm'` in `node.joined` and the join's audit row. Since d94f1f3 the proof holds through renewal: the certificate names the AK (`urn:bc:tpm-ak:<nameAlg>:<digest>` beside the SPIFFE ID), and every renewal must carry that AK's certification of the next key (`RenewRequest.key_attestation`), refused otherwise with no downgrade; bcd makes the next key in the TPM or fails the renewal loudly, and the fakehw e2e renews and keeps the statement. Open: a join, a renewal and a TPM node key on a hardware TPM with a real manufacturer's EK certificate. See bcd with its TPM
Optional workload identity	Done on the control plane	Workload OIDC tokens and the IMDS token exchange in pki, `WorkloadTokenRequest` over the session, the workload broker, and the issuer's discovery document and JWKS in `serve`. bcd sends the request from its own metadata service (`[imds] listen`), which serves the instances of any runtime it runs; end to end it has run only with fakehw builds' file-backed runtime. See workload identity
bcx and `driver-sdk`	In progress	The TypeScript SDK runs drivers out of process with mutual TLS. bcd hosts sha256-pinned plugins from `[plugins] dir` and runs `driver.invoke` in them (`daemon/internal/bcx`, with `bcx.Serve` as the start of a Go plugin SDK), tested by Go unit tests only. Plugins run sandboxed on Linux (d132fef, 3872c6b): executed from the file that was hashed, each under its own uid (root only when named in `[plugins] privileged`), with `no_new_privs`, a seccomp filter on every thread and optional cgroup limits, and restarted when they exit (9006eee: back-off from 1 s to 1 min, re-hashed and sandboxed again). The first Go plugin in the repository is `bcx-ipmi`, the IPMI BMC driver a site agent hosts (51806ab), shipped as its own release archive (348ff56). Open: an end-to-end run of bcd with a plugin, health checks, and the control plane merging advertised drivers into its registry. See drivers
`provider-proxmox`	Done
console-kit and `apps/console`	Done	Including an instance's serial console page (191898b) beside the exec terminal
Walkthrough 4, a Kata runtime in Go	In progress	bcd serves runtimes from bcx plugins: a hosted driver of kind `runtime` becomes an instances runtime (`list`, `apply`, `remove` over `Invoke`; no snapshots), covered by Go unit tests (494de69). No Kata plugin is written. See walkthrough 4
Sandboxes (PLAN.md §3.7): templates, TTL, warm pools, snapshots, exec over SSH certificates	Done on the control plane: 15 of 15 methods served	Added to v0.5 by D24. Served: `sandboxTemplates` create, list, get, update and delete (with `env` and `network`), `sandboxes` create (with `fromSnapshot`), list, get, extend, snapshot, exec and delete, and `sandboxSnapshots` list, get and delete, with migrations `vps/0004` to `vps/0012`, the `vps.sandboxes` reconciler (TTL expiry, `sandbox.*` events, warm pools, snapshot settling and purging), the `vps.sandbox.warm` meter, the instance guard and vps's tenant teardown. The CLI's `sandbox` and `sandbox-template` commands, eight MCP tools and the console's Sandboxes page use them. bcd enforces `network` (6db49c8) and runs sandboxes on `libvirt-kvm` (362317c) and `cloud-hypervisor` (2598f0a). Open: a run of either on a real host (both are tested against fake tools only). See sandboxes

Exit	State
The MIG torture test and the real-hardware nightly	The torture test runs on a simulated MIG pool; nothing has run on hardware (D15)
A `dedicated-root` lease and release yield a new identity generation	Done
A third-party TypeScript driver runs on a node via bcx	Done with a FakeDaemon as the node (`tests/e2e/bcx.test.ts`): a module profile's `power.*` steps go out as `driver.invoke` commands through the real engine, queue and gateway, and run in a pinned bcx plugin process. bcd now hosts bcx plugins and runs `driver.invoke`, but no test runs a TypeScript plugin under bcd yet
IMDS cross-instance access fails	Done with FakeDaemon hosts (`tests/e2e/imds.test.ts`, `tests/pki-workload.test.ts`): one instance's session token fails for another and on another host, a host gets nothing for an instance placed elsewhere, forwarded requests and replays are refused
`tests/e2e/sandboxes.test.ts`: 500 sandboxes expire on emulated hosts with the v0.3 zero-leak diff; a warm claim reaches `ready` without a placement; a snapshot round trip restores memory state; another tenant's snapshot is refused; instance methods on a sandbox's instance are refused	Done with FakeDaemon hosts (a8f3dfc): `tests/e2e/sandboxes.test.ts` runs the real server and gateway with three FakeDaemon VPS hosts (fake `cloud-hypervisor`, which pauses). It proves that a warm claim reaches `ready` without a placement (the paused instance resumes on the same node and allocation, booted once) and the pool refills to `warm.min`; that 500 sandboxes (`E2E_SANDBOXES`) with a 60 s TTL expire on their own, each publishing `sandbox.expired` then `sandbox.deleted` once, with the zero-leak diff clean (domains, overlays, taps, IP leases, bookings, scalar claims) and every readiness and expiry counted in the metrics; that guest memory captured on one sandbox comes back in one created `fromSnapshot`, and the stored object is deleted with its sandbox; and that deleting the warm template drains its pool until the hosts are empty. Since bfb1d45 it also asserts the last two bullets: `instances.stop`, `start` and `delete` on a sandbox's instance are refused `INVALID_STATE` naming the sandbox (`get` still works), and a user of another tenant, with its own template on the same pool, gets `NOT_FOUND` restoring from the snapshot and reading it (`sandboxSnapshots.get`); `tests/vps-sandboxes.test.ts` covers both against emulated host reports too. Nothing runs on a real host

Sandboxes

Sandboxes are part of the product beside GPU and CPU capacity. Decision D24 (PLAN.md section 17) was taken on 2026-09-24: build a dedicated sandbox API, a sandboxes group on vps(). PLAN.md drops "a sandbox product" from the v1 non-goals, describes the flow in section 3.7, adds the sandboxes and sandboxTemplates rows to section 9.2, and makes it a v0.5 deliverable with its own exit test (v0.5). What exists and what does not:

Item	State	Notes
The sandbox catalog	Done	`compute-sandbox` (parent: the pool; attributes `templateId`, `vcpu`, `memoryGiB`, `gpus`, `ttlSeconds`, `fromSnapshot`, `warm`), `compute-sandbox-template` and `compute-sandbox-snapshot` (attributes `sandboxId`, `templateId`, `includeMemory`; owned by the sandbox's owner, whoever took it), the `compute:sandboxes:*` actions, and 15 method rows at phase v0.5, all registered. See actions and resources
Templates	Done	`sandboxTemplates` create, list, get, update and delete: a hosted pool, a flavor, an image and the pool's own runtime, checked once; user data; `defaultTtlSeconds` (1 h) and `maxTtlSeconds` (24 h); `warm: { min, max }`. Also in FleetConfig (`sandboxTemplates`). See templates
Sandboxes with a lifetime	Done	`sandboxes` create, list, get, extend and delete. One decision on `compute-sandbox/{draftId}`, the pool's budget check, then one hosted instance admitted, placed and charged like `instances.create`. The TTL starts at admission; `extend` stays within `maxTtlSeconds`; the `vps.sandboxes` reconciler expires sandboxes into an ordinary instance delete
Warm pools	Done	`warm.min` instances per template, kept `paused` on runtimes that can pause (`libvirt-kvm`, `cloud-hypervisor`, `firecracker`) and `stopped` otherwise, at most 4 created per template per pass; claimed with `FOR UPDATE SKIP LOCKED` only after the sandbox's own allow, without a new placement, and only when made from the template's current shape; drained beyond `warm.max` and when stale; metered as `vps.sandbox.warm` until claimed
Paused instances	Done	`desiredState: 'paused'` in the lifecycle, `InstanceObservedState.PAUSED` in host reports, `RuntimeTraits.pause`
Instance guard and teardown	Done	`instances.start`, `stop`, `resize` and `delete` refuse a sandbox's or a warm instance with `INVALID_STATE`, and so does `tunnels.open` for every op (`reason: 'sandbox-managed'`); `instances.list` hides warm instances; `vps().tenantDeleted` deletes every instance and snapshot of a deleted tenant, sandboxes' and warm ones included
Hosted runtimes on your own servers	In progress	bcd has the `instances` converger (fab94d4), `instance.snapshot` and `restore_from` (be1e7d9), serves runtimes from bcx plugins (494de69), and runs `libvirt-kvm` on a host build with `virsh` (362317c: overlays on verified base images, NoCloud seeds, VFIO GPUs, snapshots with memory, restores checked for runtime and CPU model) and `cloud-hypervisor` on one with the `cloud-hypervisor` binary (2598f0a, the same features). Both are tested only against fake tools and no Kata plugin exists, so no sandbox has booted on a real host yet
Snapshots and restore	Done on the control plane	`sandboxes.snapshot` with `vps({ snapshots: SnapshotStore })`, the `instance.snapshot` command kind, the `sandbox_snapshots` table, `sandbox.snapshot.ready`, `.failed` and `.deleted`; `create({ fromSnapshot })` of the caller's own snapshot on the same runtime and, for memory, the same `cpu.model`; `sandboxSnapshots` list, get and delete, so a kept snapshot can be removed. bcd runs snapshots and restores on fakehw builds and, as a tar of the flattened disk, the seed and `virsh save`'s memory state, on `libvirt-kvm` (tested against a fake `virsh`), and with cloud-hypervisor's own memory snapshot on `cloud-hypervisor` (tested against a fake VMM API). See snapshots
Exec	Done	`sandboxes.exec` (tunnels module): an `ssh` ticket and a certificate of at most 15 minutes, never past the sandbox's expiry, from the deployment's sandbox SSH CA (`pki add-sandbox-ca`, `tunnelsOptions`), which guests trust through vps's user data; the tunnel is re-decided on the sandbox every minute; with `certifyHostKey`, guests present host keys the same CA certified and the CLI pins them. Exec is the CLI's `better-compute sandbox exec`, which runs your OpenSSH client through the tunnel. No agent in the guest. Deferred: the SDK helpers PLAN.md section 3.7 describes (`sandbox.exec(cmd)`, `files.put`, `files.get`), because they need an SSH client in the SDK, a new dependency decision. See running code inside
Template `env` and `network`; the `vps.sandbox.warm` meter	Done	`env` from the tenant's secrets, sealed into `/etc/sandbox/env`; `network` carried as `InstanceSpec.network_policy` and enforced by bcd with host nftables rules (6db49c8; `private` and `none` need the pool's IP range and place only on hosts labelled `bc.dev/cap.instance.network-policy`, cfdb918); `vps.sandbox.warm` from placement to claim (`iam setup --meters` creates the meter). Template secrets must be the tenant's own, written with `secrets.put` and not fleet-scoped, and naming them also decides `compute:secrets:manage`
Clients	Done	The CLI's `sandbox`, `sandbox-template` and `sandbox-volume` commands, eight MCP tools with gate rules (none takes a snapshot or runs code), the console's Sandboxes page, and FleetConfig `sandboxTemplates`. See MCP

v1.0: Freeze

v1.0 freezes the API and the protocol, guarantees N-2 version skew, adds an external security review and SLSA level 3 builds, and publishes to npm with provenance from CI. Its acceptance: upgrading from v0.5 disturbs no running allocation, with no open critical or high findings.

Item	State	Notes
API freeze	In progress	`scripts/api-surface.mjs`, run by `pnpm lint:repo`, fails on a breaking change to any served method or error code against `generated/api-surface.json`; additions are recorded with `pnpm api:surface`. See the API surface freeze
Protocol freeze	In progress	`agent/v1` and `common/v1` have been frozen since P0 (`buf breaking` in CI); the draft `site` and `driver` packages are not yet
N-2 version skew	In progress	`createGateway({ protocol })` sets the range a gateway speaks. `tests/skew-matrix.test.ts` runs the matrix on real dev stacks: FakeDaemons built like the N, N-1 (v0.4) and N-2 (v0.3) lines (`daemonSkewProfiles` in `@better-compute/testing`) are welcomed, adopted and refuse the kinds they predate `UNSUPPORTED`, and a daemon below the gateway's minimum gets `upgrade_required`. No older real bcd build runs in it, and only protocol version 1 is defined. See version skew
SLSA level 3 builds	In progress	`release.yml` builds bcd with SBOMs, keyless cosign signatures and SLSA3 provenance; no release has been cut
npm publishing with provenance	In progress	0.1.0 and 0.1.1 are on npm (`latest`: 0.1.1), published from a verified clean checkout, with Better IAM `>=0.2.0 <0.4.0` from npm. Provenance attestations come once `ci.yml`'s publish job publishes from the GitHub repository
External security review	Not started	A process item
Destructive-batch quorum (PLAN.md §13)	In progress	Fleet-wide destructive batches need k-of-n operator approval, checked on the machine. Done: the protocol seams (`CommandBody.batch_id`, `BatchApproval`, `BatchProof`, the trust bundle's `approver_keys`, `approver_threshold`, `quorum_tiers` and `unbatched_destructive_per_hour`, `BATCH_UNAPPROVED` and `UNBATCHED_LIMIT`; 525c199, 4f11548, 081928a), the pki primitives (approver keys, the `(subject, kind)` scope tree, approvals, `verifyBatchCommand`; 36387c8, 2d2fe6f), `pki.approvers` in the server's signed trust bundle (b816600), the server's `quorum` module with `commandBatches` get, list, scope, approve and cancel (ce067e0), the command queue's `createBatch` and `enqueue({ batchId })`, which holds batched commands until approval (6067791), metal batch scopes and steps that declare what they send (add7e0e), `better-compute batch keygen` and `batch sign` (0210671), and enforcement in the FakeDaemon (4b48f08) and bcd (e7ddb38). Since then: quorum review #6 (payload-bound leaves, site-qualified BMC subjects, the trust domain in approvals, each leaf run once; 77cdf4f, d566cbe, 8b0f50c), the gateway attaching approvals and proofs to batched commands (40be696), `batch list`, `show`, `approve` and `cancel` with the approver-side checks (b3fd712), approver-set changes signed by the current approvers (`batch sign-approvers`, `pki.approvers.setSignatures`; 6a7f1b7, edbd63f), a trust bundle version that follows its content (9c40cd5), and enqueue refusing an unbatched destructive command up front (`batch-required`, 508728e). Then the review 7 fixes: every command signed once signed intent is in force and `site.httpProxy` destructive (0d42de8, 12ccfa1), approver-set signatures bound to the trust domain (4835f4f), and leaves that carry their payload for approvers (`quorum/0007_leaf_payload`, 2ca0ba4), then approver sets with their own epoch (c6214dc), a first set pinned out of band (`--approver-set-sha256`, 9b7a199), and leaves limited to the tenant's own machines with payloads of their kind's type (fc2df7d); and the producers: metal's per-workflow batches and `metal.bulk` (f66e236), `metal.power` (478025a), the health reconciler's remediations (e594a4f) and the MCP server's `propose_command_batch` (171ad2f). Open: the batches built-in workflows create leave their leaves' payloads unbound, since the bytes are only known later (a BMC credential rotated mid-workflow, user data rendered by an earlier step); PLAN.md §13 (a1bd0aa) names binding a leaf to the operation instead of its bytes, for example `payloadMatch { operation, args }` checked by the agent after decoding, as a post-v1.0 option. Since then, review 10: one epoch-adoption rule for every change, the held set reported in `Hello` and `StatusReport`, recorded by the gateway (fa39201, `core/0026`) and checked by doctor and `pki trust-version`, a node holding no keys counting as lagging (5fe7cbf, e171f0d, 6612496, 08eb62e), the pin enforced by bcd, operations grouped by payload digest in the approver CLI (b118616), and `batch create` and `batch members`. See the quorum
Upgrading from v0.5 disturbs no running allocation	Done in simulation	`upgradeInPlace(rig, { migrations, rollDaemons })` in `@better-compute/testing` runs expand migrations, workers and api, each gateway killed and revived, then the daemons, and reports every disturbed allocation. `tests/sim/upgrade.test.ts` (6 nodes, 3 gateways, 6 active reservations, a signed rollout to 0.6.0) finds none, bills the hour exactly, and keeps I1, I2, I5 and I8; it also runs two instances and a sandbox on two VPS hosts (`ControlPlaneSim` `extraPools`), and every VM stays running on its host with the same boot count while the sandbox keeps its expiry. A second scenario (80195df) starts from a v0.5-era database without the quorum tables, upgrades, then turns the quorum on: no allocation moves, setting approvers moves no spec generation, the trust bundle's version bumps exactly once, every daemon takes the set, a `node.reboot` queued before the switch is sent but rejected `UNBATCHED_LIMIT`, and a new unbatched one is refused `batch-required` (see turning the quorum on); FakeDaemons only

CPU, memory and Kubernetes (v1.1)

PLAN.md §3.8 (a760805) designs the complete node daemon for v1.1: compute books a host's CPUs, memory and huge pages with its GPUs, the node enforces exactly that, bcd applies the host's configuration, and a reservation can hand its GPUs to a Kubernetes cluster. The principle stays P1: compute decides who gets what, and the node enforces exactly that and reports it. Most of it is now in the code, on both sides. Every node-side piece is tested against fakes only (a fake host, a fake root, a fake kubelet, fake VMMs); none has run on a real machine or cluster.

Item	State	Notes
Host configuration applied by bcd	Done in code, mostly fakes	The host converger applies twelve units in order, `kernel-modules`, `sysctl`, `hugepages`, `cpu`, `iommu`, `nvidia-driver`, `fabric-manager`, `gpu-settings`, `mig-layout`, `gpu-sharing`, `container-toolkit` and `k8s-join` (6c2343f, 7579184, 1cad924, 3fc3628, 3fd1c41), a failing one never holding back the rest (c21ef84), drift repaired by a pass every 10 minutes (3c0fe02), and settings that wait for a reboot reported as the informational node condition `RebootRequired` (c21ef84, 0aea5c0); under `bcd helper` each unit is applied as root only from a signed spec (03c023d, 2d18327), and root writes only under root's directories, with its records in `/var/lib/bcd-hostconfig` (f8e13ef, 02485b3, 1bb78d6). Provision renders every unit from profiles and from pool policy (8ce9140, 1f0f683, 7db110a, 09669d3, e4b0c01), and the pools API refuses a host policy nodes cannot take (37dcd02); `[host] units = false` for hosts configured by something else (a193249), which `bcd helper` also honours from root's own root-only `config.toml` (3fa9021). `TestRealKernel` runs sysctl, THP and 2 MiB pages on a real kernel (60143bf); `tests/e2e/bcd-host.test.ts` runs a pool's units through the control plane (91f5e81). See host configuration. Open: no driver install, MIG repartition, boot argument or cluster join has run on a real machine
Host resources on every host	Done	`capacity.hosts` writes `vcpu`, `memoryGiB`, `hugepages2Mi` and `hugepages1Gi` for every host, under `policy.hostReserve` and the overcommit ratio (8a7f8eb); vps's `memoryGiB` leaves out the huge page pools (9d55b36). Inventory reports per-NUMA huge pages and isolated CPUs, with the labels `cpu.count`, `cpu.sockets`, `cpu.isolated`, `numa.nodes`, `memory.totalGiB`, `hugepages.2Mi` and `hugepages.1Gi` (badb8c2). Inventory does not report available memory
Reservations book CPU and memory	Done	`reservations.create` takes `resources { vcpu, memoryGiB, hugepages1Gi }`, booked on the devices' node in the same statement; without it a request books `policy.perGpu` or the pool's smallest per-GPU share, for whole GPUs and, since 004579e, for MIG instances and slots as fractions of a GPU (a21c1c1). The share is the floor, and anything above it is charged to `vcpu` and `memoryGiB` quota (41826a2, 004579e). See host resources
Enforcement on the node	Done in code, fakes only	Device assignments carry the booked resources as limits, with a cpuset on the devices' NUMA nodes that leaves out the host's reserved CPUs, the lowest non-isolated, `policy.hostReserve.vcpu` (c5a419c, b6b804e); bcd admits a hold against the host's memory beside its VMs, failing it `resources-exhausted` for re-placement (31555bf), creates `bcd-alloc-<allocation>.slice` before the CDI spec names it, restores it after a reboot and stops it once empty, retrying every pass (1d03c28, 260e33c, e01d7ff), and reports it in `AllocationBound.cgroup_parent`, which `reservations.get` shows as `binding.cgroupParent` (6aa412b, c81f230, 19dbf60). VMs get `InstanceSpec.placement` on both runtimes (a184ec2), pinned guests take isolated CPUs first (b6b804e), guests of memory-overcommitted pools get a balloon (073324a), and bcd refuses an instance start the host cannot back (`ResourcesExhausted`, 0dd122a), which vps records as `HostResourcesExhausted` and answers by placing a not-yet-started instance on another host after a minute, at most three times (564f543). `bcd doctor` checks the cgroup controllers, huge page pools, `vfio-pci` and systemd (ab26e87). Reservations with host resources, and `kubernetes` bindings, place only on a bcd that advertises `devices.limits` or `binding.kubernetes` (409f783). Open: the workload manager must place workloads in the slice
Holds of MIG instances and slots	Done in code, fakes only	bcd holds `gpu.mig` and `gpu.slot` atoms with parent-aware conflicts and no hold beside a VM's GPU (46a61db), and `gpu.reset` and `diag.dcgm` refuse GPUs those holds or VMs use (539c4ce)
Kubernetes	Done in code, fakes only	Pools name the cluster in `policy.kubernetes`; the `k8s-join` unit installs, joins and keeps the Node's labels and taints, with the bootstrap token released by binding rule 3 (88ecb07, 7579184, e93bf9c); `binding: 'kubernetes'` (a21c1c1) binds like a hold (d02ba4f), and bcd's device plugin advertises exactly those devices as `bcd.dev/gpu`, `.mig` and `.slot` and deletes pods that keep released ones. See Kubernetes workers. The DRA driver `gpu.bcd.dev` offers the same devices through one ResourceSlice per node instead, chosen by `gpuApi` (7d24836, fa8611e, e4b0c01), and neither API offers a GPU that went bad (6f5a99f). bcd also runs as a DaemonSet on an existing cluster (8df8c70, 2c9f953), with hosted join tokens that land its nodes `tenant-controlled` unless a platform-owned pool is platform-operated (a27156a, 259d82c), in hosted mode with the device plugins or DRA and its `bcd.dev/*` labels but no join (1e5f96b), and `kubernetes manifest` renders it with operator-given pins (a2eb28f, d2cd813); see bcd on Kubernetes. Kubernetes bindings get no slice: the kubelet enforces pods' limits, and their booked limits count in host admission (e01d7ff). The pools API checks `policy.kubernetes` (37dcd02). Only a pool's owner books `binding: 'kubernetes'` (3825ec7), and `ForeignGpuExposure`, `PodResourcesUnavailable` and `KubernetesReleaseStuck` are NoSchedule node conditions, the first also degrading the node's reservations (ebd9806, 4570dfe). Security review 2 (8bcbe92, f88bcdb): a released GPU stays in use until no pod, prepared DRA claim or pod process holds it, failing closed on any node that ever was a worker, and `ForeignGpuExposure` also follows the toolkit the container runtime runs, other GPU DRA drivers and, hosted, an unreadable runtime configuration. The DaemonSet mounts only the kubelet sockets and credentials bcd uses, never `/var/lib/kubelet` or `/etc/kubernetes` (27f2c1b, a5eee84), and the runtime and toolkit configuration read-only (d11810c, 3775f46). `daemon/Containerfile` bundles a pinned, checksummed `kubectl` v1.34.1, which bcd runs as the kubelet (9bdd43a, e7d0765); CI builds the image and server-side dry-runs the manifests on kind (ac1ca4b). Open: no real cluster has run it (the CI job is a dry run), and pod requests are not derived from the reservation
Sandboxes and VM hosts	Done in code, fakes only	Flavors take `placement { numaAffine, pinned, hugepages }`, which sandbox templates inherit; vps chooses the NUMA node and pinned CPUs and marks a host that cannot honour them `PlacementDegraded` (9d55b36). Sandboxes, placed flavors and volume-bearing instances land only on a bcd that advertises the enforcing capability, `instance.ssh-host-only`, `instance.placement` or `instance.volumes` (82f4f51). Memory snapshots of GPU sandboxes and of sandboxes with a read-write volume are refused up front (2a09202). See flavors
Agent memory: sandbox volumes	Done in code, fakes only	`sandboxVolumes.create`, `list`, `get`, `resize` and `delete`: a persistent disk booked as `diskGiB` on one host (core/0028, vps/0020), attached by serial to the sandboxes of templates that name it, one writer or many readers, pinned to its host, metered as `vps.volume.gib.hours` (2a09202, 8cbece0); bcd keeps each as a sparse raw file, grows it live, keeps one writer through handoffs and destroys it when unlisted (550da91, b5415a2, 448dc5f, 39458f9, 408e002). bcd never promises more than its data directory's filesystem holds and reports the space in `StatusReport.volume_storage` (851016e, 37d6d6d, b8e7271), which vps books volumes against, never overcommitted (131f5ee). The CLI's `sandbox-volume` commands and `sandbox-template --volumes` (f181f43), four MCP tools (e6759df), FleetConfig templates (94b8b70) and the console's Volumes page (50add82) expose them. A template mounts a volume only with a decision on that volume (77bf9f5), lists are per caller by default, and create and resize are decided with the size, so a policy can bound it (6cb30e9). See volumes. The console's Sandboxes page attaches and detaches a template's volumes (dd5555b). At scale (38405a0), `tests/e2e/volumes.test.ts` churns 240 sandboxes over 12 volumes on three FakeDaemon hosts through the real server and gateway, host reboots included, with no leak, no second writer, no overbooked store and no lost data; see testing

The work's owners are in .claude/coordination.md ("Round 5").

What unblocks what

Unblocked by	Moves
Writing more of bcd (Go is installed now; P0's code generation, W1's node agent, `bcd site`, `bcd ramdisk`, the `instances` converger, `libvirt-kvm`, `cloud-hypervisor`, self-update, the WebSocket fallback, `bcd helper`, bcx plugin supervision and the `bcx-ipmi` BMC driver are done)	Vendor BMC drivers, bcx plugins beside an unprivileged node, NVML events and a go-nvml backend; an end-to-end run of the site agent and ramdisk, and of bcd with a bcx plugin
A bcd release	Installing real nodes with the install script
A PostgreSQL 17 server	The concurrent capacity and torture suites, the IAM-lane gate, the 30-minute gateway gate, every v0.4 number
The D15 hardware	CDI and a bound GPU hold on an NVIDIA host (real inventory and live GPU metrics have run, on one RTX 5090 under WSL 2), emulation CI on a KVM runner, the first `libvirt-kvm` run on real KVM, the MIG and real-hardware nightlies

For what changed recently, see the changelog.

On this page