Walkthroughs
The acceptance walkthroughs of PLAN.md 5.6 and the emulated-site example, what each one proves about extending Better Compute, and how to run them.
PLAN.md §5.6 sets the bar for the extension points: each one is proven by a walkthrough that adds something real to
Better Compute with zero edits to @better-compute/*. A walkthrough is a third-party plugin, written in a test the
way an outside package would write it, plus the test that shows the platform picks it up and runs it. They double as
the best worked examples of the contracts on the other pages of this section.
| Walkthrough | Phase | Extension points | Where | State |
|---|---|---|---|---|
| 1. Ubuntu autoinstall via Redfish virtual media | v0.2 | a control-plane step, a profile, a plugin | tests/extensions/ubuntu-autoinstall.test.ts | runs in simulation |
2. VPS provider acme-cloud | v0.3 | defineProvider, conformance cases, a provider pool | tests/extensions/acme-cloud.test.ts | runs against an in-memory cloud emulator, live and from a recording |
| 3. Talos on an agentless pool | v0.3 | a relaying step, a profile, site.httpProxy | tests/extensions/talos.test.ts, tests/metal-fakedaemon.test.ts | runs in simulation and on the dev stack |
| 4. Kata runtime in Go | v0.5 | a bcx runtime plugin in bcd's [plugins] dir | daemon/internal/agent/pluginruntime_test.go | bcd serves runtimes from bcx plugins (Go unit tests); no Kata plugin written |
| Emulated site | v0.2 | a hardware profile, the whole metal lifecycle | examples/emulated-site | configuration checked; needs Linux, KVM and a bcd build to run |
Run them all from the repository root:
pnpm vitest run tests/extensions tests/metal-fakedaemon.test.ts tests/examples-smoke.test.tsWalkthrough 1: Ubuntu autoinstall
The plugin. acme-autoinstall installs Ubuntu with its own installer from a per-machine ISO mounted through the
BMC, instead of streaming an image from a ramdisk. It is three definitions:
const renderAutoinstallIso = defineStep({ id: 'autoinstall.renderIso', runsOn: 'control-plane', /* … */ });
const ubuntuAutoinstall = defineProfile({ id: 'ubuntu-24.04-autoinstall', version: 1, target: 'metal', role: 'host', /* … */ });
const acmeAutoinstall = definePlugin({ id: 'acme-autoinstall', extensions: [renderAutoinstallIso], profiles: [ubuntuAutoinstall] });autoinstall.renderIsomints a single-useprovisionedjoin token throughctx.pki, renders it into an autoinstalllate-commandsentry withrenderTemplate, stores the ISO under its sha256, and returns{ url }, an unguessable per-node URL behind the site cache.- The profile chains built-in metal steps around it:
media.inserton the site agent withfromStep: 'autoinstall.renderIso',management.setBootDevicetocd,power.setPowercycle,wait.agentJoinedfor rolenode, andmedia.eject.
Both are written out in full on steps and profiles and deploying.
The harness. createMetalSim({ modules: [acmeAutoinstall], install }) in tests/extensions/metal-sim.ts builds a
simulated site: PGlite, the real provision engine and metal module, a site agent, and per-machine BMC state (boot
device, mounted media, power, firmware, BIOS and RAID). Booting from PXE brings up the ramdisk; booting from a CD whose
ISO the install hook accepts brings up the installed OS as role node. Results come back as each kind's binary result
message, node-signed where the kind is attested, as real agents send them.
What it proves.
profiles.validateresolves every step of the plugin's profile: the engine found a third-party step and profile.- One machine goes enroll,
ready,deployedwith rolehost, undeploy,ready. The ISO is ejected, and the join token appears inside the ISO and nowhere in any command the agents saw. - The v0.2 acceptance of PLAN.md §16 in simulation: 20 consecutive unattended deploy and release loops, a fresh
single-use token for each install, and every transition into
readypreceded by exactly one attested wipe in that workflow (checked in SQL overworkflowsandworkflow_steps). - A release whose wipe is not attested never reaches
ready: it endsfailedfromreleasing, with maintenance on.
pnpm vitest run tests/extensions/ubuntu-autoinstall.test.tsWhat it does not prove. The ISO build and hosting are stubbed (a map from digest to token), the installed OS
rejoining as the same node is simulated, and no real BMC mounts anything. A token a plugin mints itself joins through
the provisioned attestor only if it carries expect: { productUuid } or the host's resolver can trace it; see
attestors.
Walkthrough 3: Talos on an agentless pool
Talos manages itself through its own API and never runs bcd. The plugin is one step and one profile:
wait.talosReadyis a control-plane step withrelays: ['site']andsends: ['site.httpProxy'], the kind it relays.site.httpProxyis destructive (0d42de8), so the declaration is what puts it in the workflow's quorum batch: without it, a deployment withpki.approversset refuses the step's commands at enqueue (see relaying under the quorum). It reads the machine's address frommetal_nodes.addresses(recorded by Redfish inspection), and callshttpViaSite(ctx, { url, tlsSpkiSha256, poll: true })onhttps://<address>:50000/machine/version. Until Talos answers 200, it throwsstepNotReady.talos-1.8mounts the Talos ISO withmedia.insert, sets boot tocd, power-cycles, and waits inwait.talosReady.
The pool opts in to what an agentless machine cannot do: agent: 'none', policy.allowUnverifiedRelease: true, and
policy.provisioningCidrs for plain-HTTP targets.
httpViaSite is the extension point this walkthrough added to metal: a control-plane step reaches a machine only
through its site agent's site.httpProxy command, and the control plane refuses anything else before a command
exists. The request takes url, method (GET, POST, PUT, DELETE), headers, secretHeaders (secret
references only), body, tlsSpkiSha256, timeoutMs, key and poll. With poll, each call returns the newest
settled answer and sends the next request, which is what a readiness loop needs.
What it proves (tests/extensions/talos.test.ts, on PGlite with a stub command queue):
- Enrollment and commissioning run out of band (the agentless variants of metal's profiles); the recorded addresses come from Redfish.
- The deploy polls the Talos API until it answers, always at the machine's own address, with the pinned SPKI, and the
node is
deployedwith rolehost. Every command went to the site agent, never to the machine. - The node is never
connected, so it is never schedulable forholdreservations. - Release reaches
readywithout an attested wipe only because the pool opted in, and says so withnode.release.unverified. Rescue fails the node withUNSUPPORTED. site.httpProxyis SSRF-bounded: other hosts, names, credentials in URLs, plain HTTP off the provisioning network, forbidden headers and header line breaks are refused; a request for another node's IP creates no command; a redirect or an oversize response isDRIVER_ERROR; secret headers travel by reference.
tests/metal-fakedaemon.test.ts runs the same Talos plugin on the dev stack: the real control plane with provision
and metal, the real gateway, JoinService, and a site agent that is a FakeDaemon of role site, joined with a site
token (tests/metal-dev-site.test.ts), in front of simulated BMCs. Every command crosses the gateway and back, and
the real server's batch check runs (f66e236). Both tests declare sends on the Talos step.
pnpm vitest run tests/extensions/talos.test.ts tests/metal-fakedaemon.test.tsWalkthrough 2: a VPS provider
PLAN.md describes it as defineProvider(…), providerConformance(acmeCloud, { recordReplay }), a provider account, and
a class: 'provider' pool with agent: 'none' whose instances run through the generic provider steps.
tests/extensions/acme-cloud.test.ts runs it as a third party would, with zero edits to @better-compute/*:
- The plugin.
tests/extensions/acme-cloud/provider.tsis adefineProviderover Acme Cloud's HTTP API with an injectablefetch;cloud.tsis an in-memory emulator of that API, andrecord-replay.tsa transport that records every exchange and replays it. - Conformance, live and offline.
runConformance(providerConformanceCases(), subject)passes against the emulator through the recording transport, leaving no server behind, and passes again offline from the recording with every exchange consumed. A wrong account token is a configuration error, not a provider outage. - On a real server. With
vps({ providers: [acmeCloudProvider({ fetch })] })on a compute fixture: a provider account whose token is sealed in the secret store, aclass: 'provider'pool (itsagentisnone) offered to a tenant with a quota, and a flavor mapped onto the provider's offer. The tenant's instance is created once (a retried reconciler pass creates nothing new), with the SSH key and no join token in its user data, followed torunning, stopped (the machine powered off) and deleted; a third instance over the tenant's quota is refused, and the pool never gets a node.
The conformance API is providerConformanceCases() with runConformance, not a single providerConformance
function, and recording is left to the transport: adapters take a fetch. Instances of provider pools are driven by
the vps.instances reconciler calling the account's client, not by workflow steps. tests/vps-provider.test.ts
(manifest-typed accounts and shape refusals) and the Hetzner Cloud, Hetzner Robot and Proxmox suites cover the same
contract piecewise. Providers shows how to write and test one.
pnpm vitest run tests/extensions/acme-cloud.test.tsWalkthrough 4: a Kata runtime over bcx
A runtime driver written in Go, pinned in bcd's [plugins] dir, advertised in Hello.drivers, and selected by pools
through vps({ runtimes }). The TypeScript half of bcx is in @better-compute/driver-sdk, with a real-subprocess test
suite. bcd hosts pinned plugins and advertises their drivers, and bcx.Serve in daemon/internal/bcx is the start
of a Go plugin SDK.
On bcd node, a hosted driver whose manifest lists the kind runtime, and whose id is not a runtime built into bcd,
becomes an instances runtime (agent.PluginRuntime, 494de69). bcd speaks the TypeScript RuntimeDriver contract to
it over Invoke, with positional JSON arguments:
| Operation | Arguments | Answers |
|---|---|---|
list | none | [{ id, state, addresses, message? }] |
apply | core's InstanceSpec as JSON, with desiredState | { id, state, addresses, message? }; state: 'error' fails the instance |
remove | the instance id | null |
apply converges one instance, desired state included, so bcd records each spec and applies it when the state
changes. A new instance's first apply carries its user data and, for a restore, the snapshot bcd already downloaded
and verified, as a file:// URL; later ones do not. Snapshots answer unsupported, since they are not in the bcx
contract yet. A plugin never shadows a built-in runtime of the same name, and Hello lists its manifest once. Like
every plugin it runs sandboxed: under its own uid by default, with a
seccomp filter that refuses mounts, namespaces, bpf and module loading, so a runtime that needs them (Kata) must be
named in [plugins] privileged, which runs it as root without the filter. Go
unit tests (pluginruntime_test.go) cover it; no Kata plugin exists yet. The control-plane half is in place: the gateway writes each driver a daemon advertises in
Hello.drivers as the node label bc.dev/driver.<id>=<version>, and vps places an instance, whatever its runtime (bcd's
built-ins too since 0fca413), only on nodes carrying bc.dev/driver.<runtime> (runtimeNodeLabels, through
PlacementRequest.nodeLabels; see placement). See
drivers.
The emulated site
examples/emulated-site is the metal walkthrough without a rack: QEMU/KVM virtual machines stand in for servers,
sushy-emulator gives each a Redfish BMC over libvirt, and a bcd site agent on the libvirt bridge runs proxy DHCP,
iPXE and the image cache. Its better-compute.config.ts uses the shared example deployment (examples/shared/compose.ts,
every module) and adds one hardware profile:
export const hardwareProfiles = {
'emulated-redfish': {
description: 'QEMU/KVM machines behind sushy-tools (Redfish), PXE-booted by the site agent',
drivers: { power: 'redfish', management: 'redfish', boot: 'ipxe', deploy: 'image-stream' },
},
};A pool that names it in policy.hardwareProfile drives the emulated BMCs through commission, deploy, release and
retire. The run needs Linux with /dev/kvm, QEMU, libvirt, sushy-tools and a bcd build, plus iPXE builds for the
site agent's TFTP root and a ramdisk kernel and initramfs with bcd ramdisk in it, so none of it runs in CI. One
sushy-emulator serves both VMs, so each enrollment names its system in the BMC address
(https://192.168.122.1:8000/redfish/v1/Systems/<libvirt domain uuid>); bcd's Redfish client then drives that
system and its own manager (see Redfish in bcd site).
What CI checks, in tests/examples-smoke.test.ts, is that the configuration loads, migrates on PGlite and passes
better-compute doctor with no error findings (on an unconfigured machine it reports iam.not-configured).
The step-by-step run, from virt-install to metal.undeploy, is on
deploying. Two details there supersede the example's
README: site agents join with a site token from siteTokens.create, and after metal.enroll the reconciler carries a
machine to ready on its own, so metal.manage, metal.inspect and metal.provide are not needed.
Writing your own
A walkthrough for your extension follows the same recipe:
- Write the plugin in the test file, importing only public entry points, the way an outside package would.
- Register it the way a deployment would (
provisionModule({ modules }),vps({ providers }),betterCompute({ attestors })), not by reaching into internals. - Assert the platform's view first:
profiles.validatereturns no issues,drivers.listshows the driver, the conformance report is empty. - Drive the lifecycle end to end on a simulated site or the dev stack, and assert on states, events and the commands agents saw, including what must not appear (a token in a command body, a destructive command sent twice).
- Add the failure cases: an unattested result, a crash mid-step (
tests/metal-chaos.test.tskills the worker at every point of a deploy), a refused request.
Better Compute is created by Sean Filimon
Last updated