VPS
Virtual machines on your own hosts or on a provider's cloud, with desired state, flavors, provider accounts and pool IPAM from the vps() module.
Some of your capacity is best sold as whole machines, and some as virtual machines: a tenant wants a 2 vCPU box with an SSH key for a build agent, or an 8 vCPU VM with one L40S passed through, without renting a whole server. Other times you want to burst onto a cloud you do not run, and still keep one API, one quota system and one audit trail.
The vps() module does both. It adds : VMs described by a
, an and SSH keys, created in a
. Where the VM runs depends on the pool:
- A hosted pool runs VMs on your own servers. Capacity places each instance on a host, and the host's converges it.
- A provider pool creates each instance on an external cloud, through a such as Hetzner Cloud or your own Proxmox VE cluster.
Either way, the API only records what you want. A does the work, and nothing in a request ever calls a hypervisor or a cloud API.
Register the module
vps() is an injected module: the server never imports it, so you add it to modules. It requires the built-in
capacity module, which every server has.
import { betterCompute } from 'better-compute/server';
import { vps } from 'better-compute/vps';
import { hetznerCloud } from '@better-compute/provider-hetzner';
import { images } from './images';
export const compute = betterCompute({
// database, secret, iam, …
modules: [
vps({
runtimes: ['libvirt-kvm'], // hosted pools: the runtime driver bcd uses
providers: [hetznerCloud], // provider pools: the providers accounts may use
images, // hosted pools: resolves imageId for a tenant
}),
],
});The provider packages are on npm but sit outside the better-compute umbrella, so you install and import them by
their own names (npm install @better-compute/provider-hetzner).
Prop
Type
vps ships no image resolver: Better Compute checks image visibility itself, and you decide where images come from.
This one serves hosted VMs the images registered through the images group of provisionModule, when the tenant
owns them or they are public:
import type { ImageResolver } from '@better-compute/vps';
export const images: ImageResolver = {
async resolve(tx, tenantId, imageId) {
const { rows } = await tx.query<{
source: string;
sha256: string;
format: 'raw' | 'qcow2' | 'iso';
verified: boolean;
}>(
`SELECT source, sha256, format, verified FROM images
WHERE id = $1 AND deleted_at IS NULL AND (tenant_id = $2 OR visibility = 'public')`,
[imageId, tenantId],
);
const image = rows[0];
// undefined: the tenant cannot see it, and instances.create answers NOT_FOUND.
return image && { url: image.source, sha256: image.sha256, format: image.format, verified: image.verified };
},
};verified becomes the instance's imageVerified attribute, which policies can require.
Registering the module adds:
- the
instances,flavorsandproviderAccountsAPI groups (instances, flavors, providerAccounts), and the v0.5sandboxes,sandboxTemplatesandsandboxSnapshotsgroups (sandboxes, sandboxTemplates, sandboxSnapshots) for sandboxes on hosted pools; - the migrations
vps/0001_instances, with the tablesflavors,provider_accounts,instances,ip_poolsandip_leases,vps/0002_sealed_user_data(instances.user_data_sealed),vps/0003_provider_bootstraps(provider_bootstraps, for bootstrapping provider machines) and, for sandboxes,vps/0004_sandboxes(the tablessandbox_templatesandsandboxes, the columnsinstances.sandbox_idandinstances.warm_template_id, andpausedas a desired state),vps/0005_sandbox_snapshots(sandbox_snapshots,instances.restore_from,sandboxes.from_snapshot),vps/0006_warm_claims(instances.claimed_at),vps/0007_template_env(sandbox_templates.env) andvps/0008_template_network(sandbox_templates.networkandinstance_version,instances.network_policyandwarm_template_version),vps/0009_snapshot_fixes(sandbox_snapshots.principal_instance_idandpurged),vps/0010_sandbox_metric_indexes(partial indexes for the sandbox metrics),vps/0011_host_certificates(instances.host_certified,sandbox_snapshots.host_certified) andvps/0012_host_cert_validity(host_cert_valid_beforeon both); - the
vps.instancesreconciler, woken through the by every change, thevps.sandboxesreconciler (every 15 seconds: sandbox expiry, sandbox events, warm pools and settling snapshots), and thevps.usagejob, which meters hosted instances every 5 minutes; - the
instance.snapshotcommand kind (lanebulk, timeout 1 h) for sandbox snapshots; - a
tenantDeletedhook that deletes every instance of a deleted IAM tenant, sandboxes' and warm ones included; - an
instancessection in the of each hosted node; - the
runtimedriver kind, merged intoDriverKindsof@better-compute/corewhen you import the package; doctorfindings for hosted pools:vps.pools.runtime(a runtime the deployment does not list),vps.pools.single-tenant(a container pool withoutpolicy.singleTenant),vps.pools.ipam(no IP range) andvps.hosts.capacity(adopted hosts without capacity rows); see doctor.
Hosted pools also need the to feed each host's runtime report to vps. Pass
instanceStatusHandler as a status handler. It runs next to the built-in XID health handler, each under its own
savepoint:
import { defineHost } from '@better-compute/cli';
import { instanceStatusHandler } from '@better-compute/vps';
export const host = defineHost(async ({ compute }) => ({
gateway: {
// region, listen, tls, …
options: { reports: { status: instanceStatusHandler({ events: compute.context.events }) } },
},
}));Hosted pools and provider pools
A VPS pool is a pool of class vps-host or provider, created with
pools.create like any other. Instances in pools of other classes are refused
with INVALID_INPUT.
| Hosted pool | Provider pool | |
|---|---|---|
| Pool | class: 'vps-host', agent: 'bcd' | class: 'provider', agent: 'none', providerAccountId |
| Where the VM runs | A node of the pool, running bcd | The provider's cloud; there is no nodes row |
| Placement | Capacity claims vCPU, memory, disk and whole GPUs on one host, in the create request | None: the provider decides; the reconciler calls ProviderImpl.create |
| Quota | Charged by the placement | Charged at admission from the flavor's shape |
| Who converges | The host's runtime, from the instances section of its NodeSpec | The vps.instances reconciler, polling the provider |
| State comes from | The host's StatusReport.instances | ProviderImpl.get |
| Addresses | A lease from the pool's IPv4 range, plus what the runtime reports | What the provider reports |
| Resize | In place, while stopped, same GPU count | UNSUPPORTED |
| Metering | vps.vcpu.hours, vps.memory.gib.hours, gpu.<model>.hours | None: the provider bills you |
No platform agent ever runs inside a tenant VM, and no ever goes into tenant-readable user data. A provider machine that should become a host of your own (a Hetzner Robot server that runs bcd) is a different thing: see bare metal.
The instance model
An instance has two states. desiredState is what you asked for: running, stopped or deleted. state is where
the instance actually is. instances.create sets desiredState (default running); start, stop and delete
change only desiredState and wake the reconciler. The reconciler moves state toward it, one guarded transition at
a time, from what the host or the provider reports.
Every state except deleted and deleting can move to deleting, and most can move to error and lost. The
full table is instanceStateTable in @better-compute/core; since 564f543 it also allows recreate from
provisioning and stopped, for an instance whose host cannot start it (see
hosts that cannot start an instance). rebooting is in the table, but no API method enters
it in this release. paused (v0.5: running to paused by pause, back by resume) is used only by the warm
pools of sandboxes; see paused instances.
The API returns the Instance view:
Prop
Type
Instances covers each operation and how the reconciler turns reports into transitions.
What ships in v0.3
The control plane is complete and tested: the three API groups, admission, the reconciler for both pool kinds,
pool IPAM, the report seam and metering run against PGlite in tests/vps-instances.test.ts, and
tests/vps-churn.test.ts runs the v0.3 acceptance shape (200 instances churned on 3 emulated hosts with no leaked
allocation, IP lease, scalar claim or quota). What runs the VMs on a host is written but not yet proven: bcd's
libvirt-kvm and cloud-hypervisor runtimes have been tested against fake tools only.
| Part | Status in this release |
|---|---|
instances, flavors, providerAccounts groups | Served (v0.3). |
vps.instances reconciler, IPAM, vps.usage metering | Shipped. |
hetzner-cloud provider | Shipped in @better-compute/provider-hetzner, tested against an in-memory emulator of the Hetzner Cloud API. |
proxmox provider | Shipped in @better-compute/provider-proxmox (PLAN.md scheduled it for v0.5), tested against an in-memory PVE emulator. |
Host runtimes libvirt-kvm and cloud-hypervisor | bcd (v0.3), built but not released. A host build with virsh runs libvirt-kvm (362317c) under the instances converger: qcow2 overlays on sha256-verified https base images, NoCloud seeds, VFIO GPUs, pause and resume, snapshots and restores. A host build with the cloud-hypervisor binary runs cloud-hypervisor (2598f0a): one VMM per instance as a confined systemd transient unit, over its REST API, with the same overlays and seeds, VFIO GPUs, pause and resume, memory snapshots and restores. Security review 12's two high-severity findings are fixed (933c51b): taps are bridged before the VMM starts and an inet bcd_taps backstop drops a tap off its bridge, and both runtimes refuse a tenant image or restored snapshot that names a backing file or an external data file. Both have been tested only against fake tools, never on a real KVM host. A host with neither and without a runtime plugin fails a non-empty instances section; fakehw builds run a file-backed runtime. See the runtimes. |
| Runtimes as out-of-process plugins (Kata, and others) | v0.5: bcd serves a bcx plugin of kind runtime as an instances runtime (list, apply, remove; no snapshots), Go unit tests only. No Kata plugin is written. See walkthrough 4. |
| Single-tenant container runtimes | Planned for v0.3; the RuntimeTraits contract describes them, nothing enforces single tenancy yet. |
| Interactive access (exec, SSH, TCP through a tunnel) | tunnels.open with an instanceId, decided as compute:instances:exec: hosted instances only, and only when the deployment configures tunnels. The API reference lists it as planned. See tunnels. |
Sandboxes (sandboxes, sandboxTemplates, sandboxSnapshots) | v0.5: served, all 15 methods. Templates with env and network, TTL expiry, warm pools of paused instances and their vps.sandbox.warm meter, snapshots, restores and snapshot management, sandboxes.exec (from the tunnels module), the instance guard, tenant teardown (tests/vps-sandboxes.test.ts, tests/e2e/sandboxes.test.ts). bcd enforces a template's network with host nftables rules. Like every hosted instance, a sandbox needs a host runtime. See sandboxes. |
| Per-instance metadata service and workload identity | v0.5, optional. The control plane half (the workload broker, the session frames, the issuer and its published JWKS) is built and tested against the FakeDaemon's metadata service; bcd serves the same contract on [imds] listen, for the instances of whichever runtime it runs (end to end only with a fakehw build's file-backed runtime so far). See workload identity. |
The TypeScript FakeDaemon in @better-compute/testing speaks the real protocol for the simulator and
better-compute dev --sim N. As a VPS host it runs a FakeHypervisor (domains, overlays, NoCloud seeds, pause and
resume), which is how tests/e2e/vps.test.ts runs hosted instances through the real server and gateway. The unit
suites, tests/vps-sandboxes.test.ts among them, emulate hosts by calling reportInstances directly.
Create, start, stop, resize and delete; the reconciler, host reports, orphans and lost hosts.
SandboxesTemplates, a time-to-live, warm pools of paused instances, and the instance each sandbox drives.
FlavorsNamed shapes per pool, and how they become claims, quota and provider offers.
ProvidersProvider accounts, defineProvider, Hetzner Cloud, Proxmox VE and the conformance cases.
NetworkingPool IPAM: one IPv4 range per hosted pool, one lease per instance.
Better Compute is created by Sean Filimon
Last updated
Kubernetes workersexperimental
Join a pool's machines to a Kubernetes cluster and hand a reservation's GPUs to their kubelets with binding kubernetes, through bcd's device plugin or DRA driver.
Instancesbeta
Create, start, stop, resize and delete VPS instances, and how the vps.instances reconciler converges them from host reports or provider polls.