Profiles and images

A profile is the recipe for one machine: which steps run, which drivers do each job, which image goes on the disk, what user data it boots with and what host configuration the installed OS converges to. Profiles are versioned documents, and a version becomes immutable the first time a workflow uses it, so you can always tell exactly what was run on a machine.

Defining a profile

defineProfile(document) checks a profile and returns a deeply frozen copy. This GPU host profile follows the one the metal test suite deploys over PXE, with host config, acceptance and cleanup added:

import { defineProfile } from 'better-compute/provision';

export const ubuntuGpuHost = defineProfile({
  id: 'ubuntu-24.04-gpu-host',
  version: 1,
  target: 'metal',
  role: 'host',
  image: { ref: 'ubuntu-24.04-server@2026.09', format: 'raw', compression: 'zstd' },
  userData: { kind: 'cloud-init', template: 'bcd-host.yaml.hbs' },
  hostConfig: { 'nvidia-driver': { branch: '580' }, 'fabric-manager': { when: 'nvswitch' } },
  steps: [
    { use: 'userdata.render' },
    { use: 'boot.prepareRamdisk', on: 'site' },
    { use: 'management.setBootDevice', on: 'site', with: { device: 'pxe' } },
    { use: 'power.setPower', on: 'site', with: { action: 'cycle' } },
    { use: 'wait.agentJoined', with: { role: 'ramdisk' } },
    { use: 'deploy.deploy', on: 'ramdisk', timeout: '30m' },
    { use: 'management.setBootDevice', on: 'site', with: { device: 'disk' } },
    { use: 'boot.kexec', on: 'ramdisk', await: 'rejoin', timeout: '15m' }, // completes when the OS joins
  ],
  acceptance: [{ use: 'diag.dcgm', with: { level: 1 } }],
  cleanup: [{ use: 'boot.cleanUp', on: 'site' }],
});

defineProfile rejects unknown fields, malformed steps, ids and durations, values that are not JSON, and fields that belong to the other target, with INVALID_CONFIG whose details.path names the first offending field (for example steps[2].timeout). Through the API the same problems are INVALID_INPUT. The step fields (on, with, timeout, retries, await, onFailure, fallback, when) are described on provisioning workflows.

Storing and checking profiles

Profiles live in two places: those your code ships through provisionModule({ profiles }) or a module's profiles (read-only, visible to every tenant, listed under the tenant builtin), and stored ones in the profiles table, per tenant and version. A lookup prefers the tenant's own profile, then a public stored one, then a shipped one.

Run validate before the first deploy: it reports every problem at once with its path, where a deploy would fail on the first.

better-compute profiles create ./ubuntu-gpu-host.json --tenant root            # --public for every tenant
better-compute profiles validate ubuntu-24.04-gpu-host --tenant root --profile-version 1
better-compute profiles render ubuntu-24.04-gpu-host --tenant root              # the latest version

The profiles commands (171d9be) default --profile-version to the latest version where the method allows it; profiles update takes --file (a replacement document) or --visibility, and --if-version for the row's version.

The profiles metal ships

Metal runs one shipped profile per workflow-driven state. Deploying is the exception: it runs the profile named in metal.deploy.

Pools whose machines never run bcd (agent: 'none') cannot boot a ramdisk, so they get out-of-band variants: metal-inspect-oob@1 (inspect.oob), metal-commission-oob@1 (power off), metal-release-oob@1 (media.eject, then power off) and metal-retire-oob@1 (power off, then metal.revokeIdentity). Rescue on such a pool fails the node with UNSUPPORTED. The two tenant-root teardown profiles are on dedicated-root metal, and provision ships remediate-reboot@1 and remediate-gpu-reset@1 for health remediation.

Steps you can use

• metal.ensureRamdisk does nothing when the machine's ramdisk is already connected. Otherwise it sets one-time PXE, sends the boot entries and power-cycles the machine, then waits for the ramdisk to join. A machine known only by its product UUID is first power-cycled into PXE with no entry, so its site agent lists it; the step then waits until the MAC PXE discovery proposes is confirmed with metal.confirmMac, or bound at once on a pool with policy.trustProvisioningNetwork (3d26428, 8de6e2f).
• inventory.refresh checks that the machine that booted is the one enrolled (8de6e2f): a product UUID in the ramdisk's system section must be a spelling of the enrolled one, and for a MAC PXE discovery bound it waits for the ramdisk's system and nics sections and requires the NICs to include that MAC. A mismatch fails the workflow with DRIVER_ERROR (reason machine-identity-mismatch) before anything is written.
• diag.dcgm fails with DRIVER_ERROR unless the result is node-signed and every test passed.
• disk.wipe succeeds with attested: false when the signature or a disk's read-back check fails; metal then refuses to release the node to ready (UNVERIFIED_WIPE).
• deploy.deploy resolves the image in the machine owner's tenant (its own image first, then a public one), refuses an iso (INVALID_INPUT), sends deploy.image with the source URL, digest, format, compression and a user-data reference, and fails with DRIVER_ERROR when the digest the ramdisk wrote differs from the catalog's.
• media.insert with fromStep mounts the url an earlier step of the same workflow returned: how a plugin hands a per-node ISO to the BMC (see deploying).
• media.insert with image (3d26428) names an iso of the catalog: virtual media mounts its https:// source, and a BMC without virtual media boots it with iPXE sanboot from the site cache instead; a later management.setBootDevice to cd then sets pxe, and media.eject removes the boot entry. See sanboot instead of virtual media.

Driver bindings from the hardware profile

Machines in a pool usually share their BMC and boot hardware, so driver choices live in a hardware profile rather than on each node:

metal({
  hardwareProfiles: {
    'gpu-redfish': { drivers: { power: 'redfish', management: 'redfish', boot: 'ipxe', deploy: 'image-stream', inspect: 'agent' } },
  },
});
await compute.api.pools.create(credential, {
  tenantId: 'root', siteId, class: 'metal-host', policy: { hardwareProfile: 'gpu-redfish' },
});

At enrollment, resolveDriverBindings builds the node's bindings in three layers:

1. The BMC given to metal.enroll binds power, management and virtual-media, with its address and credential reference as config.
2. The hardware profile (the node's hardwareProfile, else the pool's policy.hardwareProfile) overrides per concern.
3. Per-node overrides win.

An unknown hardware profile is INVALID_INPUT. The bindings are stored on the node and served in its NodeSpec. Metal's own BMC steps use the BMC driver the node was enrolled with; a profile's drivers apply to the driver operations it invokes directly.

User data

A deployed OS has to join as the same node, so it needs a join token in its first-boot configuration. userdata.render builds that configuration from the profile's userData:

1. It mints a single-use provisioned join token for the node's pool and site (valid 2 hours, or tokenTtlSeconds).
2. It renders the named template with the variables nodeId, hostname (the node id with _ turned into -), joinToken, controlPlaneUrl, bcdUrl and bcdSha256.
3. It stores the document in the secret store and returns only { userDataRef, kind, joinTokenId }. deploy.deploy passes the reference, and the ramdisk fetches the document with a SecretRequest.

Two templates ship: bcd-host.yaml.hbs (cloud-init) and bcd-host.ign (Ignition 3.4.0, built as JSON so values cannot change its structure). The cloud-init one writes the token to a 0600 file and a bcd.service unit, downloads bcd, checks its digest, joins, then starts the node agent:

#cloud-config
hostname: {{hostname}}
write_files:
  - path: /etc/bcd/join-token
    permissions: '0600'
    content: {{joinToken}}
  - path: /etc/systemd/system/bcd.service
    permissions: '0644'
    content: |
      [Unit]
      Description=better-compute node agent
      After=network-online.target bcd-join.service
      Wants=network-online.target
      [Service]
      ExecStart=/usr/local/bin/bcd node
      Restart=always
      RestartSec=5
      [Install]
      WantedBy=multi-user.target
runcmd:
  - [sh, -c, "curl -fsSL '{{bcdUrl}}' -o /usr/local/bin/bcd && echo '{{bcdSha256}}  /usr/local/bin/bcd' | sha256sum -c - && chmod 0755 /usr/local/bin/bcd"]
  - [/usr/local/bin/bcd, join, --server, {{controlPlaneUrl}}, --token-file, /etc/bcd/join-token, --node-id, {{nodeId}}, --serving-ca-pin, {{servingCaPin}}, --join-signing-pin, {{joinSigningPin}}, --join-encryption-pin, {{joinEncryptionPin}}]
  - [systemctl, enable, --now, bcd.service]

(The template is a function; the snippet shows its output with placeholders. Without pins the three --…-pin arguments are left out.)

The Ignition one does the same with two units: bcd-join.service, a oneshot that runs only while the token file exists, and bcd.service after it. Both run exactly bcdJoinArgv() and bcdNodeArgv() (exported from @better-compute/provision), which bcd's own tests check against its flags:

• bcd join --server URL --token-file PATH --node-id ID joins with the provisioned token and removes the file once joined. bcd refuses an identity other than the node the OS was installed for, and --server supplies the join encryption key from the control plane's well-known document when join-encryption.jwk is absent (it must match the pinned key).
• The ramdisk's deploy.image installs its pinned config.toml and join-encryption.jwk into the OS it wrote, so bcd node finds its configuration at the default path.
• The three pins are the trust anchors bcd checks the well-known document against: the gateway serving CA, the key that signs JoinResponse, and the JoinService key the token is sealed to. They come from the host's authenticated PKI bundle (userData.pins, see below), never from that document, so a forged document (DNS takeover plus a Web PKI certificate) cannot swap them. A real bcd refuses to bootstrap without all three.

controlPlaneUrl must be an http(s) URL without spaces, quotes or shell and systemd special characters (it is a plain argument). bcdUrl is single-quoted for sh -c, so a presigned URL's & and % are fine; it may not hold quotes, backslashes or whitespace. bcdSha256 is 64 lower-case hex digits, and each pin is sha256/ plus the base64 SPKI digest. provisionModule refuses anything else with INVALID_CONFIG, including a partial set of pins.

• Configure it with provisionModule({ userData: { controlPlaneUrl, bcdUrl, bcdSha256, pins?, templates?, tokenTtlSeconds? } }), where pins is { servingCaPin, joinSigningPin, joinEncryptionPin } from the host's bundle. Without the URLs and the digest, rendering fails with INVALID_CONFIG. Custom templates get the pins as the variables of the same names.
• A template is a string with {{name}} placeholders or a function of the variables. Every value must be a single line without {{, so a value can never add lines to the document; a placeholder with no value is INVALID_CONFIG.
• No template ships for kind: 'talos'; bring your own.
• A profile with userData whose deploy runs before any userdata.render step fails with INVALID_CONFIG. A re-run after a crash mints a new token; the unused one is single-use and expires, so it never becomes a second way in.
• The installed bcd joins through the provisioned attestor, which maps the token back to the running workflow's node (see sites for the wiring).

Host config

hostConfig is not a step: when the workflow succeeds, its units become the node's desired state (the NodeSpec host section), and bcd's host converger applies them and sets drift back on its next pass (6c2343f). The built-in units, in the order bcd applies them:

Any unit takes when, a label expression. The pool can set units too, and a pool's unit replaces the profile's: policy.hugepages, policy.kernelModules, policy.cpu, policy.iommu, policy.gpu and policy.kubernetes, and gpu-sharing for a pool shared by mps or timeslice. Host configuration describes what each unit does on the machine, the files it writes, what it refuses and when it reports RebootRequired, and the host config units reference lists each unit's settings field by field. Add units with provisionModule({ configUnits }); bcd applies only the units it knows, and reports any other as a terminal failure once the units it knows had their turn.

The image catalog

Images are built outside the control plane (mkosi or Packer, as raw.zst or qcow2) and registered by digest. The catalog records where the artifact lives, what it is, and whether a trusted signer vouched for it. It never stores the image itself.

import { sign } from 'node:crypto';
import { imageSigningInput } from 'better-compute/provision';

const image = {
  ref: 'ubuntu-24.04-server@2026.09',
  kind: 'os',
  format: 'raw',
  compression: 'zstd',
  sha256: '<64 hex digits>',
} as const;
const signature = sign(null, imageSigningInput(image), signingKey).toString('base64'); // Ed25519

await compute.api.images.register(credential, {
  tenantId: 'root',
  ...image,
  sizeBytes: 3_221_225_472,
  source: 'https://images.example.com/ubuntu-24.04-server-2026.09.raw.zst', // https:// or oci://
  signerId: 'platform-2026',
  signature,
});

• What is signed. imageSigningInput is the UTF-8 text bc-image-v1, then the kind, ref, format, compression and lower-case digest, one per line. Sign it with Ed25519, or ECDSA P-256 over SHA-256 (IEEE P1363 or DER encoding).
• Signers. Allowlists are rows of image_signers per tenant, with SPKI PEM keys; the root tenant's signers are the platform's. Hosts add and revoke them with addImageSigner and revokeImageSigner; there is no API method. Images a revoked signer verified keep their flag, but no longer deploy (below).
• Verification. A signature must verify against the named signer of the tenant or the platform, else INVALID_INPUT. A verified image has verified: true. The default check is offline against image_signers; swap in Sigstore or a KMS with provisionModule({ imageVerifier }).
• Visibility. Unsigned images stay private to their tenant forever (a table CHECK enforces it). images.promote makes a verified image public to every tenant; on an unsigned one it is INVALID_STATE.
• At deploy time. Before metal relays deploy.image, verifyImageForUse re-verifies the image's stored signature against the signer allowlist it was verified against at registration. An unsigned image, a revoked signer or a tampered catalog row stops the deploy with INVALID_STATE (details.reason unsigned or signature), and the node fails out of deploying without any deploy.image sent. Pass the verifier you gave provisionModule({ imageVerifier }) as metal({ images: { verifier } }) when it is not the default allowlist. images: { allowUnsigned: true } lets a tenant-private unsigned OS image through, for development only; never a ramdisk, and never a signed image whose signature fails.
• Reading. images.get and images.list (filter by kind, up to 200 a page) return the tenant's own images and public ones. A profile's image.ref resolves to the tenant's own image first.

images.register and images.promote are decided as compute:images:publish, get and list as compute:images:read.

From the command line (d1e83ca), images register takes the format as --disk-format (--format is the CLI's output flag), and --signature and --signer only together:

better-compute images register --tenant root --ref ubuntu-24.04-server@2026.09 \
  --source https://images.example.com/ubuntu-24.04-server-2026.09.raw.zst \
  --disk-format raw --compression zstd --sha256 <64 hex digits> --size 3221225472 \
  --signer platform-2026 --signature "$SIGNATURE"
better-compute images list --tenant root --kind os
better-compute images promote img_01k5w… --tenant root

Site caches

The catalog also tracks which site caches hold which images (image_locations: pulling, ready, failed, evicting). The rows are derived from the site agents' own caches: each agent reports its cache in the images inventory section (node_image_cache), a change queues the provision.site-cache reconciler for the site, and syncSiteImageLocations marks a catalog image whose digest a live site agent holds as at the site (ready if any agent has it ready, else pulling or evicting), and drops the locations no agent holds any more unless a deploy pinned them. recordImageLocation and touchImageLocation record reports and use (and pin an image while a deploy streams it), imagesCachedAt(siteId) lists the ready images at a site (the image-cached placement scorer reads nodes' own node_image_cache rows instead, see scheduler plugins), and evictionCandidates(siteId, bytes, inUse) picks the least recently used ready images, never pinned or in-use ones. What to pre-warm is desired state today: a pool's policy.prewarmImages lists name@version refs, and provision puts the ones that verify into its site agents' NodeSpec site.cacheImages (see sites). bcd site pulls that list into its cache, verifies each digest before the copy becomes visible, serves it to the site's ramdisks by digest and reports it in its images inventory section; a ramdisk's deploy.image tries the cache first and falls back to the source (see the boot server). It does not pull on demand: an image no pool pre-warms streams from its source.