# Protocol (/docs/reference/protocol) > Every service, message and enum of the daemon protocol, from the .proto sources, with field numbers and types. The wire contract between bcd, the gateway and drivers. It is frozen: changes are additive only (`buf breaking`), and messages and fields marked with a phase are part of the frozen shape before they are used. For the session flow, framing, renewal and the WebSocket fallback, see [Protocol](/docs/daemon/protocol). ## commands.proto [#commandsproto] Package `bettercompute.agent.v1`, `proto/bettercompute/agent/v1/commands.proto`. Payloads and results of the built-in v0.1 command kinds (PLAN.md §7.3). A Command carries the request in CommandBody.payload (google.protobuf.Any); the CommandUpdate carries the serialized result in `result`. The kind table (lane and flags) lives in @better-compute/protocol (`commandKinds`) and in the daemon's dispatcher; both must agree. ### DiagNvmlRequest [#diagnvmlrequest] | Field | Number | Type | Description | | -------------------- | ------ | ----------------- | -------------------------------------------------------------- | | `gpu_uuids` | 1 | repeated `string` | GPUs to check; empty checks every GPU. | | `expected_gpu_count` | 2 | `uint32` | When non-zero, fewer visible GPUs fails the check "gpu.count". | ### DiagNvmlResult [#diagnvmlresult] | Field | Number | Type | Description | | ---------------- | ------ | ---------------------------------------------- | ------------------------------------------------- | | `passed` | 1 | `bool` | Every check on every GPU passed. | | `gpu_count` | 2 | `uint32` | GPUs visible to NVML. | | `driver_version` | 3 | `string` | | | `checks` | 4 | repeated [`DiagnosticCheck`](#diagnosticcheck) | Node-level checks, e.g. "nvml.init", "gpu.count". | | `gpus` | 5 | repeated [`GpuDiagnostic`](#gpudiagnostic) | Sorted by uuid. | ### GpuDiagnostic [#gpudiagnostic] | Field | Number | Type | Description | | -------- | ------ | ---------------------------------------------- | ---------------------------------------------------------------------------------------------- | | `uuid` | 1 | `string` | | | `passed` | 2 | `bool` | | | `checks` | 3 | repeated [`DiagnosticCheck`](#diagnosticcheck) | e.g. "ecc.uncorrectable", "row-remap.pending", "row-remap.failure", "pcie.link", "xid.recent". | ### DiagnosticCheck [#diagnosticcheck] | Field | Number | Type | Description | | --------- | ------ | -------- | ----------- | | `name` | 1 | `string` | | | `passed` | 2 | `bool` | | | `message` | 3 | `string` | | ### InventoryRefreshRequest [#inventoryrefreshrequest] | Field | Number | Type | Description | | ---------- | ------ | ----------------- | ---------------------------------------------------------------------------- | | `sections` | 1 | repeated `string` | Section names to send; empty sends every section. | | `rescan` | 2 | `bool` | Re-collect from the hardware first instead of resending the last collection. | ### InventoryRefreshResult [#inventoryrefreshresult] | Field | Number | Type | Description | | ---------------- | ------ | -------------------------- | ------------------------------------------------------ | | `section_hashes` | 1 | map of `string` to `bytes` | Section name → SHA-256 of the section content as sent. | ### LogsCollectRequest [#logscollectrequest] | Field | Number | Type | Description | | ------------------ | ------ | ------------------------------------ | ----------------------------------------------------------------------------- | | `upload_url` | 1 | `string` | Presigned HTTPS PUT URL. | | `upload_headers` | 2 | repeated [`HttpHeader`](#httpheader) | Headers the presigned request requires. | | `units` | 3 | repeated `string` | systemd units; empty means bcd's own units. | | `lookback_seconds` | 4 | `uint32` | How far back to collect. | | `max_bytes` | 5 | `uint64` | Upload cap; the bundle is truncated (oldest first) beyond it. | | `support_bundle` | 6 | `bool` | Include the support bundle (bcd status, doctor, journal metadata, inventory). | ### HttpHeader [#httpheader] | Field | Number | Type | Description | | ------- | ------ | -------- | ----------- | | `name` | 1 | `string` | | | `value` | 2 | `string` | | ### LogsCollectResult [#logscollectresult] | Field | Number | Type | Description | | ---------------- | ------ | -------- | ------------------------------- | | `uploaded_bytes` | 1 | `uint64` | | | `sha256` | 2 | `bytes` | SHA-256 of the uploaded object. | | `truncated` | 3 | `bool` | | | `content_type` | 4 | `string` | e.g. "application/zstd". | ### NodeRebootRequest [#noderebootrequest] | Field | Number | Type | Description | | ---------- | ------ | -------- | ------------------------------------------------------------------------- | | `reason` | 1 | `string` | | | `delay_ms` | 2 | `uint32` | Wait before rebooting (lets the CommandUpdate RUNNING reach the gateway). | ### NodeRebootResult [#noderebootresult] | Field | Number | Type | Description | | ------------------ | ------ | -------- | --------------------------------------------------------------------- | | `previous_boot_id` | 1 | `string` | The boot that ran the command; CommandUpdate.boot\_id is the new one. | ### GpuResetRequest [#gpuresetrequest] | Field | Number | Type | Description | | ----------- | ------ | ----------------- | -------------------------------------------------------- | | `gpu_uuids` | 1 | repeated `string` | At least one GPU; an empty list fails "invalid-payload". | ### GpuResetResult [#gpuresetresult] | Field | Number | Type | Description | | ------ | ------ | ---------------------------------------------- | --------------- | | `gpus` | 1 | repeated [`GpuResetOutcome`](#gpuresetoutcome) | Sorted by uuid. | ### GpuResetOutcome [#gpuresetoutcome] | Field | Number | Type | Description | | --------- | ------ | -------- | --------------------------------------------- | | `uuid` | 1 | `string` | | | `reset` | 2 | `bool` | | | `healthy` | 3 | `bool` | ECC and row-remap state read after the reset. | | `message` | 4 | `string` | | ### DiagDcgmRequest [#diagdcgmrequest] diag.dcgm: lane provision, retry-safe, attested. Commissioning (level 2) and acceptance (level 1). | Field | Number | Type | Description | | ----------- | ------ | ----------------- | ---------------------------------------------------------------------- | | `level` | 1 | `uint32` | dcgmi diag -r level: 1 (seconds), 2 (\~2 min), 3 (\~15 min), 4 (long). | | `gpu_uuids` | 2 | repeated `string` | GPUs to test; empty tests every GPU. | ### DiagDcgmResult [#diagdcgmresult] | Field | Number | Type | Description | | -------- | ------ | ---------------------------------------------- | ----------------------------------------------------------- | | `passed` | 1 | `bool` | | | `level` | 2 | `uint32` | | | `tests` | 3 | repeated [`DiagnosticCheck`](#diagnosticcheck) | Node-level tests, e.g. "dcgm.deployment", "pcie.bandwidth". | | `gpus` | 4 | repeated [`GpuDiagnostic`](#gpudiagnostic) | Per-GPU tests, sorted by uuid. | ### WipeRequest [#wiperequest] wipe: lane provision, NOT retry-safe, attested, destructive (signed from v0.2). The §11.1 invariant "ready only after a verified wipe" rests on this result's node signature. | Field | Number | Type | Description | | ------- | ------ | ----------------------- | ---------------------------------------------------------------------- | | `mode` | 1 | [`WipeMode`](#wipemode) | | | `disks` | 2 | repeated `string` | Disks to wipe, by WWN or serial; empty wipes every non-removable disk. | ### WipeResult [#wiperesult] | Field | Number | Type | Description | | ------- | ------ | -------------------------------- | ------------- | | `mode` | 1 | [`WipeMode`](#wipemode) | | | `disks` | 2 | repeated [`DiskWipe`](#diskwipe) | Sorted by id. | ### DiskWipe [#diskwipe] | Field | Number | Type | Description | | ---------- | ------ | -------- | --------------------------------------------------------------------------------------------- | | `id` | 1 | `string` | WWN, or the serial when the disk has no WWN. | | `method` | 2 | `string` | How the disk was wiped, e.g. "nvme-format", "ata-secure-erase", "blkdiscard", "crypto-erase". | | `verified` | 3 | `bool` | A read-back check confirmed the result. | | `message` | 4 | `string` | | ### DeployImageRequest [#deployimagerequest] deploy.image: lane provision, NOT retry-safe, attested, destructive. Streams the image from the site cache (HTTP range reads) through decompression straight to the root disk, checking the SHA-256 inline (§3.5 step 4). | Field | Number | Type | Description | | ---------------- | ------ | --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------ | | `url` | 1 | `string` | HTTPS URL, normally the site agent's image cache. | | `sha256` | 2 | `bytes` | SHA-256 of the image as stored (before decompression). | | `format` | 3 | [`ImageFormat`](#imageformat) | | | `compression` | 4 | [`ImageCompression`](#imagecompression) | | | `root_disk` | 5 | [`RootDiskHint`](#rootdiskhint) | | | `user_data_ref` | 6 | `string` | Secret ref of the rendered user data (it embeds a single-use `provisioned` join token); the agent fetches it with SecretRequest, never in the command. | | `user_data_kind` | 7 | `string` | "cloud-init", "ignition" or "talos". | | `uefi` | 8 | `bool` | Write for UEFI boot (EFI system partition) rather than legacy BIOS. | ### RootDiskHint [#rootdiskhint] The disk to install to; the first non-empty field wins, and a hint matching no disk fails the step. | Field | Number | Type | Description | | ---------------- | ------ | -------- | -------------------------------------------------------------- | | `wwn` | 1 | `string` | | | `serial` | 2 | `string` | | | `device` | 3 | `string` | e.g. "/dev/nvme0n1"; last resort, device names are not stable. | | `min_size_bytes` | 4 | `uint64` | Smallest disk at least this big. | ### DeployImageResult [#deployimageresult] | Field | Number | Type | Description | | --------------- | ------ | -------- | -------------------------------------------------------- | | `written_bytes` | 1 | `uint64` | | | `sha256` | 2 | `bytes` | SHA-256 verified while streaming (equals the request's). | | `root_disk` | 3 | `string` | WWN or serial of the disk written. | | `duration_ms` | 4 | `uint32` | | ### KexecRequest [#kexecrequest] boot.kexec: lane control, retry-safe, self-terminating. Boots the installed OS from the ramdisk without firmware POST; completes when the agent rejoins with a new boot\_id and role `node`. | Field | Number | Type | Description | | ---------------- | ------ | ----------------- | ------------------------------------------------------------------------------------------------ | | `kernel_path` | 1 | `string` | Kernel and initrd paths on the installed root filesystem; empty means the distribution defaults. | | `initrd_path` | 2 | `string` | | | `kernel_cmdline` | 3 | repeated `string` | Appended to the installed kernel command line. | ### KexecResult [#kexecresult] | Field | Number | Type | Description | | ------------------ | ------ | -------- | ------------------------------------------------------------------------------- | | `previous_boot_id` | 1 | `string` | The ramdisk boot that ran the command; the rejoining agent reports the new one. | ### DriverInvokeRequest [#driverinvokerequest] | Field | Number | Type | Description | | ---------------- | ------ | ------------------------ | ------------------------------------------------------------------------------------------------------------------------ | | `workflow_id` | 1 | `string` | | | `step_id` | 2 | `string` | | | `attempt` | 3 | `uint32` | | | `target_node_id` | 4 | `string` | Set when a site agent runs it for another node. | | `args` | 5 | `google.protobuf.Struct` | The expanded step: \{concern, operation, args, driver?, config?}; config holds secret:// refs only, never secret values. | ### DriverInvokeResult [#driverinvokeresult] | Field | Number | Type | Description | | ------------- | ------ | -------- | -------------------------------------------------------------- | | `result_json` | 1 | `string` | The operation's JSON result (bcx InvokeResponse.result\_json). | ### InstanceSnapshotRequest [#instancesnapshotrequest] | Field | Number | Type | Description | | ---------------- | ------ | ------------------------------------ | --------------------------------------------------------------------------- | | `instance_id` | 1 | `string` | | | `upload_url` | 2 | `string` | Presigned HTTPS PUT URL. | | `upload_headers` | 3 | repeated [`HttpHeader`](#httpheader) | | | `image_id` | 4 | `string` | The snapshot image id the control plane registers the upload under. | | `include_memory` | 5 | `bool` | Include the memory state (a restorable running snapshot); false: disk only. | ### InstanceSnapshotResult [#instancesnapshotresult] | Field | Number | Type | Description | | ---------------- | ------ | -------- | ------------------------------------------------------------------------------------------------------ | | `uploaded_bytes` | 1 | `uint64` | | | `sha256` | 2 | `bytes` | SHA-256 of the uploaded object; restoring verifies it (InstanceSpec.restore\_from.sha256). | | `runtime` | 3 | `string` | The runtime that wrote it; a restore needs the same runtime. | | `cpu_model` | 4 | `string` | The host CPU model the memory state depends on (the inventory label `cpu.model`), empty for disk only. | ### WipeMode [#wipemode] | Value | Number | Description | | ------------------------ | ------ | ------------------------------------------------------------------ | | `WIPE_MODE_UNSPECIFIED` | 0 | | | `WIPE_MODE_QUICK` | 1 | Partition tables, signatures and the first/last MiB of every disk. | | `WIPE_MODE_SECURE_ERASE` | 2 | ATA secure erase / NVMe format with user-data erase. | | `WIPE_MODE_CRYPTO_ERASE` | 3 | Rotate the self-encrypting drive's media key. | ### ImageFormat [#imageformat] | Value | Number | Description | | -------------------------- | ------ | ----------- | | `IMAGE_FORMAT_UNSPECIFIED` | 0 | | | `IMAGE_FORMAT_RAW` | 1 | | | `IMAGE_FORMAT_QCOW2` | 2 | | ### ImageCompression [#imagecompression] | Value | Number | Description | | ------------------------------- | ------ | ----------- | | `IMAGE_COMPRESSION_UNSPECIFIED` | 0 | | | `IMAGE_COMPRESSION_NONE` | 1 | | | `IMAGE_COMPRESSION_ZSTD` | 2 | | | `IMAGE_COMPRESSION_GZIP` | 3 | | ## inventory.proto [#inventoryproto] Package `bettercompute.agent.v1`, `proto/bettercompute/agent/v1/inventory.proto`. NodeInventory v1 (PLAN.md §10.1; docs/research/prior-art.md §4). ### NodeInventory [#nodeinventory] An assembled snapshot: what the control plane stores (append-only) and what an inspect driver returns. Never hashed as a whole. | Field | Number | Type | Description | | ---------------- | ------ | --------------------------------- | ------------------------------------------------------------ | | `schema_version` | 1 | `uint32` | 1 for this schema. | | `collected_at` | 2 | `google.protobuf.Timestamp` | | | `section_hashes` | 3 | map of `string` to `bytes` | Section name → SHA-256 of that section's canonical encoding. | | `system` | 10 | [`SystemSection`](#systemsection) | | | `cpu` | 11 | [`CpuSection`](#cpusection) | | | `memory` | 12 | [`MemorySection`](#memorysection) | | | `disks` | 13 | [`DisksSection`](#diskssection) | | | `nics` | 14 | [`NicsSection`](#nicssection) | | | `gpus` | 15 | [`GpusSection`](#gpussection) | | | `bmc` | 16 | [`BmcSection`](#bmcsection) | | | `os` | 17 | [`OsSection`](#ossection) | | | `images` | 18 | [`ImagesSection`](#imagessection) | (v0.3) Section "images". | ### ImagesSection [#imagessection] | Field | Number | Type | Description | | ---------------- | ------ | -------------------------------------- | --------------------------------------------------------------------------- | | `images` | 1 | repeated [`CachedImage`](#cachedimage) | Sorted by sha256; at most 1,000 entries (the most recently used). | | `capacity_bytes` | 2 | `uint64` | Cache capacity and use in bytes, when the runtime knows them (0 = unknown). | | `used_bytes` | 3 | `uint64` | | ### CachedImage [#cachedimage] | Field | Number | Type | Description | | ------------ | ------ | --------------------------------------- | ----------------------------------------------------------------------------------- | | `image_id` | 1 | `string` | The catalog image id (img\_…) the node pulled it for, when it knows it; else empty. | | `ref` | 2 | `string` | The reference it was pulled by (https URL or OCI ref). | | `sha256` | 3 | `bytes` | | | `size_bytes` | 4 | `uint64` | | | `state` | 5 | [`CachedImageState`](#cachedimagestate) | | | `last_used` | 6 | `google.protobuf.Timestamp` | Last use, truncated to the hour so the section hash does not churn on every use. | ### SystemSection [#systemsection] | Field | Number | Type | Description | | ---------------- | ------ | ----------------------- | ------------------------------ | | `vendor` | 1 | `string` | | | `product` | 2 | `string` | | | `serial` | 3 | `string` | | | `product_uuid` | 4 | `string` | Lowercase. | | `board_vendor` | 5 | `string` | | | `board_product` | 6 | `string` | | | `board_serial` | 7 | `string` | | | `chassis_serial` | 8 | `string` | | | `bios_vendor` | 9 | `string` | | | `bios_version` | 10 | `string` | | | `boot_mode` | 11 | [`BootMode`](#bootmode) | | | `secure_boot` | 12 | optional `bool` | Absent when unknown. | | `tpm` | 13 | [`Tpm`](#tpm) | Absent when no TPM is present. | ### Tpm [#tpm] | Field | Number | Type | Description | | ---------------- | ------ | -------- | ----------- | | `version` | 1 | `string` | e.g. "2.0". | | `manufacturer` | 2 | `string` | | | `ek_cert_issuer` | 3 | `string` | | ### CpuSection [#cpusection] | Field | Number | Type | Description | | --------------- | ------ | -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `vendor` | 1 | `string` | | | `model` | 2 | `string` | | | `architecture` | 3 | `string` | "x86\_64", "aarch64". | | `sockets` | 4 | `uint32` | | | `cores` | 5 | `uint32` | | | `threads` | 6 | `uint32` | | | `flags` | 7 | repeated `string` | Sorted, e.g. "amx\_tile", "avx512f". | | `numa_nodes` | 8 | repeated [`NumaNode`](#numanode) | Sorted by id. | | `isolated_cpus` | 9 | `string` | (v1.1 §3.8) CPUs the kernel isolates from the scheduler (/sys/devices/system/cpu/isolated, cpulist form; empty when none): bcd pins VMs and holds there first, and never counts them for the host's own reserve. | ### NumaNode [#numanode] | Field | Number | Type | Description | | -------------- | ------ | ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | 1 | `uint32` | | | `cpus` | 2 | `string` | Kernel cpulist form, e.g. "0-31,64-95". | | `memory_bytes` | 3 | `uint64` | | | `hugepages` | 4 | repeated [`HugepagePool`](#hugepagepool) | (v1.1 §3.8) The node's hugepage pools (/sys/devices/system/node/nodeN/hugepages), sorted by page size: NUMA-local hugepage backing books from these. | ### MemorySection [#memorysection] | Field | Number | Type | Description | | ------------- | ------ | ---------------------------------------- | ---------------------------------------------------- | | `total_bytes` | 1 | `uint64` | | | `hugepages` | 2 | repeated [`HugepagePool`](#hugepagepool) | Sorted by page size. | | `dimms` | 3 | repeated [`Dimm`](#dimm) | Sorted by slot; empty when dmidecode is unavailable. | | `edac` | 4 | [`EdacCounters`](#edaccounters) | Absent when EDAC is unavailable. | ### HugepagePool [#hugepagepool] | Field | Number | Type | Description | | ----------------- | ------ | -------- | ---------------------------------- | | `page_size_bytes` | 1 | `uint64` | 2097152 (2Mi) or 1073741824 (1Gi). | | `total` | 2 | `uint64` | | ### Dimm [#dimm] | Field | Number | Type | Description | | -------------- | ------ | -------- | ------------ | | `slot` | 1 | `string` | | | `size_bytes` | 2 | `uint64` | | | `speed_mts` | 3 | `uint32` | | | `type` | 4 | `string` | e.g. "DDR5". | | `ecc` | 5 | `bool` | | | `manufacturer` | 6 | `string` | | | `part_number` | 7 | `string` | | | `serial` | 8 | `string` | | ### EdacCounters [#edaccounters] | Field | Number | Type | Description | | --------------- | ------ | -------- | ----------- | | `correctable` | 1 | `uint64` | | | `uncorrectable` | 2 | `uint64` | | ### DisksSection [#diskssection] | Field | Number | Type | Description | | ------- | ------ | ------------------------ | --------------- | | `disks` | 1 | repeated [`Disk`](#disk) | Sorted by name. | ### Disk [#disk] | Field | Number | Type | Description | | ------------ | ------ | --------------------------------- | --------------------------------- | | `name` | 1 | `string` | Kernel name, e.g. "nvme0n1". | | `model` | 2 | `string` | | | `serial` | 3 | `string` | | | `wwn` | 4 | `string` | Stable device id when available. | | `size_bytes` | 5 | `uint64` | | | `rotational` | 6 | `bool` | | | `transport` | 7 | [`DiskTransport`](#disktransport) | | | `smart` | 8 | [`SmartStatus`](#smartstatus) | Absent when SMART is unavailable. | | `removable` | 9 | `bool` | | ### SmartStatus [#smartstatus] | Field | Number | Type | Description | | -------------- | ------ | ----------------- | ----------- | | `healthy` | 1 | `bool` | | | `wear_percent` | 2 | optional `uint32` | | | `media_errors` | 3 | optional `uint64` | | ### NicsSection [#nicssection] | Field | Number | Type | Description | | ------ | ------ | ---------------------- | ---------------------------------- | | `nics` | 1 | repeated [`Nic`](#nic) | Physical NICs only, sorted by MAC. | ### Nic [#nic] | Field | Number | Type | Description | | ----------------- | ------ | ------------------------- | ------------------------------------------------ | | `name` | 1 | `string` | | | `mac` | 2 | `string` | Permanent MAC, lowercase colon form. | | `driver` | 3 | `string` | | | `firmware` | 4 | `string` | | | `speed_mbps` | 5 | `uint32` | | | `pci_address` | 6 | `string` | e.g. "0000:17:00.0". | | `numa_node` | 7 | optional `uint32` | Absent when the device reports no NUMA affinity. | | `sriov_total_vfs` | 8 | `uint32` | | | `rdma` | 9 | [`RdmaType`](#rdmatype) | | | `link` | 10 | [`LinkState`](#linkstate) | | | `mtu` | 11 | `uint32` | | ### GpusSection [#gpussection] | Field | Number | Type | Description | | --------------------- | ------ | ---------------------- | ---------------------------------------------------- | | `driver_version` | 1 | `string` | Kernel driver version, e.g. "580.65.06" (node-wide). | | `cuda_driver_version` | 2 | `string` | e.g. "13.0". | | `nvml_version` | 3 | `string` | | | `gpus` | 4 | repeated [`Gpu`](#gpu) | Sorted by index. | ### Gpu [#gpu] | Field | Number | Type | Description | | ------------------------------- | ------ | -------------------------------------- | -------------------------------------------------------------------------------- | | `index` | 1 | `uint32` | | | `uuid` | 2 | `string` | "GPU-…". | | `pci_bus_id` | 3 | `string` | e.g. "00000000:18:00.0". | | `name` | 4 | `string` | Raw NVML name, e.g. "NVIDIA H100 80GB HBM3". | | `architecture` | 5 | `string` | e.g. "hopper", "blackwell". | | `vram_bytes` | 6 | `uint64` | | | `serial` | 7 | `string` | | | `vbios_version` | 8 | `string` | | | `board_part_number` | 9 | `string` | | | `numa_node` | 10 | optional `uint32` | | | `power_limit_mw` | 11 | `uint32` | | | `persistence_mode` | 12 | `bool` | | | `compute_mode` | 13 | `string` | e.g. "default", "exclusive-process". | | `cuda_compute_capability_major` | 14 | `uint32` | | | `cuda_compute_capability_minor` | 15 | `uint32` | | | `mig` | 16 | [`GpuMig`](#gpumig) | | | `nvlinks` | 17 | repeated [`NvlinkLink`](#nvlinklink) | Sorted by link. | | `topology` | 18 | repeated [`GpuPeerLink`](#gpupeerlink) | Topology matrix row for this GPU: one entry per other GPU, sorted by peer\_uuid. | | `fabric` | 19 | [`GpuFabric`](#gpufabric) | Absent unless the GPU is part of a multi-node NVLink domain (e.g. GB200 NVL72). | | `ecc` | 20 | [`GpuEcc`](#gpuecc) | | | `row_remap` | 21 | [`GpuRowRemap`](#gpurowremap) | Absent on GPUs without row remapping. | ### GpuMig [#gpumig] | Field | Number | Type | Description | | ----------- | ------ | -------------------------------------- | ---------------------------------------------------------- | | `capable` | 1 | `bool` | | | `current` | 2 | [`MigMode`](#migmode) | | | `pending` | 3 | [`MigMode`](#migmode) | Set when a mode change waits for a GPU reset. | | `profiles` | 4 | repeated [`MigProfile`](#migprofile) | GPU instance profiles, sorted by id. | | `instances` | 5 | repeated [`MigInstance`](#miginstance) | Current GPU/compute instances, sorted by (gi\_id, ci\_id). | ### MigProfile [#migprofile] | Field | Number | Type | Description | | --------------- | ------ | -------- | ----------------------------- | | `id` | 1 | `uint32` | NVML GPU instance profile id. | | `name` | 2 | `string` | e.g. "1g.10gb". | | `max_instances` | 3 | `uint32` | | | `memory_bytes` | 4 | `uint64` | | ### MigInstance [#miginstance] | Field | Number | Type | Description | | --------- | ------ | -------- | ----------- | | `gi_id` | 1 | `uint32` | | | `ci_id` | 2 | `uint32` | | | `profile` | 3 | `string` | | | `uuid` | 4 | `string` | "MIG-…". | ### NvlinkLink [#nvlinklink] | Field | Number | Type | Description | | --------- | ------ | ------------------------- | -------------------------------------------------------- | | `link` | 1 | `uint32` | | | `state` | 2 | [`LinkState`](#linkstate) | | | `peer` | 3 | `string` | A GPU UUID, or "nvswitch:\" behind NVSwitch. | | `version` | 4 | `uint32` | | ### GpuPeerLink [#gpupeerlink] One cell of the nvidia-smi topo -m matrix. | Field | Number | Type | Description | | ----------- | ------ | ----------------------------- | ----------------------------------------------------------------------------- | | `peer_uuid` | 1 | `string` | | | `link` | 2 | [`GpuLinkType`](#gpulinktype) | | | `nvlinks` | 3 | `uint32` | Number of NVLinks for GPU\_LINK\_TYPE\_NV (the "NV\" figure); 0 otherwise. | ### GpuFabric [#gpufabric] MNNVL / IMEX domain membership. | Field | Number | Type | Description | | -------------- | ------ | -------- | ----------- | | `cluster_uuid` | 1 | `string` | | | `clique_id` | 2 | `uint32` | | ### GpuEcc [#gpuecc] | Field | Number | Type | Description | | ------------------ | ------ | ------------------------- | ------------------------------------------ | | `enabled` | 1 | `bool` | | | `pending_enabled` | 2 | optional `bool` | Set when a mode change waits for a reboot. | | `volatile_counts` | 3 | [`EccCounts`](#ecccounts) | Since the last driver load. | | `aggregate_counts` | 4 | [`EccCounts`](#ecccounts) | Lifetime. | ### EccCounts [#ecccounts] | Field | Number | Type | Description | | --------------- | ------ | -------- | ----------------- | | `correctable` | 1 | `uint64` | Single-bit (SBE). | | `uncorrectable` | 2 | `uint64` | Double-bit (DBE). | ### GpuRowRemap [#gpurowremap] | Field | Number | Type | Description | | --------------- | ------ | -------- | --------------------------------------- | | `correctable` | 1 | `uint32` | | | `uncorrectable` | 2 | `uint32` | | | `pending` | 3 | `bool` | A remap waits for a GPU reset (XID 63). | | `failure` | 4 | `bool` | Remapping failed (XID 64): RMA. | ### BmcSection [#bmcsection] | Field | Number | Type | Description | | ------------------ | ------ | -------- | ----------- | | `ip` | 1 | `string` | | | `mac` | 2 | `string` | | | `vendor` | 3 | `string` | | | `firmware_version` | 4 | `string` | | | `redfish` | 5 | `bool` | | | `ipmi` | 6 | `bool` | | ### OsSection [#ossection] | Field | Number | Type | Description | | ---------------- | ------ | --------------------------------- | ----------------------------------------------- | | `distro` | 1 | `string` | os-release ID, e.g. "ubuntu". | | `distro_version` | 2 | `string` | os-release VERSION\_ID, e.g. "24.04". | | `kernel` | 3 | `string` | uname -r. | | `cgroup` | 4 | [`CgroupVersion`](#cgroupversion) | | | `kvm` | 5 | `bool` | | | `iommu` | 6 | `bool` | | | `runtimes` | 7 | repeated `string` | Sorted, e.g. "containerd", "docker", "libvirt". | | `hostname` | 8 | `string` | | ### CachedImageState [#cachedimagestate] | Value | Number | Description | | -------------------------------- | ------ | ----------- | | `CACHED_IMAGE_STATE_UNSPECIFIED` | 0 | | | `CACHED_IMAGE_STATE_READY` | 1 | | | `CACHED_IMAGE_STATE_PULLING` | 2 | | | `CACHED_IMAGE_STATE_EVICTING` | 3 | | ### BootMode [#bootmode] | Value | Number | Description | | ----------------------- | ------ | ----------- | | `BOOT_MODE_UNSPECIFIED` | 0 | | | `BOOT_MODE_UEFI` | 1 | | | `BOOT_MODE_BIOS` | 2 | | ### DiskTransport [#disktransport] | Value | Number | Description | | ---------------------------- | ------ | ----------- | | `DISK_TRANSPORT_UNSPECIFIED` | 0 | | | `DISK_TRANSPORT_NVME` | 1 | | | `DISK_TRANSPORT_SATA` | 2 | | | `DISK_TRANSPORT_SAS` | 3 | | | `DISK_TRANSPORT_VIRTIO` | 4 | | | `DISK_TRANSPORT_USB` | 5 | | ### RdmaType [#rdmatype] | Value | Number | Description | | ----------------------- | ------ | ----------- | | `RDMA_TYPE_UNSPECIFIED` | 0 | | | `RDMA_TYPE_NONE` | 1 | | | `RDMA_TYPE_INFINIBAND` | 2 | | | `RDMA_TYPE_ROCE` | 3 | | ### LinkState [#linkstate] | Value | Number | Description | | ------------------------ | ------ | ----------- | | `LINK_STATE_UNSPECIFIED` | 0 | | | `LINK_STATE_UP` | 1 | | | `LINK_STATE_DOWN` | 2 | | ### MigMode [#migmode] | Value | Number | Description | | ---------------------- | ------ | ----------- | | `MIG_MODE_UNSPECIFIED` | 0 | | | `MIG_MODE_DISABLED` | 1 | | | `MIG_MODE_ENABLED` | 2 | | ### GpuLinkType [#gpulinktype] Best to worst: NV > PIX > PXB > PHB > NODE > SYS (PLAN.md §10.3 topology scorer). | Value | Number | Description | | --------------------------- | ------ | --------------------------------------------------- | | `GPU_LINK_TYPE_UNSPECIFIED` | 0 | | | `GPU_LINK_TYPE_NV` | 1 | Connected by NVLink (directly or through NVSwitch). | | `GPU_LINK_TYPE_PIX` | 2 | At most one PCIe bridge. | | `GPU_LINK_TYPE_PXB` | 3 | Multiple PCIe bridges, no host bridge. | | `GPU_LINK_TYPE_PHB` | 4 | A PCIe host bridge. | | `GPU_LINK_TYPE_NODE` | 5 | Across host bridges within one NUMA node. | | `GPU_LINK_TYPE_SYS` | 6 | Across NUMA nodes (SMP interconnect). | ### CgroupVersion [#cgroupversion] | Value | Number | Description | | ---------------------------- | ------ | ----------- | | `CGROUP_VERSION_UNSPECIFIED` | 0 | | | `CGROUP_VERSION_V1` | 1 | | | `CGROUP_VERSION_V2` | 2 | | ## session.proto [#sessionproto] Package `bettercompute.agent.v1`, `proto/bettercompute/agent/v1/session.proto`. The daemon ↔ control-plane protocol (PLAN.md §7): join, renew, the bidi Session stream and its frames, desired state (NodeSpec) and tunnels. ### AgentGateway [#agentgateway] The gateway-facing service every bcd mode (node, site, ramdisk) talks to. | RPC | Request | Response | Description | | --------- | ------------------------------------ | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------ | | `Join` | [`JoinRequest`](#joinrequest) | [`JoinResponse`](#joinresponse) | Server-authenticated TLS (serving CA pinned). Forwarded to the api JoinService; the gateway cannot read the sealed token or forge the signed response. | | `Renew` | [`RenewRequest`](#renewrequest) | [`RenewResponse`](#renewresponse) | mTLS, at \~60% of the certificate lifetime. Verified by the api JoinService, not the gateway. | | `Session` | stream [`AgentFrame`](#agentframe) | stream [`ServerFrame`](#serverframe) | mTLS. One live session per node; a newer session supersedes an older one (ABORTED). | | `Tunnel` | stream [`TunnelFrame`](#tunnelframe) | stream [`TunnelFrame`](#tunnelframe) | (v0.3) mTLS. Opened by the daemon after it verified an OpenTunnel ticket. | ### JoinRequest [#joinrequest] | Field | Number | Type | Description | | ---------------- | ------ | ----------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `sealed_token` | 1 | `bytes` | UTF-8 compact JWE of the join token ("bcjt\_\.\") to the pinned JoinService encryption key. The token is read from BCD\_JOIN\_TOKEN or stdin only and never stored. | | `csr` | 2 | `bytes` | DER PKCS#10 CSR for a fresh P-256 key (TPM-resident when available). | | `claims` | 3 | [`HardwareClaims`](#hardwareclaims) | | | `attestor` | 4 | `string` | Attestor that verifies this join: "join-token" (v0.1); "x509pop", "provisioned", "discovery" (v0.2); "cloud-iid" (v0.3); "tpm" (v0.5). Empty means "join-token". | | `attestation` | 5 | `bytes` | Attestor-specific evidence (v0.2+), e.g. a site agent's SignedDiscoveryAssertion. | | `daemon_version` | 6 | `string` | | | `role` | 7 | [`AgentRole`](#agentrole) | The role the agent runs as. The token decides what is allowed; a mismatch is rejected. | ### HardwareClaims [#hardwareclaims] What the daemon reports about its hardware at join. JoinService derives `hardware_key` = sha256(product\_uuid ‖ product\_serial ‖ sorted permanent MACs ‖ sorted GPU UUIDs) after dropping placeholder values, and enforces the token's `expect` binding and reclaim rules. | Field | Number | Type | Description | | ---------------- | ------ | ----------------- | --------------------------------------------------------------- | | `product_uuid` | 1 | `string` | /sys/class/dmi/id/product\_uuid, lowercase. | | `product_serial` | 2 | `string` | /sys/class/dmi/id/product\_serial. | | `board_serial` | 3 | `string` | | | `chassis_serial` | 4 | `string` | | | `sys_vendor` | 5 | `string` | | | `product_name` | 6 | `string` | | | `permanent_macs` | 7 | repeated `string` | Permanent MAC addresses of physical NICs, lowercase colon form. | | `gpu_uuids` | 8 | repeated `string` | | | `bmc_mac` | 9 | `string` | BMC MAC when readable in-band. | | `hostname` | 10 | `string` | | | `boot_id` | 11 | `string` | | ### JoinResponse [#joinresponse] | Field | Number | Type | Description | | ------ | ------ | ------------------------- | ----------------------------------------------------------------------- | | `body` | 1 | `bytes` | Serialized JoinResponseBody. | | `sig` | 2 | [`Signature`](#signature) | Pinned JoinService signing key. Purpose "join-response", parts \[body]. | ### JoinResponseBody [#joinresponsebody] | Field | Number | Type | Description | | -------------- | ------ | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `node_id` | 1 | `string` | | | `cert_chain` | 2 | repeated `bytes` | DER certificates, leaf first. The leaf has URI SAN spiffe://\/site/\/node/\ and a role OID; no tenant. | | `trust` | 3 | [`TrustBundle`](#trustbundle) | Command/spec signing trust (empty roots in v0.1). | | `csr_sha256` | 4 | `bytes` | SHA-256 of JoinRequest.csr: the daemon rejects a response that is not for its own CSR. | | `trust_domain` | 5 | `string` | | | `site_id` | 6 | `string` | | | `role` | 7 | [`AgentRole`](#agentrole) | | | `issued_at` | 8 | `google.protobuf.Timestamp` | Server time at issue, for a first clock-offset estimate. | | `deferral` | 9 | [`JoinDeferral`](#joindeferral) | (v0.2) Set when the attestor could not decide yet: node\_id and cert\_chain are empty, nothing was issued and no token was spent. The daemon re-sends Join with the SAME csr (csr\_sha256 still binds this response to it): after `pending.retry_after_seconds`, or with its answer to `challenge` in JoinRequest.attestation. | ### JoinDeferral [#joindeferral] A two-round join (§3.3): an operator approval (discovery → joinRequests) or an attestor challenge (tpm credential activation). | Field | Number | Type | Description | | ----------- | ------ | ------------------------------------------------ | ----------- | | `pending` | 1 | [`JoinPending`](#joinpending) (oneof `kind`) | | | `challenge` | 2 | [`JoinChallenge`](#joinchallenge) (oneof `kind`) | | ### JoinPending [#joinpending] | Field | Number | Type | Description | | --------------------- | ------ | -------- | ------------------------------------------------------------------- | | `request_id` | 1 | `string` | The join request an operator approves or rejects (joinRequests.\*). | | `retry_after_seconds` | 2 | `uint32` | | ### JoinChallenge [#joinchallenge] | Field | Number | Type | Description | | ----------- | ------ | --------------------------- | ----------------------------------------------------------------------------------- | | `attestor` | 1 | `string` | The attestor that answers it, as in JoinRequest.attestor. | | `payload` | 2 | `bytes` | Attestor-specific (tpm: credential blob, encrypted secret and opaque server state). | | `not_after` | 3 | `google.protobuf.Timestamp` | Answers after this are refused. | ### RenewRequest [#renewrequest] | Field | Number | Type | Description | | ----------------- | ------ | ----------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `csr` | 1 | `bytes` | DER PKCS#10 CSR (a new key, or the current one). | | `chain` | 2 | repeated `bytes` | The current certificate chain, leaf first. | | `client_time` | 3 | `google.protobuf.Timestamp` | | | `pop_sig` | 4 | `bytes` | Proof of possession by the CURRENT node key. Purpose "renew-pop", parts \[sha256(csr), node\_id, client\_time]. Accepted within ±60 s of server time; 120 s replay cache. | | `key_attestation` | 5 | [`TpmKeyAttestation`](#tpmkeyattestation) | Required when the current certificate names a TPM attestation key (URI SAN urn:bc:tpm-ak:\:\): that AK certifies the CSR's key as one the TPM holds. The new certificate carries the statement forward from the old one; without this the renewal is refused. | ### TpmKeyAttestation [#tpmkeyattestation] TPM2\_Certify of a node key by the AK a certificate names, qualifying data sha256(csr). | Field | Number | Type | Description | | -------------- | ------ | ------- | -------------------------------------------------------------------------------------------- | | `ak_public` | 1 | `bytes` | TPMT\_PUBLIC of the AK: its TPM name must be the one in the certificate. | | `key_public` | 2 | `bytes` | TPMT\_PUBLIC of the node key: the CSR's key, fixedTPM \| fixedParent \| sensitiveDataOrigin. | | `certify_info` | 3 | `bytes` | The TPMS\_ATTEST (TPM\_ST\_ATTEST\_CERTIFY) and the AK's TPMT\_SIGNATURE over it. | | `signature` | 4 | `bytes` | | ### RenewResponse [#renewresponse] | Field | Number | Type | Description | | ------------ | ------ | --------------------------- | ----------------------------- | | `cert_chain` | 1 | repeated `bytes` | DER certificates, leaf first. | | `not_after` | 2 | `google.protobuf.Timestamp` | | ### AgentFrame [#agentframe] | Field | Number | Type | Description | | ------------------------ | ------ | ------------------------------------------------------------- | ---------------------------------------------------------------------------------- | | `hello` | 1 | [`Hello`](#hello) (oneof `msg`) | | | `heartbeat` | 2 | [`Heartbeat`](#heartbeat) (oneof `msg`) | | | `update` | 3 | [`CommandUpdate`](#commandupdate) (oneof `msg`) | | | `events` | 4 | [`EventBatch`](#eventbatch) (oneof `msg`) | | | `status` | 5 | [`StatusReport`](#statusreport) (oneof `msg`) | | | `inventory` | 6 | [`InventorySection`](#inventorysection) (oneof `msg`) | | | `spec_request` | 7 | [`SpecRequest`](#specrequest) (oneof `msg`) | | | `secret_request` | 8 | [`SecretRequest`](#secretrequest) (oneof `msg`) | | | `workload_token_request` | 9 | [`WorkloadTokenRequest`](#workloadtokenrequest) (oneof `msg`) | (v0.5) A hosted instance's IMDS asks for its workload identity token (§12.2, §13). | ### ServerFrame [#serverframe] | Field | Number | Type | Description | | ---------------- | ------ | --------------------------------------------------------------- | -------------------------------------------- | | `welcome` | 1 | [`Welcome`](#welcome) (oneof `msg`) | | | `command` | 2 | [`Command`](#command) (oneof `msg`) | | | `cancel` | 3 | [`Cancel`](#cancel) (oneof `msg`) | | | `desired` | 4 | [`DesiredState`](#desiredstate) (oneof `msg`) | | | `ack` | 5 | [`Ack`](#ack) (oneof `msg`) | | | `open_tunnel` | 6 | [`OpenTunnel`](#opentunnel) (oneof `msg`) | | | `drain` | 7 | [`Drain`](#drain) (oneof `msg`) | | | `revoked` | 8 | [`RevocationNotice`](#revocationnotice) (oneof `msg`) | | | `secret` | 9 | [`SecretResponse`](#secretresponse) (oneof `msg`) | | | `workload_token` | 10 | [`WorkloadTokenResponse`](#workloadtokenresponse) (oneof `msg`) | (v0.5) The answer to a WorkloadTokenRequest. | ### Hello [#hello] The first frame of every session. Kept small for reconnect storms: inventory travels as hashes. | Field | Number | Type | Description | | --------------------- | ------ | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `node_id` | 1 | `string` | Must equal the node id in the certificate SAN. | | `boot_id` | 2 | `string` | | | `journal_id` | 3 | `string` | The journal's id. A change (journal lost: ramdisk reboot, reinstall, corruption) makes the control plane fail in-flight commands with retry\_safe=false ("journal-lost") instead of replaying them. | | `role` | 4 | [`AgentRole`](#agentrole) | | | `daemon_version` | 5 | `string` | | | `proto_min` | 6 | `uint32` | Supported protocol range (PROTOCOL\_VERSION in @better-compute/protocol). Below the server's minimum the session is restricted to daemon.upgrade (Welcome.upgrade\_required). | | `proto_max` | 7 | `uint32` | | | `capabilities` | 8 | repeated `string` | | | `drivers` | 9 | repeated [`DriverManifest`](#drivermanifest) | | | `last_epoch` | 10 | `uint64` | The last session epoch this daemon saw (0 if none); diagnostics only. | | `applied_generation` | 11 | `uint64` | The newest NodeSpec generation the daemon holds (converged or not). | | `unacked_results` | 12 | repeated [`CommandUpdate`](#commandupdate) | Terminal results not yet acked; the control plane dedupes by command\_id. | | `running_command_ids` | 13 | repeated `string` | Commands the journal holds as running; the control plane does not re-send them. | | `last_event_seq` | 14 | `uint64` | The highest event seq written in this journal. | | `inventory_hashes` | 15 | map of `string` to `bytes` | Inventory section name → SHA-256 of its InventorySection.content. The control plane asks for sections whose hash differs (Welcome.request\_sections). | | `applied_spec_hash` | 16 | `bytes` | DesiredStateBody.spec\_hash of the applied generation, echoed so the control plane can detect a divergent spec and resend a full one. | | `trust_version` | 17 | `uint64` | TrustBundle.version the daemon holds and verifies with (0: none). The gateway records it (node\_sessions.trust\_version) so a command-root switch waits for every connected node (§13). | | `approver_set_epoch` | 18 | `uint64` | (v1.0 §13) The approver set the daemon HOLDS: a bundle's version can be taken while its quorum part is refused (unsigned change, stale epoch), so trust\_version alone would read "up to date" for a node still enforcing the old approvers. approver\_set\_sha256 is SHA-256 of the held set's approverSetInput (empty: no approver keys held). | | `approver_set_sha256` | 19 | `bytes` | | ### Welcome [#welcome] | Field | Number | Type | Description | | ----------------------- | ------ | ----------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | `epoch` | 1 | `uint64` | node\_sessions.session\_epoch after this Hello's bump. | | `session_id` | 2 | `string` | | | `gateway_id` | 3 | `string` | | | `protocol_version` | 4 | `uint32` | The negotiated version: the highest version inside both ranges. | | `heartbeat_interval_ms` | 5 | `uint32` | Heartbeat period (10 000 ms by default). Three missed heartbeats close the session. | | `lease_ttl_ms` | 6 | `uint32` | Liveness lease TTL (40 000 ms by default). | | `credits` | 7 | repeated [`LaneCredits`](#lanecredits) | Per-lane in-flight limits the gateway keeps outstanding (control 8, provision 4, bulk 2). | | `acked_event_seq` | 8 | `uint64` | Cumulative ack for Hello.journal\_id; the daemon resends events with a higher seq. | | `request_sections` | 9 | repeated `string` | Inventory sections to send now (hash mismatch or never seen). | | `server_time` | 10 | `google.protobuf.Timestamp` | Server time (compute\_now()). The daemon keeps the offset to its own clock and evaluates not\_after and not\_before in server time. | | `trust` | 11 | [`SignedTrustBundle`](#signedtrustbundle) | Newer signing trust, if any. Only accepted when signed by the pinned JoinService key and newer than the bundle the daemon holds. | | `upgrade_required` | 12 | `bool` | The daemon is below the server's proto\_min: only daemon.upgrade is accepted (v0.4). | | `capabilities` | 13 | repeated `string` | | ### LaneCredits [#lanecredits] | Field | Number | Type | Description | | --------- | ------ | --------------- | ----------- | | `lane` | 1 | [`Lane`](#lane) | | | `credits` | 2 | `uint32` | | ### Heartbeat [#heartbeat] | Field | Number | Type | Description | | -------------------- | ------ | -------------------------------------- | ----------------------------------------------------------------------------------- | | `credits` | 1 | repeated [`LaneCredits`](#lanecredits) | Free credits per lane right now; the gateway resynchronises its counters from them. | | `load` | 2 | [`NodeLoad`](#nodeload) | | | `inventory_hashes` | 3 | map of `string` to `bytes` | Every 10th heartbeat: the same map as Hello.inventory\_hashes; empty otherwise. | | `applied_generation` | 4 | `uint64` | | ### NodeLoad [#nodeload] Coarse load for admission and dashboards. Telemetry proper never uses the session (P8). | Field | Number | Type | Description | | ------------------------ | ------ | -------- | ---------------------------------------- | | `load_average_1m` | 1 | `double` | 1-minute load average. | | `memory_available_bytes` | 2 | `uint64` | | | `running_commands` | 3 | `uint32` | Commands currently running across lanes. | | `pending_events` | 4 | `uint32` | Unacked events in the journal. | ### Command [#command] A one-shot action. Delivery is at-least-once; the daemon's journal makes it at-most-once in effect. The control plane stores `body` and `signature` verbatim and resends the same bytes. | Field | Number | Type | Description | | ---------------- | ------ | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `body` | 1 | `bytes` | Serialized CommandBody. | | `signature` | 2 | [`Signature`](#signature) | Worker signing key. Purpose "command", parts \[body]. Optional in v0.1; required for destructive kinds from v0.2 and for all kinds from v0.4 (the daemon's own kind table decides, never a flag an unsigned body could carry). | | `attempt` | 3 | `uint32` | Delivery attempt (1-based). Outside the signature: informational only. | | `batch_approval` | 4 | [`BatchApproval`](#batchapproval) | (v1.0, §13 quorum) For a body with batch\_id: the operators' approval of the batch and this command's inclusion proof. Outside the worker signature: the approval carries its own. | | `batch_proof` | 5 | [`BatchProof`](#batchproof) | | ### BatchApproval [#batchapproval] k-of-n operator approval of a batch of destructive commands (PLAN.md §13). The scope is a Merkle tree (RFC 6962 style, SHA-256) with one leaf per (subject, kind, payload), leaves ordered by their hash (bytewise ascending, duplicates removed): leaf = H(0x00 ‖ lp(batch\_id) ‖ lp(subject) ‖ lp(kind) ‖ lp(payload\_sha256)), node = H(0x01 ‖ left ‖ right), with lp(x) = uint32\_be(len(x)) ‖ x, and an odd node promoted unchanged to the next level. The subject is what the command acts on, computed by the executing agent from the signed body: * kinds a node runs on itself (disk.wipe, deploy.image, node.reboot, gpu.reset, boot.kexec…): the command's node\_id; * site-relayed BMC kinds (bmc.\*): "bmc:" + \ + "/" + BmcOperationRequest.address, the BMC acted on. BMC addresses are site-local (10.0.0.5 exists at every site), so the site is part of the subject: an approval for site A never admits a command at site B; * site.httpProxy: "node:" + its target\_node\_id (global ids). payload\_sha256 is SHA-256 of the command's payload (CommandBody.payload.value), binding the leaf to exactly that operation (image, firmware, power action), or EMPTY for "any payload of this kind" where the payload is built later (approvers see unbound leaves flagged). The executing agent accepts a proof that walks to the root from either the bound leaf (its payload's digest) or the unbound one. Daemons run each leaf at most once per batch. Signing purpose "batch-approval", parts \[batch\_id, scope\_root, uint32\_be(scope\_size), int64\_be(not\_after.seconds) ‖ int32\_be(not\_after.nanos), trust\_domain]. The trust domain (the deployment's, JoinResponseBody.trust\_domain) is supplied by each verifier from its own config, so an approval for staging never admits a command in production, even with shared approver keys. | Field | Number | Type | Description | | ------------ | ------ | ---------------------------------- | ------------------------------------------------------------------------------------- | | `batch_id` | 1 | `string` | | | `scope_root` | 2 | `bytes` | | | `scope_size` | 3 | `uint32` | Leaves in the tree: what the operators saw and approved ("reimage 412 nodes"). | | `not_after` | 4 | `google.protobuf.Timestamp` | Server time after which the approval no longer admits a command. | | `signatures` | 5 | repeated [`Signature`](#signature) | Approver key signatures (Signature.kid names an approver key); duplicates count once. | ### BatchProof [#batchproof] The path from this command's leaf to BatchApproval.scope\_root. | Field | Number | Type | Description | | ---------- | ------ | ---------------- | ---------------------------------------------------------------------------------------- | | `index` | 1 | `uint32` | The leaf's index in the sorted leaves. | | `siblings` | 2 | repeated `bytes` | Sibling hashes from the leaf level up; a promoted odd node has no sibling at that level. | ### CommandBody [#commandbody] | Field | Number | Type | Description | | ------------------ | ------ | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `id` | 1 | `string` | ULID; the journal's dedupe key. | | `node_id` | 2 | `string` | Must equal the session's node id; anything else is rejected UNSUPPORTED. | | `seq` | 3 | `uint64` | Per-node monotonic sequence (nodes.next\_cmd\_seq); resume replays in seq order. | | `kind` | 4 | `string` | Registered kind, e.g. "diag.nvml". Unknown or role-disallowed kinds → REJECTED/UNSUPPORTED. | | `payload` | 5 | `google.protobuf.Any` | The kind's request message (commands.proto), e.g. bettercompute.agent.v1.DiagNvmlRequest. | | `lane` | 6 | [`Lane`](#lane) | | | `not_after` | 7 | `google.protobuf.Timestamp` | Server time after which the command must not start. Checked before accepting (REJECTED/ EXPIRED); the CommandSweeper expires undelivered commands at the same instant. | | `min_generation` | 8 | `uint64` | The NodeSpec generation the command presupposes. If the daemon's applied generation is lower it sends SpecRequest and defers; still lower at not\_after → REJECTED/STALE\_GENERATION. 0 = none. | | `retry_safe` | 9 | `bool` | Safe to execute again when the journal shows `accepted` but not `running` after a crash. Otherwise that case reports FAILED\{"interrupted"} for the controller to decide. | | `idempotency_key` | 10 | `string` | UNIQUE(node\_id, idempotency\_key); workflow steps use "wf:\:\". | | `traceparent` | 11 | `string` | W3C trace context, propagated api → worker → gateway → daemon → driver. | | `self_terminating` | 12 | `bool` | The side effect ends this daemon process (node.reboot, boot.kexec, daemon.upgrade). The journal fsyncs running\{boot\_id} right before it; after restart with another boot\_id (or on rejoin with another role) it completes as SUCCEEDED\{rebooted}. Never replayed. | | `attested` | 13 | `bool` | The result is a "this node says" statement: CommandUpdate.node\_sig is required (v0.2) and the worker verifies it against node\_certs before any state transition. | | `workflow_id` | 14 | `string` | (v0.2) The workflow step this command executes; SecretRequest binds secrets to it. | | `step_id` | 15 | `string` | | | `batch_id` | 16 | `string` | (v1.0, §13 quorum) The batch this destructive command belongs to. Inside the signed body, so a compromised gateway can neither add nor strip it; the daemon then requires Command.batch\_approval + batch\_proof. | ### CommandUpdate [#commandupdate] Progress and outcome of one command. Terminal updates are journaled until acked and replayed in Hello.unacked\_results after a reconnect. | Field | Number | Type | Description | | --------------- | ------ | --------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | `command_id` | 1 | `string` | | | `state` | 2 | [`CommandState`](#commandstate) | One of ACKED, RUNNING, SUCCEEDED, FAILED, REJECTED, CANCELLED. | | `result` | 3 | `bytes` | The kind's serialized result message (commands.proto), when there is one. | | `boot_id` | 4 | `string` | The boot the daemon was in when it produced this update. | | `node_sig` | 5 | [`Signature`](#signature) | Node key, for attested kinds (v0.2). Purpose "command-update", parts \[command\_id, uint32\_be(state), sha256(result)]. | | `reject_reason` | 6 | [`CommandRejectReason`](#commandrejectreason) | Set when state is REJECTED. | | `error` | 7 | [`CommandError`](#commanderror) | Set when state is FAILED. | | `rebooted` | 8 | `bool` | A self-terminating command completed by the reboot/exec it caused. | ### CommandError [#commanderror] | Field | Number | Type | Description | | --------- | ------ | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------- | | `code` | 1 | `string` | Stable lowercase code: "interrupted" (accepted, not retry-safe, daemon restarted), "invalid-payload", "internal", or a kind- or driver-specific code. | | `message` | 2 | `string` | | ### Cancel [#cancel] Stop a command between actions. Never hides a completed side effect: the daemon reports the real outcome, or CANCELLED if nothing ran. | Field | Number | Type | Description | | ------------ | ------ | -------- | ----------- | | `command_id` | 1 | `string` | | | `reason` | 2 | `string` | | ### DesiredState [#desiredstate] Generation-numbered desired state. Signed like commands; the daemon stores the last full spec, its hash and signature in the journal and enforces it while offline. | Field | Number | Type | Description | | ----------- | ------ | ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- | | `body` | 1 | `bytes` | Serialized DesiredStateBody. | | `signature` | 2 | [`Signature`](#signature) | Worker signing key. Purpose "desired-state", parts \[body]. Optional in v0.1; required when the spec changes `access` (v0.2) and always from v0.4. | ### DesiredStateBody [#desiredstatebody] | Field | Number | Type | Description | | ----------------- | ------ | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `node_id` | 1 | `string` | | | `generation` | 2 | `uint64` | Monotonic. A generation not greater than the applied one is ignored. | | `base_generation` | 3 | `uint64` | 0 for a full spec. For a patch: the generation it applies to; a daemon at any other generation discards the patch and sends SpecRequest\{PATCH\_BASE\_MISMATCH}. | | `spec_hash` | 4 | `bytes` | Opaque id of the full spec at `generation` (SHA-256 over the control plane's canonical encoding). The daemon never recomputes it; it echoes it as applied\_spec\_hash. | | `full` | 5 | [`NodeSpec`](#nodespec) (oneof `spec`) | | | `patch` | 6 | [`NodeSpec`](#nodespec) (oneof `spec`) | Section-level patch: every section that is SET replaces that section wholesale (an empty section message clears it); unset sections keep their current value. | | `traceparent` | 7 | `string` | | ### NodeSpec [#nodespec] What the node should look like. The converger runs one idempotent plan(desired, observed) + apply per section; observed\_generation advances only when every section converged or failed terminally. A section a daemon does not know arrives as an unknown field: the daemon reports condition "SectionUnsupported" (reason = the field number) and converges the rest. | Field | Number | Type | Description | | ----------- | ------ | --------------------------------------- | ------------------------------------------------------------------------------------------- | | `host` | 1 | [`HostSection`](#hostsection) | | | `devices` | 2 | [`DevicesSection`](#devicessection) | | | `instances` | 3 | [`InstancesSection`](#instancessection) | (v0.3) | | `access` | 4 | [`AccessSection`](#accesssection) | (v0.3) Empty in v0.1. | | `telemetry` | 5 | [`TelemetrySection`](#telemetrysection) | | | `daemon` | 6 | [`DaemonSection`](#daemonsection) | (v0.4) | | `site` | 7 | [`SiteSpec`](#sitespec) | (v0.2, site agents only) DRAFT: its content is defined in site/v1, which is not yet frozen. | ### HostSection [#hostsection] Host configuration units (profile hostConfig, v0.2): nvidia-driver, container-toolkit, mig-layout, fabric-manager, sysctl, … Converged continuously so drift is corrected. | Field | Number | Type | Description | | ------- | ------ | ------------------------------------ | ----------- | | `units` | 1 | repeated [`ConfigUnit`](#configunit) | | ### ConfigUnit [#configunit] | Field | Number | Type | Description | | ------------- | ------ | -------- | ------------------------------------------------------------------------------------- | | `unit` | 1 | `string` | The config driver id, e.g. "nvidia-driver". | | `config_json` | 2 | `string` | Canonical JSON (RFC 8785) validated against the unit's manifest by the control plane. | ### DevicesSection [#devicessection] Device assignments (§3.4 step 9). The daemon rejects overlapping assignments, writes the CDI spec /etc/cdi/bcd-\.json, an env file and device ownership, then reports a node-signed AllocationBound; removing an assignment releases it (AllocationReleased). The daemon never drops an assignment on its own, not even while disconnected (P6). | Field | Number | Type | Description | | ------------- | ------ | ------------------------------------------------ | ----------- | | `assignments` | 1 | repeated [`DeviceAssignment`](#deviceassignment) | | ### DeviceAssignment [#deviceassignment] | Field | Number | Type | Description | | ---------------- | ------ | -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `allocation_id` | 1 | `string` | | | `reservation_id` | 2 | `string` | | | `binding` | 3 | [`Binding`](#binding) | | | `devices` | 4 | repeated [`AssignedDevice`](#assigneddevice) | | | `instance_id` | 5 | `string` | (v0.3) Set when binding is BINDING\_INSTANCE. | | `env` | 6 | repeated `string` | Extra "KEY=VALUE" lines for the allocation's env file. | | `limits` | 7 | [`ResourceLimits`](#resourcelimits) | (v1.1 §3.8) The CPU, memory and hugepages booked with the devices. For HOLD, bcd creates the cgroup-v2 slice bcd-alloc-\.slice with these limits and names it in the CDI spec; for KUBERNETES they are what the kubelet may give pods using these devices. Absent: devices only. | ### ResourceLimits [#resourcelimits] Host resources an allocation may use (cgroup v2 semantics). Zero or empty = not limited. | Field | Number | Type | Description | | --------------- | ------ | -------- | ------------------------------------------------------------------------------ | | `cpus` | 1 | `string` | CPUs the allocation may run on (cpuset.cpus, cpulist form, e.g. "8-15,72-79"). | | `mems` | 2 | `string` | NUMA nodes its memory comes from (cpuset.mems, e.g. "1"). | | `cpu_millis` | 3 | `uint32` | CPU time, in thousandths of a CPU (cpu.max; 4000 = 4 CPUs). | | `memory_bytes` | 4 | `uint64` | memory.max; bcd sets memory.high at 90% of it. | | `hugepages_2mi` | 5 | `uint64` | hugetlb.2MB.max / hugetlb.1GB.max, in pages. | | `hugepages_1gi` | 6 | `uint64` | | ### AssignedDevice [#assigneddevice] | Field | Number | Type | Description | | -------------- | ------ | -------- | -------------------------------------------------------------------------------------------- | | `id` | 1 | `string` | Atom id: GPU UUID, "\/gi\/ci\" (v0.5), "\/s\" (v0.5), disk WWN, … | | `device_class` | 2 | `string` | Device class: "gpu", "gpu.mig", "gpu.slot", "nic.vf", "disk". | ### InstancesSection [#instancessection] (v0.3) VPS instances on this host; the daemon converges each toward desired\_state. | Field | Number | Type | Description | | ----------- | ------ | ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `instances` | 1 | repeated [`InstanceSpec`](#instancespec) | | | `volumes` | 2 | repeated [`HostVolume`](#hostvolume) | (v1.1, agent memory) Persistent volumes kept on this host across instances. bcd creates each missing one (a sparse raw file under its data directory; growing is a truncate, seen by the guest at its next start; never shrunk) and destroys (hole-punches, then unlinks) one the section no longer lists once no instance attaches it: an absent volume is a deleted volume, so a spec is only ever missing one the control plane deleted. Only a bcd with the `instance.volumes` capability keeps them. | ### HostVolume [#hostvolume] | Field | Number | Type | Description | | ------------ | ------ | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------- | | `volume_id` | 1 | `string` | \[A-Za-z0-9\_-]\{1,80}. | | `size_bytes` | 2 | `uint64` | A positive multiple of 512. | | `serial` | 3 | `string` | The virtio-blk serial (at most 20 bytes, \[a-z0-9-]); the guest finds the disk at /dev/disk/by-id/virtio-\ and mounts it from its user data. | ### InstanceVolume [#instancevolume] (v1.1) A HostVolume attached to an instance as an extra virtio disk, in list order after the root disk. bcd refuses (InstanceStatus error) an instance naming a volume the section lacks. | Field | Number | Type | Description | | ----------- | ------ | -------- | ----------- | | `volume_id` | 1 | `string` | | | `read_only` | 2 | `bool` | | ### InstanceSpec [#instancespec] | Field | Number | Type | Description | | --------------------- | ------ | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `instance_id` | 1 | `string` | | | `desired_state` | 2 | [`InstanceDesiredState`](#instancedesiredstate) | | | `runtime` | 3 | `string` | Runtime driver id, e.g. "libvirt-kvm", "cloud-hypervisor". | | `allocation_id` | 4 | `string` | | | `vcpu` | 5 | `uint32` | | | `memory_bytes` | 6 | `uint64` | | | `disk_bytes` | 7 | `uint64` | | | `image` | 8 | [`ImageRef`](#imageref) | | | `user_data_ref` | 9 | `string` | Secret ref of the NoCloud user-data, fetched with SecretRequest\{allocation\_id}. | | `gpu_device_ids` | 10 | repeated `string` | GPU atoms passed through with VFIO (never overcommitted). | | `nics` | 11 | repeated [`InstanceNic`](#instancenic) | | | `restore_from` | 12 | [`ImageRef`](#imageref) | (v0.5, sandboxes) Boot from this snapshot (an instance.snapshot upload) instead of `image`: disk and, when captured, memory state. The daemon verifies sha256 before restoring and fails the instance on a runtime or CPU model mismatch. | | `network_policy` | 13 | [`InstanceNetworkPolicy`](#instancenetworkpolicy) | (v0.5, sandboxes) What the instance's NICs may reach, enforced by bcd as host nftables rules on its taps, whatever the runtime. bcd's own tunnel dial from the host always works. | | `identity_rebind_ref` | 14 | `string` | (v0.5, review 13 H2) After a restore, the guest's per-instance identity (SSH host keys, machine id, workload credentials) must not stay the snapshot's. A secret ref (fetched with SecretRequest) of the new identity material; bcd delivers it to the running guest (a hotplugged raw disk read by a vps-installed guest unit) and reports the ref back in InstanceStatus.identity\_rebound\_ref once the guest acked it. Changing it asks for a new rebind. | | `ssh_from_host_only` | 15 | `bool` | (review 13 L5) Port 22 of the guest is reachable only from the host (bcd's tunnel dial), never from other guests or the network, whatever network\_policy allows. | | `placement` | 16 | [`InstancePlacement`](#instanceplacement) | (v1.1 §3.8) Where the guest's vCPUs and memory live on the host. Absent: the runtime's default (floating vCPUs, host memory). | | `volumes` | 17 | repeated [`InstanceVolume`](#instancevolume) | (v1.1, agent memory) Persistent volumes to attach (InstancesSection.volumes). | ### InstancePlacement [#instanceplacement] | Field | Number | Type | Description | | --------------------- | ------ | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------- | | `cpus` | 1 | `string` | Host CPUs for the vCPUs (cpulist). With exactly `vcpu` CPUs, vCPU i is pinned to the i-th one; otherwise the vCPUs float within the set. | | `mems` | 2 | `string` | NUMA nodes the guest memory is bound to (e.g. "1"). | | `hugepage_size_bytes` | 3 | `uint64` | Back guest memory with hugepages of this size (2097152 or 1073741824); 0 = normal pages. The pages come from the host's pools on `mems`. | | `balloon` | 4 | `bool` | Give the guest a virtio balloon (libvirt memballoon, cloud-hypervisor balloon) so the host can reclaim free guest memory; never with hugepages. | ### ImageRef [#imageref] A content-addressed image the daemon may fetch and cache. | Field | Number | Type | Description | | ------------- | ------ | -------- | ------------------------------------------------------ | | `ref` | 1 | `string` | Catalog reference, e.g. "ubuntu-24.04-cloud\@2026.09". | | `url` | 2 | `string` | | | `sha256` | 3 | `bytes` | | | `format` | 4 | `string` | "raw" or "qcow2". | | `compression` | 5 | `string` | "" or "zstd". | ### InstanceNic [#instancenic] | Field | Number | Type | Description | | ------------- | ------ | ----------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `network` | 1 | `string` | Bridge or network name on the host. | | `mac` | 2 | `string` | | | `addresses` | 3 | repeated `string` | Addresses assigned by IPAM, CIDR form. | | `gateway` | 4 | `string` | (v1.1) The IPAM range's default gateway; empty: no default route through this NIC. bcd renders it, with `addresses`, into the guest's NoCloud network-config (the guest runs no DHCP client against the host). | | `nameservers` | 5 | repeated `string` | (v1.1) DNS servers for the guest; empty: the guest keeps its image's resolver configuration. | ### AccessSection [#accesssection] (v0.3) Interactive access material. Empty in v0.1. | Field | Number | Type | Description | | ----------------------- | ------ | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `exec_ticket_jwks_json` | 1 | `string` | JWKS (JSON) the daemon verifies exec tickets against (§7.6). | | `ssh_ca_public_keys` | 2 | repeated `string` | OpenSSH-format public keys of the compute SSH CA trusted for break-glass certificates. | | `ssh_principals` | 3 | repeated `string` | Principals such certificates may use. | | `ssh_revoked_serials` | 4 | repeated `uint64` | (v0.3) Serials of break-glass certificates revoked before they expired. The daemon writes them into an OpenSSH KRL (RevokedKeys), so a revoked certificate stops working at the next spec. | ### TelemetrySection [#telemetrysection] Where bcd sends its own telemetry (OTLP). Never the session or Postgres (P8). | Field | Number | Type | Description | | --------------------- | ------ | ----------------- | ---------------------------------------------------------------------- | | `otlp_endpoint` | 1 | `string` | Empty disables export. | | `export_interval_ms` | 2 | `uint32` | | | `resource_attributes` | 3 | repeated `string` | Extra "key=value" resource attributes. | | `headers_ref` | 4 | `string` | Secret ref of the exporter's auth headers, fetched with SecretRequest. | ### DaemonSection [#daemonsection] (v0.4) Self-update target; the Rollout controller sets it within the disruption budget. | Field | Number | Type | Description | | ----------- | ------ | -------- | --------------------------------------------- | | `version` | 1 | `string` | | | `url` | 2 | `string` | | | `sha256` | 3 | `bytes` | | | `signature` | 4 | `bytes` | Detached release signature over the artifact. | ### SpecRequest [#specrequest] Ask for a full DesiredState. | Field | Number | Type | Description | | ----------------- | ------ | ----------------------------------------- | ----------- | | `have_generation` | 1 | `uint64` | | | `reason` | 2 | [`SpecRequestReason`](#specrequestreason) | | ### StatusReport [#statusreport] Convergence state, coalesced latest-wins. | Field | Number | Type | Description | | --------------------- | ------ | -------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `observed_generation` | 1 | `uint64` | Every section converged or failed terminally at this generation. Never regresses on the control plane (GREATEST). | | `applied_generation` | 2 | `uint64` | The newest generation held (converging or converged). | | `applied_spec_hash` | 3 | `bytes` | | | `sections` | 4 | repeated [`SectionStatus`](#sectionstatus) | | | `conditions` | 5 | repeated [`Condition`](#condition) | The node's full current condition set as seen by the daemon (health, drift, section errors). | | `instances` | 6 | repeated [`InstanceStatus`](#instancestatus) | (v0.3) What the host runtimes hold: every VM or container they know, managed or not, so the control plane can move instances by what actually runs and find leaks (an instance no live row places here). Latest-wins with the rest of the report. | | `instances_complete` | 7 | `bool` | (v0.3) `instances` is the runtimes' whole inventory: a placed instance missing from it is absent. | | `pressure` | 8 | [`HostPressure`](#hostpressure) | (v0.3) Host resource pressure for the Disk/Memory/CpuPressure conditions (§10.1). Absent when the kernel exposes no PSI. | | `trust_version` | 9 | `uint64` | TrustBundle.version the daemon holds, as in Hello.trust\_version, reported again once it applies a newer bundle mid-session (Welcome.trust), so the ack does not wait for a reconnect. 0: unchanged. | | `approver_set_epoch` | 10 | `uint64` | The approver set held after that bundle, as in Hello (set whenever trust\_version is). | | `approver_set_sha256` | 11 | `bytes` | | | `discovered` | 12 | repeated [`DiscoveredMachine`](#discoveredmachine) | (v1.1, site agents only) Machines seen PXE/HTTP-booting (DHCP PXEClient/HTTPClient) from a MAC with no live boot entry: never answered, never enrolled; a list for metal to match (by product\_uuid, the SMBIOS UUID a BMC also reports) or an operator to act on. The whole list, capped at 256 (oldest last\_seen evicted, forgotten 24 h after last seen). | | `volume_storage` | 13 | [`VolumeStorage`](#volumestorage) | (v1.1, hosts with a runtime) The volume store's filesystem, so capacity books persistent volumes against what can really be written (volumes are sparse). bcd refuses a volume create or growth beyond available − reserved − outstanding; capacity applies the same rule. | ### VolumeStorage [#volumestorage] | Field | Number | Type | Description | | ------------------- | ------ | -------- | ------------------------------------------------------------------------------- | | `total_bytes` | 1 | `uint64` | The data directory's filesystem (statfs). | | `available_bytes` | 2 | `uint64` | statfs Bavail. | | `reserved_bytes` | 3 | `uint64` | Kept from volumes: overlays, journal, images (5% of total, at least 2 GiB). | | `committed_bytes` | 4 | `uint64` | The sum of every volume's size. | | `outstanding_bytes` | 5 | `uint64` | The sum of every volume's unwritten (sparse) space: promised but not yet taken. | ### DiscoveredMachine [#discoveredmachine] | Field | Number | Type | Description | | -------------- | ------ | --------------------------- | --------------------------------------------------------------------------------------------- | | `mac` | 1 | `string` | Lowercase colon form. | | `product_uuid` | 2 | `string` | DHCP option 97 (the SMBIOS system UUID); empty when the client sent none. | | `arch` | 3 | `string` | DHCP option 93, as the site agent names it: x86\_64-efi, bios, arm64-efi, x86\_64-efi-http, … | | `first_seen` | 4 | `google.protobuf.Timestamp` | | | `last_seen` | 5 | `google.protobuf.Timestamp` | | | `count` | 6 | `uint32` | | | `relay` | 7 | `string` | giaddr when the request was relayed (which L2 segment it came from); empty when direct. | ### HostPressure [#hostpressure] Linux pressure stall information (/proc/pressure/\*, "some" avg10, percent of wall time) and CPU steal (percent over the last report interval). | Field | Number | Type | Description | | ------------------- | ------ | ------- | ----------- | | `cpu_some_avg10` | 1 | `float` | | | `memory_some_avg10` | 2 | `float` | | | `io_some_avg10` | 3 | `float` | | | `steal_pct` | 4 | `float` | | ### InstanceStatus [#instancestatus] (v0.3) One instance as a host runtime reports it. | Field | Number | Type | Description | | ---------------------- | ------ | ------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `instance_id` | 1 | `string` | The InstanceSpec.instance\_id, or the runtime's own name for an object bcd did not create. | | `state` | 2 | [`InstanceObservedState`](#instanceobservedstate) | | | `addresses` | 3 | repeated `string` | Addresses the guest holds, CIDR form (guest agent or DHCP leases). | | `message` | 4 | `string` | | | `runtime` | 5 | `string` | Runtime driver id, e.g. "libvirt-kvm". | | `identity_rebound_ref` | 6 | `string` | (review 13 H2) The InstanceSpec.identity\_rebind\_ref the guest acknowledged (its nonce written back); empty until then. Until it matches the spec's, the control plane refuses exec into a restored instance (it would still carry the snapshot's identity); readiness still means "runs", since guests restored from snapshots taken before the guest half existed never ack. | ### SectionStatus [#sectionstatus] | Field | Number | Type | Description | | ------------ | ------ | ------------------------------- | ----------------------------------------------------------------------------------------------- | | `section` | 1 | `string` | NodeSpec section name: "host", "devices", "instances", "access", "telemetry", "daemon", "site". | | `state` | 2 | [`SectionState`](#sectionstate) | | | `generation` | 3 | `uint64` | The generation this status refers to. | | `reason` | 4 | `string` | | | `message` | 5 | `string` | | ### Condition [#condition] A Kubernetes-style condition (PLAN.md §10.1). | Field | Number | Type | Description | | ----------- | ------ | ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `type` | 1 | `string` | e.g. "SectionUnsupported", "DriverMismatch", "Xid79" (v0.2). | | `status` | 2 | [`ConditionStatus`](#conditionstatus) | | | `reason` | 3 | `string` | | | `message` | 4 | `string` | | | `device_id` | 5 | `string` | When set, the condition concerns one device (atom id). | | `age_ms` | 6 | `uint64` | How long ago the daemon saw the last transition, on its monotonic clock. The control plane derives `since` from compute\_now() (P9: agents report ages, not timestamps). | ### InventorySection [#inventorysection] One inventory section, sent when Welcome.request\_sections asks for it, when inventory.refresh runs, or when a re-collection changes its hash. | Field | Number | Type | Description | | ---------------- | ------ | --------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------- | | `section` | 1 | `string` | Section name (INVENTORY\_SECTIONS): "system", "cpu", "memory", "disks", "nics", "gpus", "bmc", "os". It selects the message type of `content`. | | `sha256` | 2 | `bytes` | SHA-256 of `content`. The control plane verifies it and compares it with the hash stored with the last accepted section; it never re-encodes to compare. | | `content` | 3 | `bytes` | Binary encoding of the canonicalized section message from inventory.proto (e.g. GpusSection): arrays sorted by their stable key, no volatile values. | | `collected_at` | 4 | `google.protobuf.Timestamp` | Daemon clock, corrected by the Welcome offset; advisory. | | `schema_version` | 5 | `uint32` | NodeInventory schema version of `content` (1). | ### EventBatch [#eventbatch] Journaled events, sent in seq order. Results and transitions are never dropped; metric-like events are dropped oldest-first when the journal is full. | Field | Number | Type | Description | | ------------ | ------ | -------------------------- | --------------------------------------------------- | | `journal_id` | 1 | `string` | The journal the seqs belong to (Hello.journal\_id). | | `events` | 2 | repeated [`Event`](#event) | | ### Event [#event] | Field | Number | Type | Description | | ------------------- | ------ | ------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------- | | `seq` | 1 | `uint64` | Per journal\_id, strictly increasing from 1. Cumulatively acked (Ack.event\_seq); deduplicated on (node\_id, journal\_id, seq). | | `observed_at` | 2 | `google.protobuf.Timestamp` | Daemon clock, corrected by the Welcome offset; advisory. | | `attested` | 10 | [`AttestedReport`](#attestedreport) (oneof `body`) | | | `allocation_failed` | 11 | [`AllocationFailed`](#allocationfailed) (oneof `body`) | | ### AttestedReport [#attestedreport] A node-signed statement about allocations (§6 "Attested kinds"). | Field | Number | Type | Description | | ---------- | ------ | ------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `body` | 1 | `bytes` | Serialized AttestedReportBody. | | `node_sig` | 2 | [`Signature`](#signature) | Node key. Purpose "attested-report", parts \[body]. Optional in v0.1; required from v0.2, when workers verify it against node\_certs before any state transition. | ### AttestedReportBody [#attestedreportbody] | Field | Number | Type | Description | | --------------------- | ------ | ------------------------------------------------------------ | ------------------------------------------------------ | | `node_id` | 1 | `string` | The reporting node (must equal the session's node id). | | `boot_id` | 2 | `string` | | | `allocation_bound` | 10 | [`AllocationBound`](#allocationbound) (oneof `report`) | | | `allocation_released` | 11 | [`AllocationReleased`](#allocationreleased) (oneof `report`) | | ### AllocationBound [#allocationbound] The assignment is materialized: CDI spec, env file and device ownership are in place. | Field | Number | Type | Description | | --------------- | ------ | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `allocation_id` | 1 | `string` | | | `generation` | 2 | `uint64` | The NodeSpec generation that carried the assignment. | | `device_ids` | 3 | repeated `string` | | | `cdi_devices` | 4 | repeated `string` | Fully qualified CDI device names, e.g. "bcd.dev/gpu=GPU-…". | | `cdi_spec_path` | 5 | `string` | e.g. "/etc/cdi/bcd-\.json". | | `env_file` | 6 | `string` | | | `cgroup_parent` | 7 | `string` | (v1.1 §3.8) The cgroup-v2 slice a HOLD's workload must run in, e.g. "bcd-alloc-\.slice" (also in the CDI spec and env file); empty when the assignment carried no limits or its binding is KUBERNETES (the kubelet enforces those). | ### AllocationReleased [#allocationreleased] The assignment is gone: CDI spec and env file removed, ownership restored. | Field | Number | Type | Description | | --------------- | ------ | ----------------- | ----------- | | `allocation_id` | 1 | `string` | | | `generation` | 2 | `uint64` | | | `device_ids` | 3 | repeated `string` | | ### AllocationFailed [#allocationfailed] The daemon refused or failed to materialize an assignment (e.g. overlap with a live one). | Field | Number | Type | Description | | --------------- | ------ | -------- | ---------------------------------------------------------------------------- | | `allocation_id` | 1 | `string` | | | `generation` | 2 | `uint64` | | | `reason` | 3 | `string` | Stable lowercase code, e.g. "overlap", "device-missing", "cdi-write-failed". | | `message` | 4 | `string` | | ### Ack [#ack] Cumulative acknowledgement of durably stored events and results. | Field | Number | Type | Description | | ------------- | ------ | ----------------- | -------------------------------------------------------------------- | | `journal_id` | 1 | `string` | | | `event_seq` | 2 | `uint64` | Every event with seq \<= event\_seq in journal\_id is stored. | | `command_ids` | 3 | repeated `string` | Terminal CommandUpdates stored; the journal may stop replaying them. | ### SecretRequest [#secretrequest] Fetch one secret for current work. Forwarded to the api secret broker, which checks node\_sig and that the ref is bound to the agent's current step or allocation. | Field | Number | Type | Description | | --------------- | ------ | ------------------------- | ----------------------------------------------------------------------------------------------------------- | | `ref` | 1 | `string` | "secret://…", "vault://…", "env://…". | | `workflow_id` | 2 | `string` | | | `step_id` | 3 | `string` | | | `allocation_id` | 4 | `string` | | | `epk` | 5 | `bytes` | Fresh ephemeral P-256 public key, uncompressed SEC1 point (65 bytes). Used once. | | `node_sig` | 6 | [`Signature`](#signature) | Node key. Purpose "secret-request", parts \[request\_id, ref, workflow\_id, step\_id, allocation\_id, epk]. | | `request_id` | 7 | `string` | Correlates the SecretResponse. | ### SecretResponse [#secretresponse] | Field | Number | Type | Description | | ------------ | ------ | -------- | ------------------------------------------------------------------------------------------ | | `ref` | 1 | `string` | | | `jwe` | 2 | `string` | Compact JWE (ECDH-ES+A256KW / A256GCM) to SecretRequest.epk. Gateways only see ciphertext. | | `request_id` | 3 | `string` | | | `error` | 4 | `string` | Set instead of jwe on refusal: "not-found", "not-bound", "access-denied", "unavailable". | ### WorkloadTokenRequest [#workloadtokenrequest] (v0.5) Workload identity (§12.2, §13): a VPS host's IMDS, bound to one instance, asks for that instance's token. The control plane mints it only for a live instance placed on this node, in a pool with workload identity on, for a configured audience. | Field | Number | Type | Description | | ------------- | ------ | ------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `request_id` | 1 | `string` | Correlates the WorkloadTokenResponse. | | `instance_id` | 2 | `string` | | | `audience` | 3 | `string` | | | `epk` | 4 | `bytes` | Fresh ephemeral P-256 public key, uncompressed SEC1 point (65 bytes). Used once: the token is sealed to it, so a gateway relaying it never holds a usable token. | | `node_sig` | 5 | [`Signature`](#signature) | Node key. Purpose "workload-token", parts \[request\_id, instance\_id, audience, epk]. | ### WorkloadTokenResponse [#workloadtokenresponse] | Field | Number | Type | Description | | ------------ | ------ | --------------------------- | ------------------------------------------------------------------------------------------------------------------- | | `request_id` | 1 | `string` | | | `jwe` | 2 | `string` | Compact JWE (ECDH-ES+A256KW / A256GCM) of the workload JWT to WorkloadTokenRequest.epk. | | `expires_at` | 3 | `google.protobuf.Timestamp` | | | `error` | 4 | `string` | Set instead of jwe on refusal: "not-bound", "disabled", "audience", "rate-limited", "access-denied", "unavailable". | ### Drain [#drain] Reconnect elsewhere (rolling deploys, rebalancing). The daemon closes the session and redials after the delay, keeping its journal. | Field | Number | Type | Description | | -------------------- | ------ | -------- | ------------------------- | | `reconnect_after_ms` | 1 | `uint32` | rand(0, 60 s) by default. | | `reason` | 2 | `string` | | ### RevocationNotice [#revocationnotice] This agent's certificate was revoked; the gateway closes the session right after sending it. The daemon stops using the certificate and does not redial with it. | Field | Number | Type | Description | | --------- | ------ | ---------------- | ------------------------------------------- | | `reason` | 1 | `string` | | | `serials` | 2 | repeated `bytes` | Serial numbers of the revoked certificates. | ### OpenTunnel [#opentunnel] Ask the daemon to open a Tunnel stream. The daemon re-verifies the ticket against the AccessSection JWKS and its jti set before opening it. | Field | Number | Type | Description | | ----------- | ------ | -------- | ------------------------------------------------------------------------ | | `tunnel_id` | 1 | `string` | | | `ticket` | 2 | `string` | Exec ticket: ES256 JWT, 60 s, single-use jti, aud "node:\", sid, op. | ### TunnelFrame [#tunnelframe] | Field | Number | Type | Description | | ----------- | ------ | ---------------------------------------------- | ------------------------------------------------ | | `tunnel_id` | 1 | `string` | | | `channel` | 2 | `uint32` | 0 = stdin/stdout or the byte stream, 1 = stderr. | | `opened` | 10 | [`TunnelOpened`](#tunnelopened) (oneof `body`) | The daemon's first frame on the stream. | | `data` | 11 | `bytes` (oneof `body`) | | | `resize` | 12 | [`TerminalSize`](#terminalsize) (oneof `body`) | | | `closed` | 13 | [`TunnelClosed`](#tunnelclosed) (oneof `body`) | | ### TunnelOpened [#tunnelopened] No fields. ### TerminalSize [#terminalsize] | Field | Number | Type | Description | | ------ | ------ | -------- | ----------- | | `rows` | 1 | `uint32` | | | `cols` | 2 | `uint32` | | ### TunnelClosed [#tunnelclosed] | Field | Number | Type | Description | | ----------- | ------ | -------- | ----------- | | `exit_code` | 1 | `int32` | | | `reason` | 2 | `string` | | ### Binding [#binding] How an allocation is consumed (§10.5). | Value | Number | Description | | --------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `BINDING_UNSPECIFIED` | 0 | | | `BINDING_HOLD` | 1 | CDI hold, consumed by an operator-run workload manager on the node (v0.1). | | `BINDING_INSTANCE` | 2 | Passed to a VPS instance (v0.3). | | `BINDING_NODE` | 3 | The whole node, for a tenant deploy (v0.5). | | `BINDING_KUBERNETES` | 4 | (v1.1 §3.8) Given to the node's kubelet (the node joined a Kubernetes cluster through its pool): bcd's device plugin advertises exactly the devices bound this way, and withdraws them on release. | ### InstanceNetworkPolicy [#instancenetworkpolicy] | Value | Number | Description | | ------------------------------------- | ------ | ----------------------------------------------------------------------- | | `INSTANCE_NETWORK_POLICY_UNSPECIFIED` | 0 | EGRESS: existing specs do not change. | | `INSTANCE_NETWORK_POLICY_EGRESS` | 1 | Anything the pool's network allows (as before this field). | | `INSTANCE_NETWORK_POLICY_PRIVATE` | 2 | Only the pool's IPAM subnet and the host. | | `INSTANCE_NETWORK_POLICY_NONE` | 3 | Only the host: bcd's tunnel dial, DHCP and ARP; nothing else in or out. | ### InstanceDesiredState [#instancedesiredstate] | Value | Number | Description | | ------------------------------------ | ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `INSTANCE_DESIRED_STATE_UNSPECIFIED` | 0 | | | `INSTANCE_DESIRED_STATE_RUNNING` | 1 | | | `INSTANCE_DESIRED_STATE_STOPPED` | 2 | | | `INSTANCE_DESIRED_STATE_DELETED` | 3 | | | `INSTANCE_DESIRED_STATE_PAUSED` | 4 | (v0.5, sandboxes warm pools) Booted, then paused with its memory resident; resuming is desired RUNNING. A runtime that cannot pause keeps it STOPPED and reports STOPPED. | ### SpecRequestReason [#specrequestreason] | Value | Number | Description | | ----------------------------------------- | ------ | ----------------------------------------------------------------------------------- | | `SPEC_REQUEST_REASON_UNSPECIFIED` | 0 | | | `SPEC_REQUEST_REASON_PATCH_BASE_MISMATCH` | 1 | A patch's base\_generation is not the applied generation. | | `SPEC_REQUEST_REASON_MIN_GENERATION` | 2 | A command's min\_generation is above the applied generation. | | `SPEC_REQUEST_REASON_NO_SPEC` | 3 | The journal holds no usable spec (new journal, corruption, failed signature check). | ### InstanceObservedState [#instanceobservedstate] | Value | Number | Description | | ------------------------------------- | ------ | -------------------------------------------------------------- | | `INSTANCE_OBSERVED_STATE_UNSPECIFIED` | 0 | | | `INSTANCE_OBSERVED_STATE_RUNNING` | 1 | | | `INSTANCE_OBSERVED_STATE_STOPPED` | 2 | | | `INSTANCE_OBSERVED_STATE_ERROR` | 3 | | | `INSTANCE_OBSERVED_STATE_PAUSED` | 4 | (v0.5) Paused by the runtime (vCPUs stopped, memory resident). | ### SectionState [#sectionstate] | Value | Number | Description | | --------------------------- | ------ | ---------------------------------------------------------------------- | | `SECTION_STATE_UNSPECIFIED` | 0 | | | `SECTION_STATE_CONVERGED` | 1 | | | `SECTION_STATE_PROGRESSING` | 2 | | | `SECTION_STATE_FAILED` | 3 | Failed terminally at this generation (e.g. an overlapping assignment). | | `SECTION_STATE_UNSUPPORTED` | 4 | | ### ConditionStatus [#conditionstatus] | Value | Number | Description | | ------------------------------ | ------ | ----------- | | `CONDITION_STATUS_UNSPECIFIED` | 0 | | | `CONDITION_STATUS_TRUE` | 1 | | | `CONDITION_STATUS_FALSE` | 2 | | | `CONDITION_STATUS_UNKNOWN` | 3 | | ## common.proto [#commonproto] Package `bettercompute.common.v1`, `proto/bettercompute/common/v1/common.proto`. Shared types for every better-compute wire contract (PLAN.md §7). ### Signature [#signature] An ES256 signature (ECDSA P-256 with SHA-256). | Field | Number | Type | Description | | ------- | ------ | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `kid` | 1 | `string` | Identifies the signing key. For node keys: base64url (no padding) of SHA-256 over the key's SubjectPublicKeyInfo DER. For control-plane keys: the key's certificate subject key id as published in the trust bundle. | | `sig` | 2 | `bytes` | The raw 64-byte r ‖ s form (IEEE P1363 / JWS), not ASN.1 DER. | | `chain` | 3 | repeated `bytes` | Optional DER X.509 certificates, leaf first, chaining the key to a trust root the verifier already holds (worker signing keys chain to a TrustBundle root). Empty when the verifier knows the key by `kid` (node keys are looked up in node\_certs). | ### TrustBundle [#trustbundle] The command/spec signing trust the daemon accepts (PLAN.md §13). Delivered inside the signed JoinResponseBody, and refreshed through Welcome as a SignedTrustBundle. Rotation: new roots are added first; the old ones are dropped only after every daemon acknowledged the new version. | Field | Number | Type | Description | | -------------------------------- | ------ | -------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `version` | 1 | `uint64` | Monotonic. A daemon ignores a bundle whose version is not greater than the one it holds. | | `command_signing_roots` | 2 | repeated `bytes` | DER X.509 roots that certify the short-lived (24 h) worker keys signing Command and DesiredState bodies (v0.2). Empty in v0.1: signatures are optional there. | | `approver_keys` | 3 | repeated [`ApproverKey`](#approverkey) | (v1.0, §13 quorum) Operator approval keys (P-256), held off the control plane (laptops, HSMs). With any set, a destructive command carrying CommandBody.batch\_id runs only with a BatchApproval signed by `approver_threshold` distinct keys of this set. | | `approver_threshold` | 4 | `uint32` | | | `unbatched_destructive_per_hour` | 5 | `uint32` | With approver keys set: destructive commands WITHOUT a batch\_id this daemon runs per rolling hour (break-glass only; the default and recommended value is 0). A daemon cannot count fleet-wide, so any unbatched allowance lets a compromised worker touch every node once: fleet-wide limits come from the approvers (see quorum\_tiers), never from this cap. | | `quorum_tiers` | 6 | repeated [`QuorumTier`](#quorumtier) | Thresholds by batch size, ascending max\_scope\_size with non-decreasing thresholds: a batch with scope\_size ≤ max\_scope\_size needs `threshold` distinct approver signatures (the first tier that covers it); above every tier, approver\_threshold. Small batches can then be approved by a rate-limited auto-approver (one approver key held by a service outside api/worker, which is the fleet-wide counter), while large ones need k people. | | `approver_set_signatures` | 7 | repeated [`Signature`](#signature) | (v1.0) The quorum fields (3-6) are signed by the approvers themselves, not only by the join key the control plane holds: k-of-the-CURRENT-set signatures, purpose "approver-set", parts \[uint64\_be(approver\_set\_epoch), sha256(approver set input), trust\_domain] (see protocol's approverSetInput; the verifier's own trust domain). Published on every version while the set is current. One rule (review 10 M3): a daemon holding set H at epoch e adopts (set N, epoch e') only when e' > e AND k-of-H (H's threshold) signatures over (e', N, domain) verify; otherwise it keeps (H, e). That holds for N == H too: adopting a higher epoch for an unchanged set needs the set's own signatures. The list therefore carries both the previous set's signatures (which admit the change) and the new set's own signatures (which move daemons already holding it, or holding it again after a revert, to the new epoch, so a replay of an intermediate epoch fails). When the quorum part is refused, the rest of the bundle (roots, version) still applies. A daemon without keys takes the first set as-is unless pinned (approver\_set\_sha256 in its config). Residual risk: a daemon offline across two changes can be served an intermediate epoch by a compromised api; pin long-offline nodes again when they return. | | `approver_set_epoch` | 8 | `uint64` | (v1.0) The epoch of the quorum fields (3-6): monotonic, separate from `version` (roots rotate without touching it), raised with every change of the set. | ### QuorumTier [#quorumtier] | Field | Number | Type | Description | | ---------------- | ------ | -------- | ----------- | | `max_scope_size` | 1 | `uint32` | | | `threshold` | 2 | `uint32` | | ### ApproverKey [#approverkey] One operator approval key. | Field | Number | Type | Description | | ---------- | ------ | -------- | ----------------------------------------------------------------------- | | `kid` | 1 | `string` | base64url (no padding) SHA-256 of spki\_der, as Signature.kid names it. | | `spki_der` | 2 | `bytes` | SubjectPublicKeyInfo DER of the P-256 key. | ### SignedTrustBundle [#signedtrustbundle] A TrustBundle signed by the pinned JoinService signing key, so a gateway cannot inject roots. Signing purpose "trust-bundle", parts \[body]. | Field | Number | Type | Description | | ----------- | ------ | ------------------------- | ----------------------- | | `body` | 1 | `bytes` | Serialized TrustBundle. | | `signature` | 2 | [`Signature`](#signature) | | ### DriverManifest [#drivermanifest] What a daemon-side driver (built into bcd or served over bcx) offers (PLAN.md §5.4, §5.5). Advertised in Hello.drivers and returned by bcx DriverPlugin.Describe; the control plane merges it into its registry as a remote driver scoped to the advertising agents. | Field | Number | Type | Description | | ----------------------- | ------ | ------------------- | ------------------------------------------------------------------------------------ | | `id` | 1 | `string` | Driver id, e.g. "redfish", "ipmi", "libvirt-kvm", "nvidia". | | `version` | 2 | `string` | Semantic version of the driver implementation. | | `kinds` | 3 | repeated `string` | Driver kinds it implements, e.g. "power", "management", "runtime", "config". | | `runs_on` | 4 | [`RunsOn`](#runson) | | | `config_schema_json` | 5 | `string` | JSON Schema 2020-12 of the driver's config (manifest().jsonSchema() on the TS side). | | `capabilities` | 6 | repeated `string` | Free-form capability flags, e.g. "virtual-media", "uefi-http". | | `retry_safe_operations` | 7 | repeated `string` | Operations that are safe to repeat after an interruption, as "\.\". | ### RunsOn [#runson] Where a step or driver runs (PLAN.md §5.1 `RunsOn`). | Value | Number | Description | | ----------------------- | ------ | ----------- | | `RUNS_ON_UNSPECIFIED` | 0 | | | `RUNS_ON_CONTROL_PLANE` | 1 | | | `RUNS_ON_SITE` | 2 | | | `RUNS_ON_NODE` | 3 | | | `RUNS_ON_RAMDISK` | 4 | | ### AgentRole [#agentrole] The role carried by an agent's certificate (role OID) and declared in Hello (PLAN.md §6). Not the node's scheduling role (`nodes.role`: unassigned, host, dedicated, site). | Value | Number | Description | | ------------------------ | ------ | ------------------------------------------------------------------------------- | | `AGENT_ROLE_UNSPECIFIED` | 0 | | | `AGENT_ROLE_NODE` | 1 | `bcd node`: every server; 24 h certificate. | | `AGENT_ROLE_SITE` | 2 | `bcd site`: DHCP proxy, iPXE, image cache, BMC access (v0.2); 24 h certificate. | | `AGENT_ROLE_RAMDISK` | 3 | `bcd ramdisk`: PXE commissioning (v0.2); restricted certificate, at most 2 h. | ### Lane [#lane] Command lanes (PLAN.md §6, §7.5). Each lane has its own daemon worker pool and credit budget. | Value | Number | Description | | ------------------ | ------ | ------------------------------------------------------------------------------------------ | | `LANE_UNSPECIFIED` | 0 | | | `LANE_CONTROL` | 1 | Short, latency-sensitive work: diagnostics, inventory, reboots, resets. Default credits 8. | | `LANE_PROVISION` | 2 | Provisioning steps: wipe, deploy, kexec (v0.2). Default credits 4. | | `LANE_BULK` | 3 | Large transfers and slow jobs: log collection, image pre-warm. Default credits 2. | ### CommandState [#commandstate] The command lifecycle (PLAN.md §7.4). The control plane stores every state; a daemon reports only ACKED, RUNNING, SUCCEEDED, FAILED, REJECTED and CANCELLED in CommandUpdate. pending → dispatched(epoch) → acked → running → succeeded | failed | rejected pending → expired (CommandSweeper, past not\_after); any non-terminal → cancelled | Value | Number | Description | | --------------------------- | ------ | ------------------------------------------------------------------------------------------- | | `COMMAND_STATE_UNSPECIFIED` | 0 | | | `COMMAND_STATE_PENDING` | 1 | Stored, not yet sent. A BUSY rejection returns a command here with a backoff. | | `COMMAND_STATE_DISPATCHED` | 2 | Sent on the session with the given epoch. | | `COMMAND_STATE_ACKED` | 3 | The daemon journaled it as `accepted` (fsynced before any side effect). | | `COMMAND_STATE_RUNNING` | 4 | The side effect has started. | | `COMMAND_STATE_SUCCEEDED` | 5 | | | `COMMAND_STATE_FAILED` | 6 | Terminal failure; CommandUpdate.error says why (e.g. "interrupted"). | | `COMMAND_STATE_REJECTED` | 7 | Refused before any side effect; CommandUpdate.reject\_reason says why. | | `COMMAND_STATE_EXPIRED` | 8 | Never delivered before not\_after (control plane only). | | `COMMAND_STATE_CANCELLED` | 9 | Stopped by Cancel between actions. A completed side effect is reported as its real outcome. | ### CommandRejectReason [#commandrejectreason] Why a daemon refused a command before any side effect (PLAN.md §6 "Checks before accepting"). | Value | Number | Description | | ---------------------------------------- | ------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `COMMAND_REJECT_REASON_UNSPECIFIED` | 0 | | | `COMMAND_REJECT_REASON_UNSUPPORTED` | 1 | Unknown kind, or a kind this agent role may not run. | | `COMMAND_REJECT_REASON_BAD_SIGNATURE` | 2 | Missing, unknown-kid or invalid signature on a kind that requires one (destructive, v0.2). | | `COMMAND_REJECT_REASON_EXPIRED` | 3 | Past not\_after (server time, via the Welcome offset). | | `COMMAND_REJECT_REASON_STALE_GENERATION` | 4 | min\_generation still not applied at not\_after. | | `COMMAND_REJECT_REASON_BUSY` | 5 | No free credit on the command's lane. NOT terminal: the control plane puts the command back to pending with not\_before = compute\_now() + backoff (1 s → 30 s). | | `COMMAND_REJECT_REASON_BATCH_UNAPPROVED` | 6 | (v1.0) A destructive command with a batch\_id but no valid BatchApproval covering it (too few distinct approver signatures, expired, or an inclusion proof that does not reach scope\_root). | | `COMMAND_REJECT_REASON_UNBATCHED_LIMIT` | 7 | (v1.0) Over TrustBundle.unbatched\_destructive\_per\_hour. | ## driver.proto [#driverproto] Package `bettercompute.driver.v1`, `proto/bettercompute/driver/v1/driver.proto`. bcx: the out-of-process driver protocol (PLAN.md §5.5), served by plugins to both bcd and the control plane (externalDriver, remoteDriver). ### DriverPlugin [#driverplugin] | RPC | Request | Response | Description | | ---------- | ---------------------------------------- | ------------------------------------------ | ---------------------------------------------------------------------------- | | `Describe` | [`DescribeRequest`](#describerequest) | [`DescribeResponse`](#describeresponse) | The drivers this plugin serves. | | `Validate` | [`ValidateRequest`](#validaterequest) | [`ValidateResponse`](#validateresponse) | Validate a driver config against the driver's manifest and live checks. | | `Invoke` | [`InvokeRequest`](#invokerequest) | stream [`InvokeResponse`](#invokeresponse) | Run one operation; streams progress, then exactly one result or error. | | `Stream` | stream [`StreamRequest`](#streamrequest) | stream [`StreamResponse`](#streamresponse) | Duplex byte stream for consoles and similar (first request frame is `open`). | | `Health` | [`HealthRequest`](#healthrequest) | [`HealthResponse`](#healthresponse) | | ### DescribeRequest [#describerequest] No fields. ### DescribeResponse [#describeresponse] | Field | Number | Type | Description | | ------------------ | ------ | -------------------------------------------- | ------------------------- | | `drivers` | 1 | repeated [`DriverManifest`](#drivermanifest) | | | `plugin_version` | 2 | `string` | | | `protocol_version` | 3 | `uint32` | bcx protocol version (1). | ### ValidateRequest [#validaterequest] | Field | Number | Type | Description | | ------------- | ------ | -------------------------------------- | ------------------------------ | | `driver_id` | 1 | `string` | | | `kind` | 2 | `string` | | | `config_json` | 3 | `string` | Driver config, canonical JSON. | | `secrets` | 4 | repeated [`SecretValue`](#secretvalue) | Declared secret fields only. | ### ValidateResponse [#validateresponse] | Field | Number | Type | Description | | -------- | ------ | ---------------------------------------------- | ----------- | | `valid` | 1 | `bool` | | | `issues` | 2 | repeated [`ValidationIssue`](#validationissue) | | ### ValidationIssue [#validationissue] | Field | Number | Type | Description | | --------- | ------ | -------- | ----------------------------- | | `path` | 1 | `string` | JSON pointer into the config. | | `message` | 2 | `string` | | ### SecretValue [#secretvalue] One secret field value, delivered only to the plugin that declared it. | Field | Number | Type | Description | | ------- | ------ | -------- | --------------------------------------------------- | | `field` | 1 | `string` | The config field name declared with field.secret(). | | `value` | 2 | `bytes` | | ### InvokeRequest [#invokerequest] | Field | Number | Type | Description | | ----------------- | ------ | -------------------------------------- | ------------------------------------ | | `driver_id` | 1 | `string` | | | `kind` | 2 | `string` | | | `operation` | 3 | `string` | | | `config_json` | 4 | `string` | Driver config, canonical JSON. | | `args_json` | 5 | `string` | Operation arguments, canonical JSON. | | `secrets` | 6 | repeated [`SecretValue`](#secretvalue) | | | `idempotency_key` | 7 | `string` | | | `deadline` | 8 | `google.protobuf.Timestamp` | | | `traceparent` | 9 | `string` | W3C trace context. | ### InvokeResponse [#invokeresponse] | Field | Number | Type | Description | | ------------- | ------ | ------------------------------------------- | ------------------------------------------------- | | `progress` | 1 | [`Progress`](#progress) (oneof `msg`) | | | `result_json` | 2 | `string` (oneof `msg`) | The operation's result, canonical JSON. Terminal. | | `error` | 3 | [`DriverError`](#drivererror) (oneof `msg`) | | ### Progress [#progress] | Field | Number | Type | Description | | --------- | ------ | -------- | ------------------------- | | `percent` | 1 | `uint32` | 0-100, or 0 when unknown. | | `phase` | 2 | `string` | | | `message` | 3 | `string` | | ### DriverError [#drivererror] | Field | Number | Type | Description | | ----------- | ------ | -------- | -------------------------------------------------------------------------------------- | | `code` | 1 | `string` | Stable lowercase code, e.g. "unsupported", "invalid-config", "timeout", "unreachable". | | `message` | 2 | `string` | | | `retryable` | 3 | `bool` | The host may retry the operation (only for retry-safe operations). | ### StreamRequest [#streamrequest] | Field | Number | Type | Description | | ------- | ------ | ----------------------------------------------- | ------------------------------------------------------------- | | `open` | 1 | [`InvokeRequest`](#invokerequest) (oneof `msg`) | First frame: what to open, with the same fields as an Invoke. | | `data` | 2 | `bytes` (oneof `msg`) | | | `close` | 3 | [`StreamClose`](#streamclose) (oneof `msg`) | | ### StreamResponse [#streamresponse] | Field | Number | Type | Description | | ------- | ------ | ------------------------------------------- | ----------- | | `data` | 1 | `bytes` (oneof `msg`) | | | `error` | 2 | [`DriverError`](#drivererror) (oneof `msg`) | | | `close` | 3 | [`StreamClose`](#streamclose) (oneof `msg`) | | ### StreamClose [#streamclose] | Field | Number | Type | Description | | -------- | ------ | -------- | ----------- | | `reason` | 1 | `string` | | ### HealthRequest [#healthrequest] No fields. ### HealthResponse [#healthresponse] | Field | Number | Type | Description | | --------- | ------ | ------------------------------- | ----------- | | `status` | 1 | [`HealthStatus`](#healthstatus) | | | `message` | 2 | `string` | | ### HealthStatus [#healthstatus] | Value | Number | Description | | --------------------------- | ------ | ----------- | | `HEALTH_STATUS_UNSPECIFIED` | 0 | | | `HEALTH_STATUS_SERVING` | 1 | | | `HEALTH_STATUS_NOT_SERVING` | 2 | | ## site.proto [#siteproto] Package `bettercompute.site.v1`, `proto/bettercompute/site/v1/site.proto`. Site agent contracts (`bcd site`, PLAN.md §3.5, §8.5): boot lease state, proxy-DHCP entries, iPXE scripts, BMC operations and discovery assertions. ### SiteSpec [#sitespec] The `site` section of a site agent's NodeSpec. | Field | Number | Type | Description | | --------------- | ------ | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `boot` | 1 | [`BootLeaseState`](#bootleasestate) | | | `cache_images` | 2 | repeated [`CacheImage`](#cacheimage) | Images the site cache pre-warms (§11.5), from its pools' policy. The agent pulls each one and re-verifies its digest before serving it. | | `proxy_targets` | 3 | repeated [`ProxyTarget`](#proxytarget) | The machines site.httpProxy may reach through this agent, each with its addresses: the source of "an address of target\_node\_id known to the site" (HttpProxyRequest). The agent refuses a request whose URL host is not one of target\_node\_id's addresses (IP literals compared as IPs, names compared by name and then by every resolved IP), so a request "about" one node can never reach another machine or a BMC. Contributed for nodes whose profile is agent-less (e.g. Talos). | ### ProxyTarget [#proxytarget] | Field | Number | Type | Description | | ----------- | ------ | ----------------- | ---------------------------------------------------------------------------- | | `node_id` | 1 | `string` | | | `addresses` | 2 | repeated `string` | IP literals or DNS names of the node's API endpoint(s), never a BMC address. | ### CacheImage [#cacheimage] A verified catalog image the site cache holds. | Field | Number | Type | Description | | -------- | ------ | -------- | --------------------------- | | `url` | 1 | `string` | | | `sha256` | 2 | `bytes` | SHA-256 of the image bytes. | | `format` | 3 | `string` | "qcow2", "raw", "iso", … | ### BootLeaseState [#bootleasestate] The site:\:boot lease as published by the SiteBoot controller. Only the active holder answers proxyDHCP, serves iPXE scripts and vouches for discovery. An agent that loses its session self-fences after ttl\_ms: it stops all three. | Field | Number | Type | Description | | -------- | ------ | -------- | ---------------------------------------------------------------------------------------------------- | | `active` | 1 | `bool` | | | `epoch` | 2 | `uint64` | Lease epoch; every assertion the agent signs carries it, and the control plane rejects older epochs. | | `ttl_ms` | 3 | `uint32` | | ### ProxyDhcpEntry [#proxydhcpentry] One machine the active boot server answers for (boot.prepareRamdisk / boot.cleanUp). | Field | Number | Type | Description | | --------------------- | ------ | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `mac` | 1 | `string` | PXE MAC, lowercase colon form. | | `product_uuid` | 2 | `string` | | | `arch` | 3 | [`BootArch`](#bootarch) | | | `boot_file` | 4 | `string` | First-stage boot file served over TFTP or HTTP (e.g. an iPXE build). | | `ipxe_script_url` | 5 | `string` | HTTPS URL of this machine's iPXE script, served by the site agent. | | `workflow_id` | 6 | `string` | | | `not_after` | 7 | `google.protobuf.Timestamp` | Server time after which the entry is dropped. | | `discovery_token_ref` | 8 | `string` | Secret ref of this machine's MAC-bound discovery join token. The site agent fetches it with SecretRequest\{workflow\_id} and puts it only into the ramdisk's boot parameters it serves, so the token never travels in plaintext in commands.body. | | `sanboot_sha256` | 9 | `bytes` | (v1.1) Boot this ISO instead of the ramdisk, for BMC drivers without virtual media (ipmi): the sha256 of a SiteSpec.cache\_images entry with format "iso". The machine's iPXE script is then `sanboot --no-describe /cache/`, the vendor or OS ISO as is (no discovery token, no bc.\* parameters). A digest the cache does not hold verified is never answered (the site section fails) rather than falling back to the ramdisk. `boot_file` still chains iPXE; `ipxe_script_url` is ignored while this is set. | ### HttpProxyRequest [#httpproxyrequest] site.httpProxy: one HTTPS request made by the site agent on the control plane's behalf, for agent-less profiles (e.g. Talos, PLAN.md §5.6 walkthrough 3) whose API is reachable only inside the site. P5 holds: the control plane never dials into a site. The agent enforces, whatever the command says: the host resolves to an address of `target_node_id` known to the site (its inventory or BMC network), https only unless the address is on the provisioning VLAN, no redirects, a response cap (default 1 MiB) and the timeout. Anything else fails with "forbidden-target". | Field | Number | Type | Description | | ----------------- | ------ | ---------------------------------------------- | -------------------------------------------------------------------------------------------- | | `target_node_id` | 1 | `string` | The node the request is about; the target host must be one of its addresses. | | `url` | 2 | `string` | | | `method` | 3 | `string` | "GET", "POST", "PUT", "DELETE". | | `headers` | 4 | repeated [`HttpProxyHeader`](#httpproxyheader) | | | `body` | 5 | `bytes` | | | `tls_spki_sha256` | 6 | `bytes` | Pinned SubjectPublicKeyInfo SHA-256 of the target's TLS certificate (Talos uses its own CA). | | `timeout_ms` | 7 | `uint32` | | | `secret_headers` | 8 | repeated [`HttpProxyHeader`](#httpproxyheader) | Headers whose values are secret refs, fetched with SecretRequest and never logged. | ### HttpProxyHeader [#httpproxyheader] | Field | Number | Type | Description | | ------- | ------ | -------- | ----------- | | `name` | 1 | `string` | | | `value` | 2 | `string` | | ### HttpProxyResponse [#httpproxyresponse] | Field | Number | Type | Description | | ----------- | ------ | ---------------------------------------------- | -------------------------------------- | | `status` | 1 | `uint32` | | | `headers` | 2 | repeated [`HttpProxyHeader`](#httpproxyheader) | | | `body` | 3 | `bytes` | | | `truncated` | 4 | `bool` | The body exceeded the cap and was cut. | ### BootCleanUpRequest [#bootcleanuprequest] boot.cleanUp: drops the machine's proxy-DHCP entries and iPXE script (workflow cleanup). | Field | Number | Type | Description | | ------------- | ------ | ----------------- | -------------------------------------------------------------------- | | `node_id` | 1 | `string` | | | `macs` | 2 | repeated `string` | PXE MACs whose entries to drop; empty drops every entry of the node. | | `workflow_id` | 3 | `string` | | ### IpxeScriptRequest [#ipxescriptrequest] The parameters iPXE sends when chainloading ($\{mac}, $\{uuid}, $\{buildarch}, $\{platform}, …). The site agent matches them against its entries; an unknown machine becomes a discovery event. | Field | Number | Type | Description | | ----------- | ------ | -------- | ----------------------------------------------- | | `mac` | 1 | `string` | | | `uuid` | 2 | `string` | | | `ip` | 3 | `string` | | | `buildarch` | 4 | `string` | iPXE $\{buildarch}: "x86\_64", "arm64", "i386". | | `platform` | 5 | `string` | iPXE $\{platform}: "efi", "pcbios". | | `serial` | 6 | `string` | | | `asset` | 7 | `string` | | ### IpxeScript [#ipxescript] | Field | Number | Type | Description | | ----------- | ------ | --------------------------- | ----------------------------------------------------------------------------------------- | | `script` | 1 | `string` | The rendered script, starting with "#!ipxe". Carries the single-use discovery credential. | | `not_after` | 2 | `google.protobuf.Timestamp` | | ### BmcOperationRequest [#bmcoperationrequest] A BMC operation run by a site agent through bmclib (steps with runsOn: site). | Field | Number | Type | Description | | --------------------- | ------ | -------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `driver` | 1 | `string` | "redfish" or "ipmi" (or a vendor driver). | | `address` | 2 | `string` | Host or URL of the BMC. | | `credentials_ref` | 3 | `string` | Secret ref of the BMC credentials, fetched with SecretRequest; never a value. | | `operation` | 4 | `string` | "\.\", e.g. "power.setPower", "management.setBootDevice", "virtual-media.insert", "inspect.inspect". | | `args_json` | 5 | `string` | Operation arguments, canonical JSON. | | `tls_spki_sha256` | 6 | `bytes` | SHA-256 of the BMC certificate's SubjectPublicKeyInfo, when pinned. | | `driver_options_json` | 7 | `string` | Per-BMC driver options set at enrollment, canonical JSON (e.g. bcx-ipmi \{"cipherSuite": 3} for old BMCs). The site agent hands them to the driver as they are; the driver validates them and refuses unknown keys. Never credentials: those travel only as credentials\_ref. | ### BmcOperationResult [#bmcoperationresult] | Field | Number | Type | Description | | ------------- | ------ | -------- | -------------------------------------------------------- | | `ok` | 1 | `bool` | | | `result_json` | 2 | `string` | Operation result, canonical JSON (e.g. \{"power":"on"}). | | `error_code` | 3 | `string` | | | `message` | 4 | `string` | | ### SignedDiscoveryAssertion [#signeddiscoveryassertion] A site agent's statement that a PXE-booted machine asked for a boot from this MAC/IP. It vouches for the ramdisk's `discovery` join (v0.2). Signing purpose "discovery-assertion", parts \[body], by the site agent's node key. | Field | Number | Type | Description | | ------ | ------ | ------------------------- | ------------------------------ | | `body` | 1 | `bytes` | Serialized DiscoveryAssertion. | | `sig` | 2 | [`Signature`](#signature) | | ### DiscoveryAssertion [#discoveryassertion] | Field | Number | Type | Description | | -------------------- | ------ | --------------------------- | ----------------------------------------- | | `site_id` | 1 | `string` | | | `site_agent_node_id` | 2 | `string` | The vouching site agent. | | `boot_lease_epoch` | 3 | `uint64` | BootLeaseState.epoch held while vouching. | | `mac` | 4 | `string` | | | `ip` | 5 | `string` | | | `product_uuid` | 6 | `string` | | | `workflow_id` | 7 | `string` | | | `issued_at` | 8 | `google.protobuf.Timestamp` | | | `not_after` | 9 | `google.protobuf.Timestamp` | | ### BootArch [#bootarch] | Value | Number | Description | | --------------------------- | ------ | ----------- | | `BOOT_ARCH_UNSPECIFIED` | 0 | | | `BOOT_ARCH_BIOS` | 1 | | | `BOOT_ARCH_X86_64_EFI` | 2 | | | `BOOT_ARCH_ARM64_EFI` | 3 | | | `BOOT_ARCH_X86_64_EFI_HTTP` | 4 | |