bcd helper

bcd helper is the root half of bcd's privilege separation (PLAN.md §6, v0.4; 28dc2ae). With [helper] socket and [helper] user set in config.toml, bcd node runs as an unprivileged account and sends the few things that need root to bcd helper, which runs as root beside it and serves typed operations over a local unix socket. The node parses everything that comes from the network: the session, commands, specs, tunnels. The helper parses only its own small requests, and trusts nothing in them.

This page is the reference for the helper itself: who it serves, the wire format, each operation family and the checks root makes before it acts. The daemon overview gives the summary, and running bcd node unprivileged covers the install script that sets it up.

Why a helper

Without [helper], bcd node runs as root under systemd hardening (see the systemd unit). Everything the agent parses is then parsed by root: the session protocol, desired state, signed commands, tunnel frames, plugin traffic. A parser bug there gives an attacker root on the machine.

With the helper, root's attack surface is one socket that takes one JSON request per connection. Each operation is small and typed: a file is named by a kind and an id rather than a path, a GPU by a UUID the host has, a config unit by its name and settings, a domain by an instance id. Root rebuilds each privileged action from those values and its own view of the host, so a compromised node can ask only for what the node could already legitimately ask for.

What the node keeps doing with its own rights: the journal, the identity and key record in data_dir, the status socket, reading /dev/kmsg for XIDs, libvirt's read-only socket (virsh -r for list, domstate, dumpxml and domstats), the cloud-hypervisor runtime's files (shared with each VMM's uid through ACL entries) and each VMM's API socket.

An unprivileged bcd node without [helper] on a real host cannot write /etc/cdi or create hold slices. Its Hello then claims no GPU binding capability, and a hold that arrives anyway fails at once with reason node-unprivileged instead of retrying every 30 seconds (da60bbd).

The socket and the peer check

For every connection, the helper reads the caller's pid, uid and gid from the kernel with SO_PEERCRED (peer_linux.go). The helper is Linux only for this reason. A caller is served only when both of these hold:

A refused caller gets only not allowed. The helper logs helper: connection refused with the pid, uid and reason (uid <n> gid <n> is not <user>, or the peer does not run an installed bcd binary). A copy of bcd run from elsewhere by the bcd user is refused, and so is any other account.

The socket is created so that root never changes a path the node could swap for a symlink (2d67b25):

1. The socket's directory is created 0755 if missing. As root, the helper refuses to start unless that directory is root's and closed to group and other writes (must be a directory root owns that no one else can write).
2. It binds the socket inside a private .bcd-helper-* directory made with mkdtemp, chmods it 0660, chowns it to root and the node account's primary group, then renames it into place. A stale socket is replaced atomically by that rename.

The install script uses /run/bcd-helper/helper.sock, in the helper unit's own RuntimeDirectory. It never uses /run/bcd, which the bcd user owns. The package default helper.DefaultSocket is /run/bcd/helper.sock, but the configuration must name the socket explicitly: without [helper] socket the helper exits 1 ([helper] socket and user are not configured).

Requests

One connection carries one request and one answer, as newline-delimited JSON:

{"v":1,"op":"host.unit","unit":"sysctl","settings":{"net.ipv4.tcp_keepalive_time":600},"intent":{"body":"...","signature":"..."}}

{"v":1,"ok":false,"error":"host.unit: the settings of sysctl are not the signed spec's"}

Every operation is logged as helper: operation with the peer's pid, op, the file kind and name, the gpu, the update version, the config unit, the instance (for ch.*) and the outcome. The one exception is a successful tpm.sign: every TLS handshake and node statement signs, so only its refusals are logged.

A self-update replaces only the node's binary. The helper keeps running the binary it started with until it restarts, which works while both speak protocol version 1. A rerun of the install script restarts both units.

Operations

The table lists every operation Server.do dispatches. Anything else is unknown operation "<op>". An operation whose backend this helper was not configured with answers ... not configured on this helper.

PLAN.md also names runtime.* and mig.apply operations. They do not exist: runtimes go through libvirt.* and ch.*, and a MIG layout is the mig-layout config unit, applied through host.unit.

Files

The node never names a path. helper.Client.WriteFile maps the absolute path the node would have written to a kind and an id (Classify) and refuses any path outside the table before it sends anything. The helper builds the path again from its own table:

What root checks before it writes:

• The kind and id. An unknown kind is "<kind>" is not a kind of file bcd owns. An id that does not match is refused, and a kind without ids refuses any name (<kind> takes no name). Data over 1 MiB is refused.
• The directory walk. The helper opens / as an os.Root and walks each component of the kind's directory with Lstat. A symlink or anything that is not a directory is refused (... is not a plain directory), and a missing one is created 0755. Running as root on the real filesystem, every directory on the way must also be root's and closed to group and other writes (02485b3), or the write fails with ... is not a directory only root can write.
• The write. It creates a temporary .<name>.bcd-helper exclusively (O_EXCL) in that directory, writes it, sets the mode and renames it over the target. A symlink planted at the target is replaced, never followed. file.remove removes the file and treats a missing one as done.

Firewall

firewall.apply loads the ruleset the instances converger renders (nft -f -). Root accepts only that ruleset's shape (CheckRuleset):

• at the top level, only table, delete table and flush table lines, and table … { blocks, of exactly three tables: bridge bcd_instances, inet bcd_taps (933c51b) and arp bcd_taps (b12c217). The same name in another family is refused;
• inside a block, no nested table;
• no line starting with include or define, and no line containing ruleset;
• balanced braces.

firewall.check takes a rule count from 0 to 1,000,000. Root lists the three tables with nft -j and compares; an absent inet or arp bcd_taps counts as no rules. What the rules do is on instances and bcd's nftables tables.

gpu.reset, reboot and sshd

The node still refuses a gpu.reset command for a GPU with a live hold before it asks (see gpu.reset and live holds). The helper's own check is only that the GPU is the host's.

host.unit

host.unit applies one NodeSpec.host config unit as root. The node plans every unit itself, unprivileged, and sends only the units that are not converged: the unit name, its settings (none when the unit left the spec and should be cleaned) and the signed spec that asks for it. The full rules are on host configuration under bcd helper. In short, root checks:

Root ignores the node's list of busy GPUs (a894cd2) and works it out itself (rootBusyGPUs): the GPUs with compute processes (nvidia-smi --query-compute-apps=gpu_uuid; a MIG- UUID marks every GPU busy) and the GPUs named by the live holds' CDI specs in /etc/cdi/bcd-*.json. A k8s-join under the helper also needs root's own [kubernetes] allowed_clusters (RequireAllowed: true).

Root keeps two records for this operation in /var/lib/bcd-hostconfig:

Slices

slice.apply and slice.remove create and remove a GPU hold's cgroup slice (260e33c). Root checks the slice name against ^bcd-alloc-[A-Za-z0-9_]{1,80}\.slice$ and every cpulist against the cpulist grammar (slice.Check), so only such names and lists reach systemctl and cgroupfs. slice.apply needs limits. slice.remove stops a slice only once its cgroup.events shows populated 0, and otherwise answers the slice still runs processes. Both are bounded to 30 seconds (SliceTimeout). See device assignments.

Self-update

The update.* operations are self-update's privileged half, selfupdate.Manager, with the slots and records in root's [update] slots_dir (default /usr/local/lib/bcd under the helper). The node only downloads into <data_dir>/staging and restarts. Root decides from its own configuration and files, never from the node's claims:

The helper arms its own watchdog after each swap, and again at start when an update is pending. An update still unconfirmed 30 seconds after its deadline is swapped back by the helper, which then runs systemctl restart bcd.service. Self-update has the whole flow.

libvirt

Under the helper, the bcd user is not in the root-equivalent libvirt group (ed3c7d2, d418820). The libvirt-kvm runtime sends every privileged virsh verb to the helper, which runs virsh -c qemu:///system as root:

Since 5bb9f22 the helper also turns NVIDIA persistence mode off for a domain's passthrough GPUs before libvirt.start and back on after libvirt.destroy, because nvidia-persistenced holds the GPUs open and libvirt's managed hostdevs cannot unbind them while it does. The validator's admitted elements are listed on libvirt through the helper.

cloud-hypervisor

Since 7fe74a1 the cloud-hypervisor runtime runs under the helper too. Each ch.* request names an instance id (the domain-name pattern above), and root reads everything else from its own checks:

Without a cloud-hypervisor binary on root's PATH, every ch.* is refused. The details are on cloud-hypervisor through the helper.

TPM

Since 8870b8d (security review 3) an unprivileged bcd node never opens /dev/tpmrm0. The kernel's resource manager filters no command, so a user in the tss group could change the hierarchies' authorization, clear the TPM or trip its lockout. The helper opens the TPM as root and serves five operations on node keys named by a 32-byte unique:

Root builds every TPM command, always with the template's empty password. The node chooses no template, handle, hierarchy or authorization. At most 8 node keys stay loaded; the least recently used is unloaded and derived again on its next use. A helper on a host without a TPM refuses all five (node keys in the TPM are not served by this helper (the host has no TPM)). The node checks every answer: the key's attributes must be tpm.NodeKeyAttributes, every signature must verify against the key's public half, and a certification must come from the AK the helper announced. See node identity.

Inventory reads

Two read-only operations give an unprivileged node the inventory details sysfs keeps root-only. Both take no arguments: a request with any field besides v and op is refused (dmi.read takes no arguments).

The node asks for dmi.read with a 10-second timeout and hw.read with 45 seconds. Either failing reads as "no values": the inventory then carries what the unprivileged read saw. The IPMI BMC read is inventory only, never an attestation input, and its command set is allowlisted in the device layer.

Starting the helper

bcd helper --config /etc/bcd/config.toml exits 1 unless all of these hold:

The configuration check (config.Validate) also refuses a relative socket path, a user of root, and, under the helper, an [update] slots_dir or [update] entrypoint inside data_dir: the node owns that directory and could plant the binary root trusts (9e71b1f).

That root-owned config.toml decides what root does for the node, so these keys are read from root's copy, never from the node:

bcd doctor, run as root on a helper node, checks the same conditions: the helper check fails for each directory on the way to the slots, socket or entrypoint that is not root's alone, says what host units bind to, and fails when no command-signing roots are pinned or recorded. The libvirt, cloud-hypervisor and tpm checks cover the group memberships (see bcd doctor).

Install layout

daemon install-script --unprivileged (169c459) sets up this layout; running bcd node unprivileged shows the units and the steps.

The bcd account is in neither the libvirt group (since d418820) nor the tss group (since 8870b8d). On a host with /etc/libvirt/qemu.conf the script sets dynamic_ownership = 0 and puts QEMU's user in the bcd group, so QEMU reaches the node's disk files through that group. --unprivileged with --plugins is a usage error, since bcx plugins need root to start. Only bcd node uses the helper: bcd site ignores [helper], and the ramdisk has no configuration file.

Testing the helper

bcd helper-call exists only in helpere2e builds, never in a release: it reads one request from standard input and sends it to [helper] socket as the installed bcd binary would.