# Sites and site agents (/docs/guides/bare-metal/sites)

> Sites are boot and failure domains. A bcd site agent in each one runs proxy DHCP, iPXE, the image cache and BMC access, with one active boot server per site.



A site is a boot and failure domain: a room, a cage or a lab whose machines
share a provisioning network and a BMC network. Everything the control plane does to a machine's
hardware (power, boot device, network boot, image streaming) goes through a
site agent on that site, because the control plane never dials into a
site. Pools, nodes and join tokens all belong to exactly one site.

## Sites [#sites]

A site has an owner, a kind and a region:

| Kind        | Owner                      | Created under          | Use it for                                   |
| ----------- | -------------------------- | ---------------------- | -------------------------------------------- |
| `shared`    | the root (platform) tenant | `compute-fleet/global` | platform hardware shared through offerings   |
| `dedicated` | the root tenant            | `compute-fleet/global` | platform hardware set aside for one customer |
| `custom`    | the tenant                 | `compute-tenant/{t}`   | a tenant's own hardware (bring your own)     |

Every `sites.*` mutation is decided as `compute:sites:manage`. Creating a `shared` or `dedicated`
site with a `tenantId` other than the root tenant is `INVALID_INPUT`.

```ts
const site = await compute.api.sites.create(credential, {
  tenantId: 'root',
  kind: 'shared',
  region: 'eu-central-1', // a region label
  iamRegion: 'eu-central', // optional: the IAM region name, when it differs
  idempotencyKey: 'site-fsn1',
});
// { id: 'site_…', ownerTenantId: 'root', kind: 'shared', region: 'eu-central-1', iamRegion: 'eu-central', version: 1 }
```

The id derives from the tenant, the caller and the idempotency key, so a retry returns the same site
and a different request under the same key is `CONFLICT`. The site's `region`, `iamRegion` and
`kind` are the attributes IAM conditions see on `compute-site`.

* [`sites.update`](/docs/reference/api/sites#update) (v0.2) changes `region` and `iamRegion`, or clears
  `iamRegion` with `clearIamRegion: true`. Pass the `version` you read to get `CONFLICT` instead of a
  lost update.
* [`sites.delete`](/docs/reference/api/sites#delete) (v0.2) refuses with `INVALID_STATE` while the site
  still has live pools or nodes. Otherwise it revokes every join token of the site (site tokens
  included), expires its open join requests, drops its boot lease, and removes the row, or tombstones
  it when anything ever referenced it. Deleting a tombstoned site again answers `deleted: false`.

## The site agent [#the-site-agent]

A site agent is `bcd` running in `site` mode on a machine next to the fleet, one or two per site. To
the control plane it is a node like any other: a `nodes` row with `role = 'site'`, no pool, a
`site` certificate and a session through the gateway. It runs the site's commands:

| Duty                                                                                    | When                                                      | In `bcd site` today                                                                                                                                                                                                              |
| --------------------------------------------------------------------------------------- | --------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Proxy DHCP, so booting machines find the iPXE binary without replacing your DHCP server | only while holding the boot lease                         | yes, with `[boot] proxy_dhcp`                                                                                                                                                                                                    |
| TFTP and iPXE scripts, one per machine being booted                                     | only while holding the boot lease                         | yes: read-only TFTP for the iPXE builds, scripts over plain HTTP                                                                                                                                                                 |
| Vouching for ramdisks that PXE-booted from it (signed discovery assertions)             | only while holding the boot lease                         | yes                                                                                                                                                                                                                              |
| An image cache that ramdisks stream from                                                | always (a boot's grant only while holding the boot lease) | yes, by digest under a grant (`GET /cache/t/<grant>/<sha256>`, b7ed8a2)                                                                                                                                                          |
| BMC access (Redfish, IPMI, vendor drivers) on the management network                    | always                                                    | Redfish built in; IPMI power, boot device and inspection through the hosted `bcx-ipmi` plugin (51806ab, 7818f19), which `daemon install-script --install-plugins bcx-ipmi` installs from the release (adf2ae9); no vendor driver |

The command kinds it accepts are role-restricted to `site`: `bmc.query`, `bmc.boot`, `bmc.power`,
`bmc.rotate`, `bmc.firmware`, `bmc.storage`, `boot.prepareRamdisk`, `boot.cleanUp` and `site.httpProxy`, plus
the shared `inventory.refresh`, `logs.collect` and `node.reboot`. Their payloads are messages of
`bettercompute.site.v1`, which stays a draft package until v0.2 closes. Every `bmc.*` kind but `bmc.query`, and
`site.httpProxy` since 0d42de8, is destructive, so under the v1.0 quorum it runs only in an approved batch;
`site.httpProxy` also never reaches a BMC (the agent refuses the BMC hosts it drives, any `/redfish` path and its
local `[site] bmc_networks`), and reaches only the addresses of the agentless machine it names (`SiteSpec.proxy_targets`,
which metal fills from inspection; a2a853d, 8391d74). See
[BMC drivers and credentials](/docs/guides/bare-metal/bmc) and the
[command kinds reference](/docs/reference/command-kinds).

The agent's desired state is the `site` section of its NodeSpec (`SiteSpec`): the boot lease state (below) and,
since v0.2, `cacheImages`, the verified catalog images its cache pre-warms (`url`, `sha256`, `format`). The
provision module fills that list: every live pool of the agent's site can name images in `policy.prewarmImages`
(`name@version` refs, resolved in the pool owner's catalog), and only images that still verify (signed, signer not
revoked, row untampered) are listed, deduplicated by URL. The agent pulls each one and re-verifies its digest before
serving it. `images.register` and `images.promote` recompute the specs of the site agents whose pools name the ref,
and so does `pools.update` when `prewarmImages` changes. Modules contribute `site` fields independently, because
the section merges field by field (see [NodeSpec sections](/docs/extending/modules#nodespec-sections)).

> **Status.** 
  `bcd site` exists, and its contracts are in `proto/bettercompute/site/v1`. It runs BMC operations over
  Redfish, or through a bcx plugin it hosts for any other driver (such as `bcx-ipmi` for `ipmi`; see
  [IPMI through bcx-ipmi](/docs/guides/bare-metal/bmc#ipmi-through-bcx-ipmi); a driver it does not host answers
  `unsupported-driver`), `site.httpProxy` and boot entries. While it holds the boot lease,
  its boot server answers prepared machines only: proxy DHCP (boot server and file, never an address), TFTP for the iPXE
  builds, iPXE scripts and discovery assertions over plain HTTP (see [the boot server](#the-boot-server-in-bcd-site)).
  Its cache pre-warms `cacheImages`, serves them to the site's ramdisks by digest and reports them in its `images`
  inventory section, which becomes the site's `image_locations`. bcd is not released, and only its Go unit tests run
  these parts (against a fake Redfish BMC, a scripted session and real sockets): no test runs `bcd site` against the
  control plane, and no real site has run it. The dev stack joins site agents as FakeDaemons with role `site`
  (`createDevStack().joinSite()`), and the metal test suite answers site commands with a simulated agent.

## Enroll site agents with site tokens [#enroll-site-agents-with-site-tokens]

A site agent joins with a site token: a join token of role `site`, bound to one site and no pool.
[`siteTokens.create`](/docs/reference/api/site-tokens#create) is decided as `compute:sites:manage` on
`compute-site/{siteId}`, and only the site's owner can mint one.

```ts
const minted = await compute.api.siteTokens.create(credential, {
  tenantId: 'root',
  siteId: site.id,
  ttlSeconds: 3600, // 60 to 86 400
  maxUses: 2, // how many agents may enroll with it: 1 (default) to 16
  idempotencyKey: 'fsn1-site-agents',
});
// { id: 'jt_…', siteId, role: 'site', usesLeft: 2, expiresAt, token: 'bcjt_<id>.<secret>' }
```

* The bearer (`bcjt_<id>.<secret>`) is in the answer that created the token and nowhere else: only
  its hash is stored, and a replay of the same request omits it. The same key for another site is
  `CONFLICT`.
* A deleted site is `INVALID_STATE`; a site the tenant does not own is `NOT_FOUND`.
* [`siteTokens.revoke`](/docs/reference/api/site-tokens#revoke) takes `tenantId`, `siteId` and
  `tokenId`. It revokes only site tokens: a node join token is `NOT_FOUND` there.
* Both publish security events: `security.site-token.created` and `security.site-token.revoked`.

From the command line (573b0b0), where `--out` writes the one-time token to a new 0600 file instead of printing it
and `revoke` refuses a pasted token (anything with a `.`), so the secret is never sent as an id:

```bash
better-compute sites create --tenant root --kind shared --region eu-central-1 --iam-region eu-central --key site-fsn1
better-compute site-tokens create --tenant root --site site_01k5w… --ttl 3600 --max-uses 2 --out ./site-token
better-compute site-tokens revoke jt_01k5w… --tenant root --site site_01k5w…
```

`site-tokens create --ttl` is in seconds (60 to 86,400).

The agent joins through the same Join RPC as any daemon (see [joining nodes](/docs/guides/nodes/joining)),
and JoinService issues it a 24-hour certificate with role `site`. With bcd that is
`BCD_JOIN_TOKEN=bcjt_... bcd join --role site`, then `bcd site`; see
[installing bcd](/docs/daemon/install#a-site-agent).

## One active boot server per site [#one-active-boot-server-per-site]

Two agents answering DHCP for the same machine would race, and a stale agent vouching for a ramdisk
would be a way in. So exactly one agent per site is the boot server: the holder of the lease
`site:<siteId>:boot`.

* The lease has a 15-second TTL and a fencing epoch that grows on every change of holder.
* The SiteBoot controller renews it while its holder is a connected site agent. The holder learns
  that it is active, and at which epoch, from the `site` section of its NodeSpec (`BootLeaseState`).
* A holder that loses its session keeps the lease until it expires and self-fences after the TTL: it
  stops answering proxy DHCP, iPXE and discovery. Only then is the first connected standby (by node
  id) promoted, under a new epoch. A live lease never changes hands, so two agents can never both
  believe they are active at the same epoch.
* When no agent is left, the lease is released and the epoch still moves on.
* Every discovery assertion carries the epoch the agent held, and the control plane refuses an epoch
  that is not the lease's current one.

The lease logic is the server's SiteBoot controller (`tickSiteBoot`). `betterCompute()` registers it as the built-in
`site-boot` module: a singleton reconciler whose `tick` key runs every 5 seconds in the `controllers` role.

A failover does not restart a machine that was already booting. BMC operations are reroutable between a site's
agents: a power cycle the dead agent already ran is reused, not sent again, and one it never received is moved to
the new agent. Boot entries stay per agent, so the new lease holder gets its own entry (and its own single-use
discovery token) and serves the waiting machine. `tests/metal-site-failover.test.ts` kills the active agent right
after it power-cycled a machine: the standby is promoted when the dead agent's lease runs out, never earlier, the
machine boots from it within 30 seconds, the deploy completes, and the boot step's BMC commands ran once. With both
agents down the boot waits, and the first agent back serves it.

### Where site steps go [#where-site-steps-go]

`siteAgentFor(nodeId)` picks the agent for a machine's site steps: a connected site agent of the
machine's own site, the boot-lease holder first, otherwise the first connected agent by id. A machine's
site steps never go to another site's agent. When no agent is connected, a relaying step waits and
retries until its timeout, and a step that runs on `site` directly fails with `NO_SITE_AGENT` at its
deadline. PLAN.md also routes BMC and image-cache work to any agent that advertises the driver; that
is not implemented.

## Booting a machine into the ramdisk [#booting-a-machine-into-the-ramdisk]

Metal never lets a machine boot something it did not prepare. Before each network boot, the
`metal.ensureRamdisk` step sets a one-time PXE boot through the BMC, sends the site agent one boot
entry per known PXE MAC, and power-cycles the machine:

<Mermaid
  chart="sequenceDiagram
  participant W as Worker
  participant S as Site agent
  participant M as Machine
  participant J as JoinService
  W->>S: bmc.boot, one-time PXE
  W->>S: boot.prepareRamdisk, one entry per MAC
  W->>S: bmc.power, cycle
  M->>S: DHCP and iPXE script request
  S-->>M: iPXE script with the discovery token
  M->>J: Join with the token and a signed assertion
  J-->>M: ramdisk certificate, at most 2 h
  W->>M: inventory, diagnostics, wipe, image"
/>

* Each entry (`ProxyDhcpEntry`) names the MAC, the SMBIOS product UUID when known, the boot
  architecture (`BIOS`, `X86_64_EFI` or `ARM64_EFI`, from the node's `arch` and `uefi`), the workflow
  and a `notAfter` two hours out.
* Each entry carries a fresh single-use `discovery` join token bound to that MAC (`expect.mac`), valid
  for 900 seconds. The token is stored in the secret store and the entry carries only its reference;
  the site agent fetches the bearer with a SecretRequest for that workflow step and renders it into
  that MAC's iPXE script. No command body ever holds the token.
* A token is minted only when its entry's command is first sent, not on every poll, and the cleanup
  step `boot.cleanUp` removes the entries and deletes the stored tokens.
* A machine with no known MAC fails with `INVALID_STATE`: inspect it out of band first. The one exception
  (3d26428) is a machine enrolled with a usable `productUuid` and no MACs: `metal.ensureRamdisk` PXE-boots it once
  with no entry so a site agent lists it, and goes on once the MAC [discovery](#discovering-new-machines) proposes
  is confirmed with `metal.confirmMac` (8de6e2f), or bound at once on a pool that trusts its provisioning network.

### The boot server in bcd site [#the-boot-server-in-bcd-site]

`bcd site` keeps the boot entries it receives in `data_dir/boot-entries.json`, keyed by MAC, so a restart keeps
answering; `boot.cleanUp` drops a workflow's entries (or the listed MACs), and lapsed entries are dropped on their
own. With a `[boot]` section in its config (see [installing bcd](/docs/daemon/install#a-site-agent)) it serves them
in three stages:

<Mermaid
  chart="sequenceDiagram
  participant M as Machine firmware
  participant D as Site DHCP server
  participant S as bcd site
  M->>D: DHCP DISCOVER (PXEClient)
  D-->>M: an address
  S-->>M: proxy DHCP: boot server and iPXE file, no address
  M->>S: TFTP undionly.kpxe or ipxe.efi
  M->>S: DHCP again, now as iPXE
  S-->>M: proxy DHCP: /boot/chain.ipxe
  M->>S: GET /boot/ipxe?mac=&uuid=&ip=
  S->>S: SecretRequest for the discovery token, sign the assertion
  S-->>M: iPXE script: kernel, initrd, bc.* parameters"
/>

1. **Proxy DHCP** (UDP 67 and 4011, `proxy_dhcp = true`) answers `PXEClient` and `HTTPClient` requests beside your
   DHCP server and never hands out an address. It reads the MAC, the client architecture (option 93) and whether the
   client is iPXE already. Firmware gets the iPXE build for its architecture from TFTP at `server_ip`:
   `undionly.kpxe` for BIOS, `ipxe.efi` for x86-64 UEFI, `ipxe-arm64.efi` for arm64 UEFI. UEFI HTTP boot clients get
   the same build as a URL, `<url>/boot/files/<file>`. iPXE itself gets `<url>/boot/chain.ipxe`, which chains to
   `/boot/ipxe` with its MAC, SMBIOS UUID and IP.
2. **TFTP** (`tftp`, `tftp_root`) is read-only, octet mode, with block size and transfer size negotiation, and serves
   nothing outside its root. You supply the iPXE builds. Since 4d8b493 it runs at most 64 transfers at once and 4
   per source address; a request beyond either bound is dropped unanswered (answering would take a socket too), and
   the firmware retries.
3. **`GET /boot/ipxe`** renders the machine's script: the site's ramdisk `kernel` and `initrd`, and on the kernel
   command line the `bc.*` boot parameters `bcd ramdisk` joins with (the gateway, pins and join key, the machine's
   discovery token and a signed assertion, and `bc.cache` when `url` is set), then `cmdline`.

Proxy DHCP answers, and `/boot/ipxe` renders a script, only when both hold:

* the MAC has a live boot entry; an unknown machine gets no proxy DHCP answer and a 404 script that exits, so it
  falls through to its next boot device, and is listed as discovered (see
  [discovering new machines](#discovering-new-machines));
* the agent is the active boot server: its `site` section says it holds the lease at a non-zero epoch, and it has
  heard from the gateway within the lease TTL (15 seconds when the section gives none). A partitioned agent fences
  itself this way; its `/boot/ipxe` answers 503 with a script that waits 10 seconds and reboots.

The discovery token is fetched with a `SecretRequest` bound to the entry's workflow the first time the machine asks
for its script and reused if it asks again; while it is not available the script waits and retries. Each script
carries a fresh `DiscoveryAssertion` (the site, this agent's node id, the lease epoch, the MAC, the IP from the query
or the connection, the product UUID from the query or the entry, the workflow, `issued_at` in server time, and a
`not_after` at most 15 minutes out and never past the entry's), signed by the agent's node key under purpose
`discovery-assertion`.

**Sanboot of a cached ISO** (0056e1e, e763001). For a BMC without virtual media, such as `ipmi`, a boot entry may
carry `ProxyDhcpEntry.sanboot_sha256` instead of a ramdisk boot: `/boot/ipxe` then answers
`sanboot --no-describe <url>/cache/t/<grant>/<sha256>`, and iPXE reads the ISO from the image cache with HTTP range
requests. The ISO boots as it is, with no discovery token and no `bc.*` parameters. The agent serves it only when
the site spec names that digest as an `iso` cache image and the verified copy is cached. Otherwise the machine waits
(a 503 script that sleeps 30 seconds and chains again, never the ramdisk), and the `site` section fails naming the
entry. Since 3d26428 metal sets `sanboot_sha256`: a `media.insert` step that names a catalog ISO (`image`) on a
driver without virtual media pins the ISO at the site and, once the site's cache holds it verified, sends a boot
entry for it (see [sanboot instead of virtual media](/docs/guides/bare-metal/bmc#sanboot-instead-of-virtual-media)).

The image cache is served by the same HTTP server, and since b7ed8a2 only under grants: every cache URL the agent
hands out has the form `GET /cache/t/<grant>/<sha256 hex>`, with an unguessable 128-bit grant, and any other path,
a plain `/cache/<sha256>` included, is 404. The cache holds tenants' deploy images, so nothing is served to whoever
asks:

| Grant                 | Handed out as                                                  | Serves                                                                                                   | Valid                                                                                                                               |
| --------------------- | -------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- |
| A boot                | The ramdisk's `bc.cache` in its iPXE script, and a sanboot URL | Any cached image (`bc.cache`; the ramdisk checks the digest its command names) or that one ISO (sanboot) | Until the boot entry's `not_after` (6 hours for an entry without one), and only while the agent holds the boot lease: otherwise 503 |
| A BMC firmware update | The URL the BMC fetches the firmware image from                | That one digest                                                                                          | 2 hours                                                                                                                             |

The agent pulls every `cacheImages` entry of its `site` section in the background into
`data_dir/cache/<sha256>`, and a copy becomes visible only after its bytes match the digest. The `site` section
stays `PROGRESSING` while images warm and fails for an image whose bytes do not match. With `[boot] cache_max_bytes`
set, each pass trims the cache to that budget, least recently used first, and never evicts an image the section
still names (the pools' pre-warm images plus every image a deploy at the site is streaming). A ramdisk booted with
`bc.cache` tries the cache first for `deploy.image` and falls back to the image's source URL; the digest is checked
on the stream either way, so a cache can never substitute an image. The boot server caps request headers at 8 KiB
(0442561), since any machine on the segment can reach it.

### The discovery attestor [#the-discovery-attestor]

The ramdisk joins with the discovery token plus the site agent's `SignedDiscoveryAssertion` that it
served this MAC. The `discovery` attestor admits it only when all of these hold:

* the token is a live, single-use `discovery` token for the same site, and its `expect.mac` matches;
* the vouching agent is a site agent of that site, holds the live boot lease, and the assertion's
  epoch is the lease's current epoch;
* the assertion is signed by that agent's key under a `site` certificate, at most 15 minutes old, and
  used for one join only;
* the ramdisk's claimed MACs include the asserted MAC, and its product UUID matches when asserted.

The result is a ramdisk certificate valid for at most two hours and
restricted to the commissioning command kinds. When the assertion names a running workflow, the
ramdisk joins as that workflow's node, so a known machine re-commissions as itself.

Hosts wire the attestors with provision's resolvers:

```ts title="compute.ts"
import { discoveryAttestor, provisionedAttestor } from 'better-compute/pki';
import { discoveryWorkflowNodeResolver, provisionedTargetResolver } from 'better-compute/provision';

export const compute = betterCompute({
  // …
  attestors: [
    discoveryAttestor({ resolveWorkflowNode: discoveryWorkflowNodeResolver() }),
    provisionedAttestor({ resolveTarget: provisionedTargetResolver() }),
  ],
});
```

Both resolvers answer only for workflows that are still running, so a token or assertion from a
finished or aborted workflow cannot re-key a node that has moved on. See
[attestation](/docs/guides/nodes/attestation) for the other attestors.

## Discovering new machines [#discovering-new-machines]

Today a machine enters metal by its BMC: `metal.enroll` with the BMC address, then the `inspect.oob`
step asks the BMC for the machine's PXE MACs, SMBIOS UUID and IP addresses and records them. That is
what makes the boot entries above possible.

PXE discovery closes the gap for a BMC that cannot name the machine's NICs, such as IPMI. It never enrolls anything:
it lets metal learn the PXE MAC of a machine that is already enrolled.

**The site agent's side** (0056e1e, e763001). A PXE or HTTP-boot DHCP request, or an iPXE script fetch, from a MAC
without a live boot entry, while the agent holds the boot lease, is never answered but listed in
`StatusReport.discovered`:

| Field                              | Value                                                           |
| ---------------------------------- | --------------------------------------------------------------- |
| `mac`                              | The machine's MAC                                               |
| `product_uuid`                     | The SMBIOS UUID from DHCP option 97, the one a BMC also reports |
| `arch`                             | The client architecture                                         |
| `first_seen`, `last_seen`, `count` | When it was first and last seen, and how often                  |
| `relay`                            | The DHCP relay (`giaddr`) it came through                       |

The list holds at most 256 machines (the oldest `last_seen` goes first), forgets each a day after it was last seen,
and drops a machine once it gets a boot entry. A new machine asks for a status report at most once every 30 seconds.
Since 0442561 the agent takes only well-formed fields from an unknown client, so none can grow the status report past
the frame limit: a MAC must parse, a UUID is kept only in canonical 36-character form, an arch only as
`[a-z0-9_-]{1,32}`, and a relay only as an IP address. While the agent runs proxy DHCP, an iPXE script fetch counts
only for a MAC that DHCP already showed, since anyone can fetch a URL.

**The control plane's side** (3d26428, 8de6e2f). Discovery data is unauthenticated DHCP: any host on the provisioning
network can claim any MAC and any UUID. So storing the list binds nothing, and a discovered MAC boots a machine only
after an operator confirms it.

* **Storing.** `metal()` declares `metalDiscoveryStatusHandler()` as one of its module status handlers
  (`ComputeModule.statusHandlers`, 8de6e2f), and `better-compute serve` and the dev stack run every registered
  module's handlers after the gateway's XID health handler and before the configuration's own `reports.status`. A host
  that builds its gateway with `createGateway` itself adds it to `reports.status`. It stores lists from live site
  agents only (other nodes' roles are cached per process for 10 minutes, so a host's reports cost no query), in
  `metal_discovered` (migration `metal/0007_discovered`), each agent's list replaced whole and written only when its
  digest changes (`metal_discovery_reports`).
* **Reading.** [`metal.discovered`](/docs/reference/api/metal#discovered) (`{ tenantId, siteId }`, decided as
  `compute:sites:read` on the site) lists the machines seen in the last 24 hours, merged over the site's agents,
  newest first, at most 256: `mac`, `productUuid`, `arch`, `firstSeen`, `lastSeen`, `count` (summed over agents),
  `relays`, `agentNodeIds`, `nodeId` when an enrolled machine has that MAC, and `candidateFor` when it is a machine's
  candidate awaiting confirmation. Only the tenant that owns the site sees it; any other tenant gets `NOT_FOUND`,
  since the list describes the site's network.
* **Matching, only right after metal's own power cycle.** Only `metal.ensureRamdisk` matches, for a machine enrolled
  with a usable product UUID and no MACs, right after it power-cycled the machine into PXE with its BMC. A MAC becomes
  the machine's candidate when all of these hold: it was first seen within 10 minutes after that power command was
  sent (with 30 seconds of clock slack before it); it is the only MAC the UUID was seen from in the last 24 hours, in
  any of three spellings (as given, with the first three fields byte-swapped as SMBIOS stores them, or fully reversed
  as IPMI's Get System GUID prints it); no other machine of the site shares the UUID; and no enrolled machine has the
  MAC. Firmware placeholder UUIDs (all zeros, all ones, `03000200-0400-0500-0006-000700080009` and
  `00020003-0004-0005-0006-000700080009`) never match. A new candidate publishes `node.pxe-mac.discovered` (`nodeId`,
  `mac`, `productUuid`, `siteId`, `agentNodeId`, `bound`), and the machine's metal view shows it as `discoveredMac`.
* **Confirming.** The candidate gets no boot entry and no discovery token until an operator binds it with
  [`metal.confirmMac`](/docs/reference/api/metal#confirmmac) (`{ tenantId, nodeId, mac }`, decided as
  `compute:nodes:provision` on the node), which takes the candidate or any MAC the operator knows to be the machine's.
  It publishes `node.pxe-mac.confirmed` (`nodeId`, `mac`, `discovered`, `requestId`) for the audit trail, answers
  `INVALID_STATE` for a node that already has MACs or is retired and `CONFLICT` for a MAC another machine of the site
  has, and wakes the reconciler, so the waiting `metal.ensureRamdisk` goes on from that MAC. While it waits, the step
  reports why (`waiting for metal.confirmMac`, or the reason there is no candidate: `not-seen`, `ambiguous`,
  `shared-uuid`, `mac-enrolled`).

> **policy.trustProvisioningNetwork.** 
  A pool whose policy sets `trustProvisioningNetwork: true` binds the candidate at once, without `metal.confirmMac`
  (`macSource: 'discovered'`). Use it only on a provisioning network no untrusted host can reach: any host on it can
  claim a machine's UUID from its own MAC and boot that machine's ramdisk with its discovery token. As a backstop,
  `inventory.refresh` then waits for the ramdisk's own `system` and `nics` sections, received after it connected, and
  fails the workflow with `DRIVER_ERROR` (reason `machine-identity-mismatch`) before anything is written to the machine
  unless the reported product UUID is a spelling of the enrolled one and the NICs include the MAC. For MACs from any
  other source, a UUID present on both sides is still compared.

The metal view records where a machine's MACs came from as `macSource` (`operator` for enroll and `metal.confirmMac`,
`inspection`, or `discovered`; migration `metal/0008_mac_source`). From the CLI (002dd95):

```bash
better-compute metal discovered --site site_01k5w… --format table
better-compute metal confirm-mac node_01k5w… --mac 3c:ec:ef:12:34:56
```

| Command                  | Flags                                                                       | Calls                                                                                                                                                                                       |
| ------------------------ | --------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `metal discovered`       | `--site ID` (required), `--tenant`                                          | `metal.discovered`: what the site's agents saw PXE or HTTP boot from MACs with no boot entry, with the machine each is a candidate for; nothing in it is trusted or bound                   |
| `metal confirm-mac NODE` | `--mac MAC` (required, `aa:bb:cc:dd:ee:ff`; `-` also separates), `--tenant` | `metal.confirmMac`: the operator's decision that `--mac` is the machine's PXE MAC, usually the candidate `metal discovered` shows; a malformed MAC is a usage error before anything is sent |

In the console, the **PXE discovery** page (`/discovery`, 15c11ee) shows the same list for a site you pick, with a
**Confirm** button on each candidate and a form to bind a MAC by hand; both ask in a browser confirmation before
calling `metal.confirmMac`, so nothing is bound without the operator's click. See
[the console](/docs/guides/clients/console#pxe-discovery). See [IPMI](/docs/guides/bare-metal/bmc#ipmi-through-bcx-ipmi)
for the enrollment that uses this.

The discovery attestor also supports operator approval:

* `approval: 'never'` (the default) admits as described above;
* `approval: 'unknown-machines'` defers a join that no running workflow names;
* `approval: 'always'` defers every discovery join.

A deferred join becomes a pending join request (24 hours by default) that an operator decides with
[`joinRequests.approve`](/docs/reference/api/join-requests#approve) or `joinRequests.reject`; the
ramdisk retries every 30 seconds with the same CSR and consumes the approval once.

## Next [#next]

* [Provisioning workflows](/docs/guides/bare-metal/workflows): what runs on the site agent and when.
* [BMC drivers and credentials](/docs/guides/bare-metal/bmc): the BMC side of the site agent.
* [PKI](/docs/operations/pki): the node CA and JoinService keys the agents join against.
