# Fleet configuration (/docs/guides/fleet-config)

> Keep sites, provider accounts, pools, quotas, offers, flavors and sandbox templates in one reviewed FleetConfig document; plan, apply, catch drift.



The shape of a platform changes rarely and matters a lot: which sites exist, which pools they hold, how much of each
pool its owner keeps, which tenants each pool is offered to and with what quota, and which VM sizes are sold. Made
through one-off API calls, those changes have no review, staging and production slowly diverge, and nobody notices
when someone edits a pool by hand during an incident.

FleetConfig treats that shape as code, the way Better IAM's configuration as code
treats an access model. The fleet is one document keyed by name. You export it once, keep it in version control,
review changes as pull requests, plan them, and apply them in one transaction. A reconciler re-plans the applied
document every 15 minutes and raises an event when the fleet drifts from it.

Every fleet method decides `compute:fleet:admin` on `compute-fleet/global`, a scope that exists only in the root
tenant, so FleetConfig is a tool for platform operators. `fleet.apply` also decides every call it makes, as the
caller.

## Turn it on [#turn-it-on]

```ts title="compute.ts"
import { fleetModule } from 'better-compute/fleet';
import { betterCompute } from 'better-compute/server';
import { vps } from 'better-compute/vps';

export const compute = betterCompute({
  // database, secret, iam, baseURL ...
  modules: [vps(), fleetModule()],
});
```

`fleetModule({ driftEvery })` takes one option, how often the drift check runs (default `15m`). It adds the
migrations `fleet/0001_names` (table `fleet_names`) and `fleet/0002_applied` (table `fleet_applied`), and the
`fleet.drift` reconciler. Flavors, provider accounts and sandbox templates are served by
the `vps` module; without it (or, for templates, without its v0.5 migrations), `fleet.export` leaves them out and
reports them in `missing`, and a document's flavors and templates plan as blocked.

## The document [#the-document]

A document has `version: 1` and any of five lists. References between items are names from the same document or
objects fleet already manages. Secrets are references only (`secret://`, `vault://` or `env://`), never values, so
the document is safe to commit.

```json title="fleet.json"
{
  "version": 1,
  "sites": [{ "name": "fra", "kind": "shared", "region": "eu-central" }],
  "providerAccounts": [
    {
      "name": "cloud",
      "provider": "hetzner-cloud",
      "config": { "token": "secret://hetzner", "location": "fsn1" }
    }
  ],
  "pools": [
    {
      "name": "vps-fra",
      "site": "fra",
      "class": "vps-host",
      "gpuModel": "L40S",
      "policy": { "disruptionBudget": 0.1 },
      "ownerQuota": { "gpu": 8 },
      "offers": [{ "tenant": "acme", "quota": { "vcpu": 64, "gpu": 2 }, "priority": 5 }]
    },
    { "name": "burst", "site": "fra", "class": "provider", "providerAccount": "cloud" }
  ],
  "flavors": [{ "pool": "vps-fra", "name": "g1.small", "vcpu": 2, "memoryGiB": 4, "diskGiB": 40 }],
  "sandboxTemplates": [
    {
      "name": "py-small",
      "pool": "vps-fra",
      "flavor": "g1.small",
      "imageId": "ubuntu-24.04",
      "env": { "PIP_INDEX_URL": "secret://root/sec_01k…" },
      "network": "private",
      "warm": { "min": 2, "max": 4 }
    }
  ]
}
```

  **sites:**

    <TypeTable
      type="{
  name: { type: 'string', description: 'Lowercase letters, digits, &#x22;.&#x22;, &#x22;_&#x22; and &#x22;-&#x22;, up to 63 characters.', required: true },
  kind: { type: &#x22;'shared' | 'dedicated' | 'custom'&#x22;, description: 'The site kind.', required: true },
  region: { type: 'string', description: 'Lowercase letters, digits and &#x22;-&#x22;, up to 64 characters.', required: true },
  iamRegion: { type: 'string', description: 'The IAM region, same form as region.' },
}"
    />

    See [sites](/docs/guides/bare-metal/sites).
  
  **providerAccounts:**

    <TypeTable
      type="{
  name: { type: 'string', description: 'Same form as a site name.', required: true },
  provider: { type: 'string', description: 'The provider id, such as hetzner-cloud.', required: true },
  config: { type: 'object', description: 'Provider configuration; secret fields hold references.', default: '{}' },
}"
    />

    See [providers](/docs/guides/vps/providers).
  
  **pools:**

    <TypeTable
      type="{
  name: { type: 'string', description: 'Same form as a site name.', required: true },
  site: { type: 'string', description: 'A site name.', required: true },
  class: { type: 'string', description: 'metal-host, metal-dedicated, vps-host or provider.', required: true },
  gpuModel: { type: 'string', description: 'Letters, digits and &#x22;-&#x22;, up to 32 characters.' },
  sharing: { type: 'string', description: 'exclusive, mig, timeslice or mps.' },
  agent: { type: &#x22;'bcd' | 'none'&#x22;, description: 'Whether machines run the daemon.' },
  policy: { type: 'object', description: 'The pool policy, compared as a whole.', default: '{}' },
  ownerQuota: { type: 'Record<string, number>', description: &#x22;The owner's own maximum per resource, such as { gpu: 8 }.&#x22; },
  providerAccount: { type: 'string', description: 'A provider account name, for provider pools.' },
  offers: { type: 'FleetOffer[]', description: 'Who else may use the pool, one entry per tenant.' },
}"
    />

    See [pools](/docs/guides/capacity/pools).
  
  **offers:**

    <TypeTable
      type="{
  tenant: { type: 'string', description: 'The tenant the pool is offered to. Never the owner: use ownerQuota.', required: true },
  quota: { type: 'Record<string, number>', description: &#x22;That tenant's maximum per resource, such as { gpu: 16 }.&#x22;, default: '{}' },
  includeDescendants: { type: 'boolean', description: &#x22;Offer to the tenant's child tenants too.&#x22;, default: 'false' },
  priority: { type: 'number', description: 'An integer from -1000 to 1000.', default: '0' },
}"
    />
  
  **flavors:**

    <TypeTable
      type="{
  pool: { type: 'string', description: 'A pool name.', required: true },
  name: { type: 'string', description: 'Unique within the pool.', required: true },
  vcpu: { type: 'number', description: '1 to 1024.', required: true },
  memoryGiB: { type: 'number', description: '1 to 65536.', required: true },
  diskGiB: { type: 'number', description: '1 to 1048576.', required: true },
  gpus: { type: 'number', description: '0 to 64.', default: '0' },
  offerId: { type: 'string', description: 'Sell the flavor under one offer only.' },
  placement: { type: &#x22;{ numaAffine?: boolean; pinned?: boolean; hugepages?: '2Mi' | '1Gi' }&#x22;, description: 'Where its guests live on their host; sandbox templates inherit it (2e61612). Unknown keys are refused, false flags are dropped, and the plan compares it by value.' },
}"
    />

    See [flavors](/docs/guides/vps/flavors).
  
  **sandboxTemplates:**

    <TypeTable
      type="{
  name: { type: 'string', description: 'Same form as a site name; the key fleet matches on.', required: true },
  pool: { type: 'string', description: 'A pool name in this document, or the id (pool_…) of a pool offered to the tenant. Fixed once created.', required: true },
  flavor: { type: 'string', description: 'A flavor name of that pool.', required: true },
  imageId: { type: 'string', description: 'The image the template boots.', required: true },
  runtime: { type: 'string', description: &#x22;Must be the pool's runtime. Fixed once created.&#x22;, default: &#x22;the pool's&#x22; },
  userData: { type: 'string', description: 'Cloud-init of at most 32 KiB, or a secret://<tenant>/<id> reference to it.' },
  defaultTtlSeconds: { type: 'number', description: '60 to 2592000.' },
  maxTtlSeconds: { type: 'number', description: '60 to 2592000.' },
  warm: { type: '{ min: number; max?: number }', description: '0 to 64 each.' },
  env: { type: 'Record<string, string>', description: 'Variable names (A-Z, 0-9, _) to secret://<tenant>/<id> references only, written to /etc/sandbox/env in each sandbox.' },
  network: { type: &#x22;'egress' | 'private' | 'none'&#x22;, description: &#x22;What the sandbox's NICs may reach.&#x22;, default: &#x22;'egress'&#x22; },
  volumes: { type: '{ volume: string; mountPath: string; readOnly?: boolean }[]', description: '(v1.1, 94b8b70) Existing sandbox volumes every sandbox mounts, by volume name, in attach order. mountPath is an absolute path of letters, digits and . _ / -. At most 8, the API\'s limit per template (f181f43), so a longer list fails validation with INVALID_INPUT before anything is applied.' },
}"
    />

    Only the fields you list are compared, so TTL bounds and warm sizes the server filled in are not drift. `env` is
    compared and replaced as a whole map. `volumes` is compared by value and in order, since the attach order is the
    guest's disk order, and replaced as a whole list. Fleet never creates or deletes volumes, because they hold data:
    each name must already exist ([`sandboxVolumes.create`](/docs/reference/api/sandbox-volumes#create)), and an export
    names the template's volumes by name again. Each volume is decided for the caller when the apply checks the call,
    as in a direct `sandboxTemplates.create` or `update`, and the write runs on those decisions; before 7911c5f an apply
    that created or changed a template with volumes always failed `CONFLICT` (`volumes-changed`). See
    [sandboxes](/docs/guides/vps/sandboxes#templates) and [volumes](/docs/guides/vps/sandboxes#volumes).
  
Validation rejects unknown fields, a `version` other than 1, lists over 1000 items, repeated names (flavors repeat
by pool and name, offers by tenant) and malformed values, with `INVALID_INPUT` and the offending `details.path`.
Quota keys are resource names (a letter, then up to 63 letters, digits, `.`, `_` or `-`) with integer maxima of 0
or more.

### Names instead of ids [#names-instead-of-ids]

Sites and pools have no name of their own, so fleet keeps the names it gave them in `fleet_names`. Provider accounts
and sandbox templates use their `name`, flavors are named `<pool>/<name>`, offers `<pool> -> <tenant>`, and owner
quotas `<pool>:<resource>`. An exported template names a pool offered to the tenant by its id, since the pool is not
the tenant's.

* **Objects fleet did not create** appear in an export under their id (`pool_01...`). Keep those names: the plan
  matches by name, so a new name plans a new object.
* **Created objects get deterministic ids.** The idempotency key is `fleet:<tenant>:<kind>:<name>`, and the id
  derives from it and the caller, so an apply retried after a failure converges instead of creating duplicates.

## Plan [#plan]

`fleet.plan` (a read) diffs a document against the tenant's current objects and changes nothing. It returns:

* `changes`: one entry per item with `kind`, `name`, `action` (`create`, `update`, `delete` or `unchanged`), the
  differing `fields`, and `blocked` or `note` when they apply;
* `calls`: the exact API calls `fleet.apply` would make, in order;
* `summary`: counts per action, and `blocked` when any change cannot be applied;
* `planHash`: SHA-256 over the canonical call list.

What each kind can do:

| Kind             | Create                               | Update                                                                                                                                 | Delete, only with `prune`                                                                   | Cannot change                                          |
| ---------------- | ------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | ------------------------------------------------------ |
| Provider account | `providerAccounts.create`            | `providerAccounts.update` (the listed `config` fields)                                                                                 | `providerAccounts.delete`, once no remaining pool uses it                                   | `provider`                                             |
| Site             | `sites.create`                       | Never                                                                                                                                  | Never                                                                                       | `kind`, `region`, `iamRegion`                          |
| Pool             | `pools.create`, with its owner quota | `pools.update` (`policy`, `gpuModel`)                                                                                                  | `pools.archive`: running reservations keep their bookings                                   | `site`, `class`, `sharing`, `agent`, `providerAccount` |
| Owner quota      | `quotas.set`                         | `quotas.set`                                                                                                                           | No                                                                                          |                                                        |
| Offer            | `pools.offer`                        | `pools.offer` (`quota`, `includeDescendants`, `priority`)                                                                              | `pools.unoffer`                                                                             |                                                        |
| Flavor           | `flavors.create`                     | `flavors.update` (`vcpu`, `memoryGiB`, `diskGiB`, `gpus`, `offerId`, `placement`)                                                      | `flavors.delete`                                                                            |                                                        |
| Sandbox template | `sandboxTemplates.create`            | `sandboxTemplates.update` (`flavor`, `imageId`, `userData`, `defaultTtlSeconds`, `maxTtlSeconds`, `network`, `env`, `warm`, `volumes`) | `sandboxTemplates.delete`: its warm instances are deleted, running sandboxes keep their TTL | `pool`, `runtime`                                      |

A change is **blocked** when it touches a field that cannot change, names an unknown site, pool or provider account,
offers a pool to its own owner, or lists flavors or sandbox templates on a deployment without them (a template then
reads `this deployment does not run sandboxes (vps v0.5)`). Fix the document, or create a new
object instead of changing a fixed field. Without `prune`, items the document leaves out are left alone, so you can
adopt FleetConfig one list at a time.

Calls run in dependency order: provider account creates and updates, site creates, pool creates, pool updates and
quotas, offers, flavor creates and updates, sandbox template creates and updates, template deletes, flavor deletes,
unoffers, pool archives, and provider account deletes.

## Apply [#apply]

`fleet.apply` takes the same input plus an optional `planHash`, and runs the plan through the owning modules' own
methods. It is a composite method (`transaction: 'manual'`): it decides each call with `ctx.check` while no
transaction is open, then runs them with `ctx.invoke` in one transaction. A host without composite methods answers
`UNSUPPORTED`; the standard server has them (see [methods](/docs/extending/methods)).

      ### Plan and compare [#plan-and-compare]

    Fleet plans the document. A blocked plan fails with `INVALID_STATE` and the blocked changes in `details.blocked`.
    If you passed a `planHash` and the plan no longer has it, apply fails with `CONFIG_DRIFT` (409). A plan with no
    calls only records the document and returns `applied: 0`.
  
      ### Decide every call [#decide-every-call]

    Each call is decided for the caller, one IAM decision per call, with no transaction open. A single denial fails the
    apply with `ACCESS_DENIED` before anything is written.
  
      ### Run everything in one transaction [#run-everything-in-one-transaction]

    Inside one transaction fleet plans again. If the call list changed while IAM was deciding, nothing is applied and
    the apply fails with `CONFIG_DRIFT`. Otherwise every call runs, created names go into `fleet_names`, and the
    document is recorded as the tenant's applied document.
  
The caller therefore needs `compute:fleet:admin` and the permission of every call the plan makes:

| Calls                                                                           | Action                     |
| ------------------------------------------------------------------------------- | -------------------------- |
| `sites.create`                                                                  | `compute:sites:manage`     |
| `pools.create`                                                                  | `compute:pools:create`     |
| `pools.update`, `pools.archive`                                                 | `compute:pools:manage`     |
| `pools.offer`, `pools.unoffer`                                                  | `compute:pools:share`      |
| `quotas.set`                                                                    | `compute:quotas:manage`    |
| `flavors.create`, `flavors.update`, `flavors.delete`                            | `compute:flavors:manage`   |
| `providerAccounts.create`, `providerAccounts.update`, `providerAccounts.delete` | `compute:providers:manage` |
| `sandboxTemplates.create`, `sandboxTemplates.update`, `sandboxTemplates.delete` | `compute:sandboxes:manage` |

Every call writes its own audit row, and the apply writes one more after it succeeds. The result is
`{ plan, applied }`, the number of calls that ran.

```ts title="Plan, review, apply exactly that plan"
const { config } = await compute.api.fleet.export(credential, { tenantId: 'root' });

const plan = await compute.api.fleet.plan(credential, { tenantId: 'root', config, prune: true });
console.log(plan.summary, plan.changes.filter((change) => change.action !== 'unchanged'));

await compute.api.fleet.apply(credential, {
  tenantId: 'root',
  config,
  prune: true,
  planHash: plan.planHash, // refuse with CONFIG_DRIFT if the fleet moved since the review
});
```

## Drift [#drift]

Every apply, including one with nothing to change, records the document it applied. The `fleet.drift` reconciler
runs its key `all` every `driftEvery`, which queues one `tenant:<id>` key per tenant with an applied document; each
tenant is then re-planned in its own transaction, so one tenant's failure never blocks another.

A non-empty call list, or a blocked plan, means someone changed the fleet outside FleetConfig:

* `fleet.drifted` is published once per distinct drift, with the applied `planHash`, the summary, `blocked`, and up to
  50 changes;
* `fleet.converged` is published when the drift clears on its own;
* `fleet.status` shows the drift as of the last check.

Both events have the subject `fleet/<tenant>` and reach [webhooks](/docs/guides/events/webhooks) like any other.
Applying the document again resets the drift.

```json title="fleet.status result"
{
  "tenantId": "root",
  "applied": {
    "planHash": "3f1c...",
    "appliedBy": "usr_ops",
    "appliedAt": 1790251200000,
    "prune": true,
    "checkedAt": 1790252100000
  },
  "drift": {
    "changes": [{ "kind": "pool", "name": "vps-fra", "action": "update", "fields": ["policy"] }],
    "summary": { "create": 0, "update": 1, "delete": 0, "unchanged": 5 },
    "blocked": false,
    "planHash": "9ab0..."
  }
}
```

A tenant that never applied a document gets only `{ tenantId }`. The console's **Fleet drift** page (platform
administrators) shows the same: who applied last, when it was checked, and the drifted changes by kind.

## From the CLI [#from-the-cli]

The four `fleet` commands act as `BETTER_COMPUTE_TOKEN`, through the configuration or against `--url`. They read
and write JSON; the server validates the document.

```bash
better-compute fleet export --tenant root --out fleet.json   # refuses to overwrite a file
better-compute fleet plan fleet.json --tenant root            # --prune, --calls
better-compute fleet apply fleet.json --tenant root --yes
better-compute fleet status --tenant root
```

`fleet plan` prints one line per change that does something, then the counts:

```text
~ pool vps-fra (policy)
~ ownerQuota vps-fra:gpu (max)
+ offer vps-fra -> globex
+ flavor vps-fra/g1.large
2 to create, 2 to update, 0 to delete, 5 unchanged.
```

* **`fleet apply`** plans and prints the changes. Without `--yes` it stops there and prints the `--plan-hash` to
  apply that exact plan later. With `--yes` it applies what it just planned; with `--plan-hash` it applies a plan
  reviewed earlier, and fails with `CONFIG_DRIFT` if the fleet moved.
* **A blocked plan** exits `PLAN_BLOCKED` from `plan` and `apply`.
* **`--fail-on-drift`** makes `fleet plan` exit `FLEET_DRIFTED` when any call would run, and `fleet status` exit
  `FLEET_DRIFTED` while the recorded drift is set.

These codes exit with status 1 (usage errors exit 2). When nothing would change, `fleet apply` without
`--plan-hash` does not call `fleet.apply`, so it does not re-record the document; pass `--plan-hash` to make an
unchanged document the new drift baseline. See [the CLI reference](/docs/reference/cli#fleet-plan).

### In CI [#in-ci]

Plan on every pull request and fail when production no longer matches the reviewed file; apply on merge:

```yaml title=".github/workflows/fleet.yml (excerpt)"
- name: Production matches the reviewed document
  run: better-compute fleet plan fleet.json --tenant root --fail-on-drift
  env:
    BETTER_COMPUTE_URL: https://compute.example.com
    BETTER_COMPUTE_TOKEN: ${{ secrets.BETTER_COMPUTE_TOKEN }}

- name: Apply the reviewed document
  if: github.ref == 'refs/heads/main'
  run: better-compute fleet apply fleet.json --tenant root --yes
  env:
    BETTER_COMPUTE_URL: https://compute.example.com
    BETTER_COMPUTE_TOKEN: ${{ secrets.BETTER_COMPUTE_TOKEN }}
```

Give the pipeline an API key of a service account in the root tenant that holds `compute:fleet:admin` and the call
permissions above, and nothing else. Between runs, schedule `fleet status --fail-on-drift` or subscribe to
`fleet.drifted` to hear about hand edits the same day.

  - [fleet API reference](/docs/reference/api/fleet): export, plan, apply and status.

  - [CLI: fleet plan](/docs/reference/cli#fleet-plan): Flags of export, plan, status and apply.

  - [Rollouts](/docs/guides/rollouts): The other root-tenant tool: bcd versions in waves.