Fractional GPUs

A whole H100 for a notebook or a small inference server wastes most of it. A pool can instead split its GPUs, and each piece becomes an atom of its own: booked, placed, metered and kept exclusive exactly like a whole GPU. The control-plane side of this has landed ahead of its roadmap phase, v0.5; the node side has not (the last section of this page lists what runs today).

Sharing modes and atoms

A pool's sharing mode, fixed when the pool is created, decides how its GPUs are split:

Inventory derives the atoms from what the node reports (see inventory and health):

• A GPU in MIG mode becomes an unbookable gpu parent plus one gpu.mig atom per MIG instance. This happens whatever the pool's mode, but only mig pools accept gpu.mig requests, so a MIG-mode GPU in another pool cannot be reserved.
• In a timeslice or mps pool, a GPU not in MIG mode becomes an unbookable parent plus replicas slot atoms (1 to 48, the most clients MPS serves per GPU; a708aa6). In mps pools each slot also carries mpsUnits, its share of the GPU's 100 MPS units (floor(100 / replicas)).
• A GPU whose MIG mode is changing (the pending mode differs from the current one) has every atom unbookable and marked partitionPending, so nothing new lands on it until the change completes.

Atoms carry the attributes selectors match on. MIG atoms have gpuProduct, gpuModel, migProfile (such as 3g.40gb), giId, ciId, migUuid and numaNode. Slot atoms have gpuProduct, gpuModel, sharing, slot, slots, vramGiB, numaNode, and mpsUnits in MPS pools.

Reserve a fraction

Ask for the atom class and narrow it with a selector:

await compute.api.reservations.create(credential, {
  tenantId: 'acme',
  poolId: migPool,
  devices: [
    {
      class: 'gpu.mig',
      count: 1,
      selector: { matchExpressions: [{ key: 'migProfile', op: 'In', values: ['3g.40gb'] }] },
    },
  ],
  duration: '4h',
  idempotencyKey: 'notebook-ana-0924',
});

Admission checks every device request against the pool's mode, with INVALID_INPUT:

• gpu.mig only in mig pools: devices[0]: gpu.mig atoms exist only in mig pools;
• gpu.slot only in timeslice or mps pools: devices[0]: gpu.slot atoms exist only in timeslice or mps pools;
• a request's own sharing, when given, must be exclusive or the pool's mode: devices[0]: the pool shares GPUs as timeslice, not mps.

Whole gpu requests stay valid in every pool and take GPUs that are not split. Atoms of one GPU count as NV-connected to each other for the topology scorer. The better-compute reserve command asks for whole GPUs only; reserve fractions through the API, the typed client or better-compute api reservations.create.

Quota, policy and billing

• Quota is kept per atom class: a MIG instance charges gpu.mig, a slot charges gpu.slot. A pool owner's default quota covers only gpu, so set gpu.mig or gpu.slot rows with ownerQuota, pools.offer or quotas.set, or every fractional request is QUOTA_EXCEEDED.
• IAM policies see fractions as gpuSlices on the reservation, apart from whole GPUs in gpuCount, so a limit on gpuCount does not count slices.
• Billing uses the pool's GPU meter at each atom's share of a GPU hour: a timeslice slot is 1 / slots, an MPS slot mpsUnits / 100, and a MIG instance its compute slices over 7 (3g.40gb is 3/7). Two of four time-slice slots and one 3g.40gb instance for an hour bill 0.5 + 3/7, about 0.928571 gpu.h100.hours.
• Rollups keep gpu.mig and gpu.slot rows next to gpu. capacity.summary's per-model totals and capacity.timeline count whole GPUs only; read the class rows with capacity.byPool.

Single-tenant sharing

Time slicing and MPS do not isolate one workload from another the way MIG does. timeslice and mps pools are therefore single-tenant: the offering validator refuses to offer them to anyone but their owner, with INVALID_STATE. mig pools can be offered like exclusive ones. See pools.

Changing a layout: the atom rule

A flat EXCLUDE constraint cannot see a MIG instance and its parent overlapping, so repartitioning a GPU under a live booking would slip past it. The atom rule closes that gap: a mode or layout change books every atom of the GPU, parent included, for the change window. It commits only when nothing of the GPU is booked in that window, and keeps anything new off the GPU until the window is released.

better-compute/capacity exports the primitive:

import { bookPartitionChange, releasePartitionChange } from 'better-compute/capacity';

const change = await bookPartitionChange(tx, { gpuId: 'GPU-…', during: { start, end } });
if (change.status === 'conflict') {
  // something of the GPU is booked; the whole family is free from change.freeFrom
} else {
  // apply the layout, then:
  await releasePartitionChange(tx, change.bookingId);
}

The window's bookings use the allocation id partition:<gpuId>:<start> and have no allocation row. An unknown GPU is NOT_FOUND. For mig pools, policy.migLayout names the layout the mig-layout config unit applies, such as all-1g.10gb or mixed:3g.40gb+2x2g.20gb; a profile's host config can take it with from: 'pool' (see profiles and images).

The MIG layout gate

Changing policy.migLayout on a mig pool must not repartition MIG instances under their holders, so the capacity module gates the change per node, with the atom rule:

1. pools.update with a new migLayout freezes every node of the pool at its current layout, with the new one pending, in capacity_mig_layouts (migration capacity/0004_mig_layouts); a newer change replaces a pending one. Provision's mig-layout unit reads the node's frozen layout (provision_gated_mig_layout), falling back to the pool's for a node never gated.
2. The capacity.mig reconciler (sharded by node) books every GPU family of the node, parent and atoms, from the end of its last live booking onward, so idle time before that still books but nothing lands past it (node.mig.scheduled, with the time).
3. At that time it grants the new layout and invalidates the node's spec (node.mig.granted); the daemon repartitions the idle GPUs.
4. Once the node has converged a newer spec (a higher generation, observed), it releases the partition bookings (node.mig.applied) and the new atoms take work.

At the grant, the gate asks whether the node's host config takes its layout from the pool (carriesPoolLayout): provision's last succeeded workflow with a host config has mig-layout { from: 'pool' }, which is what hostSpecFor resolves. If not (or without provision), nothing will repartition the node for this change, and the holds are released at the grant. If so, they are held until a newer generation is observed, however long that takes. No timer releases a hold: with a NodeSpec backlog, work could otherwise land on the old atoms and a late repartition would destroy them. A node that never converges keeps its GPUs held, and better-compute doctor reports it as capacity.mig.stuck after 24 hours (see observability).

A node that joins later reads the pool's layout directly: it is not schedulable before its first converged spec.

What runs today

Shipped in the control plane:

• atoms derived from inventory for all three modes, with partitionPending during a MIG mode change;
• the sharing checks at admission, and placement, booking and exclusivity of gpu.mig and gpu.slot atoms (covered by tests/capacity-fractional.test.ts);
• the atom-rule primitive, fractional metering and the single-tenant offering rule;
• the MIG layout gate, which wraps a pool's layout change in bookPartitionChange (covered by tests/capacity-mig-gate.test.ts);
• the MIG torture test in tests/capacity-torture.test.ts: placers race on one MIG pool while simulated repartitions go through the gate, and no parent and child, or atoms of different layouts, are ever live at once.

On the node, bcd (built and tested end to end, not released) holds device assignments through its devices converger: CDI specs, env files and node-signed AllocationBound and AllocationReleased reports. Since 46a61db it holds gpu.mig atoms (resolved to the MIG instance's MIG-… UUID from the inventory; device-missing until the instance exists) and gpu.slot atoms (their GPU), refuses a whole GPU while part of it is held and a part while the whole is held, and gives a slot hold CUDA_MPS_PIPE_DIRECTORY when MPS runs (see device assignments).

bcd's host converger (6c2343f) applies the layout: the mig-layout unit partitions every idle MIG-capable GPU (all-disabled, all-<profile>, mixed:<n>x<profile>+…), leaves a GPU that a live hold uses alone, and asks for a reboot when a mode change needs a GPU reset that fails. For mps and timeslice pools, provision renders the gpu-sharing unit (1f0f683), with which bcd runs the MPS control daemon as bcd-mps.service. All of it is tested against fakes; no GPU has been repartitioned on a real node.

Still to come in v0.5:

• a MIG repartition and an MPS pool on real hardware;
• the remaining v0.5 exit tests and the real-hardware nightly.

PLAN.md plans to reset GPUs on release and leaves one question open: clearing memory between tenants relies on the driver's guarantees, and a zeroing pass may be added. See the roadmap.