Volumes and addresses

A tenant on a provider pool can keep data and an address that outlive any one instance: a provider volume is a block volume on the pool's provider account (or a shared NFS volume many instances mount), a provider snapshot is a point-in-time copy of one, and a provider address is a reserved public IP. All three are typed resources of the vps module, listed and authorized like instances, and converged on the provider by one reconciler, vps.provider-resources.

They are different from the sandbox and host volumes that bcd keeps on a hosted machine's own disks: those live on a node, these live in the cloud account.

A pool whose provider implements neither answers UNSUPPORTED to every create.

How a request converges

Every mutation records what the tenant wants (desiredState, the wanted attachment or assignment) in its own transaction and returns at once, usually with state: 'pending'. The reconciler then calls the provider outside any transaction and writes back what it saw:

• States are pending, ready, updating (a resize running), deleting, deleted and error. An error carries the provider's message and is retried by the next change or the periodic pass (every 5 minutes). A settled resource is read once an hour for drift, those reads spread across the hour so they leave the token's request budget to instance polling.
• Provider actions (attach, detach, resize, snapshot) never block a run. A running action is looked at again 5 seconds later; a rate-limited or unavailable provider 30 seconds later.
• A create sends the provider a name derived from the resource id, so a create whose answer was lost finds what it made instead of making a second one. A reserved IP has no name on DigitalOcean: a reserve whose answer was lost puts the address in error and is never sent again (see lost reserves).

Each create takes an idempotencyKey. The same key with the same body returns the same resource; with a different body it is CONFLICT.

Volumes

const volume = await compute.api.providerVolumes.create(token, {
  tenantId: 'ten_01jz…',
  poolId: 'pool_01jz…',
  name: 'pg-data',
  region: 'nyc3',
  sizeGiB: 100,
  filesystem: 'ext4',
  instanceId: 'inst_01jz…', // attach once both are ready
  idempotencyKey: 'pg-data-1',
});
// → { id: 'pvol_…', state: 'pending', attachment: { instanceId, state: 'pending', desiredState: 'present' }, … }

Once it is ready, device is the path the guest sees it at. On DigitalOcean that is /dev/disk/by-id/scsi-0DO_Volume_<name>, where <name> is the provider name derived from the volume id.

better-compute does not mount it yet: the volume arrives in the guest without a reboot, as a block device (formatted when you asked for a filesystem), and the guest mounts it, for example with an /etc/fstab line on that path with nofail. Mounting by bcd, and a cloud-init mount for volumes attached at create time, are planned.

The other methods of providerVolumes:

Attach refuses before any provider call: an instance of another tenant is ACCESS_DENIED; one on another provider account or in another region is INVALID_INPUT; an instance being deleted is INVALID_STATE.

Shared (NFS) volumes

A volume with kind: 'shared' is a network file share: one filesystem that many instances mount at once over NFS. On DigitalOcean it is a Network File Storage share. A share has no access control of its own beyond the network it is reachable from, so a shared volume always names a tenant network and is reachable only from that network's instances.

const shared = await compute.api.providerVolumes.create(token, {
  tenantId: 'ten_01jz…',
  poolId: 'pool_01jz…',
  name: 'assets',
  region: 'nyc3',
  sizeGiB: 50,
  kind: 'shared',
  networkId: 'net_01jz…', // required: the share lives in this network's subnet in nyc3
  idempotencyKey: 'assets-1',
});
await compute.api.providerVolumes.attach(token, { tenantId: shared.tenantId, volumeId: shared.id, instanceId: 'inst_a…' });
await compute.api.providerVolumes.attach(token, { tenantId: shared.tenantId, volumeId: shared.id, instanceId: 'inst_b…' });
// get → { kind: 'shared', networkId, attachments: [{ instanceId: 'inst_a…', state: 'ready', … }, …],
//         device: '10.110.0.5:/2633050/…', mount: { source: '10.110.0.5:/2633050/…', type: 'nfs4' }, … }

How it differs from a block volume:

• The network. networkId must be a network of the tenant, on the pool's provider account, provisioned, with a subnet in the volume's region (INVALID_INPUT otherwise; INVALID_STATE while the network or that subnet is not ready yet, or the network is being deleted). The share is made in that subnet, never in the region's default VPC. networkId on a block volume is INVALID_INPUT. A network a shared volume is on is not deleted (networks.delete answers INVALID_STATE until the volume is deleted), and neither is a subnet of it in the volume's region (networks.removeSubnet answers INVALID_STATE, with the volumes in details.volumeIds). A provider that makes no shared volumes is UNSUPPORTED (reason: 'sharedVolumes').
• Many instances. attach adds the instance (attaching one it is on returns the volume); the instance must be on the volume's network (INVALID_INPUT otherwise). The view lists every attachment in attachments (a block volume keeps the single attachment). detach names the instance, { volumeId, instanceId } (INVALID_INPUT without it). delete is INVALID_STATE while any instance still has it.
• No filesystem choice. A share is a filesystem already: leave filesystem out (none is accepted and ignored). deleteWithInstance is always false: deleting an instance detaches it from the share and keeps the share.
• Size. It grows like a block volume and books diskGiB quota the same way. DigitalOcean shares are 50 to 16384 GiB: a smaller size is INVALID_INPUT (reason: 'size') at create, before any quota is booked.
• Billing. A shared volume bills vps.share.gib.hours (its GiB), not vps.volume.gib.hours, so a rate card prices shares apart from block storage.
• Snapshots. snapshot works as for a block volume; providerSnapshots.restore of a shared volume's snapshot is UNSUPPORTED (reason: 'sharedRestore') for now.

On DigitalOcean, attaching tags the Droplet bc-nfs-<16 hex> (the record of which Droplets have the share) and adds the Droplet's VPC to the share when it is not there yet; that VPC must be one of the same network. Detaching removes the tag and takes the VPC off the share once no tagged Droplet is left in it; the share's last VPC stays. When an instance on the share is deleted, the detach that follows finds its Droplet gone and takes off every VPC no tagged Droplet is left in; deleting the share removes its tag. The hourly drift pass asks every ready attachment of a settled shared volume again (an attach is idempotent), which puts back a VPC a racing detach took off.

Mounting a shared volume

device (and mount.source) is the NFS source, <host>:<path>. better-compute does not mount it in the guest yet; sharedVolumeMount from @better-compute/vps makes the lines for cloud-init or /etc/fstab:

import { sharedVolumeMount } from '@better-compute/vps';

const mount = sharedVolumeMount(shared.mount!.source, '/mnt/assets'); // { readOnly: true } mounts it ro
mount.cloudConfig; // ['10.110.0.5:/2633050/…', '/mnt/assets', 'nfs4', 'defaults,_netdev,nofail', '0', '0']
mount.fstab; //       '10.110.0.5:/2633050/… /mnt/assets nfs4 defaults,_netdev,nofail 0 0'

Put cloudConfig in the mounts: list of an instance's cloud-config user data (the image needs an NFS client, such as nfs-common on Ubuntu). _netdev waits for the network and nofail lets the guest boot while the share is unreachable. The helper refuses a source that is not <host>:<absolute path>, and a mount path that is relative, has . or .. segments or would cover part of the system (/, /etc, /usr, …), as for a sandbox volume's mount. Mounting by bcd on pool members is planned, as for block volumes.

Snapshots

A snapshot outlives its volume. providerSnapshots has get, list (filter by volumeId), delete and restore:

const copy = await compute.api.providerSnapshots.restore(token, {
  tenantId: 'ten_01jz…',
  snapshotId: 'psnap_01jz…',
  name: 'pg-data-copy',
  idempotencyKey: 'restore-1',
  // optional: sizeGiB (not below the snapshot's), description, deleteWithInstance
});
// → a new provider volume, created from the snapshot

A restore waits until the snapshot is ready, and fails if the snapshot is deleted first. The restored volume keeps the filesystem of the volume the snapshot was taken of: a restore takes no filesystem (INVALID_INPUT), since asking the provider to format it could wipe the restored data.

Addresses

const ip = await compute.api.providerAddresses.create(token, {
  tenantId: 'ten_01jz…',
  poolId: 'pool_01jz…',
  name: 'ingress',
  region: 'nyc3',
  family: 'ipv4', // or 'ipv6'
  idempotencyKey: 'ingress-1',
});
await compute.api.providerAddresses.assign(token, { tenantId: ip.tenantId, addressId: ip.id, instanceId: 'inst_01jz…' });
// get → { address: '203.0.113.7', assignmentState: 'ready', instanceId: 'inst_01jz…', … }

assignmentState is none, pending, ready, deleting (an unassign running) or error (with assignmentMessage). An address is on one instance at a time: assigning it to a second instance is CONFLICT, and to an instance of another tenant ACCESS_DENIED. unassign releases it and keeps it reserved. delete is INVALID_STATE while it is assigned.

The reconciler never takes an address over from a machine it did not put it on: if the provider reports it on another machine, the assignment goes to error and nothing is moved.

Since a reserved IP carries no tag, the row is the ownership record, and one address is recorded by at most one live row. Where the provider has one, the row also keeps the fingerprint from the reserve (on DigitalOcean a reserved IPv6's reservation time). A reserved IPv4 has none: its project is shared by every reservation in the project, so it cannot tell a re-reservation of the same IP apart.

An address released at the provider outside better-compute is lost for good. The reconciler notices it on its next read (the IP is gone, or another reservation's fingerprint or region shows there), or when a later reserve of the same account is handed that IP. The row then goes to error saying so, forgets the provider address and its instance, and never calls the provider again: it is never re-adopted when the IP is handed out again, never assigned, and never released. assign is INVALID_STATE; delete finishes the row (refunding its quota) without a provider call. The provider refuses a mismatched address too (INVALID_STATE, reason notManaged).

A volume deleted at the provider outside better-compute is not made again either: once a volume was created, the reconciler only reads it, and one the provider no longer has goes to error (its data is gone). Delete it.

Lost reserves

DigitalOcean's reserved IPs carry no name or tag, so a reserve whose answer never arrived (a timeout or a dropped connection) cannot be found again. Rather than risk a second address, the address goes to error with a message saying so, and is never reserved again. Delete it, and check the account's reserved IPs for one that is not assigned. A reserve that was refused or rate-limited reserved nothing, and is retried.

Deleting an instance

Deleting an instance takes care of its volumes and addresses:

1. While the machine is being deleted, a volume with deleteWithInstance makes no provider call (an ensure by name could otherwise make again a volume the delete already took).
2. The machine delete names the instance's deleteWithInstance volumes. On DigitalOcean that is one DELETE /v2/droplets/{id}/destroy_with_associated_resources/selective naming only volumes that carry this deployment's managed tags and are attached to that Droplet. A tenant's own volume attached by hand is never named and survives. With nothing to name it is a plain DELETE /v2/droplets/{id}. Tag-scoped bulk deletes are never used.
3. When the delete finishes, the instance's attachments are no longer wanted, its deleteWithInstance volumes move to deleting, and every volume and address it had is woken. The reconciler settles the detach, deletes what is to go (already gone after a selective destroy; a provider without one deletes it after the machine) and unassigns its addresses, which stay reserved.

Once the instance's delete is asked, detach of a deleteWithInstance volume is INVALID_STATE: it goes with the machine. Detach it before deleting the instance to keep it; a volume detached before the delete is never deleted with it.

An instance that finishes as a detach (an attached machine with onDelete: 'detach', or one whose ownership label is gone) leaves its machine running. Its volumes are only detached, deleteWithInstance or not.

Quota, billing and events

• A volume books its sizeGiB as diskGiB quota on (tenant, pool), the resource a provider instance's disk books; growing it books the difference. An address books one addresses. A tenant without a quota row for the resource gets nothing (QUOTA_EXCEEDED, fail closed). The booking is refunded once the provider has deleted the resource. Snapshots book no quota.
• On a metered pool, each resource bills from the moment it was ready until it is deleted, in whole-hour usage windows written by vps.ancillary-usage: a snapshot vps.snapshot.gib.hours (its GiB), a block volume vps.volume.gib.hours (its GiB), a shared volume vps.share.gib.hours (its GiB) and an address vps.address.hours (one per hour). One that never became ready never bills; a BYO account's pool (platform fee) bills none of them. Volumes, snapshots and addresses are all metered by the vps.ancillary-usage pass (usage source kinds volume, snapshot and address).
• Events: providerVolume.* (created, updated, ready, attached, detached, resized, error, deleting, deleted), providerVolume.snapshot.* and providerAddress.* (created, updated, ready, assigned, unassigned, error, deleting, deleted). See the event catalog.

Permissions

list is decided on the tenant. A create is decided on the new resource, a restore on the snapshot and a snapshot on its volume.

providerVolumes.attach, providerVolumes.create with an instanceId and providerAddresses.assign put the volume or address on an instance, so they are also decided on that instance: the caller needs instances:update on instance/{instanceId} as well, with its own credential. A role that manages volumes in one project cannot attach one to an instance it cannot change.