Capacity Management — Why Clouds Run Out at Sixty Percent

The headline number is not the capacity

“The cloud is at 60%” is one figure summarising several resources that exhaust independently. What matters is not the average but the first one to run out, and it is usually not the one being watched.

Capacity is exhausted per host, not in aggregate. A cloud with 40% of its memory free and no single host able to fit a large flavor is full for that flavor, and its dashboard says 60%.

Read what placement believes, host by host

$ openstack hypervisor list –long+————+—––+———–+————+–––––+| Host | vCPUs | vCPU Used | Memory MB | Mem Used |+————+—––+———–+————+–––––+| compute-01 | 128 | 96 | 515000 | 410000 || compute-02 | 128 | 124 | 515000 | 180000 | ← CPU bound| compute-03 | 128 | 20 | 515000 | 505000 | ← memory bound+————+—––+———–+————+–––––+

Aggregate across those three and the cloud looks comfortable. In practice compute-02 can take no more vCPUs and compute-03 can take no more memory, so only one host can place a balanced flavor. This is fragmentation, and it is the most common reason a cloud stops accepting instances well before its utilisation figure suggests.

Allocation ratios move the ceiling, and only for some resources

Allocation ratios permit over-commitment. They apply differently by resource, and the difference is where the trouble is.

Resource	Typical ratio	Why
CPU	4–16	Instances are mostly idle; sharing works
Memory	1.0–1.5	Memory in use is genuinely occupied
Disk	1.0–2.0	Depends on thin provisioning

Stale allocations: capacity that exists and cannot be used

Placement tracks allocations. A failure between creating an allocation and cleaning it up leaves capacity reserved for an instance that no longer exists.

The signature is unmistakable once known: placement reports a host as full, the hypervisor shows few instances running, and the two never reconcile on their own.

# What placement thinks is consumed here$ openstack resource provider allocation list <provider-uuid># What is actually running here$ openstack server list –host compute-03 –all-projects# More consumers than servers = stale allocations

Every consumer UUID in the allocation list should correspond to a real instance. Those that do not are holding capacity that will never be released without intervention.

The resources nobody watches until they run out

Compute is watched. These are the ones that cause surprise:

Floating IPs. A finite pool. Exhaustion prevents new externally-reachable instances while every compute metric looks healthy. Frequently made worse by allocated-but-unassociated addresses held by projects that have forgotten them.

Ports and subnet addresses. A /24 tenant subnet holds about 250 addresses, minus infrastructure ports. Large deployments hit this and the error names the port, not the subnet.

Storage IOPS rather than capacity. The array has terabytes free and cannot serve more operations. Presents as every instance being slow — an application problem, apparently.

Placement inventory for specialised resources. GPUs and other passthrough devices are tracked separately and exhaust independently of anything on the dashboard.

Quota, which is not capacity. A tenant blocked by quota gets an error that reads like the cloud being full. Checking quota first costs seconds and resolves a meaningful share of “the cloud is full” reports.

Planning against lead time

Capacity planning is a race between growth and procurement, and the useful question is not “how full are we” but “will hardware arrive before we need it”.

  months of headroom remaining   >   months to procure and install

If hardware takes three months to arrive, three months of headroom is the threshold at which to order, not the point at which to become concerned. Most capacity emergencies are the result of measuring the left side and never writing down the right.

Track the trend per resource, per host aggregate, and project forward. A single cloud-wide percentage cannot answer the question because different resources exhaust at different rates.

The diagnostic path for “the cloud is full”

1. Is it quota? Check the project’s quota before anything else.
2. Which resource is short? Per host, not aggregate — CPU, memory, and disk exhaust independently.
3. Is it fragmentation? Total free capacity sufficient, no single host able to fit the flavor.
4. Are allocations stale? Compare placement consumers against running instances per host.
5. Is it a resource nobody is watching? Floating IPs, subnet addresses, specialised devices.
6. Only then is it genuinely time to add hardware.

Steps 1 through 5 resolve most reports, and each is faster than the last.

Exercise

Calculate how many instances of your most common flavor your cloud can still place. Do it properly: per host, honouring allocation ratios, taking the minimum across CPU, memory, and disk rather than the average.

Then try to boot that many. The number you achieve will be lower than the number you calculated, and the gap is fragmentation plus whatever else is holding capacity. Understanding that gap for your own cloud is the entire point — it is the difference between the number on the dashboard and the number that matters.