Kolla-Ansible Architecture — Containers, Inventory, and Where Truth Lives

One container per service, named for the service

Kolla-Ansible runs each OpenStack service as its own container. The naming is predictable, which makes it navigable without documentation.

$ docker ps –format ‘table .Names\t.Status’ | sortcinder_api Up 6 dayscinder_scheduler Up 6 dayshaproxy Up 6 dayskeystone Up 6 daysmariadb Up 6 daysnova_api Up 6 daysnova_scheduler Up 6 daysrabbitmq Up 6 days

Two habits follow, and both are worth forming early:

docker ps on the right host answers “is it running”. Not on any host — the controller runs API and scheduler services, compute hosts run nova_compute and the L2 agent. Looking for nova_compute on a controller finds nothing and means nothing.

docker logs <container> is the service log. There is no separate log file to find. This is the single largest ergonomic difference from a package-based deployment.

The inventory decides what runs where

The inventory assigns hosts to groups, and group membership decides which containers are deployed.

[control]
controller-01
controller-02
controller-03

[compute]
compute-01
compute-02

[storage]
storage-01

A host in [compute] gets compute containers. Move it to [control] and it gets control plane containers on the next deploy.

Where configuration actually comes from

This is the part that causes silent reverts, and the layering is not obvious.

  1. Kolla defaults              built into the role templates
  2. globals.yml                 your deployment-wide values
  3. passwords.yml               generated secrets
  4. /etc/kolla/config/...       your service overrides
              │
              ▼  ansible renders, during deploy or reconfigure
  5. /etc/kolla/<service>/       the generated file on the host
              │
              ▼  bind-mounted
  6. inside the container        what the service actually reads

The critical property: layer 5 is generated. It is output, not input.

Editing /etc/kolla/nova-api/nova.conf directly works — the service reads it after a restart — and is destroyed by the next reconfigure, which regenerates that file from layers 1 to 4. The change disappears, usually weeks later, during unrelated work, and the incident it was preventing returns with nothing to connect it to.

Reading the effective configuration

When behaviour disagrees with what you believe is configured, read what the service is actually using rather than what you wrote.

# The generated file on the host$ grep -n ‘^cpu_allocation_ratio’ /etc/kolla/nova-compute/nova.conf142:cpu_allocation_ratio = 4.0# Confirm the container sees the same thing$ docker exec nova_compute grep -n ‘^cpu_allocation_ratio’ /etc/nova/nova.conf142:cpu_allocation_ratio = 4.0

When those two disagree, the container was started before the file changed and needs restarting. When they agree and the behaviour still does not match, the value is being overridden somewhere else — frequently by a database-backed setting that configuration does not control at all, which is true of several placement and quota values.

Deploy, reconfigure, and what each one does

Command	Does	Use when
deploy	Full run: pulls images, creates containers, runs bootstrap tasks	First install, adding hosts
reconfigure	Regenerates configuration and restarts affected services	Changing configuration
pull	Fetches images only, changes nothing running	Staging an upgrade
upgrade	Runs migrations and moves services to new images	Version changes

reconfigure is the one used most and the one with the sharpest edge: it restarts services whose configuration changed. On a cloud with a single controller, that is an API outage. On three controllers it is a rolling restart and mostly invisible — provided the load balancer’s health checks are good enough to notice a backend restarting, which the HAProxy lesson covers.

The diagnostic path for “my change did nothing”

1. Is the change in /etc/kolla/config/? If it was made directly to a generated file, it will revert and may already have.
2. Was reconfigure run for that service? Rendering does not happen on its own.
3. Does the generated file contain the value? If not, the override path or filename is wrong.
4. Does the container see it? If the host file has it and the container does not, restart the container.
5. Is the value overridden elsewhere? Some settings live in the database and ignore configuration entirely.

Exercise

Pick one running service and trace a single configuration value all the way back: from inside the container, to the generated file on the host, to whichever of globals.yml, an override file, or the Kolla default produced it.

Then find a value in the generated file that appears in none of your inputs. It came from a role default. Knowing that those exist — and that they change between releases — is what makes an upgrade’s behaviour changes predictable rather than mysterious.