Kolla-Ansible Configuration — Making a Change That Survives

The mistake everyone makes once

You need to change a Nova setting. You find /etc/kolla/nova-scheduler/nova.conf on the control node, edit it, restart the container, and it works.

Three weeks later somebody runs a reconfigure for an unrelated reason, and your change is gone. Nothing failed. Nobody was told. The setting simply reverted, and the incident it was preventing comes back.

The three sources of truth

Configuration is assembled from three places, in increasing specificity:

/etc/kolla/globals.yml        deployment-wide: enabled services, networks,
                              the VIP, TLS, backend selection

/etc/kolla/passwords.yml      every generated credential
                              back this up -- without it you cannot redeploy

/etc/kolla/config/            per-service overrides merged into the templates

All three belong in version control, except passwords.yml, which belongs in a secret store. A change to any of them is a reviewable change, which is the point.

Override scoping

Overrides live under /etc/kolla/config/ and the path determines what they apply to:

/etc/kolla/config/nova.conf                        → every nova service
/etc/kolla/config/nova/nova-scheduler.conf         → the scheduler only
/etc/kolla/config/nova/compute-01/nova.conf        → nova on that host only

Three levels: all services in a project, one service, or one service on one host.

Kolla merges overrides into its generated template rather than replacing the file, so you specify only what differs:

# /etc/kolla/config/nova/nova-scheduler.conf
[filter_scheduler]
enabled_filters = AvailabilityZoneFilter,ComputeFilter,ComputeCapabilitiesFilter,ImagePropertiesFilter
max_attempts = 5

Everything else in the generated nova.conf remains as Kolla produced it.

Preview before you apply

genconfig generates configuration without deploying it. This is the step that separates a confident change from a hopeful one.

# Keep a copy of what is currently deployed
sudo cp -r /etc/kolla/nova-scheduler /etc/kolla/nova-scheduler.before

# Generate, without deploying
kolla-ansible genconfig -i /etc/kolla/inventory --tags nova

# What actually changed?
diff -r /etc/kolla/nova-scheduler.before /etc/kolla/nova-scheduler

The diff catches something you cannot otherwise see: unrelated changes that would be applied alongside yours. If somebody edited globals.yml a month ago and nobody has reconfigured since, that change is sitting in the templates waiting for the next run.

You want to find that out from a diff, not from an outage.

Applying, scoped

# Everything -- full risk, rarely necessary
kolla-ansible reconfigure -i /etc/kolla/inventory

# One project
kolla-ansible reconfigure -i /etc/kolla/inventory --tags nova

# One project, one host
kolla-ansible reconfigure -i /etc/kolla/inventory --tags nova --limit compute-02

# One host group
kolla-ansible reconfigure -i /etc/kolla/inventory --tags nova --limit compute

--tags and --limit are what make configuration changes safe to make during business hours. A full reconfigure regenerates configuration and restarts services across the entire deployment; a change to a Nova scheduler setting has no business restarting Neutron.

Verifying the change landed

Do not assume. Check the running container.

# 1. The generated file on the host
grep -A3 '\[filter_scheduler\]' /etc/kolla/nova-scheduler/nova.conf

# 2. The file inside the running container -- this is what the service reads
docker exec nova_scheduler grep -A3 '\[filter_scheduler\]' /etc/nova/nova.conf

# 3. The service restarted and came back
docker ps --filter name=nova_scheduler --format '{{.Names}}\t{{.Status}}'

# 4. It is still registered
openstack compute service list --service nova-scheduler

Step 2 is the one that matters. Steps 1 and 2 differing means the container did not get the new configuration — usually because the reconfigure did not restart it.

Adding a compute node, correctly

# 1. Add the host to [compute] in the inventory

# 2. Prepare the host
kolla-ansible bootstrap-servers -i /etc/kolla/inventory --limit compute-04

# 3. Pull images first, so the deploy window is short and predictable
kolla-ansible pull -i /etc/kolla/inventory --limit compute-04

# 4. Deploy -- scoped to the new host only
kolla-ansible deploy -i /etc/kolla/inventory --limit compute-04

# 5. Verify it joined everything it needs to join
openstack compute service list --host compute-04
openstack network agent list --host compute-04
openstack hypervisor list | grep compute-04
openstack resource provider list | grep compute-04

Step 5’s last line is the one people skip. A compute node can deploy successfully, register as a compute service, and never appear as a resource provider in placement — in which case it will never receive an instance and nothing will report a problem.

Rollback

Because the source of truth is a set of files, rollback is a file revert plus the same scoped command:

# Revert the override
git checkout HEAD~1 -- /etc/kolla/config/nova/nova-scheduler.conf

# Apply with exactly the same scope you used to deploy it
kolla-ansible reconfigure -i /etc/kolla/inventory --tags nova

Using the same --tags and --limit for the rollback as for the change is important. A narrower rollback leaves some hosts on the new configuration; a wider one applies unrelated pending changes.

Exercise

Change max_attempts in the Nova scheduler configuration from its default to 5.

1. Write the override at the service scope, not the project scope
2. Run genconfig and diff against the currently deployed configuration
3. Confirm the diff contains only your change — investigate anything else it shows
4. Apply with --tags nova
5. Verify the value inside the running container, not just on the host
6. Revert it, and confirm the revert landed the same way

The point of step 3 is the habit. Most of the value of genconfig is finding the change you did not know was pending.