Dynamic Inventory for the Cloud
A static inventory rots the moment your fleet autoscales. Hosts come and go on a schedule no hand-edited file keeps up with, and a list you committed this morning describes a fleet that no longer exists by afternoon. The file is stale the instant the cloud scales out.
Dynamic inventory solves this by querying the source of truth — AWS, GCP, your VMware estate, your Terraform-built fleet — at run time, so ansible-playbook always sees the servers that actually exist. The hosts are discovered, tagged, and grouped automatically, which means the inventory is generated rather than written.
The Problem Static Inventory Can't Solve
Autoscaling groups, ephemeral instances, and Terraform-built fleets change constantly by design. An instance that exists at 09:00 is gone by 09:30, replaced by one with a different address you never typed anywhere. No discipline keeps a hand-maintained file accurate against that churn — the file is stale before you finish committing it.
The conclusion is structural, not a matter of trying harder: for anything that scales, the inventory has to be generated from the cloud, not written by a person. The moment instances are created and destroyed automatically, the only inventory that can be correct is one queried at the moment it is used.
Inventory Plugins
The modern mechanism is an inventory plugin driven by a small YAML config file. A file named aws_ec2.yml or gcp_compute.yml tells the amazon.aws.aws_ec2 or google.cloud.gcp_compute plugin how to query the cloud and how to build groups from what it finds. The config is declarative; the plugin does the API calls.
The plugin name as the first line is what marks the file as a plugin config rather than a static inventory. The AWS config below filters to running instances in one region — point -i at this file and Ansible runs the plugin to populate the model.
plugin: amazon.aws.aws_ec2 regions: - eu-west-1 filters: instance-state-name: running
Grouping from Cloud Metadata
keyed_groups turns instance tags and labels into Ansible groups automatically. An AWS tag Role: web becomes the web group with no manual mapping written anywhere — the cloud's metadata becomes your inventory structure directly. The grouping you would otherwise maintain by hand is derived from data the instances already carry.
The config below builds a group per Role tag value, so web and db groups appear without listing a single host. Tag an instance Role: web at launch and it joins the web group the next time Ansible queries the cloud — the tag is the membership.
plugin: amazon.aws.aws_ec2 regions: - eu-west-1 keyed_groups: # a Role:web tag creates and populates the "web" group - key: tags.Role separator: ''
AWS and GCP Specifics
The two cloud plugins filter on the dimensions their cloud exposes. amazon.aws.aws_ec2 filters by region, tag, and instance state; google.cloud.gcp_compute filters by project, zone, and label. Both need credentials on the control node — an AWS profile or a GCP service account — and both need their collection installed before Ansible can load them.
Installing the collection is a one-time step, and a missing one is the usual reason a plugin "does nothing." The command below adds the AWS collection; the GCP one is the same shape with a different name.
# the plugin lives in a collection you install once
ansible-galaxy collection install amazon.aws
ansible-galaxy collection install google.cloud
The Terraform Handoff
Dynamic inventory completes the provisioning-to-configuration handoff cleanly. After terraform apply builds the fleet, the same cloud now holds instances tagged the way Terraform tagged them — and the inventory plugin reads that cloud, so Ansible configures exactly what Terraform created. No IP address is ever copied from one tool's output into the other's input.
That is the whole point of letting the cloud be the shared source of truth: Terraform writes to it, Ansible reads from it, and neither holds a stale copy of the other's view. The command below confirms the plugin sees the freshly built fleet before any playbook runs against it.
# after terraform apply, check what Ansible will target ansible-inventory -i aws_ec2.yml --graph
Static inventory is a file you maintain by hand. It is fine — and simpler — for a fixed set of long-lived servers whose names and addresses do not change. The Larkspur estate of a few named boxes is exactly this case: write it once, commit it, done.
Dynamic inventory queries the cloud at run time and is required for anything that autoscales or is rebuilt often. The rule of thumb: use static for a stable handful, and reach for dynamic the moment instances are cattle rather than pets — the moment something else is creating and destroying them on its own schedule.
- Maintaining a static inventory for an autoscaling fleet, then configuring instances that were terminated an hour ago while missing the ones that replaced them — the file and the fleet quietly disagree.
- Forgetting to install the collection (
amazon.aws,google.cloud) or provide cloud credentials on the control node, so the plugin returns zero hosts and the play silently does nothing at all. - Confusing the legacy inventory scripts with the modern inventory plugins — plugins are the supported path, configured by a small YAML file, not a Python script you maintain.
- Building groups by hand on top of a dynamic source instead of using
keyed_groups, re-introducing the exact manual mapping the dynamic inventory was meant to remove. - Running a playbook against a dynamic source without checking
--graphfirst, so a credential or filter problem shows up as an empty run rather than a clear error.
- Use inventory plugins, not scripts, configured by a YAML file, and let
keyed_groupsderive Ansible groups from cloud tags and labels instead of mapping them by hand. - Tag and label cloud instances deliberately, because those tags become your inventory structure — a
Roleand anEnvironmenttag map straight to the groups your plays target. - Pair dynamic inventory with Terraform: provision with Terraform, then let the plugin read the cloud so Ansible always targets exactly what exists, with no copied addresses.
- Verify with
ansible-inventory -i aws_ec2.yml --graphthat the plugin returns the expected hosts and groups before running a playbook against them. - Keep credentials on the control node out of the inventory config itself — use an AWS profile or a GCP service account the plugin picks up from the environment.
Knowledge Check
Why does static inventory fail for an autoscaling fleet?
- Instances are created and destroyed automatically, so a hand-edited file is stale before you commit it; the inventory must be generated at run time
- Static inventory has no syntax for groups at all, and the per-tier grouping that an autoscaling fleet relies on to target its layers always requires them
- Ansible flatly refuses to run a static inventory against any cloud instance and aborts the play before connecting
- Autoscaling fleets are assigned IPv6 addresses, which the older INI inventory format simply cannot represent
What does keyed_groups do?
- Turns instance tags and labels into Ansible groups automatically, so a
Role: webtag becomes thewebgroup with no manual mapping - Encrypts the generated inventory with a vault key so that the cloud provider credentials can be stored safely inside it alongside the hosts
- Locks each derived group to a single named SSH key that all of its hosts authenticate with
- Caches the cloud query results under a named key on disk so later reruns of the plugin go faster
What is the difference between inventory plugins and the legacy scripts?
- Plugins are the supported path, configured by a small YAML file; scripts are the older executables you maintained in Python
- Scripts are the modern, supported path today, while plugins were the older approach that got deprecated and dropped back in Ansible 2
- Plugins only ever work for hand-written static files, so you still need a script for any live cloud source
- There is no real difference — the two names just refer to the very same YAML-driven mechanism
How does dynamic inventory complete the Terraform → Ansible handoff?
- Terraform builds the fleet and the plugin reads the same cloud, so Ansible configures exactly what was created with no addresses copied between tools
- Terraform writes a ready-made Ansible inventory file to disk directly as the final step of
apply - Ansible parses Terraform's dependency lock file at startup to learn which of the freshly built instances it should then go on to connect to and configure
- The two tools share one state file on disk that records the current inventory between their runs
You got correct