Topic 65

The Plugin Ecosystem

ArchitecturePlugins

Modules are the part of Ansible everyone learns first, but they are one plugin type among many — almost every moving part of Ansible is a pluggable component you can swap or extend. Callback plugins shape the output you see, lookup plugins pull data at template time, filters and tests transform values in Jinja2, connection plugins reach the node, strategy plugins decide task ordering across hosts, and become plugins handle privilege escalation.

Plugin types — almost every moving part of Ansible is pluggable

Callback

Shapes the output you see — stdout, JSON, a profile_tasks timing profile, a log sink.

Lookup

Pulls external data into a play at template time — a file, an env var, a secret.

Filter

Transforms values inside Jinja2 — to_nice_json, combine, and the rest.

Connection

Reaches the node — ssh, local, winrm, network_cli.

Inventory

Builds the host list — from a static file or a live cloud API.

Knowing the plugin types is knowing where Ansible's behavior comes from and where you would hook in to change it. When the output isn't what you want, when a value needs reshaping, or when a play needs data from outside, the answer is usually "there is a plugin for that" rather than a shell task glued in to fake it. This topic maps the landscape so you can name the part you need to touch.

Modules vs Plugins

The distinction that organizes everything else: modules are code copied to the managed node and executed there — they do the work on the host, like installing a package or writing a file. Plugins run on the control node and extend Ansible's own behavior. A callback that formats output, a filter that transforms a value, a lookup that fetches data — none of these ship to the target, because they shape what Ansible does, not what happens on the host. If it runs on the managed node it's a module; if it shapes Ansible itself it's a plugin.

Output and Logging — Callback Plugins

Callback plugins consume play events and decide what reaches your screen or a log sink. The default is the human-readable stdout you see on every run, but community.general ships callbacks that send results to JSON, produce a timing profile with profile_tasks, or forward to a log aggregator. Swapping a callback re-shapes the entire run's output without touching a single playbook — which is how you get per-task timing in CI without editing any tasks, just by enabling the callback in ansible.cfg.

Data Pull-in — Lookup and Vars Plugins

Lookups fetch external data into a play at template time: lookup('file', ...) reads a file, lookup('env', ...) reads an environment variable, lookup('aws_secret', ...) pulls a secret. They run on the controller and return a value into your template, idempotently, with no shell task in sight. Vars plugins are a quieter relative — they are the mechanism that auto-loads group_vars and host_vars, so variables appearing for a group is a vars plugin doing its job, not magic.

Value Transformation — Filters and Tests

Jinja2 filters transform values — | to_nice_json pretty-prints, | ansible.builtin.combine merges two dicts — and tests ask boolean questions, like is defined or is match. Both are plugins, which is the useful part: when your template needs a transformation that isn't built in, a collection can ship the exact filter, and you install it rather than hand-rolling the logic in a set_fact chain. The filter you want usually already exists somewhere in the ecosystem.

Run Behavior — Connection, Strategy, Inventory, Become

The shape of a run is assembled from several plugin types working together. Connection plugins, the last topic, reach the node. Strategy plugins (linear, free) order tasks across hosts. Inventory plugins build the host list, whether from a static file or a live cloud API. Become plugins (sudo, su, doas) escalate privilege on the target. None of these are settings buried in the engine — they are pluggable parts, which is why you can change how privilege escalation works or how hosts are discovered without patching Ansible itself.

Where Plugins Live and Load From

Plugins come from three places. Ansible's built-ins ship in ansible-core. Installed collections carry namespaced plugins you reference as namespace.collection.plugin. And project-local plugin directories sit beside the playbook — filter_plugins/ for custom filters, callback_plugins/ for callbacks, library/ for custom modules — discovered by directory name. "Almost everything is a pluggable type" is the design, and it means your repo can carry its own filters and callbacks, versioned alongside the playbooks that use them.

Enabling a timing callback in ansible.cfg, and a lookup at template time

# ansible.cfg — turn on per-task timing for every run, no playbook edits
[defaults]
callbacks_enabled = ansible.posix.profile_tasks

# in a task — pull a secret with a lookup plugin instead of a shell command
api_token: "{{ lookup('community.general.passwordstore', 'larkspur/api_token') }}"

The callback line re-shapes output for the whole run from one config setting. The lookup pulls a secret into a variable at template time on the controller — a plugin doing cleanly what a command task would do messily and non-idempotently. Neither touches the managed node; both are controller-side plugins shaping the run.

Module vs Plugin

Module — packaged code copied to the managed node and run there to change state (apt, copy, service), reporting back what it changed. If it executes on the target, it's a module.

Plugin — runs on the control node and extends Ansible itself: callbacks format output, lookups fetch data, filters transform values, strategies order tasks. If it shapes Ansible's own behavior rather than the host's state, it's a plugin.

Common Mistakes

Calling everything a "module" and then being confused why a custom Jinja2 filter or a callback never ships to the managed node — those are plugins that run on the controller, not modules copied to the host.
Writing a shell task to fetch a secret from a file or env var when a lookup plugin already does it cleanly at template time, reinventing a built-in plugin as imperative glue.
Dropping a custom filter into the wrong directory name — not filter_plugins/ beside the play, and not namespaced in a collection — so Ansible never discovers it and the template errors on an "unknown filter."
Ignoring callback plugins and parsing raw stdout in CI for timing or results, when profile_tasks and a JSON callback give structured output for free.
Assuming a missing filter or test means you must patch Ansible, when the right one usually lives in an installable collection like community.general.

Best Practices

Enable the profile_tasks callback in the Larkspur ansible.cfg so every CI run prints per-task timing and you can see which task on the 6-host deploy is the slow one.
Pull external data with the matching lookup plugin (file, env, aws_secret, password) instead of command/shell, keeping it idempotent and at template time.
Install the collection that carries the filter, test, or callback you need (community.general) rather than hand-rolling a project-local plugin.
Keep any genuinely custom plugins in the correctly named project directories or a versioned collection, so they load predictably across the team and CI.
Reach for the plugin layer first when behavior needs changing — output, data pull-in, escalation, ordering — instead of bolting on a shell task that fakes what a plugin already does.

Comparable tools Terraform providers are roughly the module analog; functions parallel filters SaltStack renderers, returners, and grains map loosely onto Ansible's plugin types Puppet custom facts and functions are the nearest equivalents

Knowledge Check

What is the defining difference between a module and a plugin in Ansible?

A module is copied to the managed node and runs there to change state; a plugin runs on the control node and extends Ansible's own behavior
A module is always written in compiled Python while a plugin is declared purely in YAML markup
A module always ships bundled with ansible-core, whereas a plugin must always be installed separately by hand
A module only ever changes files on disk while a plugin only changes running services; the two simply target completely different resource types

You want every CI run to print per-task timing without editing any playbook. What changes that?

A callback plugin like profile_tasks — enabling it in ansible.cfg re-shapes the run's output for the whole fleet
A lookup plugin that reads per-task timing data back from each managed host at template-evaluation time
A become plugin that escalates privileges on each managed host so that the timer is able to read the system clocks accurately
A custom module copied to each managed node that records and returns its own per-task runtime

A play needs a secret from a password store pulled into a variable. What is the clean way to do it?

A lookup plugin, which fetches the value on the controller at template time, idempotently and with no shell task
A command task that shells out to the password store's own CLI and then registers its raw stdout into a variable for later use
A custom module that is copied out to each managed node and reads the secret locally on the host
A strategy plugin that injects the fetched secret into scope before it orders the play's tasks

You wrote a custom Jinja2 filter but the template errors with "unknown filter." What is the most likely cause?

It isn't in a discovered location — not in a filter_plugins/ directory beside the play, and not namespaced in an installed collection
Custom Jinja2 filters must be compiled to bytecode ahead of time before Ansible is able to load and call them
Custom filters only ever work from inside module code itself, and never from within ordinary playbook templates
The filter was evaluated out on the remote managed node, which has no Jinja2 runtime installed there at all to resolve the filter name

You got correct