Chapter 11: Configuration, Plugins & Performance
Topic 67

Accelerators and Tradeoffs

PerformanceTradeoffs

Once pipelining and multiplexing are on, the remaining overhead is Ansible re-forking and re-connecting for chunks of work, and the best-known answer is Mitogen for Ansible — a third-party plugin that holds persistent Python interpreters on each target and routes calls to them, often cutting wall-clock time dramatically. The catch is exactly that it is third-party and out-of-tree.

Two accelerators, one supported — enable the left first
Pipelining
Built-in, in-tree. An ssh_connection setting that halves SSH ops per task with zero added dependency. Supported in ansible-core — turn it on first.
Mitogen
Third-party, out-of-tree. Holds persistent interpreters for a big speedup, but patches private internals — a core upgrade can break it. Adopt only when the measured win clears the risk.

Mitogen hooks deep into Ansible's connection and execution internals, so a new ansible-core release can break it, and that compatibility risk is the whole decision. For most fleets, enabling pipelining first answers the speed question without taking on an unsupported dependency — and treating Mitogen as if it were an official, supported feature is the mistake this topic is built to prevent.

Where the Remaining Overhead Lives

With pipelining and ControlPersist on, the SSH handshake and the extra per-task round-trips are already handled. What is left is Ansible repeatedly forking a Python process and re-establishing execution context for each task — the cost of standing up the machinery to run a module, paid over and over across a long, task-heavy play. That is the gap an accelerator targets. It is not the SSH layer pipelining already optimized, so reaching for an accelerator before pipelining is solving the wrong problem.

Mitogen for Ansible

Mitogen installs a strategy plugin you select as strategy: mitogen_linear. Instead of re-forking and re-connecting per task, it keeps long-lived Python interpreters running on the targets and proxies module calls to them, so the per-task fork and context setup mostly disappears. The reported speedups are large — multiples, not percentages, on the right workload — which is exactly why people reach for it. On a fleet where per-task fork cost dominates, that is a real and substantial win.

The Third-Party, Out-of-Tree Catch

Mitogen is not part of ansible-core and is not maintained by Red Hat. It is a separate project that patches into private Ansible internals, which is what makes the speedup possible and also what makes it fragile: an ansible-core upgrade can change those internals and break Mitogen, and you then wait on a separate project to catch up before your pipeline runs again. It is not an officially supported path, and treating it as one — pinning production to it as though Red Hat stands behind it — is the central mistake.

The Deprecated History — accelerate and fireball

Accelerators have a graveyard, which is worth knowing as context for how permanent any of them is. Older Ansible shipped fireball and later the accelerate mode — daemons that opened a faster channel to targets. Both were deprecated and removed: they added a daemon to manage and a security surface to defend, and pipelining covered most of the gain they offered for none of that cost. If a stale playbook or an old guide tells you to enable accelerate or fireball, it is pointing at a feature that no longer exists.

When the Complexity Is Worth It

Mitogen earns its risk on very large fleets, or on fact-heavy and task-heavy plays where the per-task fork cost genuinely dominates and minutes of wall time matter on every run. There the measured win can clear the maintenance cost of an out-of-tree dependency. For Larkspur's 6 prod web hosts, enabling pipelining is almost certainly enough — the per-task fork cost is small at that scale, and the unsupported dependency isn't worth taking on for a win that won't move the deploy clock meaningfully. Profile first; adopt only if the number justifies the risk.

Pipelining vs Mitogen

Pipelining — a built-in, in-tree ssh_connection setting that halves SSH ops per task with zero added dependency. It is supported by Red Hat as part of ansible-core, and it is the first thing to turn on.

Mitogen — a third-party, out-of-tree strategy plugin that can cut wall time further by holding persistent interpreters, at the cost of patching Ansible internals and breaking on core upgrades. Enable pipelining first; consider Mitogen only when its measured win clears the maintenance risk.

Common Mistakes
  • Treating Mitogen as an official Ansible feature and pinning a production pipeline to it, then having an ansible-core upgrade break the whole pipeline because the out-of-tree plugin hasn't caught up.
  • Reaching for Mitogen before enabling pipelining, taking on a third-party dependency to win speed that the built-in in-tree setting would have delivered for free.
  • Resurrecting accelerate/fireball from an old playbook or guide — both are deprecated and removed; following stale advice to enable them wastes time on a dead feature.
  • Adopting Mitogen on a small fleet like Larkspur's where the per-task fork cost is negligible, paying the compatibility risk for a win that doesn't move the deploy clock meaningfully.
  • Upgrading ansible-core in CI without checking Mitogen compatibility first, so the bump that should be routine breaks every run until the plugin supports the new version.
Best Practices
  • Enable pipelining (in-tree, supported) before evaluating any accelerator, and only consider Mitogen if profiling shows per-task fork cost is the bottleneck after pipelining.
  • If you adopt Mitogen, pin both ansible-core and the Mitogen version together and gate every ansible-core upgrade on confirmed Mitogen compatibility, since it patches internals.
  • Treat Mitogen as a third-party optimization with its own lifecycle, not a core feature — document that the pipeline depends on it and who owns keeping it current.
  • Ignore accelerate and fireball entirely; they're deprecated and removed, and pipelining superseded the gain they offered.
  • Measure before and after with a timing callback so any accelerator's win is a number you can defend, not an assumption — and so a small fleet's negligible gain is visible before you take on the risk.
Comparable tools SaltStack persistent minions achieve the persistent-interpreter benefit natively and supported SSH ControlPersist the in-tree multiplexing Mitogen builds past Terraform no analog — no per-task remote-execution overhead to accelerate

Knowledge Check

Why is Mitogen's out-of-tree status the central tradeoff, rather than a footnote?

  • It patches private ansible-core internals and isn't maintained by Red Hat, so a core upgrade can break it and your pipeline waits on a separate project to catch up
  • It is measurably slower than plain pipelining despite the marketing benchmarks, so the promised speedup never actually materializes on a real production fleet under load
  • It requires installing a long-running daemon on every managed node, quietly abandoning the agentless model that makes Ansible easy to adopt
  • It only works against Windows targets reached over WinRM, so the entire Linux SSH fleet is left unable to use it at all

What does Mitogen do that pipelining does not?

  • It keeps long-lived Python interpreters on the targets and proxies module calls to them, removing the per-task fork and context setup that remains after pipelining
  • It is what opens and holds the multiplexed SSH master connection, something plain pipelining can never set up on its own
  • It gathers facts across the inventory in parallel batches, whereas pipelining is forced to run the setup phase strictly serially
  • It encrypts the wire traffic of the SSH session end to end, closing a real confidentiality gap that plain pipelining otherwise leaves fully exposed to anyone in plaintext

An old guide says to enable accelerate mode for speed. What is the right response?

  • Ignore it — accelerate and fireball were deprecated and removed, and pipelining superseded the gain they offered
  • Enable it alongside pipelining in ansible.cfg, since the two layers stack cleanly for a larger combined throughput win
  • Enable it only on the controller side, where the accelerate daemon binary is still shipped and supported
  • Replace it with Mitogen, which is simply the officially supported, renamed continuation of accelerate

When is an accelerator's complexity justified over plain pipelining?

  • On very large fleets or fact- and task-heavy plays where fork cost dominates and the measured win clears the maintenance risk of an out-of-tree dependency
  • On absolutely any fleet of any size, since the raw per-task speedup reliably outweighs the out-of-tree compatibility risk every single time without exception
  • On tiny fleets like Larkspur's six hosts, where shaving every millisecond of per-task fork cost matters the most
  • Only when the managed fleet is entirely Windows, since plain pipelining cannot apply over the WinRM transport

You got correct