Batching and Throttling

A change that is correct on one host and catastrophic on sixty is the failure mode this topic prevents. serial bounds how many hosts a bad release reaches per wave, max_fail_percentage aborts the whole play once too many hosts in a batch fail, throttle caps a single dangerous task, and --limit carves the run down to a subset. Together they are the guardrails that stop a push-based tool from pushing a mistake to the entire fleet at once.

These are complementary, not alternatives — each addresses a different way a fleet-wide change goes wrong, and a serious production rollout layers all four. The topic takes them one at a time, then shows how they stack.

serial as Blast-Radius Control

Beyond rolling deploys, serial is the primary limit on how many hosts a change can break before you notice. It takes a count, a list, or a percentage, and whatever the form, the effect is the same: a bad config at serial: 1 fails on one host, not all six. The wave size is the blast radius. Choosing it is choosing how much damage a wrong change is allowed to do before the run reaches a point where you can stop it.

max_fail_percentage — Stop the Bleed

max_fail_percentage: 20, paired with serial, aborts the entire play once more than twenty percent of a batch fails. Inside a wave each task runs on all of its hosts at once, up to the forks limit, so a broken step lands on all six hosts of a six-host wave together, and Ansible checks the threshold right after it. When two of the six fail, the play stops cold: the four survivors skip the remaining tasks and no later wave starts. This matters because the default behavior is far more forgiving: without the setting, Ansible ends the play only when every host in a batch has failed. Two broken hosts out of six is not that, so the other four finish, the next wave begins, and a change that breaks a third of every wave rolls on across the whole fleet. max_fail_percentage turns "carry on while any host survives" into "fail a little, then halt."

- hosts: web
  serial: 2                  # two hosts per wave
  max_fail_percentage: 20    # abort the whole play if >20% of a wave fails
  become: true
  tasks:
    - name: Deploy the new release
      ansible.builtin.git:
        repo: https://github.com/larkspur-io/app.git
        dest: /opt/larkspur
        version: "{{ release_tag }}"

throttle — Cap One Hot Task

throttle: 2 on a specific task limits it to two hosts at a time regardless of forks or serial. It is for the one step that hammers a shared resource — a license server, a package mirror, an upstream API — that would buckle if the whole batch hit it simultaneously. throttle is scoped to the task, not the play: the rest of the wave runs at full width, and only the dangerous task is squeezed down to a rate the shared resource can survive.

--limit Batches — Manual Carving

--limit web1.larkspur.io:web2.larkspur.io runs the play only against the named subset, and --limit @retry_file runs it only against the hosts a previous run recorded as failed. This is the operator-driven complement to serial — a human carving the run by hand to canary a change on two named hosts, or to re-run just the hosts that failed instead of the whole fleet. Where serial batches automatically, --limit is you drawing the boundary for one invocation.

Layering the Guards

The four guards stack because each catches a different failure. serial sets the wave size so a bad change reaches only a few hosts at a time. max_fail_percentage sets the abort threshold so a wave that starts failing halts the rollout. throttle protects the one task that pulls on a shared, rate-limited resource. --limit is the human override for canarying or re-running failures. Use one where you meant another and you either flood a shared resource, slow the whole play needlessly, or let a broken change roll on past the point it should have stopped.

There is a sharp ordering dependency between the first two. max_fail_percentage is evaluated per batch, so it only does its job when serial has created batches to evaluate against. Set the threshold without batching and it is measured against the whole play at once, which defeats the early-abort intent on a large run. The two are a pair: serial creates the waves, max_fail_percentage decides when a wave's failures stop the rollout.