Logging Drivers and Log Management

A container should write its logs to STDOUT and STDERR and nothing else — that is the 12-factor rule, and it exists because Docker captures those two streams and hands them to a logging driver. The driver decides where the lines go: a file on the host, the systemd journal, a remote collector. The application never opens a log file or rotates one, which keeps the image stateless and makes the log destination a deployment decision rather than a code decision.

The default driver, json-file, writes every line to a JSON file on the host that grows without bound until it fills the disk and takes the daemon down with it. Picking a driver and bounding its output is the first operational decision for a long-lived container, not an afterthought — the day driftwood/web goes from a demo to a service that runs for a month is the day the unbounded log becomes a production incident.

Log to STDOUT/STDERR, Let Docker Capture It

An application inside a container should not manage its own log files, directories, or rotation. It writes to the two standard streams — informational lines to STDOUT, errors to STDERR — and Docker's logging driver decides where they land. That split keeps the image stateless: the same driftwood/web image logs to a file on a laptop and to CloudWatch in production without a single line of code changing, because the destination is configured on the daemon and the container, not compiled into the app.

The anti-pattern is an app that opens /var/log/driftwood/app.log inside the container and rotates it itself. Those lines never reach a logging driver, they vanish when the container is removed, and they bloat the writable layer in the meantime. Logging to a file inside the container is logging into a black hole that also costs disk.

The json-file Default and the Unbounded-Growth Footgun

Out of the box, Docker uses json-file. It appends every captured line to /var/lib/docker/containers/<id>/<id>-json.log — forever. There is no rotation, no cap, no cleanup. A chatty driftwood/web at debug verbosity can write several gigabytes in a day, and the only thing standing between that and a full disk is two options you have to set yourself: max-size (rotate when the file reaches a size) and max-file (how many rotated files to keep).

You set them per container at run time, or as the daemon default so every new container inherits them. Setting them per container looks like this — three rotated files of 10 MB each, so the container's logs never exceed 30 MB on disk:

docker run -d --name web \
  --log-driver json-file \
  --log-opt max-size=10m \
  --log-opt max-file=3 \
  driftwood/web:1.4.0

The same caps in a Compose service live under a logging: key, and the daemon-wide default goes in daemon.json so you stop relying on remembering the flags. The point is not which mechanism — it is that json-file without max-size and max-file is a disk-fill waiting for a long enough uptime.

The local Driver

Docker's newer recommended local driver is local. It stores logs in a compressed binary format with rotation enabled by default — roughly max-size=20m across five files — so it is bounded the moment you choose it, with no options to remember. It is more space-efficient than json-file and still fully readable by docker logs. When logs stay on the host and you are not shipping them anywhere, local is the right default; json-file survives mainly because changing the out-of-the-box default would break tooling that parses the raw JSON files directly.

Shipping Drivers

When logs need to outlive the container — and the host — you reach for a driver that forwards them off the box. journald hands lines to the host's systemd journal, queryable with journalctl. fluentd forwards to a Fluentd or Fluent Bit collector. awslogs pushes to CloudWatch. Each moves logs somewhere durable so a destroyed container or a dead host doesn't take the evidence with it. You don't lose docker logs in the trade: since Docker 20.10, the daemon keeps a small local dual-logging cache even for drivers that can't read back, so docker logs still replays the most recent lines — the full, durable history is what lives off the box in the collector.

What docker logs Can and Can't Read

docker logs <container> replays captured output. Drivers that keep their own readable copy — json-file, local, journald — serve it directly. For drivers that cannot read back, such as fluentd or awslogs, Docker's dual-logging cache (on by default since 20.10) steps in, so docker logs web still returns recent output rather than the old configured logging driver does not support reading error. The catch is what the cache is: a small, recent ring buffer — about 20 MB across five files by default — not the full stream. During an incident you get the last few minutes locally, but the durable, complete history is in the collector — and if someone set cache-disabled, even that local replay is gone.

Configuring the Driver

You set the driver and its options in one of two places. Per container, with --log-driver and --log-opt (or Compose's logging: block); or daemon-wide in daemon.json as the default every new container inherits. A daemon default that bounds json-file for the whole host looks like this:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  }
}

The per-container setting overrides the daemon default, so a container that needs to ship to fluentd can opt out of the host default. The trap is the reverse: changing the daemon default only affects containers created after the daemon restart. An already-running driftwood/web keeps the driver it was created with until you recreate it — editing daemon.json does not retroactively rotate the log that is already growing.