The Three Signals

Telemetry comes in three shapes, and every observability discussion you will ever have sorts into them. Metrics are numeric aggregates over time — cheap, alertable, and anonymous. Logs are discrete timestamped events — detailed, expensive, and specific. Traces follow one request across service boundaries — the only signal that knows where in the path the time went.

Each answers questions the other two structurally cannot, which is why a real stack runs all three and why this book gives each its own chapters — metrics in Chapters 3–5, logs in Chapter 7, traces in Chapter 8. Learn the shapes now and the tool choices later stop looking arbitrary.

Metrics — Cheap Aggregates That Alert

A metric is a number sampled over time: request count, p95 latency, queue depth. The defining property is pre-aggregation — the counter harborline_bookings_total costs the same to store at 10 requests per second as at 10,000, because it is one number incremented in memory, not one record per event. That flat cost curve is why metrics scale to any traffic level without a budget meeting.

Cheap math over time series is also what makes metrics the alerting signal: "error ratio over 2% for 5 minutes" is a trivial computation over a counter and an impossible one over raw events at volume. The price of all this is anonymity. A metric can tell you p95 checkout latency is 2.1 seconds; it can never tell you which request, which customer, or what the slow request was doing.

Logs — What Happened at That Moment

A log line is one event with full local detail: the exception with its stack trace, the SQL statement that timed out, the request id, whatever the developer chose to record at that moment. Logs answer "what exactly did this failing request say" — the question the anonymous metric just raised. The cost model is the mirror image of metrics: every event is stored individually, so cost scales linearly with traffic. At Harborline's search service, 300 requests per second on a summer Saturday is roughly 26 million log lines a day if each request logs one INFO line — and that number sits behind every logging decision in Chapter 7.

Traces — Where in the Request Path

A trace records one request as a tree of timed spans across services. When a Harborline checkout crosses web → bookings → payments, the trace answers the question neither of the other signals can touch: where did the 2.1 seconds go? 40 ms in web, 180 ms in bookings, 1.9 s waiting on payments — read directly off the span tree, no correlation guesswork.

Traces have two prices. The admission price is context propagation: every service must pass trace context along on every outgoing call, because one hop that drops it splits the trace in two. The running cost is per-request, like logs, so nobody stores every trace — you keep a sample, and choosing that sample well is a Chapter 8 problem with real teeth.

The Extended Family

Three more signal types orbit the core three. Continuous profiling (Pyroscope, Parca) shows which line of code burns the CPU. Error events (Sentry) group exceptions with stack traces and deduplicate them. Synthetic probes test the system from outside, on a schedule, whether or not real users are hitting it. Chapter 13 covers all three as supplements — none replaces a core signal, and none should be built before the core three exist.

The Negative Space

The fastest way to internalize the three signals is by what each cannot do. Metrics cannot show you the single failed request — the aggregate discarded it at emit time. Logs cannot cheaply compute a rate: scanning 26 million lines to produce a number a counter holds in a few bytes is possible, and gets slower every week you grow. Traces cannot page anyone at 02:40 — they are per-request evidence, not an aggregate condition you can put a threshold on.

Notice that every one of those limits is a cost statement in disguise. Metrics are cheap because they discard the individual event; logs are exact because they pay for every event; traces are affordable because they keep a sample. Every "which signal do I use" decision is secretly a cost decision — topic 04 makes that explicit and gives the costs names.

One Request, Three Views

One Harborline checkout, observed three ways: a +1 on harborline_bookings_total, a structured log line inside bookings carrying the request id, and a span tree crossing three services. Same event, three records, three different questions answered. Keeping the ids and label vocabulary consistent across all three — same service name, same env, a trace id in the log line — is what makes Chapter 8's payoff possible: jumping from a latency spike on a metric panel to the exact slow trace to that request's logs, without re-typing a timestamp.