Headers That Carry Meaning

Most headers are plumbing the framework handles, and a service author who never thinks about Date or Transfer-Encoding is right not to. A handful carry decisions the service must make on purpose. Content-Type decides how the body is parsed. Accept decides what comes back. Authorization carries the identity. Idempotency-Key makes a POST safe to repeat. A request id ties one buyer's five requests together in the logs, and three forwarded headers tell the service who is really calling, if it trusts the right source.

This topic is the short list a service author must own, and for each one the decision it carries and the failure when it is left to a default. None of them is exotic. Every one of them has cost Stagedoor an evening: a form post parsed as JSON, a token in an access log, a rate limiter defeated by a header anyone could set.

Content-Type and the Parse

The service parses the body according to the type the client declared, and nothing else. A body declared application/json that is not JSON is a 400 before any handler runs. A body with no declared type, or a type the endpoint does not accept, is a 415 Unsupported Media Type, also before any handler runs. Stagedoor accepts exactly three: JSON on the API, application/x-www-form-urlencoded on the login form, and multipart/form-data on the organizer's seat-map upload, which is the one endpoint that takes a file.

The order matters. A lenient parser that tries JSON on whatever arrives will accept a form post whose body happens to start with a brace, produce a typed object with the wrong fields, and hand it to a handler that was never meant to see it. The check on the declared type is one line and it is the first line: parse only what was declared, refuse the rest with 415, and let Chapter 3's validation start from a body that is at least the kind of thing it expected.

Accept and Negotiation

The client says what it can read; the service picks from what it can produce; when the two do not overlap the answer is 406 Not Acceptable. Stagedoor produces JSON everywhere and text/csv on one resource, the organizer's sales report, where a spreadsheet is what the organizer actually wants. The Accept header chooses, not a ?format=csv parameter, because the header is the mechanism every cache and proxy already understands: a cache keyed on the URL alone would hand the JSON client the CSV that the spreadsheet client fetched a second earlier, and Vary: Accept is how the service tells it not to.

A client that sends no Accept gets JSON. A client that sends Accept: text/csv to an endpoint that produces only JSON gets a 406 with a body naming what is available, which is more useful than the JSON it said it could not read. Negotiation is a small thing at Stagedoor's scale, two types on one resource, and the point of doing it through the header is that the day a third type is needed nothing else has to change.

Authorization

Authorization: Bearer followed by the token for users, Authorization: ApiKey followed by the key for the scanner app; Chapter 5 is about what is inside each. What this topic owns is where the credential travels: in that header, and nowhere else. Not in the query string, ever, because query strings land in the load balancer's access log, in the browser's history, in the Referer header sent to the next site unless a referrer policy strips it, and in every screenshot of a URL bar. A token that has been in any of those places is a token that has been shared.

The one place the rule is tested is the download link. An organizer wants a link to the sales report she can paste into an email, and a link cannot carry a header. The answer is not a token in the query string; it is a separate, short-lived, single-purpose credential that names one report and expires in 10 minutes, so that the thing in the email is worth nothing after the report is downloaded. The session token, which opens everything, stays in the header where only code can put it.

Idempotency-Key

A client-generated UUID on POST /orders, sent as Idempotency-Key. The service stores the key with the request's hash and, once the work is done, the response's status and body, in the idempotency_keys table. A repeat with the same key returns the stored answer without touching Payrail or the orders table. That is the mechanism of Chapter 7; the header is its contract, and the one thing about it that belongs in this chapter is who makes the key.

The key must be born where the retry is born, which is the client. A key the server generates on receipt is a different key on every attempt, so the retry looks like a new order and the whole mechanism is decoration. The client makes one UUID when the buyer presses Pay, attaches it to the first attempt, and attaches the same one to every retry of that attempt. The header is an expired IETF draft rather than a standard, the pattern is a decade old at Stripe, and the book treats it as a first-class part of the contract: required on every POST that creates or charges, documented next to the endpoint, and rejected with a 400 when absent.

POST /orders HTTP/1.1
Host: api.stagedoor.example
Authorization: Bearer eyJ...
Content-Type: application/json
Idempotency-Key: 7f3c2a90-8d1e-4b7a-9c55-2e0d1f6a8b41   # made once on the client, reused on every retry
X-Request-Id: 01J8Q6M2K7V3Z9X4B0N5R8T1WC                # new per attempt; ties the attempt's logs together

HTTP/1.1 201 Created
Location: /orders/9c1f6e2a-0b3d-4f8e-a7c2-5d9e1b4f3a60
X-Request-Id: 01J8Q6M2K7V3Z9X4B0N5R8T1WC                # echoed, so the buyer can quote it to support

The exchange is a checkout request and its response, reduced to the lines that matter. The credential travels in the Authorization header and nowhere else. The idempotency key is a UUID the client made once when the buyer pressed Pay, and the same value is sent on every retry of that press, which is how a repeat is recognized. The request id is different: it is new for every attempt, because its job is to find this attempt's log lines, not to deduplicate it, and the response echoes it so a buyer reporting a problem can quote the one string that leads straight to the trace.

Correlation: X-Request-Id and traceparent

A request id is accepted from the client if it sent one, generated at the edge if it did not, echoed in the response, and stamped on every log line the request produces. Chapter 13 makes the stamping automatic; this topic makes the rule: the id is created once and travels everywhere the request's work goes. When the handler queues a job, the id goes into the job's payload. When the worker runs it, the id is on the worker's log lines. When the service calls Payrail, the id is on the outbound request. One string, searched in the log store, returns the whole story of one buyer's checkout across two instances, a worker and a provider.

traceparent is the W3C form of the same idea with more structure: a trace id, a parent span id and flags, in one header that Postgres drivers, Redis clients and HTTP clients know how to propagate. Chapter 13 uses it to draw the checkout as a tree of spans across all four hops. The request id and the trace id coexist: the request id is the string a human quotes; the trace id is the string the tracing backend joins on.

The Headers That Lie If You Let Them

Host, X-Forwarded-For and X-Forwarded-Proto are set by whoever is in front of the service, and anyone can be in front of the service. From the load balancer they are the truth: the client's real IP, the scheme the client used, the name the client asked for. From anywhere else they are an attacker's choice of all three. A rate limiter keyed on X-Forwarded-For that trusts any source is defeated by one header; the scanner app's 40 scans a second become 40 different clients at one scan each, and Chapter 14's limiter never fires.

One setting decides it: trust the forwarded headers only when the connection arrives from the load balancer's address range, and strip or overwrite them otherwise. Uvicorn's --forwarded-allow-ips takes the list of trusted addresses and --proxy-headers switches the reading on; every framework has the equivalent. A service that runs with the list set to "everyone," which is the tempting default when the balancer's address is not known at deploy time, has handed the audit log an IP the attacker chose and the redirect-to-HTTPS logic a scheme the attacker chose.