Chapter Thirteen · Replication and High Availability

Replication and High Availability

Cartwheel can now recover from a deleted table in a few hours. This chapter is the other failure mode, where the data is fine and the machine is not, and it ends with pg-primary deliberately taken out of service at 22:00, timed, with the losses counted.

5 topics

The previous chapter finished with a restore that somebody had actually performed, and with a number for how long it takes. Hours is the right answer to a migration that deleted 90,000 rows and the wrong answer to a power supply that stopped. Recovering from a backup rebuilds a database; what an outage needs is a second machine that already has one. The same write-ahead log serves both, which is why these two chapters sit next to each other and why the first sentence of this one is the last sentence of that one: a replica is not a backup.

Half the machinery is already running. pg-replica-a has been streaming from pg-primary for a year and serving the analytics dashboard, and it is the reason for a symptom the team has never named: every so often an order the customer can already see on their own page has not reached the dashboard. Nothing is broken there. The replica applies the write-ahead log a fraction of a second after the primary generates it, and a page read inside that fraction gets an honest answer to a slightly older question. Naming that gap, measuring it, and deciding per query how large it may be is most of the work in the first two topics.

From there the chapter adds pg-replica-b as the failover target, prices the durability dial, meaning what a synchronous commit costs per transaction and what naming a single standby costs the whole site, and then turns to logical replication, which ships rows instead of bytes and therefore crosses major versions in a way physical replication never will. The last topic is the rehearsal: writes paused at 22:00, pg-replica-b promoted, the application reconnected, and two numbers written down along with the three things that did not work.

Cartwheel's topology by the end of this chapter
One primary, two standbys, two different jobs
pg-primary · generates the write-ahead log
streaming for a yearpg-replica-a · analytics dashboard
added in this chapterpg-replica-b · failover target
Each replica applies the log after the primary generates it — a fraction of a second when nothing is in the way, and a read that lands inside that gap gets an honest answer to a slightly older question.

Topics in This Chapter

Topic 67
Streaming Replication
Building a standby with three commands, and why the operational weight is somewhere else entirely. Replication slots against wal_keep_size, hot standby and the conflicts it creates, cascading topologies, and the difference between a replica that is connected and a replica that is current.
Physical Replication
Topic 68
Replicas, Lag, and Read Scaling
Why replay is one process against sixteen cores of writers, and what that does to the order confirmation page. The three real fixes for read-your-writes, what hot_standby_feedback moves onto the primary, and the routing rules that make staleness a decision instead of a race.
Read Scaling
Topic 69
Synchronous Replication and the Durability Dial
The recovery point objective of an asynchronous failover is the replica's lag, which at 3,000 orders a minute is a number with customers attached. What each remote level waits for, why quorum syntax is the most important line in the config, and why the choice is per transaction.
Durability
Topic 70
Logical Replication
Publications, subscriptions, and the slot that appears on the publisher without being asked for. Replica identity and the update that errors on the source, the list of what never crosses the wire, row filters and column lists, and the conflicts that stop replication until a human acts.
Logical Decoding
Topic 71
Failover, Patroni, and Split Brain
Timelines, pg_rewind, and why the old primary cannot just reconnect. Fencing as the thing Postgres does not provide and homemade scripts always skip, Patroni and its lighter alternatives, and the 22:00 rehearsal with its recovery numbers and its three failures.
High Availability