Chapter Five · How a Row Is Stored

How a Row Is Stored

Three facts about Cartwheel that the schema cannot explain: inventory holds 12,000 rows in 900 MB, products.attributes averages 3 KB and lives outside the table it belongs to, and SELECT id, price FROM products reads far more disk than two columns account for. Five topics open the file and show what is physically on the page.

5 topics

The first four chapters treated Postgres as something you send statements to. The schema was designed, the constraints were made real, the migrations shipped without an outage, and the SQL got sharp enough to replace three round trips with one. From here the book turns the machine around and works from the inside out, and this chapter is the first look at a page of actual bytes. Everything in the next four chapters — snapshots, vacuum, indexes, the planner's cost model — is measured in the units defined here.

Nadia has three measurements and no explanation for any of them. inventory has 12,000 rows of three narrow columns and occupies 900 MB, which is roughly 1,700 times what those rows need. products reports 18 MB in the obvious size query while its data directory shows hundreds. And a query naming two columns of products reads disk as though it had named all of them. Nothing in the DDL predicts any of this, because none of it is about the schema — it is about how the engine physically arranges a row and what it does to that arrangement when the row is written to twice.

The five topics build up in order: the 8 KB page and its three regions, then the tuple header whose four fields are the entire basis of multiversion concurrency, then TOAST for the values too big to sit in a row, then the two small side files that make inserts and index-only scans cheap, and finally the data directory itself. Two questions are deliberately left open at the end. Which of two versions of a row a given transaction may see is Chapter 6's subject, and how inventory came to be that size is Chapter 7's. Neither answer can be given honestly without this chapter's vocabulary.

Where a row actually lives, from the volume down to the bytes
Data directory
One directory per database, named for its OID
Relation
One file per relation, continuing in 1 GB segments
plus the free space map and visibility map forks
Page
8,192 bytes: header, line pointers, tuples
Tuple
24 bytes of header, then the column data
values too wide to fit move to a TOAST table

Topics in This Chapter

Topic 22
The 8 KB Page
A 24-byte header, line pointers growing forward, tuples growing backward, and free space in between — read off a real Cartwheel page with pageinspect. Why the size is fixed at compile time, why a tuple cannot span pages, and why the planner counts pages rather than rows.
Page Layout
Topic 23
Tuples, Headers, and Line Pointers
Twenty-three bytes of header rounded up to twenty-four, and four of its fields carrying the whole of MVCC. Watch xmin, xmax and ctid change under an UPDATE in another session, see what a DELETE physically does, and find out why a SELECT can produce write I/O.
Row Layout
Topic 24
TOAST — Where Big Values Go
Past roughly 2 KB a row is compressed and then relocated into a side table, leaving an 18-byte pointer behind. The four storage strategies, pglz against lz4, why SELECT * on a document column is an I/O decision, and where TOAST bloat hides from the usual size query.
Large Values
Topic 25
The Free Space Map and the Visibility Map
Two side files of a few hundred kilobytes decide whether an insert has to search and whether an index-only scan touches the heap. What each visibility bit enables, why Heap Fetches is the number to read, and why an append-only table still needs vacuum.
Relation Forks
Topic 26
Files on Disk — OIDs, relfilenode, Tablespaces
From a table name to a path in five seconds, and the difference between the number that identifies a relation and the number that names its file. Gigabyte segments, tablespaces as symlinks a restore must reproduce, checksums on by default since 18, and why deleting a file ends the cluster.
Data Directory