JSON: Giving a Program a Memory

A program that forgets everything when it ends has to start from scratch every time. JSON fixes that. It is a text format for exactly the shapes you learned in Chapter 6, lists and dictionaries, and with two functions a program can save its data to a file and load it back the next time it runs.

Think of a notebook left by the door for the next shift. Before leaving, the program writes down where it got to; the first thing it does when it starts again is read the notebook.

What JSON Looks Like

JSON, short for JavaScript Object Notation, looks almost exactly like Python's lists and dictionaries: square brackets for lists, curly braces for objects with keys, strings in double quotes, and plain numbers. The small differences are in the special words. JSON writes true, false and null in lower case, where Python has True, False and None. The json module translates both ways.

>>> import json
>>> json.dumps({"done": False, "note": None, "items": [1, 2]})
'{"done": false, "note": null, "items": [1, 2]}'

Saving and Loading

json.dump writes data to an open file, and json.load reads it back. With indent=2 the file is laid out over several lines, easy for a person to read in an editor. What comes back from load is ordinary lists and dictionaries, ready to use.

A Memory Between Runs

The program below keeps a to-do list in a file called todo.json. If the file exists, it loads the list; if not, it starts with an empty one. Then it asks for a new item, adds it, saves the whole list, and prints it. Run it twice and the second run remembers the first.

import json
from pathlib import Path

todo_file = Path("todo.json")

if todo_file.exists():
    with open(todo_file, encoding="utf-8") as f:
        todo = json.load(f)
else:
    todo = []

todo.append(input("New item: "))

with open(todo_file, "w", encoding="utf-8") as f:
    json.dump(todo, f, indent=2)

print("To do:", todo)

New item: Buy milk
To do: ['Buy milk']

New item: Call the dentist
To do: ['Buy milk', 'Call the dentist']

After those two runs, the file on disk is plain text that any editor, or any other program in any language, can read.

[
  "Buy milk",
  "Call the dentist"
]

What JSON Cannot Hold

JSON has strings, numbers, true, false, null, lists and objects, and nothing else. A Decimal is not among them, and neither is a date. Ask json to save one and it refuses with a TypeError: the object of type Decimal is not JSON serializable, meaning it cannot be turned into JSON text.

>>> import json
>>> from decimal import Decimal
>>> json.dumps({"amount": Decimal("54.30")})
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    json.dumps({"amount": Decimal("54.30")})
    ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  ...
TypeError: Object of type Decimal is not JSON serializable
when serializing dict item 'amount'

The full traceback runs through several of the json module's own files, left out above; as always, the lines to read are at the bottom. The very last line is a note Python adds to say where in your data the problem was: the dictionary item called amount. The fix is to store such values as text, which keeps them exact, and convert them back when loading: str(amount) on the way out, Decimal(text) on the way in. Dates work the same way, with their ISO text.

>>> import json
>>> from decimal import Decimal
>>> text = json.dumps({"amount": str(Decimal("54.30"))})
>>> text
'{"amount": "54.30"}'
>>> Decimal(json.loads(text)["amount"])
Decimal('54.30')