Topic 36

The System Prompt

Concept

The airline's support bot will not talk about anything except bookings. Ask it for a restaurant recommendation in Lisbon and it apologizes and steers you back. It opens by using Tessa's first name. It has never once been rude, never once been chatty, and never once answered a question about a competitor.

The raw chat box does none of those things. It will discuss Lisbon restaurants at length, it does not know Tessa's name, and its tone is whatever the conversation drifts into. Same kind of model underneath. Completely different behaviour on the surface.

The reason is one message. In Chapter 7 Tessa read a request and found three roles in the message list, and the third one — system — was a line she had never typed, sitting at the top, addressed to the model rather than spoken by anyone. That is the system prompt: standing instructions the product puts into every single request, before the user's words, invisible to the user, read by the model first.

It is the single biggest reason a product behaves unlike the chat box, and this page is the introduction Chapter 7 promised.

The first message in the payload, and what is inside it
Identity and scope
who to be, and what to decline politely
Tone and format
be brief · at most three bullets · never an exclamation mark
Rules of its own
on top of the provider's: never promise a date, never name a competitor
Injected facts
today's date · your name · the passages a search just found
And beneath all of it, your turns
the only part of the list you ever see

What It Actually Is

Structurally, it is nothing exotic. It is text, in a message, with its role marked as system rather than user — sent ahead of everything Tessa typed, either as the first entry in the list or in a field of its own beside it, depending on the provider. Everything Tessa knows about how the model reads applies to it unchanged: it costs tokens, it takes up room in the window, and the model treats it as material in front of it rather than as a rule carved into anything.

What makes it different is position and purpose. It is the frame for the whole exchange, and it is written once, by the product, for every user, forever. The conversation changes with every turn; the system message does not.

Read that description again and something familiar should surface. Fixed text, written once, carefully, saying what to do and how to answer, sent again with every new input — that is Tessa's template from Chapter 5. The thing she built to stop Thursday-Tessa drifting from Monday-Tessa is the same object a product company builds so that ten thousand users get consistent behaviour. She has been writing system prompts by hand since Chapter 5 without knowing where they went. They go here.

There is a laminated card taped to the desk behind the counter at the hotel Waymark books most often. Tessa has seen it while waiting: greet by name if the booking shows one, never discuss other guests, offer the spa on Fridays and Saturdays, do not quote prices for next season. No guest is ever handed that card. Every guest experiences nothing but that card — the greeting, the discretion, the spa mentioned on exactly the right day. The receptionist is not a different person on Fridays. She is reading the same card she reads on Tuesdays.

What Products Put in It

Open enough of them and the contents fall into four groups.

Identity and scope. Who to be and what to stay away from: you are the booking assistant for an airline; only discuss reservations, refunds and schedules; if asked about anything else, say politely that it is outside what you can help with. This is the instruction that makes the bot decline to recommend restaurants — not a limit of the model, a fence drawn by the vendor.

Tone and format. Be brief. Never use exclamation marks. Answer in at most three bullets. Always end with an offer to connect the customer to a person. This is why so many AI features have a voice that is oddly consistent across every answer they ever give, while the raw chat box does not.

Rules of its own, on top of the provider's. Chapter 6 covered the limits the provider builds in, which apply to everyone using that model. A product adds a second layer for its own reasons: never promise a refund date, never speculate about a delay's cause, never give medical advice, never discuss a competitor by name. Two layers, drawn by two different companies, for two different sets of reasons — and only the outer one is written by anybody Waymark could ever phone.

Injected facts. This is the group most people never guess at, and it closes a loop the book opened in Chapter 1.

The product puts things the model could not otherwise know straight into the system message before sending: today's date, the user's name, which plan they are on, which office they belong to. The airline's bot knows Tessa's name because the program looked it up and typed it into the payload half a second before sending, not because anything remembered her.

The retrieved passages from Chapter 4 arrive the same way. A product that answers questions about your own documents runs a search, finds the three relevant sections, and puts them into the payload — very often into this same first message — so the model has them in front of it. Search plus context, done by a program, landing exactly where this page said the standing instructions live.

So the promise from Chapter 1 is finally paid in full. A product that greets you by name, remembers your preferences, and knows what you asked last month is not a model with a memory. It is a program with a database and a habit of typing what it finds into the top of every request. That is a real feature and a genuinely useful one — and it means the answer to "where do my details live?" is never "in the model". It is a company's database, subject to Chapter 6's questions, every one of which now has an address.

Instructions, Not Shackles

Here is the honest limit, and it matters more than the mechanism.

A system prompt leans the model hard. It does not bind it. Chapter 1 established that the model predicts text one piece at a time, with a measure of variation built in, and none of that stopped being true because a firm instruction rides at the top of the request. Unusual input, a long strange conversation, a question phrased in a way the vendor never imagined — and an answer can come out that the standing instructions clearly did not want.

This is not a rare exotic failure. It is the ordinary consequence of steering something that predicts rather than obeys, and every company shipping these features knows it.

Which is why products that genuinely must not misbehave do not rely on the instruction alone. They check the output outside the model: a rule that refuses to send any answer containing a price, a filter for anything that looks like a promise of a date, a second pass that verifies the reply mentions only real flight numbers. Rules, written in ordinary code, applied to what came back — Chapter 5's checking habit, industrialized and moved into the product.

Hold on to that sentence, because it is the seam where this book stops and the next one starts. Serious behaviour comes from instructions plus checks outside the model, and building those checks is the subject of the deep-dive course this one prepares you for.

Reading Behaviour Backwards

The practical use of this page is that product quirks stop being personality and start being decodable.

The assistant that refuses to leave its lane: a scope rule. The one that opens with your name: an injected fact. The one whose answers all sound like the same careful press officer: a tone rule. The one that suddenly will not discuss pricing after an update last Tuesday: somebody edited the standing instructions, and nothing about the model changed at all.

Tessa's vendor questions gain one more, and it is the sharpest of the set: what are the standing instructions, and can we see them or set them?

The answers vary and all of them are informative. Some products publish theirs. Some let a customer add rules of their own — "always use our tour names, never suggest a competitor's route" — which is a genuinely valuable feature and one Waymark should ask for by name. Some treat it as confidential, which is a defensible business position and also a plain statement that the behaviour of a tool in Waymark's workflow is not something Waymark gets to inspect. That is not a scandal. It is a fact to weigh, and weighing it requires knowing the question exists.

And there is a last, smaller consequence, which the next two pages pick up. Tessa's own review script needs standing instructions too — her template, sent as the system message with every one of 2,300 requests. She has just seen exactly where it goes.

Common Confusions
  • "The bot was trained to be a booking assistant." It is almost always the same general model with a standing instruction attached. The persona lives in the payload, is rewritten by an ordinary edit, and was never trained into anything.
  • "A system prompt is an unbreakable rule." It is a strong lean on a system that predicts rather than obeys, and unusual input can pull an answer off-script. That is why serious products also check the reply outside the model.
  • "Hidden instructions are sinister." They are how any product built on a model works at all, including the good ones. The fair question is not whether they exist but what they contain — and that question is a reasonable thing to ask a vendor.
  • "If a product greets me by name, it remembers me." A program looked your name up and wrote it into the request moments before sending. The memory is the product's database, which is where your details actually live.
Why It Matters
  • Product behaviour finally has a full explanation: model, plus standing instructions, plus gathered context. Tessa can name all three, point at where each one sits in the request, and ask a vendor about any of them.
  • It also relocates the answer to every privacy question. Names, preferences and history are not inside the model; they are in a company's records, being typed into requests — which is a thing that can be asked about, limited and audited.

Knowledge Check

What is a system prompt?

  • A configuration of the model that the provider applies per customer
  • Standing instructions the product puts at the top of each request
  • A summary the model writes of the conversation it has just had
  • The provider's built-in safety rules, delivered as text to the model

A product's assistant greets Tessa by name at the start of every session. What is happening?

  • The model kept her name from their previous conversation and used it
  • The provider retrained the model on this company's customer records
  • Her browser stored the name and the product read it back locally
  • The product looked her name up and put it into the request it sent

Why do careful products check answers outside the model as well as instructing it?

  • Because a standing instruction steers but cannot guarantee
  • Because checking the reply is cheaper than sending instructions
  • Because the provider requires every product to filter its output
  • Because checking the output keeps the answers shorter than the cap

A support bot politely refuses to discuss anything except bookings. Why?

  • The model behind it only ever learned material about travel bookings
  • The provider blocks any question that falls outside a product's field
  • The vendor's standing instructions tell it to stay on that one subject
  • The request format allows only questions about the product's own area

You got correct