CODEAI101 · chapter 2 of 7 · free to read

Prompting and Context

This is the whole chapter, not a sample of it. Nothing is shortened for the showcase and nothing here is recorded — no account, no cookie, no reading history. Every chapter of this ladder was signed by a named person before it was served; this one by Capt Anil Sharma.

Yesterday you learned that a language model predicts the next word and nothing else. Today you learn the only lever you have on that prediction: what the model can see at the moment it makes it.

WHAT A PROMPT ACTUALLY IS

A prompt is every word you put in front of the AI before asking it to produce something. That is the whole definition, and it is wider than most people assume.

Type "summarise this email in three sentences" and paste an email underneath, and the prompt is the instruction plus the email. The model reads the entire thing as one continuous block of words and starts predicting what comes next. It does not receive an instruction and a document as two separate objects. It receives one long input and begins generating.

This is unintuitive, because we think of conversation as turn-taking — I say something, you say something, each of us keeping our own memory. The model does not work that way. Every message you have sent in the chat, every reply it has given, every attachment, every instruction the product gave it behind your back, is concatenated into one long text, every single time, before the next prediction begins.

A PROMPT IS EVERYTHING THE AI SEES AT THE MOMENT OF GENERATING — EVERY PREVIOUS MESSAGE, EVERY ATTACHMENT, EVERY SYSTEM INSTRUCTION, ALL OF IT AS ONE CONTINUOUS TEXT.

Recall the courier from Day 2 of Code101, because the mechanics matter here. When you tap send, your message travels to the model's server. The server assembles the full prompt by gluing together the system instruction, the conversation history and your new message, and feeds that whole text to the model. The model reads it, predicts one token, then the next, until it decides it is finished. The server sends those tokens back to you.

Which leads to the implication worth carrying: in the moment of generation there is no difference between the AI's training and the AI's instructions. There is only the text in front of it. Trained-in patterns plus prompt equals output.

THE CONTEXT WINDOW

The context window is the limit on how much text the model can read at once, and the sizes differ enormously. A small model might hold 4,000 tokens — roughly 3,000 words, enough for a chat and a short document. A modern frontier model such as Claude, ChatGPT or Gemini holds around 200,000 tokens, roughly 150,000 words, or a four-hundred-page book. Specialised long-context models reach a million tokens, which is a small library.

If the prompt exceeds the window, the model does not remember less. It physically cannot see the part that does not fit. The product must either truncate — drop the oldest messages — or summarise, compressing earlier content into something shorter, before sending. Either way something is lost.

This is what is happening when a long conversation starts to feel forgetful. The product is silently dropping older messages to stay within the window. The model is not tired. It cannot see what it could see ten minutes ago.

Three consequences are worth holding.

CONTEXT LENGTH MATTERS. A model with 200,000 tokens is doing genuinely different work from one with 4,000. They are not the same product at two prices; they are different shapes of capability.

ORDER MATTERS. What sits near the end of the prompt, closest to where generation begins, tends to carry more weight than what sits at the start. Put your important instruction at the top of a long document and you may simply not see it obeyed. Put the important thing close to the question.

POSITION MATTERS. Information in the middle of a very long context is attended to less than information at either end. Engineers call it the lost-in-the-middle effect. For very long prompts, repeat the load-bearing instruction near the end.

The context window is not memory in any human sense. It is a long page the model is currently reading, and when the page is full something has to come off it before anything else goes on.

THE THREE VOICES

In most AI products the prompt is structured as three kinds of message, even though all three arrive at the model as one stream.

SYSTEM MESSAGE — set by the product and usually invisible to you. "You are a helpful customer service assistant. Be polite. Do not discuss these topics. Format your responses this way." It defines the persona and behaviour for the whole conversation, and you normally cannot see or edit it.

USER MESSAGES — what you type. The questions and the inputs.

ASSISTANT MESSAGES — what the AI produced on previous turns. These are fed back in as part of the next prompt, which is how it appears to remember what it said.

The three are concatenated with markers the model was trained to recognise, so what it actually reads runs something like: system, you are a customer service assistant; user, how do I cancel my subscription; assistant, to cancel you can; user, but the link does not work; assistant — and there it begins generating.

Two things follow. The first is that the system message is doing a great deal of invisible work: much of what feels like personality difference between ChatGPT, Claude and Gemini is the system message each company has tuned, and the same model behaves very differently under different ones. Where a product offers custom instructions, that is your slice of it.

The second is that you can give the model a role. Asking it to act as a careful first-grade teacher explaining something to a child is not magic. It shifts the kind of text the model is now most likely to produce, because the training data holds many examples of that role and naming it nudges the model toward them.

NO CONTEXT AGAINST GOOD CONTEXT

The single biggest skill in prompting is giving the model enough context to actually answer. Vague questions produce vague answers, and the demonstration is worth more than the principle.

Ask "should I use a database or a file for this?" and the model has no idea what "this" is. You will get a competent generic essay on the trade-offs between databases and files. Some of it will be true. Most of it will be useless to you.

Now ask the same question this way: I am building a personal grocery-tracker app for my own phone, for offline use; I expect five to ten entries a day, around two thousand a year; I will run reports like total spent this month and amount by category; I am building solo, on a budget; should I store the data in a JSON file or a SQLite database, and what are the trade-offs for my situation?

Now it can answer specifically. It will probably tell you SQLite is worth the small extra setup for the queries you described, and say why at your scale — or tell you JSON is fine and name the point at which it would stop being fine.

The difference is not the model. It is what the model can see. The first prompt withholds the substrate; the second provides it, and that is the substrate-shaped use of AI from Day 1. The same holds everywhere: "write me marketing copy" gives boilerplate, while "write three subject-line options for an email to small-business owners in Bangalore, offering thirty percent off accounting software, in a friendly Hindi-English mix" gives you something you can use. Same model, different prompt, different output.

THE AI IS ONLY AS GOOD AS THE PROMPT IS SPECIFIC.

FEW-SHOT — SHOWING BEATS TELLING

There is one more move worth learning today: showing the model examples of the shape you want.

Describe what you want in words — write it like a professional email — and the model has to interpret your description. Show it three professional emails and ask for a fourth in the same style, and it has a concrete template to match. The technique is called few-shot prompting: a few example pairs of input and output placed before the real question.

Say you want key facts pulled out of short descriptions. You give three worked pairs first. Coffee at the airport cafe, ₹240, becomes category Food, item Coffee, location Airport, amount 240. Auto rickshaw to office, ₹85, becomes category Transport, item Auto, location Office, amount 85. Movie tickets for two, ₹600, becomes category Entertainment, item Movie, location blank, amount 600. Then you give the real one — birthday dinner with parents at the new restaurant in Indiranagar, ₹3,200 — and leave the output empty.

It will produce something matching the shape: category Food, item Dinner, location Indiranagar, amount 3200. The examples did what a long instruction could not. They showed the exact target unambiguously.

Few-shot is among the most powerful and least used tools in prompting. Whenever you catch yourself describing a format in words while the model keeps drifting away from it, stop describing and show two or three examples instead. The model is a pattern matcher. Give it the pattern.

THREE MORE LEVERS

CHAIN-OF-THOUGHT — asking the model to think step by step before answering. The output gets longer and the answer gets more accurate. For anything with multi-step logic, it is the lowest-effort accuracy upgrade available to you.

STRUCTURED OUTPUT — asking for the answer in a specific format: JSON, a table, a list under named headings. Useful whenever the output will be read by another program rather than a person, and modern APIs support it directly, so the model can be forced to return only valid JSON.

ROLE AND PERSONA — act as a particular kind of person, which selects a distribution rather than conjuring an expert. Not magic; statistics.

Each is a one-line addition that materially changes what comes out. None of them is a trick. They are levers found by researchers and practitioners trying many variations and keeping what worked — and every one of them works by the same mechanism this chapter began with: changing what the model can see at the moment it predicts.

Read the rest of the ladder →

The rest of this ladder is for enrolled learners. This chapter is the whole of what is open — it is not a teaser with the ending removed.