What is few-shot prompting?

Few-shot prompting is placing a handful of already-solved input-output pairs before the real input. In practice that means two to eight. They don’t teach the model the task: they pin down the shape of the answer, the set of possible outputs, and the boundary between similar cases. No weights change, and every example costs tokens on every call, forever.

It’s the third mode of example-based prompting, after zero-shot and one-shot. “Shot” means example, not attempt, and “few” has no formal definition: any number above one and below the point where the context window starts to hurt.

The question that brings most people here isn’t what it is, though. It’s how many and which ones. Both have a partial answer in the literature and a complete answer only inside your own task.

How the block is assembled

The task I’ll use throughout: pull the delivery lead time, in days, out of a sentence written by a supplier.

The four examples weren’t picked at random. Each one settles something the instruction would settle badly:

• ships in 3 business days → 3 pins the output as a bare number, no unit, no sentence wrapped around it.
• lead time: one week → 7 shows that natural language becomes a number, and that the conversion lands on calendar days.
• ships immediately → 0 covers the edge case that would otherwise produce something creative.
• on request → null shows what to do when there is no lead time, which is the hardest instruction to write and the easiest one to show.

Writing that in prose would take three paragraphs and still leave the decimal, the business-days question and the empty case open.

What the examples actually teach

Here’s the most useful and least intuitive result in the area. A 2022 paper replaced the labels in the examples with random ones — deliberately pairing the wrong input with the wrong output — and measured the effect across 12 models, GPT-3 among them. Performance barely moved. The authors identified what carries the result instead: the set of possible labels, the distribution of the input text, and the overall format of the sequence¹.

Translated to the task above: what the model takes from those four pairs is “the output is a number or null”, “inputs look like commercial sentences from a supplier” and “the pattern is input, arrow, output”. The specific mapping from “one week” to 7 contributes less than anyone assumes.

That doesn’t licence writing wrong examples. It licences reallocating your attention: making sure the example’s format is byte-for-byte what you want back pays more than half an hour picking the most semantically representative case. And a wrong label left in a prompt is debt, because somebody will copy that block into another service six months from now.

Where the examples sit in the message structure

Two layouts produce almost the same text as far as the model is concerned. The difference is operational.

In the single block, instruction and examples live together in the system prompt and only the real input arrives as a user message. The advantage is the stable prefix: that’s precisely what prompt caching can reuse, and with fixed examples it drives the per-call cost down.

With alternating turns, each example becomes a pair of messages, one user carrying the input and one assistant carrying the output. The boundary between input and output is marked by the structure itself rather than by a separator you invented, and the model was trained on exactly this conversational shape.

In Python, the second layout is a loop over a list of tuples:

EXAMPLES = [
    ("ships in 3 business days", "3"),
    ("lead time: one week", "7"),
    ("ships immediately", "0"),
    ("on request", "null"),
]

def build(user_input: str) -> list[dict]:
    messages = []
    for question, answer in EXAMPLES:
        messages.append({"role": "user", "content": question})
        messages.append({"role": "assistant", "content": answer})
    messages.append({"role": "user", "content": user_input})
    return messages

The real win in those ten lines isn’t elegance. It’s that the examples become data: you can version them, swap them per task, and run your test set against two different lists without editing any prose.

How many examples

The GPT-3 paper is the most cited reference here and the most misread. It used 64 examples on the question answering tasks, and the zero-to-few gain varied wildly by dataset: on WebQuestions the 175B model went from 14.4% with no examples to 41.5% with 64; on TriviaQA it went from 64.3% to 71.2%². Same model, same number of examples, completely different effect sizes.

What survives is the shape of the curve, not the number. The first example fixes format. The next ones fix boundaries, and every boundary you want to pin costs at least one example on each side of it. After that the return falls off quickly while the cost stays linear.

That gives you a floor by construction rather than by guess. If the task has four categories and two of them get confused, the minimum is one example per category plus a second one for each side of the confusion, which is six. Add the empty case and it’s seven. Arriving at that number by construction is different from arriving at it because five felt thin.

Composition matters as much as count. A block of six examples where four land in the same category pushes the model that way before it reads anything, through the same label-bias mechanism described below. An even spread across the possible outputs costs nothing and avoids a failure that shows up as “the model prefers the most common category”.

There’s a regime far above all this that large windows made viable. A 2024 paper investigated hundreds to thousands of examples in context and found consistent gains across generative and discriminative tasks; notably, many-shot managed to override pretraining biases, which few-shot doesn’t, and came close to fine-tuning on part of the set. The same authors note that inference cost grows linearly in that regime³. It’s a real path and an expensive one: past a certain volume the honest comparison stops being with few-shot and becomes with training a small model.

Choosing and ordering the examples

Two decisions that look like details and aren’t.

Which examples. A 2021 paper showed that GPT-3’s results depend heavily on which examples go in, and that retrieving examples semantically close to the current input consistently beats random selection, with the largest gains on table-to-text generation and open-domain question answering⁴. In practice that means keeping a bank of solved cases and pulling the five nearest to each input. You gain relevance and lose the cached prefix, so the arithmetic changes.

In what order. A study from the same year found something more uncomfortable: with the same examples merely permuted, performance ranges from near state of the art to near random guessing. The effect shows up at every model size tested, isn’t fixed by swapping which examples you use, and a good permutation for one model doesn’t transfer to another. The authors proposed picking the order using entropy statistics over an artificial validation set generated by the model itself, which yielded a 13% relative improvement across eleven classification tasks⁵.

Part of this has a known cause. A 2021 paper showed the model arrives already preferring certain answers, particularly those common in pretraining and those appearing near the end of the prompt, and that calibrating the output against a content-free input recovers up to 30 absolute points of average accuracy⁶. If all your examples end in null, the model will develop a taste for null.

The operational conclusion is boring and correct: fix an order, version it with the prompt, and treat reordering as a code change that requires re-running the test set.

Few-shot or fine-tuning

The comparison comes up early and usually gets settled by taste. The useful criterion is where the example starts being expensive.

Few-shot pays per call. Fine-tuning pays once at training time and hands back a model that already behaves that way, with the prompt back down to zero-shot size. There is a volume beyond which the second bill is smaller, and it depends on three things: how many example tokens you carry, how many calls a month, and what it costs to keep one more trained artefact alive in your deployment process.

There’s a second criterion that isn’t financial. Fine-tuning fits when the behaviour won’t fit in examples that fit in the window: an entire writing style, a large domain vocabulary, a convention with hundreds of special cases. Few-shot fits when the behaviour is describable in half a dozen cases, which covers the overwhelming majority of extraction and classification work.

The 2024 many-shot paper found performance comparable to fine-tuning on part of its task set when using hundreds of in-context examples³. That doesn’t close the argument, but it moves the line: with a large window and prompt caching, the point where training pays off sits further out than it did in 2023.

Where it breaks

Tasks where examples change nothing. In the GPT-3 paper itself, Winograd came in at 88.3% zero-shot, 89.7% with one example and 88.6% with several². Not every task has a format or a boundary to pin, and on those you’re paying context for nothing.

Open-ended tasks. Ask for a piece of writing and hand over four examples, and you get variations on those four. The answer space shrank without anyone asking.

Reasoning models. Models trained to produce steps before the answer already have an internal thinking format. Examples with written-out reasoning compete with it and usually get in the way.

Examples that go stale. The block was written when there were three categories. There are five now, and the two new ones appear in no example. The model keeps returning the old three, confidently, and nobody reviews prompts.

Real data leaking into the prompt. Examples lifted from the production database tend to carry a real customer’s name, tax ID and order value. That sits in the system prompt, goes into your logs, and sometimes goes to a provider with a retention policy different from the one you promised.

What it costs

Order of magnitude. One example of this task, with the supplier’s full sentence, runs around 60 tokens. Five examples are 300 tokens on every call; twenty are 1,200.

At 2 million extractions a month, the difference between five and twenty examples is 1.8 billion input tokens a month. No argument about which model is cheaper makes up that gap, and it exists entirely because of a decision that almost always got made without measuring.

Two things bend that arithmetic. Prompt caching, when the examples are fixed and live in the prefix, charges a fraction for the reused span. And dynamic example selection, which improves relevance, throws the cache away precisely because the prefix changes on every request.

Where to start

1. Start with no examples and measure. Without the zero-shot baseline you can’t know what the examples bought you.
2. Add one example for the format. Write its output exactly as you want to receive it, including the absence of a code fence.
3. Add a pair for every boundary that’s being got wrong. One case on each side. A boundary without both sides isn’t pinned.
4. Include the empty case and the edge case. If null is a valid answer, one of the examples has to return null.
5. Keep the examples as data, not as prose. A list of tuples, a versioned file, the same thing you’d do with a test fixture.
6. Fix the order and run your 30 cases on every change, including when the only change is the order.

Step 3 is what separates few-shot from “I added more examples to be safe”. An example that doesn’t correspond to an observed error is 60 tokens per call with nothing on the other side of the trade.

Footnotes

1. Min et al. (2022) replaced correct labels with random ones across 12 models and measured only a small drop, pointing at the label space, the input distribution and the sequence format as what carries the result. ↩

2. Brown et al. (2020) report, with the 175B model and 64 examples, 14.4% zero-shot against 41.5% few-shot on WebQuestions and 64.3% against 71.2% on TriviaQA; on Winograd the three modes land at 88.3%, 89.7% and 88.6%. ↩ ↩²

3. Agarwal et al. (2024) measured the hundreds-to-thousands regime, with consistent gains, the ability to override pretraining biases, and inference cost growing linearly. ↩ ↩²

4. Liu et al. (2021) showed the result depends on which examples go in, and that retrieving examples semantically close to the input consistently beats random selection. ↩

5. Lu et al. (2021) found permutations of the same examples ranging from near state of the art to near random guessing, with no transfer between models, and proposed an entropy-based ordering worth a 13% relative improvement across eleven classification tasks. ↩

6. Zhao et al. (2021) describe the model’s preference for answers common in pretraining and for labels near the end of the prompt, and recover up to 30 absolute points by calibrating against a content-free input. ↩