What is example-based prompting?

Example-based prompting means showing the model what you want instead of describing it. Rather than spending three sentences explaining that the answer should be JSON with no currency symbol on the amount, you show one solved case and the model copies the pattern. The best-known variants are one-shot prompting, with a single example, and the versions with none or several. It’s one of the core techniques in prompt engineering.

The name changes with the quantity, but the mechanism is the same.

“Shot” here means example, not attempt. It’s a common mix-up: zero-shot isn’t “asking once without pushing”, it’s “giving no solved examples”. The term comes from the low-data learning literature and caught on after the GPT-3 paper used it in its title¹.

What an example conveys that an instruction doesn’t

The practical difference shows up in things that are awkward to describe in words and trivial to demonstrate.

Take classifying a support ticket. Without an example you’d write something like:

Classify this ticket into a category. Use a single lowercase word, no punctuation. The possible categories are billing, technical, cancellation and other. If you’re unsure, use other.

With one example:

Classify the ticket.

Ticket: “I was charged twice this month” Category: billing

Ticket: “…”

The second is shorter and settles something the first didn’t quite settle. Your instruction said “cancellation”, but your database uses cancel. The example communicates that without an extra sentence.

There’s an experimental result that helps explain what’s going on. A 2022 paper tested what happens when you deliberately scramble the labels in your examples, pairing wrong inputs with wrong outputs. Performance drops far less than you’d expect². The reasonable reading is that the example mainly tells the model what the format is and what the set of possible answers is, and less about the mapping between them.

That isn’t licence to ship a wrong example. It’s a hint about where attention pays off: making sure the example’s format is exactly what you want back buys more than half an hour picking the most semantically representative case.

How many examples

The gain isn’t linear, and treating it as if it were is the most common mistake.

Zero to one is the biggest jump, and it settles format. If your complaint about the model is “it answers correctly but in the wrong shape”, one example is enough.

One to five settles boundaries. One example shows one side of the line between billing and finance; it doesn’t show where the line falls. For that you need cases from both sides, and you’ve left one-shot behind.

Five to twenty usually returns little, and the cost is linear: those tokens ride along on every call, forever. It only pays off when the task has many categories or edge cases that resist description.

There’s a band above that, with hundreds of examples, which became viable once context windows grew. The gain reappears on some tasks³, but the per-call cost gets high enough that it’s worth comparing against training a small model.

Where examples hurt

Three situations where adding an example makes things worse.

Open-ended tasks. If you ask for creative text and supply an example, the model tends to produce variations of that example. You narrowed the answer space without meaning to.

A single edge case as the example. If the example is exceptional, the model learns that everything is exceptional. Examples should be median.

Reasoning models. Models trained to reason before answering already do the work internally. Giving examples with the reasoning written out usually hurts, because it imposes a thinking format that isn’t the one the model learned.

Where to put the examples

If the examples are fixed — always the same, regardless of the input — they belong in the system prompt. That puts them in the request’s stable prefix, which is what prompt caching can reuse, and you start paying a fraction of the price for them.

If the examples are selected based on the input, by similarity to the case at hand, they have to go in the user message. You lose the cache but gain relevance. That’s worth it when the domain is broad and no fixed set of five examples covers it well.

Where to start

1. Write the no-example version first. It’s cheaper and sometimes enough.
2. If the format comes out wrong, add one example with the output exactly as you want to receive it.
3. If categories get confused, add cases from both sides of the boundary that’s being confused.
4. Measure. Run the same 20 real cases through all three versions. The difference shows up in minutes and beats any general rule, including the ones in this article.

Step 4 is the one almost nobody does, and the only one that answers the question for your specific case.

Footnotes

1. The use of “shot” in this sense, and the demonstration that large models pick up a task from in-context examples, come from the paper that introduced GPT-3. ↩

2. Min et al. (2022) showed that replacing correct labels with random ones degrades performance far less than expected, suggesting demonstrations mainly specify format and label space. ↩

3. Agarwal et al. (2024) measured behaviour with hundreds to thousands of examples, made viable by large context windows. ↩