What is an agentic prompt?

An agentic prompt is the instruction written for a model that will act several times in a row instead of answering once. It carries four sections a single-task prompt does not have: the tool catalogue, the stopping criterion, what to do when a call fails, and the step budget. Without those four the loop still runs. It just doesn’t end.

The difference comes from the format. Inside an AI agent your text is not read once: it goes back to the model on every iteration, alongside a history that grows with each turn. Writing for that is closer to writing the policy for a job than to writing a request, which is why almost every agentic prompt lives in the system prompt rather than in the user message.

Two consequences show up early. Ambiguity does not average out, it compounds: a sentence that can be read two ways will be read both ways across thirty iterations. And a good share of what you want to write does not belong in the prompt at all. A rule that must always hold belongs in a guardrail in code; the production system prompt keeps what is guidance, not what is law.

The four sections a normal prompt doesn’t have

The examples below come from an agent that investigates data pipeline failures, with three tools: list_runs, read_log and reprocess_partition.

Tool policy, not a tool list

You declare each tool’s signature through the API, and the model receives that already. What is missing, and what goes in the prompt, is the usage policy: which one to pick when two would work, which one never to call before another, which one needs a human to confirm.

list_runs and read_log answer similar questions at wildly different costs. If the prompt doesn’t say the listing comes first, the model pulls a 40 MB log only to discover the job never started. Write the policy in the negative wherever you can. “Never call reprocess_partition without having read the log of the failed run” survives thirty iterations. “Use the tools sensibly” doesn’t survive one.

A definition of done your own code can check

A normal prompt is finished when the answer comes out. An agent needs an explicit definition of finished, and it can rarely be “the model thought it was done”.

“Stop once you have the failing partition and its error message, or after reading three logs without finding any error” is checkable. “Stop when it’s fixed” is not. The practical difference is who does the judging: in the first sentence the judgement sits with your code, which can verify that both fields exist; in the second it sits with the model.

And the model is weak at exactly that. A 2023 paper tested intrinsic self-correction, with no external feedback, and found the opposite of what people assumed: on reasoning tasks, performance did not improve after revision and in some setups got worse, because the model has no way to know it was wrong¹. The result is about reasoning rather than agents, but the implication carries: self assessment works as one signal among several and fails as the only one.

What to do when a tool fails

This is the section almost nobody writes and the one that changes trajectories the most. Without it, the default behaviour after an error is to repeat the same call, and then repeat it again.

Write by class of error, not by case. Bad argument: fix the argument, try once. Transient network failure: retry once, then take a different route. Permission denied: stop and report, never route around it. Empty result that is not an error (the list came back with nothing): that is an answer, treat it as one. Four lines cover most of what happens in production and save dozens of iterations spent on stubbornness.

The budget, written where the model sees it

The hard step cap lives in your code, and that is what actually interrupts. But writing the budget into the prompt as well changes behaviour before the cap is reached: “you have at most 12 tool calls for this investigation” makes the model prioritise early instead of sweeping.

Say what to do near the end, too. With no instruction, an agent that runs out of budget simply stops mid-thought and what you get is a truncated context. With “at 10 calls, write the report with whatever you have and mark it incomplete”, you get a partial result somebody can act on.

The prompt is read forty times, not once

Everything you write in the system prompt is resent on every iteration. That has three effects a single-task prompt never sees.

The first is cost. A 300-token section you added “since it costs nothing” costs 300 tokens times the number of iterations. Over twenty steps that is 6,000 input tokens, and prompt caching cuts the price of that repetition without removing it. It still occupies window.

The second is contradiction. In a one-shot call, two slightly incompatible instructions produce a slightly odd answer and nobody notices. In an agent, the same incompatibility is resolved again on every turn, sometimes one way and sometimes the other, and the result shows up as erratic behaviour that gets blamed on the model.

The third is position. On iteration 1 your instruction is nearly the whole context. On iteration 18 it is a small fraction of it, surrounded by tool results that arrived later. Models do not read the window evenly, and what sits in the middle gets less attention than what sits at either end. A constraint that must hold to the end either gets reinjected near the last message, or becomes validation inside the tool, where it depends on no reading at all.

Domain rules are where it slips

Writing “follow the refund policy” and testing it once successfully produces a feeling of resolved that repetition does not confirm.

τ-bench, published in 2024, was built to measure exactly this: an agent with domain tools and a policy document, talking to a simulated user, scored by comparing the final database state against the expected one. The authors proposed a consistency metric, pass^k, the rate of succeeding on all k attempts at the same task. State-of-the-art function calling agents of the time solved under 50% of tasks, and in the retail domain pass^8 fell below 25%². Eight runs of the same task, and in three quarters of cases at least one came out wrong.

The numbers age. The shape of the problem doesn’t: policy instructions are followed probabilistically, and one successful run is not evidence that the rule is being respected. If the rule genuinely matters, test it k times, and anything that cannot fail even once should not depend on reading.

Instructions don’t fix the interface

A common diagnostic mistake is blaming the prompt for what belongs to the tool.

SWE-agent, from 2024, attacked the problem from the other side: instead of improving the instruction, the authors designed the interface between the agent and the computer, treating the model as a new category of user with needs of its own. Editing commands with immediate syntax-error feedback, file viewing in size-controlled windows, return messages written to be read by a model. With that interface the system reached 12.5% pass@1 on SWE-bench, far above what non-interactive models had managed until then³.

The lesson that matters here is the division of labour. If read_log returns 3,000 raw lines, no amount of prompt wording saves the trajectory: the context fills up by iteration 6. Trim the output in the tool, return the 20 lines around the error, and the prompt gets shorter because it no longer has to compensate for anything.

The same logic covers the action format. A 2024 paper showed that letting the agent emit executable Python, instead of JSON in a fixed shape, gives a more expressive action space, allows composing several tools in one go and revising the previous action when a new observation arrives, with up to 20 points more success rate in their comparisons⁴. That is a harness decision, and it changes what the prompt has to explain.

Where it breaks

“Keep going until it’s fixed” is the most expensive instruction there is. It combines two bad things: a condition only the model can judge, and no ceiling. Facing an error that will never clear, a server that is down, a permission it does not hold, the agent keeps trying. The cycle in the figure is literal, and it ends when somebody notices the bill.

Too long a prompt on the first version. Writing forty rules before the first run is tempting. They will fight each other, you will not know which one is being obeyed, and each one costs context on every iteration. Start with the minimum and add a rule when a real trajectory shows the gap.

A rule written in the wrong place. “Never reprocess a partition from the previous month” in the prompt is a strong suggestion. The same rule inside reprocess_partition, refusing with a clear message, is a guarantee. The gap shows up on iteration 30, when the sentence is far away.

An example that eats the whole window. Pasting a twenty-step trajectory as a demonstration feels good and costs a lot: it is resent every time and pushes the real history into the middle of the context. A fragment of one hard call buys more than a complete run.

Tuning the prompt without reading the trajectory. Most failures blamed on wording show up in the log as a truncated tool result or an error message that said nothing. Rewriting the prompt without looking at that is guesswork with an extra step.

How to write it, in order

1. Write the definition of done before writing the task. If you cannot describe the end state in terms your code can check, the agent won’t manage it either.
2. List what it may never do. That list is short, you write it in ten minutes, and it is the one part of the prompt that should have a mirror in code.
3. Write the tool policy in two lines per tool. When to use it and when not to. If you need more than that, the problem is the tool.
4. Cover the four classes of error. Invalid argument, transient failure, permission denied, empty result. One line each.
5. Set the budget and say what to do on reaching it. Twelve calls is a reasonable start and you tune it with data.
6. Run the same task eight times before changing anything. One good run is not evidence. Two divergent trajectories on the same task show you which sentence is being read two ways.

Step 6 is what separates people who improve an agentic prompt from people who rewrite the whole thing every week.

What one more section costs

Assume an agent with a 1,500-token system prompt, a five-tool catalogue at 1,200 tokens, and a task that resolves in fifteen iterations. Those 2,700 fixed tokens go out on all fifteen calls: 40,500 input tokens of instruction alone, before any history.

Adding a 400-token section takes the fixed part to 3,100, and the fifteen steps to 46,500. That is 6,000 extra tokens for a section you might use once. Prompt caching drops the price of that part considerably, since the prefix repeats in full between iterations, but the space it occupies in the window is unchanged, and that space is what decides which iteration overflows.

The arithmetic that usually pays off most is the catalogue. Five tools described in 1,200 tokens cost 18,000 across fifteen steps; forty tools at the same density cost roughly 144,000, and the thirty-five nobody will call cost the same as the rest. Cutting the catalogue is nearly always the cheapest and most effective edit available in an agentic prompt.

Footnotes

1. Huang et al. (2023) found that, without external feedback, revising its own answer does not improve reasoning and in some setups degrades the result. ↩

2. Yao et al. (2024) measured consistency with a pass^k metric: state-of-the-art function calling agents of the time stayed under 50% success and under 25% pass^8 in the retail domain while following written policies. ↩

3. Yang et al. (2024) showed that the design of the interface between agent and computer moves performance directly, reaching 12.5% pass@1 on SWE-bench with commands and return messages built to be read by a model. ↩

4. Wang et al. (2024) compared executable code actions against fixed-format JSON actions and measured up to 20 points more success rate, with the added ability to compose tools and revise an action after a new observation. ↩