What is the difference between an agent, a workflow and a chatbot?

The difference is who picks the next step. In a chatbot there is no next step: the model answers and the conversation moves on. In a workflow you pick, in code, before anything runs. In an agent the model picks on every turn, looking at what the previous action returned. The model can be exactly the same in all three.

This is not an academic taxonomy. It is the variable that sets your cost, your latency, how you test, and what kind of failure wakes you at three in the morning. Chaining calls in a fixed order is prompt chaining, and it stays a workflow no matter how many calls it has.

The honest position, before any detail: most cases that arrive asking for an agent are better served by a deterministic workflow. When it is worth leaving the prompt behind and building an agent has a narrower answer than the word suggests, and erring toward the agent is the expensive mistake.

The three of them on the same ticket

A customer writes: “I bought this on Tuesday and the order never arrived.” It’s worth watching what each design does with that sentence.

The chatbot reads the sentence and writes a reply. It will come out polite, in the right register, saying it’s sorry and suggesting the customer check the tracking number. It looked up no order, because it can’t. The text is the product.

The workflow runs three stages in the same order, always: classify the intent, look the order up by the sender’s email, build the reply from a template with the real status in it. If the intent isn’t one of the six it knows, it hands off to a person. Two or three model calls, a few seconds, and the path is identical on every run.

The agent gets the same text plus a catalogue of six tools and decides what to do. It may fetch the order, notice the carrier marked it delivered while the customer says otherwise, look up the redelivery policy, check whether the address has a history of problems, and only then reply. Or it may find on the first call that the payment was never approved and answer in a single turn. The variation is the point: that is what it offers and that is what it charges for.

Chatbot: fine until the third message

The chatbot is the oldest of the three designs and the easiest to underrate. For content already inside the model, or content that fits in the prompt, it is the cheapest answer available. A returns-policy FAQ answered by a model, with the policy text pasted into context, works well and costs one call.

The ceiling shows up as the conversation stretches. A 2025 study compared single-turn and multi-turn performance, simulating more than 200,000 conversations across the top open- and closed-weight models of the time. The average drop was 39% across six generation tasks, and the decomposition is the interesting part: the loss in raw aptitude is small, the rise in unreliability is large¹. The authors describe the pattern plainly: models make assumptions early, try to produce the final solution before they should, and then over-rely on that attempt. Once they take a wrong turn, they get lost and don’t come back.

That changes how you design a customer support chatbot. If the conversation will run past three or four turns, rebuild the state on each message from extracted fields instead of pushing the whole history forward. It is less elegant and it is more stable.

Rebuilding state means, concretely, keeping five or six fields: which order is under discussion, what the reported problem is, what has already been checked, what has already been promised. The next message’s prompt is assembled from those fields plus the last two turns of text. The full history stays in the database for audit and never goes to the model. A wrong assumption made on turn 2 then shows up as a wrong field somebody can see, instead of quietly poisoning everything after it.

Workflow: the design that usually wins

Workflow here means a sequence of stages you wrote, with model calls inside some of them. Classify, extract, summarise, decide by rule, call an API, format. The order is yours.

Four properties explain why it wins so often.

Predictable cost. Three calls with controlled context cost three calls. There is no sum of growing contexts pushing the bill superlinearly the way an agent does.

Latency in seconds. A user can wait for that behind a button.

Localised failure. When output is wrong you know which stage produced it, and the fix is a change to one specific prompt or one specific rule. With an agent, the same investigation starts by reading the whole trajectory.

Tests that work. Each stage has defined inputs and outputs, so each stage has a test set. At temperature zero the behaviour is close to reproducible.

A frequent confusion: workflow does not mean less model. It can have five calls, with different prompts and different temperatures, and remain a workflow, because the edges between stages live in your code. What defines it isn’t how much intelligence fits inside each box, it’s who draws the arrows between them.

The case that causes the most doubt is routing. A stage where the model picks which of six intents applies, and the code then calls a different path for each, looks like the model deciding, and it is. But it is one decision, over a closed set you wrote. What makes something an agent is the choice repeating based on the result of the previous choice. A router chooses once and execution runs on rails.

The real limitation is just as clear: a workflow cannot handle what you didn’t anticipate. If the customer’s intent isn’t on the list, it hands off. That is usually acceptable and sometimes exactly what you want.

Agent: when the loop pays

The pattern nearly every agent follows came out of a 2022 paper that proposed interleaving reasoning and action instead of treating them as separate stages: the model writes what it is thinking, picks an action, reads the result, repeats². That cycle is what gives it the ability to investigate, and it is what charges you.

The loop pays when the space of paths is large and you cannot enumerate it. Investigating why a batch of charges failed. Walking a codebase nobody knows, hunting the source of a behaviour. Researching until an answer turns up, when it could be in five different places.

The loop charges in consistency. τ-bench, from 2024, built a setting close to real support work: an agent with domain tools, a policy document, a simulated user, scored on the final database state. The authors measured pass^k, the rate of succeeding on all k attempts at the same task. State-of-the-art function calling agents of the time stayed under 50% success, and in the retail domain pass^8 fell below 25%³. Eight runs of the same task, and in three out of four cases at least one came out wrong.

The numbers age quickly and the shape does not. Wherever the same input must produce the same output, the loop works against you.

The hybrid that covers most of it

The choice is rarely between three sealed boxes. The arrangement that works most often is a deterministic workflow with a single agentic step inside it, at the stage where the variation actually lives.

Back to the support ticket: classify, fetch the order and format the reply stay fixed stages. Only the middle one, “decide what to do when tracking says delivered and the customer says otherwise”, becomes a small loop with three tools and a cap of five steps. You pay for the loop in one stage out of six, and the other five stay deterministic.

A 2024 survey of agent architectures describes the same move one level up, comparing single-agent and multi-agent designs: the clearer the division of roles and the more explicit the coordination point, the more reliable the system⁴. It is a survey, with the limits of the genre, but the observation matches what shows up in practice. Structure around the loop buys more than autonomy does.

What each one costs

Order-of-magnitude numbers for the same support ticket, with assumptions on the table.

The chatbot makes one call. A 600-token instruction, a 200-token message, a 200-token reply. Around 1,000 tokens total.

The workflow makes three calls with lean context in each: 400 tokens to classify, 900 to decide, 1,200 to draft. About 2,500 tokens, and the composition is known in advance.

The agent carries a 1,500-token instruction and a 1,200-token catalogue on every iteration. Over nine turns, with each tool result adding roughly 700 tokens to the history, the sum passes 50,000 input tokens. That is twenty times the workflow for the same ticket, and it is what shows up on the invoice.

Latency tracks the same shape. One call takes seconds, three calls take a few seconds, nine iterations with tools in between take one to two minutes. If your product has a user staring at a screen, that difference decides on its own.

Where each one breaks

The chatbot invents confidently. It has no access to the order and answers anyway, because answering is what it does. Without a tool or retrieved context, the plausible answer is the only one available.

The workflow breaks outside the list. The seventh intent, the one nobody anticipated, falls through to the default path. If the default is handing off to a person, fine. If the default is answering anyway, you have a silent problem.

The agent varies. Two runs, two paths, two answers. Much of ordinary software testing strategy doesn’t apply, and measuring means running the same task several times rather than once.

The agent hides its cost. It works in the demo at four iterations and reaches thirty in production with a catalogue that grew. The curve is superlinear and the surprise arrives with the bill.

All three break when the choice was made by fashion. The costliest error in this decision isn’t picking wrong between designs: it’s picking the agent without having tried the workflow, and finding out six weeks later that the workflow handled it.

How to decide

1. Does a text answer finish the task? If nothing has to change in any system, it’s a chatbot. Adding tools to that solves a problem you don’t have.
2. Can you write out the whole sequence of steps? If you can, it’s a workflow. The test is literal: open an editor and write it. If it comes out, you’re done.
3. Does the variation fit inside a single step? If so, it’s a workflow with one agentic step inside. You keep the other stages deterministic.
4. Does the loop fit your cost and latency? Is twenty times the price and a two-minute wait acceptable in this product? If not, the answer is workflow even when the task looks like it wants an agent.

One note on question 2, where most of the mistakes happen: it asks about the sequence, not about difficulty. A task can be hard, require judgement, and still have a fixed order. Hard does not imply agentic.

And run the same input eight times before deciding the agent works. One successful run is the single most misleading data point in this entire decision.

Footnotes

1. Laban et al. (2025) simulated more than 200,000 conversations and measured an average 39% drop across six generation tasks when moving from single-turn to multi-turn, almost all of it from increased unreliability rather than lost aptitude. ↩

2. Yao et al. (2022) proposed interleaving reasoning and action in one sequence, which is the base structure of the agent loop still in use. ↩

3. Yao et al. (2024) measured consistency with pass^k: state-of-the-art function calling agents of the time stayed under 50% success and under 25% pass^8 in the retail domain. ↩

4. Masterman et al. (2024) compared single-agent and multi-agent architectures and observed that well-defined roles and explicit coordination points contribute most to reliability. ↩