What is fine-tuning?

Fine-tuning means continuing to train a finished model on your own examples, which changes its weights. It teaches behaviour: format, tone, what to do by default. It doesn’t teach new knowledge, and for that RAG is usually the answer. Before you train anything, check whether your case justifies it and whether a better-written prompt settles it.

Most requests that arrive as “we need a model trained on our data” die on those two checks. Fine-tuning comes later, once you know which behaviour you want and can show that behaviour in a few thousand examples.

What changes in the weights

The base model arrives at your run with an expensive pretraining behind it, during which it picked up English, code, the shape of an argument and most of the facts it can recite. Fine-tuning takes that starting point and runs a few more passes over a small set of input-output pairs. On each pair the model predicts the output, misses by some margin, and the weights get nudged toward whatever would have reduced that miss.

The gap in scale between the two stages explains almost everything else. Pretraining is trillions of tokens. A typical fine-tune is a few million, often far less: LIMA was tuned on top of a 65B model with a thousand hand-picked prompts and responses, and the result followed format and task well enough to hold its own against commercial assistants in the authors’ human evaluation¹. A thousand examples teach nobody medicine. They teach a model to answer the way whoever wrote those thousand examples would answer.

Keep the phrasing: fine-tuning rearranges access to what the model already has.

Behaviour goes in, knowledge doesn’t

This isn’t an architectural preference. It has been measured.

A 2024 paper built a controlled closed-book QA experiment and varied how many of the training examples carried facts the model didn’t already know². Two results came out of it. First, examples carrying a new fact are learned considerably more slowly than examples consistent with what the model already knew. Second, and worse: as those examples are eventually learned, the model’s tendency to hallucinate rises linearly.

So the process works in the least pleasant way available. You train, the new fact resists, you train harder, it lands, and the model starts inventing more confidently on questions that have nothing to do with your domain. The gain showing up on your test set hides a regression everywhere else.

The head-to-head against retrieval was run earlier. A 2023 study compared unsupervised fine-tuning with RAG on knowledge-intensive tasks and found retrieval consistently ahead, both on knowledge the model had already met during training and on entirely new knowledge³. Read the scope before generalising: the tuning under test was unsupervised, continuing training over raw documents, which happens to be exactly what most teams try first.

The approach that does work for knowledge is putting the document in the context window at question time. When a fact changes, you swap the document. With fine-tuning, you train again.

Three subjects under one word

Training methods. The base case is supervised fine-tuning: you supply pairs of input and correct output, and the model learns to produce the second given the first. It’s what most people mean when they say fine-tuning. The cheap variant trains a handful of extra parameters instead of the whole model, and the one everybody reaches for is LoRA.

LoRA freezes the original weights and injects small, low-rank matrices into each transformer layer. On GPT-3 175B the authors reported ten thousand times fewer trainable parameters and a third of the GPU memory, at quality equal to or better than full fine-tuning⁴. The side effect matters more than the memory saving: what the run produces is an adapter file of a few megabytes. You can serve ten different behaviours off one copy of the base model.

Alignment methods. When what you want isn’t a correct output but a preferable one, training changes shape, because there’s no single right answer to compare against. InstructGPT is the landmark: the authors started from demonstrations written by labellers, ran supervised fine-tuning, collected rankings between the model’s own outputs and trained further on those rankings with reinforcement learning. In their human evaluation, outputs from the 1.3B model aligned this way were preferred over those of GPT-3 175B, a model a hundred times larger⁵. It’s the cleanest demonstration that behaviour and raw capability are separate axes.

Decision and cost. The third part isn’t technical. It’s knowing when the case justifies the work, and what to compare it against.

The ladder of intervention

The order worth trying things in is fairly stable, and the test for climbing a step is always the same: the step below failed for a reason you can name.

1. A better prompt. Explicit instruction about format, role and decision criteria. It settles more than its reputation suggests, and it’s the only step you can repair in thirty seconds once you find out it was wrong.
2. Few-shot. Two to five solved examples inside the prompt itself. It teaches format through the same mechanism fine-tuning uses, without touching the weights or waiting on a training run.
3. RAG. Fetch the right document and put it in the context. This is the step for anything that counts as knowledge: internal policy, catalogue, ticket history, whatever changed last week.
4. Fine-tuning. When the behaviour you want is consistent, hard to put into words, and expensive to restate in every prompt. Or when call volume is high enough that a shorter prompt pays for the run.
5. Continued pretraining. Carrying on the pretraining over a large corpus in a domain or an under-represented language. Knowledge genuinely enters here, but the scale is billions of tokens, not thousands of examples. If you’re considering it for a folder of 400 PDFs, you’re on the wrong step.

The common mistake isn’t climbing the ladder. It’s jumping straight to the fourth step because fine-tuning sounds like the serious answer, then finding out three weeks in that the problem was the shape of the prompt.

Where fine-tuning fails

Five ways the project goes wrong, in rough order of frequency.

The dataset is the product, and it’s usually poor. If the outputs in your training set are inconsistently formatted, or were generated by another model and never reviewed, tuning learns the inconsistency as efficiently as it would have learned the right pattern. Five hundred examples reviewed one at a time beat fifty thousand scraped.

Regression on what already worked. Training on a narrow distribution degrades performance outside it, the effect the neural network literature calls catastrophic forgetting. The model gets excellent at classifying your tickets and worse at writing anything else. You only see it if your evaluation set includes tasks that aren’t yours.

The model freezes on training day. New pricing, renamed product, revised policy: none of it lands without another run. A retrieval system absorbs the change the moment somebody edits the document.

There is no evaluation. Without a held-out test set written before training, there’s no way to tell whether tuning helped. Reading ten answers and deciding it feels better is noise, and it’s how most projects declare victory.

The base model ages. You tuned on the best model available when the project started, and six months later a successor handles your case with no examples at all. Migrating the tune means redoing the work. Prompts and RAG travel between models almost for free; a set of tuned weights doesn’t travel.

What to do before training

1. Write the evaluation set first. Thirty to a hundred real cases with the output you’d accept, settled before any training. Without it none of the steps below can be judged.
2. Run the baseline. The base model with the best prompt you can write, then the same prompt with a few examples inside it.
3. Sort the failures that remain. If most read as “it stated something wrong about our company”, the problem is knowledge and the fix is retrieval. If they read as “right content, wrong format, wrong tone, wrong length”, fine-tuning is a candidate.
4. Build the smallest plausible set. A few hundred to a few thousand examples, reviewed by somebody who knows what the right answer is. Growing it later is easy; throwing away a large bad set is expensive.
5. Train with LoRA. A cheap run answers whether the signal is there at all. If an adapter doesn’t move the needle, full fine-tuning tends not to either.
6. Measure twice. On the set from step 1, and on a set of tasks outside your domain, to catch the regression the first one won’t show.

What it costs

Three bills, and compute is the smallest of them.

Compute. QLoRA set the practical floor back in 2023: tuning a 65B model on a single 48 GB GPU, quantising the model to 4 bits and training adapters on top of it. The best model in the family they released came out of 24 hours on one GPU⁶. At the sizes most teams actually use, 7B to 14B, a run takes a few hours. In July 2026, renting by the hour, that lands in the tens of dollars. Treat it as an order of magnitude rather than a quote, and remember that every wrong attempt costs the same again.

Data. This is where the money goes. A few thousand examples reviewed by somebody who knows what the right output looks like eat weeks of a person who is usually expensive and doesn’t have those weeks free. Budgeting the training without budgeting this is the most common estimating error in the whole subject.

Upkeep. The line item that rarely makes it onto the sheet. A tuned model is infrastructure: it needs versioning, somewhere to be served from, re-evaluation against every new base model, and somebody who can reproduce the run after the original author leaves. A prompt needs none of that.

Always compare against what you’d stop spending. If tuning cuts 2,000 tokens off every prompt and you make a million calls a month, the sum closes quickly. If you make a thousand calls a month, it never closes at all.

Footnotes

1. Zhou et al. (2023) tuned a 65B model on a thousand curated examples with no reinforcement learning, arguing from that result that almost all knowledge comes from pretraining. ↩

2. Gekhman et al. (2024) varied the fraction of examples carrying new facts in a closed-book QA setup and measured a linear rise in hallucination as those examples were learned. ↩

3. Ovadia et al. (2023) compared unsupervised fine-tuning against RAG on knowledge-intensive tasks, with retrieval consistently ahead. ↩

4. Hu et al. (2021) report, on GPT-3 175B, a 10,000-fold reduction in trainable parameters and a 3-fold reduction in GPU memory, with no added inference latency. ↩

5. Ouyang et al. (2022) showed that a 1.3B model aligned with human feedback produced outputs preferred over those of GPT-3 175B on their prompt distribution. ↩

6. Dettmers et al. (2023) fine-tuned a 65B model on one 48 GB GPU by quantising the model to 4 bits and training low-rank adapters on top of it. ↩