Instruction Tuning: How a Text Predictor Becomes an Assistant

A freshly pretrained model is a text continuer. Ask it "how do I reverse a list in Python" and a plausible continuation is three more questions about Python, because in its training corpus questions frequently appear in lists of questions.

It is not confused and it is not broken. It is doing exactly what it was trained to do. Instruction tuning is the stage that changes what it is trying to do.

The mechanism is ordinary supervised learning

Take a dataset of instruction and response pairs. Train the model on them with the same next-token objective used in pretraining, but compute the loss only over the response, not the instruction.

That detail is the whole trick. The model is not learning to predict instructions; it is learning what a good response to an instruction looks like. Repeated across a varied dataset, it generalises the pattern: text in this position is a request, text in that position is an answer, and answering is what I do here.

Compared with pretraining this stage is tiny — a fraction of the compute and vastly less data. It adds almost no knowledge. It reorganises access to knowledge that is already there. Pretraining versus post-training covers the division of labour.

Where the format conventions come from

Instruction tuning is also where the chat template is established. Somewhere in the training data, turns are delimited by special tokens marking system, user and assistant roles, and the model learns to expect that structure.

This is why sending a raw string to a chat model produces worse results than sending a properly formatted message list. The API usually applies the template for you, but if you are running weights locally and hand-assembling prompts, using the wrong template degrades quality in ways that look like the model being weak.

It is also where the system prompt gets its influence. The model was trained to treat that position as authoritative context, which is a learned convention rather than an architectural guarantee. System prompts explained covers what that implies for reliability.

What the dataset determines

Nearly every visible habit of a model traces back to what its instruction data looked like. Response length is the clearest example: if the training responses were long and thorough, the model is verbose, and asking for brevity fights a learned default rather than setting a parameter.

Format discipline is the same. A model trained on many examples that produce clean JSON will emit clean JSON. A model trained mostly on conversational prose will wrap JSON in an explanatory sentence and break your parser. Structured outputs and JSON mode covers constraining that at inference time.

Coverage matters too. A model whose instruction data included many multi-step tool-use traces handles tool calling far better than one that saw few, even at identical size. This is a large part of why models with similar parameter counts differ so much at agentic work. Tool calling explained covers the interface.

Where the data comes from

Early instruction datasets were written by humans, which is accurate and slow. Modern practice leans heavily on model-generated data: a strong model produces candidate responses, which are filtered, edited or verified before training.

This scales enormously but propagates the generating model's habits, including its errors. If the generator hedges excessively or formats a certain way, the trained model inherits it. Synthetic training data covers the trade-offs and the filtering that makes it work.

For verifiable domains the filtering can be mechanical. A code instruction whose response compiles and passes tests is kept; one that fails is discarded. That is a genuine quality signal and part of why coding ability improved faster than subjective tasks.

What instruction tuning does not do

It does not add knowledge in any dependable way. A few thousand examples cannot install facts that pretraining never saw. Teams routinely try to fine-tune a model onto their internal documentation and find it produces confidently wrong answers in the right style.

It also does not fully settle preferences over responses that are all reasonable. Choosing between two competent answers on tone, hedging or level of detail is a comparison problem, which is why a preference stage usually follows. RLHF explained covers what that stage adds.

And it does not remove the underlying next-token machinery. An instruction-tuned model still completes text; it has simply been shaped so that the completion it favours is a helpful answer. Every prompt-injection technique exploits exactly that. Prompt injection and agent security covers the consequences.

What this means for your own fine-tuning

If you are considering fine-tuning, be clear about which problem you have. Instruction tuning is the right tool when you need a specific behaviour, format or style reliably and prompting keeps drifting.

It is the wrong tool when you need the model to know things. That is retrieval. Putting your documentation in the prompt works; training on it mostly teaches the model to sound like your documentation. Fine-tuning versus prompting versus RAG covers the decision.

Data quality dominates data quantity here. A few hundred carefully constructed examples that show the exact behaviour you want typically outperform tens of thousands of mediocre ones, because the model is learning a pattern rather than memorising content. Write the examples you actually want copied.

The practical takeaway

When a model behaves in a way you did not ask for — too long, too hedged, wrong format, unnecessary preamble — you are usually seeing its instruction data, not a bug.

That reframes the fix. Give an explicit contract rather than a polite request: state the format, state the length, give one example of the shape you want. You are competing with a learned default, and specificity is what wins.