Pretraining vs Instruction Tuning vs Alignment: What Each Stage Changes

When someone says an LLM was "trained," they usually mean one thing. The people who build LLMs mean several. Each major stage of the modern training pipeline fixes a different failure, uses a different kind of data, and changes what you observe when you type a prompt. Learn to tell the stages apart, and you can predict what a model will do before you run it—and know what prompting can and cannot fix.

Why "Training" Is Really Several Different Jobs

Imagine hiring someone to answer customer emails. First you teach them to read and write English fluently. Then you teach them the format of a good reply. Then you teach them when not to reply, and how to phrase a refusal without being rude. Different jobs. Different lesson plans. One employee.

Modern LLMs are built in a similar pattern, but the stages are not a strict assembly line. Think of them as functional layers that usually stack in a particular order—and sometimes overlap, repeat, or get skipped:

• Pretraining builds broad language ability from massive amounts of raw text. The result is a base model that can complete text fluently but cannot follow instructions.
• Instruction tuning teaches the model to respond to requests using curated examples of prompts paired with good responses. The result is an instruct model that follows directions.
• Alignment is the broader goal of shaping behavior toward chosen human, developer, or safety criteria. Preference optimization is one important route within that goal. The result is a model whose behavior has been steered toward what humans judge as helpful, honest, and safe.

The layered view matters more than the pipeline metaphor: pretraining builds the engine, and the later stages shape how that engine gets driven. None of the later stages replace the earlier ones. They refine what pretraining created.

One boundary before we go further: this article is about the training side of the lifecycle. If you want the contrast between training and the moment a model answers your question, that is a separate distinction worth holding in mind. Here, we are comparing stages that all happen before you ever type a prompt.

Pretraining: Building the Raw Language Engine

Pretraining is the stage that consumes the most data and compute by a wide margin. The model ingests massive amounts of unlabeled text—web pages, books, code, academic papers, and more—and learns from the text itself.

The objective is deceptively simple: next-token prediction. The model reads a sequence of text and learns to guess what token comes next. Every word in the corpus serves as its own label, because the correct answer is simply the word that actually followed. No humans need to annotate anything.

What emerges is a base model with a remarkable grasp of language structure, factual patterns, reasoning traces, and stylistic conventions. It has absorbed statistical patterns from an enormous library of text.

But here is the catch: a base model is optimized for text continuation, not for answering questions. If you prompt a base model with "teach me how to bake bread," it will not give you advice. It will produce the most plausible continuation of that phrase, which might be something like "in a home oven" or "with a child." Grammatically fluent. Completely useless as a response to your request.

This is not a bug. The model was never trained to help you. It was trained to predict text, and it is doing exactly that.

The scale difference between this stage and everything that follows is enormous. Pretraining runs on thousands of specialized chips for months and costs millions of dollars in compute. The later stages are expensive by ordinary standards, but they are comparatively cheap next to the foundation they refine.

Instruction Tuning: Teaching the Model to Follow Requests

A base model that completes text is a tool with no interface. Instruction tuning gives it one.

The training data here looks completely different. Instead of raw web text, instruction tuning uses curated pairs: a request written the way a human might ask it, paired with a response the model should produce. The dataset mixes many task types—summarization, classification, question answering, rewriting, structured output—so the model learns a general pattern rather than one narrow skill.

Here is the subtle part that surprises many beginners: the underlying objective is still next-token prediction. The model is still guessing what token comes next. What changed is the data. When the training text is an instruction followed by a good response, the model learns the pattern of answering a request rather than the pattern of continuing arbitrary text. In practice, many implementations even mask the loss on the instruction portion, so the model is graded only on how well it predicts the response tokens.

This is why instruction tuning is best understood as a specific type of supervised fine-tuning, not a separate category. The mechanism is the same; the data format and goal are what differ.

The observable change is dramatic. The same "teach me how to bake bread" prompt that produced a useless continuation now yields actual advice: ingredients, steps, temperature, timing. The model has learned that when a human addresses it with a request, the expected pattern is a helpful answer.

One important limit: instruction tuning is primarily a behavior-shaping stage. It teaches the model how to use what it already has. It can also reinforce narrow information contained in its training examples, but it is not the stage that supplies broad knowledge. If the base model never encountered a fact during pretraining, no amount of instruction tuning will reliably make it know that fact.

Alignment: Steering Behavior Toward What Humans Prefer

Instruction tuning makes a model follow requests. Alignment shapes which behaviors the model favors among the many ways it could follow them.

Consider what happens after instruction tuning. Ask the model something and it will answer. But there are many possible answers to any question, and humans do not judge them all equally. Some are more helpful. Some are more honest about uncertainty. Some are safer than others. Some are sycophantic—telling you what you want to hear rather than what is true.

Alignment is the umbrella term for shaping behavior toward chosen criteria: helpfulness, honesty, safety, or whatever a developer decides matters. Preference optimization is one common family of methods within that goal. The training signal comes from human judgments: people compare multiple responses to the same prompt and indicate which they prefer. The model then learns to optimize for that preference signal rather than for matching a single target text.

The technical methods vary. Reinforcement learning from human feedback (RLHF) was the approach that popularized this stage. Direct preference optimization (DPO) achieves a similar goal with a simpler training procedure. New methods continue to appear. But alignment is the goal; RLHF and DPO are routes to it. An aligned model does not require any specific method, and some alignment work happens through supervised safety tuning or other post-training choices.

What does alignment change that instruction tuning did not? Observable behaviors like these:

• The model refuses harmful requests instead of dutifully answering them.
• The model says "I don't know" when it lacks information, rather than inventing a confident answer.
• The model avoids excessive sycophancy, pushing back when a user's assumption is wrong.
• The model exercises judgment between competing valid answers, choosing the one humans would find more useful.

A useful way to hold the distinction: instruction tuning teaches the model what to do. Alignment teaches it what humans prefer done. Both matter, and they are not the same lesson.

Side by Side: Data, Objective, and What You See

The mental rule that captures the common pattern: a base model completes text, an instruct model follows requests, and an aligned model prefers responses humans judge as good.

One caveat before you treat this table as gospel: these are functional layers, not mandatory universal checkpoints. Research has explored instruction-style pretraining, where models learn prompt-following behavior during the pretraining phase itself. Continued pretraining can blur the line between pretraining and fine-tuning. Real production pipelines sometimes add extra post-training stages, repeat stages, or skip them entirely. The table is a map, not the territory—but it is a map that will serve you well for understanding the models you actually use.

Common Mistakes When Reasoning About Training Stages

Beginners misattribute model behavior to the wrong stage all the time. Each misattribution leads to a wrong expectation about what prompting or fine-tuning can fix.

Mistake 1: Calling a base model "broken" because it does not follow instructions. A base model was never trained to follow instructions. It is not failing at its job; it is doing exactly what it was trained to do. If you need instruction-following behavior, you need an instruct or chat variant, not a base model.

Mistake 2: Expecting instruction tuning to add broad new knowledge. Instruction tuning reshapes behavior. It teaches the model to answer questions in useful patterns. It is not the stage that supplies broad factual knowledge. If a model lacks information, the usual fix is retrieval—especially for facts that change or need source grounding. More instruction tuning is rarely the answer.

Mistake 3: Treating alignment as a safety guarantee. Alignment is a preference signal learned from human judgments. It is imperfect, and it can be gamed. A model can be jailbroken. Its refusals can be inconsistent. Alignment reduces harmful behavior; it does not eliminate it. Treat it as a strong preference learned from data, not as a hardcoded safety rule.

Mistake 4: Assuming every model goes through all three stages in the same order—or at all. Many open models skip or compress stages. Some are released as base models with no instruction tuning. Others undergo alignment without a distinct instruction-tuning phase. The three-stage pattern is common, not universal. Read the model card to see what a specific model actually received.

Why This Distinction Matters for How You Use Models

This is not academic taxonomy. The stage model changes practical decisions you will make as soon as you start building with LLMs.

Choosing a model. For everyday use—chat, writing assistance, question answering, most API calls—you want an instruct or chat variant. That is the model that has been taught to respond to requests. A base model makes sense when you plan to do your own continued training or fine-tuning for a specialized purpose, or when you need full control over the training objective.

Knowing what prompting can and cannot change. Prompting steers a model's existing behavior. It cannot add instruction-following that was never trained in, and it cannot add alignment that was never learned. You can coax a base model toward better completions with careful prompting, but you cannot prompt it into being an instruct model. If the behavior you need is missing because a training stage was skipped, the fix is training, not prompting.

Reading model cards. Model cards usually tell you which stages a model received. Names like "base," "instruct," "chat," and "RLHF" are stage labels. When you see them, you know which behaviors to expect. A model labeled "base" will complete text. A model labeled "instruct" will follow requests. A model labeled with an alignment method has been steered toward human preference.

My practical rule for beginners: when you are choosing a model for a real project, start with an instruct or chat variant from a reputable provider. That is the model that has received the later training stages that make it useful for ordinary interaction. Save base models for when you have a specific reason to train further yourself.

The next time you open a model card or try a new model, ask one question: which stage likely produced the behavior I am seeing? If the model completes your prompt instead of answering it, you are probably looking at a base model. If it answers but refuses nothing and never hedges, you are probably looking at instruction tuning without much alignment. If it answers helpfully, admits uncertainty, and declines harmful requests, you are probably looking at a model that received substantial post-training.

Notice the word probably. Behavior gives you clues, not proof. A model may refuse or hedge because of system instructions, policy layers, or the surrounding application—not because of its training stages. To diagnose accurately, combine your prompt tests with the model card and the system context around the model. That combination will tell you more about what to expect—and what prompting can and cannot fix—than any single observation.