Processes and objectives that shape model parameters before or during adaptation, including pretraining, instruction tuning, fine-tuning, and optimization.
Tagged articles
9 articles in this tag.
You duplicate a handful of "good" examples to push the model toward a behavior. You retrain. The behavior barely moves — or it moves somewhere you did not…
Read tutorial
You can recite "gradient descent" and still freeze when someone hands you a loss, a gradient, and a learning rate. That freeze is not a math problem. It is…
Read tutorial
Large language models are not digital minds. They are probability engines that turn a conversation into a series of next-token guesses. The guesswork is…
Read tutorial
You have seen the chart. A straight line on a log-log plot, a caption promising that more compute buys predictable improvement, and a comment section…
Read tutorial
When someone says an LLM was "trained," they usually mean one thing. The people who build LLMs mean several. Each major stage of the modern training…
Read tutorial
You watch the loss number fall on a training chart. Something inside the model changed, but the chart never tells you what. The loss is a single number,…
Read tutorial
You are comparing models and you see it everywhere: a name, a letter, and a number. Llama 3.1 8B. Mistral 7B. Qwen 72B. The reflex is almost automatic:…
Read tutorial
You fine-tune a model on your own documents, ask it a factual question about them, and watch it hallucinate anyway. The training ran. The loss curve looked…
Read tutorial
A model can write a confident, fluent paragraph that is completely false — and its training loss curve can look excellent the whole time. That combination…
Read tutorial