beginner
9 min read
Cloud vs Local LLMs: Which Setup Fits Your First Project?
The "local is more private, cloud is more powerful" story sounds clean. It's also too simple to make a good decision with.
Read tutorialplatform
Local LLMs
Running or assessing language models on user-controlled or locally managed hardware rather than relying only on hosted inference.
Tagged articles
Local LLMs articles
10 articles in this tag.
beginner
8 min read
Estimate the Memory a Local LLM Needs
Two people run the same 7B model. One says it needs 4 GB. The other says it needs 9 GB. Neither is lying.
Read tutorial
intermediate
11 min read
Hosted vs Local LLM Cost: Calculate the Break-Even Workload
Two people run the same break-even calculator, feed it the same hardware price, and get opposite answers. Both are right.
Read tutorial
intermediate
10 min read
LLM Quantization Tradeoffs: What Lower Precision Changes
Two model files sit in your download folder. Both say 4-bit. One is smaller, one runs faster, and they give you different answers to the same prompt. If…
Read tutorial
beginner
13 min read
Open-Weight vs Closed LLMs: What Control Do You Actually Need?
You do not need the best model. You need the model that matches the control you actually require.
Read tutorial
beginner
10 min read
Practice Choosing a Local LLM Within a Memory Budget
That is the moment this exercise trains you for. You already know how to estimate weight storage, and you have already run one small model and watched its…
Read tutorial
intermediate
10 min read
Practice Comparing Quantized LLM Outputs on a Fixed Task
You swap a model to a 4-bit build, run the same prompt, and get a slightly different answer. Now what? You cannot tell whether quantization degraded the…
Read tutorial
beginner
9 min read
Run a Small Local LLM and Measure Its Resource Use
The model finishes downloading. You type a prompt. The cursor blinks, the fan spins up, and nothing happens for a while. That pause is the whole question:…
Read tutorial
beginner
9 min read
Using Open Source LLMs
Running an open source LLM on your own machine today feels closer to installing a desktop app than building a neural network. You don't need a data center,…
Read tutorial
beginner
8 min read
What Are LLM Parameters? A Practical Mental Model for Model Size
You are comparing models and you see it everywhere: a name, a letter, and a number. Llama 3.1 8B. Mistral 7B. Qwen 72B. The reflex is almost automatic:…
Read tutorial