What Are LLM Tokens? A Beginner's Guide to Text Units

You paste a long document into a chatbot and hit an error about a "token limit." Or you open an API pricing page and see costs quoted per token, with no clear sense of what you're actually paying for. If you've assumed a token is basically a word, you're not wrong—but you're not quite right either.

Here's the distinction that matters: tokens are the unit the model actually reads and writes. They don't map cleanly to words, and understanding why will save you from surprise bills, truncated responses, and a lot of confusion about how these systems behave.

Why Tokens Confuse Beginners

The natural assumption is that a token is just a word. Count the words in your prompt, multiply by some rate, and you've got your cost. That mental model works right up until it doesn't.

Here's where it breaks down. You paste what looks like a modest document into a chat interface and get an error about exceeding the context limit. Or you send a short message in another language and notice it consumes far more tokens than the English equivalent would. The word-count model doesn't explain any of this.

If you're new to large language models, here's the short version of what you need to know: an LLM is a system trained to predict what text comes next. It doesn't read your words the way you do. It reads a sequence of smaller pieces, and those pieces are tokens.

The crowded-desk image will help throughout this article: every LLM works with a finite working surface, and every token you add takes up space on that surface. Keep that image in mind—we'll come back to it.

What a Token Actually Is

A token is a small chunk of text drawn from the model's vocabulary. That chunk might be a whole short word, part of a longer word, a piece of punctuation, or even a space attached to the next word.

Think of it like LEGO bricks. Language gets broken into reusable pieces the model already knows. Common words like "the" or "cat" might each be a single brick. Longer or rarer words get assembled from several smaller bricks.

Here's a concrete example. The word "understanding" might split into something like "under," "stand," and "ing"—three tokens instead of one. Meanwhile, a common word like "hello" might be a single token. Punctuation and spaces often attach to neighboring tokens, which is exactly why token counts and word counts diverge.

One important detail: each model has its own vocabulary of tokens. The same sentence can tokenize differently depending on which model you're using. That's not a bug—it's just how different systems were built.

How Text Becomes Tokens

Before a model can do anything with your text, that text has to be converted into a sequence of tokens. This process is called tokenization.

The steps are straightforward:

1. Raw text goes in.
2. A tokenizer breaks it into pieces from the model's vocabulary.
3. Each token maps to a number called a token ID.
4. The model works with those numbers, not with letters.

So when you send "I love ice cream!" to a model, it might become something like five tokens: "I," " love," " ice," " cream," and "!"—each with its own ID. The model sees a sequence of numbers, and it processes that sequence to figure out what should come next.

You don't need to understand the algorithm behind tokenization. You need the mental model: text in, numbered pieces out, model processes the pieces.

Why Token Counts Matter: Context Limits

Every model has a maximum number of tokens it can handle at once. This is called the context window, and it covers both your input and the model's output together.

Remember the crowded desk. Your prompt is a stack of papers on that desk. The model's response needs space too. Every token you add—whether it's your instructions, a pasted document, or earlier conversation history—takes space away from what the model can use to respond.

When you exceed the limit, a few things can happen. The model might truncate your input, dropping the oldest parts. It might return an error. Or in a long chat, it might quietly lose track of earlier messages because they fell outside the window.

Long documents and long chat histories consume this budget fast. That's why a chatbot might forget something you said twenty messages ago, and why pasting an entire book into a prompt usually doesn't work.

Why Token Counts Matter: Cost and Generation

Tokens drive two other practical concerns: pricing and how the model generates text.

Most paid AI tools and APIs bill by tokens. They count both what you send in and what the model sends back. A rough rule of thumb: plain English runs about 0.75 words per token. So 1,000 words is roughly 1,300 tokens. But that's a heuristic, not a guarantee. Code, punctuation-heavy text, and non-English languages can tokenize very differently.

Generation is where tokens really show their nature. The model doesn't write your response as a whole. It predicts one token at a time, appends it to what it has, then predicts the next. That loop continues until the model decides it's done. Every word you read from an LLM arrived one small piece at a time.

Output tokens are typically billed at a higher rate than input tokens. That's because generating text is computationally heavier than reading it. The exact rates change and vary by provider, so don't memorize specific numbers—just know that input and output are usually priced differently.

Note: The 0.75-words-per-token figure works best for ordinary English prose. If you're working with code, structured data, or other languages, expect the ratio to shift.

Common Token Mistakes Beginners Make

Once you think in tokens, several common mistakes become obvious.

Mistake 1: Assuming word count predicts token count. Code, punctuation, and non-English text tokenize differently than plain English prose. A short piece of code can consume far more tokens than its word count suggests.

Mistake 2: Treating the context window as unlimited memory. It's a finite budget, not a storage system. Long conversations degrade because earlier messages fall out of the window, not because the model is malfunctioning.

Mistake 3: Forgetting that output consumes the budget too. If a model has a 4,000-token context window and you send 3,800 tokens of input, you've left almost no room for a response.

The practical habit: when a tool shows you a token count or a token limit, read it as the real unit of measure. It's not a curiosity—it's the model telling you how it sees your text.

That said, don't obsess over exact counts for casual use. If you're chatting with an AI assistant, rough awareness is enough. Count tokens carefully when cost or limits genuinely matter—like when you're building an application or processing large documents.

The Mental Model That Matters

Tokens are the model's real unit of reading and writing. Words are how you think about language; tokens are how the model processes it. The gap between those two perspectives explains context limits, pricing surprises, and why long conversations lose the plot.

The crowded desk is the image to keep. Every token takes space. Input and output share the same surface. And the model builds its response one small piece at a time until the desk runs out of room or the response feels complete.

A good next step is to find a tokenizer visualization tool and run a few samples through it. Paste in a paragraph of English, then a few lines of code, then a sentence in another language. Watch how the token counts differ. That hands-on observation will cement the mental model faster than any explanation.

From there, you're ready to think about how context windows shape prompting—what deserves space on the desk, and what should be left off.