Token counting explained for builders and budget owners

Tokens are the billing and context unit for most LLMs. If you cannot estimate them, you cannot forecast cost or fit prompts into context windows. This guide explains token counting in plain language for engineers and budget owners. For interactive estimates, use the token & cost estimator.

What is a token?

A token is a chunk of text produced by a tokenizer — often a subword piece. Rough intuition for English prose:

• ~4 characters per token
• ~0.75 words per token (or ~100 tokens ≈ 75 words)

These are rules of thumb, not laws. Code, URLs, non-English text, and whitespace-heavy JSON tokenize differently.

Chinese often uses fewer characters per token than a naive English rule suggests — still measure with a real tokenizer when precision matters.

Why vendors count differently

Each model family ships (or implies) a tokenizer. The same string can yield different token counts on different APIs. Therefore:

• Budget with the tokenizer of the model you call
• Do not reuse GPT heuristics blindly for other families
• When vendors expose usage.prompt_tokens in responses, treat that as ground truth for billing

Offline estimators (including ours) approximate for planning. They are for forecasts and teaching, not invoices.

What consumes tokens in a request?

Typical chat/completions request:

1. System message
2. Developer / tool instructions (if any)
3. Conversation history
4. Retrieved documents / RAG chunks
5. Tool schemas and tool results
6. The new user message
7. Output tokens from the model reply (billed separately, often at a different rate)

Agents multiply (4)–(6) across loop iterations. That is why research agents need max-call budgets (system prompts for research agents).

Context window vs cost

• Context window — hard maximum tokens the model can consider (input + output constraints vary by API)
• Cost — usually dominated by how many tokens you actually send and generate, not the maximum window

A 128k window does not mean you should send 128k tokens. More context is not free and often hurts quality when noisy.

Practical estimation workflow

1. Paste a representative prompt (system + user + sample RAG) into an estimator
2. Set expected output length (max_tokens or observed average)
3. Multiply by monthly calls × (1 + retry rate)
4. Apply vendor $/1M input and output rates
5. Add 10–20% buffer for prompt creep

Do this per feature, not as one blob.

Why your estimator disagrees with the API

Cause	What to do
Different tokenizer	Prefer API usage fields
Chat template tokens	Account for special tokens / role markers
Tools JSON	Schemas are not free
Invisible system additions	Log full request payload size in staging
Compression / caching	Cached tokens may bill at different rates

Counting for non-English and code

• CJK text: do not assume 4 chars/token; sample real counts
• Code: operators and identifiers can be token-heavy; indentation matters less than you think, but long minified lines can be worse
• Base64 / hex: extremely token-inefficient — avoid stuffing blobs into prompts
• Markdown tables: usually fine; huge HTML dumps are not

Output token control

Output is often pricier per token than input on frontier models.

• Set max_tokens to the task (titles vs essays)
• Ask for compact formats (bullets, JSON without comments)
• Forbid restating the user question
• For creative tasks, accept higher caps deliberately — and budget them

Token budgets in product design

Examples:

• Free tier: 50k input tokens / day / user
• Support bot: hard cap 8k context; summarize older turns
• Doc assistant: top-5 chunks ≤ 3k tokens combined
• Eval suite: separate budget so it cannot starve production

See LLM cost control for teams.

Teaching teammates a shared language

Print this on the wiki:

• “We’re not short on ideas; we’re short on tokens and edit time.”
• “Expanding the system prompt by 1k tokens at 1M calls/month is a budget change — review it.”
• “Estimator ≠ invoice; API usage = invoice.”

Quick lab (10 minutes)

1. Take a 500-word blog paragraph; estimate tokens at 4 chars/token
2. Run the same text through your provider’s tokenizer or a library when available
3. Compare error %
4. Repeat with 50 lines of Python and a Chinese paragraph
5. Update your team’s rule of thumb per content type

Related tools and guides

• Token estimator tool
• Prompt cleaner — reduce junk tokens before counting
• Prompt engineering checklist