Tokens & Pricing

AI models do not process text as words. They process text as tokens.

A token can be a word, part of a word, punctuation, or whitespace. The exact split depends on the tokenizer used by the model.

Why tokens matter

Tokens matter for two reasons:

1. They determine how much context the model can handle.
2. They affect how much the request costs.

When you send a long prompt, paste large files, or include a long conversation history, you are spending input tokens. When the model answers, it spends output tokens.

Input and output cost

Most model providers price input and output separately. Output tokens are often more expensive because the model has to generate them step by step.

That means a short prompt that asks for a huge answer can still be expensive. It also means a long pasted context can be expensive even if the answer is short.

Context windows

Every model has a context window: the maximum amount of information it can consider at once.

If the conversation becomes too large, older or less relevant context may be dropped or compressed by the tool. When that happens, the model can lose details that mattered earlier.

This is why focused conversations work better. Start a fresh thread for a new task. Include the relevant files and errors, not every possible file.

Streaming

Streaming lets you see output as the model generates it. It does not make the model free or change the core token cost, but it improves the interaction because you can stop a bad answer early.

For coding agents, streaming also helps you observe whether the model is heading in the wrong direction before it spends more time and tokens.

Optimize usage

You do not need to minimize tokens at all costs. You need to spend them intentionally.

Good patterns:

• Give the exact files, errors, or requirements the model needs.
• Avoid dumping unrelated context.
• Ask for a plan before a large implementation.
• Use smaller models for simple tasks and stronger models for harder reasoning.
• Stop and restart when a conversation has drifted.

Tokens are the budget for the model's attention. Use them on the context that changes the answer.