How Many Tokens Is My Prompt? Counting and Why It Matters

A token is the unit of text an AI model actually processes. Models bill per token and cap context per token, so token count determines both your cost and whether your input fits at all.

You can count tokens for any text here, free and entirely in your browser. This article explains what the number means.

How tokenization works

Models do not read letters or words. Text is first split into tokens by a tokenizer, and each token becomes a number the model can process.

The split is learned rather than rule-based. Tokenizers are trained to find the most efficient set of subword fragments across a large corpus, so common words become single tokens while rare words break into pieces.

• "the" → 1 token
• "tokenization" → 2-3 tokens
• "antidisestablishmentarianism" → around 7 tokens

The rule of thumb

For English prose:

1 token ≈ 4 characters ≈ 0.75 words

So 1,000 words is roughly 1,350 tokens, and 1,000 tokens is roughly 750 words or a page and a half.

This approximation is accurate within 10-15% for ordinary English. It degrades for other content:

• Code tokenizes less efficiently — unusual whitespace and symbols cost more.
• JSON is expensive relative to its information content, because braces, quotes and keys all consume tokens.
• Non-Latin scripts can cost several times more per visible character.
• Images consume tokens too, often the equivalent of several pages of text for a high-resolution image.

Why it matters

Cost. Billing is per million tokens, with output typically three to five times input. Halving prompt length halves input cost.

Context limits. Every model caps total tokens — prompt, documents, history and response together. The detail people miss is that the response is included: fill the window entirely with input and there is no room to answer. Reserve 2,000-4,000 tokens for output.

Quality. Long contexts degrade attention. Models attend well to the beginning and end of a long input and less well to the middle. A 100,000-token prompt does not give you 100,000 tokens of reliable attention.

Useful thresholds

How to reduce token count

Cut preamble. "I was wondering if you could possibly help me with something" is fifteen tokens of nothing.

Trim retrieved context. RAG pipelines commonly pass ten chunks where three would do. Rerank and pass fewer.

Summarise history. In long conversations, replace the full transcript with a running summary every few turns. This cuts cost and improves focus.

Cap output. Because output costs several times input, an explicit length limit is the highest-leverage single change.

Reconsider JSON. If you are passing large data structures as context rather than as output, a compact format costs meaningfully fewer tokens.

Getting exact counts

Our estimator blends a character-based and word-based approximation, which lands within 10-15% for English — accurate enough for planning prompts and budgeting.

For exact counts, use your provider's official tokenizer library. You need exactness when you are close to a hard context limit or reconciling a bill; for everything else, estimation is fine.

Related tools: the context window calculator shows whether your prompt fits each model's limit, and the cost calculator converts token counts into monthly spend.