What Are AI Tokens and Context Windows? A Plain-English Guide
Tokens and context windows decide how much text an AI model can handle at once, and why Hindi or Tamil prompts often cost more than English ones.
Type a long question into ChatGPT in Hindi and watch it hit a length limit far sooner than the same question typed in English would. Users report this constantly and assume it's a bug. It isn't. It's tokens, and understanding them explains a surprising amount about why AI chatbots behave the way they do.
A token isn't a word, and that trips people up
Every AI chatbot you've used — ChatGPT, Gemini, Claude, whatever your bank's support bot runs on — is built on a large language model (LLM), a system trained on huge amounts of text to predict what comes next in a sequence. But an LLM doesn't actually read in words. It breaks text into smaller chunks called tokens, using a process called tokenization, and those chunks can be whole words, parts of words, or even single characters.
"Unbelievable" might become one token. A rarer word like "tokenization" itself might split into two or three pieces: "token," "iza," "tion." As a rough rule of thumb for English, one token works out to about four characters, or three-quarters of a word. Every piece of text you send an AI model, and everything it sends back, gets counted and billed in tokens, not words or characters.
Why context windows have a hard ceiling
The context window is the maximum number of tokens a model can hold in its "working memory" at once — your prompt, any documents you've pasted in, the model's own reply, and (in a long chat) everything said earlier in that conversation. GPT-5-class models, Gemini, and Claude all advertise context windows ranging from roughly 100,000 tokens up to a million or more, and bigger sounds strictly better. It isn't free, though.
The mechanism that lets a model weigh every word against every other word in a passage — called attention — gets more computationally expensive as the input grows, not in a straight line but closer to a curve. Doubling the context doesn't just double the work; it can multiply it several times over. That's why providers charge more per token for longer prompts, why "unlimited context" isn't actually unlimited, and why a model can lose track of something you mentioned at the very start of a long chat even though it was technically "in" the context window the whole time — being present isn't the same as being weighted heavily.
A chatbot doesn't forget because it's being careless. It forgets because the detail slipped outside the window of text it's actually paying attention to.
The India angle: some languages pay a bigger "token tax"
Here's where it gets genuinely relevant for Indian users and builders, not just a trivia fact. Most major tokenizers were trained on text that's overwhelmingly English and Latin-script. When they encounter Hindi, Tamil, Bengali, or other Indic scripts, the same sentence often breaks into noticeably more tokens per word than an English sentence of equivalent meaning would — sometimes two to four times as many, depending on the model and language.
That has two real consequences, not just an academic one. First, cost: if you're an Indian startup building a customer support bot in Hindi and paying per token through an API, you're paying more for the same conversation than an equivalent English-only product would. Second, capacity: the effective context window shrinks for Indic-language use, because more tokens get consumed saying the same thing, leaving less room for the model to "remember" earlier parts of a long document or chat.
This is a large part of why Indian AI efforts — Sarvam AI, AI4Bharat's work out of IIT Madras, and Ola's Krutrim among them — have specifically built or fine-tuned tokenizers around Indian languages rather than just wrapping a Western model in a translation layer. A tokenizer trained on real Hindi, Marathi, or Telugu text represents those languages far more efficiently, which directly cuts cost and stretches the usable context for the same conversation. It's also one reason techniques like retrieval-augmented generation (RAG) matter for multilingual products in India: instead of stuffing an entire policy document into the context window and eating the token cost on every query, a RAG setup fetches only the relevant passage first. Code24 covered how that works in RAG Explained: How AI Chatbots Pull Real Facts Instead of Guessing.
What this means if you're actually using these tools
You don't need to count tokens by hand, but a few habits make a real difference once you understand what's happening under the hood:
- Trim the clutter. Pasting an entire PDF when you only need one section burns context you'll want later in the conversation.
- Start a fresh chat for a new topic. Long-running threads quietly accumulate tokens, and very old messages get weighted less even if they're technically still "in" the window.
- Watch for a model switching language mid-task. If you're building something that needs to handle Hindi or another Indic language at scale, ask what tokenizer the underlying model uses — it changes your cost per user meaningfully.
- Bigger context window isn't automatically the better tool. For most everyday use, how well a model uses the tokens it has matters more than the sticker number of tokens it can theoretically hold.
None of this is a reason to distrust AI tools — it's just the plumbing behind them. The next time a chatbot forgets something from earlier in a long conversation, or a Hindi-language assistant feels more expensive or more limited than its English counterpart, you'll know it's not a glitch. It's tokens doing exactly what they were built to do, within a window that was never actually infinite to begin with.
What's Your Reaction?
Like
0
Dislike
0
Love
0
Funny
0
Angry
0
Sad
0
Wow
0