The 1 Million Token Context Window, Explained

A 1 million token context window is the amount of text a large language model can read and reason over in a single request. According to Wikipedia’s entry on large language models, context length determines how much prompt, history, and reference material a model can hold at once, and 1M tokens works out to roughly 750,000 words. A wave of frontier models now ships this size, including Moonshot’s Kimi K3.

This guide explains what a token is, what fits in a million of them, why the number matters, and the limits nobody puts on the marketing slide.

What a Context Window Actually Is

Tokens, not words

A token is the unit a model reads — roughly 3 to 4 English characters, or about 0.7 words. The context window is the total number of tokens the model can hold at once; everything below counts against the same budget:

• The current prompt or question
• The full conversation history so far
• Any pasted documents, code, or transcripts
• The system instructions that shape the model’s behavior
• Tool outputs an agent has accumulated during a task

That’s a different unit than word count, which is why “1M tokens” and “1M words” aren’t the same claim — tokenization and context length are defined the same way across most modern transformer architectures.

Why the window has a hard edge

Everything the model “sees” in one turn lives inside this window; nothing outside it exists for that response. When you exceed it, the oldest tokens fall out of view. There’s no partial memory beyond the edge — it’s a hard cutoff, not a gradual fade.

Window size	Approx. words	Approx. pages
8,000 tokens	~6,000 words	~20 pages
128,000 tokens	~96,000 words	~320 pages
1,000,000 tokens	~750,000 words	~2,500 pages

What Fits in 1 Million Tokens

1M tokens is roughly 750,000 words, about 2,500 pages, or around 8 average-length paperback novels back to back. In concrete terms, one million tokens is enough to hold:

• A full production codebase of 50,000–100,000 lines
• A year of Slack history for a 20-person team
• An entire annual compliance or policy document set
• Several hundred short PDF contracts, side by side
• A full novel series, not just a single book

The scale, calibrated

Numbers like “a million tokens” are hard to picture until anchored to something concrete. Next to a codebase or a contract set, the same number becomes a planning constraint — how much of your actual work fits in one shot, and how much still needs splitting.

From chunks to whole objects

The practical shift is architectural: instead of chunking a codebase or contract into pieces, you load the whole thing at once and ask questions across all of it. A function call and its definition, or a clause and the exhibit it references, stay in the same request instead of landing in separate chunks that never get compared directly.

Why the Number Matters

Whole-repository and whole-corpus reasoning. Feed an entire repo for architecture review, or a full policy set for compliance Q&A, without RAG chunking. Cross-file reasoning improves because nothing is split before the model sees it — a bug spanning three files becomes visible in one pass.

Long-running agents. An agent can run a 50-step workflow, accumulate every tool result, and still reference a decision from step 3 while executing step 47. Big context is what makes durable, stateful agents practical.

Broader model access to this scale. Many open-weight models now ship large context windows as a default rather than a premium tier, one of the trends covered in this site’s guide to open-weight AI models.

To try this in practice:

1. Identify the full object you want reasoned over — a repo, a contract set, a document archive.
2. Estimate its token count (roughly 1.3 tokens per word) and confirm it fits the model’s limit.
3. Load it whole rather than pre-chunking it.
4. Put the most important facts at the very start or end of the prompt, not buried in the middle.
5. Ask the cross-cutting question you couldn’t ask with retrieval alone.
6. Check the answer against a known fact from the middle of the input, not just the edges.
7. If accuracy drops on middle content, restructure so critical material sits near the start or end.

The Limits Nobody Puts on the Slide

Lost in the middle

Models don’t attend uniformly across a long input. Accuracy is highest for content near the start and end of the window; information buried in the middle can degrade by 30% or more on multi-needle retrieval tasks. Put critical context first or last — an architectural decision, not a prompting tip. This traces back to how transformer attention weighs tokens against each other rather than treating every position identically.

Latency, cost, and the KV cache

Filling a million-token window raises time-to-first-token and cost, and inflates the KV cache the model keeps in memory during generation. Sparse architectures such as mixture of experts help keep large-context inference affordable by activating only a subset of parameters per token. This is also why efficient attention designs matter for cost at scale — see this site’s breakdown of Kimi Delta Attention.

Factor	Small context (8K–32K)	Large context (1M)
Time-to-first-token	Low	Higher
Cost per request	Low	Higher
Cross-document reasoning	Limited by chunking	Full corpus visible
Accuracy on buried facts	Less of an issue at small scale	Lost-in-the-middle risk

None of that makes large context impractical — it just has a cost curve worth managing:

• Prompt caching, so repeated system instructions and reference material aren’t re-processed on every call
• Sparse or mixture-of-experts architectures that activate only part of the model per token
• Trimming conversation history instead of resending the entire transcript every turn
• Summarizing older tool outputs in long-running agent loops instead of keeping every raw result

Cost scales with how much of the window actually gets filled, not the maximum size, so a request using 50,000 of the available million tokens costs roughly what a 50,000-token request would cost.

Researchers documented the lost-in-the-middle effect directly in a widely cited study on long-context language models:

We find that performance can degrade significantly when changing the position of relevant information, indicating that current language models do not robustly make use of information in long input contexts.

Liu et al., “Lost in the Middle: How Language Models Use Long Contexts”

Big Context vs RAG

When to just load everything

For small, static corpora, full-context loading is simpler than retrieval-augmented generation and often produces better output, since there’s no retrieval step that can miss a relevant chunk. Use the simplest approach that meets the requirement rather than building a retrieval pipeline for a document set that already fits in one prompt.

When RAG still wins

For huge, fresh, or frequently changing knowledge bases — and for low-latency conversational systems — targeted retrieval still beats loading a million tokens every turn. RAG tends to win when:

• The knowledge base is far larger than any context window, even a million tokens
• Content changes frequently and re-indexing beats re-sending the whole corpus
• Response latency matters more than exhaustive cross-document reasoning
• Only a small, targeted slice of the corpus is relevant to any single query

FAQ

What is a context window in AI?

The maximum number of tokens a model can process in one request, covering the prompt, conversation history, pasted documents, and system instructions.

What can you fit in a 1 million token context window?

About 750,000 words — roughly 2,500 pages, a 50,000 to 100,000 line codebase, or hundreds of documents.

Does a larger context window mean better AI performance?

Not automatically. Bigger windows enable new workflows but introduce lost-in-the-middle degradation, higher latency, and higher cost.

Is 1M token context a replacement for RAG?

Sometimes. For small, static corpora, yes; for large, fresh, or latency-sensitive knowledge bases, RAG still wins.

What is the 'lost in the middle' problem in LLMs?

Models retrieve information near the start and end of a long context more reliably than content buried in the middle, sometimes with 30% or more accuracy degradation on middle content.