Research · AI security · LLM security · Context window

Context windows explained: from 10K tokens to 1,000 trillion

Tokens, context and memory explained, then a climb from 10K to 1,000 trillion tokens: what fits, what breaks, the security risks at each size, and where quantum and AGI fit.

18 min read

Short answer

A context window is how much text a model can see at once, counted in tokens (about three quarters of a word each). In November 2026 the flagship models from Anthropic, Google and OpenAI list about 1 million tokens. Bigger windows let a model read whole codebases, but cost grows fast, accuracy drops well before the limit, and every extra document is one more place for a prompt injection or a leaked secret. Past about 100 billion tokens the window stops being a prompt and becomes a database, with database security problems.

A context window is the amount of text a language model can look at in one go. In November 2026 the flagship models from Anthropic, Google and OpenAI each list a window of about 1 million tokens, which is roughly 750,000 words.

Researchers have already shown designs that stretch toward a billion tokens, so it’s fair to ask what happens if the number keeps climbing: to a trillion, or to 1,000 trillion. This guide starts from the basics, then climbs the ladder one step at a time. At each step we look at what fits, what gets better, what breaks for the engineers building it, and what goes wrong for security.

The context ladder, 10K to 1,000 trillion tokens

IN USE TODAY RESEARCH ONLY NO SYSTEM YET 10Ka long article 100Ka full novel 1M~100,000 lines of code · flagships now 1 billionmore words than a person hears in life 100 billionabout a million novels 1 trillionabout 10 million novels 100 trilliona third of all public human text 1,000 trillionover 3x all public human text
Our diagram. Word counts use OpenAI's rule of thumb (100 tokens is about 75 words); lines of code use Magic's ratio; human-text totals use Epoch AI's estimate of about 300 trillion tokens. Sources are below.

What a token is

Models don’t read letters or words. They read tokens: chunks of text that a tokenizer cuts out of the input. A common word is often one token, and a rare word may be split into several. OpenAI’s rule of thumb for English is that “1 token ~= 4 chars” and “100 tokens ~= 75 words”.

Everything is measured in tokens: what you send, what the model writes back, what you pay for and how much fits.

What a context window is

The context window is the model’s working space for one request: everything it can see while it writes the next token. That includes the system prompt, the chat so far, any files or web pages a tool pulled in, and the answer as it’s being written.

If something isn’t in the window, the model can’t use it in that request. If the window is full, something has to be cut, summarized or left behind.

Context is one of three kinds of memory

People say “the model remembers” for three different things, and the security story is different for each.

Three places a model keeps information

1. WeightsWhat it learned in training. Fixed until the model is retrained.
long-termpoisoning
2. Context windowWhat it can see in this request. Wiped when the request ends.
working memoryprompt injection
3. External memoryDatabases, search indexes and memory tools it reads into the window when needed.
retrieval (RAG)access control
Our layout. The chips name the main security concern for each one.

There’s a fourth thing engineers care about, the KV cache. When a model reads your context it computes a key and a value for every token at every layer and keeps them in GPU memory, so it doesn’t redo that work for each new token it writes. That cache is why long contexts are so expensive, and it’s a copy of your context sitting in someone’s memory.

Why the size matters

A bigger window means fewer tricks. With 10K tokens you have to chop documents into pieces, search for the right piece and hope you picked well. With 1 million you can often paste the whole thing. Google’s Gemini 1.5 report described “near-perfect retrieval (>99%) up to at least 10M tokens” in its tests, and Meta launched Llama 4 Scout in April 2025 with what it called “an industry-leading context window of 10M”.

The catch has two parts: cost and quality.

Cost: attention grows with the square

In the original Transformer paper, the cost of a self-attention layer is O(n² · d), where “n is the sequence length”. In plain words, every token looks at every other token. Double the context and the work goes up four times. Go from 1 million to 1 trillion tokens, a million times longer, and plain attention does a million million times more work.

Why doubling the context quadruples the work

2 tokens 4 pairs 4 tokens 16 pairs 8 tokens 64 pairs … 1 million tokens = 10¹² pairs
Our diagram of the n² term in "Attention Is All You Need" (Vaswani et al., 2017). Each square is one token looking at one other token.

Engineers have cut the time cost a lot. FlashAttention (2022) is “an IO-aware exact attention algorithm”. Ring Attention (2023) spreads one sequence across many chips so the length can be “up to device count times longer”. LongNet (2023) claims “a linear computation complexity” and a sequence length of “more than 1 billion tokens”. Memory is the harder wall.

Memory: the KV cache

Here is our arithmetic for one real model shape. Meta’s Llama 3 70B has 80 layers and 8 key/value heads, each 128 numbers wide (a model dimension of 8,192 split over 64 heads). Stored at 2 bytes per number, each token needs 2 × 80 × 8 × 128 × 2 bytes, about 0.33 MB of KV cache.

KV cache for one request, Llama 3 70B shape (log scale)

10K tokens
3.3 GB
100K tokens
33 GB
1M tokens
328 GB
1 billion
328 TB
100 billion
33 PB
1 trillion
328 PB
100 trillion
33 EB
1,000 trillion
328 EB
Our arithmetic from the Llama 3 paper's 70B settings, 16-bit cache, no compression. Real systems compress, share and offload the cache, so treat these as the size of the problem, not a quote. GB, TB, PB, EB = giga-, tera-, peta-, exabytes.

Magic, a startup that trained a 100M-token model called LTM-2-mini, made the same point with a bigger model. By its numbers, “running Llama 3.1 405B with a 100M token context requires 638 H100s per user just to store a single 100M token KV cache.” Magic says its own design needs “a small fraction of a single H100’s HBM per user” for the same context. That’s the company’s claim about its own product, and we haven’t seen it independently tested.

The vLLM team found a second problem: in the serving systems they studied, “only 20.4% - 38.2% of the KV cache memory is used to store the actual token states”. Long contexts waste memory unless the serving software is built for them.

Quality: the window is bigger than the attention span

A model that accepts 1 million tokens doesn’t use all of them equally well. The research keeps finding the same pattern.

Claimed window vs what models actually use

Halfof models claiming 32K+ held up at 32KRULER, 2024
11models fell below 50% of their short-context score at 32KNoLiMa, 2025
99.3 → 69.7%GPT-4o, short vs 32K, on NoLiMaone of the best
2x–30xmore misses by AI monitors after 800K tokensMartin and Roger, 2026
Sources: Hsieh et al. (RULER), Modarressi et al. (NoLiMa), Martin and Roger, "Classifier Context Rot". Full links below.

“Lost in the Middle” (Liu et al., 2023) found that performance “is often highest when relevant information occurs at the beginning or end of the input context”. Chroma tested 18 models in 2025 and reported that “model performance degrades as input length increases”, even on simple tasks. A June 2026 paper adds a twist: “effective context capacity is a function of lexical density”. Text packed with many distinct terms wears out the useful part of a window faster than loose prose, and our guess is that code and logs fall on the dense side.

So the honest size of a context window has two numbers: what the vendor accepts, and what the model reliably uses for your kind of data.

Climbing the ladder

Size What you gain What it costs
10K Cheap, fast, easy to inspect Forgets long chats; needs retrieval
100K Whole documents fit Bigger bills, misses in the middle
1M Whole repos and long agent runs Slow, costly, quality drops early
1B A company’s text in one place Racks of memory; permissions mix
100B+ Live access to huge archives Becomes a database to secure

10K tokens: a long article

Ten thousand tokens is a few pages, a short contract or one source file. It’s cheap and quick, and a person can read everything the model saw, which makes mistakes easy to debug.

The pain is everything that doesn’t fit. A long chat loses its beginning, and a folder of documents has to go through retrieval (RAG): split into chunks, turned into embeddings, searched, and only the best few chunks pasted in. Most quality problems at this size are chunking problems.

Retrieval brings its own security job. The search index has to respect who may see which document, or the chatbot hands one user another user’s files.

100K tokens: a novel

At about 75,000 words you can fit a book, a long report or a medium codebase, so whole documents go in without chunking. Each full request costs much more, though, and this is where the middle of the context starts getting missed. Prompt caching starts to pay off here, so you don’t pay to re-read the same 100K tokens every turn.

This is also where indirect prompt injection gets room to hide. Greshake and colleagues showed it in 2023 “by strategically injecting prompts into data likely to be retrieved”, and warned that these apps “blur the line between data and instructions”. OWASP’s wording: “Indirect prompt injections occur when an LLM accepts input from external sources, such as websites or files.” More pages in the window means more places for a planted instruction. Our note on the OWASP LLM Top 10 covers the rest of the list.

1 million tokens: where we are

Anthropic’s current Claude models list “1M tokens”. Google lists Gemini 4 Argon at 1,048,576 tokens and OpenAI lists GPT-6 Astra at 1.05M (vendor documentation, checked 17 November 2026). By Magic’s ratio of 100M tokens to “~10 million lines of code”, a million tokens is about 100,000 lines.

That’s enough for a whole repo or an agent that works for hours without losing track of the task. It’s also slow to start, expensive for every full-window request, and less accurate than the label suggests. Managing context becomes real engineering work: what goes in, in what order, what gets summarized and when the cache gets reused. Long agent runs fill up with tool output nobody reads.

Some security problems grow with the window. Anthropic’s 2024 research on many-shot jailbreaking found the attack “takes advantage of a feature of LLMs that has grown dramatically in the last year: the context window”. It works better as examples are added, “according to a scaling trend known as a power law”.

The safety checks also get worse. Martin and Roger found that AI monitors reading agent transcripts missed the harmful actions they were looking for 2 to 30 times more often when those actions came after 800K tokens of harmless activity.

Long sessions collect secrets too. Our note on the Cursor breach shows what happens when stored agent sessions hold credentials.

1 billion tokens: the research frontier

LongNet’s 2023 paper is titled “Scaling Transformers to 1,000,000,000 Tokens”. Infini-attention (Munkhdalai et al., 2024) took another route. It adds “a compressive memory” so inputs can be infinitely long “with bounded memory and computation”. Titans (Behrouz et al., 2024) describes “a neural long-term memory module” that can “scale to larger than 2M context window size”. None of these is a product with a billion-token window you can buy today.

A billion tokens is about 750 million words. The BabyLM researchers estimate children hear “2M-7M words per year”, so even across an 80-year life (our arithmetic) a person hears about 160 to 560 million words. A billion-token context holds more language than one human ever takes in.

The payoff would be a company’s code, wiki and tickets in one place, with little need for search. The price, by our KV cache numbers, is around 328 TB for one request on the 70B model shape. That’s racks of hardware. The window stops being part of a request and becomes a shared, long-lived object: loaded once, kept warm, hit by many questions. Someone has to version it, refresh it when a file changes and split the bill between teams.

Access control breaks at this size. RAG can check permissions per document before fetching it. A single shared context puts the HR folder and the sales deck in the same prompt, and the model can’t be trusted to keep them apart. The fix is a separate context per permission level, which brings back much of the complexity big windows were meant to remove.

100 billion tokens: about a million novels

No published system works at this size. From here on, everything is our reasoning from the numbers above.

A large organization’s whole written history could sit in one model: every email, ticket, log and document. It would need around 33 PB of cache on our model shape, so loading it is a data-center job, and it needs the tools of a database. Sharding, replication, backups, a record of what’s inside and a way to delete one item without rebuilding the lot.

The security picture changes shape. One prompt now reaches everything, so a single successful injection from one poisoned email among millions can ask about anything in the company. Nobody can review what’s loaded, which makes provenance the main control: where each piece came from, and whether it’s trusted.

1 trillion tokens: about 10 million novels

Most of a trillion-token context would be machine data, like logs, metrics and transcripts, because organizations don’t write that much prose. Keeping it current means streaming new data in all the time. One copy is around 328 PB of cache on our model shape.

Whoever runs it holds one of the largest single stores of sensitive data anywhere. That’s concentration risk: one breach, one insider or one bad prompt exposes everything at once. A request to delete one person’s data means finding them in every cached copy.

100 to 1,000 trillion tokens: more than all human writing

Epoch AI estimates “the total effective stock of human-generated public text data is on the order of 300 trillion tokens”. So 100 trillion is a third of everything people have published, and 1,000 trillion is more than three times all of it. A context this big would be filled mostly with machine data and AI-generated text, at around 33 to 328 EB of cache on our model shape.

In theory a model could consult a large share of human knowledge live, instead of a blurred version learned in training. In practice the line between training and context fades, and whoever controls what goes in controls how the model behaves. It would be a global data platform split by region and permission. Data residency rules alone stop it being one context: EU data has to stay in the EU, for example.

Prompt injection at this size turns into data poisoning. An attacker doesn’t need to touch the model, only to get text into the pipe that feeds it.

The main security risk at each size

10K–100KRetrieval leaks the wrong document. Hidden instructions in fetched pages.
RAG accessindirect injection
1M (today)Many-shot jailbreaks. Monitors miss more. Secrets pile up in long sessions.
jailbreakcontext rot
1 billionOne shared context mixes permission levels.
access control
100B and upContext becomes a database. Poisoning, concentration, deletion, residency.
provenancedata platform
Our summary. The 10K to 1M risks come from the research cited above; the 1 billion and larger risks are our analysis, since no system runs at those sizes.

What we don’t know: quantum, humans and AGI

This section mixes sourced facts with our own opinion, and we label which is which.

Quantum and very large contexts

What’s known. In August 2024 NIST released its first three post-quantum encryption standards: FIPS 203 (ML-KEM), FIPS 204 (ML-DSA) and FIPS 205 (SLH-DSA). NIST names the risk “harvest now, decrypt later”: an attacker copies encrypted data today and waits for a quantum computer that can break it, which is one reason NIST gives for moving to the new standards soon.

Our view. A giant context and its KV cache are a huge pile of sensitive data that has to move between machines and sit in storage. If it’s copied today under today’s encryption, it could be read later by a quantum computer. So the bigger the context, the more it matters to move its storage and transport to post-quantum encryption early. In this research we found no evidence that quantum computers will make attention itself cheaper, so treat that claim with caution when you see it.

Could a human hold that much context?

What’s known. Human working memory is tiny. Cowan’s 2001 review found “a single, central capacity limit averaging about four chunks”. Long-term memory is much larger. Bartol and colleagues estimated synapses store about “4.7 bits of information” each. The Salk Institute’s press release turned that into an estimate of “at least a petabyte” for the brain. The paper itself doesn’t give that figure.

Our view. People don’t work like a context window at all. We hold four or so things in mind and rebuild the rest from compressed long-term memory. A person who hears a few hundred million words in a lifetime still outthinks a model holding a million tokens on many tasks, which tells you the raw size of the working space isn’t what makes a mind good. A human won’t hold a trillion tokens word for word, and doesn’t need to.

Does a huge context mean AGI?

What’s known. Researchers who define AGI don’t define it by memory size. François Chollet’s 2019 paper describes “intelligence as skill-acquisition efficiency”. The “Levels of AGI” paper by Morris and colleagues rates systems by “depth (performance) and breadth (generality) of capabilities”. It also says “The majority of definitions focus on what an AGI can accomplish, not on the mechanism by which it accomplishes tasks.”

Our view. A bigger window is a bigger desk, not a better brain. It helps a model use more information, and the research above shows models already struggle to use the window they have. Long memory might be one ingredient of more general systems, but a 1,000-trillion-token window on its own wouldn’t make a model generally intelligent. It would make it a very powerful search-and-summarize engine sitting on a very large, very attractive target.

What to do as a builder today

  • Measure your real window. Test the model on your own data at the lengths you plan to use. The vendor’s number is the most you can send, and says nothing about how well the model reads it.
  • Put the important parts at the start and the end. Instructions, rules and the question itself shouldn’t sit in the middle of a long context.
  • Keep secrets out of context. Long sessions get stored and cached. Don’t paste keys, and keep .env files away from agents.
  • Treat everything the model reads as untrusted. Every fetched page, file and tool result can carry instructions. The longer the context, the more of them.
  • Check permissions before data goes in. Once text is in the window, assume any user of that session can get the model to repeat it.
  • Watch the bill. Cap tokens per request and per user. A full 1M-token request repeated in a loop gets expensive fast.

For a wider view of these risks, start with LLM security: the risks, the fixes and where to start.

Sources

Checked on 17 November 2026.

Foundations and cost:

Long-context quality:

Current models (vendor docs, 17 November 2026):

Security:

Data, humans, quantum and AGI:

AI Security Engineer

Move into AI security from the role you have

340+ videos, 33 modules, labs on AWS, Azure and Google Cloud, a graded capstone and a certificate.

Launch price $199, regular price $299 one payment. Final price: nothing is added at checkout.

Enroll today See the full curriculum

AI Security Engineer

Enroll in the full course

A practical, applied AI security course. Every module, lab, quiz, handbook and the graded capstone open the day you join.

  • 340+videos
  • 33modules
  • 3clouds included
  • 340+ videos, plus every reading page and lab across all 6 parts
  • Local and cloud lab solutions for AWS, Azure and Google Cloud
  • Downloadable slide deck (PDF) for every module
  • Graded capstone and certificate
  • Every future update, including the quarterly future-watch

Launch price $199, regular price $299

Launch price until Nov 30, 2026One-time payment. No subscription. Final price: nothing is added at checkout.

You pay $199 USDUS dollars, paid by card. Final price: nothing is added at checkout.

You pay on Razorpay's secure payment page. We never see your card or bank details. One-time payment, no subscription.

  • Paid on Razorpay's secure page: Card, in US dollars
  • Refund policy