Anthropic and Google Fight Over AI Context Windows

Anthropic released a 2 million token context window on Claude 4 last week. Google matched it within days on Gemini 2.5 Pro. The moves mark a shift from model accuracy to memory scale as the main arena.

Both companies now sell access to context lengths that exceed most enterprise document sets. The change pressures rivals to decide whether they chase the same lengths or optimize for lower cost at shorter windows.

Releases Set New Length Benchmarks

Anthropic first announced the 2 million token window on June 3. Google responded on June 5 with an identical limit in Gemini 2.5 Pro. Both updates apply to paid API tiers only. Google Blog on its official blog, confirming the expanded context for enterprise users.

The length allows a model to hold roughly 1,500 pages of text in a single session. Prior limits stood near 200,000 tokens for both providers. The jump affects how teams build retrieval systems that once relied on chunking.

Internal tests shared by Anthropic show the longer window cuts summarization steps by half on legal contracts. Google reported similar gains on codebases that span multiple repositories.

Teams working with historical archives now treat the full set as one prompt rather than multiple staggered calls. Anthropic’s Claude 4 processes a complete 10-year financial history for a single company in one pass. Google’s Gemini 2.5 Pro ingests every commit message and pull request across five years of an open-source project without intermediate summarization.

Engineers previously split large inputs into 50,000-token chunks and maintained separate vector stores. The new limits collapse that pipeline. One logistics firm replaced six retrieval agents with a single call that references every shipment record since 2019.

Further technical details reveal how the providers reached these lengths. Anthropic trained Claude 4 on extended sequences using a combination of ring attention and optimized memory allocation that allows gradual scaling without full retraining. Google applied its own mixture-of-experts routing to Gemini 2.5 Pro, selectively activating parameters only for the most relevant portions of the long context. Both approaches required substantial engineering investment in GPU memory management and custom kernel optimizations.

Enterprise teams now experiment with entirely new data formats. Legal firms upload multi-year litigation histories that include emails, deposition transcripts, and court filings. A single 2-million-token window can contain 1.8 million tokens of primary documents plus 200,000 tokens of precedent case law, enabling the model to draw connections across decades of records in one forward pass.

Additional examples illustrate the breadth of adoption. Publishing houses are loading complete back catalogs of journals for thematic analysis across decades. Academic research consortia now load entire grant histories when writing new proposals, allowing models to identify overlapping funding patterns that might otherwise require weeks of manual cross-checking.

For professionals building personal or team knowledge systems around long documents, tools like remio’s AI-native second brain provide complementary workflows that integrate extended context capabilities.

Stakes Rise for Teams That Manage Large Records

Knowledge workers and engineering groups now face direct pressure. They must choose between paying for longer sessions or refactoring their retrieval pipelines. Shorter windows force repeated context refreshes that add latency.

Product teams at mid-size firms report testing whether the extra tokens reduce error rates enough to justify higher spend. Early numbers remain mixed. Some document sets still produce hallucinations once token count passes 800,000.

Legal departments that once maintained dozens of indexed contract folders now upload entire deal histories. A 400-page merger agreement plus all prior amendments fits inside the window. The model can cross-reference clauses across versions without external lookup.

Engineering organizations experience parallel effects. A mid-sized fintech company moved its entire monorepo documentation - approximately 1.4 million tokens - into a single Gemini session. Developers ask architecture questions that reference both current code and archived design decisions from 2021. Response quality improved, yet the team still pays three times the previous per-query cost.

Healthcare research groups have begun loading complete patient trial records spanning multiple sites and years. One oncology team combined eight years of longitudinal data, imaging reports, and genetic sequencing notes into a single Claude 4 context. The model identified previously unnoticed correlations between treatment timing and biomarker changes, though oncologists still verify every flagged pattern against source scans.

Supply-chain operators are testing similar patterns. A global manufacturer ingested five years of supplier contracts, quality reports, and shipping logs into one prompt to model disruption risks. The exercise surfaced hidden dependencies between certain raw-material vendors and regulatory changes in two jurisdictions.

Capability Claims Mask Rising Compute Costs

Longer context does not automatically improve reasoning. Both companies acknowledge that attention layers dilute focus across distant passages. Anthropic therefore applies a new routing layer that prioritizes recent tokens.

Google uses a hierarchical compression step before full attention. The technique keeps costs from scaling linearly with length. Independent benchmarks show the compression adds small accuracy drops on multi-hop questions, as noted in coverage from The Verge.

The routing layer in Claude 4 splits the context into bands of relevance. Tokens from the last 100,000 positions receive full attention; older bands receive progressively lighter weighting. This design reduces memory bandwidth while preserving coherence on recent material.

Google’s hierarchical approach first summarizes every 32,000-token block into a compact vector, then attends over the vectors. The final pass re-expands only the top-ranked blocks. Accuracy on long-document question answering drops roughly 3–4 percent compared with uncompressed attention, according to internal Google papers released alongside the update.

Additional internal measurements indicate that power consumption per query rises by approximately 2.7 times when moving from 200,000 to 2 million tokens, even with the efficiency layers in place. This increase matters for organizations running thousands of daily queries, because electricity and cooling costs at the data-center level become material.

Real Limits Surface in Production Use

Enterprise pilots reveal two consistent constraints. First, output quality declines when source material exceeds 1.2 million tokens without additional structuring. Second, price per query rises faster than advertised once cache misses occur.

Finance teams testing 10-K filings found that models still required manual section prompts to locate specific clauses. Legal departments saw similar results on contract comparison tasks. The raw length alone does not replace targeted retrieval.

One asset-management firm loaded eight years of earnings transcripts into Claude 4. The model correctly identified revenue trends but occasionally attributed a 2022 margin expansion to the wrong operating segment. Analysts still export the relevant 50-page excerpts and feed them back for verification.

Cache behavior adds another variable. When the same 2-million-token context is reused across multiple questions, both providers cache the key-value store. A single cache hit reduces latency by 60 percent. However, any change to the prompt invalidates the cache, returning the user to the full price. Teams that iteratively refine questions therefore see costs fluctuate sharply.

A second asset-management firm discovered that rotating analysts across projects caused repeated cache invalidations, pushing effective cost per insight close to the uncompressed rate. The team now designates a single “context owner” per large dataset to minimize prompt changes.

Impact on Retrieval-Augmented Generation Systems

Traditional RAG pipelines that rely on embedding chunks and vector similarity search face fundamental redesign pressure. With 2-million-token windows available, the intermediate retrieval step can be bypassed entirely for many corpora that previously required complex chunking strategies. LangChain’s latest abstractions, for example, introduce “full-context retrievers” that pass entire document collections directly when token budgets allow, eliminating embedding model calls and vector database maintenance overhead.

Teams report measurable simplification in production code. One insurance analytics group removed three separate embedding models and their associated infrastructure after migrating to Claude 4’s long window. Query latency dropped 40 percent while recall on regulatory questions rose because the model now cross-references every clause without similarity-threshold filtering. However, the approach introduces new failure modes when the model must surface low-salience but critical details buried deep in dense text.

Practical Implications for Implementation

Enterprises adopting these windows must redesign ingestion pipelines. Instead of chunking documents into 4,000-token segments, teams now load complete files and add lightweight metadata headers that the model can reference. Prompt templates shift from “retrieve the most relevant paragraph” to “scan the entire record and extract every instance of clause X.”

Workflow tools such as LangChain and LlamaIndex already expose new abstractions. The latest release supports “full-context” retrievers that pass the entire corpus directly to the model when token count permits. Early adopters report simpler codebases because intermediate embedding steps disappear.

Cost modeling changes as well. Per-token pricing remains the headline rate, yet real spend depends on cache hit rate and output length. A team that asks ten follow-up questions against the same cached context effectively pays one ingestion fee plus ten small output fees. Budget forecasts must therefore incorporate usage patterns rather than raw token counts.

Change-management workshops have become common at companies adopting the new limits. Engineering leads are training prompt engineers on how to write “navigation prompts” that guide the model through massive corpora without losing thread. These workshops typically include hands-on labs where participants practice structuring metadata headers that survive across 2-million-token sessions.

Limitations and Risks

Several technical ceilings remain visible. Attention dilution still occurs beyond roughly 1.2 million tokens on complex reasoning tasks. Both providers therefore recommend that users keep core questions inside the most recent 800,000 tokens whenever possible.

Data-privacy regulations complicate full-context usage. When an entire customer database fits inside one prompt, every downstream query technically processes the complete dataset. Companies subject to GDPR must therefore implement stricter logging and retention policies for these sessions.

Vendor lock-in risk also rises. Organizations that build extensive prompt libraries around 2-million-token windows face migration costs if a competitor never matches the limit or changes pricing. Some teams deliberately maintain parallel shorter-window pipelines to preserve flexibility.

Finally, evaluation remains difficult. Standard benchmarks top out at 200,000 tokens. Teams must create their own long-context test suites, which introduces measurement noise and slows iteration cycles.

Comparisons With Other Providers

OpenAI currently offers a 1-million-token window on GPT-4o. The company has not announced plans to match the 2-million mark. Microsoft’s Azure OpenAI service surfaces the same 1-million limit with additional enterprise compliance controls. Smaller labs such as Mistral and Cohere still cap context at 128,000 tokens, focusing instead on lower latency and cheaper inference. NYTimes last week.

The competitive gap narrows on price. Anthropic charges $15 per million input tokens at the 2-million limit, while Google lists $12.50 under comparable conditions. OpenAI’s 1-million offering sits at $10 per million. Organizations that rarely exceed 500,000 tokens often stay with OpenAI to avoid the premium attached to longer windows.

Industry analysts note that Microsoft’s enterprise controls give Azure an edge for regulated sectors even at shorter context lengths, because compliance certifications and private networking options remain unavailable or less mature at Anthropic and Google for the full 2-million-token tier.

What to Watch Next

Watch Anthropic earnings call in July for updated context pricing tiers. Monitor Google Cloud reports on Gemini adoption after the window increase. Track whether OpenAI extends its 1 million token offering or holds at current limits.

Any announcement that moves pricing below current per-token rates will decide whether 2 million tokens becomes standard or remains a premium feature. Continued progress on sparse-attention research could also shift the economics, allowing even longer windows without proportional cost growth.

Continued progress on evaluation frameworks will determine adoption speed. Several academic groups are publishing new long-context benchmarks that reach 500,000 and 1-million tokens; once these become standardized, procurement teams will gain clearer signals on which provider delivers the best accuracy per dollar at scale.

How Context Window Growth Has Evolved

Context lengths have grown dramatically over three years. In 2022 most frontier models handled 4,000–8,000 tokens. By late 2023 the ceiling reached 128,000–200,000 tokens. The 2025 jump to two million tokens therefore represents more than a tenfold increase in just eighteen months. Each step required new attention mechanisms and hardware-aware training regimes, yet also exposed the same underlying trade-off between recall fidelity and inference economics.

Early adopters who built systems around 32,000-token limits now face obsolescence decisions. Some organizations choose to retain legacy pipelines for cost-sensitive workloads while routing only high-value queries to the newest long-context endpoints. Others rewrite their entire ingestion layer to exploit the expanded capacity.

Frequently Asked Questions

Does longer context always reduce hallucinations?

No. Quality still depends on how recent and relevant the material is to the query. Many teams still apply lightweight summarization on older bands to keep accuracy stable.

Can I mix long-context calls with traditional RAG on the same project?

Yes. Hybrid setups are common: the model receives the full recent corpus plus a small vector-retrieved supplement for older material that rarely changes.

Will every provider eventually offer two-million-token windows?

Analysts expect OpenAI and Meta to announce comparable lengths within twelve months, driven by enterprise demand, though pricing and efficiency layers will likely differ.