AT&T Cuts AI Costs 90% With Model Routing and Open Models

AT&T says it has cut some artificial intelligence costs by as much as 90%, despite processing an average of 45 billion tokens daily. The claim reaching Google News readers is striking, but the mechanism matters more than the headline percentage.

The company did not find one cheaper model that could replace every premium system. It built a gateway that selects different models according to each request’s expected difficulty, speed, quality, and cost.

That approach challenges the assumption that enterprises should standardize on one frontier model from OpenAI, Anthropic, Google, or another major provider. AT&T is instead combining commercial systems with smaller open models trained for telecommunications work.

The strategy creates a clear tradeoff. Model choice can lower inference spending and reduce dependence on one supplier. It also creates a complicated operating layer that must evaluate quality, security, latency, and failures across many systems.

AT&T’s results remain company-reported, and the public evidence does not provide a complete independent audit. Still, its scale makes this more than another corporate AI experiment.

AT&T Put Its Model Router Into Production

AT&T’s important change is not simply that it adopted open models. It placed model selection between employees, applications, and the models answering their requests.

AT&T Chief Data and AI Officer Andy Markus described the strategy in a July 23 company post about its AI economics. The company processes an average of 45 billion tokens daily, according to that AI cost update.

A token is a small unit of text or data processed by a model. Token volume becomes a useful operating measure because most hosted AI services charge according to input and output consumption.

Sending every token to the most capable model would be simple. It would also waste expensive reasoning capacity on routine extraction, classification, summarization, and data-preparation tasks.

AT&T’s proprietary AI Gateway tries to prevent that waste. The gateway uses cache-aware routing, which considers whether reusable information already exists before choosing a model.

For every turn, the system weighs expected answer quality against speed and cost. It can also change models during a multi-turn conversation instead of keeping the entire session with one provider.

That mid-session capability is important. A conversation can begin with a basic lookup, move into specialized network analysis, and finish with a complicated recommendation.

A static assignment would treat those steps as one workload. Dynamic routing can send them to different systems according to their requirements.

AT&T says this process has lowered relevant AI costs by up to 90% and is already saving millions. Those statements describe production use, not a laboratory benchmark.

However, AT&T has not publicly provided the denominator behind the 90% figure. It has not disclosed the share of traffic receiving the maximum reduction or a detailed quality comparison.

The claim should therefore be read as an upper-bound company result. It should not be treated as a guaranteed saving for every application or enterprise.

Even with that limitation, the disclosed volume gives the result weight. Small routing improvements compound when a business processes tens of billions of tokens each day.

The system also reflects AT&T’s earlier architectural choices. Ask AT&T initially relied on OpenAI technology, but the carrier designed it to support algorithms from other companies.

That flexibility was visible when Ask AT&T launched in 2023. The internal assistant supported coding, customer service, translation, document search, network optimization, and legacy software work.

AT&T’s current gateway turns that optionality into an operating policy. Applications no longer need to make every model decision independently.

The gateway becomes a control point for routing, caching, measurement, and governance. It can apply consistent rules even as providers release new models or change their service terms.

That is why the story appearing through Google News is larger than a cost-cutting announcement. AT&T is treating model selection as enterprise infrastructure rather than an application setting.

Why Model Routing Changes Enterprise AI Economics

The router shifts competition away from finding one universally superior model and toward finding the least costly model that meets each task’s requirements.

Enterprise AI adoption often begins with a straightforward pattern. A team chooses a leading hosted model, connects company information, and expands access after a successful pilot.

Costs become harder to predict when usage moves beyond employees asking occasional questions. Automated workflows can generate repeated prompts, long contexts, tool calls, retries, and intermediate outputs.

Agents increase that pressure because one user request can trigger many model calls. A workflow might retrieve records, classify documents, create a plan, call software tools, inspect results, and revise its answer.

The user sees one completed task. The enterprise pays for every step required to produce it.

AT&T’s router addresses this problem before consumption occurs. It tries to select the appropriate model at each turn instead of analyzing overspending after an invoice arrives.

This is comparable to assigning work across a mixed team. Routine requests do not always require the most specialized expert, while difficult decisions still justify escalation.

The analogy has limits because a router must make that judgment automatically. A wrong decision can produce a weak answer before anyone recognizes that a stronger model was needed.

AT&T says its gateway estimates expected output quality alongside cost and speed. That quality threshold is the central component, even though the saving percentage attracts more attention.

Cost-only routing would consistently favor the smallest available model. Production routing must instead identify the lowest-cost option that remains acceptable for a specific task.

The resulting system can give enterprises more negotiating leverage. A company that can shift workloads among providers faces less pressure to accept one vendor’s technical limitations or commercial terms.

The strategy also changes how applications are built. Developers can request a capability or service level without permanently binding every feature to one model.

That abstraction can shorten future migrations. It can also let teams evaluate a new model against live workload categories before granting it wider access.

AT&T’s approach places frontier models in a narrower role. They become escalation targets for reasoning-intensive requests rather than the default engines for every interaction.

This does not mean frontier systems lose their value. It means their value must be matched to work that smaller models cannot complete reliably.

Independent observers see the approach as part of a wider enterprise pattern. AvidThink principal Roy Chua told Mobile World Live that leading enterprises use routing gateways for cost, security, caching, and auditing controls.

Futuriom founder Scott Raynovich argued that open models can support proprietary enterprise data without leaving companies dependent on the most expensive frontier systems.

Those comments support the architecture, but they do not independently validate AT&T’s reported saving. The distinction matters because architecture and measured business results are separate questions.

The router also concentrates authority. Whoever controls the gateway determines which models receive traffic, what quality thresholds apply, and how failures are handled.

That position is becoming strategically valuable. Model providers compete over intelligence, while routing layers observe demand across applications and providers.

A router can learn which model performs well for coding, customer support, telecom standards, fraud analysis, or document extraction. That performance history becomes useful operational data.

For enterprise buyers, the lesson is practical. Model benchmarks alone cannot explain the economics of a production AI program.

Workload composition matters just as much. An organization needs to know which tasks are common, which are expensive, and which truly require frontier-level reasoning.

Teams also need traceable evaluation criteria. Without them, routing can become an opaque mechanism that quietly trades answer quality for lower consumption.

A searchable AI workflow can help teams retain decisions, evidence, and outputs. That record becomes more important when several models contribute to one process.

Google News coverage can make the 90% figure appear like a single technical breakthrough. AT&T’s actual method is an ongoing discipline involving classification, evaluation, caching, and workload measurement.

Open Telecom Models Give AT&T Another Cost Lever

Routing lowers unnecessary consumption, while domain-specific open models give the router cheaper options that understand telecommunications terminology and operations.

AT&T paired its gateway with OTel 2.0, an open model family adapted for telecommunications. The company developed the models with industry data and a mixture of computing platforms.

A domain-specific model is trained or adapted for a bounded field rather than general conversation. Its narrower focus can improve performance on specialized vocabulary, documents, and operational questions.

Telecommunications presents a difficult test. Models may need to interpret technical standards, network topology, equipment telemetry, radio configurations, and multi-vendor systems.

A general model can discuss these subjects broadly. It may still lack the precision needed for network operations, where a plausible but incorrect answer can create serious consequences.

GSMA Intelligence argues that domain-adapted models can be smaller while remaining competitive on telecom tasks. Its OTel analysis frames AT&T’s strategy as an alternative to routing every request through a general frontier model.

AT&T says it post-trained OTel 2.0 using more than 400 billion tokens on AMD GPUs. The wider development workflow processed approximately one trillion tokens.

Microsoft says AT&T used about 530 GPUs through Foundry Managed Compute. That total included 430 AMD Instinct MI300X GPUs and multiple hardware architectures.

The model-development pipeline also distributed work across several open models. Microsoft’s Phi-4 handled data preparation and synthetic data generation, processing more than 700 billion tokens monthly.

Other models supported reasoning and development tasks. This mirrors the production gateway’s philosophy: one model does not need to dominate every stage.

Microsoft says open-model data generation saved tens of millions compared with using frontier models for the same development work. That is a partner-reported comparison, not a neutral financial audit.

Still, the Microsoft account provides technical detail beyond AT&T’s shorter announcement. It shows that model choice and hardware choice were treated as related decisions.

Open models can run on dedicated infrastructure instead of only through a vendor-controlled application programming interface. Companies can therefore evaluate different combinations of models, accelerators, clouds, and on-premises systems.

That flexibility supports AT&T’s stated interest in sovereignty. In this context, sovereignty means controlling data flows, deployment choices, and supplier dependencies.

It does not mean AT&T has eliminated outside technology. The project uses Microsoft infrastructure, AMD hardware, GSMA data, and underlying models created elsewhere.

The strategy is better understood as diversified dependence. AT&T is trying to prevent any one model, chip supplier, cloud, or development environment from becoming irreplaceable.

That position can improve bargaining power. If the company demonstrates comparable results across several platforms, switching becomes more credible during procurement discussions.

Open models also allow deeper domain adaptation. AT&T can train against approved telecom material and tune evaluation around network-specific tasks.

GSMA says the initial source material included standards and industry documents. Contributors included bodies and projects covering radio networks, interfaces, APIs, and telecom operations.

The model family is also intended for broader industry use. Making the output available to others can encourage shared testing, extensions, and specialized applications.

However, “open source” requires careful interpretation in AI. A release may provide weights and usage rights without exposing every training record, filtering decision, or development method.

Organizations evaluating OTel 2.0 must inspect its actual license, documentation, data disclosures, and deployment requirements. The label alone does not answer governance questions.

The size of the training workflow also shows that open does not mean free. Dedicated GPUs, engineering labor, security controls, evaluation, and ongoing inference all carry costs.

AT&T can justify those investments because it operates at unusual scale. A smaller company might spend more building and managing a model portfolio than it saves through routing.

That scale difference limits direct imitation. The transferable idea is not that every company should train a telecom model.

The broader lesson is to combine model choice with workload-specific evidence. An enterprise should use an open model where domain performance, control, and operating economics justify the additional responsibility.

The 90% Saving Still Needs a Quality Audit

AT&T has disclosed a credible mechanism and substantial operating scale, but it has not published enough evidence to independently verify the headline result.

The first uncertainty concerns scope. “As much as 90%” describes the strongest reported reduction, not necessarily the average across AT&T’s entire AI estate.

Public disclosures do not identify the baseline model mix. They also do not show how much traffic moved to smaller models, how often the router escalated, or how savings varied by application.

The second uncertainty concerns quality. AT&T says routing does not compromise quality, but readers cannot inspect task-level acceptance thresholds or evaluation results.

That omission is significant because quality is not one measurement. A customer-service summary, network recommendation, software patch, and fraud alert each require different standards.

A router can lower average spending by sending more traffic to smaller models. The result is only valuable if those models remain reliable for their assigned work.

Evaluation must also detect rare failures. An average score can hide a small number of costly mistakes in network operations or regulated workflows.

Human review remains important. When Ask AT&T launched, Markus said subject-matter experts still needed to verify generated code and other consequential outputs.

Routing does not remove that requirement. It adds another decision that teams must monitor, because an application may behave differently when the selected model changes.

Cache-aware operation introduces related concerns. Reusing prior computation can reduce consumption, but cached information must remain current, properly scoped, and protected.

A response suitable for one employee or customer cannot automatically be reused for another. Identity, permissions, geography, and data retention policies must follow the request.

Security teams must also evaluate every connected provider and model. A multi-model system creates more integrations, credentials, logs, policy boundaries, and possible failure modes.

The gateway can centralize those controls, which is an advantage. Yet centralization also makes the gateway a high-value target and a critical dependency.

If its classification logic fails, a sensitive task might reach an unapproved model. If the service becomes unavailable, many downstream applications can fail together.

Latency presents another tradeoff. The router must inspect enough information to choose a model, and complicated evaluation can delay the answer.

Caching can offset some delay, while smaller models may respond faster. The combined result depends on routing overhead, application design, and the selected model.

Vendor diversity creates operational work as well. Providers use different interfaces, context limits, tool-calling formats, safety controls, and update schedules.

A gateway can normalize part of that variation. It cannot guarantee that two models interpret every instruction or structured output format identically.

Model updates make the problem continuous. A route that performed well last month can become inferior after a provider changes behavior or releases a replacement.

AT&T must therefore maintain evaluations rather than certify each route once. Production traces should reveal failure rates, escalations, latency, and human corrections by workload.

Open-model deployments add supply-chain considerations. Teams need to track weights, libraries, licenses, model provenance, and vulnerabilities across the serving stack.

Data quality is another constraint. A telecom model trained on standards and synthetic material can still inherit gaps, outdated assumptions, or weak coverage of unusual equipment.

A leaderboard result offers useful evidence, but it does not reproduce every live network condition. Production evaluations must reflect AT&T’s own data, tools, and operational consequences.

The final uncertainty is organizational. A technically sound router cannot fix an unclear workflow or an application lacking accountable owners.

AT&T executives have emphasized end-to-end workflows instead of isolated AI tasks. That distinction matters because local savings can disappear when employees must repair weak outputs later.

Companies considering similar systems should calculate total process cost. The accounting should include model consumption, infrastructure, evaluations, engineering, security, review, and failure recovery.

They should also compare completed outcomes, not just tokens. A cheaper answer is not economical when it causes another call, another model run, or manual rework.

None of these risks disproves AT&T’s result. They explain why the reported percentage should begin a due-diligence process rather than end one.

For Google News readers, the responsible interpretation is narrow. AT&T says its gateway has produced reductions reaching 90%, but broader applicability remains unverified.

What to Watch After AT&T’s AI Cost Claim

Three signals will show whether AT&T has built a repeatable enterprise architecture or documented an unusually favorable set of workloads.

The first signal is a more detailed production scorecard. AT&T should disclose average savings across workload categories, not only the largest reduction.

Useful reporting would separate customer care, coding, document retrieval, network analysis, fraud detection, and autonomous workflows. Each category has a different quality threshold and model profile.

A strong scorecard would include escalation rates, latency, error rates, and human corrections. It would also show whether consumption keeps growing after unit costs fall.

If average costs remain controlled while volume expands, AT&T’s model-routing case becomes stronger. If savings concentrate in a small group of simple tasks, the claim becomes less transferable.

The second signal is independent evaluation of OTel 2.0. Industry leaderboards provide a starting point, but operators need tests tied to real network decisions.

Evaluators should compare the model with larger general systems on standards interpretation, topology analysis, telemetry reasoning, troubleshooting, and tool use.

They should also document where the smaller model fails. A trustworthy domain model needs clear escalation boundaries, not only high aggregate scores.

OTel 2.0 adoption outside AT&T would add another form of evidence. Other operators can test whether its reported advantages survive different networks, vendors, languages, and regulations.

Meaningful external deployment would support AT&T’s argument that specialized open models can serve an industry. Limited adoption would suggest the economics depend heavily on AT&T’s scale and internal data.

The third signal is how frontier-model and infrastructure suppliers respond. AT&T’s strategy pressures them to justify premium consumption for every workload.

Providers can answer with smaller models, lower-latency systems, stronger caching, automated routing, or more predictable enterprise contracts. They can also improve domain adaptation and private deployment options.

Competition at the routing layer deserves particular attention. Cloud companies, model providers, independent gateways, and enterprise software vendors all want to influence model selection.

If routing becomes standardized, enterprises can move workloads more freely. If each platform develops a proprietary router, supplier dependence may simply shift upward.

Hardware diversity will matter too. AT&T’s use of AMD and other architectures tests whether enterprises can avoid binding their AI economics to one accelerator roadmap.

Consistent performance across hardware would strengthen the sovereignty argument. Difficult migrations or uneven software support would expose the cost of maintaining that flexibility.

AT&T’s experience also gives enterprise buyers a practical checklist. They can begin by measuring requests according to task, risk, latency, quality, and completed business outcome.

Next, they can test whether smaller models satisfy narrowly defined workloads. Only then should they introduce automated routing between approved options.

Governance must arrive before broad traffic. Teams need access controls, logs, evaluation gates, fallback rules, incident procedures, and named owners for every production route.

The objective is not to maximize the share of requests handled cheaply. It is to minimize the cost of acceptable, completed work.

That distinction keeps the router aligned with business outcomes. It also prevents a cost target from quietly weakening customer service, network reliability, or employee trust.

AT&T has presented one of the clearest large-enterprise examples of this architecture. The combination of 45 billion daily tokens, dynamic routing, and domain-specific models deserves attention.

The company has not shown that every organization can reproduce a 90% reduction. It has shown why paying one model to handle every task becomes difficult at production scale.

The next phase will depend on evidence rather than another headline. Watch for average savings, external OTel 2.0 validation, and supplier responses to multi-model buying.

Those signals will determine whether the Google News claim becomes an industry playbook or remains an impressive result shaped by AT&T’s unusual scale. Enterprise teams should start measuring their own workload mix now, before choosing a router or training another model.