SK hynix Processing-in-Memory Takes Aim at AI Bottlenecks Beyond GPU and HBM

SK hynix has put processing-in-memory back at the center of its AI strategy, despite years of industry dependence on GPU and HBM systems. At AI Infra Summit 2026, the company showed an AiM chip, an AiMX accelerator card, and a server running an interactive large language model demonstration. The message was direct: moving every operation through a GPU is becoming an expensive bottleneck.

The September 16 presentation did not introduce PIM as a new invention. SK hynix announced its first GDDR6-AiM sample in 2022 and demonstrated an AiMX card in 2023. What changed is the company’s argument about where the technology belongs. SK hynix now positions it as a specialized partner to GPU and high-bandwidth memory infrastructure, especially for fast AI inference.

That distinction matters because the primary contest is not SK hynix against Nvidia or another chipmaker. It is the conventional GPU-HBM architecture against a heterogeneous design that sends memory-intensive work to dedicated memory hardware. The new approach promises to reduce data movement, but commercial adoption still depends on software, production economics, and independently verified performance.

What SK hynix showed in Santa Clara

SK hynix presented PIM as an operating part of a broader AI-serving system, not simply a laboratory memory chip.

The company participated in AI Infra Summit 2026 from September 15 through September 17 in Santa Clara, California. SK hynix said the event attracted roughly 6,000 attendees, about twice the previous year’s figure. Its booth also doubled in size and combined several memory technologies under a “New Spectrum” theme.

The PIM display included the company’s AiM memory chip, an AiMX accelerator card, and a server containing the card. Processing-in-memory, or PIM, places limited computational functions inside or closely alongside memory. It reduces the need to shuttle every piece of data between memory and a separate processor.

Visitors could interact with an LLM service running on the displayed system. That detail moves the presentation beyond a static chip sample, although a conference demonstration does not establish production readiness. The AI infrastructure display showed how SK hynix wants customers to understand the product: as a deployable component within a server.

Vice President Lim Eui-cheol presented the strategy during the conference’s Data Movement track. His session argued that increasingly diverse workloads no longer fit a single hardware design. Ultra-low-latency agents, long-context applications, and high-volume inference place different demands on memory capacity, bandwidth, and computing resources.

Lim said the conventional GPU-HBM architecture alone is becoming unable to satisfy all these requirements. This is a company position, not an independent verdict on current GPU systems. However, it explains why SK hynix brought three different technologies to the same event.

PIM targets fast decoding and memory-bound operations. High Bandwidth Flash, or HBF, is intended to place larger NAND capacity between HBM and SSD storage. SALT-KV is a software-led system for distributing an LLM’s key-value cache across HBM, DRAM, and SSD tiers.

A key-value cache stores previously calculated attention information during language-model generation. Its memory footprint grows as conversations become longer and more requests run together. Moving or retaining that data efficiently can influence inference speed and infrastructure utilization.

The portfolio therefore reflects one central judgment. AI serving will need several specialized memory layers instead of a single, uniformly fast pool. SK hynix is trying to turn its role from selling high-speed memory into designing where AI data should reside and where selected operations should execute.

That ambition also explains why the 2026 appearance should not be described as SK hynix’s first PIM unveiling. The company announced GDDR6-AiM in February 2022. It introduced its first AiMX prototype card in 2023 and doubled the card’s demonstrated capacity to 32 GB in 2024.

The latest event advances the narrative from component development toward system deployment. Yet SK hynix has not publicly supplied the shipment volumes, named customer deployments, or production schedule needed to confirm broad commercial adoption.

That gap creates the article’s central tension. SK hynix can show an integrated PIM server and explain a persuasive architectural problem. It must still prove that customers will rework software and infrastructure around the solution.

Why GPU and HBM systems face a data-movement problem

The constraint SK hynix is targeting appears when expensive processors wait for data rather than perform useful calculations.

Modern AI accelerators combine highly parallel processors with high-bandwidth memory, or HBM. HBM stacks DRAM dies and connects them through dense vertical links. Placing those stacks close to a GPU supplies substantially more bandwidth than conventional server memory.

This arrangement has become central to AI training and inference. However, additional processor throughput does not remove every memory constraint. A workload can remain limited by repeatedly reading model weights, attention data, or intermediate results.

Data movement consumes time and energy even when the mathematical operation itself is simple. A conventional system fetches data from memory, sends it to a processor, performs an operation, and writes the result back. Repeating that sequence across large models can leave arithmetic hardware underused.

The problem becomes particularly visible during LLM decoding. Decoding generates output tokens sequentially, so each step depends on preceding state. The process often performs relatively modest computation while reading large quantities of model and context data.

Training presents a different balance. It groups larger mathematical operations and can keep GPU units heavily occupied. Inference workloads vary more widely because batch sizes, context lengths, latency targets, and request patterns change from service to service.

SK hynix is using that diversity to challenge the assumption that every important operation belongs on the GPU. Its argument is not that general-purpose accelerators have become unnecessary. The company proposes offloading suitable memory-bound operations while leaving compute-heavy work on established processors.

That approach pressures several parts of the current system design. GPU operators must consider whether additional accelerator capacity is the most efficient answer to slow inference. Server vendors must decide whether another class of cards provides enough benefit to justify added complexity.

Cloud providers face a related question. A faster component has limited value if software cannot schedule work onto it reliably. Hardware utilization, model compatibility, system management, and maintenance can matter as much as theoretical bandwidth.

The economics also depend on what service a customer operates. A premium conversational agent prioritizing immediate token delivery has different needs from a batch service processing thousands of requests. Long-context retrieval introduces another memory pattern.

SK hynix says PIM is best suited to fast-decoding workloads in premium services. That qualification is important. It narrows the proposal to workloads where low latency and repeated memory access justify specialized hardware.

The company is not presenting one memory technology as the answer to every AI bottleneck. HBF addresses capacity and bandwidth for long-context use. SALT-KV decides where cached information should reside. PIM performs selected calculations near stored data.

This division challenges the prevailing preference for scaling a familiar GPU-HBM platform. Adding more GPUs and faster HBM remains operationally simpler when existing software already supports them. A heterogeneous design offers efficiency only after workloads are partitioned correctly.

Researchers continue to find the same tradeoff. A 2026 study of heterogeneous DRAM-PIM-GPU systems concluded that meaningful efficiency requires system-wide co-optimization. Its PIM design analysis found that static memory and idle GPU power can materially change apparent efficiency.

The study also found that workload mapping produced limited end-to-end gains in its tested configurations. Those results do not evaluate SK hynix’s 2026 server directly. They show why component-level measurements cannot predict complete application performance by themselves.

For enterprise buyers, this means PIM should be evaluated as a system architecture. The relevant question is not whether computation can occur inside memory. It is whether a supported application completes useful work faster and more efficiently after every overhead is included.

SK hynix processing-in-memory shifts selected work toward data

SK hynix processing-in-memory changes the location of computation, but it does not turn DRAM into a complete replacement for a GPU.

The company’s AiM design adds computational functions to GDDR6 memory. GDDR6 is a high-speed graphics memory standard that provides substantial bandwidth without using the stacked structure of HBM. Multiple GDDR6-AiM devices can be combined on the AiMX accelerator card.

In 2022, SK hynix said its first GDDR6-AiM sample operated at 16 gigabits per second and used 1.25 volts. Standard comparison memory cited by the company operated at 1.35 volts. The company also claimed up to 16 times faster performance for certain operations and an 80 percent reduction in power consumption.

Those figures came from SK hynix and described selected conditions. They should not be interpreted as universal gains across AI applications. The original GDDR6-AiM announcement did not establish how every production model, server, or software stack would perform.

The next step was AiMX. SK hynix demonstrated the first accelerator-card prototype in 2023 using Meta’s OPT 13B language model. The company said that system processed data more than ten times faster than a GPU-based comparison while using one-fifth of the power.

That claim included a significant condition. SK hynix said the performance assumed an application-specific integrated circuit for the card’s AiM Control Hub. The demonstrated result therefore depended on a particular controller implementation, not just the memory chips.

The first AiMX prototype nevertheless established the intended mechanism. Instead of asking a GPU to fetch every value, the card performs supported operations near the GDDR6-AiM devices. Reduced traffic can free the GPU for work that better matches its strengths.

SK hynix updated the prototype in 2024. The new card carried 32 GB, twice the capacity of the previous version. The company demonstrated it with Meta’s Llama 3 70B model and focused on multi-batch inference.

Multi-batch processing groups several requests for execution. It can improve utilization, but it also increases pressure on memory and scheduling. SK hynix presented AiMX as an attention accelerator within that environment.

Attention allows an LLM to relate the current token to relevant earlier tokens. It is essential to model behavior, but it involves moving and processing substantial quantities of stored state. That makes parts of attention an attractive target for near-memory acceleration.

The company also claimed its 2024 design could improve mobile LLM speed threefold over mobile DRAM at the same power level. Again, that was a company-reported comparison rather than a broad independent benchmark. The 32 GB AiMX demonstration did not announce mass-market availability.

At the 2026 conference, Lim supplied another comparison between PIM and static random-access memory, or SRAM. SRAM is fast but consumes considerable chip area and usually provides less capacity than DRAM.

According to a Korean report on the presentation, Lim said PIM could provide approximately 300 times the SRAM capacity within the same area. It could also retain high internal bandwidth. The conference remarks framed PIM as a way to avoid SRAM’s capacity and cost limits.

That comparison addresses a real architectural tradeoff. Fast local memory helps minimize latency, but large SRAM arrays are expensive and area-intensive. DRAM offers much higher density, although ordinary DRAM lacks the same direct computing ability.

SK hynix proposes separating the memory die from the computing die and connecting them vertically through hybrid bonding. Hybrid bonding joins polished surfaces and copper connections without conventional solder bumps. This can support denser links between independently optimized dies.

The separation matters because embedding substantial logic directly within a DRAM die can reduce memory capacity and raise manufacturing costs. A logic die can use a process suited to computation, while a memory die retains a DRAM-focused design.

However, hybrid bonding introduces its own production demands. Alignment, surface quality, heat, yield, and testing all influence cost. A technically elegant stack does not automatically become an economical volume product.

The mechanism is therefore more precise than the phrase “memory that computes” suggests. AiM handles supported operations near data. AiMX combines those devices into an accelerator. A host processor, GPU, memory hierarchy, controller, and software stack still coordinate the complete workload.

That limitation is also the source of the potential benefit. Specialized hardware does not need to replace every component to provide value. It only needs to remove enough repetitive data movement from an important workload.

The real opponent is a one-size-fits-all architecture

SK hynix is challenging the idea that faster GPU-HBM combinations should handle every stage of AI inference.

This is a route-versus-route competition. One route extends the dominant architecture by adding faster accelerators, larger HBM stacks, and additional systems. The other divides work across GPUs, PIM, storage, and software-managed memory tiers.

The first route benefits from a mature software base. Developers already use established GPU frameworks, optimized kernels, monitoring systems, and deployment tools. Model providers often publish configurations designed around those platforms.

The second route promises better alignment between hardware and workload. Memory-bound operations can run near data, while GPUs continue handling compute-intensive tasks. Slower but larger memory tiers can hold information that does not require immediate access.

That division resembles a specialized production line. Each component performs the job matching its characteristics. The resulting system can become more efficient, but coordination becomes harder.

SK hynix’s strategy pressures GPU vendors indirectly. If PIM cards serve a meaningful share of decoding, customers may need fewer GPU cycles for the same service volume. Yet successful PIM systems could also increase demand for GPUs by making complete AI services more economical.

For that reason, describing PIM as a direct GPU replacement would distort the company’s current presentation. SK hynix showed AiMX inside a server and discussed heterogeneous infrastructure. Its target is inefficient data movement, not the elimination of the processor.

Samsung offers the clearest competitive reference among major memory suppliers. It has developed HBM-PIM and demonstrated the technology with AMD accelerator hardware. Samsung has also explored processing-near-memory and PIM designs for on-device AI.

Samsung’s approach confirms that the industry sees computation near memory as more than an SK hynix experiment. Its HBM-PIM architecture offloads selected operations into HBM rather than using GDDR6-based AiM cards.

The difference in memory format does not create a simple winner. HBM provides very high bandwidth near an accelerator, while GDDR6 can offer different capacity, packaging, and cost characteristics. Customers will evaluate complete systems and supported workloads.

PIM developers also face a familiar standardization problem. Hardware becomes easier to adopt when software can target it without extensive rewrites. A design tied to one vendor’s custom libraries or narrow operator set risks remaining a specialist product.

Compilers must identify operations suitable for offloading. Runtimes must schedule those operations and manage data placement. Monitoring tools must explain failures and performance changes across several processing locations.

Models also evolve faster than server refresh cycles. An accelerator optimized for one attention pattern can lose relevance if new architectures change memory access. Reconfigurability and software updates therefore matter alongside raw silicon performance.

This is where SK hynix’s broader portfolio becomes strategically relevant. SALT-KV attempts to make placement decisions across memory and storage. HBF supplies another capacity tier. PIM handles selected operations.

Together, these technologies suggest that SK hynix wants to influence the full data path. That role extends beyond manufacturing memory devices with higher bandwidth. It requires system software, customer integration, and sustained developer support.

Engineering teams evaluating such hardware will need to preserve benchmark conditions, model versions, power measurements, and deployment findings. A searchable engineering knowledge base can keep those assumptions attached to later purchasing decisions.

The competitive question is therefore broader than which company builds the fastest memory. The winner must make specialized memory understandable and usable within existing AI operations. A component advantage without an adoption path will not change infrastructure design.

What the PIM demonstrations still do not prove

SK hynix has established technical continuity, but it has not yet established large-scale commercial adoption.

The AiM effort now spans several public milestones. SK hynix announced the memory sample in 2022, demonstrated a card in 2023, doubled its displayed capacity in 2024, and showed a server in 2026.

That progression is meaningful. It indicates continuing investment rather than a single conference announcement. However, public demonstrations cannot answer several questions that matter to buyers.

First, SK hynix has not disclosed a general production date for the 2026 PIM configuration. The latest presentation did not identify a hyperscale customer, contracted deployment, or confirmed shipment volume.

Second, the company’s most striking performance claims use selected comparisons. The 2022 figures covered certain computations. The 2023 AiMX result depended on an ASIC-based control hub condition. The 2024 demonstration focused on a specific model and processing configuration.

A fair evaluation requires matched hardware, identical model quality, equivalent precision, and complete system power. It should also include host processors, idle components, memory refresh, networking, and software overhead.

Latency should be measured across multiple batch sizes and context lengths. Throughput should include request scheduling and data transfers outside the card. Reliability testing should cover sustained operation, not only a controlled demonstration.

Third, PIM does not accelerate every part of an AI workload equally. Arithmetic-heavy operations can remain better suited to GPUs or dedicated tensor processors. Cross-device synchronization can erase gains when tasks are partitioned poorly.

Recent academic work reinforces that caution. System-level efficiency depends on channel configuration, static power, model structure, and workload mapping. A favorable kernel result can shrink after the full inference pipeline is measured.

Fourth, manufacturing economics remain uncertain. Adding logic and dense interconnects can increase design complexity. Separating memory and logic through hybrid bonding can protect memory density, but it adds packaging steps and yield considerations.

A chiplet-like structure also requires testing before and after assembly. A defect in one layer can affect the value of the combined package. Vendors must show that performance gains survive those production costs.

Fifth, software portability will shape customer willingness. Enterprises do not deploy an accelerator for one impressive benchmark. They need support for evolving models, common frameworks, security updates, observability, and predictable maintenance.

SK hynix’s claims should therefore be read as a technical case awaiting market validation. The company has shown that PIM can progress from a memory sample toward an interactive server. It has not shown that the resulting platform is ready for broad replacement cycles.

This distinction does not make the announcement unimportant. It defines the next standard of evidence. Another prototype with a larger number will matter less than a named deployment operating under real service conditions.

Three signals will show whether AiMX is becoming infrastructure

The next phase depends on customer evidence, reproducible system benchmarks, and a usable software ecosystem.

The first signal is a production commitment tied to a named system or customer. A shipment date, server qualification, or cloud deployment would move AiMX beyond repeated conference demonstrations. Volume information would strengthen that evidence further.

If SK hynix announces such a commitment, its memory-centric architecture will look closer to a commercial platform. Another showcase without deployment details would leave the adoption question unresolved.

The second signal is an independently reproducible benchmark. It should compare AiMX against a defined GPU-HBM system across multiple models, context lengths, and batch sizes. The results must include total server power and end-to-end latency.

A transparent test would strengthen claims that SK hynix processing-in-memory reduces real inference costs. Narrow component measurements or unmatched systems would weaken the comparison, even if their headline numbers appear large.

The third signal is broader software support. Buyers should watch for integration with common inference frameworks, documented operator coverage, profiling tools, and support from server partners. Public developer access would reveal how much model adaptation the platform requires.

Strong software integration would indicate that SK hynix understands adoption as more than a semiconductor problem. A closed demonstration stack would keep AiMX dependent on custom engineering.

These signals also provide a practical framework for technical buyers. Ask whether the exact model runs without quality changes. Measure the complete server rather than one card. Record which operations execute on PIM and which remain on the GPU.

Then test how performance changes as traffic, context, and batch size shift. AI services rarely stay within one carefully selected workload. Infrastructure must remain useful when models and user behavior change.

SK hynix has made a credible case that data movement deserves more attention. Its 2026 presentation also shows that PIM has advanced beyond the company’s first chip sample. The unresolved issue is whether customers will accept another accelerator and its software requirements.

Watch the first production deployment, the first reproducible system benchmark, and the first broadly accessible development stack. Those milestones will determine whether AiMX becomes part of AI infrastructure or remains an impressive demonstration.