Apple Reopens a Hardware Argument: Retrospectively Reverse-Engineering Apple's Neural Engine
Apple's M1 Neural Engine has returned to scrutiny, despite a three-year pause in the Linux driver project created to understand it. Retrospectively Reverse-Engineering Apple's Neural Engine documents why that accelerator excelled at earlier neural networks but struggles with today's transformer workloads.
Eileen Yoon, a former Asahi Linux contributor, returned to the abandoned work after Apple changed its hardware direction. The M5 places a Neural Accelerator inside every GPU core while retaining a separate 16-core Neural Engine. That combination challenges the idea that one fixed-function accelerator should carry Apple's expanding generative AI workloads.
This is more than a delayed hardware teardown. The investigation connects choices made for 2017-era convolutional neural networks, or CNNs, with Apple's response to modern language models. GPUs now pressure the standalone Neural Engine because transformers depend on flexible scheduling and sustained memory movement.
A Dormant M1 Driver Became a Hardware Autopsy
The new work turns an unfinished Linux driver into an account of what Apple originally expected machine learning to become.
Yoon previously built a Linux ANE driver that could communicate with the Neural Engine inside M1 chips. The project included a kernel module, a userspace library, tests, and Python bindings. It offered a route around Apple's public software abstraction, although it did not make the hardware broadly programmable.
The project then went quiet for three years. Yoon writes that the ANE appeared too specialized to justify further driver work. Opening its hardware interface would not change which operations its fixed datapath could execute efficiently.
That distinction matters. A conventional driver can expose capabilities that the hardware already contains. It cannot turn a purpose-built dataflow accelerator into a general processor.
Yoon's architecture retrospective therefore asks a different question from the original driver effort. Instead of asking how Linux can submit work, it examines the compute array, scheduler, memory system, and execution model. Those components reveal the workload assumptions embedded in the M1 design.
The investigation describes 16 compute cores arranged around a central block of local memory. Each core contains parallel multiply-accumulate units, or MACs, which multiply inputs and add the results into accumulators. These operations underpin convolutions, matrix multiplication, and the dot products used by attention mechanisms.
The MAC units do not explain the Neural Engine's specialization by themselves. CNNs and transformers both need multiplication and accumulation. The decisive difference lies in how weights and activations reach those units, remain available, and move through the chip.
Apple introduced its first Neural Engine with the A11 Bionic in 2017. At that time, consumer neural networks centered on image classification, facial analysis, and other dense CNN workloads. Those networks offered regular tensor shapes and predictable reuse.
The M1 inherited that design lineage when Apple brought its own processors to the Mac. Its Neural Engine was optimized for compiled models whose dimensions and movement patterns were substantially known in advance. That specialization reduced latency and energy consumption for supported work.
The retrospective does not claim that the M1 Neural Engine cannot execute transformer operations. It argues that the surrounding dataflow makes some transformer patterns inefficient, especially autoregressive decoding. That process generates one token at a time while repeatedly reading model weights and a growing attention cache.
This reframing creates the article's central tension. Apple built an efficient engine by constraining data movement around an expected workload. Modern AI changed the dominant workload faster than a fixed hardware architecture could change.
Retrospectively Reverse-Engineering Apple's Neural Engine Exposes the Real Constraint
The M1 Neural Engine's defining limitation is not arithmetic capacity; it is the route that data must follow around that arithmetic.
The reverse-engineered driver never sends high-level commands such as CONV, MATMUL, or RELU directly to the hardware. Apple's compiler has already converted those neural operations into task descriptors before execution begins.
A task descriptor is a structured block of configuration data. It programs register groups controlling tensor dimensions, memory addresses, activation functions, dependencies, and data transfers. The driver places the descriptor in memory, points the task manager toward it, and rings a hardware "doorbell."
After that submission, the Neural Engine controls the job until completion. It raises an interrupt when the task finishes. The host processor does not direct each mathematical instruction while the operation is running.
Yoon concludes that the ANE lacks an instruction set in the familiar CPU or GPU sense. Its task descriptors configure a domain-specific datapath rather than supplying an arbitrary program. Each descriptor represents one pass through that datapath.
The task begins by loading configuration registers. Dedicated transfer blocks then move weights and input activations from main memory into separate local stores. The compute cores perform reductions, post-processing applies an activation, and another transfer block returns the result.
That sequence favors operations with predictable reuse. A convolution can apply the same compact set of learned filters across many image regions. The accelerator can keep its arithmetic lanes occupied without repeatedly retrieving a large new set of weights.
Transformer decoding changes that balance. Each new token can require streaming through a substantial portion of the model's parameters. The arithmetic remains recognizable, but data movement becomes the limiting cost.
The M1 layout compounds the issue because it separates memory used for weights from local memory used for activation tiles. Yoon identifies approximately 1 MiB of kernel memory and 2 MiB of tile memory in the design. The investigation argues that the architecture was not arranged to reinterpret locally produced tensors as weights efficiently.
That was a reasonable assumption for the models Apple targeted in 2017. CNN inference generally treats learned weights as fixed kernels and activations as data flowing between layers. Transformer attention blurs that distinction because values produced during execution can feed later matrix operations.
A key-value cache illustrates the problem. The cache stores representations of earlier tokens so the model can reuse them during generation. Its contents grow as the conversation or document gets longer, making efficient memory access increasingly important.
Fixed tensor dimensions are not the central obstacle. A compiled task can loop over a changing cache length, and task submission overhead can remain small. The harder issue is repeatedly feeding data through pathways designed around CNN reuse.
This is why raw operations-per-second figures offer an incomplete comparison. Peak arithmetic throughput describes how quickly the MAC array can work under favorable conditions. It does not reveal how often those units wait for weights, activations, or intermediate results.
Independent research has reached a compatible conclusion from a different direction. The 2026 Orion research paper describes 20 constraints encountered while programming the ANE through private interfaces. Its authors identify compilation, memory layout, and numerical behavior as practical barriers.
Orion still reports meaningful transformer results. On an M4 Max, the system produced more than 170 tokens per second for GPT-2 with 124 million parameters. It also trained a 110-million-parameter model for 1,000 steps in 22 minutes.
Those measurements show that the ANE can run language-model workloads. They do not establish it as the best target for every large model or every stage of inference. The distinction between technical possibility and architectural fit remains essential.
Apple's Public Software Keeps the Hardware at Arm's Length
Developers can request the Neural Engine through Core ML, but Apple still controls how workloads are compiled, divided, and dispatched.
Apple exposes the Neural Engine primarily through Core ML, its public framework for deploying machine-learning models. Developers provide a compatible model, while the framework decides whether individual operations should use the CPU, GPU, or Neural Engine.
Apple's compute-unit controls let an application permit combinations of those processors. A developer can allow every available unit or exclude the GPU or Neural Engine. The public interface does not offer direct programming of the ANE's task descriptors.
This model protects portability across Apple devices. An application can describe the prediction it needs without encoding the register layout of one chip generation. Apple can revise compilers and scheduling policies while keeping the application interface stable.
The tradeoff is visibility. Developers cannot rely on Core ML to place every supported operation on a specific engine. They also cannot inspect the final low-level program with the control expected from GPU compute APIs.
Apple explains that Core ML execution can use the CPU, GPU, and Neural Engine while reducing memory consumption and power use. This approach is appropriate for applications seeking efficient on-device inference without hardware-specific tuning.
It is less satisfying for researchers investigating architectural limits. A benchmark might fall back to another processor, split a graph across processors, or encounter compiler transformations that obscure the underlying hardware behavior.
The reverse-engineering work removes part of that uncertainty. It examines task descriptors, register writes, queues, interrupts, and memory pathways below Core ML. That view helps distinguish restrictions imposed by software from constraints created by the silicon.
However, private interfaces create their own uncertainty. They lack Apple's public compatibility guarantees and can change with an operating-system update. Research code that works today might fail after the compiler, model format, or runtime service changes.
This gap places Apple in an unusual position. The company ships dedicated machine-learning hardware across phones, tablets, Macs, and headsets. Yet independent developers have limited control over one of the most distinctive blocks inside those processors.
For mainstream applications, that limitation can be an intentional product choice. Apple optimizes the complete device and decides where each operation runs. Most developers benefit more from predictable deployment than from direct register access.
Generative AI developers often need the opposite. They experiment with quantization formats, attention kernels, cache layouts, and fused operations. They also compare performance across rapidly changing model architectures.
GPUs accommodate that experimentation because their programming models expose more general compute. Developers can implement new kernels without waiting for a dedicated compiler path. The cost is more responsibility for synchronization, memory access, and performance tuning.
The standalone Neural Engine represents the other end of that spectrum. Its compiler and fixed datapath can deliver efficient execution when a model fits. When the workload shifts, specialization becomes a constraint instead of an advantage.
Apple's software strategy prevents developers from directly resolving that tension. Core ML can hide hardware variation, but it cannot make a CNN-oriented memory system behave like a flexible GPU. Reverse engineering exposes the boundary that the framework normally conceals.
The M5 Makes the GPU the Main Opponent
Apple's M5 does not remove the standalone Neural Engine, but it gives the GPU a more direct role in the company's AI roadmap.
Apple announced the M5 in October 2025 with a 10-core GPU containing a Neural Accelerator in each core. The chip also retained an improved 16-core Neural Engine. That design places specialized matrix hardware on both sides of the architectural contest.
According to Apple's M5 chip announcement, the new GPU provides more than four times the peak AI compute performance of the M4 GPU. Apple also raised unified memory bandwidth to 153GB/s, nearly 30 percent above M4.
Those are Apple-controlled measurements, and workload details determine real application performance. Still, the location of the new accelerators is more revealing than the headline multiplier. Apple put them inside programmable GPU cores rather than relying solely on the separate Neural Engine.
The GPU combines specialized matrix execution with an environment already suited to changing algorithms. Apple says developers can program the Neural Accelerators through tensor APIs in Metal 4. That provides a public path toward GPU-based AI workloads without exposing the standalone ANE's private command format.
Yoon interprets this shift as the beginning of the end for the standalone NPU, or neural processing unit. The wording is intentionally provocative, and Apple's product decisions do not yet confirm an actual retirement.
The M5 still includes a separate Neural Engine. Apple describes it as faster and associates it with system features, including photo processing and spatial Persona generation. These tasks resemble the bounded, predictable inference workloads that dedicated accelerators handle well.
The more defensible conclusion is narrower. Apple now treats the GPU as a primary destination for demanding generative AI workloads, while the Neural Engine retains a role in efficient system inference.
That division matches the reverse-engineered evidence. A GPU can combine matrix acceleration with flexible memory operations, general kernels, and direct developer access. A fixed-function NPU can minimize overhead for stable graphs with known execution patterns.
Neither design wins every workload. A general processor spends area and energy supporting flexibility that a fixed pipeline avoids. A specialized engine loses adaptability when models demand new movement patterns.
The M5 suggests Apple wants both. Its separate Neural Engine can support established on-device features, while GPU Neural Accelerators target models whose operators and memory behavior continue changing.
This hybrid strategy also pressures Apple's software stack. Core ML must choose among increasingly capable processors. Metal must give developers enough control to take advantage of the new GPU units. The compiler must avoid moving data between processors so often that transfer costs erase acceleration.
The pressure is therefore not simply Nvidia versus Apple, or macOS versus Linux. It is a contest inside Apple's own silicon between fixed-function efficiency and programmable acceleration.
That contest began long before generative AI. Apple's chips already divide work among CPUs, GPUs, media engines, image processors, and security hardware. The difference now is that AI model design changes at a pace that makes long hardware planning cycles especially risky.
A dedicated block can take years to design and validate. Transformer architectures, attention variants, and quantization techniques can shift within months. Embedding more adaptable acceleration in the GPU reduces the cost of guessing incorrectly.
Reverse Engineering Does Not Prove the Neural Engine Is Finished
The retrospective explains an architectural mismatch, but it cannot establish Apple's future product plan or measure every newer ANE implementation.
The deepest findings concern the M1 generation. Apple has shipped several processor families since then, and internal details can change without public documentation. Conclusions about later chips require direct measurements rather than visual similarity or marketing names.
Yoon acknowledges uncertainty in parts of the physical-layout analysis. Die images can reveal major memory blocks and repeated compute structures, but they do not explain every routing decision. Some conclusions remain informed interpretations.
The investigation also focuses on hardware structure rather than a comprehensive application benchmark. It shows why memory movement should limit certain workloads. It does not compare every model across the ANE, GPU, and CPU under identical power limits.
Orion provides useful newer measurements, but it uses private APIs and research software. Its GPT-2 and TinyStories experiments demonstrate access and capability, not broad production readiness for current large language models.
Another open project has reported direct training through reverse-engineered private interfaces. Its M4 measurements place FP16 throughput around 18.6 trillion operations per second and INT8 throughput around 35.1 trillion operations per second. These figures depend on selected convolution configurations and should not be generalized to complete models.
Software maturity matters as much as hardware. A highly optimized compiler can restructure graphs, fuse operations, and reduce transfers. A research driver might expose the engine correctly while leaving much performance unused.
The opposite risk also applies. Peak microbenchmarks can keep arithmetic units busy under ideal conditions while hiding real model bottlenecks. End-to-end latency, memory use, energy consumption, and compilation time determine whether an accelerator helps an application.
Apple could also redesign the standalone Neural Engine while keeping its product name. Larger shared memory, revised data paths, or new task formats could address limitations found in the M1. The M5 announcement does not disclose that level of detail.
Security presents another reason for controlled access. Apple uses Neural Engine hardware in protected biometric workflows. Its platform security documentation describes state resets and memory controls for secure Neural Engine operation on newer systems.
That role does not require opening the same hardware to arbitrary Linux workloads. It also means the block's continued presence can reflect system architecture beyond consumer language models.
Power efficiency remains another missing comparison. Autoregressive decoding might run more naturally on programmable GPU hardware, yet a dedicated engine can still outperform it for vision, audio, and classification tasks. Apple sells battery-powered devices where those savings matter.
The credible claim is therefore not that the Neural Engine has died. It is that its original design assumptions no longer cover the full range of strategically important AI workloads.
That distinction keeps Retrospectively Reverse-Engineering Apple's Neural Engine grounded in evidence. The project illuminates an architectural fork, while Apple's product releases will determine how far the company follows either route.
Three Signals Will Show Which Architecture Wins
Apple's next APIs, benchmarks, and chip layouts will reveal whether the M1 Neural Engine was a durable template or a specialized branch.
The first signal is developer access to the M5 GPU's Neural Accelerators. Metal 4 needs to expose useful tensor operations without hiding so much scheduling that researchers face another black box.
Working tools will strengthen the case that Apple has chosen programmable GPU acceleration for fast-changing models. Limited APIs or narrow operator support would weaken that interpretation and preserve a larger role for Core ML-managed hardware.
The second signal is end-to-end performance across representative transformer workloads. Useful comparisons must include prompt processing, token generation, long-context memory behavior, energy use, and model-loading time.
Microbenchmarks alone will not settle the issue. A processor can lead in matrix throughput while losing time to weight transfers, cache movement, or graph compilation. Measurements should also identify which compute unit executed each operation.
Results from M5 applications will be especially important. Local language models and diffusion software can test the new GPU accelerators under workloads Apple explicitly highlighted. Consistent gains would validate the move toward acceleration inside programmable cores.
The third signal is the architecture of Apple's next standalone Neural Engine. Apple can keep the 16-core label while changing memory size, interconnects, scheduling, and supported precision beneath it.
A redesigned local memory system would challenge the idea that the standalone block is approaching its end. Minimal changes, combined with larger GPU investments, would support Yoon's interpretation.
Linux progress offers a secondary verification path. The Asahi community has extensive experience documenting Apple silicon through clean-room observation and experimentation. Its reverse-engineering work has already produced open drivers for other undocumented blocks.
A usable ANE driver would let researchers compare Core ML's choices against direct task submission. It could also reveal whether alternative compilers can recover performance that Apple's public framework leaves inaccessible.
Yet Linux support should not be mistaken for the main commercial outcome. The driver matters because it turns hidden hardware behavior into testable evidence. Apple's own devices, frameworks, and workloads will decide the architecture's future.
Developers should watch where Apple places new public programmability. They should also separate theoretical throughput from complete application performance. The processor with the largest headline number is not necessarily the one moving model data efficiently.
Teams conducting similar technical investigations need a durable record of experiments, register findings, benchmarks, and rejected hypotheses. A searchable engineering knowledge base can keep that evidence connected as tools and chip generations change.
Retrospectively Reverse-Engineering Apple's Neural Engine ultimately captures a rare moment when old silicon explains a new strategy. The M1 shows the benefits and costs of committing CNN assumptions to hardware. The M5 shows Apple adding flexibility without immediately abandoning specialization.
The next question is concrete: will future Apple chips expand programmable GPU acceleration while leaving the Neural Engine to stable system tasks, or will Apple rebuild the standalone block for transformers? Watch the APIs and memory behavior, not just the TOPS figure.
