MiniCPM LES Brings 35B MoE to Phones Without Restricting Experts

MiniCPM says its LES scheduling technology helps a 35-billion-parameter model generate 20.51 tokens per second on an iPhone 18 Pro Max. MiniCPM LES addresses a specific obstacle: moving model weights into memory quickly enough without restricting which parts of the model can participate.

That distinction matters for MoE, or mixture of experts, an architecture that activates selected parameter groups instead of using the entire model for every token. MiniCPM-V 4.7’s 35B-A3B designation describes approximately 35 billion total parameters and 3 billion active parameters. Lower active computation does not eliminate the need to store and access the remaining weights.

The underlying contest is between preserving dynamic expert selection and reducing loading delays. Apple’s earlier flash-based inference research established a useful reference point: storage can extend limited memory, but data movement needs careful management. MiniCPM’s contribution targets expert scheduling within that broader problem. Its published results also require careful attribution: the highest phone speed combines several optimizations, and “lossless” needs a narrower interpretation than the headline suggests.

MiniCPM LES Turns Expert Loading Into a Scheduling Problem

The meaningful change is an attempt to preserve the model’s original expert choices while reducing the time spent waiting for their weights.

The team’s 35B model card describes a deployment approach that retains token-level routing, the process that selects experts for each generated token. It also reports a separate result of more than 13 tokens per second on an iPhone 17 Pro Max, using approximately 3 GB of memory.

That earlier phone result should remain separate from the newer iPhone 18 Pro Max benchmark. They concern different devices and documented configurations. Treating them as interchangeable would obscure what the public evidence actually establishes.

LES means Lookahead Expert Scheduling, a system that anticipates which experts the next model layer will need. Its purpose is to move loading work earlier. The prediction helps arrange computation; it does not replace the model’s final expert selection.

Consider the deployment problem from an application developer’s perspective. An assistant can have enough computational capacity to generate an answer yet remain slow because its next operation lacks the necessary weights. Making the arithmetic faster does little when arithmetic waits for data.

Keeping every expert in working memory avoids that particular delay, but demands more memory. Keeping weights in storage reduces the resident footprint, but introduces loading work. Restricting the available experts simplifies access while potentially changing the answer-producing computation.

MiniCPM’s proposed route preserves access to the full expert pool and manages when weights arrive. That creates a different engineering target. The question becomes whether enough necessary data can arrive during computation already underway.

This is why the scheduling announcement deserves attention beyond a model-size headline. A large parameter count on a phone says little about responsiveness by itself. The arrangement of loading and execution determines whether that capacity supports an interactive application.

For users, the distinction appears as continuity. An answer that starts producing text and then repeatedly pauses feels different from an answer with steadier output. Average generation speed captures part of that experience, but the sequence of delays matters too.

The practical significance therefore rests on the runtime, the software responsible for executing the model. Model weights alone cannot establish that every compatible application reproduces the same behavior. The relevant deployment combines the model, storage arrangement, scheduling implementation, and numerical configuration.

That combined system is the actual subject of the claim. It provides a concrete basis for investigation without requiring acceptance of every superlative attached to the release.

Preserving Expert Choice Puts Pressure on Mobile Deployment Shortcuts

The immediate pressure falls on deployment strategies that gain speed by narrowing the model’s available expert pool.

An expert restriction can look attractive when memory and loading time dominate an application’s budget. Fewer available weights mean fewer possible loading requirements. The difficulty is that an optimization affecting expert availability can also affect model behavior.

This creates a decision that developers need to make explicitly. Are they optimizing how the original model executes, or deploying a modified computational path? Both approaches deserve evaluation, but their quality claims require different evidence.

A scheduling system promises a useful separation. Developers can investigate execution efficiency while keeping the model’s selection mechanism intact. That makes it easier to identify whether a change affects speed, answer quality, or both.

It also changes the comparison with smaller models. A compact model that stays in memory can offer a simpler deployment path. A larger streamed model needs to justify its additional storage and runtime complexity through better results on relevant tasks.

Neither route wins through parameter count alone. An application that extracts a short field from a familiar document might benefit from predictable latency more than broader reasoning capability. A system interpreting unfamiliar diagrams might place greater weight on answer quality.

These are deployment scenarios, not verified customer outcomes for MiniCPM. They illustrate why the same benchmark can support different product decisions. The value depends on what users ask and what failures the application can tolerate.

The historical comparison is also instructive. Apple’s flash-based inference research investigated running models beyond available working memory through selective loading and more efficient storage access. Its methods differ from LES, but the shared constraint is data movement.

That precedent limits the novelty claim appropriately. Using flash storage to support larger models has an established research history. The specific contribution under examination concerns anticipating expert requirements and coordinating their execution.

For mobile hardware teams, this framing redirects attention toward the entire memory and storage path. Compute capability remains relevant, but a deployment can stall before reaching its arithmetic limits. Evaluating the processor in isolation misses that possibility.

For application teams, it raises a procurement question. A device with attractive compute specifications still needs testing against the intended runtime and workload. Storage behavior, available memory, and operating conditions become part of the application’s performance profile.

The competitive implication is therefore an engineering demand rather than a declared market victory. Alternatives need to show how they balance quality, responsiveness, and resource use under comparable conditions. MiniCPM needs to meet that same standard.

If the scheduling approach reproduces reliably, expert restriction becomes harder to justify as an automatic necessity. It remains a design choice whose quality cost and performance benefit should be visible.

How MiniCPM LES Preserves Routing While Hiding Loading Delays

LES uses prediction to prepare computation, while the actual router remains responsible for selecting the experts that run.

The official expert scheduling documentation describes cross-layer prediction followed by prefetching, which loads likely-needed weights before execution reaches them. Experts that were correctly anticipated run first. Other required experts load alongside that computation.

A missed prediction therefore creates additional waiting rather than permission to skip a required expert. This distinction explains the team’s lossless scheduling claim. It concerns preserving expert participation, rather than assuming predictions always match the eventual selection.

The documentation also describes arranging weights contiguously, reducing scattered reads, and applying layer-specific mixed precision. Quantization, the representation of weights with reduced numerical precision, lowers storage and transfer demands. These changes address different parts of the loading problem.

The mechanism suggests several useful implications, although they require measurement on actual deployments. Prediction accuracy alone cannot determine the total benefit. A correct prediction that arrives too late might save little waiting time.

Likewise, loading the right weights early helps only if the device can overlap useful work. If another resource becomes the limiting factor, the same scheduler can produce a different gain. Performance belongs to the complete execution environment.

Incorrect predictions also deserve attention. Prefetching something unnecessary consumes resources, even when it leaves expert selection unchanged. The useful question is whether avoided delays outweigh that extra work across representative requests.

This makes workload selection central to evaluation. A benchmark should expose situations where scheduling works well and situations where it struggles. An average without that distribution can conceal patterns that matter to application reliability.

For example, a document assistant might encounter familiar layouts during one session and unusual visual structures during another. Evaluators should investigate whether those changes alter responsiveness. The mechanism gives them a reason to ask; it does not establish the answer.

The approach also separates two predictions that readers can easily confuse. Expert prefetching anticipates which weights an upcoming computation will require. Speculative decoding, which drafts candidate output tokens for later verification, concerns the generation process itself.

MiniCPM integrates DSpark as its speculative component. That creates another opportunity to improve throughput, but it also makes attribution more important. An acceleration attributed broadly to LES can contain contributions from scheduling, storage access, numerical representation, and token drafting.

For developers, the conceptual advantage is clear enough to test. Preserve the required work, then rearrange preparation and execution to reduce idle time. The practical challenge is making that rearrangement beneficial under the device’s actual constraints.

That is a stronger and more specific proposition than claiming a large model simply fits inside a small memory allocation. It identifies the waiting time the implementation targets and the model behavior it says it preserves.

The 20.51-Token Result Combines Several Optimizations

The published phone result belongs to the full optimization stack, rather than lookahead scheduling alone.

OpenBMB’s on-device benchmarks list the iPhone 18 Pro Max with 12 GB of device memory. The detailed results begin at 3.38 tokens per second for the baseline. Lookahead scheduling reaches 6.57; additional scheduling and loading changes reach 11.34.

Mixed-precision quantization raises the reported result to 17.16 tokens per second. Adding DSpark brings it to 20.51, or 6.07 times the baseline. The corresponding fully optimized iPad Pro M4 result is 17.39 tokens per second.

Those figures support a concrete reading: multiple stages contribute to the outcome. They do not support describing the entire acceleration as the isolated effect of expert prediction. Developers trying to reproduce the headline need the relevant configuration for every stage.

The baseline deserves equal scrutiny. A comparison against an implementation without these optimizations demonstrates improvement over that implementation. It does not automatically establish superiority over every alternative runtime or deployment method.

This is a common interpretive problem with systems benchmarks. A large relative gain can coexist with a modest starting point. Readers need both the absolute result and a clear description of what changed.

The phone’s total memory also answers a different question from the model’s reported working-memory footprint. Device capacity describes the hardware environment. Application memory describes resources consumed within that environment, subject to the measurement method.

Persistent storage adds another dimension. The official quantized model files list the Q4_K_M variant at 21.2 GB. That listing illustrates why a low resident-memory claim should never be read as an equally small download.

The file listing does not establish that this exact artifact produced every published benchmark. Matching the benchmark to its precise weight files remains necessary. Otherwise, developers risk comparing different numerical representations under the same model name.

The distinction has direct product consequences. A mobile application must accommodate installation, model updates, available storage, and launch behavior. A configuration that generates efficiently after preparation still needs an acceptable experience before generation begins.

Decode speed measures output generation once decoding is underway. It should not be treated as the complete interval between submitting an image and receiving a useful answer. Input handling and preparation require their own measurements.

Similarly, tokens are units used by the model, rather than a fixed number of English words. A throughput figure provides a technical comparison under stated conditions. It does not translate directly into a universal reading or speaking experience.

The reported result is therefore meaningful but bounded. It shows what the team says its combined implementation reaches on named hardware. Independent testing should determine how that result transfers to realistic application sessions.

“Lossless” Needs a Defined Reference Point

Preserving expert routing supports a narrow technical claim; it does not establish identical behavior across every precision setting, task, and device.

The strongest interpretation of the scheduling description concerns selection. Required experts remain eligible, and a prediction miss does not remove them from computation. That property addresses a specific quality risk introduced by restricting the available expert pool.

The complete mobile deployment raises additional questions. Quantization changes weight representation. The appropriate quality comparison therefore needs to identify which reference model and numerical configuration define the baseline.

A claim about preserving routing should not silently become a claim about matching every output from a higher-precision model. Those statements concern different properties. Evaluators need evidence appropriate to each one.

Likewise, preserving a generation procedure does not establish that its answers are correct. A model can execute as intended and still misread a chart, misunderstand a request, or invent a detail. Runtime fidelity and task accuracy require separate evaluation.

No independent reproduction of the headline phone benchmark was established through the sources reviewed for this analysis. That limitation does not invalidate the reported result. It determines how confidently readers should generalize from it.

The same caution applies to the company’s “first” framing. Establishing priority requires a defined comparison set and consistent definitions. Terms such as smooth, real-time, and lossless need measurable boundaries before they support a universal ranking.

Real-time is particularly dependent on the application. A streamed written answer can tolerate delays that would interrupt a live interaction. A technical throughput threshold cannot establish suitability for every interface.

Longer sessions introduce another important test. Evaluators should measure repeated requests, changing device conditions, and competing application activity. These are questions about operational durability, rather than evidence that the reported implementation necessarily deteriorates.

Answer-quality testing should include failures that users can recognize. In a hypothetical expense workflow, that means checking whether extracted entries match the receipt. In document analysis, it means checking whether explanations remain grounded in the supplied page.

A broader knowledge workflow adds retrieval, the process of selecting relevant stored material for an answer. Readers comparing personal knowledge bases should evaluate that layer separately from model generation speed. Faster output cannot compensate for selecting the wrong source document.

Local execution also needs careful interpretation. It offers a deployment option for processing inputs on the device, but does not describe every data flow in a finished application. Application architecture determines what stays local.

These distinctions strengthen the analysis by making the claims testable. The question is no longer whether “lossless mobile AI” sounds plausible. It is which behavior remains unchanged, which resources the system consumes, and which measurements demonstrate the result.

MiniCPM LES gives developers a specific scheduling proposition to investigate. Acceptance should follow reproducible results and application-level quality checks, with each conclusion tied to the property actually measured.

Three Signals Will Show Whether the Approach Transfers to Products

The next evidence should connect reproducibility, sustained use, and implementation availability, in that order.

The first signal is an independent reproduction with identifiable artifacts and settings. Over the next few months, the useful result would include exact model files, runtime revision, prompts, and measurement boundaries. Those details would let other developers examine the same claim.

A reproduction should report more than its best run. Variation between requests matters when teams plan an interactive service. Consistent results would strengthen confidence that the published speed represents a usable configuration.

A lower result would need interpretation rather than immediate dismissal. Different weight files or settings might explain a gap. A reproducible mismatch under matched conditions would more directly weaken the original performance claim.

The second signal is sustained testing on representative workloads. Evaluators should follow complete requests from input submission through answer completion. They should also check quality alongside responsiveness, because faster incorrect output creates little product value.

This is where mobile MoE inference moves from a technical demonstration toward an application decision. A useful evaluation would show whether benefits persist during repeated document or image requests. It would also identify conditions where a smaller resident model remains preferable.

Cross-device results would clarify the scope. Similar behavior across additional hardware would support broader applicability. Results concentrated on a narrow configuration would indicate a more specialized deployment opportunity.

The third signal is accessible integration of the required optimization stack. Developers need a clear path from published weights to the implementation behind the benchmark. Documentation should make the necessary runtime and artifact choices unambiguous.

That availability matters because compatibility and equivalent performance are separate achievements. Loading a supported model does not guarantee that an application enables every scheduling and storage optimization. Reproducing the complete configuration is the relevant milestone.

Clear integration would strengthen the case that this is reusable infrastructure for mobile applications. Fragmented instructions or configuration-dependent results would make adoption harder, even if the underlying technique remains sound.

For teams considering a pilot, the appropriate next step is a bounded evaluation using material that resembles actual user requests. Include the difficult documents and awkward images that reveal whether added model capacity improves the answer. Measure the waiting time users experience, alongside resource use and correctness. Keep the reference configuration fixed while assessing each optimization, so improvements remain attributable. MiniCPM LES deserves attention because it offers a concrete way to preserve expert choice while reducing loading delays. Before building around the headline speed, ask whether that same configuration delivers reliable answers throughout your intended mobile workflow.