Kimi K3 Runs on an M1 Max, and openai max Is the Wrong Frame

Kimi K3 now runs on a 64GB M1 Max at a reported 0.0687 token per second, reframing what an openai max search should surface. Deltafin, a small research project, executes Moonshot AI’s 2.8-trillion-parameter model without loading its entire weight set into memory. The result is technically remarkable and practically glacial.

The project does not turn a 2021 Mac into a competitive AI server. Its median decoding time is 14.6 seconds per token with a complete local installation. A moderately long response can occupy the machine for hours, while an agentic coding session remains more demonstration than workflow.

That gap between possible and usable is the story. Deltafin challenges the assumption that enormous models require enormous memory capacity. It does not challenge the responsiveness of hosted systems from OpenAI, Anthropic, Moonshot, or other infrastructure providers.

The experiment instead exposes a different frontier. Open weights let independent developers redesign inference around storage, memory pressure, and sparse computation. The resulting system expands where a model can run, even when it cannot deliver the speed users expect.

Deltafin Gets 2.8 Trillion Parameters Through 64GB

Deltafin changes the meaning of “running locally” by separating model access from model residency.

Moonshot AI introduced Kimi K3 on July 16, 2026. The company describes it as a 2.8-trillion-parameter, natively multimodal model with a one-million-token context window. Its published architecture activates 104 billion parameters during a forward pass rather than using every parameter for every token.

That design is called a mixture of experts, or MoE. An MoE divides parts of a neural network into specialized parameter groups and routes each token through a limited selection. Kimi K3 contains 896 routed experts, with 16 selected at each relevant layer.

The distinction makes Deltafin possible. A dense 2.8-trillion-parameter model would need to read and process the full parameter pool for every token. Kimi K3’s sparse routing lets the runtime retrieve only the experts chosen by the model’s router.

Deltafin’s maintainer reports that the published Kimi K3 weights occupy about 1.56TB. Roughly 114GB forms the model’s resident spine, including attention components, embeddings, shared experts, and latent projections. Another 1.45TB contains 82,432 routed experts.

Neither group fits inside 64GB of unified memory in its original form. Deltafin therefore treats fast local storage as another level in the memory hierarchy. It reads the spine layer by layer and loads selected experts from disk as each token moves through the network.

The project’s Deltafin repository reports a median steady decoding rate of 0.0687 token per second. That equals 14.6 seconds per token, or approximately 4.1 tokens per minute. The measured range across six full-model runs was 0.0503 to 0.0779 token per second.

Those numbers came from one 64GB M1 Max with a 10-core CPU, 32-core GPU, and internal solid-state storage. The full model was stored locally. The test used greedy decoding, exact numerical settings, an int8 spine, and tracing disabled.

The benchmark prompt contained five tokens. Deltafin produced a verified three-token continuation, with the initial decoding step excluded from the steady-rate calculation. The median model time for those three generated tokens was 56.5 seconds, while fresh-process wall time reached 64.1 seconds.

This is not an independent benchmark or a representative cross-device average. It is a maintainer-run reference on one machine. The repository publishes its configuration and run range, but broader replication remains limited.

Even with that caveat, the experiment crosses a meaningful boundary. The original working version reportedly needed about 20 minutes per token. Deltafin’s current result represents an approximately 82-fold improvement over that internal starting point.

The improvement turns an almost static demonstration into observable generation. It still does not produce an interactive conversation. That distinction drives every practical judgment about the project.

Why the M1 Max Can Participate but Cannot Keep Up

The M1 Max contributes useful compute, but Deltafin is fundamentally constrained by moving weights through storage.

Apple announced the M1 Max in October 2021 with support for up to 64GB of unified memory. Unified memory gives the CPU and GPU access to one shared pool, reducing some copying between separate memory spaces.

Apple also specified up to 400GB per second of memory bandwidth for the chip. Those characteristics help local AI software avoid the rigid split between system memory and discrete GPU memory. They do not make terabytes of model weights fit into 64GB.

Deltafin works around capacity rather than eliminating the mismatch. It converts the resident spine to int8, an eight-bit representation that reduces its storage requirement from roughly 114GB to around 60GB. Selected layers can then pass through memory while other data remains on disk.

The routed experts use MXFP4 weights, a four-bit floating-point format intended to reduce model size and data movement. Moonshot says Kimi K3 received quantization-aware training from supervised fine-tuning onward. That means low-precision behavior was considered during training instead of being added only after release.

The official Kimi K3 code recommends server-oriented inference engines such as vLLM and SGLang. Deltafin follows a different path, adding native kernels and storage-aware execution for hardware far below a conventional deployment target.

For every generated token, the router selects 16 experts across 92 routed layers. Deltafin says that process reads about 25.8GB of expert data per token. Local storage can serve those reads in seconds, while network retrieval can take minutes.

This explains why the M1 Max’s headline memory bandwidth does not determine final performance. The GPU cannot process data that has not arrived. Disk latency, storage throughput, decompression, weight preparation, and synchronization all enter the critical path.

The project uses background loading and expert prefetching to overlap some work. Memory mapping lets the runtime access expert data without repeatedly copying entire files. Native Metal kernels perform the selected expert computation on Apple’s GPU path.

Deltafin also includes fused kernels, which combine multiple operations to reduce intermediate data movement and launch overhead. These optimizations matter because small inefficiencies repeat across dozens of layers for every token.

However, no kernel can erase the 25.8GB expert-reading requirement while preserving the same routing and weights. The full local installation shifts that traffic to an internal drive. Streaming shifts misses to a remote host and magnifies latency.

Apple’s original M1 Max specifications provide an important reality check. The chip was designed for high-performance laptop workloads, including graphics and media processing. It was not designed to act as memory for a 1.56TB language model.

The result is an impressive use of the machine’s architecture, not evidence that hardware limits have disappeared. Deltafin keeps the computation moving by repeatedly exchanging capacity for time.

That trade explains why the experiment matters to systems researchers. It shows that “too large to load” is not identical to “impossible to execute.” It also shows that execution alone is an incomplete measure of deployment viability.

The openai max Comparison Is Possibility Versus Responsiveness

Deltafin competes with the assumption behind hosted inference, not with the experience hosted inference delivers.

A search for openai max suggests a user looking for greater model capability, higher reasoning effort, or an upper product limit. Deltafin answers another question: what is the largest open model a determined developer can force through one workstation?

Hosted AI systems optimize for a service contract. Users expect prompt processing, generated tokens, concurrency, availability, and predictable latency. Providers distribute that work across accelerators and supporting infrastructure that ordinary users never manage.

Deltafin optimizes for local executability. Its goal is to preserve Kimi K3’s published weights and model path while operating under extreme memory constraints. That is a research target, not a substitute for a responsive commercial endpoint.

The project exposes an OpenAI-compatible server, meaning its HTTP endpoints follow familiar request and response shapes. It implements chat completions, text completions, model listing, and streamed output. Developers can point compatible clients at a local base URL.

Compatibility does not imply equivalent behavior. Deltafin handles one generation request at a time. A second concurrent request receives an HTTP 429 response. Temperature and top-p parameters are accepted but ignored because the current system uses greedy decoding.

The maintainer recommends setting client timeouts in hours rather than seconds. That advice captures the gap more clearly than any architecture diagram. A familiar API can hide interface differences, but it cannot hide physical waiting time.

Coding agents illustrate the problem. Such tools often send long system prompts, repository context, tool definitions, and conversation history before requesting the first useful action. Deltafin warns that these prompts make prefill, the initial processing of input tokens, especially expensive.

A coding assistant may also require many sequential model calls. One response proposes a command, another interprets its result, and later calls revise files or inspect tests. At 14.6 seconds per generated token, latency compounds across every step.

Moonshot’s own deployment supports OpenAI-compatible and Anthropic-compatible interfaces. Its model documentation says Kimi K3 always uses reasoning and returns a separate reasoning field. The default reasoning effort is “max,” with lower and higher settings also available through the API.

That official service is the relevant reference for someone evaluating everyday Kimi K3 use. Deltafin is the relevant reference for someone studying local execution, sparse routing, weight streaming, or reproducible inference.

Privacy offers another distinction. A full Deltafin installation can generate tokens without network access after downloading the weights. Prompts and generated text can remain on the workstation, assuming the user also controls connected applications and logging.

That property can interest researchers working with sensitive drafts or proprietary code. Yet local processing does not automatically make a system suitable for regulated or production data. Operators must still examine dependencies, access controls, logs, model licensing, and application behavior.

The Kimi K3 paper describes a model with native visual capability and a one-million-token context window. Deltafin’s benchmark does not establish practical performance across that full context, multimodal inputs, or sustained agentic workloads.

This is why an openai max comparison should remain narrow. Deltafin expands the set of hardware that can execute a frontier-scale open model. It does not match the latency, concurrency, operational maturity, or convenience of a hosted service.

The pressure falls less on commercial APIs than on conventional deployment assumptions. Infrastructure developers can no longer treat RAM capacity as the only hard boundary. Storage-aware runtimes now provide another route, although it comes with severe latency.

Full Installation and Streaming Produce Two Different Experiments

Deltafin’s two installation modes prove that storage location matters almost as much as model size.

The recommended full installation needs approximately 1.7TB of local disk space. Deltafin estimates a download time between five and ten hours, with resumable transfers. Once installed, inference no longer needs network access.

This configuration produced the reported 14.6-second median token time. It stores the entire expert pool locally, allowing each routed expert to be read from disk instead of fetched over HTTP.

The streaming installation needs approximately 215GB. Its initial download takes around 30 minutes according to the project. Missing experts are fetched from the Kimi K3 files on Hugging Face and added to a growing local cache.

That lower entry requirement carries a brutal performance penalty. Deltafin estimates more than three minutes per token when the required experts are not already cached. Chat prefill can take hours because a prompt touches many experts before generation begins.

A 60-token input is not unusual for even a minimal chat template. Production assistants often send thousands of tokens. Streaming mode can therefore spend substantial time downloading experts before the user sees the first generated token.

The cache makes repeated paths faster when future prompts route through experts already stored locally. However, sparse routing is input dependent. A cache warmed by one task does not guarantee that a different task will reuse the same expert pattern.

Deltafin includes an idle-time warming tool that ranks missing experts using recorded router traces. The mechanism can prefetch likely experts and convert older cache entries into a faster raw format. Network fetching remains an explicit operator action.

Users can also begin with streaming and later download the complete expert pool. The process is resumable and preserves existing cache data. That upgrade path turns streaming into a trial mode rather than a permanent architectural choice.

Still, even the full installation requires more than free disk capacity. Reading 25.8GB per generated token places continuous pressure on the drive. Long runs create a storage-intensive workload that differs from ordinary application use.

Solid-state storage also has finite write endurance, although Deltafin’s steady full-install path primarily reads existing weights. Streaming and cache conversion add writes. The actual effect depends on workload length, caching behavior, drive design, and available spare capacity.

The project’s benchmark uses an internal M1 Max drive. Results from external drives, network storage, nearly full volumes, or thermally constrained systems should not be assumed equivalent. The repository does not present a broad storage comparison.

A full model installation also creates operational friction. Users must reserve terabytes, maintain dependencies, compile native libraries, and manage updates. A compatible Python environment and Apple development tools are required on macOS.

These constraints do not invalidate the project. They define its audience. Deltafin fits researchers, inference engineers, and local-model enthusiasts who treat setup and measurement as part of the value.

Streaming mode serves another purpose. It shows that complete possession of all expert weights is not required before execution starts. That idea could inform future runtimes using tiered storage, shared network caches, or predictive expert placement.

The current numbers also reveal the limit. Fetching experts across the internet moves generation from slow to barely interactive. The model technically runs, but most practical conversations collapse under the waiting time.

The Benchmark Is Real Enough to Study, Not Broad Enough to Generalize

The reported result is transparent, but one machine and one short completion cannot establish everyday Kimi K3 performance.

Deltafin publishes more methodological detail than many hobby inference claims. The maintainer identifies the processor, memory capacity, GPU configuration, storage location, numerical settings, prompt, expected output, and treatment of the first token.

The six full-model runs used balanced execution ordering intended to reduce drift between configurations. Reported values are medians rather than a single best result. The repository also provides a range showing meaningful variation on the same machine.

That transparency strengthens the result. It does not turn it into independent validation. The project maintainer developed the optimizations and ran the reference benchmark, creating the usual need for replication by other operators.

The test prompt is also intentionally small. “The capital of France is” exercises the full forward path, but it does not represent long reasoning, coding, vision input, tool calling, or a large context window.

A three-token verified continuation confirms that the model produced the expected short sequence under the tested configuration. It does not measure answer quality across tasks. It also cannot show whether extended generation maintains the same throughput.

Greedy decoding makes exact reproduction easier because the runtime always selects the highest-scoring next token. Typical hosted applications may use sampling or other decoding controls to create varied outputs. Deltafin currently accepts some sampling fields without applying them.

The project reports exact numerical behavior for its principal benchmark path. An optional approximate mode uses lower-precision arithmetic, but the maintainer warns that near-tied outputs may not remain reproducible.

Kimi K3 itself introduces further uncertainty. Moonshot’s official materials claim strong performance across coding, reasoning, visual, and agentic evaluations. Those claims involve specific harnesses, settings, and competing model configurations.

Deltafin does not validate those benchmark claims. It runs Moonshot’s published modeling code with compatibility changes around unsupported kernels. Readers should separate successful weight execution from independent confirmation of model quality.

The model’s scale can also mislead. Kimi K3 has 2.8 trillion total parameters, but 104 billion are active during a forward pass. Total parameter count determines storage pressure, while active parameters better describe part of the compute burden.

Even 104 billion active parameters is substantial for a workstation. Yet the sparse architecture makes it wrong to compare Kimi K3 directly with a dense 2.8-trillion-parameter model. The systems would move and compute very different amounts of data.

Newer hardware should improve the result, but the degree remains uncertain. More unified memory could retain additional spine data and experts. Faster storage and memory bandwidth could shorten data movement. Better kernels could reduce compute overhead.

Those improvements will not scale every phase equally. If storage dominates one configuration, a faster GPU alone provides limited benefit. If more memory changes cache behavior, performance can jump by more than the raw bandwidth increase suggests.

Deltafin cites a community result from an NVIDIA DGX Spark with 128GB of unified memory. The contributor reported a short end-to-end run, but the maintainer labels it as a single community measurement rather than a reproduced benchmark.

That restraint is appropriate. Hardware comparisons need the same prompts, software revision, cache state, numerical path, and measurement method. Otherwise, a faster completion might reflect setup differences rather than the device.

The largest risk is therefore not that Deltafin’s reference number is meaningless. It is that readers turn a narrow systems result into a broad claim about local AI readiness.

Local generation at four tokens per minute supports experimentation and offline inspection. It does not support the responsive interaction most people associate with chat, code completion, or autonomous agents.

What to Watch After the 0.0687 Token Result

Deltafin’s significance now depends on replication, cache economics, and whether the approach survives realistic prompts.

The first signal is independent performance on newer Apple hardware. Results should report exact chip configurations, memory capacity, storage devices, software commits, cache state, and decoding settings.

A newer Max or Ultra system with more memory can retain a larger fraction of the model’s working set. If throughput improves substantially under controlled tests, Deltafin’s architecture will look like a scalable local-inference path rather than an isolated M1 Max trick.

If gains remain small, storage traffic is probably enforcing a harder ceiling. That outcome would weaken the case for interactive use while preserving the project’s research value.

The second signal is long-prompt behavior. Deltafin’s current headline centers on steady decoding after a five-token prompt. Real chat and coding workloads depend heavily on prefill, which processes all input tokens before generation starts.

Useful tests should include multi-thousand-token code contexts, tool definitions, conversation history, and repeated agent calls. They should report time to first token separately from steady decoding speed.

This distinction matters for the openai max audience. A system can improve its generation rate while still making users wait hours before the first answer. Practical responsiveness depends on both phases.

Kimi K3’s one-million-token context is especially important here. Supporting a context length in the model architecture does not mean every runtime can process that length within acceptable memory and time constraints.

The third signal is whether expert caching becomes predictable across workloads. Deltafin’s streaming mode relies on accumulated local experts, trace-guided warming, and eventual reuse. Researchers need hit-rate data from varied task sequences.

A coding project may repeatedly activate a useful expert subset, improving over time. Switching from code to visual analysis or broad research may change routing patterns and erase much of that benefit.

If small caches deliver high reuse across realistic sessions, the 215GB mode could become more than a preview. If expert selection remains widely distributed, the full 1.7TB installation will remain the only tolerable local path.

Software optimization will continue alongside these tests. Deltafin already reports improvements from an int8 output head, speculative decoding changes, buffer reuse, native kernels, and background loading.

Those gains show how quickly an early runtime can move. They also suggest that the current 0.0687 token-per-second figure should be treated as a dated reference, not a permanent ceiling.

However, future headlines should preserve benchmark discipline. A changed prompt, shorter output, warm cache, approximate numerical mode, or different expert count can improve speed while changing the comparison.

Reducing the number of selected experts is an obvious speed lever because it cuts expert data movement. Deltafin exposes that control, but fewer experts can change outputs and model quality. Such runs should not be presented as equivalent to the default top-16 path.

Moonshot’s ongoing weight and code releases matter as well. The official Kimi K3 model gives developers access to the files that make experiments like Deltafin possible. Runtime compatibility will depend on future model revisions and documentation.

For developers, the immediate lesson is not to replace a hosted model with an M1 Max. It is to reconsider which constraints are absolute. Storage-aware inference can execute a model far larger than available memory when latency is negotiable.

For enterprise buyers, the project clarifies the difference between local control and production suitability. Data residency, model access, response time, concurrency, maintenance, and licensing remain separate decisions.

For knowledge workers, the result points toward a future where local systems can access increasingly large models. Today, managing the output is much easier than waiting for it. Teams experimenting with slow local runs should preserve prompts, configurations, results, and decisions in searchable engineering knowledge.

Deltafin has not compressed a data center into an old laptop. It has built a carefully managed path through a model that should not fit there. That is why the result matters, even at 14.6 seconds per token.

The next question is measurable: can independent developers reproduce the number, then cut it without changing Kimi K3’s default computation? Until those results arrive, openai max remains the wrong contest. Deltafin’s contest is against the boundary between impossible execution and impractical execution.