Bonsai 27B First 27B Scale Multimodal Model to Run on a Phone

Bonsai 27B ships a 27 billion parameter multimodal model that runs fully on the iPhone 17 Pro.

The 1 bit variant uses 3.9 GB of memory. Its ternary counterpart uses 5.9 GB. Both descend from the Qwen3.6 27B base.

Apple silicon has limited unified memory. A full precision 27B model exceeds that budget on current phones.

Bonsai solves the constraint through extreme quantization without large accuracy loss.

Model sizes and memory footprint

The 1 bit model stores 1.125 effective bits per weight. The ternary model stores 1.71 effective bits per weight.

These figures come from the official release notes posted with the open source weights.

The 3.9 GB 1 bit checkpoint loads directly on the iPhone 17 Pro without external offloading.

Previous 27B class models required at least 14 GB at 4 bit or higher.

Retained benchmark performance

On fifteen standard benchmarks the ternary variant keeps 95 percent of the dense baseline score. The 1 bit version keeps 90 percent.

Math and coding tasks show almost no drop. Vision and agent loop tasks lose a few points.

The team attributes the retention to a training stage that applies quantization aware fine tuning on the final 10 percent of tokens.

Developers can therefore run structured tool calls and multi step reasoning locally.

Context window and inference tricks

Both variants support a 262 K token context window. Speculative decoding speeds up generation by 1.8 times on the phone NPU.

The same weights run on laptops and servers when users need higher throughput. The Apache 2.0 license permits commercial fine tuning.

The release includes reference code for an on device computer use agent loop.

Practical use cases on mobile

A knowledge worker can attach screenshots and receive structured answers without cloud calls.

An engineer can run the model on local codebases for inline suggestions while traveling. The 262 K context lets the model read an entire technical manual in one pass.

Privacy sensitive users gain a local alternative to sending documents to remote servers.

Limits and remaining gaps

The 1 bit model occasionally drops tokens in long agent loops that exceed ten turns. Latency on the phone NPU averages 28 tokens per second for the 1 bit variant.

Neither checkpoint matches the speed of smaller distilled models optimized for mobile. Battery impact stays measurable after twenty minutes of continuous inference.

The team published the weights but has not released the full training dataset or the exact quantization schedule.

Next signals to watch

Watch whether Apple adds hardware support for sub 2 bit operations in the next silicon cycle. Track adoption through GitHub stars and reported on device benchmarks from independent labs.

Observe whether competing labs release similar 1 bit 30 B scale checkpoints within the next quarter.

The outcome will show whether extreme quantization becomes the default path for on device frontier models.