Bonsai 27B: First 27B-Class Model Runs on a Phone — Benchmark Scores and Specs
PrismML has released Bonsai 27B, the first 27-billion-parameter class model that fits and runs on a phone. Based on Qwen 3.6 27B, it uses extreme low-bit quantization — ternary or binary weights — to shrink the model to 3.9 GB (1-bit variant) or 5.9 GB (ternary variant), down from 54 GB in 16-bit precision.
Two Variants: Ternary vs 1-bit
- Ternary Bonsai 27B: weights in {−1, 0, +1} with FP16 group-wise scaling, 1.71 effective bits per weight. Size: 5.9 GB. Targets laptops with full reasoning, tool calling, and agentic capability.
- 1-bit Bonsai 27B: binary weights {−1, +1}, 1.125 effective bits per weight. Size: 3.9 GB. Fits in iPhone 17 Pro memory budget.
Both variants run the entire network (embeddings, attention, MLPs, LM head) in low-bit — no higher-precision escape hatches. They support 262K-token context, multimodal vision (4-bit vision tower), and speculative decoding.
Benchmark Scores (thinking mode)
Across a 15-benchmark suite:
- Math (GSM8K, MATH-500, AIME25, AIME26): Qwen 3.6 27B 95.3, Ternary 93.4, 1-bit 91.7
- Coding (HumanEval+, MBPP+, LiveCodeBench): 88.7 → 86.0 → 81.9
- Agentic/Tool-calling (BFCL v3, TauBench): 80.0 → 74.0 → 66.0
- Instruction following (IFEval, IFBench): 78.4 → 71.8 → 65.8
- Knowledge/STEM (MMLU-Redux, MuSR): 83.1 → 77.0 → 73.4
- Vision (MMMU Pro, OCRBench): 72.6 → 65.2 → 59.6
- Overall: 85.0 → 80.5 (95% retention) → 76.1 (90% retention)
Math and coding are hit the least — key for agentic workloads. The 1-bit variant at 3.9 GB is smaller than a full-precision 2B model, yet delivers 27B-class intelligence.
Why Local Execution Matters for Agents
Agentic workloads require many sequential model calls — each carrying context and producing structured output. Cloud-only execution means per-step latency, accumulating token costs, and sending private data (screenshots, files) over the network. Running locally eliminates those constraints. Bonsai 27B enables on-device multi-step reasoning, tool calls, and computer-use agentic loops.
Licensing and Availability
Both variants are available today under the Apache 2.0 License.
📖 Read the full source: HN AI Agents
👀 See Also
Inference Pricing Analysis Shows 4.4x Spread for Same Model Across Providers
Analysis of inference pricing for Llama 3.1 70B Instruct shows a 4.4x cost difference between providers, with DeepInfra at $0.20/$0.27 per million tokens and Together at $0.88/$0.88. For reasoning models, the spread reaches ~30x between DeepSeek R1 and OpenAI o1.
llama.cpp Q8_0 quantization gets 3.1x speedup on Intel Arc GPUs with SYCL reorder fix
A fix to llama.cpp's SYCL backend brings Q8_0 quantization on Intel Arc GPUs from 21% to 66% of theoretical memory bandwidth, achieving 15.24 tokens/second versus 4.88 tokens/second previously on an Arc Pro B70 with Qwen3.5-27B.
AI Zombification of Universities: A Firsthand Account of LLM Cheating at Elite Colleges
An analysis of how LLMs are systematically destroying academic integrity at elite universities, with specific examples from UChicago: 40-point score gaps between take-home and in-person tests, students photographing exams during tests, and professors writing lectures with ChatGPT.
New AI Tutor Achieves 0.71-1.30 SD Effect Size in Dartmouth Course
A new AI tutor for a Dartmouth introductory CS course showed learning gains of 0.71 to 1.30 standard deviations compared to a control group.