GitHub - StayLameBro/backburner: Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable
Short summary
Backburner splits a 27-billion-parameter language model between a MacBook Pro and iPhone connected via USB-C, achieving 29-44% faster inference on long contexts by distributing computation across both devices' GPUs. The system maintains identical outputs to running the model on the Mac alone while supporting up to 196k tokens of context on an iPhone 17 Pro Max.
- Split prefill workload: Mac runs layers 1-40 while iPhone runs layers 41-64 in parallel, with the iPhone holding older KV cache pages past 64k context and computing attention over them during both prefill and decoding.
- Measured performance gains: 44% faster throughput at 16k context (157 vs 109 tok/s), and ability to maintain 8-bit precision at 128k tokens versus the Mac alone dropping to 4-bit to fit in memory.
- Greedy decoding is token-identical between Mac-only and Mac+iPhone configurations across tested context lengths (8k, 32k, 140k), confirming the split computation produces correct results.
Full summary
Backburner is a llama.cpp fork that splits model inference between a Mac and iPhone connected via USB-C, letting you run Qwen 3.8-27B locally with better performance than either device alone. The setup splits the 64-layer model across devices: the Mac handles layers 1-40 during prefill while the iPhone runs layers 41-64 in parallel, then streams results back. This pipelined approach cuts prefill time by 29-44% on 16k-48k context windows. The system also extends context capacity from 64k tokens on a 24GB Mac to 196k-229k tokens by offloading older key-value pages to the iPhone's GPU, which computes attention over them during generation. Token outputs remain identical whether you use the phone or not—the fork achieves this through careful synchronization of computation across devices.
The performance gains come from both the split architecture and Mac-specific optimizations. The fork adds custom Metal kernels and SME2 (the M4's specialized matrix units) that speed up prefill by ~30% even running solo. For real-world agent workloads, the system processes the first response in 168 seconds versus 245 seconds on stock llama.cpp, then handles subsequent tool results in 14.5 seconds versus 17.9 seconds. At 128k context with 8-bit precision, it maintains 12.6 tokens/second generation speed and successfully recalls facts planted across the entire context window. The iPhone's Neural Engine compiles old key pages into specialized models, cutting per-token latency from 279ms to 176ms during generation at extreme context lengths.
The architecture makes practical trade-offs. Small reads under ~512 tokens stay on the Mac since USB latency outweighs the benefit. Past 64k context, the iPhone switches from running layers 41-64 to only computing old-key attention while the Mac runs all layers—this simplification works because the phone's layers can't see keys it's holding anyway. Two iPhones can share old context pages. The system handles memory carefully: the model loads wired so macOS can't page it, and the token embedding table gets memory-mapped rather than fully loaded. This keeps the server's footprint around 18-20GB on a 24GB machine, leaving room for context.