← All Tags
# qwen → llama.cpp

1 link tagged with all of: qwen + llama.cpp

Click any tag below to further narrow down your results

+ backburner (1) + split inference (1) + context window (1) + apple silicon (1)

Links

GitHub - StayLameBro/backburner: Your iPhone helps your Mac run a 27B model: faster prompt reading and more context over a USB-C cable

Backburner splits a 27-billion-parameter language model between a MacBook Pro and iPhone connected via USB-C, achieving 29-44% faster inference on long contexts by distributing computation across both devices' GPUs. The system maintains identical outputs to running the model on the Mac alone while supporting up to 196k tokens of context on an iPhone 17 Pro Max.
  • Split prefill workload: Mac runs layers 1-40 while iPhone runs layers 41-64 in parallel, with the iPhone holding older KV cache pages past 64k context and computing attention over them during both prefill and decoding.
  • Measured performance gains: 44% faster throughput at 16k context (157 vs 109 tok/s), and ability to maintain 8-bit precision at 128k tokens versus the Mac alone dropping to 4-bit to fit in memory.
  • Greedy decoding is token-identical between Mac-only and Mac+iPhone configurations across tested context lengths (8k, 32k, 140k), confirming the split computation produces correct results.
Last saved Oct 08, 2026 · 7 min read
+ backburner qwen llama.cpp + split inference + context window + apple silicon