Someone built a multi-GPU homelab for local AI inference using a Framework Desktop, RTX 5090 eGPU, and two DGX Spark units, routing requests between local and cloud models depending on latency needs. They're currently at 60-70% local inference and plan to move toward 100%, though they admit the eGPU was a mistake and you don't need this much hardware to start.

• The Qwen 3.8 27B model on the 5090 eGPU hits 150+ tokens/second for fast inference, while the Deepseek v4 Flash models on the DGX Sparks handle slower but higher-quality batch processing and background jobs.
• A custom routing plugin (Arch-Router) decides whether each request goes to local models or cloud/frontier models based on latency requirements, currently keeping about 60-70% of workloads local.
• The hardware stack is overkill for most people—started with just a Mac mini and kept adding. The eGPU specifically isn't worth the investment.

Perplexity's Portable Computer lets Windows users run AI agents locally on NVIDIA RTX GPUs, keeping sensitive data on-device while handling multi-step tasks without consuming cloud credits. The tool connects to common apps like Outlook, Gmail, and GitHub, with an option to offload complex work to cloud models when needed.

• Runs on local NVIDIA GeForce RTX and RTX PRO GPUs with 24GB+ VRAM, requiring no manual model selection or complex software setup
• Handles real workflows: reviewing GitHub PRs, analyzing financial documents with exact file citations, identifying user drop-off points in product funnels
• Sensitive information stays on the device; users approve before any data leaves for cloud processing

Apple released updated Mac mini and Mac Studio with new M6 and M5 Ultra chips, positioning them as machines for running large language models locally. The hardware refresh is purely specs-focused, but Apple's marketing now emphasizes AI workloads after macOS improvements last year made multi-Mac setups viable for distributed inference.

• Apple shipped macOS 26.2 in December with Thunderbolt 5 support for low-latency distributed AI inference, which triggered developer interest in daisy-chaining multiple Macs for larger models
• The M6 is Apple's first 2nm chip for Macs, and the M5 Ultra is now the most powerful option in the lineup, especially for AI tasks
• Developers and researchers are using stacked Mac minis and Studios as an alternative to expensive Nvidia GPU hardware for running local LLMs that exceed single-device capacity

This article argues that local-AI performance on Macs depends on memory bandwidth, not CPU cores, GPU cores, or the Neural Engine. Using a simple formula (bandwidth ÷ model size × efficiency), it shows a 2021 M1 Max outperforms a 2024 M4 base chip by over 3× on a 7B model. It recommends buying used Max-tier machines and highlights lineup quirks like the M3 Pro’s bandwidth regression.

• Memory bandwidth, not CPU/GPU/Neural Engine specs, determines local-AI token generation speed on Macs
• A 2021 M1 Max (400 GB/s) hits ~64 tok/s on a 7B model vs. ~19 tok/s on a base 2024 M4 (120 GB/s) — over 3× faster despite being three years older
• Tier jumps (base→Pro→Max) matter far more than generational upgrades: four generations of base chips only went from 68 to 120 GB/s, while switching tiers can triple or quadruple bandwidth
• Used Max-tier MacBook Pros often cost the same as a new M4 MacBook Air but outperform it on every local-AI task except power efficiency and media engines, making RAM/bandwidth the specs to prioritize when buying used

This article shows how macOS 27 includes a built-in LLM accessible via /usr/bin/fm that runs entirely offline and needs no account. The author tests it on an M1 MacBook Air, noting its limited memory, occasional inaccuracies, and a range of Terminal-based tricks.

• macOS 27 includes an offline, account-free LLM ("Apple Foundation Models") accessible via /usr/bin/fm in Terminal
• Runs smoothly on a base M1 MacBook Air with 8GB RAM, though it has limited memory and no awareness of current dates
• Useful for practical tasks like drafting emails, translation, summarizing Markdown, generating regex/SQL, and shell scripting help
• Prone to hallucinating dates or inventing functions when pushed beyond simple tasks

The article breaks down which AI models and setups you can afford to run or train at home by 2026, comparing GPU costs, power use, and performance. It highlights efficient small-scale models, quantization tricks, and DIY hardware options to save money without sacrificing too much accuracy.

• Nvidia Blackwell cards should hit 150–200 TFLOPS FP16 under $1,500, making home rigs viable for large-model inference by 2026.
• 4-bit quantization plus FlashAttention already lets a 4090 run Llama 2-70B for under $0.02/inference and fit 13B models in 20GB VRAM, undercutting cloud A100 rental costs ($0.10–0.50/min).
• LoRA fine-tuning a 7B model on 8x4090s or two Blackwells takes a few hours and under $10 in electricity, though full 70B training from scratch still needs real clusters.
• A $3,000–4,000 home setup (with ~$50–100/month power costs for 24/7 use) will be enough to prototype LLM applications without cloud fees.

The author swaps ChatGPT Plus, Cursor and Midjourney for local AI on a 14″ MacBook Pro M5 Max. Two setups failed; a third ran locally by day nine and convinced him to re-subscribe.

• CUDA is irrelevant on Apple Silicon, yet the author describes "CUDA fallbacks" as part of the local setup overhead—an inconsistency suggesting the account may be unreliable
• Local LLMs (Qwen, Mistral) hit memory bottlenecks and slow inference on long prompts, even on an M5 Max
• Local Stable Diffusion couldn't match Midjourney's compositional consistency and prompt refinement, driving a resubscription
• Cursor's extension ecosystem proved hard to replicate locally, undermining the coding-assistant replacement