Someone built a multi-GPU homelab for local AI inference using a Framework Desktop, RTX 5090 eGPU, and two DGX Spark units, routing requests between local and cloud models depending on latency needs. They're currently at 60-70% local inference and plan to move toward 100%, though they admit the eGPU was a mistake and you don't need this much hardware to start.

• The Qwen 3.8 27B model on the 5090 eGPU hits 150+ tokens/second for fast inference, while the Deepseek v4 Flash models on the DGX Sparks handle slower but higher-quality batch processing and background jobs.
• A custom routing plugin (Arch-Router) decides whether each request goes to local models or cloud/frontier models based on latency requirements, currently keeping about 60-70% of workloads local.
• The hardware stack is overkill for most people—started with just a Mac mini and kept adding. The eGPU specifically isn't worth the investment.

Most prompts degrade into contradictory messes because teams keep adding to them without removing anything, causing real performance hits. Treating prompts like code—structured, modular, and maintained—can unlock 30-50% improvements in speed, retention, and costs.

• Prompts grow through accretion without review, creating contradictions and hidden ambiguities that agents can't resolve; one startup cut LLM spend by $20M/year just by removing duplication
• Prompt decisions are product decisions—agent behavior shapes user experience directly, so vague specifications like "never mention competitors" fail because they lack concrete rules
• MECE structure (mutually exclusive, collectively exhaustive sections) eliminates overlap and makes prompts maintainable like modular code; separate concerns into context, behavior, and output sections