Someone built a multi-GPU homelab for local AI inference using a Framework Desktop, RTX 5090 eGPU, and two DGX Spark units, routing requests between local and cloud models depending on latency needs. They're currently at 60-70% local inference and plan to move toward 100%, though they admit the eGPU was a mistake and you don't need this much hardware to start.

• The Qwen 3.8 27B model on the 5090 eGPU hits 150+ tokens/second for fast inference, while the Deepseek v4 Flash models on the DGX Sparks handle slower but higher-quality batch processing and background jobs.
• A custom routing plugin (Arch-Router) decides whether each request goes to local models or cloud/frontier models based on latency requirements, currently keeping about 60-70% of workloads local.
• The hardware stack is overkill for most people—started with just a Mac mini and kept adding. The eGPU specifically isn't worth the investment.

This article digs into why repeated LLM calls can produce different outputs even at zero temperature. It shows that floating-point non-associativity and kernel implementation details—rather than thread scheduling or atomic adds—are the real sources of run-to-run variation and outlines ways to make inference fully reproducible.

• Bit-identical results across thousands of runs of the same GPU matmul disprove the standard "concurrency + floating-point non-associativity" explanation for LLM nondeterminism.
• The actual cause is that libraries like cuBLAS/cuDNN nondeterministically pick among multiple reduction/tiling algorithms at runtime, each summing floats in a different order.
• Forcing a single deterministic algorithm choice (via flags like CUBLAS_WORKSPACE_CONFIG, torch.use_deterministic_algorithms(True), and seeded CUDA RNG) makes LLM inference output bit-for-bit reproducible.