More on the topic...
Generating detailed summary...
Failed to generate summary. Please try again.
Meta released Muse Glimmer, a 30-billion-parameter model designed to run locally on consumer hardware like a Mac or PC with a single GPU. The whole point here is practical: you get an AI agent that works offline, handles tool use, writes code, and manages multi-step tasks without needing cloud infrastructure. They're open-sourcing it under Apache 2.0, so anyone can download the weights from Hugging Face and build with it. The model handles the core things agents actually need — reliable function calling, failure recovery when tools break, multimodal input (text and images), and reasoning across long task chains.
The training approach involved three phases. First, they used logit distillation from a larger teacher model (Muse Spark) during pre-training. Then mid-training added longer-context, agent-heavy data with reasoning traces. Finally, post-training mixed supervised fine-tuning with on-policy distillation and reinforcement learning across general, reasoning, coding, and agentic domains. The result performs competitively against Gemma 4-31B and Qwen 3.6-27B on standard benchmarks, which matters because those are the actual competitors in this size class.
To make it actually usable on consumer hardware, they compressed the model to 4-bit precision, shrinking it from 55GB to under 20GB — tight enough to fit on a 24GB or 32GB GPU alongside working memory and other components. They also implemented speculative decoding with a lightweight drafter model that proposes token blocks at once rather than generating one token at a time, which speeds up long reasoning chains and tool calls without changing the output. Testing on MacBook M4-Max and RTX-5090 showed the model runs fast enough for real-time interaction entirely on-device.
Integrations with llama.cpp, MLX, and ExecuTorch are coming in the next few days, plus support from partners like Ollama, LM Studio, and Together AI. They're also working with hardware makers including NVIDIA, Intel, AMD, and Arm on optimization. The model supports over 100 languages and includes documentation for setting up custom agent scaffolds, so developers can start building local agents immediately.
Questions about this article
No questions yet.