Introducing Muse Glimmer: An Open Agentic Model That Runs on Your Device
Short summary
Meta open-sourced Muse Glimmer, a 30-billion-parameter model designed to run on consumer GPUs and laptops for local AI agent tasks like scheduling, coding, and tool use. The model uses quantization and speculative decoding to fit within 20-32GB of memory while maintaining speed for real-time interaction. It's available now under Apache 2.0 license with integrations for llama.cpp, MLX, and other frameworks.
- Meta open-sourced Muse Glimmer, a 30B-parameter agentic model quantized to under 20GB so it runs locally on consumer GPUs/laptops (24-32GB) while competing with Gemma 4-31B and Qwen 3.6-27B.
- Speculative decoding with a lightweight drafter model proposes token blocks at once instead of one token at a time, speeding up reasoning and tool calls without changing outputs.
- Training combined logit distillation from a larger teacher model, agent-heavy mid-training data, and post-training RL/distillation across reasoning, coding, and agentic tasks.
- Released under Apache 2.0 with upcoming llama.cpp, MLX, and ExecuTorch integrations and support from Ollama, LM Studio, Together AI, and chipmakers like NVIDIA, Intel, AMD, and Arm.
Full summary
Meta released Muse Glimmer, a 30-billion-parameter model designed to run locally on consumer hardware like a Mac or PC with a single GPU. The whole point here is practical: you get an AI agent that works offline, handles tool use, writes code, and manages multi-step tasks without needing cloud infrastructure. They're open-sourcing it under Apache 2.0, so anyone can download the weights from Hugging Face and build with it. The model handles the core things agents actually need — reliable function calling, failure recovery when tools break, multimodal input (text and images), and reasoning across long task chains.
The training approach involved three phases. First, they used logit distillation from a larger teacher model (Muse Spark) during pre-training. Then mid-training added longer-context, agent-heavy data with reasoning traces. Finally, post-training mixed supervised fine-tuning with on-policy distillation and reinforcement learning across general, reasoning, coding, and agentic domains. The result performs competitively against Gemma 4-31B and Qwen 3.6-27B on standard benchmarks, which matters because those are the actual competitors in this size class.
To make it actually usable on consumer hardware, they compressed the model to 4-bit precision, shrinking it from 55GB to under 20GB — tight enough to fit on a 24GB or 32GB GPU alongside working memory and other components. They also implemented speculative decoding with a lightweight drafter model that proposes token blocks at once rather than generating one token at a time, which speeds up long reasoning chains and tool calls without changing the output. Testing on MacBook M4-Max and RTX-5090 showed the model runs fast enough for real-time interaction entirely on-device.
Integrations with llama.cpp, MLX, and ExecuTorch are coming in the next few days, plus support from partners like Ollama, LM Studio, and Together AI. They're also working with hardware makers including NVIDIA, Intel, AMD, and Arm on optimization. The model supports over 100 languages and includes documentation for setting up custom agent scaffolds, so developers can start building local agents immediately.