Introducing LFM3: Every Modality for Specialized Intelligence | Blog
Liquid AI has released LFM3, a family of roughly 2B-parameter open-weight models covering text, vision, audio, omni, and decision tasks on one shared backbone. The pitch is strong capability per parameter that runs fast on phones, laptops, and single GPUs, and can be fine-tuned locally.
- The LFM3-1.7B text model has a 256K-token context window and leads several agentic and tool-use benchmarks, including BFCL and ToolSandbox, while beating Qwen3.5-9B on WebArena-Verified and AutomationBench.
- On-device, it decodes 1.8 to 2.8 times faster than the next-best model tested and needs only 1.6 GB of KV cache for a full 128K context, versus 4.3 GB for Qwen3.5-4B.
- The architecture mixes gated short convolutions, sliding-window attention, and global attention, and the team reports about 25% faster inference than LFM2 at 8K context.
Liquid AI's LFM3 is a new family of about 2B-parameter models, and the first release includes six of them, all branched from one 1.7B base backbone. Four cover different inputs (text, vision, audio, and omni), and two are decision models specialized from the vision and omni variants. The base was pre-trained on 22T tokens, with a final 3.5T-token mid-training stage that pushes the context window to 256K. Weights are open and available on Hugging Face, the Liquid Playground, and the Liquid AI API, and the pitch is that a small model you can fine-tune on a single GPU and run on a phone or laptop can be a reliable specialist.
The architecture is where the interesting engineering sits. LFM2 mixed short convolutions with attention; LFM3 adds a third operator, sliding-window attention with a 256-token window, and ends up with 13 convolution layers, 5 SWA layers, and 6 global attention layers across 24 total. The convolution kernel grows from 3 to 7. Head-wise sigmoid gating lets individual attention heads switch off, which the authors say helps suppress activation outliers that hurt quantization. Partial RoPE leaves the low-frequency half of global attention heads unrotated so they match on content over long distances, and sandwich normalization wraps each block's input and output. The architecture search ran across 14 device and backend combinations, including a Raspberry Pi 5, two phones, Apple M4 Max and AMD Ryzen AI laptops, cloud CPUs, GPUs, and the Qualcomm Hexagon NPU. The headline speed claim is that at 8K context, LFM3-1.7B is about 25% faster than its predecessor per parameter, though the speedup is specifically a long-context result.
The benchmark table is the part to read carefully, because it is mostly a story about where a 1.7B model punches above its weight and where it does not. On agent and tool benchmarks it does well: it posts 45.6 on LiveClaw against 35.8 for Qwen3.5-9B, leads on BFCL V4 (71.4) and ToolSandbox (80.4), and beats the 9B Qwen on WebArena-Verified (38.4 vs. 28.9) and AutomationBench (41.3 vs. 15.1). It also does well on instruction following, scoring 96.7 on IFEval. It trails clearly on agentic coding, with 43.4 on SWE-bench Verified against 51.2 for the 9B Qwen, and on knowledge, where GPQA Diamond is 63.4 against 80.4. Long-context results are mixed: strong on AA-LCR (52.8), but RULER 128K at 65.6 sits well below Qwen3.5-4B's 84.6. Post-training runs through four stages, including 1.75T tokens of supervised fine-tuning and mixed reinforcement learning with a length penalty to keep answers short. Anyone thinking about deploying this should weigh the agentic and tool-use strengths against the coding and long-context gaps, and verify the numbers on their own workload, since these are vendor-reported results.