2 links tagged with all of: mixture-of-experts + long-context
Click any tag below to further narrow down your results
Links
DeepSeek V4.1 Flash is a 552B-parameter mixture-of-experts model that activates only 8B-16B parameters per token and supports a 1M-token context window for handling long documents and agentic workloads. The model cuts KV-cache memory usage by 4x compared to its predecessor while improving coding, reasoning, and agent performance.
- Activates just 8B parameters during input processing and 16B during generation, making it efficient despite the massive 552B backbone
- Reduces memory footprint by roughly 4x versus DeepSeek-V4-Flash-0731 through Compressed Sparse Attention 2 and a Causal Encoder-Decoder architecture
- Offers configurable reasoning-effort levels so developers can adjust latency and cost versus reasoning depth
- Handles multimodal input (text and images) natively with a 1M-token context window, designed for agents processing large amounts of data
Laguna XS 2.1 is a 33B Mixture-of-Experts model with 3B activated parameters per token, tuned for agentic coding and long-horizon tasks on local machines. It outperforms XS.2 on SWE-bench Multilingual by 5.4 points, adds support for major runtimes, offers FP8/INT4/NVFP4 checkpoints, and runs with a permissive OpenMDW-1.1 license. You can access it via API, OpenRouter, or locally with quantized or DFlash-boosted builds.
- Laguna XS 2.1 jumps from 57.7% to 63.1% on SWE-bench Multilingual, beating Qwen 3.6 (35B) and North Mini Code (30B) despite only 3B active params per token
- DFlash speculator models double token throughput versus stock XS 2.1 when run locally
- Same API pricing as XS.2 ($0.10 input/$0.20 output/$0.05 cache-read per million tokens), now served at 256K context
- XS.2 leaves the API in one week, pushing users to migrate to 2.1 (though it remains available on Baseten's Model Library)