This article introduces the Gemma 4 family of models from Google DeepMind, detailing their architectures and improvements over the previous version, Gemma 3. It highlights key features such as interleaved attention layers and efficiency enhancements in global attention mechanisms.

• Gemma 4 comes in four sizes—E2B, E4B, 31B, and a 26B MoE model that only activates 4B parameters at inference—and handles text, images, and audio
• Local-to-global attention layers are interleaved at a fixed 4:1 ratio in smaller models and 5:1 in larger ones, with sliding windows of 512 tokens (E2B/E4B) or 1024 tokens (larger models) to cut compute costs
• Global attention memory use is reduced through Grouped Query Attention, a K=V trick that makes keys and values identical, and p-RoPE for improved positional encoding

Qwen has released the Qwen3-VL-Embedding and Qwen3-VL-Reranker models, designed for advanced multimodal information retrieval and cross-modal understanding. These models support various inputs, including text and images, and enhance retrieval accuracy through a two-stage process of initial recall and precise re-ranking.

• Qwen3-VL-Embedding and Qwen3-VL-Reranker pair up for a two-stage retrieval pipeline (recall then rerank) covering text, images, screenshots, and video across 30+ languages.
• Embedding model uses a dual-tower design for independent encoding, while the reranker uses a single-tower architecture with cross-attention for deep query-document interaction.
• Achieves state-of-the-art results on image, visual document, and video retrieval benchmarks.
• Trails the text-only Qwen3-Embedding model on pure text retrieval, showing a tradeoff for its multimodal gains.