Cohere Labs released North Small Translate, a sparse Mixture-of-Experts model built for machine translation across 50 languages. The model has 25 billion active parameters out of 218 billion total, meaning most weights stay dormant unless needed—a design choice that keeps inference faster than running the full network. It supports a 16K context window for both input and output, and Cohere offers three quantization levels (BF16, FP8, and 4-bit) depending on your hardware. The smallest version runs on a single Nvidia B200 GPU with 4-bit weights, while the full precision version needs 8 H100s. The model uses a CC BY-NC license but requires accepting Cohere's Acceptable Use Policy, and you need to sign up to download the weights. Using the model requires some care with memory management and output parsing. If you load it with PyTorch's `device_map="auto"`, you need to explicitly set `max_memory` to leave room for temporary buffers when fusing the Mixture-of-Experts layers—otherwise you'll run out of VRAM partway through loading. The model wraps its responses in structural markers like `<|START_TEXT|>` and `<|END_TEXT|>` that aren't registered as special tokens, so `skip_special_tokens=True` won't remove them. Cohere provides a parsing library called melody that strips these markers cleanly, which works with both direct transformers calls and vLLM deployments. The documentation covers three main ways to run it: directly via transformers with `AutoModelForCausalLM`, through the transformers pipeline abstraction, or via vLLM for serving. You can override the default system instructions by passing `platform_instruction_override` to the chat template, or append additional instructions as a regular system message. Cohere recommends greedy decoding (no sampling) for production use, which matches how they serve the model themselves. The vLLM setup requires installing melody 0.9.0+ and configuring tensor parallelism based on your GPU count—for H100s, they suggest `tp 8` for the full model.