GitHub - meituan-longcat/LongCat-Video

Open original

This repo introduces LongCat-Video, a 13.6B-parameter model that handles text-to-video, image-to-video, and video continuation within a single framework. It uses block sparse attention and a coarse-to-fine strategy to produce minutes-long 720p/30fps videos without quality drift. The project also includes an audio-driven avatar extension with Whisper-based lip sync and distillation-accelerated inference.

  • A single 13.6B-parameter model handles text-to-video, image-to-video, and video continuation, producing minutes-long 720p/30fps clips without quality drift.
  • Avatar 1.5 (May 2026) switched from wav2vec2 to Whisper-Large-v3 for better lip sync, improved long-video stability, and added stylized domains like anime and animal content.
  • Step-distilled inference cuts video generation down to just eight steps, with INT8 quantization also supported.

LongCat-Video packs a 13.6 billion-parameter transformer into one model that handles text-to-video, image-to-video and video-continuation tasks. Its backbone uses a coarse-to-fine strategy along both time and space axes, plus block-sparse attention, to crank out 720 p, 30 fps clips in minutes. Pretraining on video-continuation makes it resilient to color drift and quality loss, even over minutes-long outputs. Multi-reward Group Relative Policy Optimization (GRPO) fine-tunes it via RLHF, pushing its results on public and internal benchmarks up to par with leading open-source and commercial systems. Since the initial October 2025 release, the project launched two avatar-focused versions. LongCat-Video-Avatar debuted in December 2025 for audio-driven character animation, covering Audio-Text-to-Video, Audio-Text-Image-to-Video and continuation modes. In May 2026, Avatar 1.5 swapped wav2vec2 for Whisper-Large-v3, boosted lip sync, improved long-video stability, and added stylized domains (anime, animal, complex scenes). Step-distilled inference cuts generation to eight steps. The GitHub repo includes conda setup commands, FlashAttention-2 installation, demo scripts for single- and multi-GPU runs, plus huggingface-cli commands to pull weights for all three models. User notes cover audio CFG settings, prompt detail, reference-image indexing, mask-frame ranges, resolution flags, dual-audio modes, distillation and INT8 quantization options.

Ask about this article