Runway's new world model generates 720p video and audio in real time as you interact with it through text commands, letting you control characters, camera movement, and scene events in a continuously evolving environment. It works by splitting world state into persistent context (scene description, rules, first frame) and timestamped events (actions and camera input), then generating video and audio autoregressively frame by frame.

• Generates 24 fps video at 720p with synchronized 48kHz audio in real time, responding to text actions and camera input without following a fixed script
• Uses a two-layer "WorldPrompt" format that separates unchanging world rules from dynamic events, allowing the model to handle rich control across different use cases
• Supports multiple interaction modes: ahead-of-time scripting for filmmaking, turn-based for interactive stories, and real-time for games, plus multiplayer where different users control different characters
• Currently trades fidelity for speed in real-time mode—fast camera movements and long-term consistency degrade quality, though the researchers expect these constraints to improve with further development