Large language models (LLMs) excel in language tasks but often falter in dynamic environments where they must anticipate the effects of their actions. The authors introduce a method called Reinforcement World Model Learning (RWML), which aims to address this gap by enabling LLMs to learn action-conditioned world models based on textual states. RWML employs a self-supervised approach, using rewards that bridge the difference between simulated outcomes and actual environmental feedback. This method promotes coherence between the internal models and real-world dynamics, which is essential for effective decision-making in agentic settings. RWML distinguishes itself from traditional next-state token prediction. While the latter focuses on mimicking specific wording, it risks oversimplifying the learning process and possibly leading to model failure. RWML, on the other hand, provides a stronger training signal and demonstrates greater resilience against reward hacking, making it a more reliable framework for training LLM-based agents. The authors tested RWML on the ALFWorld and τ² Bench environments, where it showed substantial improvements over baseline models. Specifically, when combined with task-success rewards, RWML outperformed direct reinforcement learning methods by 6.9 points on ALFWorld and 5.7 points on τ² Bench. This performance is on par with models trained on expert data, suggesting that RWML can effectively bridge the gap between simulated and real-world learning without requiring extensive labeled datasets.