A novel actor-critic algorithm is introduced that achieves optimal sample efficiency in reinforcement learning, attaining a sample complexity of \(O(dH^5 \log|\mathcal{A}|/\epsilon^2 + d H^4 \log|\mathcal{F}|/\epsilon^2)\). This algorithm integrates optimism and off-policy critic estimation, and is extended to Hybrid RL, demonstrating efficiency gains when utilizing offline data. Numerical experiments support the theoretical findings of the study.

The neural motion simulator (MoSim) is introduced as a world model that enhances reinforcement learning by accurately predicting the future physical state of an embodied system based on current observations and actions. It enables efficient skill acquisition and facilitates zero-shot learning, allowing for a decoupling of physical environment modeling from the development of RL algorithms, thus improving sample efficiency and generalization.