Dream-RSI Summary The core problem here is that autonomous AI agents need to explore search spaces effectively to find good solutions, but exploration strategies themselves become a bottleneck as problems get harder. Standard approaches hit a wall: you can use fixed exploration tactics, but they don't adapt when the search space grows. Alternatively, you can optimize exploration policies online, but that means running expensive trials repeatedly with slow feedback from long-horizon attempts. The researchers behind Dream-RSI spotted the real issue — we're treating exploration as something that happens in a vacuum rather than something we can learn from systematically. Their solution uses a replay simulator built from the agent's own discovery history. Instead of running new expensive trials every time you want to test a better exploration strategy, you can "dream" — simulate what would happen using the tree of past discoveries you've already accumulated. This gives you instant, cheap feedback on whether a new exploration policy would have worked better on problems you've already seen. Once you've refined the policy this way, you deploy the improved version to find new solutions, which then expands your simulator pool. It's a self-reinforcing loop where each round of dreaming produces a better exploration strategy, which finds more discoveries, which builds a richer simulator for the next round. They tested this across three different domains: algorithm engineering, mathematical optimization, and GPU kernel engineering. Dream-RSI matched or beat existing approaches while cutting discovery costs substantially. The framework keeps the underlying coding agent unchanged and just adds a lightweight layer on top that makes exploration explicit and programmable. The practical payoff is clear — you get better solutions without burning through as many expensive evaluations.