Thread by @SakanaAILabs on Thread Reader App

Open original

AB-MCTS applies an adaptive branching Monte Carlo tree search at inference time to let top LLMs (o4-mini, Gemini 2.5 Pro, R1-0528) collaborate on hard problems. On the ARC-AGI-2 benchmark, this multi-model approach solves cases no single model can and outperforms each model alone. The team open-sourced their algorithm and benchmark code.

  • Combining Gemini 2.5 Pro, o4-mini, and DeepSeek-R1-0528 via AB-MCTS on ARC-AGI-2 outperforms any single model, and solves problems none of them can solve alone.
  • A wrong step from one model (e.g., o4-mini) often becomes the hint another model (Gemini or R1-0528) uses to reach the correct answer.
  • The method is training-free, adapting Monte Carlo Tree Search to let models propose, vote, and guide exploration, so new models can be swapped in as they emerge.
  • Sakana open-sourced the approach as TreeQuest on GitHub alongside a published paper (arxiv.org/abs/2503.04412).

Sakana AI just rolled out AB-MCTS, an inference-time scaling method that links several top LLMs—Gemini 2.5 Pro, o4-mini and DeepSeek-R1-0528—into a single problem solver. It borrows its search structure from Monte Carlo Tree Search but adapts branching to let each model propose moves, vote on them and guide exploration. On the ARC-AGI-2 benchmark, the trio combined via AB-MCTS beats each one on its own by a clear margin. In trials, a wrong step by o4-mini often becomes the key hint for Gemini or R1-0528 to find the right answer. This effort follows Sakana’s 2024 work on evolutionary model merging but shifts focus from fusing weights to orchestrating existing models at inference time. They’ve open-sourced the TreeQuest code on GitHub and published a paper (arxiv.org/abs/2503.04412). Early results show that complex reasoning tasks unsolvable by any single LLM fall more often to the collaborative setup. The idea is simple: treat each model’s biases as complementary strengths and let them iterate until they converge on a solution. All the training-free mechanics live in the AB-MCTS wrapper, so you can drop in new models as they emerge.

Ask about this article