Click any tag below to further narrow down your results
Links
OpenAI's GPT-6 Astra achieved near-perfect scores on ARC-AGI-3, a benchmark measuring agentic intelligence through novel puzzle environments, and matched human efficiency by solving 96% of levels with fewer actions than the median human participant.
- Astra scored 62.7% with standard evaluation but 99.9% when using OpenAI's provider-specific context management features, showing significant performance variation based on how the model manages information between requests.
- The model surpassed human action efficiency on 96% of levels, using 51.7% fewer actions per level on average—a shift from the long-held assumption that AI would require more exploration than humans to solve problems.
- Astra spontaneously developed compact algebraic notation and domain-specific languages to track game states, building custom symbolic world models for each environment without explicit instruction to do so.
Researchers at Latch Bio created a benchmark of 100 experimentally grounded tasks to test whether AI agents can make defensible decisions in antibody discovery. Claude Opus 5 performed best at 53% pass rate, but all models frequently failed by answering scientifically adjacent questions rather than the actual problem at hand.
- Opus 5 with Claude Code achieved the highest pass rate at 53%, with xAI and Google models within 3 percentage points, but no model was reliably accurate
- Model rankings shifted depending on the specific competency and decision type—higher cost and token usage didn't correlate with better performance
- Most failures came from scientific framing errors, not computational mistakes: agents performed internally consistent calculations on the wrong question
- The benchmark spans 10 discovery stages from target assessment through engineering and candidate de-risking, with diverse evidence types (binding kinetics, dose-response, sequence data, structural info)