MatrAIx: Simulating the World with 8.3 Billion Persona Agents
Short summary
MatrAIx is an evaluation platform that uses 8.3 billion AI-powered persona agents to test how AI systems and digital products perform with diverse user types. The system includes a dataset of 1 million personas (half human-grounded, half synthetic), four interactive environments (survey, chatbot, web, app), and over 1,000 tasks across 25 domains. Testing showed the personas accurately express behavioral attributes 91.5% of the time, capturing real variation in user preferences like price sensitivity and failure tolerance.
- MatrAIx uses 8.3 billion simulated personas (built from ~1 million curated profiles) to test AI products instead of relying on expensive, slow human testing.
- Across 18,189 trials using three LLMs as persona "brains," the personas stuck to their declared traits (like price sensitivity or failure tolerance) 91.5% of the time.
- The system is designed to expose how different user segments react differently to the same product, countering the common practice of testing against a flattened "average user."
Full summary
MatrAIx: Testing AI Systems at Scale with Simulated Users
Researchers have built MatrAIx, a system for evaluating AI products and services using 8.3 billion simulated user personas instead of real humans. The motivation is straightforward: human testing is expensive, slow, and doesn't scale. Offline testing works better at scale but typically flattens out human diversity—everyone gets treated like an average user. MatrAIx solves this by creating a population of varied personas with 1,290 different categorical attributes (think demographics, preferences, behaviors, risk tolerance). They released a curated set of about 1 million personas: roughly 600,000 grounded in actual human data and 400,000 generated synthetically. The personas themselves come from two sources—either sampled from a dependency graph that respects how real attributes correlate (income and education, for example) or built from hand-written profiles.
The system has three working parts. Persona 8B is the database of personas. The MatrAIx Playground provides four different environments where these simulated users interact with products: surveys, chatbots, websites, and mobile apps. Then there's a library of 1,010 application tasks spanning 25+ domains—commerce, finance, healthcare, software, and more. The researchers ran 18,189 evaluation trials using three different language models (Claude Opus 4.8, GPT 5.5, and Claude Haiku 4.5) as the brains behind the personas. What they captured was granular: how different user types react to price increases, whether they'll stick with an AI assistant after it fails, how much latency they'll tolerate.
Two validation studies checked whether this actually works. In a controlled test of 400 trials, the personas behaved consistently with their declared attributes 91.5% of the time—they either showed the expected behavior or correctly didn't show it when they shouldn't. A second study had both humans and LLM judges rate how well the human-grounded personas were extracted from real data. The system isn't perfect, but it's reliable enough to surface how user preferences fragment across different backgrounds and circumstances. This matters because most product testing treats users as homogeneous, missing how a feature that works for one segment might alienate another.