Probably raised $9 million to build an AI system that catches hallucinations and factual errors before they reach users. Their data-science tool wraps LLM outputs in a deterministic validator “mech suit,” letting it run smaller models locally while ensuring each answer matches the source data.

This article examines the reliability issues of large language models (LLMs) used in AI, highlighting their tendency to hallucinate and produce incorrect information. New research indicates that these problems stem from the models' inherent design, raising concerns about their suitability for high-stakes applications like law and accounting. Investors may need to reconsider the viability of AI business models given these risks.