More on the topic…
Dan Luu argues that he has an unusual ability to spot bugs and quality issues that others miss, and he's developed this skill enough that executives sometimes hire him to evaluate products before launch. When he finds severe problems—where a product barely functions without workarounds—he's often puzzled to discover internal discussions praising the same product. Over time, he's become confident that when something seems broken to him, it actually is broken. He's now using LLMs to verify that issues reproduce across different user scenarios, not just edge cases he's hitting by accident. His track record of identifying problems before products fail publicly supports this confidence.
Luu illustrates his point with search engines. When he tested Google, Bing, and Kagi, all three returned poor results filled with SEO spam and scams. He rated this as "moderate" rather than "severe" quality issues. Almost nobody disputed his findings on Google and Bing, but Kagi fans insisted the results were good—even when they showed him their actual results, which contained the same spam and missing information he'd found. Some users worked around the problems by pinning GitHub to the top of their filters, which worked only for software-related queries and failed completely for things like seasonal forecasts or tire mechanics explanations.
The deeper mystery, Luu argues, is how people blind themselves to obvious failures in products they like. Volvo forum members insist their cars are reliable despite decades of mediocre reliability data. Blackboard employees thought their software was widely loved while it was so despised that 93% of survey respondents said they "hate" the company—a fact mentioned on its Wikipedia page. Most striking: Discourse employees believed their forum software had great performance while it contained code deliberately designed to cheat on performance metrics like LCP, actually harming user experience rather than helping it. The gap between what fans believe and what users experience appears to be nearly total.
Questions about this article
No questions yet.