we don't know how to hire. we're just very confident. - Log - nibzard
Short summary
The author argues that standard hiring practices—especially heavy reference checking—don't actually predict future performance better than we think they do. After 50+ hires, he's learned that we mistake subjective judgment and past narratives for objective truth, while ignoring how much a person's performance depends on their environment and manager.
- Multiple references from the same company or network often reflect a shared narrative rather than independent information, and references themselves are biased participants, not neutral observers.
- Past experience is a poor predictor for roles involving rapidly changing technologies and requirements; someone's ability to learn and adapt matters more than years in adjacent roles.
- Most hiring advice is anecdotal autobiography—individual companies rarely hire enough people in similar roles to prove statistically which methods actually work, yet we treat personal success as universal methodology.
- We evaluate candidates heavily but rarely assess whether the company, manager, or system will actually enable good performance, treating professional capability as an individual trait rather than a system outcome.
Full summary
The author challenges the popular idea that exhaustive reference checks—especially the 12-15 calls Brett Berson recommends for executives—actually improve hiring decisions. After hiring 50+ people across different roles and companies, they've learned that more references don't necessarily mean better predictions about future performance. The problem runs deeper than just doing references poorly. Eight references from the same company under the same leadership often tell you the company narrative, not the truth. References are participants in the story, not neutral observers. Someone might seem "difficult" for challenging bad decisions, or their departure might reflect nothing about their actual capability. Add in confidentiality agreements, professional courtesy, and bruised egos, and you're not getting a complete picture. For younger candidates or people in new fields, finding 12 meaningful, independent references becomes impossible—you can find 12 people, but that's not the same as finding 12 independent sources of information.
Experience itself has become a shaky predictor because the jobs people are hired for keep changing. Someone with six months of intense hands-on work with new tools might be more useful than someone with five years of adjacent experience in a field that's now obsolete. References measure past behavior in past contexts, not someone's ability to learn, unlearn, and adapt when the job itself hasn't been invented yet. The author also points out that hiring expertise doesn't follow from being good at other things—a great engineer might be terrible as a team lead, a brilliant CTO might flop as CEO. Yet we treat hiring like a skill we can master, when most of us are just confident we know what we're doing.
The real problem is that hiring advice is almost entirely anecdotal. Most companies don't hire enough people into similar enough roles under comparable conditions to draw meaningful conclusions about which methods actually work. You only observe the performance of people you hire, not rejected candidates who might have been stars. Even measuring "good hire" is fuzzy—is it retention, revenue, productivity, or team satisfaction? Someone might leave after six months having created enormous value, or stay five years contributing nothing. You can't separate a good hire from a bad manager, a shifting strategy, or pure luck. At some point, the author suggests, after checking references on references and validating the credibility of those validators, you should probably just hire someone.