The Revenge of the Data Scientist – Hamel’s Blog - Hamel Husain
Short summary
The article discusses the shifting landscape for data scientists and machine learning engineers in the age of large language models (LLMs). It emphasizes the importance of data science fundamentals in evaluating AI systems, addressing common pitfalls in metrics, experimental design, and data quality. The author argues that the core work of data scientists remains vital, even as their roles evolve.
- Off-the-shelf eval framework metrics often mislead teams; digging into your own data to find relevant metrics is what data scientists actually do.
- Using LLMs as judges without validating them against human labels is a common, risky shortcut.
- Synthetic test data that isn't grounded in real production logs leads to flawed experimental design and misleading results.
- Outsourcing labeling away from domain experts degrades data quality and undermines the whole evaluation process.
Full summary
The article tackles concerns about the relevance of data scientists in the current AI landscape, particularly with the rise of large language models (LLMs) and foundation-model APIs. Once hailed as the "sexiest job of the 21st century," the role of data scientists is now questioned as companies find ways to integrate AI without them. However, the author argues that the core responsibilities of data scientists—such as setting up experiments, debugging systems, and designing metrics—remain essential. Simply calling an LLM via an API doesn't eliminate these tasks.
Several pitfalls in AI evaluation practices are highlighted, emphasizing the need for data scientists' expertise. Firstly, using generic metrics from evaluation frameworks can mislead teams. Instead, data scientists would explore their specific data to identify relevant metrics. Next, relying on LLMs as judges for model performance is risky without verification; human labels and robust testing are necessary to ensure reliability. Experimental design is another area where many teams falter, often using synthetic data that doesn't reflect real-world conditions. Good practices include grounding synthetic examples in actual logs and using actionable metrics tied to business outcomes.
The author also points out issues with data quality and labeling. Many teams delegate labeling tasks, which can lead to poor data quality. Data scientists prioritize involvement from domain experts to ensure accurate labeling and to better understand the criteria for evaluating outputs. Finally, the tendency to automate everything can overlook the nuanced human work that data scientists bring to the table. While LLMs can assist in some tasks, they can’t replace the critical analysis and insight that comes from human expertise.