The article tackles concerns about the relevance of data scientists in the current AI landscape, particularly with the rise of large language models (LLMs) and foundation-model APIs. Once hailed as the "sexiest job of the 21st century," the role of data scientists is now questioned as companies find ways to integrate AI without them. However, the author argues that the core responsibilities of data scientists—such as setting up experiments, debugging systems, and designing metrics—remain essential. Simply calling an LLM via an API doesn't eliminate these tasks. Several pitfalls in AI evaluation practices are highlighted, emphasizing the need for data scientists' expertise. Firstly, using generic metrics from evaluation frameworks can mislead teams. Instead, data scientists would explore their specific data to identify relevant metrics. Next, relying on LLMs as judges for model performance is risky without verification; human labels and robust testing are necessary to ensure reliability. Experimental design is another area where many teams falter, often using synthetic data that doesn't reflect real-world conditions. Good practices include grounding synthetic examples in actual logs and using actionable metrics tied to business outcomes. The author also points out issues with data quality and labeling. Many teams delegate labeling tasks, which can lead to poor data quality. Data scientists prioritize involvement from domain experts to ensure accurate labeling and to better understand the criteria for evaluating outputs. Finally, the tendency to automate everything can overlook the nuanced human work that data scientists bring to the table. While LLMs can assist in some tasks, they can’t replace the critical analysis and insight that comes from human expertise.