The article discusses the shifting landscape for data scientists and machine learning engineers in the age of large language models (LLMs). It emphasizes the importance of data science fundamentals in evaluating AI systems, addressing common pitfalls in metrics, experimental design, and data quality. The author argues that the core work of data scientists remains vital, even as their roles evolve.

• Off-the-shelf eval framework metrics often mislead teams; digging into your own data to find relevant metrics is what data scientists actually do.
• Using LLMs as judges without validating them against human labels is a common, risky shortcut.
• Synthetic test data that isn't grounded in real production logs leads to flawed experimental design and misleading results.
• Outsourcing labeling away from domain experts degrades data quality and undermines the whole evaluation process.

The article discusses how the rise of AI tools, particularly LLMs, has affected software engineering and data work. While some engineers are concerned about the declining quality of code, data professionals find value in these tools for generating quick, low-maintenance solutions. It emphasizes the need for careful evaluation of the new data generated by these systems.

• Fast, disposable code from LLMs isn't a problem for data work the way it is for engineers building long-term systems, since data scientists often throw away code after one use anyway
• LLMs are genuinely useful for building data extraction tools (e.g., pulling usable data from messy PDFs), unlocking previously inaccessible data sources with minimal effort
• Using LLMs to generate synthetic data is risky and shouldn't replace real human data, especially in user experience research
• As LLMs improve at generating and explaining code, data professionals need less hand-holding from software engineers for low-risk coding tasks, shrinking the traditional divide between the two roles