Good evaluations are essential for developing AI agents effectively. They help teams catch issues before they reach users, minimizing the reactive debugging that often leads to further complications. The article emphasizes the complexity of evaluating AI agents, which operate over multiple turns and can adapt their behavior based on interactions. This complexity necessitates a rigorous evaluation framework, especially as agents become more autonomous and intelligent. Evaluations, or "evals," consist of tasks with defined inputs and success criteria. The process involves running multiple trials to ensure consistent results, scoring outputs with grading logic, and recording a complete transcript of each interaction. The article breaks down key components of evals, including the evaluation harness that conducts tests and the agent harness that enables the AI to perform tasks. It highlights the importance of having an evaluation suite to measure specific capabilities, as well as the role of different types of graders—code-based, model-based, and human—in assessing agent performance. Building evaluations is particularly important as agents scale. Early on, teams may rely on intuition and manual testing, but this approach becomes insufficient as user feedback reveals shortcomings. Evals help clarify expectations for agent behavior and provide a structured way to measure improvements. They can also streamline the adoption of new models, allowing teams to quickly assess strengths and optimize their systems. By establishing baselines and regression tests, evals facilitate communication between product and research teams, ensuring both sides are aligned on performance metrics.