Bloom is an open-source tool designed to automate the evaluation of behavioral traits in AI models. It allows researchers to specify a behavior and generates various scenarios to quantify how frequently and severely that behavior occurs in different models. Bloom's results align well with human evaluations and distinguish between baseline models and those with intentional misalignments. The tool can produce evaluations in just a few days, a significant improvement over traditional methods that can take much longer and become outdated as models evolve. The evaluation process consists of four automated stages: Understanding, Ideation, Rollout, and Judgment. The understanding stage analyzes behavior descriptions to determine what to measure, while the ideation stage creates scenarios to elicit those behaviors. The rollout stage tests these scenarios, and the judgment phase scores the results. Bloom's flexibility means it can generate varied scenarios for the same behavior across different runs, helping maintain reproducibility through a configuration seed file that documents the evaluation setup. Bloom has shown promising results in validation tests. It accurately differentiated between models with distinct behavioral tendencies, succeeding in nine out of ten tests against deliberately quirky models. It also correlated well with human judgments, especially in identifying extreme behaviors, which is crucial for determining the presence or absence of specific traits. In a case study on self-preferential bias, Bloom not only confirmed previous findings but also revealed that increased reasoning effort in models led to reduced bias, demonstrating its potential for deeper insights into model behaviors. Researchers are already applying Bloom for various tasks, including evaluating jailbreak vulnerabilities and measuring evaluation awareness. As AI capabilities expand, tools like Bloom are essential for understanding and addressing behavioral alignment in complex AI systems.