The HateBenchSet is a dataset designed to benchmark hate speech detectors on content generated by various large language models (LLMs). It comprises 7,838 samples across 34 identity groups, including 3,641 labeled as hate and 4,197 as non-hate, with careful annotation performed by the authors to avoid exposing human subjects to harmful content. The dataset aims to facilitate research into LLM-driven hate campaigns and includes predictions from several hate speech detectors.

OpenAI has launched BrowseComp, a new benchmark designed to evaluate the browsing capabilities of AI agents in locating difficult-to-find information across the internet. This benchmark includes 1,266 challenging questions that require persistence and creativity, distinguishing it from existing benchmarks that focus on simpler fact retrieval. Researchers are invited to utilize BrowseComp to improve the reliability and performance of AI systems.