Tristan Hume from Anthropic has been grappling with how to effectively evaluate technical candidates as AI improves rapidly. Since early 2024, his team has used a take-home test where candidates optimize code for a simulated accelerator, a method that has allowed them to hire dozens of performance engineers. However, with each new Claude model, including Claude Opus 4 and 4.5, the ability of these AI systems to outperform human candidates within the test's time constraints has prompted ongoing redesigns. Hume has created three versions of the test to maintain its effectiveness in distinguishing candidate skills from AI capabilities. The original take-home test was designed to provide a realistic coding environment, allowing candidates to work without pressure while still reflecting job demands. It featured a Python simulator that mimicked a TPU-like accelerator, focusing on tasks that would be relevant for performance engineers. Early iterations showed promise, with about 1,000 candidates completing it and several high performers being hired directly from undergrad programs. The test's design aimed to be engaging and fun while ensuring that candidates demonstrated their skills without being limited by narrow domain knowledge. As AI models advanced, the effectiveness of the test declined. By May 2025, Claude Opus 4 produced better solutions than most candidates could achieve in the allotted time. Hume had previously experienced similar challenges when designing interview questions that early AI models could easily solve. To counteract this trend, Hume is now releasing the original take-home test as an open challenge, inviting individuals to surpass the performance of Claude Opus 4.5 in an effort to find innovative ways to identify strong engineering talent amid the growing capabilities of AI.