Evaluating Large Language Models (LLMs) is crucial due to their widespread use in generative AI applications, which presents unique challenges such as hallucination and instruction adherence. Booking.com developed a framework using a judge-LLM to automate the evaluation process, significantly reducing the need for human involvement while ensuring high-quality assessments through the creation of a golden dataset. This approach enables continuous monitoring of LLM performance in production environments.