Principles and Guidelines for the Use of LLM Judges
Aug 1, 2025·
,,,,,,,·
0 min read
Laura Dietz
Oleg Zendel
Peter Bailey
Charles L. A. Clarke
Ellese Cotterill
Jeff Dalton
Faegheh Hasibi
Mark Sanderson
Nick Craswell
Abstract
Comprehensive principles and guidelines for using large language models as judges in information retrieval evaluation, addressing reliability, bias, calibration, and best practices for LLM-as-a-judge methodologies.
Type
Publication
Proceedings of the 2025 International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval
