advanced-evaluation

This skill covers production-grade techniques for evaluating LLM outputs using LLMs as judges. It synthesizes research from academic papers, industry practices, and practical implementation experience into actionable patterns for building reliable evaluation systems.

Key insight: LLM-as-a-Judge is not a single technique but a family of approaches, each suited to different evaluation contexts. Choosing the right approach and mitigating known biases is the core competency this skill develops.

Evaluation approaches fall into two primary categories with distinct reliability profiles:

当用户要求“实施LLM作为法官”、“比较模型输出”、“创建评估标准”、“减轻评估偏差”或提到直接评分、成对比较、位置偏差、评估流程或自动质量评估时，应使用此技能。 来源：chakshugautam/games。