evaluating-llms

SKILL.md

Evaluate Large Language Model (LLM) systems using automated metrics, LLM-as-judge patterns, and standardized benchmarks to ensure production quality and safety.

| Task Type | Primary Approach | Metrics | Tools |

| Classification (sentiment, intent) | Automated metrics | Accuracy, Precision, Recall, F1 | scikit-learn | | Generation (summaries, creative text) | LLM-as-judge + automated | BLEU, ROUGE, BERTScore, Quality rubric | GPT-4/Claude for judging | | Question Answering | Exact match + semantic similarity | EM, F1, Cosine similarity | Custom evaluators |

자동화된 지표, 판사로서의 LLM 및 벤치마크를 사용하여 LLM 시스템을 평가합니다. 신속한 품질을 테스트하고, RAG 파이프라인을 검증하고, 안전성(환각, 편향)을 측정하거나 프로덕션 배포를 위한 모델을 비교할 때 사용합니다. 출처: ancoleman/ai-design-components.

원본 보기