evaluating-llms

SKILL.md

Evaluate Large Language Model (LLM) systems using automated metrics, LLM-as-judge patterns, and standardized benchmarks to ensure production quality and safety.

| Task Type | Primary Approach | Metrics | Tools |

| Classification (sentiment, intent) | Automated metrics | Accuracy, Precision, Recall, F1 | scikit-learn | | Generation (summaries, creative text) | LLM-as-judge + automated | BLEU, ROUGE, BERTScore, Quality rubric | GPT-4/Claude for judging | | Question Answering | Exact match + semantic similarity | EM, F1, Cosine similarity | Custom evaluators |

Evalúe los sistemas LLM utilizando métricas automatizadas, LLM como juez y puntos de referencia. Úselo al probar la calidad inmediata, validar tuberías RAG, medir la seguridad (alucinaciones, sesgos) o comparar modelos para el despliegue de producción. Fuente: ancoleman/ai-design-components.

Ver original