llm-evaluation

SKILL.md

Evaluate and validate LLM outputs for quality assurance using RAGAS and LLM-as-judge patterns.

| Faithfulness | RAG grounding | ≥ 0.8 | | Answer Relevancy | Q&A systems | ≥ 0.7 | | Context Precision | Retrieval quality | ≥ 0.7 | | Context Recall | Retrieval completeness | ≥ 0.7 |

| Judge model | GPT-4o-mini or Claude Haiku | | Threshold | 0.7 for production, 0.6 for drafts | | Dimensions | 3-5 most relevant to use case | | Sample size | 50+ for reliable metrics |

LLM output evaluation and quality assessment. Use when implementing LLM-as-judge patterns, quality gates for AI outputs, or automated evaluation pipelines. Source: yonatangross/orchestkit.

View raw