promptfoo-evaluation

SKILL.md

This skill provides guidance for configuring and running LLM evaluations using Promptfoo, an open-source CLI tool for testing and comparing LLM outputs.

Embed examples directly in prompt or use chat format with assistant messages:

Create a Python file for custom assertions (e.g., scripts/metrics.py):

Configures and runs LLM evaluation using Promptfoo framework. Use when setting up prompt testing, creating evaluation configs (promptfooconfig.yaml), writing Python custom assertions, implementing llm-rubric for LLM-as-judge, or managing few-shot examples in prompts. Triggers on keywords like "promptfoo", "eval", "LLM evaluation", "prompt testing", or "model comparison". Source: daymade/claude-code-skills.

View raw