failure-taxonomy

Transform raw, freeform trace annotations from open coding sessions into a structured taxonomy of binary failure modes, following the grounded theory methodology from the Analyze-Measure-Improve evaluation lifecycle.

The user has already completed open coding — they've read through LLM pipeline traces and written short, freeform notes describing what went wrong (the "point of first failure").

Now they need to move from that chaotic pile of observations into an organized, actionable taxonomy. This is the axial coding step.

从开放编码的跟踪注释构建故障模式的结构化分类法。每当用户通过查看 LLM 跟踪获得自由格式注释并希望将它们聚类为连贯的、不重叠的二进制故障类别集（轴向编码）时，请使用此技能。当用户提到“故障模式”、“错误分类”、“轴编码”、“聚类注释”、“错误分类”、“故障分析”或想要从原始观察笔记转向结构化评估标准时也可以使用。这项技能涵盖了整个流程：对开放代码进行分组、定义故障模式、重新标记跟踪以及量化错误率。 来源：maragudk/evals-skills。