failure-taxonomy

Transform raw, freeform trace annotations from open coding sessions into a structured taxonomy of binary failure modes, following the grounded theory methodology from the Analyze-Measure-Improve evaluation lifecycle.

The user has already completed open coding — they've read through LLM pipeline traces and written short, freeform notes describing what went wrong (the "point of first failure").

Now they need to move from that chaotic pile of observations into an organized, actionable taxonomy. This is the axial coding step.

從開放編碼的追蹤註釋建構故障模式的結構化分類法。每當使用者透過查看 LLM 追蹤獲得自由格式註釋並希望將它們聚類為連貫的、不重疊的二進位故障類別集（軸向編碼）時，請使用此技能。當使用者提到「故障模式」、「錯誤分類」、「軸編碼」、「聚類註釋」、「錯誤分類」、「故障分析」或想要從原始觀察筆記轉向結構化評估標準時也可以使用。這項技能涵蓋了整個流程：對開放程式碼進行分組、定義故障模式、重新標記追蹤以及量化錯誤率。 來源：maragudk/evals-skills。