generate-synthetic-data

Generate diverse, realistic test inputs that cover the failure space of an LLM pipeline.

Before generating synthetic data, identify where the pipeline is likely to fail. Ask the user about known failure-prone areas, review existing user feedback, or form hypotheses from available traces. Dimensions (Step 1) must target anticipated failures, not arbitrary variation.

Dimensions are axes of variation specific to your application. Choose dimensions based on where you expect failures.

使用基於維度的元組產生為 LLM 管道評估建立多樣化的綜合測試輸入。在引導評估資料集、真實使用者資料稀疏或對特定故障假設進行壓力測試時使用。當您已經擁有 100 多個代表性真實追蹤（改為使用分層採樣）或任務正在收集生產日誌時，請勿使用。 來源：hamelsmu/evals-skills。