generate-synthetic-data

Generate diverse, realistic test inputs that cover the failure space of an LLM pipeline.

Before generating synthetic data, identify where the pipeline is likely to fail. Ask the user about known failure-prone areas, review existing user feedback, or form hypotheses from available traces. Dimensions (Step 1) must target anticipated failures, not arbitrary variation.

Dimensions are axes of variation specific to your application. Choose dimensions based on where you expect failures.

使用基于维度的元组生成为 LLM 管道评估创建多样化的综合测试输入。在引导评估数据集、真实用户数据稀疏或对特定故障假设进行压力测试时使用。当您已经拥有 100 多个代表性真实跟踪（改为使用分层采样）或任务正在收集生产日志时，请勿使用。 来源：hamelsmu/evals-skills。