ai-data-engineering

SKILL.md

Build data infrastructure for AI/ML systems including RAG pipelines, feature stores, and embedding generation. Provides architecture patterns, orchestration workflows, and evaluation metrics for production AI applications.

RAG pipelines have 5 distinct stages. Understanding this architecture is critical for production implementations.

Chunking is the most critical decision for RAG quality. Poor chunking breaks retrieval.

Data pipelines, feature stores, and embedding generation for AI/ML systems. Use when building RAG pipelines, ML feature serving, or data transformations. Covers feature stores (Feast, Tecton), embedding pipelines, chunking strategies, orchestration (Dagster, Prefect, Airflow), dbt transformations, data versioning (LakeFS), and experiment tracking (MLflow, W&B). Source: ancoleman/ai-design-components.

View raw