Welcome to the comprehensive data engineering skill suite. This hub organizes all data engineering knowledge into logical, non-overlapping domains.
| Core | @data-engineering-core | Polars, DuckDB, PyArrow fundamentals; ETL patterns; error handling; performance optimization | | Storage | @data-engineering-storage-lakehouse | Delta Lake, Apache Iceberg, Apache Hudi | | | @data-engineering-storage-remote-access | fsspec, pyarrow.fs, obstore; cloud access patterns |
| | @data-engineering-storage-authentication | AWS, GCP, Azure auth - IAM roles, managed identity, secrets management | | | @data-engineering-storage-formats | Parquet optimizations, Lance, Zarr, Avro, ORC | | Orchestration | @data-engineering-orchestration | Prefect, Dagster, dbt, workflow scheduling |
Suite complète de compétences en ingénierie de données couvrant les bibliothèques principales (Polars, DuckDB, PyArrow), les formats Lakehouse, le stockage cloud, l'orchestration, le streaming, la qualité, l'observabilité et les pipelines IA/ML. Source : legout/data-platform-agent-skills.