Welcome to the comprehensive data engineering skill suite. This hub organizes all data engineering knowledge into logical, non-overlapping domains.
| Core | @data-engineering-core | Polars, DuckDB, PyArrow fundamentals; ETL patterns; error handling; performance optimization | | Storage | @data-engineering-storage-lakehouse | Delta Lake, Apache Iceberg, Apache Hudi | | | @data-engineering-storage-remote-access | fsspec, pyarrow.fs, obstore; cloud access patterns |
| | @data-engineering-storage-authentication | AWS, GCP, Azure auth - IAM roles, managed identity, secrets management | | | @data-engineering-storage-formats | Parquet optimizations, Lance, Zarr, Avro, ORC | | Orchestration | @data-engineering-orchestration | Prefect, Dagster, dbt, workflow scheduling |
Umfassendes Wissenspaket zur Datentechnik, das Kernbibliotheken (Polars, DuckDB, PyArrow), Lakehouse-Formate, Cloud-Speicher, Orchestrierung, Streaming, Qualität, Beobachtbarkeit und AI/ML-Pipelines abdeckt. Quelle: legout/data-platform-agent-skills.