vaex

SKILL.md

Vaex is a high-performance Python library designed for lazy, out-of-core DataFrames to process and visualize tabular datasets that are too large to fit into RAM. Vaex can process over a billion rows per second, enabling interactive data exploration and analysis on datasets with billions of rows.

Vaex provides six primary capability areas, each documented in detail in the references directory:

Load and create Vaex DataFrames from various sources including files (HDF5, CSV, Arrow, Parquet), pandas DataFrames, NumPy arrays, and dictionaries. Reference references/coredataframes.md for:

使用此技能來處理和分析超出可用 RAM 的大型表格資料集（數十億行）。 Vaex 擅長核外 DataFrame 操作、惰性求值、快速聚合、大數據的高效視覺化以及大型資料集上的機器學習。當使用者需要處理大型 CSV/HDF5/Arrow/Parquet 檔案、對海量資料集執行快速統計、建立大數據視覺化或建立不適合記憶體的 ML 管道時適用。 來源：jackspace/claudeskillz。

查看原文