vaex

SKILL.md

Vaex is a high-performance Python library designed for lazy, out-of-core DataFrames to process and visualize tabular datasets that are too large to fit into RAM. Vaex can process over a billion rows per second, enabling interactive data exploration and analysis on datasets with billions of rows.

Vaex provides six primary capability areas, each documented in detail in the references directory:

Load and create Vaex DataFrames from various sources including files (HDF5, CSV, Arrow, Parquet), pandas DataFrames, NumPy arrays, and dictionaries. Reference references/coredataframes.md for:

使用此技能來處理和分析超出可用 RAM 的大型表格數據集（數十億行）。 Vaex 擅長核外 DataFrame 操作、惰性求值、快速聚合、大數據的高效可視化以及大型數據集上的機器學習。當用戶需要處理大型 CSV/HDF5/Arrow/Parquet 文件、對海量數據集執行快速統計、創建大數據可視化或構建不適合內存的 ML 管道時適用。 來源：ovachiever/droid-tings。

查看原文