Data acceleration materializes working sets of data locally, reducing query latency from seconds to milliseconds. Hot data gets materialized for instant access while cold data remains federated.
Unlike traditional caches that store query results, Spice accelerates entire datasets with configurable refresh strategies and the flexible compute of an embedded database.
| Small datasets (<1 GB), max speed | arrow | In-memory, lowest latency | | Medium datasets (1-100 GB), complex SQL | duckdb | Mature SQL, memory management | | Large datasets (100 GB-1+ TB), analytics | cayenne | Built on Vortex (Linux Foundation), 10-20x faster scans | | Point lookups on large datasets | cayenne | 100x faster random access vs Parquet |
Accelerate data locally for sub-second query performance. Use when enabling data acceleration, choosing an engine (Arrow, DuckDB, SQLite, Cayenne), configuring refresh modes, setting up retention policies, creating snapshots, adding indexes, or materializing datasets. Source: spiceai/skills.