Polars

Polars is an ultra-fast DataFrame library written in Rust, leveraging Apache Arrow columnar format, lazy evaluation, and multi-threaded parallel query execution.

When to Use

• High-Performance In-Memory & Streaming Dataframes: Rust-backed, multi-threaded columnar data processing 5x-30x faster than Pandas.
• Out-of-Core Larger-Than-Memory Analytics: Streaming datasets larger than RAM with the Polars Lazy API (collect(streaming=True)).
• Parallel Query Execution: Automatic query optimization, predicate pushdown, and projection pushdown via the Catalyst-like engine.
• Zero-Copy Apache Arrow Interoperability: Seamlessly sharing memory with PyTorch, DuckDB, and Parquet readers.

Quick Start

import polars as pl