pandas DataFrame AI Skill Guide

Overview & Engine Architecture

pandas is the default Python tabular toolkit: Series and DataFrame over NumPy (and Arrow-backed dtypes in newer builds). IO loaders materialize tables; vectorized ops and groupby/merge reshape them. Agents prefer explicit dtypes, avoid row-wise apply on large frames, and profile memory before chaining copies.

CSV/Parquet/SQL
      -> DataFrame (columns = Series)
          -> filter / assign / groupby / merge
          -> to_parquet / to_sql / plot

When to use this skill

• Cleaning and joining tabular files
• Feature tables for ML (@scikit-learn, @pytorch)
• Quick exploratory aggregates before moving to Polars/DuckDB/Spark