Data Export

Saving financial data as CSV loses type information, bloats file size 5-10x, and makes every downstream read parse strings back into numbers - a tax paid on every pipeline run.

The Problem

CSV is human-readable but machine-hostile: no type information (dates become strings, integers become floats), no compression (a 100MB Parquet file becomes 500MB CSV), no predicate pushdown (you must read the entire file to filter one column), and no schema enforcement (a renamed column breaks silently). For financial data pipelines where the same dataset is read hundreds of times by different notebooks, the cumulative cost of CSV is enormous in both time and correctness.

The Pattern

WRONG

import pandas as pd

# CSV: no types, no compression, slow reads, 5x file size
df.to_csv("prices.csv", index=False)