Harvard Art Museum ETL Pipeline

Skill by ara.so — Data Skills collection.

This skill enables you to build end-to-end data engineering and analytics applications using the Harvard Art Museums API. The project demonstrates real-world ETL pipelines, SQL database design, analytical queries, and interactive Streamlit dashboards for museum artifact data.

What This Project Does

The Harvard Artifacts Collection app provides a complete data engineering workflow:

• Extract: Fetch artifact data from Harvard Art Museums API with pagination and rate limiting
• Transform: Convert nested JSON into normalized relational tables (metadata, media, colors)
• Load: Batch insert data into MySQL/TiDB Cloud with proper foreign key relationships
• Analyze: Execute 20+ predefined SQL queries for insights
• Visualize: Display results in interactive Plotly charts via Streamlit

The architecture follows: API → ETL → SQL → Analytics → Visualization

Installation