Harvard Art Museums ETL Pipeline

Skill by ara.so — Data Skills collection.

This project provides an end-to-end data engineering solution for extracting, transforming, and analyzing artifacts from the Harvard Art Museums API. It demonstrates real-world ETL patterns, SQL analytics, and interactive visualization using Streamlit.

What It Does

• Extracts artifact data from Harvard Art Museums API with pagination and rate limiting
• Transforms nested JSON into normalized relational tables (metadata, media, colors)
• Loads data into MySQL/TiDB Cloud with proper foreign key relationships
• Analyzes data using predefined SQL queries for cultural insights
• Visualizes results through interactive Plotly charts in a Streamlit dashboard

Installation