Harvard Artifacts Data Engineering Analytics

Skill by ara.so — Data Skills collection

This project provides an end-to-end data engineering and analytics application for the Harvard Art Museums API. It demonstrates real-world ETL pipelines, SQL database design, analytical queries, and interactive visualization using Streamlit.

What This Project Does

The application implements a complete data pipeline:

• Extract: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting
• Transform: Processes nested JSON into relational database tables (metadata, media, colors)
• Load: Batch inserts transformed data into MySQL/TiDB Cloud
• Analyze: Executes 20+ predefined SQL queries for insights
• Visualize: Renders interactive dashboards with Plotly charts in Streamlit

Installation