Harvard Art Museums Data Engineering Pipeline

Skill by ara.so — Data Skills collection.

Overview

This project demonstrates a production-grade data engineering workflow that:

• Extracts artifact data from Harvard Art Museums API with pagination and rate limiting
• Transforms nested JSON into normalized relational tables
• Loads data into MySQL/TiDB Cloud databases
• Executes analytical SQL queries for insights
• Visualizes results through interactive Streamlit dashboards

The application showcases real-world ETL patterns, database design, and data visualization techniques used in analytics engineering roles.

Installation