Harvard Artifacts Data Engineering App

Skill by ara.so — Data Skills collection.

This project demonstrates end-to-end data engineering using the Harvard Art Museums API. It extracts artifact data, transforms it into relational tables, loads it into SQL databases (MySQL/TiDB), and provides interactive analytics through Streamlit dashboards with Plotly visualizations.

What It Does

• API Integration: Fetches paginated artifact data from Harvard Art Museums API
• ETL Pipeline: Transforms nested JSON into normalized relational tables
• SQL Storage: Creates and populates artifactmetadata, artifactmedia, and artifactcolors tables
• Analytics Queries: 20+ predefined SQL queries for artifact insights
• Interactive Visualization: Streamlit dashboard with Plotly charts

Installation