Harvard Artifacts Collection Data Engineering & Analytics

Skill by ara.so — Data Skills collection.

This project provides an end-to-end data engineering and analytics application for the Harvard Art Museums API. It demonstrates real-world ETL pipelines, SQL database design, analytical queries, and interactive Streamlit visualizations for museum artifact data.

What This Project Does

• Data Collection: Fetches artifact data from Harvard Art Museums API with pagination and rate limiting
• ETL Pipeline: Extracts, transforms, and loads artifact metadata, media, and color information into relational SQL databases
• SQL Analytics: Runs 20+ predefined analytical queries for insights on artifacts by culture, century, department, and media
• Visualization: Creates interactive Plotly dashboards in Streamlit for exploring query results
• Database Management: Manages MySQL/TiDB Cloud schemas with proper foreign key relationships

Installation

Prerequisites