Harvard Artifacts Collection Data Engineering & Analytics App

Skill by ara.so — Data Skills collection.

This skill enables AI coding agents to help developers build and work with an end-to-end data engineering application that collects artifact data from the Harvard Art Museums API, performs ETL operations, stores data in SQL databases, and creates interactive analytics dashboards using Streamlit.

What This Project Does

The Harvard Artifacts Collection app demonstrates a complete data pipeline:

1. Extract: Fetches artifact data from Harvard Art Museums API with pagination
2. Transform: Processes nested JSON into relational database schema
3. Load: Batch inserts into MySQL/TiDB Cloud with foreign key relationships
4. Analyze: Runs 20+ predefined SQL analytical queries
5. Visualize: Displays results in interactive Streamlit dashboards with Plotly charts

The architecture follows: API → ETL → SQL → Analytics → Visualization

Installation