Data Pipeline Engineering Skill

Purpose

Design and implement robust data pipelines for the CIA platform that extract, transform, and load Swedish political data from multiple sources into the internal data model. Covers Spring Integration, batch processing, and monitoring patterns.

When to Use

• ✅ Building data import pipelines for Riksdagen data
• ✅ Designing ETL workflows for political data aggregation
• ✅ Implementing batch processing for large datasets
• ✅ Creating data refresh and synchronization jobs
• ✅ Monitoring pipeline health and data quality

Do NOT use for:

• ❌ Real-time API request handling (use api-integration skill)
• ❌ UI data binding (use vaadin-component-design skill)

Pipeline Architecture