data-pipeline-engineering
Data Pipeline Engineering Skill
Purpose
Design and implement robust data pipelines for the CIA platform that extract, transform, and load Swedish political data from multiple sources into the internal data model. Covers Spring Integration, batch processing, and monitoring patterns.
When to Use
- ✅ Building data import pipelines for Riksdagen data
- ✅ Designing ETL workflows for political data aggregation
- ✅ Implementing batch processing for large datasets
- ✅ Creating data refresh and synchronization jobs
- ✅ Monitoring pipeline health and data quality
Do NOT use for:
- ❌ Real-time API request handling (use api-integration skill)
- ❌ UI data binding (use vaadin-component-design skill)
Pipeline Architecture
More from hack23/cia
iso-27001-controls
Verify implementation of ISO 27001:2022 information security controls across CIA platform development and operations
15playwright-ui-testing
Playwright browser automation, visual regression testing, accessibility testing, and E2E workflow validation for CIA platform
15ui-ux-design-system
Design system management, Vaadin component library patterns, consistent UI/UX, accessibility integration
15code-quality-checks
Enforce code quality with SonarCloud, CheckStyle, SpotBugs, and maintain quality gates
14business-model-canvas
Business Model Canvas framework for value proposition, customer segments, revenue streams, and sustainable business model design
14legislative-monitoring
Voting pattern analysis, committee effectiveness, bill tracking, parliamentary oversight for Swedish intelligence operations
13