data-pipeline-engineering
Data Pipeline Engineering Skill
🔴 AI FIRST Quality Principle
Apply the AI FIRST principle: never accept first-pass quality. Minimum 2 iterations. Read all output, improve every section. No shortcuts.
Purpose
Expert knowledge in designing robust ETL (Extract, Transform, Load) pipelines for automated data processing, focusing on reliability, monitoring, and maintainability.
Core Principles
- Idempotency - Pipeline runs produce same results
- Observability - Full visibility into pipeline health
- Error Recovery - Graceful handling of failures
- Version Tracking - Track all data changes
- Monitoring - Real-time pipeline health checks
Enforces
- ETL workflow patterns (Extract → Transform → Load)
More from hack23/riksdagsmonitor
osint-methodologies
OSINT collection, source evaluation, data integration, verification techniques for Swedish political intelligence
41economic-policy-analysis
Fiscal policy, budget analysis, economic forecasting, monetary policy, trade policy for political journalists
33electoral-analysis
Election forecasting models, campaign analysis, coalition prediction, voter behavior analysis for Swedish elections
25vulnerability-management
Vulnerability scanning, assessment, prioritization, and remediation processes following NIST and CIS Controls
25nist-csf-mapping
NIST Cybersecurity Framework 2.0 mapping for static HTML/CSS websites
24testing-strategy
Comprehensive testing strategy covering unit, integration, E2E, security, accessibility, and performance testing
23