data-quality-frameworks
Validate data pipelines with Great Expectations, dbt tests, and data contracts.
- Covers three complementary frameworks: Great Expectations for statistical and schema validation, dbt tests for transformation layer checks, and data contracts for cross-team data agreements
- Includes six core quality dimensions (completeness, uniqueness, validity, accuracy, consistency, timeliness) with ready-to-use expectation patterns and custom test examples
- Provides checkpoint automation for CI/CD integration, Slack notifications on failure, and orchestrated validation pipelines across multiple tables
- Supports both generic reusable tests and singular business-logic tests, with data contract specifications for SLA, freshness, and PII classification
Data Quality Frameworks
Production patterns for implementing data quality with Great Expectations, dbt tests, and data contracts to ensure reliable data pipelines.
When to Use This Skill
- Implementing data quality checks in pipelines
- Setting up Great Expectations validation
- Building comprehensive dbt test suites
- Establishing data contracts between teams
- Monitoring data quality metrics
- Automating data validation in CI/CD
Core Concepts
1. Data Quality Dimensions
| Dimension | Description | Example Check |
|---|
More from wshobson/agents
tailwind-design-system
Build scalable design systems with Tailwind CSS v4, design tokens, component libraries, and responsive patterns. Use when creating component libraries, implementing design systems, or standardizing UI patterns.
41.0Ktypescript-advanced-types
Master TypeScript's advanced type system including generics, conditional types, mapped types, template literals, and utility types for building type-safe applications. Use when implementing complex type logic, creating reusable type utilities, or ensuring compile-time type safety in TypeScript projects.
40.4Knodejs-backend-patterns
Build production-ready Node.js backend services with Express/Fastify, implementing middleware patterns, error handling, authentication, database integration, and API design best practices. Use when creating Node.js servers, REST APIs, GraphQL backends, or microservices architectures.
31.8Kpython-performance-optimization
Profile and optimize Python code using cProfile, memory profilers, and performance best practices. Use when debugging slow Python code, optimizing bottlenecks, or improving application performance.
22.1Kapi-design-principles
Master REST and GraphQL API design principles to build intuitive, scalable, and maintainable APIs that delight developers. Use when designing new APIs, reviewing API specifications, or establishing API design standards.
20.3Kpython-testing-patterns
Implement comprehensive testing strategies with pytest, fixtures, mocking, and test-driven development. Use when writing Python tests, setting up test suites, or implementing testing best practices.
19.7K