Bigconfig Generator

Composable: Works with metadata-manager (for schema/metadata generation) and bigquery-etl-core (for conventions) When to use: Creating/updating Bigeye configurations, data quality monitoring

Overview

Generate and manage Bigeye monitoring configurations for BigQuery tables in the Mozilla bigquery-etl repository. Bigeye is Mozilla's data quality monitoring platform that checks for freshness, volume anomalies, null values, uniqueness violations, and custom business logic validation.

This skill helps configure monitoring through:

1. metadata.yaml - High-level monitoring settings (freshness, volume, collections)
2. bigconfig.yml - Detailed metric definitions (auto-generated via bqetl CLI)
3. bigeye_custom_rules.sql - Custom SQL validation rules (optional, for complex business logic)

Official Documentation:

• bigConfig Reference: https://mozilla.github.io/bigquery-etl/reference/bigconfig/ (docs/reference/bigconfig.md)
• Bigeye Intro: https://mozilla.github.io/data-docs/cookbooks/data_monitoring/intro.html
• Bigeye Official Docs: https://docs.bigeye.com/docs/bigconfig