Dataform Engineering Fundamentals

Overview

Core principle: Safety practices and proper architecture are NEVER optional in Dataform development, regardless of time pressure or business urgency. The time "saved" by shortcuts gets multiplied into hours of debugging, broken dependencies, and production issues.

REQUIRED FOUNDATION: This skill builds upon superpowers:test-driven-development. All TDD principles from that skill apply here; this skill adapts them for BigQuery Dataform SQLX files.

For PostgreSQL / Supabase: use sqlanvil-engineering-fundamentals instead (or alongside) — the architecture, ${ref()}, TDD, and columns:{} rules below carry over unchanged; that skill covers the Postgres/Supabase deltas (config blocks, credentials, DDL, named connections).

Official docs: syntax/config/API at https://cloud.google.com/dataform/docs · repository structure & naming at https://cloud.google.com/dataform/docs/best-practices-repositories

When to Use

Use this skill for ANY Dataform work — new SQLX transformations, modifying tables, adding sources, troubleshooting pipelines, and "quick" reports or ad-hoc analysis.

Especially when under time pressure, stakeholders are waiting, you're working late and exhausted, or you're tempted to "just make it work" — discipline matters most exactly when it's tempting to skip.