Databricks CI Integration

Overview

Automate Databricks deployments with Declarative Automation Bundles (DABs) and GitHub Actions. Covers bundle validation, unit testing PySpark transforms locally, deploying to staging on PR, production on merge, and integration testing against live workspaces. Uses databricks/setup-cli action and OAuth M2M for secure CI auth.

Prerequisites

• Databricks workspace with service principal (OAuth M2M)
• Asset Bundle (databricks.yml) configured
• GitHub repo with Actions enabled
• GitHub environment secrets: DATABRICKS_HOST, DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET

Instructions

Step 1: GitHub Actions — Validate and Test on PR

# .github/workflows/databricks-ci.yml
name: Databricks CI