databricks-execution-compute
Installation
SKILL.md
Databricks Execution & Compute
Run code on Databricks. Three execution modes—choose based on workload.
Execution Mode Decision Matrix
| Aspect | Databricks Connect ⭐ | Serverless Job | Interactive Cluster |
|---|---|---|---|
| Use for | Spark code (ETL, data gen) | Heavy processing (ML) | State across tool calls, Scala/R |
| Startup | Instant | ~25-50s cold start | ~5min if stopped |
| State | Within Python process | None | Via context_id |
| Languages | Python (PySpark) | Python, SQL | Python, Scala, SQL, R |
| Dependencies | withDependencies() |
CLI with environments spec | Install on cluster |