Dagster Development Expert

Core Philosophy

Think in Assets: Dagster is built around the asset abstraction—persistent objects like tables, files, or models that your pipeline produces. Assets provide:

• Clear Lineage: Explicit dependencies define data flow
• Better Observability: Track what data exists and how it was created
• Improved Testability: Assets are just Python functions that can be tested directly
• Declarative Pipelines: Focus on what to produce, not how to execute

Assets over Ops: For most data pipelines, prefer assets over ops. Use ops only when the asset abstraction doesn't fit (non-data workflows, complex execution patterns).

Environment Separation: Use resources and EnvVar to maintain separate configurations for dev, staging, and production without code changes.

Quick Reference