Prefect

Overview

Prefect 3 turns Python functions into flows (@flow) made of tasks (@task), with retries, caching, logging, a UI and scheduled deployments. Two ways to run on a schedule: flow.serve(...) starts one long-running process that executes runs itself, and work pools plus workers (prefect deploy and prefect worker start) execute runs in managed infrastructure. Agents and prefect.deployments.Deployment from Prefect 2 are removed: importing Deployment raises an error.

Instructions

Install and run a server

python3 -m venv .venv && source .venv/bin/activate
pip install prefect httpx            # needs Python 3.11 or newer
prefect version

prefect server start                 # UI and API at http://127.0.0.1:4200
# In another shell, point clients at it:
prefect config set PREFECT_API_URL=http://127.0.0.1:4200/api