Run Evals

Guide users through writing tasks, running evaluations, scoring results, and configuring execution with the ZeroEval Python SDK.

When To Use

• Defining a @ze.task to run against a benchmark dataset.
• Running evals with dataset.eval().
• Writing row-level, column-level, or run-level evaluations.
• Using column_map to bind evaluator arguments to dataset columns.
• Emitting runtime signals during task execution.
• Configuring workers, retries, timeouts, and checkpoints.
• Repeating evals or resuming interrupted runs.
• Inspecting eval results, metrics, and health.

Execution Sequence

Follow these steps in order. Load reference files for detailed patterns and configuration.