promptfoo-evaluation

A promptfooconfig.yaml declares providers (LLMs under test), prompts (templates with {{var}} placeholders), tests (input variables + assertions); promptfoo eval runs the cross-product (per pf-config).

When to use

• The repo has a promptfooconfig.yaml or the user wants to author one.
• The user needs prompt regression suites - same inputs, multiple providers, fail-on-diff.
• A CI workflow needs an eval gate on prompt or model changes.
• The team prefers vendor-neutral eval (vs OpenAI Evals' OpenAI-first posture).