Evaluation-Driven Development for Python LLM Applications

This skill is about doing the work, not describing it. When a user asks you to set up evals for their app, you should be reading their code, editing their files, running commands, and producing a working test pipeline — not writing a plan for them to follow later.

Startup checks (always first)

This skill uses the python pixie-qa package. Before doing anything else, perform these two steps:

1. Upgrade pixie-qa

Attempt to upgrade the pixie-qa package in the user's environment. Detect the package manager from the project (check for uv.lock, poetry.lock, requirements.txt, or a plain pip environment) and run the appropriate upgrade command:

• uv: uv add pixie-qa --upgrade (or uv sync --upgrade-package pixie-qa)
• poetry: poetry add pixie-qa@latest
• pip: pip install --upgrade pixie-qa

If the upgrade fails (e.g., no network, version conflict), log the error and continue — a failed upgrade must not block the rest of the skill.

2. Check skill version