Skillgrade Evaluation Setup

Procedures

Step 1: Install Skillgrade

1. Verify Node.js 20+ and Docker are available.
2. Run npm i -g skillgrade to install the CLI globally.

Step 2: Initialize an Eval Configuration

1. Navigate to the skill directory (must contain a SKILL.md).
2. Set the appropriate API key environment variable (GEMINI_API_KEY, ANTHROPIC_API_KEY, or OPENAI_API_KEY).
3. Run skillgrade init to generate an eval.yaml with AI-powered tasks and graders.
4. If an eval.yaml already exists, pass --force to overwrite: skillgrade init --force.
5. Without an API key, a well-commented template is generated instead.