Write MCP Eval Datasets

Generate data-driven eval datasets for MCP server tools using @gleanwork/mcp-server-tester. Eval datasets are JSON files containing test cases that are loaded and executed by the eval runner.

When to Use Evals vs Inline Tests

Use eval datasets when:

• You have many similar test cases for the same tool (10+ cases)
• Test data changes more frequently than test logic
• Non-engineers need to add or review test cases
• You want to track accuracy over time with baselines

Use inline Playwright tests (see write-mcp-test skill) when:

• You need complex setup or teardown logic
• Tests have unique validation requirements
• You're testing tool interactions or multi-step flows