The Agent Skills Directory

Command Execution: The skill instructs the agent to execute shell commands such as npm run test:vally and npx @microsoft/vally-cli. These commands are used to initiate the evaluation runner and validate the structure of test files within a local development environment.
External Resource Access: The skill references and executes the @microsoft/vally-cli package from the public npm registry and links to documentation on Microsoft's GitHub pages. These resources are utilized to provide the core functionality of the Vally evaluation framework.
Dynamic Plugin Loading: The configuration allows for loading custom TypeScript modules (e.g., vally-executor.ts and vally-graders.ts) as plugins. This mechanism is designed to extend the test runner's capabilities with custom logic for specific testing scenarios.
CI/CD Integration Patterns: The documentation describes how to incorporate testing into GitHub Actions, which includes standard practices like environment provisioning and test result collection to monitor agent performance over time.

vally-eval