The Agent Skills Directory

[COMMAND_EXECUTION]: The skill provides instructions to execute several shell commands for managing the evaluation environment. This includes unlocking datasets with transcrypt, running validation tests with pytest, and executing the core evaluation runner main.py using the uv tool.
[INDIRECT_PROMPT_INJECTION]: The skill defines a workflow for authoring multi-turn conversation datasets that include system prompts and tool call examples which are processed by an LLM-as-a-judge.
Ingestion points: Data is read from YAML files located in eval/datasets/.
Boundary markers: The instructions do not specify the use of delimiters or boundary markers for these inputs.
Capability inventory: The main.py script performs model inference and scoring based on the messages, function calls, and qualitative targets defined in the datasets.
Sanitization: No explicit sanitization or filtering of the dataset content is described in the workflow.

a2ui-add-eval-datapoint