The Agent Skills Directory

[EXTERNAL_DOWNLOADS]: The skill downloads the 'uv' tool from the well-known service 'astral.sh' and executes its installation script via the shell in scripts/nel-next.sh and recipes/examples/gym/example_mrcr.yaml.
[REMOTE_CODE_EXECUTION]: The skill installs the nemo-evaluator and nemo-evaluator-launcher packages directly from GitHub using the uvx tool at specific commit SHAs. It also fetches the livecodebench and NVIDIA-NeMo/Gym packages from GitHub repositories as part of its setup routine.
[COMMAND_EXECUTION]: The skill uses shell scripts (scripts/nel-next.sh, scripts/nel-gym.sh) for environment orchestration and includes pre_cmd instructions in its configuration files to execute setup commands inside containers, such as package installation or file retrieval.
[DYNAMIC_EXECUTION]: In references/doom_loops.md, the skill provides a script to dynamically patch the source code of the vllm library at runtime within the serving container to remap status codes for specific benchmarks.
[INDIRECT_PROMPT_INJECTION]: The skill evaluates untrusted data in the form of model outputs and benchmark results, creating a potential surface for indirect prompt injection.
Ingestion points: Untrusted data enters the agent context through model responses and benchmark rollout artifacts processed in references/run-validation.md and recipes/tasks/gym/mrcr.md.
Boundary markers: The skill uses structured YAML and JSON formats for communication, though it does not explicitly implement natural language boundary markers for all ingested model outputs.
Capability inventory: The skill possesses the capability to execute shell commands, perform file operations, and make network requests to judge and inference endpoints.
Sanitization: A 'Bounded Evaluation-Failure Policy' described in references/run-validation.md is implemented to detect and handle malformed or systemic failure cases in model responses before scores are processed.

modelopt-evaluation