llm-eval-harness
Pass
Audited by Gen Agent Trust Hub on Sep 7, 2026
Risk Level: SAFEINDIRECT_PROMPT_INJECTIONCOMMAND_EXECUTIONEXTERNAL_DOWNLOADS
Full Analysis
- [INDIRECT_PROMPT_INJECTION]: The skill ingests text generated by external, untrusted LLMs during quality benchmarks and interpolates it into prompts for "blind judge" agents to evaluate. This creates a vulnerability surface where a malicious model could provide a response containing instructions designed to override the judge's grading criteria.
- Ingestion points:
scripts/usecase_runner.py(line 78) extracts the model's message content, andreferences/quality_blind_judge.mdcontains the template for the grading prompt. - Boundary markers: The judge prompt template uses structural delimiters like
MODEL ANSWER:andRUBRIC:, but it lacks explicit instructions for the judge to disregard any commands contained within the model's output. - Capability inventory: The evaluation scripts possess the capability to perform network requests to arbitrary endpoints and write results to the local filesystem.
- Sanitization: The skill does not perform sanitization or escaping of the model-generated text before using it in judging prompts.
- [COMMAND_EXECUTION]: The skill consists of several Python scripts that execute network requests and file system operations to perform benchmarks. These operations are triggered by the user and target user-provided endpoints.
- [EXTERNAL_DOWNLOADS]: The skill documentation recommends using the
uvtool to install standard, well-known libraries such asaiohttp,openai, andtiktokenfrom the official PyPI registry.
Audit Metadata