llm-eval-harness

Pass

Audited by Gen Agent Trust Hub on Sep 7, 2026

Risk Level: SAFEINDIRECT_PROMPT_INJECTIONCOMMAND_EXECUTIONEXTERNAL_DOWNLOADS
Full Analysis
  • [INDIRECT_PROMPT_INJECTION]: The skill ingests text generated by external, untrusted LLMs during quality benchmarks and interpolates it into prompts for "blind judge" agents to evaluate. This creates a vulnerability surface where a malicious model could provide a response containing instructions designed to override the judge's grading criteria.
  • Ingestion points: scripts/usecase_runner.py (line 78) extracts the model's message content, and references/quality_blind_judge.md contains the template for the grading prompt.
  • Boundary markers: The judge prompt template uses structural delimiters like MODEL ANSWER: and RUBRIC:, but it lacks explicit instructions for the judge to disregard any commands contained within the model's output.
  • Capability inventory: The evaluation scripts possess the capability to perform network requests to arbitrary endpoints and write results to the local filesystem.
  • Sanitization: The skill does not perform sanitization or escaping of the model-generated text before using it in judging prompts.
  • [COMMAND_EXECUTION]: The skill consists of several Python scripts that execute network requests and file system operations to perform benchmarks. These operations are triggered by the user and target user-provided endpoints.
  • [EXTERNAL_DOWNLOADS]: The skill documentation recommends using the uv tool to install standard, well-known libraries such as aiohttp, openai, and tiktoken from the official PyPI registry.
Audit Metadata
Risk Level
SAFE
Analyzed
Sep 7, 2026, 03:06 AM
Security Audit — agent-trust-hub — llm-eval-harness