02-ref-hallucination-arena

Pass

Audited by Gen Agent Trust Hub on Oct 25, 2026

Risk Level: SAFEEXTERNAL_DOWNLOADSCOMMAND_EXECUTIONINDIRECT_PROMPT_INJECTION
Full Analysis
  • [EXTERNAL_DOWNLOADS]: The skill requires the installation of the py-openjudge and matplotlib packages via pip. These are standard libraries used for LLM evaluation and data visualization respectively.
  • [COMMAND_EXECUTION]: The skill provides CLI instructions to execute the evaluation pipeline using python -m cookbooks.ref_hallucination_arena. This command runs the benchmarking logic contained within the installed or local package.
  • [INDIRECT_PROMPT_INJECTION]: The skill processes external data that could contain adversarial instructions.
  • Ingestion points: Data enters the agent context through the dataset.path (JSON/JSONL files) and raw responses from target LLM endpoints.
  • Boundary markers: The instructions do not define specific delimiters or "ignore previous instructions" guards for the interpolated data.
  • Capability inventory: The skill performs network operations (API calls to Crossref, PubMed, arXiv, DBLP, and model providers) and file system operations (writing results and reports to the output directory).
  • Sanitization: There is no explicit mention of sanitization or validation of the content within the queries or model responses before they are processed or written to reports.
  • [SAFE]: The skill implements secure credential handling by instructing the user to provide API keys (OpenAI, Dashscope, etc.) via environment variables rather than hardcoding them. It also interacts exclusively with well-known, legitimate academic and technology services.
Audit Metadata
Risk Level
SAFE
Analyzed
Oct 25, 2026, 04:47 AM