hebrew-llm-eval-suite

Pass

Audited by Gen Agent Trust Hub on Oct 14, 2026

Risk Level: SAFEINDIRECT_PROMPT_INJECTION
Full Analysis
  • [INDIRECT_PROMPT_INJECTION]: The skill exhibits an inherent attack surface for indirect prompt injection through its primary functionality of processing external datasets.
  • Ingestion points: The scripts/run_eval.py script utilizes the datasets library to fetch content from HuggingFace repositories (e.g., Etelis/HeQ_v1, HebArabNlpProject/HebrewSentiment).
  • Boundary markers: Prompts in references/prompt-templates.md use basic labels (e.g., Passage: {passage}) to separate instructions from data, but do not employ advanced delimiters or explicit warnings to the model to ignore instructions embedded within the processed Hebrew text.
  • Capability inventory: The skill possesses network access to call AI provider APIs (Anthropic, OpenAI, Google, AI21) and file-system access to write evaluation results to disk via scripts/run_eval.py and scripts/make_scorecard.py.
  • Sanitization: The harness performs minimal sanitization, relying on direct string interpolation of external data into the prompt templates.
  • [SAFE]: The skill performs network operations to fetch configuration and datasets from HuggingFace, which is a well-known service for machine learning resources. These actions are documented and consistent with the skill's purpose.
  • [SAFE]: The evaluation harness (scripts/run_eval.py) connects to official model provider endpoints for Anthropic, OpenAI, Google, and AI21. These are recognized, well-known services and the network activity is expected for an LLM benchmarking tool.
Audit Metadata
Risk Level
SAFE
Analyzed
Oct 14, 2026, 08:28 AM