hebrew-llm-eval-suite
Pass
Audited by Gen Agent Trust Hub on Oct 14, 2026
Risk Level: SAFEINDIRECT_PROMPT_INJECTION
Full Analysis
- [INDIRECT_PROMPT_INJECTION]: The skill exhibits an inherent attack surface for indirect prompt injection through its primary functionality of processing external datasets.
- Ingestion points: The
scripts/run_eval.pyscript utilizes thedatasetslibrary to fetch content from HuggingFace repositories (e.g.,Etelis/HeQ_v1,HebArabNlpProject/HebrewSentiment). - Boundary markers: Prompts in
references/prompt-templates.mduse basic labels (e.g.,Passage: {passage}) to separate instructions from data, but do not employ advanced delimiters or explicit warnings to the model to ignore instructions embedded within the processed Hebrew text. - Capability inventory: The skill possesses network access to call AI provider APIs (Anthropic, OpenAI, Google, AI21) and file-system access to write evaluation results to disk via
scripts/run_eval.pyandscripts/make_scorecard.py. - Sanitization: The harness performs minimal sanitization, relying on direct string interpolation of external data into the prompt templates.
- [SAFE]: The skill performs network operations to fetch configuration and datasets from HuggingFace, which is a well-known service for machine learning resources. These actions are documented and consistent with the skill's purpose.
- [SAFE]: The evaluation harness (
scripts/run_eval.py) connects to official model provider endpoints for Anthropic, OpenAI, Google, and AI21. These are recognized, well-known services and the network activity is expected for an LLM benchmarking tool.
Audit Metadata