ai-observability

Pass

Audited by Gen Agent Trust Hub on Oct 8, 2026

Risk Level: SAFEINDIRECT_PROMPT_INJECTION
Full Analysis
  • [INDIRECT_PROMPT_INJECTION]: The skill implements 'LLM-as-judge' patterns for RAG evaluation and hallucination detection that are susceptible to indirect prompt injection.
  • Ingestion points: External data such as retrieved contexts, user questions, and AI-generated answers are interpolated directly into evaluation prompts in references/patterns.md (specifically in functions evaluateFaithfulness, evaluateAnswerRelevancy, evaluateContextPrecision, and factCheckAgainstContext).
  • Boundary markers: The skill uses basic text labels (e.g., Context:, Question:, Answer:) to separate data from instructions. These markers are insufficient to prevent adversarial data from breaking out of its context and overriding the evaluation task.
  • Capability inventory: The skill utilizes anthropic.messages.create to execute these evaluations. A successful injection could lead to poisoned metrics, where an attacker-controlled context forces the evaluator to report a high faithfulness score regardless of actual quality.
  • Sanitization: Although the skill includes a detailed pattern for PII sanitization in references/sharp_edges.md, it does not provide sanitization or escaping mechanisms specifically designed to prevent prompt injection within the evaluation pipeline.
  • [SAFE]: The skill follows security best practices by recommending the use of environment variables for API keys and providing regex-based sanitization patterns to prevent PII leakage into tracing services. It uses well-known services for observability (Langfuse, Helicone) and standard SDKs.
Audit Metadata
Risk Level
SAFE
Analyzed
Oct 8, 2026, 09:51 PM