ai-observability
Pass
Audited by Gen Agent Trust Hub on Oct 8, 2026
Risk Level: SAFEINDIRECT_PROMPT_INJECTION
Full Analysis
- [INDIRECT_PROMPT_INJECTION]: The skill implements 'LLM-as-judge' patterns for RAG evaluation and hallucination detection that are susceptible to indirect prompt injection.
- Ingestion points: External data such as retrieved contexts, user questions, and AI-generated answers are interpolated directly into evaluation prompts in
references/patterns.md(specifically in functionsevaluateFaithfulness,evaluateAnswerRelevancy,evaluateContextPrecision, andfactCheckAgainstContext). - Boundary markers: The skill uses basic text labels (e.g.,
Context:,Question:,Answer:) to separate data from instructions. These markers are insufficient to prevent adversarial data from breaking out of its context and overriding the evaluation task. - Capability inventory: The skill utilizes
anthropic.messages.createto execute these evaluations. A successful injection could lead to poisoned metrics, where an attacker-controlled context forces the evaluator to report a high faithfulness score regardless of actual quality. - Sanitization: Although the skill includes a detailed pattern for PII sanitization in
references/sharp_edges.md, it does not provide sanitization or escaping mechanisms specifically designed to prevent prompt injection within the evaluation pipeline. - [SAFE]: The skill follows security best practices by recommending the use of environment variables for API keys and providing regex-based sanitization patterns to prevent PII leakage into tracing services. It uses well-known services for observability (Langfuse, Helicone) and standard SDKs.
Audit Metadata