creating-online-evaluations

Pass

Audited by Gen Agent Trust Hub on Nov 1, 2026

Risk Level: SAFEINDIRECT_PROMPT_INJECTIONDYNAMIC_EXECUTION
Full Analysis
  • [INDIRECT_PROMPT_INJECTION]: The skill is designed to ingest and process untrusted data (AI-generated text and traces) to perform automated evaluations, creating a surface for indirect prompt injection.
  • Ingestion points: Untrusted data enters the evaluation context through $ai_generation events and execution traces, which are then processed by Hog scripts or LLM judges.
  • Boundary markers: The instructions do not mandate the use of delimiters or 'ignore' instructions when the agent constructs prompts for the llm_judge that include untrusted generation content.
  • Capability inventory: The skill utilizes posthog:llma-evaluation-create to deploy logic that triggers LLM calls or executes domain-specific scripts on the ingested data.
  • Sanitization: No explicit sanitization or escaping of the external AI-generated content is required before it is interpolated into evaluation prompts or scripts.
  • [DYNAMIC_EXECUTION]: The skill involves generating and deploying Hog source code, a domain-specific language, to be executed on the PostHog server at runtime.
  • Evidence: Phase 2.4 and the HogEvalConfig section describe the creation of custom Hog scripts (via evaluation_config.source) that are executed to score events. While this is a controlled DSL environment, it constitutes dynamic logic generation based on agent-authored templates.
Audit Metadata
Risk Level
SAFE
Analyzed
Nov 1, 2026, 01:15 AM