creating-online-evaluations
Pass
Audited by Gen Agent Trust Hub on Nov 3, 2026
Risk Level: SAFEINDIRECT_PROMPT_INJECTIONCOMMAND_EXECUTIONDYNAMIC_EXECUTION
Full Analysis
- [INDIRECT_PROMPT_INJECTION]: The skill configures evaluators to process untrusted data from production AI traffic, such as
$ai_generationevents, traces, and sessions. - Ingestion points: Untrusted content enters the evaluation context through the
evaluation_eventsglobal in Hog scripts or as input to LLM judges. - Capability inventory: Evaluators can execute logic and produce events; the skill uses
posthog:execute-sqlandposthog:llma-evaluation-createfor configuration and verification. - Boundary markers: There are no explicit instructions for using delimiters to isolate untrusted data from evaluation instructions.
- Sanitization: No specific sanitization or validation of the ingested generation data is mentioned.
- [COMMAND_EXECUTION]: Uses a platform-specific SQL tool to query event counts for verification.
- Evidence: The skill uses
posthog:execute-sqlto performSELECT count()queries to validate the volume of events matching evaluation conditions. - [DYNAMIC_EXECUTION]: Involves the creation and testing of user-defined Hog source code.
- Evidence: Uses
posthog:llma-evaluation-test-hogandposthog:llma-evaluation-createto execute and persist evaluation logic provided as strings.
Audit Metadata