The Agent Skills Directory

[INDIRECT_PROMPT_INJECTION]: The skill establishes a process for creating online evaluations that ingest untrusted data from AI generations and traces. This creates a potential surface for indirect prompt injection if the processed content contains instructions that could influence the evaluator's scoring or behavior.
Ingestion points: Content is ingested through the evaluation_events global and item.input/item.output projections in Hog scripts, as well as via transcripts provided to LLM judges (referenced in SKILL.md and references/evaluation-payload.md).
Boundary markers: The instructions do not mandate the use of specific delimiters or markers to isolate untrusted data within evaluation prompts.
Capability inventory: The skill uses tools like posthog:llma-evaluation-create and posthog:llma-evaluation-update to deploy logic that scores generations and emits $ai_evaluation events.
Sanitization: No specific sanitization or filtering logic is prescribed for the input generation data before it is processed by the evaluator.
[DYNAMIC_EXECUTION]: The skill involves the runtime generation of Hog source code (a domain-specific language) and LLM judge criteria. This logic is assembled by the agent to define how evaluations score incoming data and is then deployed via posthog:llma-evaluation-create. This dynamic code generation is a functional requirement for the skill's purpose of authoring monitoring tools.
[COMMAND_EXECUTION]: The skill utilizes the posthog:execute-sql tool to perform analysis on the events table. This is used to verify the expected volume and scope of evaluation triggers during the configuration phase.
[CREDENTIALS_UNSAFE]: The skill manages AI provider API keys through vendor-specific tools such as posthog:llma-provider-key-list and posthog:llma-evaluation-config-get. These keys are used to configure and pin specific models for use in evaluation judges.

creating-online-evaluations