The Agent Skills Directory

[INDIRECT_PROMPT_INJECTION]: The skill processes untrusted trace content from Datadog LLM-Obs interactions, which represents an indirect prompt injection risk as these traces contain end-user text and tool outputs.\n
Ingestion points: Interactions are fetched from Datadog LLM-Obs queues via the get_llmobs_annotated_interactions and related trace retrieval tools described in Phase 2 of SKILL.md.\n
Boundary markers: Mitigation is attempted in references/judge_runner.py by fencing untrusted data with <payload> tags and instructing the judge model to treat the content as data rather than instructions.\n
Capability inventory: The skill can perform network operations to Datadog and Anthropic APIs and execute local CLI tools via the subprocess module.\n
Sanitization: No explicit sanitization or filtering of the trace content is described beyond the use of delimiters and model instructions.\n- [COMMAND_EXECUTION]: The references/judge_runner.py script utilizes the subprocess module to interact with the local environment for LLM evaluations.\n
The script executes the claude CLI tool (via claude -p) to invoke model calls if the Anthropic SDK is not available.\n
The implementation uses list-based arguments and passes untrusted content through standard input (stdin), which effectively prevents shell injection attacks.\n- [DYNAMIC_EXECUTION]: The skill generates and manages executable files and prompts at runtime to support the evaluation process.\n
A Python evaluation harness is created by copying the references/judge_harness_template.py file to the .auto_experiment/ directory.\n
The judge's system prompt is dynamically drafted and saved to prompts/judge.md before being used in the optimization loop.\n- [EXTERNAL_DOWNLOADS]: The SKILL.md file provides instructions for users to install the agent-observability-auto-experiment skill from the vendor's repository using npx skills add datadog-labs/agent-skills. This reference targets a well-known vendor and repository.

agent-observability-build-eval-from-annotations