agent-observability-eval-pipeline
Pass
Audited by Gen Agent Trust Hub on Aug 24, 2026
Risk Level: SAFEPROMPT_INJECTIONCOMMAND_EXECUTIONCREDENTIALS_UNSAFE
Full Analysis
- [PROMPT_INJECTION]: The skill processes untrusted trace data from production environments via the search_llmobs_spans tool. This creates a surface for indirect prompt injection where malicious instructions embedded in production data could influence the agent's behavior during classification, root-cause analysis, or evaluator bootstrapping. 1. Ingestion points: Production traces are ingested via search_llmobs_spans in SKILL.md (Precheck) and passed to sub-skills. 2. Boundary markers: Explicit delimiters or warnings to ignore embedded instructions in the trace content are absent. 3. Capability inventory: The skill can execute shell commands via Bash, write state files to the local filesystem, and perform network operations to Datadog's API. 4. Sanitization: The skill performs PII scrubbing and tag normalization for published data, but not for the data interpreted by the LLM.
- [COMMAND_EXECUTION]: The skill generates a Python experiment script and subsequently executes it using Bash. Evidence exists in SKILL.md where it instructs running 'python <generated_path>'. This risk is mitigated by a mandatory user review checkpoint before execution.
- [CREDENTIALS_UNSAFE]: The skill is configured to discover and load credentials from standard local files, including .env files and ~/.datadog/credentials. While this is the intended mechanism for Datadog authentication, it involves searching and reading sensitive local paths.
Audit Metadata