langsmith-observability

Pass

Audited by Gen Agent Trust Hub on Oct 29, 2026

Risk Level: SAFEINDIRECT_PROMPT_INJECTIONDATA_EXFILTRATIONREMOTE_CODE_EXECUTION
Full Analysis
  • [INDIRECT_PROMPT_INJECTION]: The LLM-as-Judge evaluator pattern in references/advanced-usage.md is susceptible to indirect prompt injection.\n
  • Ingestion points: The llm_judge_evaluator function in references/advanced-usage.md ingests untrusted data from run.outputs and example.outputs.\n
  • Boundary markers: The data is directly interpolated into a triple-quoted f-string without separation markers or explicit instructions to ignore embedded content.\n
  • Capability inventory: The skill uses client.chat.completions.create (OpenAI API) to process the constructed prompt.\n
  • Sanitization: No sanitization, escaping, or filtering is applied to the untrusted inputs before interpolation.\n- [DATA_EXFILTRATION]: The skill is designed to transmit execution traces, metadata, and inputs/outputs to the LangSmith cloud backend at smith.langchain.com.\n
  • While this is the intended functionality for observability, users must be aware that sensitive data like PII or credentials may be included in these traces.\n
  • The skill provides examples of using process_inputs and process_outputs to redact such information before transmission, which is a necessary security practice.\n- [REMOTE_CODE_EXECUTION]: The skill fetches prompt templates dynamically at runtime from the LangChain Hub.\n
  • Using client.pull_prompt as shown in SKILL.md allows the agent to retrieve instructions from a remote service at smith.langchain.com.\n
  • This mechanism introduces a remote dependency where the agent's behavior and instructions can be modified externally, representing a shift in the control plane for agent behavior.
Audit Metadata
Risk Level
SAFE
Analyzed
Oct 29, 2026, 03:23 PM