langsmith-dataset

Pass

Audited by Gen Agent Trust Hub on Oct 28, 2026

Risk Level: SAFEEXTERNAL_DOWNLOADSREMOTE_CODE_EXECUTIONINDIRECT_PROMPT_INJECTION
Full Analysis
  • Remote Script Installation: The skill includes a command to install the LangSmith CLI by downloading and executing a shell script from the vendor's official GitHub repository (https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh). Although this is a common installation method for this utility, executing remote code from a script is a point for security consideration.
  • Indirect Prompt Injection Surface: The skill processes external data from trace logs and local files to build evaluation datasets, which introduces a potential surface for indirect instructions to enter the agent's context.
  • Ingestion points: The skill reads JSONL files from the ./traces directory and local files like /tmp/dataset.json (as seen in SKILL.md) for processing and upload.
  • Boundary markers: No explicit delimiters or developer instructions to disregard embedded instructions within the processed data are specified in the processing logic.
  • Capability inventory: The skill uses the langsmith CLI for network operations and includes Python and TypeScript scripts for reading and writing data to the local file system.
  • Sanitization: There is no mention of explicit validation or sanitization steps for the content of the data being ingested into the datasets.
  • Credential Management: To function, the skill requires a LangSmith API key. It correctly suggests using environment variables or specific command flags for authentication rather than hardcoding secrets, which is a recommended security practice for token management.
Audit Metadata
Risk Level
SAFE
Analyzed
Oct 28, 2026, 05:18 AM