langsmith
Fail
Audited by Gen Agent Trust Hub on Jun 17, 2026
Risk Level: HIGHCOMMAND_EXECUTIONEXTERNAL_DOWNLOADSREMOTE_CODE_EXECUTIONPROMPT_INJECTION
Full Analysis
- [COMMAND_EXECUTION]: The script
scripts/setup.shmodifies system shell profile files (~/.bashrcand~/.zshrc) to persist environment variables such asLANGSMITH_API_KEY. This represents a persistence mechanism that modifies shell configuration. - [COMMAND_EXECUTION]: The
scripts/setup.shscript is vulnerable to command injection. It accepts user input for an API key and appends it directly to shell profile files without sanitization, which could allow a malicious actor to inject arbitrary commands that execute when a new shell is opened. - [REMOTE_CODE_EXECUTION]: The skill downloads and executes an installation script for the LangSmith CLI from the official repository of a well-known service (
langchain-ai) using a shell pipe pattern. - [EXTERNAL_DOWNLOADS]: The skill automates the installation of several software packages (
langsmith,openai,openevals) from public registries usingpip,npm, andyarnwithin thescripts/setup.shfile. - [PROMPT_INJECTION]: The skill exhibits an indirect prompt injection surface as it ingests untrusted user requests and interpolates them into YAML structures for routing decisions.
- Ingestion points: User prompts analyzed in
SKILL.md. - Boundary markers: None present.
- Capability inventory: The skill uses
BashandWebFetchtools and includes scripts (setup.sh,quickstart.py) capable of network operations and file system modification. - Sanitization: No input validation or sanitization is implemented for the processed data.
Recommendations
- INFO: Downloads code from trusted source(s): https://raw.githubusercontent.com/langchain-ai/langsmith-cli/main/scripts/install.sh
- AI detected serious security threats
Audit Metadata