langsmith-evaluator
Pass
Audited by Gen Agent Trust Hub on Sep 19, 2026
Risk Level: SAFEEXTERNAL_DOWNLOADSREMOTE_CODE_EXECUTIONCOMMAND_EXECUTION
Full Analysis
- Remote CLI Installation: The skill includes a command to download and execute an installation script from
https://cli.langsmith.com/install.sh. This is a common pattern for CLI tools; because the domain is a known vendor-owned resource for LangSmith, it is considered a standard operational procedure for this skill. - Environment Variable Configuration: The setup instructions suggest setting several environment variables, including
LANGSMITH_API_KEYandOPENAI_API_KEY. The skill correctly advises against placing these keys directly in commands or logs, directing users toward using CLI profiles or environment variables instead. - Package Dependencies: The skill lists standard Python and Node.js dependencies (
langsmith,langchain-openai,openai) from official registries (PyPI and NPM), which are expected for its stated purpose of building evaluation pipelines. - Command Execution: The instructions use the
langsmithCLI tool for managing evaluators. These commands are localized to the user's authenticated session and perform intended CRUD operations on the LangSmith platform.
Audit Metadata