langsmith-evaluator

Pass

Audited by Gen Agent Trust Hub on Sep 19, 2026

Risk Level: SAFEEXTERNAL_DOWNLOADSREMOTE_CODE_EXECUTIONCOMMAND_EXECUTION
Full Analysis
  • Remote CLI Installation: The skill includes a command to download and execute an installation script from https://cli.langsmith.com/install.sh. This is a common pattern for CLI tools; because the domain is a known vendor-owned resource for LangSmith, it is considered a standard operational procedure for this skill.
  • Environment Variable Configuration: The setup instructions suggest setting several environment variables, including LANGSMITH_API_KEY and OPENAI_API_KEY. The skill correctly advises against placing these keys directly in commands or logs, directing users toward using CLI profiles or environment variables instead.
  • Package Dependencies: The skill lists standard Python and Node.js dependencies (langsmith, langchain-openai, openai) from official registries (PyPI and NPM), which are expected for its stated purpose of building evaluation pipelines.
  • Command Execution: The instructions use the langsmith CLI tool for managing evaluators. These commands are localized to the user's authenticated session and perform intended CRUD operations on the LangSmith platform.
Audit Metadata
Risk Level
SAFE
Analyzed
Sep 19, 2026, 03:12 PM