The Agent Skills Directory

[Dynamic Code Deployment]: The skill describes the creation and deployment of Python code strings to be executed as evaluators within the LangSmith runtime environment. This is a core functionality of the platform designed for automated quality checks. The instructions specify that the runtime environment is restricted and lacks network access, which serves as a security control for these operations.
[Processing Untrusted Production Data]: The skill is designed to ingest and evaluate production traces, which may contain sensitive or attacker-controlled data. The instructions proactively address this by advising that trace content be treated as sensitive, suggesting the truncation of payloads, and explicitly warning against exposing secrets through prompts, code, or logs.
[Dependency Management]: The skill utilizes the LangSmith SDK and mentions support for standard data science libraries such as NumPy and pandas within the evaluation runtime. These are used for deterministic data processing and analysis within the platform's infrastructure.
[Indirect Prompt Injection Surface]: Because the skill processes external data (trace inputs and outputs) through LLM judges, there is a potential for indirect prompt injection. The documentation mitigates this by recommending structured rubrics, narrow output schemas, and specific variable mappings to limit the context provided to the LLM judge.

langsmith-online-eval-engineering