The Agent Skills Directory

Local Execution of Evaluation Logic: The skill supports the creation and local testing of Python-based evaluators. These functions are executed by the agent against sample traces to verify logic and field access before deployment to the LangSmith platform.
Evidence: SKILL.md (Section 4) describes executing the perform_eval function directly against fetched root-level traces to catch runtime errors.
Interaction with Official API Services: The skill communicates with the LangSmith API to create, list, and manage evaluators and run rules. These network operations are directed exclusively to the platform's official infrastructure.
Evidence: references/langsmith-api.md demonstrates the use of the langsmith SDK and httpx to interact with the api.smith.langchain.com endpoint.
Processing of Trace Context: The skill ingests input and output data from existing LangSmith traces. This data is summarized for the user and used to inform the design of LLM prompts and code evaluators.
Evidence: references/trace-inspection.md outlines procedures for fetching runs and extracting field names and sample content for analysis.

langsmith-online-eval-engineering