langsmith — LLM Observability, Evaluation & Prompt Management

Keyword: langsmith · llm tracing · llm evaluation · @traceable · langsmith evaluate

LangSmith is a framework-agnostic platform for developing, debugging, and deploying LLM applications. It provides end-to-end tracing, quality evaluation, prompt versioning, and production monitoring.

When to use this skill

• Add tracing to any LLM pipeline (OpenAI, Anthropic, LangChain, custom models)
• Run offline evaluations with evaluate() against a curated dataset
• Set up production monitoring and online evaluation
• Manage and version prompts in the Prompt Hub
• Create datasets for regression testing and benchmarking
• Attach human or automated feedback to traces
• Use LLM-as-judge scoring with openevals
• Debug agent failures with end-to-end trace inspection

Instructions