improve-setup
Improve Your LangWatch Setup
This recipe acts as your expert AI engineering consultant. It audits everything, delivers quick fixes, then guides you deeper.
Phase 1: Full Audit
Before suggesting anything, read EVERYTHING:
Code Audit
- Read the full codebase — every file, every function, every system prompt
- Study
git log --oneline -50— read commit messages for WHY things changed. Bug fixes reveal edge cases. Refactors reveal design decisions. These are goldmines for what to test and evaluate. - Read README, docs, comments for domain context
LangWatch Audit (via MCP)
- Call
search_traces— check trace quality (inputs/outputs populated? spans connected? labels present?) - Call
platform_list_scenarios— what scenarios exist? Are they comprehensive or shallow? - Call
platform_list_evaluators— what evaluators are configured? - Call
platform_list_prompts— are prompts versioned or hardcoded? - Call
get_analytics— what's the cost, latency, error rate?
More from langwatch/skills
evaluations
Set up comprehensive evaluations for your AI agent with LangWatch — experiments (batch testing), evaluators (scoring functions), datasets, online evaluation (production monitoring), and guardrails (real-time blocking). Supports both code (SDK) and platform (CLI) approaches. Use when the user wants to evaluate, test, benchmark, monitor, or safeguard their agent.
51scenarios
Test your AI agent with simulation-based scenarios. Covers writing scenario test code (Scenario SDK), creating platform scenarios via the `langwatch` CLI, and red teaming for security vulnerabilities. Auto-detects whether to use code or platform approach based on context.
50tracing
Add LangWatch tracing and observability to your code. Use for both onboarding (instrument an entire codebase) and targeted operations (add tracing to a specific function or module). Supports Python and TypeScript with all major frameworks.
46level-up
Take your AI agent to the next level with full LangWatch integration. Adds tracing, prompt versioning, evaluation experiments, and simulation tests in one go. Use when the user wants comprehensive observability, testing, and prompt management for their agent.
38prompts
Version and manage your agent's prompts with LangWatch Prompts CLI. Use for both onboarding (set up prompt versioning for an entire codebase) and targeted operations (version a specific prompt, create a new prompt version). Supports Python and TypeScript.
37analytics
Analyze your AI agent's performance using LangWatch analytics. Use when the user wants to understand costs, latency, error rates, usage trends, or debug specific traces. Works with any LangWatch-instrumented agent.
32