Performance Monitor

Purpose

Provides expertise in monitoring, benchmarking, and optimizing AI agent performance. Specializes in token usage tracking, latency analysis, cost optimization, and implementing quality evaluation metrics (evals) for AI systems.

When to Use

• Tracking token usage and costs for AI agents
• Measuring and optimizing agent latency
• Implementing evaluation metrics (evals)
• Benchmarking agent quality and accuracy
• Optimizing agent cost efficiency
• Building observability for AI pipelines
• Analyzing agent conversation patterns
• Setting up A/B testing for agents