eval-harness
Eval Harness
Overview
A systematic framework for evaluating agent performance. Measures accuracy, efficiency, and reliability across defined test scenarios. Enables data-driven decisions about agent quality and improvement.
When to Use
- Before deploying agent changes to production
- Comparing different agent configurations
- Identifying weaknesses in agent behavior
- Tracking agent quality over time
- Validating prompt improvements
Evaluation Dimensions
1. Accuracy
Does the agent produce correct outputs?
More from v1truv1us/ai-eng-system
coolify-deploy
Deploy applications to Coolify self-hosting platform. Use when deploying to Coolify, configuring build settings, setting environment variables, managing health checks, or performing rollbacks.
106prompt-refinement
Transform prompts into structured TCRO format with phase-specific clarification. Automatically invoked by /ai-eng/research, /ai-eng/plan, /ai-eng/work, and /ai-eng/specify commands. Use when refining vague prompts, structuring requirements, or enhancing user input quality before execution.
18text-cleanup
Comprehensive patterns and techniques for removing AI-generated verbosity and slop
15plugin-dev
This skill should be used when creating extensions for Claude Code or OpenCode, including plugins, commands, agents, skills, and custom tools. Covers both platforms with format specifications, best practices, and the ai-eng-system build system.
14incentive-prompting
Research-backed prompting techniques for improved AI response quality (+45-115% improvement). Use when optimizing prompts, enhancing agent instructions, or when maximum response quality is critical. Invoked by /ai-eng/optimize command. Includes expert persona, stakes language, step-by-step reasoning, challenge framing, and self-evaluation techniques.
10git-worktree
Manage Git worktrees for parallel development. Use when creating isolated workspaces for parallel feature work, running multiple Claude sessions simultaneously, or managing concurrent development tasks.
9