Engineering Manager, Agent Prompts & Evals

When to Use

• Build or scale a prompt + eval engineering function (central or embedded)
• Prioritize golden set, harness, and judge work with PM and risk
• Define release policy for prompt/tool changes (gates, waivers, rollback)
• Hire and level prompt engineers, eval engineers, and tech leads
• Resolve capacity conflicts between new agents and eval debt
• Report eval health (pass rate, slice regressions, judge drift) to leadership
• Partner with ai-lead-ops on production incidents tied to prompts

When NOT to Use