PerfGuard: Performance-Aware Agent for Visual Content Generation

This skill enables Claude to design and implement multi-tool orchestration systems for AI-generated visual content (AIGC) that select tools based on measured performance boundaries rather than static descriptions. Derived from the PerfGuard framework (ICLR 2026), the approach replaces the naive assumption that "all tools work equally well" with a structured scoring, ranking, and adaptive update system across three mechanisms: Performance-Aware Selection Modeling (PASM), Adaptive Preference Update (APU), and Capability-Aligned Planning Optimization (CAPO).

When to Use

• When the user is building a pipeline that routes tasks to multiple image generation or editing models (e.g., FLUX, SDXL, DALL-E, Midjourney APIs)
• When the user wants to automatically pick the best tool for a specific visual subtask (e.g., "which model handles spatial relationships better?")
• When the user needs a scoring/evaluation system for comparing AI-generated images across quality dimensions
• When the user asks to build a self-improving agent that learns which tools work best over time
• When the user wants to decompose complex visual tasks into subtasks matched to tool strengths
• When the user is implementing fallback/retry logic for visual generation where output quality is uncertain

Key Technique

The core insight: Generic tool descriptions ("generates high-quality images") are useless for selection. PerfGuard replaces them with multi-dimensional performance scores measured on fine-grained capability axes. For text-to-image tools, these axes are: color accuracy, shape fidelity, texture realism, 2D spatial, 3D spatial, numeracy (correct object count), and non-spatial semantics. For image editing tools: addition, removal, replacement, attribute alteration, motion change, style transfer, and background change. Each tool gets a score (0.0-1.0) per dimension, producing a capability fingerprint.

Selection works by weighted dot product. Given a task, the system identifies which dimensions matter (e.g., a request involving "three red cups arranged left to right" weights numeracy, color, and 2D-spatial heavily). The preference score for each candidate tool is sum(tool.score[dim] * task.weight[dim]). Tools are ranked by this score; the top-N candidates execute. Results are evaluated via a multi-dimensional quality assessment (object count, position, attributes, style, background, semantic alignment) scored at three levels (L1=1.0, L2=0.66, L3=0.33). If the composite score falls below a threshold (default 0.8), the system re-plans and retries with alternative tools.