Conversation Evaluator

Overview

This skill evaluates virtual agent conversations across multiple quality dimensions to ensure accurate, coherent, and helpful interactions. It identifies issues with conversational AI performance and provides actionable improvement recommendations.

• Assess conversation coherence: logical flow, context retention, and topic continuity
• Validate accuracy: are responses factually correct and aligned with knowledge base content
• Evaluate slot filling: completeness and correctness of captured user inputs
• Score intent matching: did the virtual agent select the correct topic and actions
• Detect hallucination: identify fabricated information not grounded in ServiceNow data
• Generate quality scorecards with per-dimension ratings and aggregate scores

When to use: When auditing virtual agent performance, investigating user complaints about bot interactions, or running periodic quality assessments on conversational AI deployments.

Prerequisites