Auto Context Generation - Evaluation Workflow

This skill guides the process of rigorously evaluating an existing ContextSet against a specific golden truth dataset using Google's Evalbench architecture. It structures the evaluation experiments and executes the binaries.

Input

Before beginning the workflow, you explicitly require:

• A tools.yaml file securely located in the workspace root directory containing the target database connection details.

• A golden evaluation dataset (golden_dataset_path), formatted as an absolute system path. The file must be in the simplified user-facing format.

  Simplified User-Facing Dataset Format: A JSON list of objects, where each object must have the following keys:

  • id: Unique string identifier (e.g., eval_001).
  • database: Target database name.
  • nlq: Natural language question.
  • golden_sql: The correct reference SQL query.