genie-benchmark-generator
Installation
SKILL.md
Genie Benchmark Generator
Creates, validates, and syncs benchmark question suites for Genie Space evaluation. Handles three intake paths, ground truth SQL validation, and MLflow Evaluation Dataset synchronization.
When to Use This Skill
- Creating benchmark questions for a new Genie Space optimization
- Refreshing benchmarks after arbiter corrections (ground truth was wrong)
- Generating synthetic benchmarks from Genie Space trusted assets
- Validating ground truth SQL against the live warehouse
Inputs (from Orchestrator)
| Input | Type | Description |
|---|---|---|
space_id |
str | Genie Space ID |
uc_schema |
str | Unity Catalog schema for MLflow datasets |
domain |
str | Domain key (e.g., "cost") |
user_questions |
list | Optional user-provided benchmark questions |