Creating Eval Scenarios

Generate evaluation scenarios that measure whether agents follow instructions from skills.

Prerequisites

Skills must be packaged in a Tessl tile (directory with tile.json + skill folders). If not, use the converting-skill-to-tessl-tile skill first. Ask the user where to put the tile if its not specified.

Its possible for a tile to contain multiple skills. In this case, split the tile into multiple tiles, one for each skill first.

Quick Start

Read references/scenario-generation.md before starting. It will guide you through the workflow of researching the tile and creating all the expected files in the correct formats.

Output Structure