Trace Annotation Tool Generator

Generate a custom local web application for open coding of LLM traces — the first qualitative pass of error analysis in the Analyze phase of the evaluation lifecycle.

Core Workflow

Step 1: Understand the User's Trace Data

1. Ask the user to point to their trace data file (CSV, JSONL, JSON, or any structured format).
2. Read a sample of the data to understand its structure: field names, nesting depth, which fields represent the user query, intermediate steps, tool calls, and final output.
3. Identify a unique trace identifier field (or generate sequential IDs if none exists).
4. Confirm the structure with the user: "I see fields X, Y, Z — which represent the trace steps, and which is the user query?"

Step 2: Ask About Additional Features

The tool includes these features by default: