Write Eval Specs

Spec File Anatomy

Eval specs use the evaluate Playwright fixture (not test). A spec file follows this structure:

import { evaluate, tags, selectEvaluators, type Example, type TaskOutput } from '@kbn/evals';

evaluate.describe('Suite name', { tag: tags.serverless.observability.complete }, () => {
  evaluate.beforeAll(async ({ fetch, log }) => {
    // one-time setup: install docs, create agents, load archives
  });

  evaluate.afterAll(async ({ fetch, log }) => {
    // teardown: uninstall docs, delete agents, unload archives
  });