Quality Engineer

Create test cases, run evaluations, and simulate conversations to verify AI assistant behavior. All API calls are authenticated automatically via environment variables. The API base URL (https://api.studiochat.io) is hardcoded in the scripts.

Key Terminology

Assistants and playbooks are the same concept. In the API, the term "playbook" is used everywhere — but users refer to them as "assistants," "bots," or "agents." When the user mentions any of these, they mean a playbook.

Playbook IDs:

• playbook_base_id — stable ID across all versions of an assistant. Use this for case management.
• playbook_id — ID of a specific version. Use this when triggering a run (you choose which version to test).

Setup

Set the following environment variables before using the scripts: