Quick Start

Available Commands

User Says	Action
"/test" or "/t"	Create and run tests for current changes
"/test-quick"	Quick test without PRD
"/test-with-prd"	Comprehensive tests using PRD
"run tests"	Execute existing tests
"validate dsl"	Check test syntax
"list projects"	Show all test projects
"list templates"	Show all DSL templates
"use template"	Create test from a template
"save as template"	Save test case as template

MCP Tools

Tool	Description	Parameters
`list_projects`	List all test projects	-
`get_project`	Get project details with test cases	`project_id`
`create_project`	Create a new project	`name`, `target_type`
`delete_project`	Delete a project permanently	`project_id`
`validate_dsl`	MUST USE Validate DSL YAML syntax and schema	`dsl_yaml`
`save_case`	Save a test case	`project_id`, `name`, `dsl_yaml`, `case_id` (optional)
`delete_case`	Delete a test case	`project_id`, `case_id`
`run_case`	Execute a single test case	`project_id`, `case_id`
`run_project`	Execute all cases in a project	`project_id`, `case_ids` (optional)
`stop_execution`	Stop a running execution	`execution_id`
`get_execution_status`	Get execution status	`execution_id`
`get_execution_log`	Get detailed execution log	`execution_id`

Template Tools

Tool	Description	Parameters
`list_templates`	List all available templates	`category`, `target_type`, `tag` (optional)
`get_template`	Get template details	`template_id`
`create_template`	Create a new reusable template	`name`, `dsl_template`, `description`, `target_type`, `category`, `tags`
`update_template`	Update an existing template	`template_id`, `name`, `dsl_template`, etc.
`delete_template`	Delete a template	`template_id`
`instantiate_template`	Create test case from template	`template_id`, `project_id`, `case_name`, `variables`
`create_template_from_case`	Convert existing case to template	`project_id`, `case_id`, `template_name`, `variable_mappings`

DSL Validation Workflow (CRITICAL)

ALWAYS validate DSL before saving!

1. Generate DSL YAML
2. Call validate_dsl(dsl_yaml) to check syntax
3. IF validation fails:
   - Read the error message
   - Fix the DSL according to error
   - Call validate_dsl again
   - REPEAT until validation passes
4. IF validation passes:
   - Call save_case to store the test
5. Call run_case to execute

Validation Error Handling

Common errors and fixes:

Error	Cause	Fix
`Unknown assertion type`	Using `type` directly in assert step	Use `assertion: { type: ..., expected: ... }`
`field required`	Missing required field	Add the missing field
`YAML error`	Indentation/syntax error	Fix YAML formatting

DSL Format (Correct)

Basic Structure

version: "1.0"
meta:
  name: "Test Name"
  description: "What this test verifies"
  author: "Author"           # Optional
  tags: ["tag1", "tag2"]     # Optional

target:
  type: cli                  # cli, api, or tui

timeout: 30                  # Optional: Default timeout (seconds)

vars:                        # Optional: Variables
  base_url: "http://localhost:3000"

steps:
  - name: "Step description"
    action: exec|assert|wait
    # Action-specific fields below

Action: exec (Execute Command)

CLI Target:

- name: "Execute shell command"
  action: exec
  command: "echo Hello World"     # Required: Command to execute
  working_dir: "/path/to/dir"     # Optional: Working directory
  env:                            # Optional: Environment variables
    KEY: "value"
  timeout: 10                     # Optional: Timeout in seconds

API Target:

- name: "Send HTTP request"
  action: exec
  method: GET                     # HTTP method: GET, POST, PUT, DELETE
  path: "/api/endpoint"           # API path (appended to base_url)
  body:                           # Optional: Request body
    key: "value"
  headers:                        # Optional: Additional headers
    X-Custom: "value"

TUI Target:

- name: "Send input to TUI"
  action: exec
  input: "text to type"           # Text input
  # OR
  key: "enter"                    # Special key: enter, tab, escape, etc.

Action: assert (Verify Conditions)

⚠️ CRITICAL: assert MUST use assertion field, NOT type directly!

CLI Assertions:

- name: "Verify exit code"
  action: assert
  assertion:
    type: exit_code               # Assertion type
    expected: 0                   # Expected value (use 'expected', not 'value'!)

- name: "Verify stdout contains text"
  action: assert
  assertion:
    type: stdout_contains
    expected: "expected text"

- name: "Verify stderr contains error"
  action: assert
  assertion:
    type: stderr_contains
    expected: "error message"

- name: "Verify stdout matches pattern"
  action: assert
  assertion:
    type: stdout_matches
    expected: "^pattern$"

- name: "Verify stderr is empty"
  action: assert
  assertion:
    type: stderr_empty
    expected: true

API Assertions:

- name: "Verify HTTP status"
  action: assert
  assertion:
    type: status_code
    expected: 200

- name: "Verify JSON response"
  action: assert
  assertion:
    type: json_path
    path: "$.data.name"           # JSON path
    expected: "expected value"

TUI Assertions:

- name: "Verify screen content"
  action: assert
  assertion:
    type: screen_contains
    expected: "visible text"

- name: "Verify cursor position"
  action: assert
  assertion:
    type: cursor_at
    expected: "prompt"

Action: wait (Pause Execution)

- name: "Wait for process"
  action: wait
  duration: 5                     # Seconds to wait

Error Handling

- name: "Optional step that may fail"
  action: exec
  command: "optional-command"
  on_error: continue              # Options: fail (default), continue, stop

Target Types

CLI Target

target:
  type: cli
  working_dir: "/path/to/dir"     # Optional
  env:                            # Optional
    KEY: "value"

API Target

target:
  type: api
  base_url: "http://api.example.com"  # Required
  headers:                            # Optional default headers
    Authorization: "Bearer token"

TUI Target

target:
  type: tui
  start_command: "my-tui-app"     # Required: Command to start TUI
  terminal:                       # Optional
    rows: 24
    cols: 80

Variables

vars:
  base_url: "http://localhost:3000"
  timeout: 30

steps:
  - name: "Use variable"
    action: exec
    command: "curl ${vars.base_url}/api"
    timeout: "${vars.timeout}"

Complete Example (CLI)

version: "1.0"
meta:
  name: "Echo Command Test"
  description: "Test basic echo functionality"

target:
  type: cli

timeout: 10

steps:
  - name: "Execute echo command"
    action: exec
    command: echo "Hello World"

  - name: "Verify exit code is 0"
    action: assert
    assertion:
      type: exit_code
      expected: 0

  - name: "Verify output contains expected text"
    action: assert
    assertion:
      type: stdout_contains
      expected: "Hello World"

Complete Example (API)

version: "1.0"
meta:
  name: "API Health Check"
  description: "Test API health endpoint"

target:
  type: api
  base_url: "http://localhost:8000"

timeout: 10

steps:
  - name: "Call health endpoint"
    action: exec
    method: GET
    path: "/health"

  - name: "Verify status code"
    action: assert
    assertion:
      type: status_code
      expected: 200

  - name: "Verify response body"
    action: assert
    assertion:
      type: json_path
      path: "$.status"
      expected: "ok"

Workflows

Create and Run Tests Workflow

When user asks to create tests:

Discover: Call list_projects to find existing projects
Context: Read relevant source files and understand changes
Generate: Create DSL YAML following the CORRECT format above
Validate: Call validate_dsl(dsl_yaml)
Check Result:
- If ❌ validation fails:
  - Read error message carefully
  - Fix the DSL (usually: change type/value to assertion: { type, expected })
  - Call validate_dsl again
  - Repeat until ✅ validation passes
Save: Call save_case with validated DSL
Execute: Call run_case to run tests
Report: Show execution results

Delete Test Case Workflow

When user asks to delete a test case:

Find: Call get_project(project_id) to confirm project and case exist
Confirm: Show user the case details and ask for confirmation
Delete: Call delete_case(project_id, case_id)
Verify: Call get_project(project_id) to confirm deletion
Report: Show deletion result

User: "Delete test case 'login-test' from project 'auth-module'"

Agent:
1. Get project details to find the case
2. Confirm: "Delete test case 'login-test' (ID: xxx)?"
3. delete_case(project_id="auth-module", case_id="xxx")
4. Report: "✅ Test case 'login-test' deleted successfully"

Delete Project Workflow

When user asks to delete a project:

Warn: Show warning that this will delete ALL test cases in the project
List: Call get_project(project_id) to show what will be deleted
Confirm: Ask user to confirm the deletion
Delete: Call delete_project(project_id)
Report: Show deletion result

User: "Delete project 'legacy-tests'"

Agent:
1. Get project details
2. Warn: "⚠️ This will permanently delete project 'legacy-tests' with 12 test cases"
3. Confirm: "Are you sure? (yes/no)"
4. delete_project(project_id="legacy-tests")
5. Report: "✅ Project 'legacy-tests' and all its test cases deleted"

Cleanup Old Executions Workflow

When user wants to clean up old test data:

List: Call list_projects to show all projects
Identify: Ask user which projects/test cases to delete
Delete: Follow delete workflow above
Report: Summary of cleanup actions

Template Workflow - List and Use Templates

When user wants to use a template:

List: Call list_templates to show available templates
Filter (optional): Use category, target_type, or tag parameters
Get Details: Call get_template(template_id) to see variable requirements
Collect Variables: Ask user for variable values
Instantiate: Call instantiate_template with variables
Report: Show created test case

User: "Use the API health check template"

Agent:
1. list_templates(category="api")
2. get_template(template_id="api-health-check")
3. "Template requires: base_url (default: http://localhost:8000)"
4. instantiate_template(
     template_id="api-health-check",
     project_id="my-project",
     case_name="My API Health Test",
     variables={"base_url": "http://localhost:3000"}
   )
5. Report: "✅ Test case created from template"

Template Workflow - Create Template from Case

When user wants to save a test case as a template:

Get Project: Call get_project(project_id) to find the case
Identify Variables: Ask user which values should become variables
Create: Call create_template_from_case with variable mappings
Report: Show created template

User: "Save 'login-test' as a template"

Agent:
1. get_project(project_id="auth-module")
2. "Which values should become variables?"
   - User: "http://localhost:3000 -> base_url"
   - User: "test@example.com -> email"
3. create_template_from_case(
     project_id="auth-module",
     case_id="login-test",
     template_name="User Login Template",
     variable_mappings={
       "http://localhost:3000": "base_url",
       "test@example.com": "email"
     }
   )
4. Report: "✅ Template created with 2 variables"

Template Workflow - Create New Template

When user wants to create a template from scratch:

Define: Ask for template name, description, category
Create DSL: Create DSL with ${variable} placeholders
Create: Call create_template with the DSL
Report: Show created template

User: "Create a template for database backup tests"

Agent:
1. "Template name? Category?"
2. Create DSL with variables like ${db_name}, ${backup_path}
3. create_template(
     name="Database Backup Test",
     dsl_template="version:... steps:...",
     category="database"
   )
4. Report: "✅ Template created with 2 variables"

Template DSL Format

Templates use ${variable_name} placeholders that get replaced when instantiated.

version: "1.0"
meta:
  name: "${test_name}"
  description: "API test for ${endpoint}"

target:
  type: api
  base_url: "${base_url}"

steps:
  - name: "Call ${endpoint}"
    action: exec
    method: GET
    path: "${endpoint}"

  - name: "Verify status"
    action: assert
    assertion:
      type: status_code
      expected: ${expected_status}

When instantiating, provide variable values:

instantiate_template(
    template_id="...",
    project_id="...",
    case_name="My Test",
    variables={
        "test_name": "User API Test",
        "endpoint": "/api/users",
        "base_url": "http://localhost:8000",
        "expected_status": 200
    }
)

Common Mistakes to Avoid

❌ WRONG - assert with direct type/value:

- action: assert
  type: stdout_contains      # ❌ Wrong!
  value: "text"              # ❌ Wrong!

✅ CORRECT - assert with assertion object:

- action: assert
  assertion:                 # ✅ Correct!
    type: stdout_contains
    expected: "text"         # ✅ Use 'expected', not 'value'!

❌ WRONG - Missing name field:

- action: exec
  command: "echo test"

✅ CORRECT - With name:

- name: "Execute test command"  # ✅ Always add name
  action: exec
  command: "echo test"

opentester

OpenTester Testing Skill

Prerequisites