Load Testing Your Databricks App

Goal: Find the maximum QPS (queries per second) your Databricks App can support.

Assumes MLflow AgentServer. These templates serve the agent with the MLflow AgentServer — it exposes the POST /invocations endpoint (streaming @invoke/@stream, SSE terminated by [DONE]) the scripts below hit, and emits the per-request MLflow trace used in Step 6. Any app works if it exposes the same /invocations streaming contract.

Before You Start — Gather Parameters

Before beginning, use the AskUserQuestion tool to collect the following from the user:

1. Do they already have deployed apps to test, or do they need to set up new apps?
2. Do they want to mock LLM calls? Mocking isolates infrastructure throughput from LLM latency — useful for capacity planning. Testing without mocks measures end-to-end performance.
3. What compute sizes do they want to test? (Medium, Large, or both)
4. How many worker configurations do they want to test? (e.g., 2, 4, 6, 8 workers)
5. Do they have M2M OAuth credentials (service principal client_id/client_secret)? — Recommended for tests longer than ~30 minutes. If not, guide them to create one.
6. What is their DATABRICKS_HOST? (workspace URL)