Load Testing

Measure how a service behaves as offered load rises, in one named environment: where latency or errors bend upward, the highest load that still meets the latency budget, and what the numbers do not cover. The work ends with a short result record that another person can rerun and compare, or with a test plan when only a design was requested. A load test answers a capacity question; beyond checking that responses are valid, it says nothing about output quality.

Evaluation numbers are not performance evidence

Throughput and latency logged by an evaluation run are not capacity evidence. An evaluation harness sends requests at a pace of its own choosing, usually one at a time or at a small fixed concurrency, with inputs chosen to cover behaviors rather than to match traffic, and often with settings that differ from serving. It is unloaded by design and measures model quality. When a document, plan, or decision needs capacity or latency under load, cite a load test or say that none exists. The reverse also holds: a load test establishes capacity, not quality, so quality claims stay with the evaluation.

Route other questions elsewhere

• Why the service got slower, why latency jumps at some load, or what causes a bottleneck: bug-diagnosis. This skill locates where the limit appears; finding its cause is a diagnosis.
• Which model, quantization, mode, or serving configuration gives better outputs, or pinning a serving setup at fixed revisions before a campaign, with no load involved: ml-experiment.
• What an evaluation measures, or running an evaluation campaign: evaluation-design or evaluation-operations.

Comparing serving configurations for latency and throughput under load stays here: run the same workload and step plan against each configuration and change one setting at a time.

Authority and early questions

Send load only to systems the user owns or operates. Shared environments and production need explicit authorization from whoever runs them, with agreed limits: maximum rate or concurrency, duration, time window, abort conditions such as an error-rate or latency ceiling, and who to notify. A third-party API the user calls but does not operate is not a load-test target; loading it also spends money and runs into its terms and rate limits.