Spot GPU Evaluation

Run LLM and agent evaluations on GPU capacity the provider can take back at any time: a model server plus many independent benchmark episodes on a spot or preemptible node. The work ends with one of two results. A placement decision states the workload's numbers and why spot, on-demand, or a non-preemptible box fits. An operated run has a measured restart path, results that reach object storage while the run is live, interruptions recorded apart from model failures, and a log of every start, stop, and preemption with its reason and price.

The method applies to any provider with interruptible capacity: Nebius preemptible VMs, AWS, GCP, and Azure spot instances, Runpod interruptible pods, and similar offers. Notice periods, restart semantics, and address behavior differ by provider and change over time; read provider notes and recheck the provider's current documentation before relying on one.

Scope and authority

• Evaluation workloads only: serving a model and running benchmark episodes against it. Training and fine-tuning, with their checkpoints, optimizer state, and data progress, belong to ml-experiment. Campaign bookkeeping across many runs belongs to evaluation-operations; this skill supplies the node-level operation those runs depend on.
• Creating, starting, and stopping instances, disks, buckets, and addresses spends money, and deleting a disk is irreversible. Act within the user's grant: budget, hourly price cap, regions, instance types, and stop rules. A status question does not authorize a start. When a price cap exists, read the current spot price before each start.
• Keep GPUs stopped whenever they are not serving or running evaluations, including while waiting for a decision, a code change, or a review. Place the holds described below before stopping.
• Record instance IDs, addresses, bucket names, account names, and prices in the project's private run record. Shared reports carry the method and the measurements without them.

Decide placement before building recovery

Recovery machinery keeps completed work. Whether a workload fits spot depends on three numbers: the episode length D, the mean lifetime τ of an allocation from start to preemption on the actual pool, and the restart cost S from preemption until the server serves again.