Overview

When to Use

Use this skill when you need run evaluations for Hugging Face Hub models using inspect-ai and lighteval on local hardware. Use for backend selection, local GPU evals, and choosing between vLLM / Transformers / accelerate. Not for HF Jobs orchestration, model-card PRs, .eval_results publication, or community-evals...

This skill is for running evaluations against models on the Hugging Face Hub on local hardware.

It covers:

• inspect-ai with local inference
• lighteval with local inference
• choosing between vllm, Hugging Face Transformers, and accelerate
• smoke tests, task selection, and backend fallback strategy