EvalScope Documentation

EvalScope is ModelScope's LLM/VLM evaluation framework providing benchmarking, performance testing, arena mode, and visualization.

• GitHub | Docs

CRITICAL: grep references/ for detailed docs. references/benchmarks/ has 153 individual benchmark files with full details, metrics, and usage examples.

Quick Start

pip install evalscope

# Evaluate a model on datasets (CLI)
evalscope eval --model Qwen/Qwen2.5-0.5B-Instruct --datasets gsm8k arc --limit 5