modelopt-evaluation

Installation
SKILL.md

NeMo Evaluator Launcher Assistant

Guide the user through creating NEL YAML configs, running evaluations, and monitoring progress. NEL manages serving for canaries and full runs; see Step 3's serving-handoff check for external endpoints.

Completed-run analysis

For completed runs, skip launch/config; use launching-evals for artifacts/analysis and apply run-validation.md before scores. Its policy overrides blanket uncapping and fixed token/context limits.

Workspace integration

If MODELOPT_WORKSPACE_ROOT is set, use the common skill's workspace-management.md and reuse existing workspaces (PTQ → checkpoint smoke test → NEL evaluation; carry compatibility requirements into NEL's deployment config).

Workflow

Installs
1
GitHub Stars
5.6K
First Seen
1 day ago