update-golden-values

Installation
SKILL.md

Update golden values + signed per-model percentage summary

End-to-end workflow for refreshing golden values from a GitHub Actions workflow run, reporting signed percentage changes per test/model environment, and writing a PR-ready summary.

The skill orchestrates two scripts that already live in the repo:

  • tests/test_utils/python_scripts/download_golden_values.py — pulls artifacts from a workflow run and overwrites tests/functional_tests/test_cases/**/golden_values_*.json.
  • tests/test_utils/python_scripts/compare_golden_values_kl.py — diffs the working-tree goldens against git HEAD and reports per-metric avg_rel_diff = mean((old − new) / old). (Filename keeps the legacy _kl suffix; the script no longer computes KL divergence.)

Inputs to gather from the user

  1. GitHub Actions workflow run ID (e.g. 25341543542). It's the numeric ID in the run URL.

  2. Source: should be github for this workflow. (gitlab is supported by the download script but uses a different env path.)

  3. Scope — accept one of:

    • only-failing → run with --only-failing (download from failing/cancelled jobs only). Use this for "fix the broken tests" workflows.
    • all → run without --only-failing (download from every job that produced golden values). Use this when the user wants a full refresh.

    If the user doesn't specify, ask. Don't silently default.

Installs
4
GitHub Stars
18.1K
First Seen
Jul 3, 2026
update-golden-values — nvidia/megatron-lm