update-golden-values

Installation
SKILL.md

Update golden values + relative-diff summary

End-to-end workflow for refreshing golden values from a GitHub Actions workflow run, scoring the update with a per-metric average normalized relative difference, and writing a PR-ready summary.

The skill orchestrates two scripts that already live in the repo:

  • tests/test_utils/python_scripts/download_golden_values.py — pulls artifacts from a workflow run and overwrites tests/functional_tests/test_cases/**/golden_values_*.json.
  • tests/test_utils/python_scripts/compare_golden_values_kl.py — diffs the working-tree goldens against git HEAD and reports per-metric avg_rel_diff = mean((old − new) / old). (Filename keeps the legacy _kl suffix; the script no longer computes KL divergence.)

Inputs to gather from the user

  1. GitHub Actions workflow run ID (e.g. 25341543542). It's the numeric ID in the run URL.

  2. Source: should be github for this workflow. (gitlab is supported by the download script but uses a different env path.)

  3. Scope — accept one of:

    • only-failing → run with --only-failing (download from failing/cancelled jobs only). Use this for "fix the broken tests" workflows.
    • all → run without --only-failing (download from every job that produced golden values). Use this when the user wants a full refresh.

    If the user doesn't specify, ask. Don't silently default.

Installs
4
GitHub Stars
17.3K
First Seen
Jul 3, 2026
update-golden-values — nvidia/megatron-lm