update-golden-values
Installation
SKILL.md
Update golden values + relative-diff summary
End-to-end workflow for refreshing golden values from a GitHub Actions workflow run, scoring the update with a per-metric average normalized relative difference, and writing a PR-ready summary.
The skill orchestrates two scripts that already live in the repo:
tests/test_utils/python_scripts/download_golden_values.py— pulls artifacts from a workflow run and overwritestests/functional_tests/test_cases/**/golden_values_*.json.tests/test_utils/python_scripts/compare_golden_values_kl.py— diffs the working-tree goldens againstgit HEADand reports per-metricavg_rel_diff = mean((old − new) / old). (Filename keeps the legacy_klsuffix; the script no longer computes KL divergence.)
Inputs to gather from the user
-
GitHub Actions workflow run ID (e.g.
25341543542). It's the numeric ID in the run URL. -
Source: should be
githubfor this workflow. (gitlabis supported by the download script but uses a different env path.) -
Scope — accept one of:
only-failing→ run with--only-failing(download from failing/cancelled jobs only). Use this for "fix the broken tests" workflows.all→ run without--only-failing(download from every job that produced golden values). Use this when the user wants a full refresh.
If the user doesn't specify, ask. Don't silently default.