ml-hyperparameter-tuning
Installation
SKILL.md
ML Hyperparameter Tuning
Overview
Use this skill for designing and running hyperparameter searches. Tuning should improve a valid baseline under a fixed evaluation protocol. Do not tune before data validation, leakage-safe splits, metric selection, reproducible training, and a simple baseline are in place.
Search Strategy Selection
| Strategy | Use when | Notes |
|---|---|---|
| Manual informed search | Early debugging or very small budgets | Best when guided by learning curves and domain knowledge |
| Grid search | Few categorical/discrete parameters | Wasteful in high dimensions |
| Random search | Strong default for broad spaces | Often beats grid when only some parameters matter |
| Bayesian/TPE | Moderate budgets and expensive trials | Good for structured continuous/discrete spaces |
| Hyperband/ASHA | Many deep-learning trials with early signal | Requires comparable learning curves and sensible early-stopping metric |
| Population-based training | Schedules and nonstationary hyperparameters | More complex; useful for RL and large training budgets |
| AutoML | Need strong baseline or tabular productivity | Validate leakage, explainability, and deployment constraints |
Optuna is a flexible default for Python search. Ray Tune is strong for distributed sweeps, schedulers, and Ray Train integration. FLAML emphasizes cost-effective AutoML. AutoGluon is productive for tabular, multimodal, and time-series baselines. W&B sweeps integrate well with experiment tracking.