Numerai Experiment Design

Use this workflow to plan, run, and report Numerai experiments for any model idea.

Note: run commands from numerai/ (so agents is importable), or from repo root with PYTHONPATH=numerai.

Persistence expectation (required)

This skill is not complete after a single promising run. You must run experiments in rounds (typically 4–5 configs per round), synthesize results, and decide what to try next. Only finalize when you reach a plateau and additional rounds stop improving the primary metric.

Planning checklist (answer before running)

• State the model idea and novelty.
• Choose the initial baseline and feature set. Default to deep_lgbm_ender20_baseline (feature_set=all) unless the user explicitly requests the small baseline; keep experiments' feature_set aligned with the chosen baseline.
• Decide the primary metric (bmc_mean and bmc_last_200_eras) where BMC = Benchmark Model Contribution vs official v52_lgbm_ender20.
• Decide which parameter dimensions to explore based on the core idea (targets, model hyperparameters, ensemble weights, data settings).
• Or decide that only a minimal round is needed because the change is tiny — but still run multiple variants unless the user explicitly requested exactly one run.