Using Morphogenetic RL

Overview

A morphogenetic controller is an RL agent acting on a non-stationary environment whose state space includes its own past structural decisions. Its mistakes change the shape of the network it is trying to optimize.

That sentence is the whole problem. It implies four load-bearing properties that this pack designs into the system:

• Reward must net structural cost against utility. "Loss went down after I grew" is not enough — the network may have improved despite the growth, not because of it. Counterfactual baselines matter.
• Catastrophic actions must be vetoed by something the controller does not own. A governor running outside the policy is not a nice-to-have; without it, the policy will eventually take an action that destroys training, and there is no gradient signal informative enough to teach it not to.
• Determinism across topology change is load-bearing, not optional. Without it, you cannot reproduce a failure, cannot ablate, cannot compare runs, and cannot debug the controller.
• Evaluation under topology change is its own discipline. Two checkpoints with different shapes cannot be compared on raw parameter count, FLOPs, or loss curves alone.

Key tensions: exploration vs. stability, controller autonomy vs. governor veto, reward density vs. reward honesty, determinism vs. performance. Every sheet in the pack resolves one or more of these.

When to Use

Use this pack when: