agent-validation-experiment
Installation
SKILL.md
Agent Validation Experiment
Overview
| Item | Details |
|---|---|
| Date | 2026-02-18 |
| Goal | Determine if Claude agent integration improves model predictive power |
| Status | v2.4 showed agents HURT performance (fitness -38.2%, PF -6.3%). v3.0 implements guardrails: phase gates, cumulative entropy bounds, fitness-gated checkpoints, rollback, institutional knowledge, agent memory persistence. Awaiting re-run. |
| Files | scripts/agent_validation_experiment.py, notebooks/agent_validation_analysis.ipynb, alpaca_trading/training/multi_agent.py, alpaca_trading/training/agent_memory.py, alpaca_trading/gpu/vectorized_env.py, tests/test_multi_agent.py |
The Question
Does integrating Claude agents during RL training provide measurable improvement to model quality?
The codebase has ~4,744 lines of agent integration code. Before relying on it:
- We need empirical evidence that agents help
- We need to quantify the cost-benefit tradeoff