Actor-Critic Methods

When to Use This Skill

Invoke this skill when you encounter:

• Algorithm Selection: "Should I use actor-critic for my continuous control problem?"
• SAC Implementation: User implementing SAC and needs guidance on entropy coefficient tuning
• TD3 Confusion: "Why does TD3 have twin critics and delayed updates?"
• Training Instability: "My actor-critic diverges. How do I stabilize it?"
• A2C/A3C Questions: "What's the difference between A2C and A3C?"
• Continuous Control: User has continuous action space and needs appropriate algorithm
• Critic Issues: "My critic loss isn't decreasing" or "Advantage estimates are wrong"
• SAC vs TD3: "Which algorithm should I use for my problem?"
• Entropy Tuning: "How do I set the entropy coefficient α in SAC?"
• Policy Gradient Variance: "My policy gradients are too noisy. How do I reduce variance?"
• Implementation Bugs: Critic divergence, actor-critic synchronization, target network staleness
• Continuous Action Handling: Tanh squashing, log determinant Jacobian, action scaling