Entropy

When to Use

Use this skill when working on entropy problems in information theory.

Decision Tree

1. Shannon Entropy
   • H(X) = -sum p(x) log2 p(x)
   • Maximum for uniform distribution: H_max = log2(n)
   • Minimum = 0 for deterministic (one outcome certain)
   • scipy.stats.entropy(p, base=2) for discrete