Channel Capacity

When to Use

Use this skill when working on channel-capacity problems in information theory.

Decision Tree

1. Mutual Information

   • I(X;Y) = H(X) + H(Y) - H(X,Y)
   • I(X;Y) = H(X) - H(X|Y) = H(Y) - H(Y|X)
   • Symmetric: I(X;Y) = I(Y;X)
   • scipy.stats.entropy(p) + scipy.stats.entropy(q) - joint_entropy

2. Channel Model

   • Input X, output Y, channel P(Y|X)
   • Channel matrix: rows = inputs, columns = outputs
   • Element (i,j) = P(Y=j | X=i)