Distribution Search

Overview

This skill provides structured guidance for finding probability distributions that satisfy specific statistical constraints. These problems typically involve constructing a discrete probability distribution over a large vocabulary that achieves target values for metrics like KL divergence, entropy, or other information-theoretic quantities.

Mathematical Analysis First

Before writing any code, perform thorough mathematical analysis to constrain the solution space:

1. Derive analytical relationships between the target metrics and distribution properties

   • For KL divergence from uniform: KL(P||U) = -H(P) + log(V) where H(P) is entropy and V is vocabulary size
   • For KL divergence to uniform: KL(U||P) = log(V) - (1/V) * Σ log(P(i))

2. Calculate fixed quantities from the problem specification

   • Vocabulary size V determines log(V)
   • Target values combined with log(V) constrain feasible entropy ranges