Quantities of information and probability theory

7 minute read

Entropy, conditional entropy, mutual information, cross entropy, Kullback-Leibler divergence, and perplexity