Maximum Entropy
Maximum entropy is a general approach of inferring a probability distribution from constraints that do not uniquely determine that probability.
Shannon’s perspective
- Intuitively, information I, is related to the measure of uncertainty in the outcome of an experiment.