What is the best distribution?
The sender and receiver can use any distribution they
But what distribution minimizes the expected
message length
The minimum occurs when we pick codes using a
Boltzmann distribution:
This gives the best trade-off between entropy and
expected energy.
It is how physics behaves when there is a system that
has many alternative configurations each of which
has a particular energy (at a temperature of 1).