What is the best distribution?
The sender and receiver can use any distribution they
like
But what distribution minimizes the expected
message length
The minimum occurs when we pick codes using a
Boltzmann distribution:
This gives the best trade-off between entropy and
expected energy.
It is how physics behaves when there is a system that
has many alternative configurations each of which
has a particular energy (at a temperature of 1).