 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
• |
The sender and
receiver can use any distribution they
|
|
|
like
|
|
|
|
– |
But
what distribution minimizes the expected
|
|
|
message
length
|
|
|
• |
The minimum
occurs when we pick codes using a
|
|
|
Boltzmann
distribution:
|
|
|
• |
This gives the
best trade-off between entropy and
|
|
|
expected energy.
|
|
|
|
– |
It
is how physics behaves when there is a system that
|
|
has
many alternative configurations each of which
|
|
|
has
a particular energy (at a temperature of 1).
|
|