The bits-back argument
data
Gaussian 0
Gaussian 1
• Consider a datapoint that is equidistant from two cluster
centers.
– The sender could code it relative to cluster 0 or
relative to cluster 1.
–  Either way, the sender has to send one bit to say
which cluster is being used.
• It seems like a waste to have to send a bit when you don’t
care which cluster you use.
• It must be inefficient to have two different ways of encoding
the same point.