Rewarding softness
If a datapoint is exactly halfway
between two clusters, each
cluster should obviously have the
same responsibility for it.
The responsibilities of all the
clusters for one datapoint should
add to 1.
A sensible softness function is
the entropy of the
responsibilities.
Maximizing the entropy is like
saying: be as uncertain as
you can about which cluster
has responsibility
We want high entropy
responsibilities, but we also
want to focus the
responsibility for a data-point
on the nearest cluster
centers.
Responsibility
of cluster i for
datapoint j
Number of
clusters, k
Entropy of the
responsibilities
for datapoint j