Rewarding softness
If a datapoint is exactly halfway
between two clusters, each
cluster should obviously have the
same responsibility for it.
The responsibilities of all the
clusters for one datapoint should
add to 1.
A sensible softness function is
the entropy of the
Maximizing the entropy is like
saying: be as uncertain as
you can about which cluster
has responsibility
We want high entropy
responsibilities, but we also
want to focus the
responsibility for a data-point
on the nearest cluster
of cluster i for
datapoint j
Number of
clusters, k
Entropy of the
for datapoint j