CSC321: Neural
Networks
Lecture 14: Clustering
Clustering
The k-means algorithm
Why K-means converges
Local minima
Soft k-means
A generative view of
clustering
The mixture of Gaussians
generative model
The E-step: Computing
responsibilities
The M-step: Computing new
mixing proportions
More M-step: Computing the
new means
More M-step: Computing the
new variances
How many Gaussians do we
use?
Avoiding local optima
Speeding up the fitting
The next 5 slides are
optional extra material that will not be in the final exam
Why EM converges
The expected energy of a
datapoint
The entropy term
The E-step chooses the
responsibilities that minimize the cost function (with the parameters of the
Gaussians held fixed)
The M-step chooses the
parameters that minimize the cost function (with the responsibilities held
fixed)