12
The batch learning algorithm
Positive phase
Clamp a datavector on the visible units.
Let the hidden units reach thermal equilibrium at a
temperature of 1 (may use annealing to speed this up)
Sample for all pairs of units
Repeat for all datavectors in the training set.
Negative phase
Do not clamp any of the units
Let the whole network reach thermal equilibrium at a
temperature of 1 (where do we start?)
Sample for all pairs of units
Repeat many times to get good estimates
Weight updates
Update each weight by an amount proportional to the
difference in in the two phases.