Fine-tuning for discrimination
First learn one layer at a time greedily.
Then treat this as “pre-training” that finds a good
initial set of weights which can be fine-tuned by
a local search procedure.
Contrastive wake-sleep is one way of fine-
tuning the model to be better at generation.
Backpropagation can be used to fine-tune the
model for better discrimination.
This overcomes many of the limitations of
standard backpropagation.