 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| • |
We could find
good rivals by repeatedly making a random
|
|
perturbation to
the data and accepting the perturbation
|
|
|
with a
probability that depends on the energy change.
|
|
|
– |
Diffuses
very slowly over flat regions
|
|
|
– |
Cannot
cross energy barriers easily
|
|
|
| • |
In
high-dimensional spaces, it is much better to use the
|
|
|
gradient to
choose good directions.
|
|
| • |
HMC adds a
random momentum and then simulates a
|
|
|
particle moving
on an energy surface.
|
|
|
– |
Beats
diffusion. Scales well.
|
|
|
– |
Can
cross energy barriers.
|
|
|
– |
Back-propagation
can give us the gradient of the
|
|
|
energy
surface.
|
|