An RBM with real-valued visible units
(you dont have to understand this slide!)
In a mean-field logistic unit, the total
input provides a linear energy-
gradient and the negative entropy
provides a containment function with
fixed curvature. So it is impossible
for the value 0.7 to have much lower
free energy than both 0.8 and 0.6.
This is no good for modeling real-
valued data.
Using Gaussian visible units we can
get much sharper predictions and
alternating Gibbs sampling is still
easy, though learning is slower.
Text Box: F
- entropy
0 output-> 1