The free-energy of a mean-field logistic unit
In a mean-field logistic unit, the
total input provides a linear
energy-gradient and the negative
entropy provides a containment
function with fixed curvature.
So it is impossible for the value
0.7 to have much lower free
energy than both 0.8 and 0.6.
This is no good for modeling
real-valued data.
- entropy
0 output-> 1