Products of Experts and energies
Products of Experts multiply probabilities together. This
is equivalent to adding log probabilities.
Mixture models add contributions in the probability
Product models add contributions in the log
probability domain. The contributions are energies.
In a mixture model, the only way a new component can
reduce the density at a point is by stealing mixing
In a product model, any expert can veto any point by
giving that point a density of zero (i.e. an infinite energy)
So its important not to have overconfident experts in a
product model.
Luckily, vague experts work well because their
product can be sharp.