How sharp are products of experts?
If each of the M experts is a Gaussian with the
same variance, the product is a Gaussian with a
variance of 1/M on each dimension.
But a product of lots of Gaussians is just a
Gaussian
Adding Gaussians allows us to create
arbitrarily complicated distributions.
Multiplying Gaussians doesn’t.
So we need to multiply more complicated
“experts”.