The Markov chain for unigauss experts

j

j

j

j

a fantasy

i

i

i

i

t = 0                 t = 1                  t = 2                               t = infinity

Each hidden unit has a binary state which is 1 if the unigauss chose its Gaussian. Start with a training vector on the visible units. Then alternate between updating all the hidden units in parallel and updating all the visible units in parallel. Update the hidden states by picking from the posterior. Update the visible states by picking from the Gaussian you get when you multiply together all the Gaussians for the active hidden units.