Why the autoregressive connections do not cause
problems
The autoregressive connections do not mess up
contrastive divergence learning because:
We know the initial state of the visible units, so we
know the initial effect of the autoregressive
connections.
It is not necessary for the reconstructions to be at
equilibrium with the hidden units.
The important thing for contrastive divergence  is to
ensure the hiddens are in equilibrium with the visibles
whenever statistics are measured.