Why the lateral connections do not cause problems
The lateral connections do not mess up contrastive
divergence learning because:
We know the initial state of the visible units, so we
know the initial effect of the lateral connections.
It is not necessary for the reconstructions to be at
equilibrium with the hidden units.
The important thing for contrastive divergence  is to
ensure the hiddens are in equilibrium with the visibles
whenever statistics are measured.