 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| • |
Inference is
difficult in directed models of time series if
|
|
|
we use
distributed representations in the hidden units.
|
|
|
– |
So
people tend to avoid distributed representations
|
|
|
(e.g.
HMM’s)
|
|
| • |
If we really
need distributed representations (which we
|
|
|
nearly always
do), we can make inference much simpler
|
|
by using three
tricks:
|
|
|
– |
Use
an RBM for the interactions between hidden and
|
|
|
visible
variables.
|
|
|
– |
Include
temporal information in each time-slice by
|
|
|
concatenating
several frames into one visible vector.
|
|
|
– |
Treat
the hidden variables in the previous time slice
|
|
|
as
additional fixed inputs.
|
|