The lectures are
3:00-5:00pm on Wednesdays in IB 210.
Tentative Schedule:
January
4:
Lecture 1: What are neural networks?
(notes as .ppt ) (notes as .pdf))
Reading: How neural networks learn from experience (.pdf)
January
4:
Lecture 2: Two simple learning algorithms
(notes as .ppt ) (notes as .pdf))
Reading: Connectionist Learning Procedures, pp 185-190; 193-198. (.pdf)
January
11:
Lecture 3: Learning in multilayer networks
(notes as .ppt) (notes as .pdf))
Reading: Connectionist Learning Procedures, pp 198-205. (.pdf)
January
11:
Lecture 4: Learning to model relationships and word sequences
(notes as .ppt ) (notes as .pdf))
Reading: A neural probabilistic language model. (.ps
file)
(a paper written for researchers about predicting the next word in a
sentence).
January
18:
Lecture 5: Distributed representations
(notes as .ppt ) (notes as .pdf))
Reading: Book chapter on "Distributed Representations" (.pdf)
Reading: Scientific American article on "Simulating Brain
Damage" (.pdf)
January
25:
Lecture 7: Learning in recurrent networks
(notes as .ppt ) (notes as .pdf))
Reading: Learning internal representations by error propagation, pp
354-362. (.pdf)
January
25:
Lecture 8: Modeling text using a recurrent neural network trained with a
really fancy optimizer
(notes
as .ppt ) (notes as
.pdf))
Reading: Generating text with recurrent neural networks (hardcopy will
be handed out in class)
February
15:
Lecture 13: Learning without a teacher: Autoencoders and PCA
(notes
as .ppt ) (notes as
.pdf )
Reading: ()
February
15:
Lecture 14: Clustering: The EM algorithm for fitting mixtures of
Gaussians
(notes
as .ppt ) (notes as
.pdf )
Reading: ()
February
22: No Lectures (reading week)
February
29:
Lecture 15: Mixtures of experts
(notes
as .ppt ) (notes as
.pdf )
Reading: Adaptive mixtures of local experts (.pdf)
February
29:
Lecture 16: Hopfield Nets and simulated annealing
(notes
as .ppt ) (notes as
.pdf )
Reading: For a gentle introduction to the idea of memories as energy
minima (.ps
) (.html
)
Reading: For a gentle introduction to how to add new memories by
creating new minima (.ps
) (.html
)
March
14:
Lecture 20: Learning features one layer at a time
(notes
as .ppt ) (notes as
.pdf )
Reading for lectures 19 and 20: Optional extra reading for lectures 19
and 20: "Learning multiple layers of representation". (.pdf)
March
21:
Lecture 21: Using backpropagation to fine-tune deep networks
(notes
as .ppt ) (notes as
.pdf )
Reading for lecture 21: "Reducing the dimensionality of data with
neural networks" (.pdf)
March
21:
Lecture 22: Transforming Autoencoders for learning the right
representation of shapes.
(notes
as .ppt ) (notes as
.pdf )
Reading for lecture 22: "Transforming Autoencoders" (.pdf)