CSC321: Neural Networks

Lecture 4: Learning to model relationships and word sequences

Learning by back-propagating error derivatives

Some Success Stories

An example of relational information

Another way to express the same information

A relational learning task

The structure of the neural net

How to show the weights of hidden units

The features it learned for person 1

What the network learns

Another way to see that it works

Why this is interesting

A subtelty

A basic problem in speech recognition

The standard “trigram” method

Why the trigram model is silly

Bengio’s neural net for predicting the next word

An alternative architecture