CSC321: Neural Networks

 Lecture 5: Applying backpropagation to shape recognition

Applying backpropagation to shape recognition

The invariance problem

The invariant feature approach

The normalization approach

The replicated feature approach

Backpropagation with weight constraints

Combining the outputs of replicated features

The hierarchical partial invariance approach

Le Net

The architecture of LeNet5

Slide 12

Slide 13

Slide 14

Slide 15

Slide 16

Slide 17

Slide 18

Slide 19

Slide 20

Slide 21

The 82 errors made by LeNet5

A brute force approach

Making dumb backpropagation work really well for recognizing digits

Problems with squared error

Softmax