An alternative architecture

A single output unit that gives a score for the candidate word in this context

Units that discover good or bad combinations of features

Learned distributed encoding of word t-1

Learned distributed encoding of candidate

Learned distributed encoding of word t-2

Index of word at t-2

Index of word at t-1

Index of candidate

Try all candidate words one at a time