Softmax units (one per possible word)

output

Skip-layer connections

Units that learn to predict the output word from features of the input words

Learned distributed encoding of word t-2

Learned distributed encoding of word t-1

Table look-up

Table look-up

inputs

Index of word at t-2

Index of word at t-1