Finding binary codes for documents

2000  reconstructed counts

• Train an auto-encoder using 30 logistic units for the code layer. • During the fine-tuning stage, add noise to the inputs to the code units. – The “noise” vector for each training case is fixed. So we still get a deterministic gradient. – The noise forces their activities to become bimodal in order to resist the effects of the noise. – Then we simply round the activities of the 30 code units to 1 or 0.

500 neurons

250 neurons

30

noise

250 neurons

500 neurons

2000  word counts