In thie project, you will work on extending min-char-rnn.py. You will experiment with the Shakespeare dataset.
Write a function to sample text from the model using different temperatures (i.e., 1/alpha on Slide 5). Try different temperatures, and, in your report, include examples of texts generated using different temperatures. Briefly discuss what difference the temperature makes.
Include the listing (i.e., source code) of the function you wrote/mofigied to accomplish the task in the report.
You should either train the RNN yourself, or use the weights from Part 3 -- up to you.
Write a function that uses an RNN to complete a string. That is, the RNN should generate text that is a plausible continuation of a given starter string. In order to do that, you will need to compute the hidden activity h at the end of the starter string, and then to start generating new text.
Include 5 interesting examples of outputs that your network generated using a starter string. (This part need not be easily reproducible).
Include the listing (i.e., source code) of the function you wrote in the report.
You should either train the RNN yourself, or use the weights from Part 3 -- up to you.
The weights for a trained RNN are available here. Some sample from the RNN (at temeperature=1) are here, and code to read in the weights is here (if this doesn't work, try the pickle file, and get a using import cPickle as pickle; a = pickle.load(open("char-rnn-snapshot.pkl")).
In the samples that the RNN generated, it seems that a newline or a space usually follow the colon (i.e., ":") character. In the weight data provided, identify the specific weights that are responsible for this behaviour by the RNN. In your report, specify the coordinates and values of the weights you identified, and explain how those weights make the RNN generate newlines and spaces after colons.
Identify another interesting behaviour of the RNN, identify the weights that are responsible for it. Specify the coordinates and the values of the weights, and explain how those weights lead to the behaviour that you identified. To obtain more than 2/10 for the bonus, the behaviour has to be more interesting than the behaviour in Part 3 (i.e., charachter A following character B).
The project should be implemented using Python 2. Your report should be in PDF format. You should use LaTeX to generate the report, and submit the .tex file as well. A sample template is on the course website. You will submit three files: p4.py, p4.tex, and p4.pdf. You can submit other Python files as well: we should have all the code that's needed to run your experiments. Y
Reproducibility counts! We should be able to obtain all the graphs and figures in your report by running your code. Submissions that are not reproducible will not receive full marks. If your graphs/reported numbers cannot be reproduced by running the code, you may be docked up to 20%. (Of course, if the code is simply incomplete, you may lose even more.) Suggestion: if you are using randomness anywhere, use numpy.random.seed().
You must use LaTeX to generate the report. LaTeX is the tool used to generate virtually all technical reports and research papers in machine learning, and students report that after they get used to writing reports in LaTeX, they start using LaTeX for all their course reports. In addition, using LaTeX facilitates the production of reproducible results.
Using my code
You are free to use any of the code available from the course website.
Important: