Project Report Grading Rubric
Students can work on projects individually,in pairs, or even in triplets. The grade will depend on the ideas, how well you present them in the report, how clearly you position your work relative to existing literature, how illuminating your experiments are, and well-supported your conclusions are.
You can use word or whatever you like to format your report, but ideally it will be in the format of a machine learning conference paper such as NIPS.
The idea is that this project report should be a manageable amount of work, but that if you want to turn your project into a paper, everything in the project report will need to be done anyways. If you feel that your project won't fit into this rubric, please talk to me. There are many ways to make contributions to a field!
Length: 4 to 8 pages, not including appendices. Don't be afraid to keep the text short and to the point, and to include large illustrative figures.
Abstract (5 points) that summarizes the main idea of the project and its contributions.
- Should be understandable to anyone in the course.
- You don't need to say everything you did, just what the main idea was and one or two takeaways.
Introduction (5 points) that states the problem being addressed and why we might want to solve it.
Figure or diagram (10 points) that shows the overall model or idea. The idea is to make your paper more accessible, especially to readers who are starting by skimming your paper.
- For the project, taking a picture of a hand-drawn diagram is fine, as long as it's legible.
- For camera-ready diagrams, we recommend using Tikz, a LaTeX package.
- Try to be clear whether arrows indicate computational flow, or conditional dependencies, or both.
Formal description (20 points) of the model / loss function / conjecture / problem domain.
Include at least one of:
- An algorithm box.
- Equations describing your model.
- A theorem or formally stated conjecture.
A formal description of a problem domain.
Highlight how your model is different from other approaches, or what the main relevant considerations are for the domain. This can be done by comparing it to an existing model, perhaps by using another diagram or in words. E.g. if you are proposing a new algorithm that only changes one line in an existing algorithm, highlight that one line, or do a side-by-side comparison.
Related work (20 points) section and bibliography.
- If your project builds on previous work, clearly distinguish what they did from what your new contribution is.
- Include a 1-2 sentence summary of other closely related papers.
- You might not know about all related papers (or have time to carefully read all related papers), and that's OK for this project. A rough guide is that you should be able to find 3-4 closely related papers, and another 3-4 papers that all those papers cite as foundational work. These foundational papers are often cited in the introduction.
- Using bibtex is annoying at first, but Google Scholar can give you the bibtex entries, and it will save you time in the long run.
Comparison or demonstration (20 points). Include at least one of:
- A demonstration of a theorem or conjecture. For example, an example or counter-example.
- A comparison of data generated by your model to a baseline model. Qualititative evaluation is OK for the project.
- An experiment demonstrating a property that your model has that a baseline model does not. Experiments should also include a description of how you prepared your datasets, how you trained your model, and any tricks you used to get it to work.
If doing a review, include a table comparing the properties of the different approaches.
Toy data is OK! The point is to help the reader understand why or when we would want to use one approach over another, or to understand something better. Try to summarize the main takeaways.
Limitations (15 points) of your approach.
- Describe some settings in which we'd expect your approach to perform poorly, or where all existing models fail.
- Try to guess or explain why these limitations are the way they are.
- Give some examples of possible extensions, ways to address these limitations, or open problems.
Conclusions (5 points)
- State the results achieved in relation to the problem described in the introduction.
- Repeat the main takeaways from your paper.
Extra guidelines:
- You're free to play with the format of your paper, as long as all the above content is there and easy to find.
- Put long mathematical derivations that interrupt the flow of the paper in an appendix. This is also a good place to put extra generated examples.
- We encourage you to attach your code, although this is not required.
- Axes should be labeled.
- Text in figures should be about the same size as the text in the rest of the paper.
- For figures, try to use a vector graphics format such as pdf, eps or svg so that your figures don't look blurry.
Positive examples:
PixelVAE: A Latent Variable Model for Natural Images
Your project report doesn't have to be as polished as this, of course!