SML310 Course Project

General guidelines

The idea of the course project is for you to get some experience trying to do a piece of original research in data science and writing up your results in a format that is similar to what you would see in published literature. We expect to see an idea/task that you describe clearly, relate to existing work, and implement, as well as a report on the results. You will need to write code, run it on some data, make some figures, read a few background papers, collect some references, and write a few pages describing your task, the algorithm(s) you used (if any) and the results you obtained.

You should submit both your write-up and the code that you wrote.

Requirements

There is no specific length requirement, but we would expect that good projects would require about 4-8 pages of text (not including appendices and figures). Note that many journals and conferences would limit you to 6-10 pages in total. For the course project, you are allowed to write more, which is sometimes easier than writing less and still covering all the ground that you need to cover.

It is to be expected that the rubric below won't be a perfect fit for some people's projects, so don’t feel obligated to always follow the instructions below exactly if there is a way to make your paper better by deviating from them. However, if you want to structure your paper significantly differently, please consult us before doing so! When grading, we will consider the problem at hand. For example, if it is very difficult to produce interesting visualizations for your problem (because you are mainly developing an algorithm), we will take that into account when grading.

1. Abstract (5 points): a summary of the main idea of the project and its contributions.
   

   
   
   • Should be understandable to anyone in the course.
     
   • You don't need to cover everything you did, just say what the main idea was and what were one or two of the main takeaways.
   
   Grading scheme :
   
   
   
   • 5/5: A clear and concise abstract that gives the reader a clear idea of what the project is about and why it is interesting.
   • 3/5: The abstract is difficult to read and/or is very vague and/or doesn’t sell the project as well as it might have.

2. Introduction (10 points): a statement of the problem being addressed and why we might want to solve it. In this section, it makes sense to link the work that you are doing to prior work and explain what your contribution is; to explain in more detail why the work you are doing is potentially imporant; to briefly state your results; and to outline the structure of the paper and what each of the following sections in the paper achieves.

   Grading scheme :

   • 5/5: A readable write-up that explains what the problem is and why it is of interest.
   • 3/5: The write-up is difficult to read, somewhat vague, or doesn’t make a really good case for why the problem is of interest.

3. Background (10 points): an explanation of where the data comes from and how it is collected. In this section, you should explain your dataset. For example, if you are working with finance data, briefly explain any financial terminology (e.g., if you are working with stock prices, explain briefly how stock trading works); if you are working with sociological data, explain each variable being measured and why it is important to measure it.
   

   Grading scheme :

   • 10/10: A readable and insightful write-up that gives the reader everything that they need to understand the dataset and its significance.
   • 9/10: A coherent write-up, but the reader would have some difficulty understanding what’s going on, and might need to do their own research.
   • 7/20: A flawed and/or perfunctory write-up. Little insight is gained from it, and the reader may need to do significant research to understand what’s going on.

4. Prior work (15 points): a survey of at least 4-5 research papers that are important for understanding the dataset you are working with and the methods you are using (there should be at least one paper related to the data, and one paper related to the methods). This would usually be in the form of an essay rather than in the form of a list. You should relate the background work to what you are doing. For each work you survey, the reader should understand the main contribution of the paper (i.e., what the paper did that was new and interesting) and the methods that were used. While you do not need to re-explain things like linear regression, it is appropriate to discuss the methods being used if they were not mentioned in the course and are not explained in something like OpenIntro Statistics or PRML .
   

   Grading scheme :

   • 15/15: A readable write-up that gives the reader a clear idea of the current state of the problem, the methods used, and the open problems. The relationship between what the current paper is doing and prior work is made clear. The technical details given are pertinent and understandable.
   • 13/15: The write-up is readable, but the reader needs to do some work (including their own research sometimes) to understand the current state of the problem and how it relates to the paper.
   • 12/15: The student clearly understands the state of the field, but the reader needs to do their own research to gain an understanding of what is going on.
   • 11/15: The write-up is flawed: it is impossible to understand what is going on without reading the sources being cited, and/or the sources being cited are not necessarily the most relevant sources there could be.
   • 9/15: The write-up is significantly flawed, but at least 5 works are surveyed.

5. Exploratory Analysis and Visualization (15 points). Produce figures and tables that summarize the data in an understandable way. It is important that your analysis both tells the reader about the dataset, and gives the reader insights into the problem that you are addressing. For example, if you are trying to predict Y using X, there should be a visualization of the relationship between Y and X. If you are fitting a statistical model, you should present visual evidence that the model makes sense, if possible. You should produce at least 2 figures. You will likely want visualize the results of your analysis/experiments as well.
   

   Grading scheme :

   • 15/15: Figures and tables that give good insight into the dataset. The figures are neat and easily readable.
   • 13/15: The figures and tables are good, but do not give particularly interesting insights into the dataset.
   • 11/15: The figures are okay, but they seem perfunctory, and no particular insight is gained from them.

6. Method description (10 points) A description of your method. You should include a description of everything you did here (including model checking etc.). All terms must be defined, and all algorithms that you use explained. Highlight how your model is different from other approaches, or what the main relevant considerations are for the domain. Including a diagram is usually useful. This can be done by comparing it to an existing model, perhaps by using another diagram or in words. For example, if you are proposing a new algorithm that only changes one line in an existing algorithm, highlight that one line, or do a side-by-side comparison.
   

   Grading scheme :

   • 10/10: A full and comprehensible description.
   • 9/10: A somewhat flawed description, with some things unclear.
   • 7/10: A flawed description: some things are just unclear.

7. Design of the Analysis and/or Experiments (15 points) This might or might not be a separate section in your paper. You should design the analysis and/or experiments carefully, and justify the design choices that you made. You will be graded both on the clarity of the description and on the design itself. The grading will vary from person to person.
   

   Grading scheme :

   • 15/15: A full and comprehensible description. The author has put thought into designing and running analyses/experiments that would test whether the idea/hypothesis works or not.
   • 13/15: A somewhat flawed description. The experiments/analysis are OK, but could be better.
   • 11/15: A flawed description and/or analysis.

8. Limitations (5 points): A section describing the limitations of your approach with respect to being able to draw conclusions about the population you are researching and/or the method. For example, the dataset might not be fully representative of the population you are trying to research; or the algorithm you are using might only work in some situations, but might fail on new data under some circumstances.
   

   Grading scheme :

   • 5/5: An insightful analysis of the limitations.
   • 3/5: A perfunctory analysis that doesn’t give much insight.

9. Results and Conclusion (10 points): A section describing the results. Note that you are not graded on whether your hypothesis turned out to be correct – the only thing that’s important is that the results are presented well. For the conclusion section, present the main takeaways, and give some ideas for future work.
   

   Grading scheme :

   • 10/10: The results are presented in an understandable and insightful way. An insightful and correct analysis in the conclusion. Future work ideas are insightful.
   • 9/10: The results are presented well, and the conclusion is warranted and correct. Future work ideas make sense.
   • 8/10: The results are presented correctly, and the conclusion is warranted.

10. Depth and Originality (5 points): 5 points are assigned for the depth of the idea. Essentially, 5/5 will be awarded for original ideas and/or especially difficult-to-execute projects. 4/5 will be awarded for good ideas that make sense. Fewer points will be awarded for projects that fall short of that, but I hope that this will rarely happen since we will work together to make the project make sense.