Overview

Each student, or pair of students, will complete a substantial data-science project. PhD students whose research has a data-science component are welcome to base the project on that work. Students are encouraged to pick a dataset of interest to them and to aim for one or more of the following:

• Obtaining interesting insights from the dataset.
• Applying interesting data-analysis techniques to the dataset.
• Demonstrating a new data-analysis technique and applying it to the dataset.
• Collecting a novel dataset.

Components of a complete project

A complete project includes:

• A description of the dataset and a summary of the relevant domain knowledge (e.g., for audio data, a summary of how sound is generated, perceived, and processed; for hospital length-of-stay data, a summary of the relevant literature).
• A summary of previous work related to the analysis.
• An exploratory analysis of the dataset.
• A description of the method applied and a summary of related work.
• A description of the outcomes of the analysis, or an evaluation of a predictive system that was built.
• A unique dataset (e.g., scraped from Reddit, collected through a survey, or assembled from public APIs — not obtained from Kaggle, UCI, or other curated repositories).
• At least one statistical-analysis technique drawn from the course.
• A dashboard. Building one with claude code is straightforward, and a dedicated tutorial covers this.

Deliverables and deadlines

Preliminary Proposal (approximately one page, due May 19): identifies the data source, summarizes at least two related papers, and lists the team (one or two students per project). See the proposal handout for full instructions.

Preliminary Project Presentation and Document (due June 15): each student or pair submits a preliminary document containing a full project description and initial exploratory-analysis results, and gives a short presentation to the class.

Final Project (due August 10): the completed analysis, with all components listed above.

Worked example

A representative project is the “culture of honor” analysis: scrape Reddit, use an LLM to flag personal attacks, geo-locate users, and test the hypothesis that users in the U.S. South retaliate more often than users elsewhere. The project illustrates the full cycle of data acquisition, preprocessing, hypothesis specification, and statistical testing. See the course-introduction notes for details.

Resources

• Proposal handout.
• Course-introduction notes (worked example).
• Lecture calendar for technique references.
• Syllabus (PDF).