CSC2552 Topics in Computational Social Science: AI, Data, and Society

Winter 2021

Lecture hours: Thu 3-5pm
Lecture link: https://utoronto.zoom.us/j/88282089787

Instructor: Ashton Anderson, Assistant Professor
Email: ashton [at] cs [dot] toronto [dot] edu
Office location: online
Office hours: by appointment

Course News

Course Description

The mass migration of life onto online platforms has ushered in a new era of social research. The recent availability of large-scale datasets of human behaviour and the new potential for massive online experimention introduce both great opportunities and challenges. In this class, a seminar course on computational social science, we will survey the explosion of research at the intersection of computer science and the social sciences.

The goals of this seminar class are twofold: first, students will be acquainted with computational tools and techniques for processing and thinking about social data, and be exposed to papers spanning the full spectrum of computational social science methods from large-scale empirical data analysis to online experimetation. Second, students will develop research skills by reading, reviewing, presenting, and discussing recent academic papers.

Every week, we will cover a different computational social science topic by reading two papers and discussing them. Before class, everyone will write a review of the papers, identifying their research questions, strengths and weaknesses, and connection to other literature. Each paper will be assigned to 2-3 people who will lead a group discussion of it in class. Throughout the term, everyone will get the chance to present one paper.

The major coursework component of the course, besides the weekly reviews, will be a term project. Students will propose a topic, give a presentation on their work, and submit a final report. The project will give students a chance to identify an interesting computational social science problem and implement it, and could potentially lead to publication in a workshop or conference.

Grading scheme: Textbook

We will read the book Bit By Bit: Social Research in the Digital Age by Matthew Salganik. It's available online for free, and in print form at a reasonable cost.

Schedule

The following is a tentative schedule of topics we'll cover (subject to change).

Week Date Topic Reviews Due Textbook Readings
1 1/14 Introduction to computational social science Video | Slides Ch. 1
2 1/21 Introduction to computational social science cont'd Video | Slides Ch. 1
3 1/28 Observational studies 1 1/27 9:00pm Ch. 2
4 2/4 Observational studies 2 2/3 9:00pm Ch. 2
5 2/11 Experiments 1 2/10 9:00pm Ch. 4
6 2/25 Experiments 2 2/24 9:00pm Ch. 4
7 3/4 Project proposals
8 3/11 Asking questions 3/10 9:00pm Ch. 3
9 3/18 Mass collaboration 3/17 9:00pm Ch. 5
10 3/25 Ethics in computational social science 3/24 9:00pm Ch. 6
11 4/1 Project presentations (Part 1)
12 4/8 Project presentations (Part 2)


Classes

1/28: Observational studies 1

Main papers:

  1. Fake news on Twitter during the 2016 U.S. presidential election (and online supplement).
    N. Grinberg, K. Joseph, L. Friedland, B. Swire-Thompson, D. Lazer.
    Science, 2019.
  2. Dissecting racial bias in an algorithm used to manage the health of populations (and online supplement).
    Z. Obermeyer, B. Powers, C. Vogeli, S. Mullainathan.
    Science, 2019.

2/4: Observational studies 2

Main papers:

  1. Exposure to ideologically diverse news and opinion on Facebook (and online supplement).
    E. Bakshy, S. Messing, L. Adamic.
    Science, 2015.
  2. Human Decisions and Machine Predictions.
    J. Kleinberg, H. Lakkaraju, J. Leskovec, J. Ludwig, S. Mullainathan.
    Quarterly Journal of Economics (QJE), 2017.

2/11: Experiments 1

Main papers:

  1. The Role of Social Networks in Information Diffusion.
    E. Bakshy, I. Rosenn, C. Marlow, L. Adamic.
    WWW, 2012.
  2. Algorithm aversion: People erroneously avoid algorithms after seeing them err.
    B. Dietvorst, J. Simmons, C. Massey.
    Journal of Experimental Psychology, 2014.

2/25: Experiments 2

Main papers:

  1. The Welfare Effects of Social Media.
    H. Allcott, L. Braghieri, S. Eichmeyer, and M. Gentzkow.
    American Economic Review, 2020.
  2. Manipulating and Measuring Model Interpretability.
    F. Poursabzi-Sangdeh, D.G. Goldstein, J.M. Hofman, J.W. Vaughan, H.Wallach.
    CHI, 2021.

3/4: Project proposals

Students will present their project proposals.

3/11: Asking questions

Main papers:

  1. Predicting poverty and wealth from mobile phone metadata (and online supplement).
    J. Blumenstock, G. Cadamuro, R. On.
    Science, 2015.
  2. The association between adolescent well-being and digital technology use (and online supplement).
    A. Orben and A. K. Przybylski.
    Nature Human Behaviour, 2019.

3/18: Mass collaboration

Main papers:

  1. Crowd-sourced Text Analysis: Reproducible and Agile Production of Political Data (and online supplement).
    K. Benoit, D. Conway, B. E. Lauderdale, M. Laver, and S. Mikhaylov.
    American Political Science Review, 2016.
  2. Measuring the predictability of life outcomes with a scientific mass collaboration.
    M. Salganik, I. Lundberg, A. Kindel, et al.
    PNAS, 2020.

3/25: Ethics in computational social science

Main papers:

  1. Experimental Evidence of Massive-scale Emotional Contagion through Social Networks (see also the Editorial expression of concern).
    A.D.I. Kramer, J.E. Guillory, and J.T. Hancock.
    PNAS, 2014.
  2. Critical questions for big data: Provocations for a cultural, technological, and scholarly phenomenon.
    d. boyd and K. Crawford.
    Information, Communication & Society, 2012.

4/1 and 4/8: Final project presentations

Students will present their final projects.



Writing Reviews

The main point of this class is to engage with important and cutting-edge research at the interface of computer science and the social sciences. This involves reading, reviewing, discussing, and presenting papers. Reviewing papers for CSC2552 will help you develop your reviewing and critical thinking skills, as well as prepare you for in-class discussions. In what follows, I've written some thoughts on how to write a good review. Every week that we have a discussion class, your paper reviews will be due on Wednesday at 9pm before class.

Your reviews in this class are meant to be similar to real reviews that you would write in a peer-review process, but they will be somewhat different. When you are reviewing for a real conference or journal, the main point of the review is to determine if it should or should not be published in that venue. In this class, we will be reviewing established published papers, so the main point will be to think critically about them. There will be less emphasis (but not no emphasis) on the soundness of the results, and more emphasis on the decisions the researchers made, the interpretation of the results, and the implications of the research.

As part of your review, provide a concise (1-2 sentence) summary of the paper. What is the main result? This demonstrates that you understood the high-level point of the work, and it's often a useful exercise to circumscribe the domain the paper is exploring.

List the strengths and weaknesses of the paper. These should be major pros and cons, not little nitpicks. It's extremely rare that a paper doesn't have both several strengths and several weaknesses. Research is difficult, and there are almost always tradeoffs. What are the compromises the authors made? What is good about their approach, and what is bad?

Then the main part of the review will be a discussion and response to the work. What exactly have the authors shown? Do you agree with their interpretation of the results? Do the strengths of their approach justify the weaknesses? Should the authors have done anything differently, in your opinion? What else should they have done? What are the implications of the results? How does this research inform or compliment other work in computational social science, or society in general?

Reviews should be concise. Papers are "big" things; they represent an entire research project that a group of people have spent significant time pursuing. Resist the temptation to address every detail, or go off on a small tangent. Keep to the main and most important points. Reviews should not be longer than 500 words.

Projects

One of the main goals of CSC2552 is to introduce you to research in computational social science. The best way to do that is to get your hands dirty and try to study something yourself, and the final project offers you an opportunity to do exactly this.

The project has two main deliverables: the project proposal and the final report. Students will present each to the class (proposals on March 4 and final reports on April 1 and April 8). Projects will be done in teams of one or two people.

Your first task is to pick a project topic. If you are looking for project ideas, please email me, and I'd be happy to brainstorm and suggest some project ideas. Also check this resources page.

Project proposal (2 pages). Your proposal should outline the motivation for your project, the realistic research question you wish to answer, and your chosen operationalization. Articulate these as crisply as you can. Next, survey a bit of related work (around 1 paragraph). The proposal should identify the CSS methodology you will use and lay out a plan for your project. How exactly do you plan to pursue your research questions? What data/methods will you use? You should provide a concrete proposal for a data analysis, experiment, survey, or other method that helps you answer your research question. The analysis plan will probably be the biggest section, at least 3/4 page. Finally, mention any of the steps you have taken so far and share your preliminary progress. Ideally, the first 2 pages of your final report will be quite similar to this proposal.

Proposal components:



Acknowledgments

Thanks to Sharad Goel, Jon Kleinberg, Jure Leskovec, Matt Salganik, Johan Ugander, and Bob West for course advice and inspiration.