Lecture 1: Quick review of Maximum Likelihood, Bayesian inference, torch_coin_manual.py.

Lecture 2: Bayesian inference (and unicorns). Code: coin_bayes.py

Lecture 3.1 Generative models

Lecture 3: Causal inference

Videos:

• Lecture 1
• Lecture 2
• Lecture 3

Reading:

• Maximum Likelihood: MacKay Ch. 22.
• Bayesian Inference: MacKay Ch. 3.
• Generative models: Bishop Ch. 4.2, Bishop Ch. 8.1-8.2.
• Causal inference Shalizi Ch. 18-20 (18 repeats stuff from Bishop). Or Ch. 5 of the Fair ML book (less technical than Shalizi, and doesn't quite tell you everything you need to know, but an easier read so maybe a good start).

• Just for fun: Google study on productivity gains from code completion

Slides: Intro to fairness in ML. Zoom recording

Reading:

• Angwin et al, Machine Bias (Equivant response, Propublica's response to the response)
• Corbett-Davies et al, https://5harad.com/papers/fair-ml.pdf
• COMPAS data
• Fairnes and Machine Learning book for more details on everything.

Lecture 1a: causal.py, explain_away.py

Lecture 1b: Fairness and causality.

Lecture 2: Causality and ML

Zoom recording (notes coming up)

Reading: Kusner et al., Counterfactual Fairness, NeuriPS 2017

Reading: Mitchell at al., Model Cards for Model Reporting, FAT* 19

Reading: Shoelkopf et al., Toward Causal Representation Learning, Proc. of the IEEE, 2021

Video: Yoshua Bengio, Towards Causal Representation Learning

Lectures 1, 2: Causality and ML, continued

Lecture 3: Word embeddings, subword models from the Transformers lecture

Zoom recording

Reading: continue reading last week's readings on causaltion

Reading: We mentioned in lecture that PCA can be seen as finding the basis vectors that minimize the reconstruction error. Also see Slides 1-10 here

Reading: Notes on Word2vec and GLoVe. The original GloVe paper, original Word2vec paper (not super-readable), an explanation of the original Word2vec paper

Reading: Byte-pair encoding

Just for fun: Why Athletes' Birthdays Affect Who Goes Pro — And Who Becomes A Star

Just for fun: Publication bias

Lecture: Transformers. Zoom recording.

Reading: The Illustrated Transformer

Lecture: Transformers, cont'd. Capabilities of transformers. Zoom recording

Code: MinGPT

Reading:

• Rogers et al., A Primer in BERTology: What We Know About How BERT Works, Transactions of the Association for Computational Linguistics, 2020
• Hewitt and Manning, A Structural Probe for Finding Syntax in Word Representations, Proc. NAACL-HLT 2019
• Ettinger, What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models, Transactions of the Association for Computational Linguistics, 2020
• Petroni et al., Language Models as Knoweldge Bases?, EMNLP 2019
• Bender and Koller, Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data, ACL 2020
• Aguera y Arcas, Do large language models understand us?, medium.com, 2021
• Good and readable summary of arguments around Koller and Bender:Juian Michael, To Dissect an Octopus: Making Sense of the Form/Meaning Debate, blog post, 2020

Lecture: More details on PyTorch. GANs. Zoom recording

Reading:

• Deep Learning Ch. 20.10.3-20.10-4
• Goodfellow et al., Generative Adversarial Nets, NeurIPS 2014

  Readable summary: Weng, From GAN to WGAN, blog post, 2017

GAN implementation

Zoom recording

Reading:

• Deep Learning Ch. 20.10.3-20.10-4
• Goodfellow et al., Generative Adversarial Nets, NeurIPS 2014

  Readable summary: Weng, From GAN to WGAN, blog post, 2017

WGANs continued.

Zoom recording

Reading:

• Kingma and Welling, Auto-Encoding Variational Bayes, ICLR 2013
• Arjovsky et al, Wasserstein Generative Adversarial Networks, ICML 2017

  Readable summary: Weng, From GAN to WGAN, blog post, 2017

Handout: working with distances between distributions

WGANs continued.

Zoom recording

Variational autoencoders.

Variational Autoencoders, cont'd

AI Ethics

Zoom recording

Readable summary: Weng, From Autoencoder to Beta-VAE: the VAE section, blog post, 2018

How ConvNets see, adversarial examples. Zoom recording

Grokking, scaling, and all that

Metalearning (not on the exam)

Zoom recording