The Weekly Newsletter of MIT Linguistics

24.S96, Fall 2022: “Methods in Computational Linguistics”

Instructor: Forrest Davis
Tuesday 10-1; 32-D461, 




Current models in natural language processing are trained on large amounts of text with a simple objective: predict the next word (or predict a word in context). These models have garnered a lot of attention, and there are claims that they can learn non-trivial aspects of human linguistic knowledge. A growing body of literature, framed as “model interpretability”, has attempted to address what exactly such computational models know about linguistic structure. Exploration of these linguistically “naive” models can be used to clarify claims about the nature (and origin) of linguistic knowledge.


In this course, we will survey papers and methods in computational linguistics and natural language processing with an aim towards understanding five key approaches to evaluating neural network models:


  • Targeted syntactic evaluations

  • Representational probing

  • Direct comparison to human behavioral measures

  • Priming/fine-tuning

  • Cross-linguistic comparison


The course is intended to be a hands-on experience. We will follow a cyclic pattern. First a model will be introduced with a particular approach to evaluation, with students implementing core aspects of both. Then, student led presentations will explore replications, extensions, and challenges to the existing empirical results, broadening our understanding of how to use and evaluate neural models, and how these findings may relate to a theory of language.

A tentative syllabus is attached. The first classes will introduce students to the relevant computational tools, so no background in computer science or machine learning is assumed.