Computational Linguistics Syllabus

Course Outline

Linguistics 581

Week

Day

Reading

Assignment

Lecture

Background

Code
  Thu Jan 21 Chapter 1 of Jurafsky and Martin (J&M) History of Computational Linguistics. Chapter 2 J&M [Canvas], Finite-State Automata, Regular Expressions Read BOTH Chapter 2 and the extra file labeled "Finite-State Automata" Assignment 1, Regular expression Python demo script. Textbook Ch. 2 slides Non-deterministic automata, epsilon transitions, [Also see Canvas slides] Transitions demo notebook (Python FSA implementation]. Not runnable as Colab NB w/o first running install directions at Tal Yarkoni's github transitions page.    
Week 1 Tue Jan 26 Chapter 4: J&M. Language modeling (Ngram models). Word counting, frequency dictionaries, simple ngram models, the training corpus. Introduction to NLTK. Python Distributions, Demo Script for computing ngram counts, Github Google colab extension for Chrome, Pollard assignment, Smoothing assignment. Ngrams, Brief probability intro.    
  Thu Jan 28       Peter Norvig on ngrams.  
Week 2 Tue Feb 02 Chapter 4: Section 4.1-4.5 of chapter 4. Practicalities. Sections 4.5.3 Chapter 4. Sections 4.4 and 4.5.1, 4.5.2 Chapter 4. Smoothing, Add-1 smoothing, Kneser-Ney smoothing. Unknown words. Assignment One due! Entropy and Cross-Entropy. (Entropy as expected information value; cross-entropy as an evaluation tool) Smoothing Lecture. Kneser-Ney Lecture.    
  Thu Feb 04          
Week 3 Tue Feb 09 Chapter 4: Section 4.5 Kneser-Ney smoothing. Pollard assignment and smoothing assignment due. Smoothing, Pollard.      
  Thu Feb 11          
Week 4 Tue Feb 16 Chapter 5: J&M. [Canvas] Spelling Correction. Noisy Channel. Generative models. Smoothing assignment solution. , Pollard assignment solution.      
  Thu Feb 18          
Week 5 Tue Feb 23 Chapter 6: J&M. [Canvas] Naive Bayes Classification. More generative models. Noisy channel assignment Google Colab notebook. Due Feb. 23.      
  Thu Feb 25   Word senses. Computing word senses. Naive Bayes assignment: Disambiguating a word.      
Week 6 Tue Mar 02 Chapter 7. [Canvas] Logistic Regression        
  Thu Mar 04          
Week 7 Tue Mar 09   Logistic Regression Notebook (Not an assignment; does not need turning in).      
  Thu Mar 11   Assignments: Naive Bayes assignment (Due Mar. 11).      
Week 8 Tue Mar 16 Chapter 9: J&M. [Canvas] Noisy channel models, Sequence models. HMMs. Assignments: Naive Bayes solution, Noisy Channel assignment solution. Intro to HMMs, Forward/Backward Algorithm on Jason Eisner's ice cream HMM.    
  Thu Mar 18   Assignments: Midterm 2020! Midterm 2020 solution, Midterm 2021. (This is it!)      
Week 9 Tue Mar 23 Chapter 11. J&M Context Free Grammars of English, Treebanks, [Jurafsky and Martin 3rd Ed, Canvas]   Jurafsky/ Martin Grammar chapter lecture (see Canvas), Tree structure: Main ideas.    
  Thu Mar 25   Assignments: Midterm due Mar. 25 (as a Google Colab notebook). Turn in on Canvas.      
Week 10 Tue Mar 30 H'day H'day H'day H'day H'day
  Thu Apr 01 Chapter 12 Parsing. [Jurafsky and Martin 3rd Ed, Canvas] CKY algorithm (bottum up parsing with a chart),   Prob parsing lecture, also see Jurafsky and Martin prob parsing slides (see Canvas),    
Week 11 Tue Apr 06   Viterbi assignment (due on Canvas Apr. 6 ), Practice Viterbi Problem, Practice Problem solution, Practice problem template. Grammar assignment (Due Apr. 13).     td_parser-0.1:Python implementation of a recursive descent top down recognizer.
  Thu Apr 08 Chapter 13 Probabilistic Parsing. [Jurafsky and Martin 3rd Ed, Canvas] probabilistic extension of CKY. Shift-reduce Dependency parsing, Deep learning parsers (RNNs)   Prob parsing lecture.    
Week 12 Tue Apr 13          
  Thu Apr 15 H'day H'day H'day H'day H'day
Week 13 Tue Apr 20   Assignment: Parsing assignment (Due Apr. 20) Solution to Midterm. Midterm interpretation. Grammar assignment solution.      
  Thu Apr 22   Notes on probability parsing assignment, CKY assignment solution, Viterbi assignment solution.      
Week 14 Tue Apr 27   Chapter 15. Vector Semantics [Jurafsky and Martin 3rd Ed, Canvas] Quiz on Vector Semantics & Neural Nets.      
  Thu Apr 29   Word vector calculation example. Word Vector practice NB,      
Week 15 Tue May 04 Chapter 16. Word Embeddings [Jurafsky and Martin 3rd Ed, Canvas] Final review materials, 2020 Final. Practice Viterbi Problem, Practice Problem solution.      
  Thu May 06 Chapter 8, 12-13, 15, 16. Review Probability Parsing assignment, Due Thurs., May 6), Notes on probability parsing assignment, Word Vector Assignment, due Thurs. May 6.   Last class day    
Week 16 Tue May 11          
  Thu May 13          
Week 17 Tue May 18          
  Thu May 20          
Week 18 Tue May 25