Computational Linguistics Syllabus

Course Outline

Linguistics 581

Week

Day

Reading

Assignment

Lecture

Background

Code

  Thu Jan 23 Chapter 1 of Jurafsky and Martin (J&M) History of Computational Linguistics. Chapter 2 J&M [Blackboard], Finite-State Automata, Regular Expressions Read BOTH Chapter 2 and the extra file labeled "Finite-State Automata" Assignment 1, Regular expression Python demo script. Textbook Ch. 2 slides Non-deterministic automata, epsilon transitions. [Also see blackboard slides]    
Week 1 Tue Jan 28 Chapter 4: J&M. Language modeling (Ngram models). Word counting, frequency dictionaries, simple ngram models, the training corpus. Introduction to NLTK. Python releases, Demo Script for computing ngram counts, The Pollard assignment and smoothing assignment are due Feb. 11. Ngrams, Brief probability intro, Python & unicode.    
  Thu Jan 30       Peter Norvig on ngrams.  
Week 2 Tue Feb 04 Chapter 4: Section 4.1-4.5 of chapter 4. Practicalities. Sections 4.5.3 Chapter 4. Sections 4.4 and 4.5.1, 4.5.2 Chapter 4. Smoothing, Add-1 smoothing, Kneser-Ney smoothing. Unknown words.   Entropy and Cross-Entropy. (Entropy as expected information value; cross-entropy as an evaluation tool) Smoothing Lecture. Kneser-Ney Lecture.    
  Thu Feb 06          
Week 3 Tue Feb 11 Chapter 4: Section 4.5 Kneser-Ney smoothing.        
  Thu Feb 13          
Week 4 Tue Feb 18 Chapter 5: J&M. [Blackboard] Spelling Correction. Noisy Channel. Generative models.        
  Thu Feb 20   Noisy channel assignment zip file (contains Python notebook). Due Feb. 27. Regular expressions assignment solution script (Python).      
Week 5 Tue Feb 25 Chapter 6: J&M. [Blackboard] Naive Bayes Classification. More generative models.        
  Thu Feb 27   Naive Bayes assignment (Due Mar. 5).      
Week 6 Tue Mar 03 Chapter 7. [Blackboard] Logistic Regression        
  Thu Mar 05   Logistic Regression assignment (Due Mar. 12).      
Week 7 Tue Mar 10 Chapter 9: J&M. [Blackboard] Noisy channel modeleduxs r, Sequence models. HMMs.   Intro to HMMs, Forward/Backward Algorithm on Jason Eisner's ice cream HMM.    
  Thu Mar 12     Naive Bayes lecture.    
Week 8 Tue Mar 17 Chapter 10: J&M [Blackboard] Word-class and part of speech tagging. Rule-based taggers, decoding with HMMs. Viterbi assignment (email due due Mar. 18 or Ma. 25 ), Practice Viterbi Problem, Practice Problem solution, Practice problem template, Solution to Noisy channel spelling problem. Lecture. Tagging slides HMM Taggers/HMM models.    
  Thu Mar 19          
Week 9 Tue Mar 24 Chapter 11. J&M Context Free Grammars of English, Treebanks, [Jurafsky and Martin 3rd Ed, Blackboard] Brown tagging NB (class today). Jurafsky/ Martin Grammar chapter lecture (see blackboard), Tree structure: Main ideas.    
  Thu Mar 26   Brown tagging NB, Naive Bayes solution, Noisy channel assignment solution, Viterbi assignment part one solution, Viterbi assignment answer, Viterbi assignment calculations, Midterm 2020!      
Week 10 Tue Mar 31 H'day H'day H'day H'day H'day
  Thu Apr 02 H'day H'day H'day H'day H'day
Week 11 Tue Apr 07          
  Thu Apr 09          
Week 12 Tue Apr 14 Chapter 12 Parsing. [Jurafsky and Martin 3rd Ed, Blackboard] CKY algorithm (bottum up parsing with a chart), Grammar assignment (Due Apr. 16). Jurafsky Martin parsing lecture (see blackboard).    
  Thu Apr 16         td_parser-0.1:Python implementation of a recursive descent top down recognizer.
Week 13 Tue Apr 21 Chapter 12 Probabilistic Parsing. [Jurafsky and Martin 3rd Ed, Blackboard] probabilistic extension of CKY. Shift-reduce Dependency parsing, Deep learning parsers (RNNs) Prob parsing lecture, also see Jurafsky and Martin prob parsing slides (see BlackBoard), Notes on Part C of parsing assignment,      
  Thu Apr 23   CKY assignment solution      
Week 14 Tue Apr 28     Prob parsing lecture.    
  Thu Apr 30   Parsing assignment (CKY) (Part C: Probabilities, Due Thurs., May 7), Word Vector Assignment, due Thurs. May 7, Solution for Grammar assignment, CKY assignment solution, Prob parsing assignment notes.      
Week 15 Tue May 05   Chapter 15. Vector Semantics [Jurafsky and Martin 3rd Ed, Blackboard] Quiz on Vector Semantics & Neural Nets.      
  Thu May 07   2020 Compling final, Solution for Word Vector problem, Word vector calculation example. Word Vector practice NB, Solution to parsing problem, Part C. Solution for Parts A and B, Prob parsing assignment notes,   Last class day    
Week 16 Tue May 12 Chapter 16. Word Embeddings [Jurafsky and Martin 3rd Ed, Blackboard]        
  Thu May 14 Chapter 9. Neural Nets. Sequence RNNs [Jurafsky and Martin 3rd Ed, Blackboard] Chapter 10. Encoder Decoder attention. Chapter 8, 12-13, 15, 16. Review        
Week 17 Tue May 19          
  Thu May 21   2019 Final,      
Week 18 Tue May 26          
  Thu May 28          
Week 19 Tue Jun 02