Computational Linguistics Syllabus

Course Outline

Linguistics 581








Week 0 Wed Jan 22 Chapter 1 of Jurafsky and Martin (J&M) History of Computational Linguistics. Chapter 2 J&M [Blackboard], Finite-State Automata, Regular Expressions Read BOTH Chapter 2 and the extra file labeled "Finite-State Automata" Assignment 1, Regular expression Python demo script. Textbook Ch. 2 slides Non-deterministic automata, epsilon transitions. [Also see blackboard slides]    
Week 1 Wed Jan 29 Chapter 4: J&M. Language modeling (Ngram models). Word counting, frequency dictionaries, simple ngram models, the training corpus. Introduction to NLTK. Python releases, Demo Script for computing ngram counts, The Pollard assignment and smoothing assignment are due Feb. 11. Ngrams, Brief probability intro, Python & unicode. Peter Norvig on ngrams.  
Week 2 Wed Feb 05 Chapter 4: Section 4.1-4.5 of chapter 4. Practicalities. Sections 4.5.3 Chapter 4. Sections 4.4 and 4.5.1, 4.5.2 Chapter 4. Smoothing, Add-1 smoothing, Kneser-Ney smoothing. Unknown words.   Entropy and Cross-Entropy. (Entropy as expected information value; cross-entropy as an evaluation tool) Smoothing Lecture. Kneser-Ney Lecture.    
Week 3 Wed Feb 12 Chapter 4: Section 4.5 Kneser-Ney smoothing.        
Week 4 Wed Feb 19 Chapter 5: J&M. [Blackboard] Spelling Correction. Noisy Channel. Generative models. Noisy channel assignment zip file (contains Python notebook). Due Feb. 26. Regular expressions assignment solution script (Python).      
Week 5 Wed Feb 26 Chapter 6: J&M. [Blackboard] Naive Bayes Classification. More generative models. Chapter 7. [Blackboard] Logistic Regression Naive Bayes assignment (Due Mar. 4). Naive Bayes lecture.    
Week 6 Wed Mar 04 Chapter 7. [Blackboard] Logistic Regression Logistic Regression assignment (Due Mar. 11).      
Week 7 Wed Mar 11 Chapter 9: J&M. [Blackboard] Noisy channel model redux, Sequence models. HMMs.   Intro to HMMs, Forward/Backward Algorithm on Jason Eisner's ice cream HMM.    
Week 8 Wed Mar 18 Chapter 10: J&M [Blackboard] Word-class and part of speech tagging. Rule-based taggers, decoding with HMMs. Viterbi assignment (Hardcopy due Mar. 18 or Mar. 25). Practice Viterbi Problem, Practice Problem solution, Practice problem template. Lecture. Tagging slides HMM Taggers/HMM models.    
Week 9 Wed Mar 25   Brown tagging NB, Naive Bayes solution, Noisy channel assignment solution, Viterbi assignment part one solution, Viterbi assignment solution, Viterbi assignment calculations, Midterm 2020! Lecture: Review    
Week 10 Wed Apr 01 H'day H'day H'day H'day H'day
Week 11 Wed Apr 08 Chapter 11. J&M Context Free Grammars of English, Treebanks, [Jurafsky and Martin 3rd Ed, Blackboard] Grammar assignment (Due Apr. 15).      
Week 12 Wed Apr 15 Chapter 12 Parsing. [Jurafsky and Martin 3rd Ed, Blackboard] CKY algorithm (bottum up parsing with a chart), Parsing assignment (CKY) (Due Tues., April 22, Note: Part C, using probabilities is Due Tu, Apr 29. Solution for Grammar assignment. Jurafsky/ Martin Grammar chapter lecture (see blackboard), Tree structure: Main ideas. Jurafsky Martin parsing lecture (see blackboard).   td_parser-0.1:Python implementation of a recursive descent top down recognizer.
Week 13 Wed Apr 22 Chapter 12 Probabilistic Parsing. [Jurafsky and Martin 3rd Ed, Blackboard] probabilistic extension of CKY. Shift-reduce Dependency parsing, Deep learning parsers (RNNs) Parsing assignment (CKY) (Part C: Probabilities, Due Wed., May 6), Solution for Grammar assignment, CKY assignment solution      
Week 14 Wed Apr 29 Chapter 15. Vector Semantics [Jurafsky and Martin 3rd Ed, Blackboard] Solution for Parts A and B, Prob parsing assignment notes, Parsing assignment (CKY) (Part C: Probabilities, Due Wed., May 6), Word Vector Assignment, due Wed. May 6,      
Week 15 Wed May 06 Chapter 16. Word Embeddings [Jurafsky and Martin 3rd Ed, Blackboard] 2020 Compling final, Solution for Word Vector problem, Word vector calculation example. Word Vector practice NB, Solution to parsing problem, Part C. Solution for Parts A and B, Prob parsing assignment notes,   Last class day    
Week 16 Wed May 13 Chapter 8. Neural Nets [Jurafsky and Martin 3rd Ed, Blackboard] Quiz on Vector Semantics & Neural Nets.      
Week 17 Wed May 20          
Week 18 Wed May 27 Chapter 8, 12-13, 15, 16. Review        
Week 19 Wed Jun 03   2019 Final,      
Week 20 Wed Jun 10          
Week 21 Wed Jun 17          
Week 22 Wed Jun 24