Semantic decision/classification systems
Knowledge extraction systems
Information Extraction systems
Command systems
Dialog systems
Translation systems

Text
Speech
Multi-modal (primarily visual +)

Applications

Semantic Decision/classification systems

News article classification (profit/earnings article, terrorist attack article)

Sentiment detection (good/bad review; left/right political orientation)

Spam detection

Complaint classification

Knowledge extraction systems (Ontology Building)

Named entity extraction (persons, organizations, places, events)

Clustering terms into categories

Hierarchy construction (finding the term "fruit" for the "orange", "apple", "grape",... cluster )

Relation discovery

place/superplace
event time, temporal event/event relations
other part-whole relations (structural component, chemical constituent) [Medical, chemical weapon detection]

Information Search Systems

Database query

Bill Woods Moonrocks system (70s)
HP NL system (80s)
NLI Intellect System (Ginsparg, 90s)
Microsoft NL DB toolkit

Information Retrieval (IR) [find and display relevant information containers]

Document retrieval
Web search
Why this is not just simple document retrieval: Google
Text segmentation

Information Extraction (IE) [manipulate information into canonical forms]

Text Summarization
Find the most important paragraphs or sentences
Try to generate an abstract
Interacts with translation: Do this in funny languages!
Get help from the world! Marti Hearst: Find papers citing this paper. The text around the citation will often be a summary. Works best in science papers.
Document database query
Doc DB: Find the query-relevant paragraph or cluster of sentences (Litigation support)
Online help system query

Command-oriented Systems

Online desktop

Text editor

Unix shell

Robot [SRI's Flakey, with speech interface]

Simulated armed forces (user=commander)

Kitchen appliances

Dialog Systems

Intelligent Tutors (Language, Reading, Math, etc. )

Student Modeling

Behavior, skill, and preference modeling
Error analysis

Translation Systems

MT systems: Not very good, not very widely used, except perhaps on the Web ("Translate this page" button. Type "Poisson frit" to Google; Type "ryba smazona" to Google The translation, The original)

Translator's assistant: a resource for a human translator

Theoretical Applications

Theory-specification tool

Zellig-Harris Transformational Grammar and the Linguistic String Project (Transformational and Discourse Analysis Project, Sager, NYU String Grammar project)

Chomskyan Transformational Grammars and ATNs (Woods, Moonrock system)

Lexical Functional Grammar (LFG, Ron Kaplan, Xerox PARC system)

Generalized Phrase Structure Grammar (GPSG, HP system)

Head-Driven Phrase Structure Grammar (HPSG, HP system, Verbmobil English grammar at Stanford)

Theoretical modeling

Processing models

Syntax
Garden path models (Bever 1970, McDonald 1993 and Jurafsky 1996)
The horse raced past the barn fell.
The complex houses married and single students and their families.
The student forgot the solution was in the back of the book.
Parse preference models (attachment preferences modeled; Pereira 1985, models "minimal attachment" and "right association" of Kimball 1973)
"Shallow" parsing (finite-state) models of human processing (Church 1980, Ramshaw and Marcus 1995, Argammom, et al. 1998, Munoz et al. 1999, Chapter 10, our text)
Semantics
The astronomer married the star.
The movie director married a star.(Reder 1983, Uszkoreit 1990)
Speech recognition systems as crude phonological processing models

Language Acquisition models

Unsupervised learning systems

Segmentation (Michael Brent)

Derivational Morphology (Watkinson and Manandhar)

PDP models

Language change models (simulation)
Briscoe 2000: biased learning creates linguistic change

History

Foundational insights (40s and 50s)

McCulloch-Pitts neuron: a simplified computational model of a neuron
Shannon (1948): automata for language, incorporating probablistic models
Chomsky (1956): formal language theory
Sound spectrograph (Koenig et al. 1946): foundation for instrumental phonetics.
First machine speech recognizers (Bell Labs, Davis et al. 1952)
Shannon-Weaver information theory: Noisy channel model

Two camps (57-70)

AI
Beginnings of Artifical Intelligence as a field (John McCarthy, Marvin Minsky, Clause Shannon, Nathaniel Rochester)
Newell and Simon Logic Theorist and Problem Solver. computable models of reasoning and logic
Subjects speak aloud as they solve problems
Problem solving modeled with a production-system where the productions (or reasoning steps) correspond to the steps human reasoners took for those kinds of problems.
George Miller and Donald Broadbent: importing computational ideas into psychology.
Bayesian method and optical character recognition: Using probabilistic methods on recognition problems (see Chapter 5, history section, our text)
Zellig Harris project (mentioned above)
Brown Corpus

Four paradigms (70-83)

Stochastic: HMMs in speech recognition (Jelinek, Bahl and Mercer at IBM)
Logic-based programming Prolog:
Q-systems aand metamorphosis grammars (Colmerauer)
Prolog, Definite Clause Grammars (Pereira and Warren 1980)
Unification grammar (Kay, Bresnan and Kaplan)
Natural language understanding(Winograd, serious attention to semantics)
Winograd SHRDLU, blocks world
Yale school: Scripts, plans, goals (Schank and Abelson, Wilensky, Lehnert). Story and text understanding.
Discourse-modeling (Grosz, Sidner, Perrault, Allen, Cohen). Discourse as plans guided by intentions and beliefs. Communicative acts as steps in plans.

Empiricism and Finite-state models redux (83-99)

Finite-state models
phonology and morphology (Kaplan and Kay 1981, Koskenniemi, Karttunen)
syntax (Church 1980)
Probabilistic models
Speech recognition work at IBM
Part of speech tagging (history section, chapter 8)
utter, direct: adj, v
walk, pilot, sneer, help: N,V
hard: adj, adv
Probabilistic parsing (history section, chapter 12)

The field comes together (94-99)

Spread of probabilistic methods to all kinds of problems
Commercial ventures using speech, some NLP
The web
Some lessened emphasis on theoretical work

Reasons for the expansion of Computational Linguistics in recent years

Success of Speech recognition

Increase in computing power and storage capacity of machines

Availability of online text and speech resources in unprecedented quantities

Success of statistical methods

World-wide web (applications and data)

Increasing demand for translation

Globalization of information, standards, software

Availability of venture capital (no longer so true!)