What you need for this learning path
by Iti Mathur, Nisheeth Joshi, Deepti Chopra, Jacob Perkins, Nitin Hardeniya
Natural Language Processing: Python and NLTK
- Natural Language Processing: Python and NLTK
- Table of Contents
- Natural Language Processing: Python and NLTK
- Natural Language Processing: Python and NLTK
- Credits
- Preface
- 1. Module 1
- 1. Introduction to Natural Language Processing
- 2. Text Wrangling and Cleansing
- 3. Part of Speech Tagging
- 4. Parsing Structure in Text
- 5. NLP Applications
- Building your first NLP application
- Other NLP applications
- Summary
- 6. Text Classification
- 7. Web Crawling
- 8. Using NLTK with Other Python Libraries
- 9. Social Media Mining in Python
- 10. Text Mining at Scale
- 2. Module 2
- 1. Tokenizing Text and WordNet Basics
- Introduction
- Tokenizing text into sentences
- Tokenizing sentences into words
- Tokenizing sentences using regular expressions
- Training a sentence tokenizer
- Filtering stopwords in a tokenized sentence
- Looking up Synsets for a word in WordNet
- Looking up lemmas and synonyms in WordNet
- Calculating WordNet Synset similarity
- Discovering word collocations
- 2. Replacing and Correcting Words
- 3. Creating Custom Corpora
- Introduction
- Setting up a custom corpus
- Creating a wordlist corpus
- Creating a part-of-speech tagged word corpus
- Creating a chunked phrase corpus
- Creating a categorized text corpus
- Creating a categorized chunk corpus reader
- Lazy corpus loading
- Creating a custom corpus view
- Creating a MongoDB-backed corpus reader
- Corpus editing with file locking
- 4. Part-of-speech Tagging
- Introduction
- Default tagging
- Training a unigram part-of-speech tagger
- Combining taggers with backoff tagging
- Training and combining ngram taggers
- Creating a model of likely word tags
- Tagging with regular expressions
- Affix tagging
- Training a Brill tagger
- Training the TnT tagger
- Using WordNet for tagging
- Tagging proper names
- Classifier-based tagging
- Training a tagger with NLTK-Trainer
- 5. Extracting Chunks
- Introduction
- Chunking and chinking with regular expressions
- Merging and splitting chunks with regular expressions
- Expanding and removing chunks with regular expressions
- Partial parsing with regular expressions
- Training a tagger-based chunker
- Classification-based chunking
- Extracting named entities
- Extracting proper noun chunks
- Extracting location chunks
- Training a named entity chunker
- Training a chunker with NLTK-Trainer
- 6. Transforming Chunks and Trees
- Introduction
- Filtering insignificant words from a sentence
- Correcting verb forms
- Swapping verb phrases
- Swapping noun cardinals
- Swapping infinitive phrases
- Singularizing plural nouns
- Chaining chunk transformations
- Converting a chunk tree to text
- Flattening a deep tree
- Creating a shallow tree
- Converting tree labels
- 7. Text Classification
- Introduction
- Bag of words feature extraction
- Training a Naive Bayes classifier
- Training a decision tree classifier
- Training a maximum entropy classifier
- Training scikit-learn classifiers
- Measuring precision and recall of a classifier
- Calculating high information words
- Combining classifiers with voting
- Classifying with multiple binary classifiers
- Training a classifier with NLTK-Trainer
- 8. Distributed Processing and Handling Large Datasets
- Introduction
- Distributed tagging with execnet
- Distributed chunking with execnet
- Parallel list processing with execnet
- Storing a frequency distribution in Redis
- Storing a conditional frequency distribution in Redis
- Storing an ordered dictionary in Redis
- Distributed word scoring with Redis and execnet
- 9. Parsing Specific Data Types
- A. Penn Treebank Part-of-speech Tags
- 1. Tokenizing Text and WordNet Basics
- 3. Module 3
- 1. Working with Strings
- 2. Statistical Language Modeling
- Understanding word frequency
- Applying smoothing on the MLE model
- Develop a back-off mechanism for MLE
- Applying interpolation on data to get mix and match
- Evaluate a language model through perplexity
- Applying metropolis hastings in modeling languages
- Applying Gibbs sampling in language processing
- Summary
- 3. Morphology – Getting Our Feet Wet
- 4. Parts-of-Speech Tagging – Identifying Words
- 5. Parsing – Analyzing Training Data
- 6. Semantic Analysis – Meaning Matters
- 7. Sentiment Analysis – I Am Happy
- 8. Information Retrieval – Accessing Information
- 9. Discourse Analysis – Knowing Is Believing
- 10. Evaluation of NLP Systems – Analyzing Performance
- B. Bibliography
- Index
Module 1:
We need the following software for this module:
|
Chapter number |
Software required (with version) |
Free/Proprietary |
Download links to the software |
Hardware specifications |
OS required |
|---|---|---|---|---|---|
|
1-5 |
Python/Anaconda NLTK |
Free |
Common Unix Printing System |
any | |
|
6 |
scikit-learn and gensim |
Free |
Common Unix Printing System |
any | |
|
7 |
Scrapy |
Free |
Common Unix Printing System |
any | |
|
8 |
NumPy, SciPy, pandas, and matplotlib |
Free |
Common Unix Printing System |
any | |
|
9 |
Twitter Python APIs and Facebook python APIs |
Free |
Common Unix Printing System |
any |
Module 2:
You will need Python 3 and the listed Python packages. For this learning path, the author used Python 3.3.5. To install the packages, you can use pip (https://pypi.python.org/pypi/pip/). The following is the list of the packages in requirements format with the version number used while writing this learning path:
- NLTK>=3.0a4
- pyenchant>=1.6.5
- lockfile>=0.9.1
- numpy>=1.8.0
- scipy>=0.13.0
- scikit-learn>=0.14.1
- execnet>=1.1
- pymongo>=2.6.3
- redis>=2.8.0
- lxml>=3.2.3
- beautifulsoup4>=4.3.2
- python-dateutil>=2.0
- charade>=1.0.3
You will also need NLTK-Trainer, which is available at https://github.com/japerk/nltk-trainer.
Beyond Python, there are a couple recipes that use MongoDB and Redis, both NoSQL databases. These can be downloaded at http://www.mongodb.org/ and http://redis.io/, respectively.
Module 3:
For all the chapters, Python 2.7 or 3.2+ is used. NLTK 3.0 must be installed either on 32-bit machine or 64-bit machine. Operating System required is Windows/Mac/Unix.
-
No Comment