5. Naïve Bayes Classifiers
by Patrick R. Nicolas, Arun Manivannan, Pascal Bugnion
Scala: Guide for Data Science Professionals
- Scala: Guide for Data Science Professionals
- Table of Contents
- Scala: Guide for Data Science Professionals
- Scala: Guide for Data Science Professionals
- Credits
- Preface
- 1. Module 1
- 1. Scala and Data Science
- 2. Manipulating Data with Breeze
- Code examples
- Installing Breeze
- Getting help on Breeze
- Basic Breeze data types
- Vectors
- Dense and sparse vectors and the vector trait
- Matrices
- Building vectors and matrices
- Advanced indexing and slicing
- Mutating vectors and matrices
- Matrix multiplication, transposition, and the orientation of vectors
- Data preprocessing and feature engineering
- Breeze – function optimization
- Numerical derivatives
- Regularization
- An example – logistic regression
- Towards re-usable code
- Alternatives to Breeze
- Summary
- References
- 3. Plotting with breeze-viz
- 4. Parallel Collections and Futures
- 5. Scala and SQL through JDBC
- Interacting with JDBC
- First steps with JDBC
- JDBC summary
- Functional wrappers for JDBC
- Safer JDBC connections with the loan pattern
- Enriching JDBC statements with the "pimp my library" pattern
- Wrapping result sets in a stream
- Looser coupling with type classes
- Creating a data access layer
- Summary
- References
- 6. Slick – A Functional Interface for SQL
- 7. Web APIs
- 8. Scala and MongoDB
- 9. Concurrency with Akka
- GitHub follower graph
- Actors as people
- Hello world with Akka
- Case classes as messages
- Actor construction
- Anatomy of an actor
- Follower network crawler
- Fetcher actors
- Routing
- Message passing between actors
- Queue control and the pull pattern
- Accessing the sender of a message
- Stateful actors
- Follower network crawler
- Fault tolerance
- Custom supervisor strategies
- Life-cycle hooks
- What we have not talked about
- Summary
- References
- 10. Distributed Batch Processing with Spark
- 11. Spark SQL and DataFrames
- DataFrames – a whirlwind introduction
- Aggregation operations
- Joining DataFrames together
- Custom functions on DataFrames
- DataFrame immutability and persistence
- SQL statements on DataFrames
- Complex data types – arrays, maps, and structs
- Interacting with data sources
- Standalone programs
- Summary
- References
- 12. Distributed Machine Learning with MLlib
- 13. Web APIs with Play
- Client-server applications
- Introduction to web frameworks
- Model-View-Controller architecture
- Single page applications
- Building an application
- The Play framework
- Dynamic routing
- Actions
- Interacting with JSON
- Querying external APIs and consuming JSON
- Creating APIs with Play: a summary
- Rest APIs: best practice
- Summary
- References
- 14. Visualization with D3 and the Play Framework
- A. Pattern Matching and Extractors
- II. Module 2
- 1. Getting Started with Breeze
- Introduction
- Getting Breeze – the linear algebra library
- Working with vectors
- Getting ready
- How to do it...
- Creating vectors
- Constructing a vector from values
- Creating a vector out of a function
- Creating a vector of linearly spaced values
- Creating a vector with values in a specific range
- Creating an entire vector with a single value
- Slicing a sub-vector from a bigger vector
- Creating a Breeze Vector from a Scala Vector
- Vector arithmetic
- Scalar operations
- Calculating the dot product of two vectors
- Creating a new vector by adding two vectors together
- Appending vectors and converting a vector of one type to another
- Concatenating two vectors
- Standard deviation
- Find the largest value in a vector
- Finding the sum, square root and log of all the values in the vector
- Working with matrices
- Vectors and matrices with randomly distributed values
- How it works...
- Creating vectors with uniformly distributed random values
- Creating vectors with normally distributed random values
- Creating vectors with random values that have a Poisson distribution
- Creating a matrix with uniformly random values
- Creating a matrix with normally distributed random values
- Creating a matrix with random values that has a Poisson distribution
- How it works...
- Reading and writing CSV files
- 2. Getting Started with Apache Spark DataFrames
- Introduction
- Getting Apache Spark
- Creating a DataFrame from CSV
- Manipulating DataFrames
- Creating a DataFrame from Scala case classes
- 3. Loading and Preparing Data – DataFrame
- 4. Data Visualization
- 5. Learning from Data
- Introduction
- Supervised and unsupervised learning
- Gradient descent
- Predicting continuous values using linear regression
- Binary classification using LogisticRegression and SVM
- Binary classification using LogisticRegression with Pipeline API
- How to do it...
- Importing and splitting data as test and training sets
- Construct the participants of the Pipeline
- Preparing a pipeline and training a model
- Predicting against test data
- Evaluating a model without cross-validation
- Constructing parameters for cross-validation
- Constructing cross-validator and fit the best model
- Evaluating the model with cross-validation
- How to do it...
- Clustering using K-means
- Feature reduction using principal component analysis
- How to do it...
- Dimensionality reduction of data for supervised learning
- Mean-normalizing the training data
- Extracting the principal components
- Preparing the labeled data
- Preparing the test data
- Classify and evaluate the metrics
- Dimensionality reduction of data for unsupervised learning
- Mean-normalizing the training data
- Extracting the principal components
- Arriving at the number of components
- Evaluating the metrics
- How to do it...
- 6. Scaling Up
- 7. Going Further
- 1. Getting Started with Breeze
- III. Module 3
- 1. Getting Started
- 2. Hello World!
- 3. Data Preprocessing
- 4. Unsupervised Learning
- 5. Naïve Bayes Classifiers
- 6. Regression and Regularization
- 7. Sequential Data Models
- 8. Kernel Models and Support Vector Machines
- 9. Artificial Neural Networks
- Feed-forward neural networks (FFNN)
- The multilayer perceptron (MLP)
- The activation function
- The network architecture
- Software design
- Model definition
- Training cycle/epoch
- Training strategies and classification
- Evaluation
- Benefits and limitations
- Summary
- 10. Genetic Algorithms
- 11. Reinforcement Learning
- 12. Scalable Frameworks
- B. Basic Concepts
- C. Bibliography
- Index
This chapter introduces the most common and simple generative classifiers—Naïve Bayes. As a reminder, generative classifiers are supervised learning algorithms that attempt to fit a joint probability distribution, p(X,Y), of two events X and Y, representing two sets of observed and hidden (or latent) variables, x and y.
In this chapter, you will learn, and hopefully appreciate, the simplicity of the Naïve Bayes technique through a concrete example. Then, you will build a Naïve Bayes classifier to predict stock price movement, given some prior technical indicators in the analysis of financial markets.
Finally, you will apply Naïve Bayes to text mining by predicting stock prices, using financial news feed and press releases.
Let's start with a refresher course in basic statistics.
Given two events or observations, X and Y, the joint probability of X and Y is defined as 
. If the observations X and Y are not related, an assumption known as conditional independence, then p(X,Y) = p(X).p(Y). The conditional probability of event Y, given X, is defined as p(Y|X)=p(X,Y)/p(X).
These two definitions are quite simple. However, probabilistic reasoning can be difficult to read in the case of large numbers of variables and sequences of conditional probabilities. As a picture is worth a thousand words, researchers introduced graphical models to describe a probabilistic relation between random variables [5:1].
There are two categories of graphs, and therefore, graphical models:
- Directed graphs such as Bayesian networks
- Undirected graphs such as conditional random fields (refer to the Conditional random fields section in Chapter 7, Sequential Data Models)
Directed graphical models are directed acyclic graphs that have been introduced to:
- Provide a simple way to visualize a probabilistic model
- Describe the conditional dependence (or independence) between variables
- Represent statistical inference in terms of graphical manipulation
A Bayesian network is a directed graphical model defining a join probability over a set of variables [5:2].
The two join probabilities, p(X,Y) and p(X,Y,Z), can be graphically modeled using Bayesian networks, as follows:
Examples of probabilistic graphical models
The conditional probability p(Y|X) is represented by an arrow directed from the output (or symptoms) Y to the input (or cause) X. Elaborate models can be described as a large directed graph between variables.
Here is an example of a real-world Bayesian network; the functioning of a smoke detector:
- A fire may generate smoke.
- Smoke may trigger an alarm.
- A depleted battery may trigger an alarm.
- The alarm may alert the homeowner.
- The alarm may alert the fire department.
A Bayesian network for smoke detectors
This representation may be a bit counterintuitive, as the vertices are directed from the symptoms (or output) to the cause (or input). Directed graphical models are used in many different models, besides Bayesian networks [5:3].
The Naïve Bayes models are probabilistic models based on the Bayes's theorem under the assumption of features independence, as mentioned in the Generative models section in Chapter 1, Getting Started.
-
No Comment