Performance considerations
by Alex Kozlov, Patrick R. Nicolas, Pascal Bugnion
Scala:Applied Machine Learning
- Scala:Applied Machine Learning
- Table of Contents
- Scala:Applied Machine Learning
- Scala:Applied Machine Learning
- Credits
- Preface
- I. Module 1
- 1. Scala and Data Science
- 2. Manipulating Data with Breeze
- Code examples
- Installing Breeze
- Getting help on Breeze
- Basic Breeze data types
- Vectors
- Dense and sparse vectors and the vector trait
- Matrices
- Building vectors and matrices
- Advanced indexing and slicing
- Mutating vectors and matrices
- Matrix multiplication, transposition, and the orientation of vectors
- Data preprocessing and feature engineering
- Breeze – function optimization
- Numerical derivatives
- Regularization
- An example – logistic regression
- Towards re-usable code
- Alternatives to Breeze
- Summary
- References
- 3. Plotting with breeze-viz
- 4. Parallel Collections and Futures
- 5. Scala and SQL through JDBC
- Interacting with JDBC
- First steps with JDBC
- JDBC summary
- Functional wrappers for JDBC
- Safer JDBC connections with the loan pattern
- Enriching JDBC statements with the "pimp my library" pattern
- Wrapping result sets in a stream
- Looser coupling with type classes
- Creating a data access layer
- Summary
- References
- 6. Slick – A Functional Interface for SQL
- 7. Web APIs
- 8. Scala and MongoDB
- 9. Concurrency with Akka
- GitHub follower graph
- Actors as people
- Hello world with Akka
- Case classes as messages
- Actor construction
- Anatomy of an actor
- Follower network crawler
- Fetcher actors
- Routing
- Message passing between actors
- Queue control and the pull pattern
- Accessing the sender of a message
- Stateful actors
- Follower network crawler
- Fault tolerance
- Custom supervisor strategies
- Life-cycle hooks
- What we have not talked about
- Summary
- References
- 10. Distributed Batch Processing with Spark
- 11. Spark SQL and DataFrames
- DataFrames – a whirlwind introduction
- Aggregation operations
- Joining DataFrames together
- Custom functions on DataFrames
- DataFrame immutability and persistence
- SQL statements on DataFrames
- Complex data types – arrays, maps, and structs
- Interacting with data sources
- Standalone programs
- Summary
- References
- 12. Distributed Machine Learning with MLlib
- 13. Web APIs with Play
- Client-server applications
- Introduction to web frameworks
- Model-View-Controller architecture
- Single page applications
- Building an application
- The Play framework
- Dynamic routing
- Actions
- Interacting with JSON
- Querying external APIs and consuming JSON
- Creating APIs with Play: a summary
- Rest APIs: best practice
- Summary
- References
- 14. Visualization with D3 and the Play Framework
- A. Pattern Matching and Extractors
- II. Module 2
- 1. Getting Started
- 2. Hello World!
- 3. Data Preprocessing
- 4. Unsupervised Learning
- Clustering
- K-means clustering
- The expectation-maximization algorithm
- Dimension reduction
- Performance considerations
- Summary
- Clustering
- 5. Naïve Bayes Classifiers
- 6. Regression and Regularization
- 7. Sequential Data Models
- 8. Kernel Models and Support Vector Machines
- 9. Artificial Neural Networks
- Feed-forward neural networks
- The multilayer perceptron
- Evaluation
- Convolution neural networks
- Benefits and limitations
- Summary
- 10. Genetic Algorithms
- 11. Reinforcement Learning
- 12. Scalable Frameworks
- A. Basic Concepts
- III. Module 3
- 1. Exploratory Data Analysis
- 2. Data Pipelines and Modeling
- 3. Working with Spark and MLlib
- 4. Supervised and Unsupervised Learning
- 5. Regression and Classification
- 6. Working with Unstructured Data
- 7. Working with Graph Algorithms
- 8. Integrating Scala with R and Python
- 9. NLP in Scala
- 10. Advanced Model Monitoring
- A. Bibliography
- Index
The three unsupervised learning techniques share the same limitation—a high computational complexity.
The K-means has the computational complexity of O(iKnm), where i is the number of iterations (or recursions), K is the number of clusters, n is the number of observations, and m is the number of features. Here are some remedies to the poor performance of the K-means algorithm:
- Reducing the average number of iterations by seeding the centroid using a technique such as initialization by ranking the variance of the initial cluster, as described in the beginning of this chapter
- Using a parallel implementation of K-means and leveraging a large-scale framework such as Hadoop or Spark
- Reducing the number of outliers and features by filtering out the noise with a smoothing algorithm such as a discrete Fourier transform or a Kalman filter
- Decreasing the dimensions of the model by following a two-step process:
- Execute a first pass with a smaller number of clusters K and/or a loose exit condition regarding the reassignment of data points. The data points close to each centroid are aggregated into a single observation.
- Execute a second pass on the aggregated observations.
The computational complexity of the expectation-maximization algorithm for each iteration (E + M steps) is O(m2n), where m is the number of hidden or latent variables and n is the number of observations.
A partial list of suggested performance improvement includes:
- Filtering of raw data to remove noise and outliers
- Using a sparse matrix on a large feature set to reduce the complexity of the covariance matrix, if possible
- Applying the Gaussian mixture model wherever possible—the assumption of Gaussian distribution simplifies the computation of the log likelihood
- Using a parallel data processing framework such as Apache Hadoop or Spark, as discussed in the Apache Spark section in Chapter 12, Scalable Frameworks
- Using a kernel function to reduce the estimate of covariance in the E-step
The computational complexity of the extraction of the principal components is O(m2n + n3), where m is the number of features and n is the number of observations. The first term represents the computational complexity for computing the covariance matrix. The second term reflects the computational complexity of the eigenvalue decomposition.
The list of potential performance improvements or alternative solutions for PCA includes the following:
- Assuming that the variance is Gaussian
- Using a sparse matrix to compute eigenvalues for problems with large feature sets and missing data
- Investigating alternatives to PCA to reduce the dimension of a model such as the discrete Fourier transform (DFT) or singular value decomposition (SVD) [4:17]
- Using PCA in conjunction with EM (at the research stage)
- Deploying a dataset on a parallel data processing framework such as Apache Hadoop or Spark, as described in the Apache Spark section in Chapter 12, Scalable Frameworks
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.