In this subsection, we represent a semi-automated technique of TM using Spark. Using other options as defaults, we train LDA on the dataset downloaded from GitHub at https://github.com/minghui/Twitter-LDA/tree/master/data/Data4Model/test. However, we will use more well-known text datasets in the model reuse and deployment phase later in this chapter.
Topic modeling with Spark MLlib and Stanford NLP
by Md. Rezaul Karim
Scala Machine Learning Projects
- Preface
- Analyzing Insurance Severity Claims
- Machine learning and learning workflow
- Hyperparameter tuning and cross-validation
- Analyzing and predicting insurance severity claims
- LR for predicting insurance severity claims
- GBT regressor for predicting insurance severity claims
- Boosting the performance using random forest regressor
- Comparative analysis and model deployment
- Summary
- Analyzing and Predicting Telecommunication Churn
- High Frequency Bitcoin Price Prediction from Historical and Live Data
- Population-Scale Clustering and Ethnicity Prediction
- Topic Modeling - A Better Insight into Large-Scale Texts
- Topic modeling and text clustering
- Topic modeling with Spark MLlib and Stanford NLP
- Implementation
- Step 1 - Creating a Spark session
- Step 2 - Creating vocabulary and tokens count to train the LDA after text pre-processing
- Step 3 - Instantiate the LDA model before training
- Step 4 - Set the NLP optimizer
- Step 5 - Training the LDA model
- Step 6 - Prepare the topics of interest
- Step 7 - Topic modelling
- Step 8 - Measuring the likelihood of two documents
- Implementation
- Other topic models versus the scalability of LDA
- Deploying the trained LDA model
- Summary
- Developing Model-based Movie Recommendation Engines
- Recommendation system
- Spark-based movie recommendation systems
- Item-based collaborative filtering for movie similarity
- Model-based recommendation with Spark
- Data exploration
- Movie recommendation using ALS
- Step 1 - Import packages, load, parse, and explore the movie and rating dataset
- Step 2 - Register both DataFrames as temp tables to make querying easier
- Step 3 - Explore and query for related statistics
- Step 4 - Prepare training and test rating data and check the counts
- Step 5 - Prepare the data for building the recommendation model using ALS
- Step 6 - Build an ALS user product matrix
- Step 7 - Making predictions
- Step 8 - Evaluating the model
- Selecting and deploying the best model
- Summary
- Options Trading Using Q-learning and Scala Play Framework
- Clients Subscription Assessment for Bank Telemarketing using Deep Neural Networks
- Client subscription assessment through telemarketing
- Dataset description
- Installing and getting started with Apache Zeppelin
- Exploratory analysis of the dataset
- Label distribution
- Job distribution
- Marital distribution
- Education distribution
- Default distribution
- Housing distribution
- Loan distribution
- Contact distribution
- Month distribution
- Day distribution
- Previous outcome distribution
- Age feature
- Duration distribution
- Campaign distribution
- Pdays distribution
- Previous distribution
- emp_var_rate distributions
- cons_price_idx features
- cons_conf_idx distribution
- Euribor3m distribution
- nr_employed distribution
- Statistics of numeric features
- Implementing a client subscription assessment model
- Hyperparameter tuning and feature selection
- Summary
- Client subscription assessment through telemarketing
- Fraud Analytics Using Autoencoders and Anomaly Detection
- Outlier and anomaly detection
- Autoencoders and unsupervised learning
- Developing a fraud analytics model
- Description of the dataset and using linear models
- Problem description
- Preparing programming environment
- Step 1 - Loading required packages and libraries
- Step 2 - Creating a Spark session and importing implicits
- Step 3 - Loading and parsing input data
- Step 4 - Exploratory analysis of the input data
- Step 5 - Preparing the H2O DataFrame
- Step 6 - Unsupervised pre-training using autoencoder
- Step 7 - Dimensionality reduction with hidden layers
- Step 8 - Anomaly detection
- Step 9 - Pre-trained supervised model
- Step 10 - Model evaluation on the highly-imbalanced data
- Step 11 - Stopping the Spark session and H2O context
- Auxiliary classes and methods
- Hyperparameter tuning and feature selection
- Summary
- Human Activity Recognition using Recurrent Neural Networks
- Working with RNNs
- Human activity recognition using the LSTM model
- Implementing an LSTM model for HAR
- Step 1 - Importing necessary libraries and packages
- Step 2 - Creating MXNet context
- Step 3 - Loading and parsing the training and test set
- Step 4 - Exploratory analysis of the dataset
- Step 5 - Defining internal RNN structure and LSTM hyperparameters
- Step 6 - LSTM network construction
- Step 7 - Setting up an optimizer
- Step 8 - Training the LSTM network
- Step 9 - Evaluating the model
- Tuning LSTM hyperparameters and GRU
- Summary
- Image Classification using Convolutional Neural Networks
- Other Books You May Enjoy
Topic modeling with Spark MLlib and Stanford NLP
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.