After the transformation process has been defined, we can execute it in a controlled pipeline. It can be executed using batch processing, or we can distribute the effort to a Spark cluster. Previously, we look at TransformProcessRecordReader, which automatically does the transformation in the background. We cannot feed and execute the data if the dataset is huge. Effort can be distributed to a Spark cluster for a larger dataset. You can also perform regular local execution. In this recipe, we will discuss how to execute a transform process locally as well as remotely.
- Title Page
- Copyright and Credits
- Dedication
- About Packt
- Contributors
- Preface
- Introduction to Deep Learning in Java
- Technical requirements
- Deep learning intuition
- Determining the right network type to solve deep learning problems
- Determining the right activation function
- Combating overfitting problems
- Determining the right batch size and learning rates
- Configuring Maven for DL4J
- Configuring DL4J for a GPU-accelerated environment
- Troubleshooting installation issues
- Data Extraction, Transformation, and Loading
- Building Deep Neural Networks for Binary Classification
- Technical requirements
- Extracting data from CSV input
- Removing anomalies from the data
- Applying transformations to the data
- Designing input layers for the neural network model
- Designing hidden layers for the neural network model
- Designing output layers for the neural network model
- Training and evaluating the neural network model for CSV data
- Deploying the neural network model and using it as an API
- Building Convolutional Neural Networks
- Technical requirements
- Extracting images from disk
- Creating image variations for training data
- Image preprocessing and the design of input layers
- Constructing hidden layers for a CNN
- Constructing output layers for output classification
- Training images and evaluating CNN output
- Creating an API endpoint for the image classifier
- Implementing Natural Language Processing
- Technical requirements
- Reading and loading text data
- Tokenizing data and training the model
- Evaluating the model
- Generating plots from the model
- Saving and reloading the model
- Importing Google News vectors
- Troubleshooting and tuning Word2Vec models
- Using Word2Vec for sentence classification using CNNs
- Using Doc2Vec for document classification
- Constructing an LSTM Network for Time Series
- Constructing an LSTM Neural Network for Sequence Classification
- Performing Anomaly Detection on Unsupervised Data
- Using RL4J for Reinforcement Learning
- Developing Applications in a Distributed Environment
- Technical requirements
- Setting up DL4J and the required dependencies
- Creating an uber-JAR for training
- CPU/GPU-specific configuration for training
- Memory settings and garbage collection for Spark
- Configuring encoding thresholds
- Performing a distributed test set evaluation
- Saving and loading trained neural network models
- Performing distributed inference
- Applying Transfer Learning to Network Models
- Benchmarking and Neural Network Optimization
- Other Books You May Enjoy
Executing a transform process
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.