We need to perform tokenization in order to build the Word2Vec models. The context of a sentence (document) is determined by the words in it. Word2Vec models require words rather than sentences (documents) to feed in, so we need to break the sentence into atomic units and create a token each time a white space is hit. DL4J has a tokenizer factory that is responsible for creating the tokenizer. The TokenizerFactory generates a tokenizer for the given string. In this recipe, we will tokenize the text data and train the Word2Vec model on top of them.
- Title Page
- Copyright and Credits
- Dedication
- About Packt
- Contributors
- Preface
- Introduction to Deep Learning in Java
- Technical requirements
- Deep learning intuition
- Determining the right network type to solve deep learning problems
- Determining the right activation function
- Combating overfitting problems
- Determining the right batch size and learning rates
- Configuring Maven for DL4J
- Configuring DL4J for a GPU-accelerated environment
- Troubleshooting installation issues
- Data Extraction, Transformation, and Loading
- Building Deep Neural Networks for Binary Classification
- Technical requirements
- Extracting data from CSV input
- Removing anomalies from the data
- Applying transformations to the data
- Designing input layers for the neural network model
- Designing hidden layers for the neural network model
- Designing output layers for the neural network model
- Training and evaluating the neural network model for CSV data
- Deploying the neural network model and using it as an API
- Building Convolutional Neural Networks
- Technical requirements
- Extracting images from disk
- Creating image variations for training data
- Image preprocessing and the design of input layers
- Constructing hidden layers for a CNN
- Constructing output layers for output classification
- Training images and evaluating CNN output
- Creating an API endpoint for the image classifier
- Implementing Natural Language Processing
- Technical requirements
- Reading and loading text data
- Tokenizing data and training the model
- Evaluating the model
- Generating plots from the model
- Saving and reloading the model
- Importing Google News vectors
- Troubleshooting and tuning Word2Vec models
- Using Word2Vec for sentence classification using CNNs
- Using Doc2Vec for document classification
- Constructing an LSTM Network for Time Series
- Constructing an LSTM Neural Network for Sequence Classification
- Performing Anomaly Detection on Unsupervised Data
- Using RL4J for Reinforcement Learning
- Developing Applications in a Distributed Environment
- Technical requirements
- Setting up DL4J and the required dependencies
- Creating an uber-JAR for training
- CPU/GPU-specific configuration for training
- Memory settings and garbage collection for Spark
- Configuring encoding thresholds
- Performing a distributed test set evaluation
- Saving and loading trained neural network models
- Performing distributed inference
- Applying Transfer Learning to Network Models
- Benchmarking and Neural Network Optimization
- Other Books You May Enjoy
Tokenizing data and training the model
-
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.