How it works...

Out-of-memory errors are an indication that VM options need to be adjusted. How you adjust these parameters will depend on the RAM capacity of the hardware. For step 1, if you're using an IDE such as IntelliJ, you can provide the VM options using VM attributes such as -Xmx, -Xms, and so on. VM options can also be used from the command line. 

For example, to increase the maximum memory consumption to 8 GB, you will need to add the -Xmx8G VM argument to your IDE. 

To mitigate StackOverflowError mentioned in step 2, we need to delete the temporary files created under the project directory where our Java program is executed. These temporary files should look like the following:

With regard to step 3, if you observe that your Word2Vec model doesn't hold all the words from the raw text data, then you might be interested in increasing the layer size of the Word2Vec model. This layerSize is nothing but the output vector dimension or the feature space dimension. For example, we had layerSize of 100 in our code. This means that we can increase it to a larger value, say 200, as a workaround:

Word2Vec model = new Word2Vec.Builder()
.iterate(iterator)
.tokenizerFactory(tokenizerFactory)
.minWordFrequency(5)
.layerSize(200)
.seed(42)
.windowSize(5)
.build();

If you have a GPU-powered machine, you can use this to accelerate the Word2Vec training time. Just make sure that the dependencies for the DL4J and ND4J backend are added as usual. If the results still don't look right, then make sure there are no normalization issues.

Tasks such as wordsNearest() use normalized weights by default, and others require weights without normalization applied.

With regard to step 4, we can use the conventional approach. The weights matrix has the most memory consumption in Word2Vec. It is calculated as follows: 

NumberOfWords * NumberOfDimensions * 2 * DataType memory footprint

For example, if our Word2Vec model with 100,000 words uses long as the data type, and 100 dimensions, the memory footprint will be 100,000 * 100 * 2 * 8 (long data type size) = 160 MB RAM, just for the weights matrix. 

Note that DL4J UI will only provide a high-level overview of memory consumption. 

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset