How to do it...

  1. Create a tokenizer factory and set the token preprocessor:
TokenizerFactory tokenFactory = new DefaultTokenizerFactory();
tokenFactory.setTokenPreProcessor(new CommonPreprocessor());
  1. Add the tokenizer factory to the Word2Vec model configuration:
Word2Vec model = new Word2Vec.Builder()
.minWordFrequency(wordFrequency)
.layerSize(numFeatures)
.seed(seed)
.epochs(numEpochs)
.windowSize(windowSize)
.iterate(iterator)
.tokenizerFactory(tokenFactory)
.build();

  1. Train the Word2Vec model:
model.fit();
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset