How to do it...

  1. Create a sentence iterator using BasicLineIterator:
File file = new File("raw_sentences.txt");
SentenceIterator iterator = new BasicLineIterator(file);

For an example, go to https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/05_Implementing_NLP/sourceCode/cookbookapp/src/main/java/com/javadeeplearningcookbook/examples/BasicLineIteratorExample.java.

  1. Create a sentence iterator using LineSentenceIterator:
File file = new File("raw_sentences.txt");
SentenceIterator iterator = new LineSentenceIterator(file);

For an example, go to https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/05_Implementing_NLP/sourceCode/cookbookapp/src/main/java/com/javadeeplearningcookbook/examples/LineSentenceIteratorExample.java.

  1. Create a sentence iterator using CollectionSentenceIterator:
List<String> sentences= Arrays.asList("sample text", "sample text", "sample text");
SentenceIterator iter = new CollectionSentenceIterator(sentences);

For an example, go to https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/05_Implementing_NLP/sourceCode/cookbookapp/src/main/java/com/javadeeplearningcookbook/examples/CollectionSentenceIteratorExample.java.

  1. Create a sentence iterator using FileSentenceIterator:
SentenceIterator iter = new FileSentenceIterator(new File("/home/downloads/sentences.txt"));

For an example, go to https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/05_Implementing_NLP/sourceCode/cookbookapp/src/main/java/com/javadeeplearningcookbook/examples/FileSentenceIteratorExample.java.

  1. Create a sentence iterator using UimaSentenceIterator.

Add the following Maven dependency:

<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-nlp-uima</artifactId>
<version>1.0.0-beta3</version>
</dependency>

Then use the iterator, as shown here:

SentenceIterator iterator = UimaSentenceIterator.create("path/to/your/text/documents"); 

You can also use it like this:

SentenceIterator iter = UimaSentenceIterator.create("path/to/your/text/documents");

For an example, go to https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/05_Implementing_NLP/sourceCode/cookbookapp/src/main/java/com/javadeeplearningcookbook/examples/UimaSentenceIteratorExample.java.

  1. Apply the preprocessor to the text corpus:
iterator.setPreProcessor(new SentencePreProcessor() {
@Override
public String preProcess(String sentence) {
return sentence.toLowerCase();
}
});

For an example, go to https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/05_Implementing_NLP/sourceCode/cookbookapp/src/main/java/com/javadeeplearningcookbook/examples/SentenceDataPreProcessor.java.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset