How to do it...

  1. Manage a range of records using FileSplit:
String[] allowedFormats=new String[]{".JPEG"};
FileSplit fileSplit = new FileSplit(new File("temp"), allowedFormats,true)
  1. Manage the URI collection from a file using CollectionInputSplit:
FileSplit fileSplit = new FileSplit(new File("temp"));
CollectionInputSplit collectionInputSplit = new CollectionInputSplit(fileSplit.locations());
  1. Use NumberedFileInputSplit to manage data with numbered file formats:
NumberedFileInputSplit numberedFileInputSplit = new NumberedFileInputSplit("numberedfiles/file%d.txt",1,4);
numberedFileInputSplit.locationsIterator().forEachRemaining(System.out::println);
  1. Use TransformSplit to map the input URIs to the different output URIs: 
TransformSplit.URITransform uriTransform = URI::normalize;

List<URI> uriList = Arrays.asList(new URI("file://storage/examples/./cats.txt"),
new URI("file://storage/examples//dogs.txt"),
new URI("file://storage/./examples/bear.txt"));

TransformSplit transformSplit = new TransformSplit(new CollectionInputSplit(uriList),uriTransform);
  1. Perform URI string replacement using TransformSplit:
InputSplit transformSplit = TransformSplit.ofSearchReplace(new CollectionInputSplit(inputFiles),"-in.csv","-out.csv");      
  1. Extract the CSV data for the neural network using CSVRecordReader:
RecordReader reader = new CSVRecordReader(numOfRowsToSkip,deLimiter);
recordReader.initialize(new FileSplit(file));
  1. Extract image data for the neural network using ImageRecordReader:
ImageRecordReader imageRecordReader = new ImageRecordReader(imageHeight,imageWidth,channels,parentPathLabelGenerator);
imageRecordReader.initialize(trainData,transform);

You can find the ImageRecordReader example at https://github.com/PacktPublishing/Java-Deep-Learning-Cookbook/blob/master/02_Data%20Extraction%2C%20Transform%20and%20Loading/sourceCode/cookbook-app/src/main/java/com/javadeeplearningcookbook/app/recordreaderexamples/ImageRecordReaderExample.java.

  1. Transform and extract the data using TransformProcessRecordReader:
RecordReader recordReader = new TransformProcessRecordReader(recordReader,transformProcess);
  1. Extract the sequence data using SequenceRecordReader and CodecRecordReader:
RecordReader codecReader = new CodecRecordReader();
codecReader.initialize(conf,split);

The following code shows how to use RegexSequenceRecordReader:

RecordReader recordReader = new RegexSequenceRecordReader((d{2}/d{2}/d{2}) (d{2}:d{2}:d{2}) ([A-Z]) (.*)",skipNumLines);
recordReader.initialize(new NumberedFileInputSplit(path/log%d.txt));

The following code shows how to use CSVSequenceRecordReader:

CSVSequenceRecordReader seqReader = new CSVSequenceRecordReader(skipNumLines, delimiter);
seqReader.initialize(new FileSplit(file));
  1. Extract the JSON/XML/YAML data using JacksonLineRecordReader:
RecordReader recordReader = new JacksonLineRecordReader(fieldSelection, new ObjectMapper(new JsonFactory()));
recordReader.initialize(new FileSplit(new File("json_file.txt")));
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset