How to do it...

  1. Load the dataset into RecordReader. Load the CSV data in the case of CSVRecordReader:
RecordReader reader = new CSVRecordReader(0,',');
reader.initialize(new FileSplit(file));
  1. Execute the transforms in local using LocalTransformExecutor: 
List<List<Writable>> transformed = LocalTransformExecutor.execute(recordReader, transformProcess)
  1. Execute the transforms in Spark using SparkTransformExecutor: 
JavaRDD<List<Writable>> transformed = SparkTransformExecutor.execute(inputRdd, transformProcess)
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset