How to do it...

  1. Start the Spark shell:
        $ spark-shell
  1. Perform the required imports:
        scala> import org.apache.spark.ml.classification.NaiveBayes
scala> import
org.apache.spark.ml.evaluation.MulticlassClassificationEvaluator
  1. Load the data into the DataFrame from S3:
        scala> val data = 
spark.read.format("libsvm").load("s3a://sparkcookbook/patientdata")
  1. Split the data into training and test datasets:
        scala> val Array(trainingData, testData) = 
data.randomSplit(Array(0.7, 0.3))
  1. Train the model with the training dataset:
        scala> val model = new NaiveBayes().fit(trainingData)
  1. Do the prediction:
        scala> val predictions = model.transform(testData)
  1. Evaluate the accuracy:
        scala> val evaluator = new MulticlassClassificationEvaluator()
.setMetricName("accuracy")
scala> val accuracy = evaluator.evaluate(predictions)

Here the accuracy is only 55 percent, which shows Naive Bayes is not the best algorithm for this dataset. 

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset