There's more...

You can use Kryo by doing the following settings in your SparkConf:

  1. Start the Spark shell by setting Kryo as a serializer:
        $ spark-shell --conf spark.serializer=org.apache.spark.serializer.KryoSerializer
  1. Kryo automatically registers most of the core Scala classes, but if you would like to register your own classes, you can use the following command:
        scala> sc.getConf.registerKryoClasses(Array(classOf[com.infoobjects.CustomClass1],classOf[com.infoobjects.CustomClass2])
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset