You can use Kryo by doing the following settings in your SparkConf:
- Start the Spark shell by setting Kryo as a serializer:
$ spark-shell --conf spark.serializer=org.apache.spark.serializer.KryoSerializer
- Kryo automatically registers most of the core Scala classes, but if you would like to register your own classes, you can use the following command:
scala> sc.getConf.registerKryoClasses(Array(classOf[com.infoobjects.CustomClass1],classOf[com.infoobjects.CustomClass2])