"It's supposed to be automatic, but actually you have to push this button."
- John Brunner
In this chapter, you learn how RDDs can be tailored to different needs, and how these RDDs provide new functionalities (and dangers!) Moreover, we investigate other useful objects that Spark provides, such as broadcast variables and accumulators.
In a nutshell, the following topics will be covered throughout this chapter:
- Types of RDDs
- Aggregations
- Partitioning and shuffling
- Broadcast variables
- Accumulators