Spark Streaming adds the holy grail of big data processing—that is, real-time analytics—to Apache Spark. It enables Spark to ingest live data streams and provides real-time intelligence at a very low latency of a few seconds.
In this chapter, we are going to cover the following recipes:
- WordCount using Structured Streaming
- Diving into Structured Streaming
- Streaming Twitter data
- Streaming using Kafka
- Understanding streaming challenges