In all those instances where data is collected from various sources, brought to a centralized location, and stored for analysis, that data is called as data at rest. There is a huge time delay between the time the data was recorded and when the analysis was performed. Analyzing the last 6 months' inventory data is an example of data at rest. Today, majority of data analysis is performed using data at rest.

With the number of Internet of Things projects on the rise, there is a great demand today to perform analysis on data in motion, also called streaming data. Streaming data is becoming ubiquitous with the number of addressable sensors and devices being added to the internet. As an example from computer network monitoring: an intrusion detection system analysis, the network packets received in real time to quickly determine if the system is compromised and takes an appropriate action. Latency is key when analyzing data in motion.

A data stream is a continuous inflow of ordered points in a multi-dimensional space. The ordering can be done either explicitly through timestamps or by some other index. The major challenge in analysis is the unbounded nature of such streams. At any point in time, we don't see all the data. This poses a lot of restrictions on the algorithms that can process this data. The algorithm has to work on a limited set of data at any point in time. It has to make quick decisions in a single pass. The real-time nature of this process demands high throughput. Lastly, but most importantly, the algorithm has to deal with any drift present in the data streams. A future stream may have completely drifted in terms of the distribution or the properties of the data compared to the present stream.

In this chapter, we will cover the following topics:

• Introducing streaming data and the challenges it comes with
• Introducing stream clustering
• Introducing stream package in R to work on streaming data
• Adopting a stream clustering algorithm for a simulated sensor network

The code for this chapter was written in RStudio Version 0.99.491. It uses R version 3.3.1. As we work through our example, we will introduce the R stream packages we will be using. During our code description, we will be using some of the output printed in the console. We have included what will be printed in the console, immediately following the statement which prints the information to the console, so as to not disturb the flow of the code.