One of the biggest advantages of using the cloud is the ability to process large amounts of data quickly and efficiently. The two most common high-level approaches are real-time and batch data processing. Similar to the previous example, each of these patterns has been around for a while and there are ways to achieve them using more traditional tools and approaches. However, adding serverless will increase velocity and reduce pricing.

We have the real-time processing and analysis of a social media data stream, and it could just as easily be a click stream analysis of all web properties to support ad analytics. The stream or click stream comes in via a streaming data service, Amazon Kinesis, which executes a Lambda function on each micro batch of data that is processed and analyzed, based on the requirement. The Lambda function then stores the important information in a DynamoDB table, which could be sentiment analysis (if the social media stream is used) or popular ad clicks (if the click stream is used). Once the metadata is stored in the table, additional reporting, analysis, or dashboards can be designed and implemented as another subsystem using additional serverless services.

To show how data processing can be achieved for batch data processing, consider the case where batch files are placed in an object storage, Amazon S3. These files could be anything, images, CSV files, or other blobs that need to be processed or modified in some way before the data processing is completed. Once the file is placed in the object store, an AWS Lambda function executes and modifies the file for the purposes required. This batch process could be to execute other subsystems that use the data, modify the image to change its size, or to update various data stores with parts of the data required. In this example, a DynamoDB table is updated, a message is put in a queue, and a notification is sent to the user.