Example use case – serverless file transformer

Here is an example use case, to give you an idea of how different managed AWS systems can fit together as a solution. The requirements are that a third-party vendor is sending us a small 10 MB file daily at random times, and we need to transform the data and write it to a NoSQL database so it can be queried in real time. If there are any issues with the third-party data, we want to send an alert within a few minutes. Your boss tells you that you they don't want to have an always-on machine just for this task, the third party has no API development experience, and there is a limited budget. The head of security also finds out about this project and adds another constraint. They don't want to give third-party access to your AWS account beyond one locked-down S3 bucket:

This can be implemented as an event-driven serverless stack. On the left, we have an S3 bucket where the third party has access to drop their file. When a new object is created, that triggers a Lambda invocation via the built-in event source mapping. The Lambda executes code to transform the data, for example, extracts key records such as user_id, date, and event type from the object, and writes them to a DynamoDB table. The Lambda sends summary custom metrics of the transformation, such as number of records transformed and written to CloudWatch metrics. In addition, if there are transformation errors, the Lambda sends an SNS notification with a summary of the transformation issues, which could generate an email to the administrator and third-party provider for them to investigate the issue.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset