Integrating Amazon SNS and Amazon SQS

One of the key features of Amazon SQS is that it can easily be integrated with other AWS services, such as Amazon SNS. Why would I need something like that? To begin with, let us quickly recap the things we have learned so far about both SNS and SQS:

Amazon SNS

Amazon SQS

Leverages the push mechanism

Leverages the polling mechanism

Amazon SNS messages can push messages to mobile devices or other subscribers directly

Amazon SQS needs a worker to poll the messages

Persistence of messages is not supported

Amazon SQS supports message persistence which can come in really handy if you can't reach your consumers due to a network failure

 

From the table, it is easy to see that both the services offer their own pros and cons when it comes to working with them. However, when we join the two services, you can actually leverage them to design and build massively scalable yet decoupled applications. One common architectural pattern that you can leverage by combining both SNS and SQS is called the fan out pattern.

In this pattern, a single message published to a particular SNS topic can be distributed to a number of SQS queues in parallel. Thus, you can build highly-decoupled applications that take advantage of parallel and asynchronous processing. Consider a simple example to demonstrate this pattern. A user uploads an image to his S3 bucket, which triggers an SNS notification to be sent to a particular SNS topic. This topic can be subscribed by a number of SQS queues, each running a completely independent process from the other. For example, one queue can be used to process the image's metadata while the other can be used to resize the image to a thumbnail, and so on. In this pattern, the queues can work independently of each other without even having to worry about whether or not the other completed its processing or not. Here is a representational figure of this pattern:

  1. To integrate both the SNS and SQS services, you will first be required to create a simple SNS topic of your own. Go ahead and create a new SNS topic using the AWS Management Console, as performed earlier in this chapter.
  2. Once the topic is ready, the next step involves the creation of an associated subscription. To do so, from the SNS dashboard, select the Subscriptions option from the navigation pane and click on Create subscription to get started.
  1. In the Create subscription dialog box, copy and paste the newly created topic's ARN in the Topic ARN field, as shown in the following screenshot:
  1. Once the Topic ARN is pasted, select the Amazon SQS option from the Protocol drop-down list, followed by pasting a queue's ARN in the Endpoint field. In this case, I'm using the standard queue's endpoint that we created a while back in this chapter.
  2. With the required fields filled out, select Create subscription to complete the process.
  3. Next, from the SQS dashboard, select the queue that you have identified for this integration and, from the Permissions tab, select Add a Permission to allow the SNS service to send messages to the queue. To do so, provide the following set of permissions:
    • Effect: Allow
    • Principal: Everybody
    • Actions: SendMessage
  1. Once done, click on Add Permission to grant the SNS service the required set of permissions.
  2. We are now ready to test the integration! To do so, simply fire a sample message using the Publish to Topic option from the SNS dashboard. Once the message is successfully sent, cross over to the SQS dashboard and poll the queue using the View/Delete Messages option from under the Queue Actions drop-down list.

Here is a snippet of the Message Body obtained after long polling the queue:

Similarly, you can use such a fan out pattern to design and build your very own highly scalable and decoupled cloud-ready applications.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset