Some of the more popular message queue services are:
Let's say you have a web application (that is probably running on an EC2 instance) that writes to a database. Perhaps there are periodic spikes where there are perhaps millions of write requests causing significant latency. Worse case scenario, this could perhaps causes the web app to load endlessly, fail to load at all, return timeout errors, display wonky errors, yada yada yada. Not good.
SQS to the rescue! The web application could publish messages to the Simple Queue Service and then the database could consume the messages in queue. This is basically the publish/subscribe architecture (pub/sub), where the web app publishes the message to the queue and the database subscribes to the queue.
In the Amazon Web Services Simple Queue Service (SQS) console, select Create queue.
Since this is just a Getting Started article, we don't want to get into all of the various configuration options. Just keep it simple! Give your queue a name, scroll down to the bottom of the page, and select Create queue.
Let's publish a message to the queue for proof of concept testing. Select your queue and then go with Send and receive messages.
Create a message and select Send message.
There should now be one message in queue. Nice!
Ok, that's enough with the console (for now). Since ultimately, at the end of the day, we are going to using code to put and get messages, let's start with the easiest "code", the AWS CLI.
The aws sqs list-queues command can be used to list your queues. There it is!
~]$ aws sqs list-queues
{
"QueueUrls": [
"https://queue.amazonaws.com/123456789012/my-first-queue"
]
}
And the aws sqs get-queue-url command can be used to get the URL of your queue. We are going to need this URL for the web app that is going to publish messages to the queue and for the database that is going to subscribe to the queue.
~]$ aws sqs get-queue-url --queue-name my-first-queue
{
"QueueUrl": "https://queue.amazonaws.com/123456789012/my-first-queue"
}
And then the aws sqs receive-message command can be used to get the message from the queue. The body is "hello". Hey great, that's the message I created. Nice.
~]$ aws sqs receive-message --queue-url https://queue.amazonaws.com/123456789012/my-first-queue
{
"Messages": [
{
"Body": "hello",
"ReceiptHandle": "AQEBqy3nHXKLjlXFCw7ZMky0z6bMgEG0MvvnNBLnQ/1drFX+9Oqh291bmiu3AqQjKni+REeYhyidtW5m+blD5WDA66dqfmGsk6q79jfrZf5GjNwgWU7FNJlm6//Sw21HdezqygzlM4slAQ54URQ9CqaTTUSmjWK6rwPuXrG1ULswGdUzp2ZNtxvSUy1km+nLy7Yp9rGhwUAX3vJlAZAGKKvce6owT/Tnf/uSi68dpv91mJH6LR23QUOdnBk8Bgl1v1SF1QkIo+DiMhY3/IoSu9FpnCZj3rSbafY32FcTLmqfctuVGAJnU9mkNRQuCMATFoSVkkjAgOrKjUZaQMZ6eyNdIS+l67N2QNbUpgsBBQtLaGg8CBHIkGCGJkKV+/ekgUXWj6/godVCkgOwo/hEeZeegQ==",
"MD5OfBody": "5d41402abc4b2a76b9719d911017c592",
"MessageId": "5508026b-e650-45d8-8482-8d13623d6a6b"
}
]
}
Back to the console for a moment. Let's select Poll for messages. There is our message! Be aware that each time you "get" the message, the receive count will be incremented by one.
And the details of the message will have the timestamp for when the message was sent (created) and the first time the message was received.
It is noteworthy that Simple Notification Service (SNS) and Simple Queue Service (SQS) are similar but not identical.
- SNS uses the pub/sub model whereas SQS is a queueing system. The pub/sub model is typically used to deliever a message to multiple subscribers wereas messages in a queue are typically consumed by a single consumer
- SNS immediately pushes messages to subscribers whereas consumers must initiate the request to get (pull) messages in an SQS queue
- SNS is often uses for real time notifications/alerts whereas SQS is more suited for scenarios where messages can be retrieved on-demand, such as once an hour or once daily, such as some sort of nightly batch job
- SNS does not hold onto messages, they are either successfully delivered to subscribers or discarded (or optionally put onto an SQS dead letter queue) whereas messages can remain in an SQS queue for a period of time
Did you find this article helpful?
If so, consider buying me a coffee over at