Welcome to part 4 of a multiple part course on passing your AWS Architect, Developer & Sysops Associate exams. The best part…this course is totally free of charge!

When I was studying for the AWS Architect Associate exam I knew that Simple Queue Service (SQS) would show up. So I did a bunch of research, asked people who’ve recently passed the exam and documented exactly what you need to know about SQS.

In this article we will look at Amazon’s Simple Queue Service (SQS for short). SQS a fully managed queuing system, exposed via a web service API. It allows for clean separation and decoupling of components that make up your applications.

The article will take just 15 minutes to read and after doing so you’ll have all you need to know to answer any question around SQS in the AWS certifications. It is guaranteed to appear in all the associate exams.

Who should read this?

If you are studying for the:

• AWS Associate Architect
• AWS Associate Developer
• AWS Associate SysOps

Or you are using AWS and want to learn more about SQS then this is the article for you.

What is AWS SQS?

Ok, let’s start with the basics. What exactly is SQS?

SQS is a web service that gives you access to a message queue that stores messages while waiting for a computer to process them.

SQS was the first service to be made publicly available from AWS almost 14 years ago in 2004.

The typical use case for SQS is to act as a buffer between discrete components that make up a system. By having components interact with the simple queue service and not directly with one another you reduce coupling between the modules that make up your application. The benefits of architecting your system in this way means you can gear your system to react differently based on the amount of messages queued to be processed.

For instance, you can increase the number of worker nodes based on the number of tasks stacking up in the queue. (We’ll go into more detail about this later in the article)

Why is separation of concerns important?

Separation of concerns is a design principle that aims to separate a computer program into discrete components. The benefit being that you could modify one section of code without having to worry about its effects on the other components (often referred to as modules).

Provided that the contracts that define the communications channels between the components is not altered.

From the perspective of the components that interact with the modified component nothing has changed.

This design philosophy simplifies development and allows development teams to focus exclusively on the component they are building.

Obvious examples of this include separating Front end (presentation layer) from a backend server via an API. You can then further modularize by breaking your backend down into smaller chunks.

Things you should know about SQS!

PULL not PUSH!

SQS is PULL based! What does this mean?

Well it means that message sit on the queue until a worker node requests to read the next message in the queue. So basically worker nodes will poll the SQS queue looking for work to carry out.

SQS will NOT push messages to the worker nodes. For that you’ll need something like Simple Notification Service SNS (which we’ll cover in a later post).

Message Size

The messages that SQS stores can be upto 256 kb in size.

Those messages can be in data formats such as TEXT, JSON, XML etc…

Messages typically contain information about the task you want to perform. They wouldn’t necessarily contain the information to process. Instead they could point to a storage location such as S3 if for instance the queue was encoding video files.

Message Life Span

Messages by default are kept for 4 days.

However their life span can be reduced to just 60 seconds or alternatively they can be increased to 14 days.

There are 2 Types of SQS queues

I could see knowing the distinction between the 2 queue types coming up in the exam for sure. So here’s a quick summary of the 2.

Standard queues

This is the queue type you get by default. It offers a few advantages over FIFO queues.

Such as:

Unlimited throughput basically meaning you can send as many messages into this queue type as you want without worrying about hitting limits.

At-least-once-delivery guaranteeing that each message will be delivered at least once. But occasionally more than once.

Best-Effort Ordering means basically that the order that the messages were originally sent to the queue. However sometimes they can come out of order.

FIFO queues

FIFO or First In First Out queues are where the order of the messages in the queues is guaranteed. With this guarantee there comes a few trade offs:

Throughput FIFO is limited to 300 messages per second. Although you can increase this by batching the messages.

Exactly-Once Processing messages are delivered exactly once and remains available until a consumer node deletes it.

Polling Types

What are polling types?

Well, remember how I said SQS is a PULL based system?

Short polling returns immediately even if the queue is empty. You get charged per usage, so the costs can rack up.

This can pose a problem for developers because if you want to know if there is work to be processed in the queue then you will have to constantly talk to (or poll) the queue to find that out. This incurs charges and can make running SQS expensive.

There is however a solution to this:

Long polling doesn’t return a message until there is a message in the queue or the long polling times out. This helps save money because you are not constantly polling the queue to check if there are messages to process

SQS Visibility Timeout

Before we go into a real world example, let’s discuss SQS’s visibility timeout feature.

When a message is polled and pickup up by a worker node, that message is rendered invisible in the queue. This is because if it weren’t then there is a possibility that another worker node would see it and pick it up from the queue.

The visibility timeout is triggered and the message is added back to the queue if a worker node has not completed the tasks within the timeout period. The worker would inform SQS by sending a delete command to the queue.

If it is not processed before the visibility timeout expires then the message can appear back in the queue and another reader can process it again. Meaning the same message could be processed twice.

The default timeout is 30 seconds and you can increase it all the way up to 12 hours.

A Real World Example of SQS

Ok, let’s imagine we’re running the latest and greatest video streaming service. Orange Tube.

Well Orange Tube has tons of users who are constantly uploading videos to it’s servers hosted on AWS.

Those videos are stored on S3 and a message is created each time that is added to the SQS queue. These messages are then picked up by worker nodes that are LONG polling the queue for jobs.

The messages have a 30 minute visibility timeout setting and this gives a worker node that amount of time to encode the uploaded video in the format the Orange Tube understands and then deletes the message from the queue.

To deal with sudden spikes in upload traffic, we can place the worker nodes behind a auto scaling group and then monitor the SQS queue for if the number of tasks grows beyond a certain value then we can simply increase (or decrease) the number of nodes available to process videos.

Let’s Review What We’ve Learnt

Ok so that wasn’t hard was it. SQS in all seriousness won’t be a major player on the exam. But if you understand it at a high level and know a few usage cases for it, you should have your bases covered.

• Pull based system
• 256kb messages size
• Messages can be kept in queue for 1 minute to 14 days
• Default retention is 4 days
• SQS guarantees that your messages will be processed at least once
• 2 Types of queue, FIFO and Standard Queues.
• Checkout AWS’s FAQ on SQS for more info.

In our next guide, we look into Simple Notification Service and Simple Workflow Service. They follow closely with SQS and are worth learning to bag a few extra points in the AWS certifications. Check them out here AWS Simple Notification Service & Simple Workflow Service Guides