AWS Simple Queue Service(SQS)

Configure Serverless Offline and AWS Simple Queue Service(SQS)

Introduction

The Serverless Framework is a free and open-source web framework written using Node.js [Wiki] that helps you develop and deploy AWS infrastructure resources. It’s best known for its development for AWS Lambda functions along with other AWS infrastructure resources it requires. One of the excellent plugins it offers is serverless-offline, which enables you to test your AWS resources directly from your local environment.

However, although Serverless Offline is a nice plugin for testing our lambda functions locally, it requires some additional setups for other services. AWS SQS is one of those services and if you attempt to invoke any SQS that is linked to its consumer lambda, it will either directly execute the AWS account’s SQS or fail if the resource is not available.

In this blog, let’s explore how we can streamline the process and put AWS SQS to the test. We will discuss how you can run your AWS SQS service with Serverless Offline locally using serverless-offline-sqs and ElasticMQ.

Installations and Getting Started

Initially, we will need to install serverless-offline-sqs as a dev dependency in your project root. This serverless-offline-sqs plugin emulates SQS queue on your local system, listening to the SQS queue and invoking lambda handlers.

npm install --save-dev serverless-offline-sqs

 

Later inside your project’s serverless.yml file add the following entry under plugins and custom

custom:
serverless-offline-sqs:
    autoCreate: true                 
    apiVersion: '2012-11-05'
    endpoint: http://0.0.0.0:9324	# Endpoint where you will publish SQS message
    region: eu-west-1  			# Your preferred AWS region
    accessKeyId: root
    secretAccessKey: root
    skipCacheInvalidation: false

plugins:
  - serverless-webpack                  # or serverless-esbuild
  - serverless-offline-sqs
  - serverless-offline

 

The custom’s serverless-offline-sqs is defined so that it can be used with ElasticMQ which is an in-memory message queue system that can be run on your local machine.

ElasticMQ Installation

To be able to replicate an AWS SQS queue on a local machine, a queue system must be functioning. ElasticMQ is one of the available solutions that is suited for the task. It is a stand-alone in-memory queue system that supports the AWS SQS interface. There are two ways you can install and run ElasticMQ on your local machine. You can either run the jar file with java run time or run elasticmq as a docker container.

JAVA

For running the jar file you will need to have Java version 8 or above. You can download the elasticmq-server v1.3.9 jar file or download the latest version from ElasticMQ GitHub repository.

Run the following command to start the elasticmq server on your local machine and it will bind it to which binds to localhost:9324 / 0.0.0.0:9324

java -jar elasticmq-server-1.3.9.jar

 

Docker

For running as a docker container you will need to have docker installed on your local machine first. After installation, you need to pull the elasticmq-native docker image with the following command.

docker pull softwaremill/elasticmq-native

 

According to their documentation, The docker image is much smaller and starts up much faster, compared to the full JVM version.

Now, run the image via docker with the following command and name the container Offline-SQS. (Note that if you have already started the server via jar, you need to close that and release the port, also if you run this command directly, the docker will first pull the image and later run it)

docker run --name Offline-SQS -d -p 9324:9324 -p 9325:9325 softwaremill/elasticmq-native

 

After your work is done you need to stop the container, you have to either restart that container you just created or remove it and re-run the above command again.

You can combine the start and stop docker commands and add a script to your package.json file. For example,

"scripts": {
    "start-elastic-mq": "docker run --name Offline-SQS --rm -it -p 9324:9324 -p 9325:9325 softwaremill/elasticmq-native",
}

Configurations in Source Code

In your source code whenever you publish a message into your SQS, it needs a QueueUrl. Generally, the queue URL looks something like this,

https://sqs.{aws-region}.amazonaws.com/{aws-accountId}/{aws-sqsQueueName}

Now to send messages to our local SQS we need to convert the URL with the endpoint we bound the server with

http://0.0.0.0:9324/{aws-accountId}/{aws-sqsQueueName}

We can have a helper method that will convert our SQS queue URL for local. For example,

export const getOfflineSqsQueueUrl = (sqsQueueUrl: string) => {
  const url = new URL(sqsQueueUrl);
  return `http://0.0.0.0:9324${url.pathname}`;
};

 

We can choose which URL we want to use based on the event header’s Host property to know if the call was made from a local machine. With the above configuration, we will now be able to send a message to your local SQS server.

Clone and run the following GitHub repository to see it in action. https://github.com/sayefreyadh/serverless-offline-sqs-example-typescript

Thank you for your time.

Picture of Abdus Sayef Reyadh

Abdus Sayef Reyadh

Software Engineer-II

Hire Exceptional Developers Quickly

Share this blog on

Hire Your Software Development Team

Let us help you pull out your hassle recruiting potential software engineers and get the ultimate result exceeding your needs.

Contact Us Directly
Craftsmen Bangladesh

Address:

Plot # 272, Lane # 3 (Eastern Road) DOHS Baridhara, Dhaka 1206

Craftsmen Norway

Address:

Kong Oscars gate 66, 68, 5017 Bergen,
Norway

Copyright © Craftsmen

Scroll to Top