Introduction
The Serverless Framework is a free and open-source web framework written using Node.js [Wiki] that helps you develop and deploy AWS infrastructure resources. It’s best known for its development for AWS Lambda functions along with other AWS infrastructure resources it requires. One of the excellent plugins it offers is serverless-offline, which enables you to test your AWS resources directly from your local environment.
However, although Serverless Offline is a nice plugin for testing our lambda functions locally, it requires some additional setups for other services. AWS SQS is one of those services and if you attempt to invoke any SQS that is linked to its consumer lambda, it will either directly execute the AWS account’s SQS or fail if the resource is not available.
In this blog, let’s explore how we can streamline the process and put AWS SQS to the test. We will discuss how you can run your AWS SQS service with Serverless Offline locally using serverless-offline-sqs and ElasticMQ.
Installations and Getting Started
Initially, we will need to install serverless-offline-sqs as a dev dependency in your project root. This serverless-offline-sqs plugin emulates SQS queue on your local system, listening to the SQS queue and invoking lambda handlers.
npm install --save-dev serverless-offline-sqs
Later inside your project’s serverless.yml file add the following entry under plugins and custom
custom: serverless-offline-sqs: autoCreate: true apiVersion: '2012-11-05' endpoint: http://0.0.0.0:9324 # Endpoint where you will publish SQS message region: eu-west-1 # Your preferred AWS region accessKeyId: root secretAccessKey: root skipCacheInvalidation: false plugins: - serverless-webpack # or serverless-esbuild - serverless-offline-sqs - serverless-offline
The custom’s serverless-offline-sqs is defined so that it can be used with ElasticMQ which is an in-memory message queue system that can be run on your local machine.
ElasticMQ Installation
To be able to replicate an AWS SQS queue on a local machine, a queue system must be functioning. ElasticMQ is one of the available solutions that is suited for the task. It is a stand-alone in-memory queue system that supports the AWS SQS interface. There are two ways you can install and run ElasticMQ on your local machine. You can either run the jar file with java run time or run elasticmq as a docker container.
JAVA
For running the jar file you will need to have Java version 8 or above. You can download the elasticmq-server v1.3.9 jar file or download the latest version from ElasticMQ GitHub repository.
Run the following command to start the elasticmq server on your local machine and it will bind it to which binds to localhost:9324 / 0.0.0.0:9324
java -jar elasticmq-server-1.3.9.jar
Docker
For running as a docker container you will need to have docker installed on your local machine first. After installation, you need to pull the elasticmq-native docker image with the following command.
docker pull softwaremill/elasticmq-native
According to their documentation, The docker image is much smaller and starts up much faster, compared to the full JVM version.
Now, run the image via docker with the following command and name the container Offline-SQS. (Note that if you have already started the server via jar, you need to close that and release the port, also if you run this command directly, the docker will first pull the image and later run it)
docker run --name Offline-SQS -d -p 9324:9324 -p 9325:9325 softwaremill/elasticmq-native
After your work is done you need to stop the container, you have to either restart that container you just created or remove it and re-run the above command again.
You can combine the start and stop docker commands and add a script to your package.json file. For example,
"scripts": { "start-elastic-mq": "docker run --name Offline-SQS --rm -it -p 9324:9324 -p 9325:9325 softwaremill/elasticmq-native", }
Configurations in Source Code
In your source code whenever you publish a message into your SQS, it needs a QueueUrl. Generally, the queue URL looks something like this,
https://sqs.{aws-region}.amazonaws.com/{aws-accountId}/{aws-sqsQueueName}
Now to send messages to our local SQS we need to convert the URL with the endpoint we bound the server with
http://0.0.0.0:9324/{aws-accountId}/{aws-sqsQueueName}
We can have a helper method that will convert our SQS queue URL for local. For example,
export const getOfflineSqsQueueUrl = (sqsQueueUrl: string) => { const url = new URL(sqsQueueUrl); return `http://0.0.0.0:9324${url.pathname}`; };
We can choose which URL we want to use based on the event header’s Host property to know if the call was made from a local machine. With the above configuration, we will now be able to send a message to your local SQS server.
Clone and run the following GitHub repository to see it in action. https://github.com/sayefreyadh/serverless-offline-sqs-example-typescript
Thank you for your time.