AWS Simple Queue Service(SQS)

Configure Serverless Offline and AWS Simple Queue Service(SQS)

Introduction

The Serverless Framework is a free and open-source web framework written using Node.js [Wiki] that helps you develop and deploy AWS infrastructure resources. It’s best known for its development for AWS Lambda functions along with other AWS infrastructure resources it requires. One of the excellent plugins it offers is serverless-offline, which enables you to test your AWS resources directly from your local environment. Learn how to configure Serverless Offline for local development and testing of serverless applications, including setup steps and integration with AWS Lambda.

However, although Serverless Offline is a nice plugin for testing our lambda functions locally, it requires some additional setups for other services. AWS SQS is one of those services and if you attempt to invoke any SQS that is linked to its consumer lambda, it will either directly execute the AWS account’s SQS or fail if the resource is not available.

In this blog, let’s explore how we can streamline the process and put AWS SQS to the test. We will discuss how you can run your AWS SQS service with Serverless Offline locally using serverless-offline-sqs and ElasticMQ.

Installations and Getting Started

Initially, we will need to install serverless-offline-sqs as a dev dependency in your project root. This serverless-offline-sqs plugin emulates SQS queue on your local system, listening to the SQS queue and invoking lambda handlers.

npm install --save-dev serverless-offline-sqs

Later inside your project’s serverless.yml file add the following entry under plugins and custom

custom:
serverless-offline-sqs:
    autoCreate: true                 
    apiVersion: '2012-11-05'
    endpoint: http://0.0.0.0:9324	# Endpoint where you will publish SQS message
    region: eu-west-1  			# Your preferred AWS region
    accessKeyId: root
    secretAccessKey: root
    skipCacheInvalidation: false

plugins:
  - serverless-webpack                  # or serverless-esbuild
  - serverless-offline-sqs
  - serverless-offline

The custom’s serverless-offline-sqs is defined so that it can be used with ElasticMQ which is an in-memory message queue system that can be run on your local machine.

ElasticMQ Installation

To be able to replicate an AWS SQS queue on a local machine, a queue system must be functioning. ElasticMQ is one of the available solutions that is suited for the task. It is a stand-alone in-memory queue system that supports the AWS SQS interface. There are two ways you can install and run ElasticMQ on your local machine. You can either run the jar file with java run time or run elasticmq as a docker container.

JAVA

For running the jar file you will need to have Java version 8 or above. You can download the elasticmq-server v1.3.9 jar file or download the latest version from ElasticMQ GitHub repository.

Run the following command to start the elasticmq server on your local machine and it will bind it to which binds to localhost:9324 / 0.0.0.0:9324

java -jar elasticmq-server-1.3.9.jar

Docker

For running as a docker container you will need to have docker installed on your local machine first. After installation, you need to pull the elasticmq-native docker image with the following command.

docker pull softwaremill/elasticmq-native

According to their documentation, The docker image is much smaller and starts up much faster, compared to the full JVM version.

Now, run the image via docker with the following command and name the container Offline-SQS. (Note that if you have already started the server via jar, you need to close that and release the port, also if you run this command directly, the docker will first pull the image and later run it)

docker run --name Offline-SQS -d -p 9324:9324 -p 9325:9325 softwaremill/elasticmq-native

After your work is done you need to stop the container, you have to either restart that container you just created or remove it and re-run the above command again.

You can combine the start and stop docker commands and add a script to your package.json file. For example,

"scripts": {
    "start-elastic-mq": "docker run --name Offline-SQS --rm -it -p 9324:9324 -p 9325:9325 softwaremill/elasticmq-native",
}

Configurations in Source Code

In your source code whenever you publish a message into your SQS, it needs a QueueUrl. Generally, the queue URL looks something like this,

https://sqs.{aws-region}.amazonaws.com/{aws-accountId}/{aws-sqsQueueName}

Now to send messages to our local SQS we need to convert the URL with the endpoint we bound the server with

http://0.0.0.0:9324/{aws-accountId}/{aws-sqsQueueName}

We can have a helper method that will convert our SQS queue URL for local. For example,

export const getOfflineSqsQueueUrl = (sqsQueueUrl: string) => {
  const url = new URL(sqsQueueUrl);
  return `http://0.0.0.0:9324${url.pathname}`;
};

We can choose which URL we want to use based on the event header’s Host property to know if the call was made from a local machine. With the above configuration, we will now be able to send a message to your local SQS server.

Clone and run the following GitHub repository to see it in action. https://github.com/sayefreyadh/serverless-offline-sqs-example-typescript

Conclusion

Finally, there are a number of benefits for local development and testing environments when utilizing serverless-offline-sqs and ElasticMQ to configure Serverless Offline with AWS SQS. Although integrating AWS SQS initially requires additional setup because it relies on real AWS services or compatible emulators like ElasticMQ, Serverless Offline makes local testing of AWS Lambda functions simpler.

Development efficiency is increased by installing serverless-offline-sqs and configuring it in your serverless.yml. This allows you to enable a local SQS queue that emulates AWS SQS behavior. ElasticMQ integrates easily into your local development workflow as an in-memory queue system that is compatible with the AWS SQS API.

ElasticMQ can be configured to run in a Docker container or a Java JAR, giving you flexibility based on the needs of your local environment. With this configuration, you can make sure that your serverless apps can process SQS messages locally without using AWS services. This can speed up development cycles and lessen the need for cloud resources when testing.

Additionally, modifying queue URLs for the local ElasticMQ instance is necessary when configuring the source code to communicate with the local SQS queue. This allows for smooth message handling in the logic of your application.

Developers can effectively create, test, and debug serverless applications locally by putting these configurations into practice, guaranteeing reliable functionality before deploying to production environments. Examine the given GitHub repository to see these ideas in action and learn how to properly optimize your serverless development processes.

We appreciate you taking the time to read through this extensive guide on maximizing Serverless Offline integration with AWS Lambda and SQS.

Picture of Abdus Sayef Reyadh

Abdus Sayef Reyadh

Software Engineer-II

Hire Exceptional Developers Quickly

Share this blog on

Hire Your Software Development Team

Let us help you pull out your hassle recruiting potential software engineers and get the ultimate result exceeding your needs.

Contact Us Directly

Address:

Plot # 272, Lane # 3 (Eastern Road) DOHS Baridhara, Dhaka 1206

Talk to Us
Scroll to Top