Hello. In this tutorial, we will interact with Amazon AWS SQS (simple queue service) to create a Node.js app with the help of a popular emulator known as LocalStack.
1. Introduction
Localstack is an aws cloud service emulator that runs in a single container and allows us to run the aws applications on our local machines without connecting to a remote cloud provider. This tool helps to speed up and simplify the testing and development workflow. To work with localstack we will use docker so that we have the setup ready within minutes and without any dependency. If someone needs to go through the Docker installation, please watch this video.
1.1 Setting up Node.js
To set up Node.js on windows you will need to download the installer from this link. Click on the installer (also include the NPM package manager) for your platform and run the installer to start with the Node.js setup wizard. Follow the wizard steps and click on Finish when it is done. If everything goes well you can navigate to the command prompt to verify if the installation was successful as shown in Fig. 1.
1.2 Setting up LocalStack
To set up the localstack on our machine we will write a simple docker-compose file and execute the docker-compose commands to quickly start/stop the container. Add the below code the yml file to have the container up and running in minutes –
docker-compose.yml
services:
localstack:
environment:
- AWS_DEFAULT_REGION=ap-southeast-1
- EDGE_PORT=4566
- SERVICES=sqs
image: "localstack/localstack:latest"
ports:
- "4566:4566"
container_name: docker-localstack
volumes:
- "${TMPDIR:-/tmp/localstack}:/tmp/localstack"
- "/var/run/docker.sock:/var/run/docker.sock"
version: "3.0"
You’re free to change the details in the above yml file as per your requirements. For this tutorial, I am keeping some of the details are default.
1.3 Start/Stop the container
To start the localstack container we will execute the docker-compose command in the terminal window. Open a new terminal, navigate to the path containing the yml file and execute the below command.
Start the container
-- pull the image, create and start the container -- docker compose up -d
The above command will start the container on port – 4566 in daemon mode. If you will be doing this activity for the first time then the Docker tool will first pull the image from the hub repository. You can hit the health endpoint in the browser to confirm that the localstack container is up and running successfully.
Localstack health check endpoint
-- to check whether the localstack is up and running -- http://localhost:4566/health
Once you are done with the tutorial use the below command to stop and remove the running localstack container.
Stop and remove the container
-- stop and remove the container – docker compose down
2. LocalStack SQS Node.js Example
To set up the application, we will need to navigate to a path where our project will reside. For programming stuff, I am using Visual Studio Code as my preferred IDE. You’re free to choose the IDE of your choice.
2.1 Setting up the implementation
Let us write the different files which will be required for practical learning.
2.1.1 Setting up dependencies
Navigate to the project directory and run npm init -y to create a package.json file. This file holds the metadata relevant to the project and is used for managing the project dependencies, script, version, etc. Add the following code to the file wherein we will specify the required dependencies.
package.json
{
"name": "localstack-sqs",
"version": "1.0.0",
"description": "localstack sqs",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"dev": "nodemon index.js"
},
"keywords": [
"localstack sqs",
"nodejs",
"expressjs"
],
"author": "javacodegeeks",
"license": "ISC",
"dependencies": {
"aws-sdk": "^2.1043.0",
"express": "^4.17.1",
"underscore": "^1.13.1"
},
"devDependencies": {
"nodemon": "^2.0.15"
}
}
To download the dependencies navigate to the directory path containing the file and use the npm install command. If everything goes well the dependencies will be loaded inside the node_modules folder and you are good to go with the further steps.
2.1.2 Creating an environment config
Once the localstack is up and running we will be creating the environment configuration file required for this tutorial. Add the below code to the file present in the config folder. The file contains the localstack service endpoint and other required information for this tutorial.
env.js
const env = {
REGION: "ap-southeast-1", // should match the one given in the docker-compose.yml file
ACCOUNT_ID: "000000000000", // represents the dummy aws account id for localstack
IAM: {
ACCESS_KEY_ID: "na",
SECRET_ACCESS_KEY: "na"
},
SERVICE_ENDPOINT: "http://localhost:4566", // represents the service point for localstack
QUEUE_NAME: "geek" // queue name used in this tutorial for implementation
};
module.exports = env;
2.1.3 Creating a request handler class
Create a request handler class that will be responsible to handle the incoming requests from the client. The file will also contain the information about the sqs service object creation. Add the below code to the file present in the handler folder.
requests.js
const _ = require("underscore");
const env = require("../config/env");
// configuring local-stack aws
const AWS = require("aws-sdk");
AWS.config.update({
region: env.REGION
});
// create an sqs service object
const sqs = new AWS.SQS({
endpoint: env.SERVICE_ENDPOINT,
accessKeyId: env.IAM.ACCESS_KEY_ID,
secretAccessKey: env.IAM.SECRET_ACCESS_KEY,
region: env.REGION
});
// handler methods
// method1 - to get sqs config status
const getConfigurationStatus = (req, res) => {
console.log("fetching configuration status");
res.status(200).json({
status: "ok",
data: sqs
});
};
// method2 - to list all queues
const listQueues = (req, res) => {
console.log("fetching all queues");
sqs.listQueues({}, (err, data) => {
if (err) {
res.status(500).json({
status: "internal server error",
error: err
});
} else {
res.status(200).json({
status: "ok",
urls: data.QueueUrls
});
}
});
};
// method3 - to create a new sqs queue
const createQueue = (req, res) => {
let createParams = {
QueueName: env.QUEUE_NAME, // in real example client should provide queue name
Attributes: {
DelaySeconds: "60",
MessageRetentionPeriod: "86400"
}
};
console.log(createParams);
sqs.createQueue(createParams, (err, data) => {
if (err) {
res.status(500).json({
status: "internal server error",
error: err
});
} else {
res.status(201).json({
status: "created",
name: data.QueueName,
message: "queue created successfully"
});
}
});
};
// method4 - to purge the given sqs queue
const purgeQueue = (req, res) => {
let queueName = req.query.name;
if (_.isEmpty(queueName)) {
res.status(400).json({
status: "bad request",
message: "queue name cannot be empty"
});
} else {
console.log("purging queue = " + queueName);
let purgeParams = {
QueueUrl: env.SERVICE_ENDPOINT + "/" + env.ACCOUNT_ID + "/" + queueName
};
console.log(purgeParams);
sqs.purgeQueue(purgeParams, (err, data) => {
if (err) {
res.status(500).json({
status: "500",
err: err
});
} else {
res.status(200).json({
status: "ok",
data: "queue purged"
});
}
});
}
};
// method5 - to publish the message to the sqs queue
const publishMsg = (req, res) => {
//todo - add empty queue name validation in the request body
let msg = {
id: req.body["id"],
name: req.body["name"],
email: req.body["email"],
phone: req.body["phone"]
};
let msgParams = {
QueueUrl:
env.SERVICE_ENDPOINT + "/" + env.ACCOUNT_ID + "/" + req.body["queueName"], // queueName will never be empty
MessageBody: JSON.stringify(msg)
};
console.log(msgParams);
sqs.sendMessage(msgParams, (err, data) => {
if (err) {
res.status(500).json({
status: "internal server error",
error: err
});
} else {
res.status(202).json({
status: "accepted",
messageId: data.MessageId,
message: "sent to queue"
});
}
});
};
// method6 - to receive the message from the sqs queue
const receiveMsg = (req, res) => {
let queueName = req.query.name;
if (_.isEmpty(queueName)) {
res.status(400).json({
status: "bad request",
message: "queue name cannot be empty"
});
} else {
console.log("Fetching messages from queue = " + queueName);
let receiveParams = {
QueueUrl: env.SERVICE_ENDPOINT + "/" + env.ACCOUNT_ID + "/" + queueName,
MessageAttributeNames: ["All"]
};
console.log(receiveParams);
sqs.receiveMessage(receiveParams, (err, data) => {
if (err) {
res.status(500).json({
status: "internal server error",
error: err
});
} else {
if (!data.Messages) {
res.status(200).json({
status: "ok",
data: "no message in the queue"
});
} else {
let items = [];
_.each(data.Messages, function (message) {
let ele = {
id: message.MessageId,
receiptHandle: message.ReceiptHandle,
data: JSON.parse(message.Body)
};
items.push(ele);
});
res.status(200).json({
status: "ok",
messages: items
});
}
}
});
}
};
// method7 - to delete the message from the sqs queue
const deleteMsg = (req, res) => {
let id = req.query.id;
let queueName = req.query.name;
if (_.isEmpty(id) || _.isEmpty(queueName)) {
res.status(400).json({
status: "bad request",
message: "receipt handle id or queue name cannot be empty"
});
} else {
console.log("Deleting message id = " + id + " from queue");
let deleteParams = {
QueueUrl: env.SERVICE_ENDPOINT + "/" + env.ACCOUNT_ID + "/" + queueName,
ReceiptHandle: id
};
console.log(deleteParams);
sqs.deleteMessage(deleteParams, (err, data) => {
if (err) {
res.status(500).json({
status: "internal server error",
error: err
});
} else {
res.status(202).json({
status: "accepted",
message: "message deleted from queue"
});
}
});
}
};
module.exports = {
getConfigurationStatus,
listQueues,
createQueue,
purgeQueue,
publishMsg,
receiveMsg,
deleteMsg
};
2.1.4 Creating a routing class
Create a routing class that will be responsible to map the incoming requests from the client with the handler method. Add the below code to the file present in the routes folder.
routes.js
const express = require("express");
const router = express.Router();
const {
getConfigurationStatus,
listQueues,
createQueue,
publishMsg,
receiveMsg,
deleteMsg,
purgeQueue
} = require("../handler/requests");
// get sqs config status
// endpoint - http://localhost:6001/status
router.get("/status", getConfigurationStatus);
// list sqs queues
// endpoint - http://localhost:6001/list
router.get("/list", listQueues);
// create a sqs queue
// endpoint - http://localhost:6001/create
router.post("/create", createQueue);
// purge a sqs queue
// endpoint - http://localhost:6001/purge
router.delete("/purge", purgeQueue);
// send message to sqs queue
// endpoint - http://localhost:6001/send
router.post("/send", publishMsg);
// receive message from sqs queue
// endpoint - http://localhost:6001/receive
router.get("/receive", receiveMsg);
// delete message from sqs queue
// endpoint - http://localhost:6001/delete
router.delete("/delete", deleteMsg);
module.exports = {
appRoutes: router
};
2.1.5 Creating an index file
Create an index file that will be acting as the entry point for the application.
index.js
// application
const express = require("express");
const app = express();
app.use(express.json());
// application routes
const routes = require("./routes/routes");
app.use("/", routes.appRoutes);
// start application
const port = process.env.PORT || 6001;
app.listen(port, () => {
console.log(`Service endpoint = http://localhost:${port}`);
});
3. Run the Application
To run the application navigate to the project directory and enter the following command as shown below.
Command
$ nodemon
If everything goes well the application will be started successfully on port – 6001 as shown in Fig. 2.
4. Demo
You are free to use postman or any other tool of your choice to make the HTTP request to the application endpoints. Download the endpoints collection from the downloads section for an easy setup.
Application endpoints
// get sqs config status // http get endpoint - http://localhost:6001/status // list sqs queues // http get endpoint - http://localhost:6001/list // create a sqs queue // http post endpoint - http://localhost:6001/create // purge a sqs queue // http delete endpoint - http://localhost:6001/purge // send message to sqs queue // http post endpoint - http://localhost:6001/send // receive message from sqs queue // http get endpoint - http://localhost:6001/receive // delete message from sqs queue // http delete endpoint - http://localhost:6001/delete
That is all for this tutorial and I hope the article served you with whatever you were looking for. Happy Learning and do not forget to share!
5. Summary
Localstack is the most popular aws cloud emulator which is used by individuals to run the aws applications on local machines and without connecting to the remote cloud provider. In this tutorial, we learned how to set up the localstack and integrate it with a simple nodejs application. You can download the source code and the postman collection from the Downloads section.
6. Download the Project
This was a tutorial to understand the aws cloud emulator popularly known as the localstack and its simple working in a nodejs application.
You can download the full source code of this example here: LocalStack SQS Node.js Example
