Hello. In this tutorial, we will explain readable and writable Streams in a Node.js application.
1. Introduction
Streams in the node are the objects that help you to write and read data to and from the destination source. There are four different types of streams in node.js:
- Readable – Stream uses for reading operations
- Writable – Stream used for write operations
- Duplex – Stream used for both reading and write operations
- Transform – Type of duplex stream where the output is computed according to the input
Each stream consists of several Event emitter events fired at different times –
- Data – Event is fired when data is available to read
- End – Event is fired when there is no more data available to read
- Error – Event is fired when there is an error receiving or writing the data
- Finish – Event is fired when all the data has been flushed
1.1 Setting up Node.js
To set up Node.js on windows you will need to download the installer from this link. Click on the installer (also include the NPM package manager) for your platform and run the installer to start with the Node.js setup wizard. Follow the wizard steps and click on Finish when it is done. If everything goes well you can navigate to the command prompt to verify if the installation was successful as shown in Fig. 1.
2. Readable and Writable Streams in Node.js
To set up the application, we will need to navigate to a path where our project will reside. For programming stuff, I am using Visual Studio Code as my preferred IDE. You’re free to choose the IDE of your choice.
2.1 Setting up the implementation
Let us write the different files which will be required for practical learning.
2.1.1 Setting up dependencies
Navigate to the project directory and run npm init -y to create a package.json file. This file holds the metadata relevant to the project and is used for managing the project dependencies, script, version, etc. Add the following code to the file wherein we will specify the required dependencies.
package.json
{
"name": "streamdata",
"version": "1.0.0",
"description": "",
"main": "index.js",
"scripts": {
"start": "nodemon index.js",
"test": "echo \"Error: no test specified\" && exit 1"
},
"keywords": [],
"author": "",
"license": "ISC",
"devDependencies": {
"faker": "^5.5.3",
"nodemon": "^2.0.13"
}
}
To download the dependencies navigate to the directory path containing the file and use the npm install command. If everything goes well the dependencies will be loaded inside the node_modules folder and you are good to go with the further steps.
2.1.2 Creating controller
In the root folder add the following content to the index file. The file will be exposing 2 endpoints –
/writeto create a writeable stream and write mock data into a file/readto create a readable stream and read data from the file
index.js
const http = require("http");
const fs = require("fs");
const faker = require("faker");
const file = "data.txt";
const server = http.createServer(function (req, resp) {
if (req.method === "POST" && req.url === "/write") {
// http://localhost:5001/write
console.log("writing data to stream.");
const writeStream = fs.createWriteStream(file);
for (let i = 0; i <= 1000; i++) {
writeStream.write(faker.lorem.words(20) + "\n");
}
writeStream.end();
resp.end();
} else if (req.method === "GET" && req.url === "/read") {
// http://localhost:5001/read
console.log("reading data.");
// getting data in a single chunk
/* fs.readFile(file, (err, data) => {
if (err) throw err;
resp.end(data);
}); */
// doing data streaming
var data = "";
const readStream = fs.createReadStream(file);
readStream.on("data", (chunk) => {
console.log("chunk data received.");
data += chunk;
});
// method will be called when no more data is available for read.
readStream.on("end", () => {
resp.end(data);
});
} else {
throw new Error("NOT_YET_IMPLEMENTED");
}
});
// start app
const PORT = process.env.PORT || 5001;
server.listen(PORT, () => {
console.log(`Server started on port ${PORT}`);
});
3. Run the Application
To run the application navigate to the project directory and enter the following command as shown in Fig. 2. If everything goes well the application will be started successfully on a port number read from the .env file or 5001.
4. Demo
You are free to use postman or any other tool of your choice to make the HTTP request to the application endpoints.
Endpoints
// http post - http://localhost:5001/write // http get - http://localhost:5001/read
That is all for this tutorial and I hope the article served you with whatever you were looking for. Happy Learning and do not forget to share!
5. Summary
In this tutorial, we explained readable and writable Streams in a Node.js application. You can download the source code and the postman collection from the Downloads section.
6. Download the Project
This was a tutorial to implement Streams in the nodejs application.
You can download the full source code of this example here: Readable and Writable Streams in Node.js
