The Ultimate Guide to Reading Files in Node.js

From a programming standpoint —  A file is essentially a container for storing information. For example, you may have a text-based file containing a list of grades for a course. You may also have an image file containing a JPEG image of your dog. A file could be anything, but it usually represents some form of data that is structured in a way that makes sense to people.

Prerequisite

The following prerequisites are assumed before getting started with this tutorial:

• Basic familiarity with Javascript
• Node.js 14 or later must be installed.
• A text editor

Node.js fs Module

As mentioned previously, the file system module is built right into node.js, which means that we don’t have to install any other packages to get started. And we can include it in a node.js project like below:

// sample.js
const fs = require("fs");

If you'd prefer to import it as an ECMA Script module instead. Using the import statement, we can do so like below:

// sample.cjs
import * as fs from 'fs';

This will import the whole node.js file system module, which gives us direct access to creating a new file or altering existing ones on our local machine or server.

Also, rather than importing the whole fs package into our project, we can choose to import a specific method from it using javascript destructuring. An example would look like this:

const { methodName } = require("fs");

So that methodName is now available as a standalone function that we can call directly. However, for the sake of clarity and conciseness, we will not be employing this format in this article moving forward.

Ways to Read files in Node.js

Naturally, node.js includes two different methods for reading files: fs.readFile() and the fs.readFileSync() method. The first method will read the file content in a non-blocking asynchronous manner and return the content in a callback function. The readFileSync() method, on the other hand, will read the file synchronously i.e, code executions are blocked until this process is completed. Also, unlike the readFile() method, this method returns its result as a return value.

The fs.readFile () method takes three parameters, which are described further below:

• File: A required string specifying the path and file name of the file to read.
• Encoding: An optional string indicating the type of file encoding.
• A callback function is used to handle the data returned from the request as well as any errors that may occur.

A sample request to read a file's content with fs.readFile() would look like this:

const fs = require("fs");

fs.readFile("/path/to/file", "encoding-type", (err, data) => {
  // ...
});

The fs.readFileSync() method likewise accepts the same parameters, except with the callback function, and a sample request would look like this:

const fs = require("fs");

let fileContent = fs.readFileSync("/path/to/file", "encoding-type");
console.log(fileContent);

It's also worth mentioning that if no encoding is specified, the data is returned as a buffer object. Otherwise, it will be returned as a string.

fs.readFile() vs fs.readFileSync()

Let's try a short experiment to better grasp the difference between these two functions. Assume we wish to read the contents of a file and then log another message to the console in another code block. Our code example would be as follows:

fs.readFile("path/to/file", "utf-8", (err, data) => {
  console.log(data);
});
console.log("Read file content ✅");

If our sample file contains “Hello world”, the output we get from this code is:

Read file content ✅
Hello World

I.e the console.log() is invoked before fs.readFile() completes reading the file. This is to emphasize that this method is asynchronous (non-blocking), and its execution will not prevent other code blocks from running. However, this is not the case with fs.readSync(), as the result is returned as a value that we can act on right away.

Please keep in mind that, in order to avoid ambiguity, we will only use the fs.readFile() in this guide moving forward.

Reading Text File

Reading text file with both functions is quite straightforward. Lets say we have a file structure like below:

.
├── sample.txt
└── app.js

And we want to read the contents from sample.txt with app.js, we can do so with the code below:

// app.js
const fs = require("fs");

fs.readFile("sample.txt", "utf-8", (err, data) => {
  if (err) {
    console.log(err);
  } else {
    console.log(data);
  }
});

From the callback function, we’ve made the file available via variable data, and when we run this code, we should have the content from our sample.txt file logged to the console.

And, yes, the file content has now been converted to a string, i.e we can now perform string-related operations with it.

Below is an updated code from our previous example that splits our file content and checks if its first word is “hello”.

fs.readFile("sample.txt", "utf-8", (err, data) => {
  if (err) {
    console.log(err);
  } else {
    let splitWords = data.split(" ");
    let firstWord = splitWords[0];

    console.log(
      firstWord == "hello"
        ? "Sample.txt starts with 'hello'"
        : "Sample.txt doesn't starts with 'hello'"
    );
  }
});

Reading HTML Files

We can read HTML files the same way we’d read a .txt file using the fs.readFile() method, also specifying the encoding type as utf-8. If for example, have an index.html, its possible to read this file content like below:

const fs = require("fs");

fs.readFile("test.html", "utf-8", (err, data) => {
  if (err) {
    console.log(err);
  } else {
    console.log(data);
  }
});

You can then choose to proceed to parse this markup with packages like cheerio or maybe even take it one step further and render it with node http package:

const fs = require("fs");
const http = require("http");
http
  .createServer(function (req, res) {
    fs.readFile("test.html", "utf-8", (err, data) => {
      if (err) {
        console.log(err);
      } else {
        console.log(data);
        res.writeHead(200, { "Content-Type": "text/html" });
        // render returned data in http://localhost:8080
        res.write(data);
        res.end();
      }
    });
  })
  .listen(8080);

Reading file by URL

Reading a remote file using its url is more tricky than the previous examples we've seen. We'll need to first create a file stream, then use node http request to send a request to the remote file, and finally pipe the response to our file stream.

An example is provided below:

const http = require("http");
const fs = require("fs");

const file = fs.createWriteStream("image.jpg");
const request = http.get(
  "http://i.imgur.com/noKHQD5.jpeg",
  function (response) {
    response.pipe(file);
  }
);

In the example provided above, we used the fs.createWriteStream() function to create a file stream that allows us to write raw data to this file, and then we used Node.js' built-in https package to send a request to a sample image hosted on Imgur. After that, we piped the result from our request to our file, so when you run the code above, the image file from our remote URL is downloaded as a new file — image.jpg — and saved on our local machine.

Reading a JSON file

We can read JSON files following the same process we did for reading a .txt and the .html file in our previous examples. Suppose we have a file called cities.json that contains the following data:

[
  {
    "city": "Tokyo",
    "population": "37843000",
    "country": "Japan"
  },
  {
    "city": "Jakarta",
    "population": "30539000",
    "country": "Indonesia"
  },
  {

    "city": "Lagos",
    "population": "15388000",
    "country": "Nigeria"
  }
]

We can read its content like so:

fs.readFile("cities.json", "utf-8", (err, data) => {
  console.log(data);
});

Running the code above, our JSON content is converted to a string. However, we can easily convert it to a valid JSON object using the JSON.parse() function, and can then proceed to loop each item in the json object, or perform any other action as deemed fit:

fs.readFile("cities.json", "utf-8", (err, data) => {
  let jsonData = JSON.parse(data);
  jsonData.forEach((city) => {
    console.log(`${city.city} population is ${city.population}`);
  });
});

fs.promises

Node.js version 10 also introduces a new promise based solution for performing file operations using the async/await syntax.

This method is available in the file system module as fs.promises, and performing a basic readFile operation would look like this:

async function loadSampleFile() {
  let fileContent = await fs.promises.readFile("path/to/file", { encoding: "utf-8" });
  console.log(fileContent);
}

The main difference between this new syntax and earlier examples is that we removed the callback function, avoiding callback hell. Furthermore, this is a lot cleaner syntax, allowing us to write better, more performant code.

Conclusion

In this article, we took a closer look at some of the most common Node.js techniques for reading and processing various file types. We looked into a variety of ways for extracting data from text, HTML, and even image files. These strategies come in handy for creating a file system, document parsing, or data processing programs. It's now your turn to use these techniques to create something very cool.