Module Core Session 6 - Tame the Node.js Callback Hell - Promises to the Rescue

In Session 1 we were introduced to the world of Node.js callbacks, and while it's powerful, it's much more cumbersome than the synchronous approach.

We need to know how the callback way works, because they are baked into the official Node.js API, so they are here to stay, but it doesn't mean that we have to write our own code in callbacks (though it's certainly something to consider if you are releasing an API). Progresses have been made in order to tame the callback hell, and today we'll take a look at the first approach - Promises.

If you are already a seasoned frontend web developer, you probably already know and work extensively with promises. In that case, feel free to skip over this session.

What Is a Promise

As its name implies, a promise is something that'll happen in the future (like when someone makes you a promise). If we think about it, that's exactly what a callback is doing - when you pass in a callback, you are promised that the callback will be called with the error or the result sometime in the future.

Instead of taking in a callback, a promise-based function returns a Promise object to be the access point of the result (or error) when the computation is completed.

I.e. instead of using a callback like:

fs.readFile(<file path>, function (err, data) {
  // this is the callback.
});

We return a promise object like:

// notice the lack of the callback, and the promise is now returned as a result of the function call.
var promise = fs.readFileAsync(<file path>); 
promise.then(function (data) {
  // do something if it's successfully.
})
.catch(function (e) {
  // handle error.
});

Since functions that return promises do not take callbacks, we have shifted the callback hell like:

fs.readFile(<filePath1>, (err1, data1) => {
  if (err1) {
    // handle error.
  } else {
    // ... some process, then
    fs.readFile(<filePath2>, (err2, data2) => {
      if (err2) {
        // handle error
      } else {
        // ... some process, then
        fs.readFile(<filePath3>, (err3, data3) => {
          if (err3) {
            // handle error
          } else {
            // ... some process, then
            fs.readFile(<filePath4>, (err4, data4) => {
              if (err4) {
                // handle error
              } else {
                // process...
              }
            })
          }
        })
      }
    })
  }
})

to something much more manageable:

fs.readFileAsync(<filePath1>)
  .then((data1) => {
    // ... some process, then
    return fs.readFileAsync(<filePath2>)
  }) // promises are chainable.
  .then((data2) => {
    // ... some process, then
    return fs.readFileAsync(<filePath3>)
  })
  .then((data3) => {
    // ... some process, then
    return fs.readFileAsync(<filePath4>)
  })
  .then((data4) => {
    // process...
  })
  .catch((e) => {
    // handle error.
  })

If you squint to block out the structures, it almost look like how the synchronous code would have been written:

// pseudo code
fs.readFileAsync(<filePath1>)
   // ... some process, then
   fs.readFileAsync(<filePath2>)
   // ... some process, then
   fs.readFileAsync(<filePath3>)
   // ... some process, then
   fs.readFileAsync(<filePath4>)
   // process...
// catch 
   // handle error.   

So Promises helps a lot with the taming of the callback hell already. It is also the backbone of the newer patterns for async code like Generators and Async / Await, so it's important that we start with learning Promises as the next step of callbacks.

Using a Promise

Promise has been widely implemented in the current browsers as a built-in object, so unless you need to support older browsers like IE, you can make use of the built-in Promise object.

However, at this time the built-in Promise objects apparently suffer from some implementation issues and aren't as performant as an external Promise library like Bluebird, and the built-in Promise also lack some of the nice helper functions that comes with Bluebird, so you might find it easier to work with Bluebird instead.

npm install --save-dev @types/bluebird
npm install --save bluebird

Then you can import the bluebird module as a stand-in for the built-in Promise:

import * as Promise from 'bluebird';

Wrapping Node.js Callbacks

To wrap around the existing Node.js callback code, you can use the constructor as follows:

function promiseBasedFunction(...args) {
  return new Promise((resolve, reject) => { // resolve & reject are functions, when one is called, the promise is "completed".
    callbackBasedFunction(...args, (err, result) => {
      if (err) {
        reject(err)
      } else {
        resolve(result)
      }
    });
  })
}

The constructor takes in a function that takes 2 parameters, resolve and reject, both of which are functions to be called when we have gotten back the result (or the error) from the async function.

You can call resolve without passing in the result for functions that do not produce a result, like fs.writeFile.

In TypeScript's world, since a promise might return any value, it's written as a generic class, Promise<T>. You'll need to make sure that the types of the result matches up in order to pass the type checking.

// the callback's result is of type T
function callbackBasedFunction(...args, (err : Error, result : T) => void);
// the promise must also be of the same type T
new Promise<T>((resolve : (val : T) => void, reject : (err : Error) => void) => {
  callbackBasedFunction(...args, (err, result) => {
    if (err) {
      reject(err);
    } else {
      resolve(result)
    }
  })
})

If the function doesn't return any result, the type T is void.

With this pattern you can convert any Node.js's callback-based functions into Promise-based functions.

Bluebird has nice helpers like .promisify() and .promisifyAll() to help you convert the callback-based functions to promise-based functions. However, these helpers won't automatically generate new function signatures for TypeScript, so you'll need to manually specify the additional signatures if you choose to use them.

Continue To the Next Computation

Once you created a promise, you can call its .then() to proceed to the next computation (assuming success).

.then() takes either one or two parameters:

.then(fulfilledHandler[, rejectedHandler])

If the rejectedHandler is passed in, it'll handle the error of the current promise object.

.then returns another Promise, so promises can be chained like:

promise.then(() => {
  // next computation.
})
.then(() => {
  // the following computation.
})
.then(() => {
  // ...
})

You can return either a regular value or another promise inside of the fulfilledHandler, which means that you can nest chain promises.

It's a common error to forget to return the nested promises. If you find your promise code "hangs", chances are that you have forgotten to return a nested promise somewhere in your code. A good way to alleviate this mistake is to setup the convention that all fulfilledHandler must end with a return.

Handling Errors

Error condition can be handled as part of .then as shown above, or it can be handled via .catch, which is usually specified at the end of a promise chain call, like the catch statement in a try / catch block.

promise.then(() => {
  // next computation.
})
// .then chains...
.catch((e) => {
  // handle error.
})

Without a .catch, the code can end up missing an error condition, which will be an issue if it's the top-level chain. However, it's okay as an nested chain as long as there is a higher level chain that would handle the error, similar to how try / catch works.

Providing a Callback-based API with Promises

As stated earlier, since callbacks are prevalent, you might want to provide a callback-based API of your library, but you want to take advantage of Promises. In such case you can have a callback-based version of your code that does the following:

function promiseBased(...args) {
  // this is the promise-based version.
  return promise;
}

function callbackBased(...args, cb) {
  promiseBased(...args)
    .then((res) => cb(null, res))
    .catch(cb);
})

Conclusion

The above is enough to get us started to make use of Promise, which is what we would do going forward in Module Core and beyond. Definitely refers to the official Promise documentation and Bluebird for references.