Module Core - Session 2 - Extend Routing

Welcome to Session 2, we will be continuing where we left off of from Session 1.

Goal

In session 1, we covered the following:

Although a hello world server seems simple - it's certainly the seed in which that we can grow into a complex web app, so in session 2 we will take the next logical step.

We will try to implement what's knowing as "routing" in Node.js.

HTTP Routing

What does "routing" mean in Node.js? In the simplest terms, it means that given different URL paths, the server should take different actions, and each action is a "route".

The term "dispatch" was more prevalent before mid-2000's, and I'm more used to that term so I'll be using them interchangeably unless otherwise noted. i.e. a dispatcher == a router.

A concrete example - let's say I have two files in my web server, index.html and logo.png, then when the user requests for /index.html, the index.html file should be served, and when the user requests for /logo.png, the logo.png should instead be served.

While the above makes sense, obviously if we don't change our existing applications, it would always do the same thing (serve the read.js script) no matter what path we request. While a full blown web server comes with this seemingly incredibly obvious capability built-in (how else should it work???), since we are programming our own web server, we need to provide it ourselves.

This is the difference between "using" something versus "building" something. When you use something, there are conveniences done for you, but you are at the whim of the creator's design. When you build something, you have all the freedom in the world, but freedom isn't free.

Simple Dispatcher

So let's get started with a simple dispatcher. We'll be dispatching to a /foo endpoint that simply prints out a message that we have reached Foo, and a /bar endpoint that prints out the message that we have reached Bar, with all the rest of endpoints going to the original function. It'll look like the following:

var server = http.createServer(function (req, res) {
  switch (req.url) {
  case: '/foo':
    // the `/foo endpoint handler
  case '/bar':
    // the `/bar endpoint handler
  default:
    // the handler function for the rest of the endpoints.
  }
})

The /foo handler would be as simple as:

    res.setHeader('content-type', 'type/plain');
    return res.end('You have reached page foo');

The /bar handler would be:

    res.setHeader('content-type', 'type/plain');
    return res.end('You have reached bar');

Then default handler would be the same as the old handler from Module 1.

When you restart the server and test for /foo and /bar in the browser, you'll see the corresponding text being printed out, which shows us that the pattern of a big switch statement can serve as the basis of creating a complete dispatcher. We can further refactor the handlers into their own functions as well.

function fooHandler(req, res) {
  res.setHeader('content-type', 'type/plain');
  return res.end('You have reached page foo');
}

function barHandler(req, res) {
  res.setHeader('content-type', 'type/plain');
  return res.end('You have reached bar');
}

function defaultHandler(req, res) {
	// pull the default code here as an exercise.
}

And we can extend the pattern indefinitely by adding more and more handlers to match them against the url paths.

Obviously, we can't be satisfied with such a dispatcher, because:

So it means that we would want to search for other solutions, but manually writing out a basic dispatcher teaches us what is fundamentally we are trying to do.

Express & NPM to the rescue

As we said earlier, it seems that routing, at least serving static files, is something that a full-blown web server does, and feels like something that ought to be built-in. If not built-in, at least there should have been many attempted solutions, and potentially some might have already been published and shared.

It turns out that there is, and quite many as well.

If you go onto npmjs.com and search for routing, you'll find quite a few solutions. As stated before, npm is currently the largest code repository in the world, and chances are you can find at least someone have published a package attempting a solution at what you are trying to do (the quality though isn't guaranteed).

In the HTTP routing space, express arguably is the most popular web framework in Node.js. It considers itself a minimal framework (meaning that it doesn't really dictate and change how you would write your code), and having used it extensively I believe it's a valid claim (personally I usually avoid opinionated framework). Many other frameworks are built on top of express, so learning how to use it will serve as a good basis for using other frameworks should you desire arise.

Let's see how we would do the same thing above via express.

First we'll have to get express installed, and that means that we need to learn how to use npm.

How to use NPM

It's pretty easy to install express via npm - you just need to be sure to do so in the root of your project directory:

npm install express

Once you have done so, you should see some output that shows what you have installed if it's successful, or print out error messages when it's not.

You are also likely to be given warnings that you are lacking a package.json file. What is a package.json file?

NPM and Package.json

The existence of a package.json helps to inform npm that the files inside this directory is considered a "package", which is a unit of code distribution for npm. Remember, there are hundreds of thousands of packages that live in npm, and each package is a collection of files, in which one of them would be called package.json, and it contains metadata information about the package. Things like the name of the package, its version, its author, dependencies, etc.

Even if you aren't considering releasing your package to npm, it's still a good idea to make use of package.json to maintain the basic metadata.

For example, if you use package.json to maintain the dependency list, instead of typing out the dependencies manually like:

npm install <dependency 1> <dependency 2> ... <dependency N>

You can just type:

npm install

and let npm install all the listed dependencies in package.json.

It's easy to create a package.json. As it's a regular JSON file with some specific keys and values, you can just do so via any text editor if you know the keys. Or if you want a wizard to walk you through the process, just type:

npm init

And then just follow the instructions.

The wizard isn't going to ask you for a list of dependencies though. To add dependencies, use the following command:

npm install --save <package>

If it's a "dev" dependency (meaning that the module is meant for development purpose only and not important in a production environment, like a test utility), then you use the following command:

npm install --save-dev <package>

The two commands have the equivalent effect as:

  1. download and install the package like npm install package
  2. edit package.json by adding the new dependency to either the dependencies field, or the devDependencies field.

If you want to remove a particular package from package.json, use one of the following to remove from either dependencies or devDependencies.

npm uninstall --save <package>

npm uninstall --save-dev <package>

So after you have created package.json either manually or via npm init, add express via:

npm install --save express

NPM Scripts

Besides the basic package metadata and the dependencies, package.json has other uses, one of which is to be used a "shortcut" for running scripts.

We have been typing out the following to run our web app:

node ./server.js

That requires us to remember the script's name and path to get to them. While it's simple in this case, sometimes the script can be quite complex, potentially with many default parameters.

Instead, we can leverage package.json's scripts field.

If you use npm init, a test key with an error message is already pre-populated for you as a template:

  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  }

To simplify what we do to start our server, add the following to the scripts field:

    "start": "node ./server.js",

Keep the , above if you add before test, and remove it if you add at the end of the object to comply with the JSON format - and remember, no comments in JSON, either.

Then you can simply run:

npm start

to run your web service!

You can do the same with any other commands, however, npm has a "limitation" in that it only has a short list of recognized commands that you can run under the pattern of npm <command>. This is easily verified via typing in an arbitrary command, like:

npm something # note that it's not in angular bracket - yes "something" is the command

And you'll get back a usage output that show you a list of recognized commands.

If you add a command to scripts that isn't already one fo the recognized commands, you need to type:

npm run <command>
# or
npm run-script <command>

in order to run it. It's less appealing than just npm <command>, but as long as your command is long, the shortcut might still be worth it.

Rewrite the Dispatcher in Express

So, let's redo our simple dispatcher with express.

First - require the express package.

var express = require('express');

Second - create an app object with express.

var app = express(); // the express module is a function.

This app object is the top level "express application object". It is also a router / dispatcher that we can add routes to it.

So the next step is to replace the current dispatcher function with app as follows:

var server = http.createServer(app);

Finally, we can add the routes to app as follows:

app.get('/foo', fooHandler); // equivalent to the case '/foo' above

app.get('/bar', barHandler);

app.use(defaultHandler);

The get function of app means that you are setting up a route for handling HTTP method GET. The first parameter takes the route. In the example, we have the static routes /foo and /bar, but express can handle dynamic routes too (we'll cover that in the future). The second parameter is the handler, which we are able to use our already-written fooHandler and barHandler without change.

This is why express is minimal. It's very similar to how one would write the routes without it.

Yes, functions like post, put, and head exists for other default HTTP methods as well.

Finally, we also see the use function, which means that it's used for all methods. In this example we are seeing it called without a route, and that means it's used for all routes.

It's possible to use get, post, put, and head without routes as well.

With the above we are done converting our dispatcher over to express. Let's rerun npm start, and you can verify that the code is still behaving the same as before the rewrite.

Middlewares & Error Handlers

With express, you can write your dispatchers much more flexibly - you can handle multiple HTTP methods, multiple routes, dynamic routes, programmatically adding routes, and adding routes at multiple files (not just in a single large switch statement).

Beyond that express further brought you additional concepts that you can leverage for code organization, and it's called "middlewares".

A middleware is a "handler" function that you can add before your "final handler", like this:

app.get('/foo', <middleware 1>, <middleware 2>, ... <middleware N>, fooHandler); // fooHandler is the final handler for /foo

The number of middleware is limited by the number of parameters JavaScript could take as a function, which should be enough for most practical purposes.

Each of the middleware has the following signature:

function (req, res, next) {

}

The difference versus the final handler is that the middlewares as a next function, which helps transition the call to the next middleware in the chain above, or transition to handling errors.

function (req, res, next) {
  next(); // if success - go to the "next" middleware.
  next(<some error>); // if failure - go to the "error" middleware.
}

You must call next somewhere in your middleware, or it effectively degenerates to a final handler. As a habit I usually write the final handler with the same signature as well, since I often want to explicitly handle passing the error via next.

The error middleware has the following signature:

function (err, req, res, next) {

}

express comes with a default error middleware .If you want to overwrite what the default error middleware does, then create a function with the above signature, and add it to app as the last route (so it doesn't escape beyond this function).

function errorMiddleware(err, req, res, next) {
  res.statusCode = 500;
  res.end('This is the default error middleware');
}

app.use(errorMiddleware); // after all other routes.

To test that we can add a route that specifically throws an error:

function throwsErrorIfReached(req, res) {
  throw new Error("I am reached");
}

app.get('/error', throwsErrorIfReached); // add this before the defaultHandler.

Notice that we have specified the orders of where to add the routes. So far we want the final 3 routes to be

app.get('/error', throwsErrorIfReached);
app.use(defaultHandler);
app.use(errorMiddleware);

In express, the order of the routes matter in case of conflicts. The first one that matches win. So in the above, defaultHandler matches anything (besides error condition) because of use without a route, so in order for throwsErrorIfReached to get triggered, it must be added before the "everything else" route. The errorMiddleware route also matches everything plus the error condition, so it comes after error handler.

Restart the service and try to request for /error - you'll see the error message being printed.

Conclusion

The above shows us that we can do quite a bit of express. When combined with the available third-party middlewares written for express framework, it certainly makes express a powerful framework as a foundation that we can built on top of. We will introduce other third-party middlewares, and create our own as necessary as we continue further into our journey.

This concludes our goal in this session, so we will continue at another time.