Welcome to Session 3. We will continue to push our small web server toward our dream app.
In Session 2, we covered the following:
npm
express
Knowing how to route give us the blueprint on how to structure the rest of the web application. We will continue to examine how to handle other common routing scenarios:
We have actually see a basic version of this already - in Session 1 we have been reading the read.js
and serving it out as a plain text file, however, we are only serving a single file no matter what URL paths was specified, so we need to change that to the following:
Let's make that a bit more concrete, let's say that I have a folder called static
, and it has a couple of files:
And I want to map the URL /index.html
to static/index.html
, and then /test.html
to static/test.html
, in a general pattern being that anything under /
should be mapped to the static
folder.
The structure of the handler would then look like this.
// NOTE that I'm using next in a handler to handle async errors that would be lost otherwise.
function staticFileHandler(req, res, next) {
var mappedPath = mapUrlToFilePath(req.url); // translates the url to underlying file path.
fs.readFile(mappedPath, 'utf8', function (err, data) {
if (err) {
next(err);
} else {
res.end(data);
}
});
}
Then we just need to define the mapUrlToFiePath
function to do what we want. For now, we assume that the static
folder is inside the project folder, so we can hardcode that with __dirname
.
function mapUrlToFilePath(url) {
// as index.js is at project root, this translates to <project root>/static/<url>
return path.join(__dirname, 'static', url);
}
Then we just need to include it in the express routes.
app.get(staticFileHandler);
Restart and test that /index.html
and /test.html
returns you the desired files, but /bar.html
will throw an error as there is no bar.html
in the static
folder.
We should test that with other types of files as well, like for example, images.
Create a test.png
file and put it into the static
folder, then try to retrieve it via /test.png
. It turns out that in this case we would end up with a corrupted image that we cannot open (try it - your image viewer will complain that it's a corrupted file).
The reason it's corrupted is because that when we read the file content via fs.readFile
, we have passed in the encoding of utf8
, which will treat the read-in file as text. That works for text files, but doesn't work for images, which is just a binary file. So let's remove the encoding.
fs.readFile(mappedPath, function (err, data) {
When fs.readFile
is called without the encoding, it returns a Buffer
object instead of a string
, and the Buffer
object simply holds the binary data as is. Luckily we don't have to change our output function, as res.end
can take either a string
or Buffer
.
Restart the server and now you'll see /test.png
retrieves a valid image.
express
Serves Static FilesAs stated before, serving static file is a very common use case, so express
has this particular functionality via express.static
like below:
app.use(express.static(<the root path of the static file folder>))
I.e, with our existing root path of the static
folder, it would be:
app.use(express.static(path.join(__dirname, 'static')));
Which is the equivalent of what we have done, except that express.static
has a different error handler for non-existent files.
Since raw html files are cumbersome to write, people have invented a few different ways of creating the HTML files, some of which were WYSIWYG-based, while others are text-based.
WYSIWYG-based means that we need to create a WYSIWYG editor for editing the HTML, that's a more involved process. So let's first address the simpler process, which is to have a text-based markup file that can get converted over to HTML. Markdown has emerged as the go-to format for such choices.
Therefore, besides serving out raw static files, we also want to serve out the markdown files. But we will not do it in the format that we would write in, but in HTML format after it's been converted.
Let's tackle this step by step. Before we try to serve out the dynamically generated HTML, let's just first be sure that we could serve out the raw markdown files.
Let's create a couple of markdown files in a different directory, call it content
.
test.md
test2.md
Then let's add another express.static
route that points to the content
directory.
app.use(express.static(path.join(__dirname, 'content')));
As stated,
express
routes are prioritized by the order, and the first one wins, so put it to where you believe it should go in terms of priority.
After restarting the app you'll be able to point to /test.md
and /test2.md
in your browser and see the raw markdown files that you have added.
Unfortunately, express.static
doesn't give us a hook to transform the files while it's being served, so we'll have to go back to make use of our raw static file handler to do our own file handling.
The first thing we need to do is to bring it to par, since we have previously hardcoded the static
path into the functions, so we must refactor it in order for us to pass in the content
directory.
function mapUrlToFilePath(url, folderName) {
return path.join(__dirname, folderName, url); // refactor static into a parameter.
}
Then we must change our handler to a handler factory that takes in the folderName
variable and creates the handler.
function configurableStaticFileHandler(folderName) {
return function staticFileHandler(req, res, next) {
var mappedPath = mapUrlToFilePath(req.url, folderName);
fs.readFile(mappedPath, function (err, data) {
if (err) {
next(err);
} else {
res.end(data)
}
})
}
}
And change the actual route instantiation to:
app.use(configurableStaticFileHandler('content'));
Restart the web server to verify that we have fully replaced express.static
call with our own configurableStaticFileHandler
.
The last step then is to process the data
parameter before it gets returned via res.end
. This is the place where we can transform the markdown into HTML.
// something like this...
res.end(markdownToHtml(data))
Let's find ourselves a markdown parser and transformer - as a popular format there are already plenty third-party libraries we could use. We'll use the one called markdown-it
(you can use other ones if you choose to, there are no harms to try multiple libraries to find the one that you like).
npm install --save markdown-it
Then load the library:
var MarkdownIt = require('markdown-it');
Then make use of it based on its documentation, and change the encoding back to utf8
since we are now dealing with text:
function configurableStaticFileHandler(folderName) {
return function staticFileHandler(req, res, next) {
var mappedPath = mapUrlToFilePath(req.url, folderName);
let md = new MarkdownIt();
fs.readFile(mappedPath, 'utf8', function (err, data) {
if (err) {
next(err);
} else {
res.end(md.render(data))
}
})
}
}
Save and restart the web server and browse to /test.md
, voila, we are now seeing the generated HTML file!
Since we change the encoding back to utf8
, we are encountering the same issue as before that this handler could only serve text files. Perhaps in the future we might want to consider handling binary files and text files through this handler as well. To do so, let's remove utf8
again, but we now need to determine if the file is a text file or binary file, so we can decide how we need to process it.
A simple way to determine the processing rule is via the file extension, such as:
.md
- markdown file, transform into html..html
- html file, pass through.png
- image file, pass throughWith that we can rewrite our data handling branch as follows:
...
fs.readFile(mappedPath, 'utf8', function (err, data) {
if (err) {
next(err);
} else {
switch (path.extname(data)) {
case '.md':
try {
return res.end(md.render(data.toString('utf8'))) // use Buffer's toString('utf8') to get the text.
} catch (e) { // render could fail synchronously, so try / catch
return next(e);
}
default:
return res.end(data);
}
}
})
}
}
With this new function, we could actually combine the static
folder, which holds the HTML and image files, with the content
folder, which holds the markdown files. And we could also consolidate the two routes into one.
Although, keep in mind that if we only keep our static handler route, it does not allow serving out of the raw markdown files anymore. We might need to add it back in in the future when we need to serve out the raw markdown files.
With the ability to serve out raw files and generated files, we basically have a very primitive CMS (Content Management System) in our hands. We can write out the files in either Markdown or HTML, and then serve them out with our web server. Although the pages would be really ugly at this time, and there are still other functionalities that we want (for example, allow us to write file snippets and combine them together), they can be built on top of what we have done so far, and we just need to keep enhance our system to get there.
Let's continue in the next session.