Rigging up a simple static-site generator

In many cases, coding something of your own from scratch is a bad idea, especially if it's a library that already exists. Why reinvent the wheel? Why waste the time working on your own solution when you have bigger fish to fry, and well-tested, well-documented, open-source solutions already exist?

One answer is simple: you start from scratch to learn. It's not a waste of time if you learn something. But good luck trying to justify that at your place of employment. "We could use React, or we could write our own front-end framework because then we would learn more!" That doesn't sound too convincing in almost any context. React is probably a bad example since it's a remarkably large project, but the point is the same. And when you write something yourself, you take on the responsibility of maintaining what you write - and in a workplace, the more things you've written yourself, the easier it is to incur technical debt.

But as a side-project, go ahead and implement something from scratch! You will learn more. In the end it might be largely useless, but you can work on it at your own pace, and it will give you greater insight into how other implementations work. Take for example the promise library I wrote, called denote, which I wrote mostly in one weekend just because I wanted to learn more about the Promises/A+ spec, specifically inter-library operability. The last thing anyone needs is another JavaScript promise library, but I worked on it in my spare time and I had a lot of fun doing it. #noragrets

Another good reason to write something from scratch is because you didn't find something existing that suited your purposes. When I decided I wanted to use a static-site generator for this website, I intended to use a library. There are a ton out there, and I like the looks of both Metalsmith and Wintersmith. But when I started to play around with them, I found that Metalsmith wasn't "opinionated" enough for me, and Wintersmith was too opinionated. Opinionated is a pretty loosey-goosey term; what I mean is essentially I couldn't find something that exactly fit my needs. Everything is a plugin in Metalsmith, so you have to write a config file and tell it exactly how you want to build the site and where to find all your files. This requires a lot of searching for libraries and figuring out how to write config files. It's a good approach and pretty easy to hop on board, but I wanted something a bit more seamless. Wintersmith provides that with less config and a bunch of nice things like themes and markdown support by default, but it was a bit too much for me right off the bat since I'm planning to keep doing my own styling. I hate feeling the need to rip out the stuff a framework gives me.

I took a look at the directory structure of Wintersmith, and the build it generates, and I thought to myself, "Hey, I can do a simple static-site generator that better suits my needs." So I went in with the goal of being able to write blog posts in markdown and define HTML templates with nunjucks, and I set to work.

I started by looking for a markdown-to-HTML library. I came across marked and it worked great, but went for meta-marked, which uses marked under the hood but accounts for YAML metadata at the top of markdown files.

I also used the nunjucks library of course, and the rest was basically reading and writing files. At one point I wanted to copy a directory over, and was sad that there was no fs.copy function in Node.js. So I came across the fs-extra package which adds a bunch of handy methods to the file system module, and returns promises if you don't provide a callback. So instead of writing file system code like this:

const fs = require('fs');
const marked = require('meta-marked');

fs.readFile('post1.md', 'utf8', (err, md) => {
  if(err) {
    return console.error(err);
  }
  const { html } = marked(md);
  fs.writeFile('post1.html', html, (err) => {
    if(err) {
      return console.error(err);
    }
  });
});

which quickly enters callback hell as you do more file system manipulations, you can write:

const fs = require('fs');
const marked = require('meta-marked');

fs.readFile('post1.md', 'utf8').then((md) => {
  const { html } = marked(md);
  return fs.writeFile('post1.html', html);
}).catch((err) => {
  console.error(err);
});

One of the main advantages of promises here is the single catch call for all error handling, as long as you return your promises correctly. (You could also use async and await for this, but I'm not that hip yet.) I was always hesitant to use libraries that claim to improve Node native modules, almost feeling as if I'm betraying Node, but fs-extra really helped me out here (especially for debugging) so I'll keep using it for the time being.

(For what it's worth, it's pretty easy to copy files with the default Node.js fs module. Aside from the obvious readFile followed by writeFile, you can do fs.createReadStream('file').pipe(fs.createWriteStream('copy')) which is cool because streams are cool. I used a bunch of other methods from fs-extra as well so it wasn't worth writing my own copy function; plus, I wanted to focus on writing this static-site generator! Gotta pick your learnings one at a time.)

Once I got all my dependencies hooked up I needed to determine the file structure for the website. Through a bit of trial-and-error I came up with the following structure:

site/
├── assets
│   ├── files
│   │   └── ...
│   ├── images
│   │   └── ...
│   └── styles
│       └── ...
├── index.html
└── sections
    ├── blog
    │   ├── 1.md
    │   ├── 2.md
    │   ├── 3.md
    │   ├── 4.md
    │   ├── 5.md
    │   ├── index.html
    │   └── template.html
    ├── contact
    │   └── index.html
    └── projects
        └── index.html

Everything lives in the site folder. site/index.html acts as the "master" template for all pages. Everything folder in site/sections represents a different part of your site. Each section folder needs at least an index.html nunjucks template which will become the main page of that section. Any .md files within a section folder are considered "posts" for that section. They are compiled to HTML and rendered using the template.html file found in the same section folder. template.html should inherit from the master template, site/index.html. Each section's index.html is given information about any posts in that section. For example, blog/index.html gets rendered with information about the five posts in the blog section. Everything in the assets folder is just copied directly over, so you can put any scripts and styles there and reference them in the HTML. When I run my script on the site folder, the following build results:

build/
├── blog
│   ├── A-new-era-begins
│   │   └── index.html
│   ├── I-Let-It-Die
│   │   └── index.html
│   ├── index.html
│   ├── Rigging-up-a-simple-static-site-generator
│   │   └── index.html
│   ├── The-Glory-of-Reading-Week
│   │   └── index.html
│   └── Update-ALL-the-projects
│       └── index.html
├── contact
│   └── index.html
├── files
│   └── thesched.pdf
├── images
│   ├── ...
├── projects
│   └── index.html
└── styles
    ├── styles.css
    └── tomorrow.css

Everything from the build folder can just be copied to a web server. There is a folder for each section and a folder for each folder in assets.

You can view the entire script for the site generation on GitHub. There's still some work to do on it and it's not perfect, but it's short, was easy to write, and serves my purposes almost exactly. Once I polish it up I'll probably extract it into its own package with a command line utility and publish it on NPM.