Abstraction for the sake of Abstraction

I wanted to talk a bit about task runners and automation. Personally, I've utilized NPM and GruntJS to automate repetitive tasks on multiple projects. That includes file compression, JavaScript/PHP linting, Sass compilation, script/style minification, amd running PHP/JS unit tests. I've even used it to keep mirrored SVN and Git repos in sync.

Having just started a new project, I went back to my previous projects for a refresher. I didn't want to reinvent the wheel, so I was going to scavenge whatever I needed from the old projects. Looking back, I used the following NPM packages:

grunt
grunt-cli
grunt-concurrent
grunt-contrib-concat
grunt-contrib-cssmin
grunt-contrib-jshint
grunt-contrib-sass
grunt-contrib-uglify
grunt-contrib-watch
grunt-exec
grunt-mocha-test
load-grunt-tasks
time-grunt
sinon
jsdom
mocha
chai

That didn't seem too bad, but then I looked at the Gruntfile. It was 195 lines long! One of my other projects used less packages, but the Gruntfile was 425 lines long...Yeah.

I came to the sudden realization that I had put a ton of effort into utilizing Grunt for these projects. The reason being that I was learning to use NPM and Grunt and the like. Although it was useful for learning automation and how to write unit tests, it was otherwise unnecessary. Nearly all the tasks that I needed to accomplish could be accomplished without Grunt. I realized that I was adding abstraction for the sake of abstraction.

When that bubble popped, I remembered the NPM has built-in functionality for running commands using npm run-script. Yup, I totally forgot about that!

So, I thought, "how could this be done with NPM only?"

Here's what I've come up with, for the above project, with comparisons between using Grunt and the default NPM run-script feature.

Sass Compilation and Minification

Grunt setup:

require( 'load-grunt-tasks' )( grunt );

grunt.initConfig({
    paths: {
        sass: {
            dir: 'styles',
            files: '<%= paths.sass.dir %>/**/*.scss'
        },
        css: {
            dir: 'public/styles'
        },
    },
    sass: {
        options: {
            style: 'expanded'
        },
        dist: {
            files: [{
                expand: true,
                cwd: '<%= paths.sass.dir %>',
                src: ['**/*.scss'],
                dest: '<%= paths.css.dir %>',
                ext: '.css'
            }]
        }
    }
});

grunt.registerTask( 'scss', 'sass' );

Grunt usage:

$ grunt scss

NPM setup:

"scripts": {
    "sass": "sass --scss -t compressed styles/*.scss public/styles/style.min.css"
}

NPM usage:

$ npm run sass

JavaScript Linting, Compression, and Minification

Grunt setup:

require( 'load-grunt-tasks' )( grunt );

grunt.initConfig({
    paths: {
        js: {
            source: 'scripts/*.js',
            public_dir: 'public/scripts/',
            public_dest: '<%= paths.js.public_dir %>main.js',
            public_ugly: '<%= paths.js.public_dir %>main.min.js',
            files: [
                '<%= paths.js.source %>',
                'Gruntfile.js',
                'test/**/*.js',
                '!test/utils/**/*.js'
            ]
        },
    },
    concat: {
        js: {
            src: '<%= paths.js.source %>',
            dest: '<%= paths.js.public_dest %>'
        }
    },
    uglify: {
        options: {
            mangle: {
                except: ['jQuery']
            }
        },
        target: {
            files: {
                '<%= paths.js.public_ugly %>': ['<%= paths.js.public_dest %>']
            }
        }
    },
    jshint: {
        options: {
            curly: true,
            eqeqeq: true,
            browser: true,
            devel: true,
            undef: true,
            unused: false,
            mocha: true,
            globals: {
                'jQuery': true,
                'module': true,
                'require': true,
                'window': true,
                'global': true
            }
        },
        dist: '<%= paths.js.files %>'
    }
});

grunt.registerTask( 'js', [ 'jshint', 'uglify', 'concat' ] );

Grunt usage:

$ grunt js

NPM setup:

"scripts": {
    "js": "jshint scripts/*.js test/*.test.js && uglifyjs scripts/*.js -cmo public/scripts/word_search.min.js"
}

NPM usage:

$ npm run js

Mocha Unit Tests

Grunt setup:

require( 'load-grunt-tasks' )( grunt );

grunt.initConfig({
    paths: {
        test: {
            files: 'test/**/*.test.js'
        },
    },
    mochaTest: {
        test: {
            options: {
                reporter: 'spec',
                require: 'test/utils/jsdom-config.js'
            },
            src: '<%= paths.test.files %>'
        }
    }
});

grunt.registerTask( 'mocha', 'mochaTest' );

Grunt usage:

$ grunt mocha

NPM setup:

"scripts": {
    "test": "mocha -R spec -r test/utils/jsdom-config.js test/*.test.js"
}

NPM usage:

Since npm run-script accepts test as a predefined task, the command is even shorter!

$ npm test

Deployment

My source directory is right next to my distribution directory. The Grunt task for deployment requires the grunt-exec dependency, which allows running command line expressions. The deploy task runs all the linting, uglification, and sass compilation before copying the public/ directory to the distribution directory. For the sake of brevity, I'm not going to list all the tasks out again, just the key ones.

Grunt setup:

require( 'load-grunt-tasks' )( grunt );

grunt.initConfig({
    pkg: grunt.file.readJSON( 'package.json' ),
    paths: {
        host: {
            dir: '../foresthoffman.github.io/<%= pkg.name %>/'
        },
        source: {
            dir: 'public/'
        }
    },
    /* Other task definitions */
    exec: {
        copy: {
            cmd: function () {
                var host_dir_path = grunt.template.process( '<%= paths.host.dir %>' );
                var source_dir_path = grunt.template.process( '<%= paths.source.dir %>' );

                var copy_command = 'cp -r ' + source_dir_path + ' ' + host_dir_path;

                return copy_command;
            }
        }
    },
});

grunt.registerTask( 'build', [
    'jshint', 
    'sass', 
    'cssmin', 
    'mochaTest', 
    'concat', 
    'uglify', 
    'exec:zip'
]);
grunt.registerTask( 'deploy', [ 'build', 'exec:copy' ] );

Grunt usage:

$ grunt deploy

NPM setup:

I made a simple bash script, so that I could completely drop Grunt for this task.

##
# cp_public
##

if [ ! $# == 2 ] || [ ! -d $1 ] || [ ! -d $2 ]; then
    echo "cp_public /path/to/source /path/to/dist"
    exit
fi

# the name property line from the package.json file in the current directory
name_line=$(grep -Ei "^\s*\"name\":\s\".*\",$" package.json | grep -oEi "[^\",]+")

# the package name by itself
name=$(echo $name_line | cut -d " " -f 3)

path_reg="\/"

source_path="$(echo ${1%$path_reg})/"
dist_path="$(echo ${2%$path_reg})/$name"

cp -r $source_path $dist_path

"scripts": {
    "deploy": "./cp_public public/ ../foresthoffman.github.io/"
}

NPM usage:

$ npm run deploy

Watchers

Rather than using Grunt to run concurrent file watchers I opted to use the npm-watch package. This allows me to indicate X number of scripts (from the scripts property of my package.json file) and the files that should trigger the scripts while the watcher is active. I only really needed to have the watcher handle js files changing, since the sass command has it's own --watch argument.

Then the configuration for the watch statement looks like this:

"watch": {
    "js": "scripts/*.js"
},
"script": {
    "js": "uglifyjs scripts/*.js -cmo public/scripts/main.min.js",
    "watch": "npm-watch"
}

Which can then be run via, npm run watch.

While writing this I actually implemented the changes that I've mentioned here. I've deleted my Gruntfile and am left with the configurations in my package.json and my new bash script in cp_public. Everything is working as intended. Woot!

package.json...

"devDependencies": {
    "chai": "^3.4.1",
    "jsdom": "^7.2.2",
    "mocha": "^2.3.4",
    "npm-watch": "^0.1.7",
    "sinon": "^1.17.2"
}
...
"watch": {
    "js": "scripts/*.js"
},
"scripts": {
    "deploy": "./cp_public public/ ../foresthoffman.github.io/ || true",
    "sass": "sass --scss -t compressed styles/*.scss public/styles/style.min.css",
    "sassWatch": "sass --watch --scss -t compressed styles/style.scss:public/styles/style.min.css",
    "js": "uglifyjs scripts/*.js -cmo public/scripts/main.min.js",
    "test": "mocha -R spec -r test/utils/jsdom-config.js test/*.test.js || true",
    "testWatch": "mocha -w -R spec -r test/utils/jsdom-config.js test/*.test.js || true",
    "watch": "npm-watch"
}

and cp_public...

#!/bin/bash

##
# cp_public
#
# Copies the source directory to the target directory.
#
# Usage: cp_public /path/to/source /path/to/dist
# 
# The package.json file, from which the package name is collected is relative to where this script
# is executed. It uses the current directory. That is why this script should be placed next to the
# package.json file in the project's hierarchy.
##

if [ ! $# == 2 ] || [ ! -d $1 ] || [ ! -d $2 ]; then
    echo "cp_public /path/to/source /path/to/dist"
    exit
fi

# the name property line from the package.json file in the current directory
name_line=$(grep -Ei "^\s*\"name\":\s\".*\",$" package.json | grep -oEi "[^\",]+")

# the package name by itself
name=$(echo $name_line | cut -d " " -f 3)

path_reg="\/"

source_path="$(echo ${1%$path_reg})/"
dist_path="$(echo ${2%$path_reg})/$name"

cp -r $source_path $dist_path

That's all folks!

I will admit that there aren't many good packages for concurrency without having to take on a lot more dependencies. In that way using Grunt (or another runner) is beneficial. Personally, I can no longer justify adding that complexity for what seems like a relatively minute benefit.

I like tooling and automation. I think writing unit tests is a freaking fantastic and very rewarding endeavor. However, I feel that there is a point at which the tools we use to automate rudimentary tasks add too much complexity. I'm seeing more and more of this everyday, thanks to the speed at which the web development ecosystem is progressing.

Anyone else feel the same? Anyone stuck in situation where they can't actually justify removing these kinds of dependencies? Anyone still a huge fan of runners, and will use them in future projects?

Obligatory xkcd comic plug:

Blog

Abstraction for the sake of Abstraction

Forest Hoffman

Sass Compilation and Minification

JavaScript Linting, Compression, and Minification

Mocha Unit Tests

Deployment

Watchers

That's all folks!

Join Our Newsletter. No Spam, Only the good stuff.

Related