In this section, we will install and configure the tools that we will use to build our package. We will assume that you know how to use the command line and you have Node installed on your system. You can install Node by either following the instructions from Node.js's website (http://nodejs.org/download/) or using a package manager in Unix-like systems.
The Node Package Manager (npm) is a program that helps us to manage dependencies between Node projects. As our project could be a dependency of other projects, we need to provide information about our package to npm. The package.json file is a JSON file that should contain at least the project name, version, and dependencies for use and development. For now, we will add just the name and version:
{
"name": "windmill",
"version": "0.1.0",
"dependencies": {},
"devDependencies": {}
}We will install Grunt, Vows, Bower, and D3 using npm. When installing a package, we can pass an option to save the package that we are installing as a dependency. With the --save-dev option, we can specify the development dependencies:
$ npm install --save-dev grunt vows bower
D3 will be a dependency for our package. If someone needs to use our package, D3 will be needed; the previous packages will be necessary only for development. To install the project dependencies, we can use the --save option:
$ npm install --save d3
A directory named node_modules will be created at the topmost level of the project. This directory will contain the installed modules:
node_modules/ bower/ d3/ grunt/ vows/
The package.json file will be updated with the dependencies as well:
{
"name": "windmill",
"version": "0.1.0",
"dependencies": {
"d3": "~3.4.1"
},
"devDependencies": {
"grunt": "~0.4.2",
"vows": "~0.7.0"
}
}Note that the dependencies specify the version of each package. Node.js packages should follow the Semantic Versioning specification. We will include additional modules to perform the building tasks, but we will cover that later.
Grunt is a task runner for Node.js. It allows you to define tasks and execute them easily. To use Grunt, we need to have a package.json file with the project information and the Gruntfile.js file, where we will define and configure our tasks. The Gruntfile.js file should have the following structure; all the Grunt tasks and configurations should be in the exported function:
module.exports = function(grunt) {
// Grunt initialization and tasks
};The Grunt tasks may need configuration data, which is usually passed to the grunt.initConfig method. Here, we import the package configuration from the package.json file. This allows you to use the package configuration values in order to generate banners in target files or to display information in the console when we run tasks:
module.exports = function(grunt) {
// Initialize the Grunt configuration
grunt.initConfig({
pkg: grunt.file.readJSON('package.json')
});
};There are hundreds of Grunt plugins to automate every development task with minimal effort. A complete list of the Grunt plugins is available at http://gruntjs.com/plugins.
The grunt-contrib-concat plugin will concatenate our source files for us. We can install the plugin as any other Node.js module:
$ npm install --save-dev grub-contrib-concat
To use the plugin, we should enable it and add its configuration as follows:
module.exports = function(grunt) {
grunt.initConfig({
pkg: grunt.file.readJSON('package.json'),
concat: {
// grunt-contrib-concat configuration...
}
});
// Enable the Grunt plugins
grunt.loadNpmTasks('grunt-contrib-concat'),
};We add the grunt-contrib-concat configuration. The concat object can contain one or more targets, each containing an array of sources and a destination path for the concatenated file. The files in src will be concatenated in order:
// Initialize the Grunt configuration
grunt.initConfig({
// Import the package configuration
pkg: grunt.file.readJSON('package.json'),
// Configure the concat task
concat: {
js: {
src: [
'src/start.js',
'src/svg/svg.js',
'src/svg/transform.js',
'src/chart/chart.js',
'src/chart/heatmap.js',
'src/layout/layout.js',
'src/layout/matrix.js',
'src/end.js'
],
dest: 'windmill.js'
}
},
});There are options to add a banner too, which can be useful to add a comment indicating the package name and version. We can run the concat task from the command line as follows:
$ grunt concat Running "concat:js" (concat) task File "windmill.js" created. Done, without errors.
If we have several targets, we can build them individually by passing the target after the concat option:
$ grunt concat:js Running "concat:js" (concat) task File "windmill.js" created. Done, without errors.
The windmill.js file contains the sources of our package concatenated in order, preserving the original spaces and comments.
It's common practice to distribute two versions of the library: one version with the original format and comments for debugging and another minified version for production. To create the minified version, we will need the grunt-contrib-uglify plugin. As we did with the grunt-contrib-concat package, we need to install this and enable it in the Gruntfile.js file:
module.exports = function(grunt) {
// ...
// Enable the Grunt plugins
grunt.loadNpmTasks('grunt-contrib-concat'),
grunt.loadNpmTasks('grunt-contrib-uglify'),
};We need to add the uglify configuration to the grunt.initConfig method as well. In uglify, we can have more than one target. The options attribute allows us to define the behavior of uglify. In this case, we set mangle to false in order to keep the names of our variables as they are in the original code. If the mangle option is set to true, the variable names will be replaced with shorter names, as follows:
// Uglify Configuration
uglify: {
options: {
mangle: false
},
js: {
files: {
'windmill.min.js': ['windmill.js']
}
}
}We can run the minification task in the command line using the same syntax as in the concatenation task:
$ grunt uglify Running "uglify:js" (uglify) task File windmill.min.js created. Done, without errors.
This will generate the windmill.min.js file, which is about half the size of the original version.
In JavaScript, it is very easy to write code that doesn't behave as we expect. It could be a missing semicolon or forgetting to declare a variable in a certain scope. A linter is a program that helps us to detect potential errors and dangerous constructions by enforcing a series of coding conventions. Of course, a static code analysis tool can't detect these if your program is correct. JSHint is a tool that helps us to detect these potential problems by checking the JavaScript code against code conventions. The behavior of JSHint can be configured to match our coding conventions.
In JSHint, the code conventions can be set by writing a jshintrc file, a JSON file containing a series of flags that will define the JSHint behavior. For instance, a configuration file might contain the following flags:
{
"curly": true,
"eqeqeq": true,
"undef": true,
// ...
}The curly option will enforce the use of curly braces ({ and }) around conditionals and loops, even if we have only one statement. The eqeqeq option enforces the use of === and !== to compare objects, instead of == and !=. If you don't have a set of coding conventions already, I would recommend that you read the list of JSHint options available at http://www.jshint.com/docs/options/ and create a new .jshintrc file. Here, the options are listed and explained, so you can make an informed decision about which flags to enable.
Many editors have support for live linting, but even if the text editor checks the code as you write, it is a good practice to check the code before committing your changes. We will enable the grunt-contrib-jshint module and configure it so that we can check our code easily. We will enable the plugin as follows:
// Enable the Grunt plugins
grunt.loadNpmTasks('grunt-contrib-concat'),
grunt.loadNpmTasks('grunt-contrib-uglify'),
grunt.loadNpmTasks('grunt-contrib-jshint'),
Next, we configure the plugin. We will check the Gruntfile.js file, our tests, and the code of our chart:
jshint: {
all: [
'Gruntfile.js',
'src/svg/*.js',
'src/chart/*.js',
'src/layout/*.js',
'test/*.js',
'test/*/*.js'
]
}We can check our code using the following command lines:
$ grunt jshint Running "jshint:all" (jshint) task >> 11 files lint free. Done, without errors.
Distributing a software package is a great responsibility. The users of our charting package rely on our code and assume that everything works as expected. Despite our best intentions, we may break the existing functionality when implementing a new feature or fixing a bug. The only way to minimize these errors is to extensively test our code.
The tests should be easy to write and run, allowing us to write the tests as we code new features and to run the tests before committing changes. There are several test suites for JavaScript code. In this section, we will use Vows, an asynchronous behavior-driven test suite for Node.js.
In Vows, the largest test unit is a
suite. We will begin by creating a simple test using JavaScript without any libraries. In the test directory, we create the universe-test.js file.
We will load the vows and assert modules and assign them to the local variables:
// Load the modules
var vows = require('vows'),
assert = require('assert'),We can create a suite now. The convention is to have one suite per file and to match the suite description with the filename. We create a suite by invoking the vows.describe method:
// Create the suite
var suite = vows.describe('Universe'),Tests are added to the suite in batches. A suite can have zero or more batches, which will be executed sequentially. The batches are added using the suite.addBatch method. Batches allow you to perform tests in a given order:
suite.addBatch({
//...
});A batch, in turn, contains zero or more contexts, which describe the behaviors or states that we want to test. Contexts are run in parallel, and they are asynchronous; the order in which they will be completed can't be predicted. We will add a context to our batch, as follows:
suite.addBatch({
'the answer': {
//...
}
});A context contains a topic. The topic is a value or function that returns an element to be tested. The vows are the actual tests. The vows are the functions that make assertions about the topic. We will add a topic to our context as follows:
suite.addBatch({
'the answer': {
topic: 42,
//...
}
});In this case, all our vows in the context the answer will receive the value 42 as the argument. We will add some vows to assert whether the topic is undefined, null, or a number, and finally, whether the topic is equal to 42. Refer to the following code:
suite.addBatch({
'the answer': {
topic: 42,
"shouldn't be undefined": function(topic) {
assert.notEqual(topic, undefined);
},
"shouldn't be null": function(topic) {
assert.notEqual(topic, null);
},
"should be a number": function(topic) {
assert.isNumber(topic);
},
"should be 42": function(topic) {
assert.equal(topic, 42);
}
}
});To execute all the tests in the test directory as a single entity (instead of having to run each one separately), we need to export the suite:
suite.export(module);
We can run these tests individually by passing the test path as the argument:
$ vows test/universe-test.js --spec ♢ Universe the answer to the Universe √ shouldn't be undefined √ shouldn't be null √ should be a number √ should be 42 √ OK » 4 honored (0.007s)
We will temporarily modify our topic to introduce an error as follows:
suite.addBatch({
'the answer': {
topic: 43,
//...
}
});The output of the test will show which vows were honored and which failed, displaying additional details for the broken vows. In this case, three vows where honored and one was broken.
♢ Universe the answer √ shouldn't be undefined √ shouldn't be null √ should be a number x should be 42 » expected 42, got 43 (==) // universe-test.js:27 x Broken » 3 honored ∙ 1 broken (0.564s)
This simple example shows you how to create a suite, context, topics, and vows to test a simple feature. We will use the same structure to test our heat map chart.
The tests for the heat map chart will be more involved than the test from the previous example; for one thing, we need to load D3 and the windmill library as Node modules.
D3 is a library that can be used to modify DOM elements based on data. In node applications, we don't have a browser and the DOM doesn't exist. To have a document with a DOM tree, we can use the JSDOM module. When we load D3 as a module, it creates the document and includes JSDOM for us; we don't need to load JSDOM (or create the document and window objects).
To create a test for the heat map chart, we create the test/chart/heatmap-test.js file and load the vows, assert, and d3 modules. We also load our charting library as a local file:
// Import the required modules
var vows = require("vows"),
assert = require("assert"),
d3 = require("d3"),
windmill = require("../../windmill");We will also add a data array and use it later to create the charts. This array will be accessible for the vows and contexts in the module, but it won't be exported.
// Sample Data Array
var data = [
{row: 1, column: 1, value: 5.5},
{row: 1, column: 2, value: 2.5},
// ...
{row: 2, column: 4, value: 7.5}
];The suite will contain tests for the heat map chart. We will describe the suite. It is not necessary to describe the suite with the path to the method being tested, but it's a good practice and helps you to locate errors when the tests don't pass.
// Create a Test Suite for the heatmap chart
var suite = vows.describe("windmill.chart.heatmap");We will add a batch that contains the contexts to be tested. In the first context topic, we will create a div element, create a chart with the default options, bind the data array with the div element, and create a chart in the first context topic:
// Append the Batches
suite.addBatch({
"the default chart svg": {
topic: function() {
// Create the chart instance and a sample data array
var chart = windmill.chart.heatmap();
// Invoke the chart passing the container div
d3.select("body").append("div")
.attr("id", "default")
.data([data])
.call(chart);
// Return the svg element for testing
return d3.select("div#default").select("svg");
},
// Vows...
}
});We will create vows to assert whether the svg element exists, its width and height match the default values, it contains groups for the chart and axis, and the number of rectangles match the number of elements in the data array:
// Append the Batches
suite.addBatch({
"the default chart svg": {
topic: function() {...},
"exists": function(svg) {
assert.equal(svg.empty(), false);
},
"is 600px wide": function(svg) {
assert.equal(svg.attr('width'), '600'),
},
"is 300px high": function(svg) {
assert.equal(svg.attr('height'), '300'),
},
"has a group for the chart": function(svg) {
assert.equal(svg.select("g.chart").empty(), false);
},
"has a group for the xaxis": function(svg) {
assert.equal(svg.select("g.xaxis").empty(), false);
},
"has a group for the yaxis": function(svg) {
assert.equal(svg.select("g.yaxis").empty(), false);
},
"the group has one rectangle for each data item": function(svg) {
var rect = svg.select('g').selectAll("rect");
assert.equal(rect[0].length, data.length);
}
}
});We can run the test with vows and check whether the default attributes of the chart are correctly set and the structure of the inner elements is organized as it should be:
$ vows test/chart/heatmap-test.js --spec ♢ windmill.chart.heatmap the default chart svg √ exists √ is 600px wide √ is 300px high √ has a group for the chart √ has a group for the xaxis √ has a group for the yaxis √ the group has one rectangle for each data item √ OK » 7 honored (0.075s)
In a real-world application, we would have to add tests for many more configurations.
The matrix layout is simpler to test, because we don't need the DOM or even D3. We begin by importing the required modules and creating a suite, as follows:
// Create the test suite
var suite = vows.describe("windmill.layout.matrix");We add a small data array to test the layout:
// Create a sample data array
var data = [
{a: 1, b: 1, c: 10},
// ...
{a: 2, b: 2, c: 5}
];We define an average function, as we did in the example file:
var avgerage = function(values) {
var sum = 0;
values.forEach(function(d) { sum += d; });
return sum / values.length;
};We add a batch and a context to check the default layout attributes and generate the layout in the context's topic:
// Add a batch to test the default layout
suite.addBatch({
"default layout": {
topic: function() {
return windmill.layout.matrix();
},We add vows to test whether the layout is a function and has the row, column, and value methods:
"is a function": function(topic) {
assert.isFunction(topic);
},
"has a row method": function(topic) {
assert.isFunction(topic.row);
},
"has a column method": function(topic) {
assert.isFunction(topic.column);
},
"has a value method": function(topic) {
assert.isFunction(topic.value);
}
}
});We can run the tests using vows test/layout/matrix-test.js --spec, but we will automate the task of running the tests with Grunt.
We will add a test task to the Gruntfile.js file in order to automate the execution of tests. We will need to install the grunt-vows module:
$ npm install --save-dev grunt-vows
As usual, we need to enable the grunt-vows plugin in the Gruntfile.js file:
// Enable the Grunt plugins
grunt.loadNpmTasks('grunt-contrib-concat'),
// ...
grunt.loadNpmTasks("grunt-vows");
We will configure the task to run all the tests in the test directory. As we have done when running the tests, we will add the spec option to obtain detailed reporting. Removing this option will use the default value, displaying each test as a point in the console:
vows: {
all: {
options: {reporter: 'spec'},
src: ['test/*.js', 'test/*/*.js']
}
},We could create additional targets to test the components individually as we modify them. We can now run the task from the command line:
$ grunt vows Running "vows:all" (vows) task (additional output not shown) Done, without errors.
Testing the code doesn't guarantee that you will have bug-free code, but it will certainly help you to detect unexpected behaviors. A mature software usually has thousands of tests. At the time of writing, for instance, D3 has about 2,500 tests.
We have created and configured the essential tasks for our project, but we can automate the process further. For instance, while modifying the code, we need to check and test the code, but we won't need a minified version until we are ready to push the changes to the repository. We will register two groups of tasks, test and build. The test task will check the code, concatenate the source files, and run the tests. To register a task, we give the task a name and add a list of the subtasks to be executed:
// Test Task
grunt.registerTask('test', ['jshint', 'concat', 'vows']);We can execute the test task in the command line, triggering the jshint, concat, and vows tasks in a sequence:
$ grunt test
We register the build task in a similar way; this task will run jshint, concat, vows, and uglify in order, generating the files that we want to distribute:
// Generate distributable files
grunt.registerTask('build', ['jshint', 'vows', 'concat', 'uglify']);A default task can be added too. We will add a default task that just runs the build task:
// Default task
grunt.registerTask('default', ['build']);To run the default task, we invoke Grunt without arguments. There are hundreds of Grunt plugins available; they can be used to automate almost everything. There are plugins to optimize images, copy files, compile the LESS or SASS files to CSS, minify the CSS files, monitor files for changes, and run tasks automatically, among many others. We could automate additional tasks, such as updating the version number in the source files or running tasks automatically when we modify source files.
The objective of automating tasks is not only to save time, but it also makes it easier to actually do the tasks and establish a uniform workflow among peers. It also allows developers to focus on writing code and makes the development process more enjoyable.
If our package is to be used in web applications, we should declare that it depends on D3 and also specify the version of D3 that we need. For many years, projects just declared their dependencies on their web page, leaving the task of downloading and installing the dependencies to the user. Bower is a package manager for web applications (http://bower.io/). It makes the process of installing and updating packages easier. Bower is a Node module; it can be installed either locally using npm, as we did earlier in the chapter, or globally using npm install –g bower:
$ npm install --save bower
This will install Bower in the node_modules directory. We need to create a bower.json file containing the package metadata and dependencies. Bower can create this file for us:
$ bower init
This command will prompt us with questions about our package; we need to define the name, version, main file, and keywords, among other fields. The generated file will contain essential package information, as follows:
{
"name": "windmill",
"version": "0.1.0",
"authors": [
"Pablo Navarro"
],
"description": "Heatmap Charts",
"main": "windmill.js",
"keywords": ["chart","heatmap","d3"],
"ignore": [
"**/.*","**/.*",
"node_modules",
"bower_components",
"app/_bower_components",
"test",
"tests"
],
"dependencies": {}
}Bower has a registry of frontend packages; we can use Bower to search the registry and install our dependencies. For instance, we can search for the D3 package as follows:
$ bower search d3 Search results: d3 git://github.com/mbostock/d3.git nvd3 git://github.com/novus/nvd3 d3-plugins git://github.com/d3/d3-plugins.git ...
The results are displayed, showing the package name and its Git endpoint. We can use either the name or the endpoint to install D3:
$ bower install --save d3
This will create the bower_components directory (depending on your global configuration) and update the bower.json file, including the D3 library in its most recent release. Note that we included D3 both in our Node dependencies and in the Bower dependencies. We included D3 in the Node dependencies to be able to test our charts (which depend on D3); here, we include D3 as a frontend dependency, so the other packages that use our charting package can download and install D3 using Bower:
{
"name": "windmill",
"version": "0.1.0",
...
"dependencies": {
"d3": "~3.4.1"
}
}We can specify which version of the package we want to include. For instance, we could have installed the release 3.4.0:
$ bower install d3#3.4.0
We don't need to register the packages to install them with Bower; we can use Bower to install the unregistered packages using their Git endpoint:
$ bower install https://github.com/mbostock/d3.git
The Git endpoint could also be a local repository, or even a ZIP or TAR file:
$bower install /path/to/package.zip
Bower will extract and copy each dependency in the bower_components directory. To use the packages, we can use a reference to the bower_components directory, or write a Grunt task to copy the files to another location. We will use the bower_components directory to create example pages for our charts.