Chapter 6

Intro to Node.js

If you've read anything about web development in the past couple years, you've certainly heard a lot about Node.js. But why all the hype? It seems clear at this point that Node is more than a fad, so why are so many developers excited about Node?

In this chapter you learn what Node is good for, which is directly related to how it works internally. You also learn about the differences between Node and client-side JavaScript, as well as strategies for coming into Node from the frontend. Next you install Node and build your first Node app. You learn how to use the Node REPL, and about Node's module system and installing third party modules using NPM. Finally, you learn about some general patterns and best practices in Node, including:

• Global variables and scope across modules

• Asynchronous and synchronous functions

• Streams

• Custom events and event handlers

• Child processes

Why Node?

It's no accident that Node became so popular. So many people are talking about Node because it's an excellent solution for building many modern web apps. Node is a great fit for apps with heavy input / output (I/O) and light computation. That means you should use Node if your app relies on heavy communication between the server and client, but doesn't need to do anything too complex on the server-side.

Using Node with Real-Time Apps

You've probably heard about Node in relation to “real-time” apps, such as collaborative document editors. That's because these types of apps require a lot of I/O and also benefit from Node's asynchronous nature. For example, a document app for a single user might autosave the doc every minute or so. But with a collaborative app, you need to save it a lot more often to make sure any changes get relayed to each user in relative real-time. Additionally, you need to check frequently for any changes that may have been pushed from other users. All things considered, real-time collaborative apps make for a ton of I/O.

You learn how to build real-time apps in Chapter 9.

Don't get me wrong; just about any server can handle heavy I/O for a handful of users at a time. The question is how well your app is going to scale. Of course, having a ton of users is a good problem to have. But this is one issue you can't afford to wait to optimize. The conventional logic of avoiding premature optimization doesn't apply when you're talking about potentially overhauling the entire app. If your app is going to need to handle a heavy I/O load, you'll need to anticipate and build for scale from day one. You'll need to build it with Node or another I/O friendly server.

Understanding How Node Works

The reason Node is able to handle large amounts of I/O gracefully, has to do with how it handles multiple concurrent connection requests.

As you might have guessed, the traditional multi-threaded approach doesn't scale well enough for many modern web apps. But Node handles things differently. In a nutshell, Node processes requests using a single non-blocking thread with an event loop. Multiple requests are queued up, and processed in turn using asynchronous callback functions.

Node isn't the only server to use a single thread; other high-speed servers such as Nginx take the same approach. But Node is particularly good at it, since it runs JavaScript. Over the years, web browsers have made substantial efforts to optimize their JavaScript engines. Since these engines run code as a single, non-blocking thread, with an asynchronous event loop, they have become very good at passing around asynchronous functions, both in timing events such as setTimeout and interface events such as onclick. Which brings me to the V8 Engine.

Using the V8 JavaScript Engine with Node

Node uses the lightning-fast V8 engine that Google programmed for Chrome. V8 compiles JavaScript to native machine code before executing it. This approach produces significantly faster code than other engines.

The compiled code is further optimized at runtime, dynamically caching portions of the code, and eliding expensive runtime properties. When all of this is put together, it produces the fastest JavaScript engine ever built. But don't trust me; see for yourself. Visit the V8 benchmarks at http://v8.googlecode.com/svn/data/benchmarks/v7/run.html and compare Chrome to another browser.

Coming to Node from the Front End

Although you will be more familiar with the syntax, working with Node is very different from working with JavaScript on the front end. Sure, you'll be able to use the core language you're already used to, but you will have to learn entirely new concepts. The problems you'll solve with Node are fundamentally different from those on the client-side.

That said, there are some aspects of Node development that make it easier to learn than other backend languages. Namely, event loops and the way that Node runs a single thread of asynchronous calls will be very familiar to anyone who has worked with JavaScript.

Just like in front-end JavaScript, Node executes code line-by-line, and allows you to set up callbacks to handle any events that need to be processed. These callbacks are then handled in order. However, the way these asynchronous callbacks are processed isn't entirely the same. Whereas on the client, you have one user triggering various callbacks, your Node app can have any number of users at once. That means there are even fewer guarantees of execution order—you need to be much more careful to avoid race conditions stemming from dependencies between one asynchronous function firing before another.

Installing Node

Before you can get started with Node, you need to install it. There are a few ways to do this:

• Clone the git repo and compile

• Download a tarball and install via the command line

• Download an automated package installer for your platform

• Use a package manager such as MacPorts

Unless you're already using a package manager, installing from the repo is the best option—it's really easy and allows you to update to different versions of Node quickly.

If you're not comfortable with the command line, Node development is going to be pretty difficult. So it's a good idea to get started with one of the command line installations rather than the automated package installer. Additionally, since Node is such a new technology, it's always best to use the latest stable build.

If you installed Weinre in Chapter 1, you already have Node installed.

Mac/Linux Installation

Installing the Node core on Mac and Linux is pretty easy. You just have to download the Node source and compile it.

Getting and Compiling the Source

First, make sure you have git installed: https://help.github.com/articles/set-up-git. Then, clone the repo from the command line (in Terminal):

git clone https://github.com/joyent/node.git

cd node

git checkout

These commands clone the Node repo and then check out the latest version of the source code.

If you'd like to avoid using git, you can also download the source from http://nodejs.org/dist/latest/.

Once you have the source, you can compile it:

./configure --prefix=/opt/node

make

sudo make install

These commands configure the installation, and then make and install. Node takes a few minutes to compile, so grab a cup of coffee or read on to the next section.

It's also a good idea to add the Node executables to your system path for ease of use. Add the following line to your ~/.profile, ~/.bash_profile, ~/.bashrc or ~/.zshenv:

export PATH=$PATH:/opt/node/bin

Using a Package Installer

Alternatively, you can use a package installer. Simply go to http://nodejs.org/dist/latest/ and download the latest installer (the .pkg file). Just open up the file and follow the instructions in the wizard to install Node automatically.

Using a Package Manager

There are also a few package managers you can use to install Node quickly. For instance, on Mac you can install Node using MacPorts (http://www.macports.org/):

port install nodejs

Or with Homebrew (http://mxcl.github.com/homebrew/):

brew install node

For a list of other package managers that support Node, visit: https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager.

Compiling with Xcode 4.5

If you're using Mac, you can configure Node to work with Xcode. First, download and install Xcode, which is free from the App Store: https://developer.apple.com/technologies/tools/.Then, install the command line tools by going to Xcode: Preferences->Downloads->Install Command Line Tools. Finally, compile the source code with Xcode paths:

export CC=/Applications/Xcode.app/Contents/Developer/Toolchains/

XcodeDefault.xctoolchain/usr/bin/clang

export CXX=/Applications/Xcode.app/Contents/Developer/Toolchains/

XcodeDefault.xctoolchain/usr/bin/clang++

./configure

make

sudo make install

Windows Installation

Node is really geared to a Linux / UNIX type environment. But you can still install it on Windows if you're running a Windows machine and don't want to install Ubuntu. Just like on Mac / Linux there are two ways to go about this installation. You can either build an install if you want more control over the process, or install a precompiled version if you want to get up and running quickly.

Building a Windows Install

To start, make sure you have Python and Visual Studio installed. Then in cmd.exe, do the following:

C:Users yan>tar -zxf node-v0.8.16.tar.gz

C:Users yan>cd node-v0.8.16

C:Users yan ode-v0.8.16>vcbuild.bat release

[Wait 20 minutes]

C:Users yan ode-v0.8.16>Release ode.exe

> process.versions

{ node: '0.8.16',

  v8: '3.6.6.11',

  ares: '1.7.5-DEV',

  uv: '0.6',

  openssl: '0.9.8r' }

>

The executable will then be in Release ode.exe.

Installing Without Building

You can also install a pre-compiled version of Node on your Windows system. First, download it from http://nodejs.org/dist/latest/node.exe. Put node.exe in a clean directory, and add that directory to your PATH variable, so the node commands are available throughout cmd.exe.

Next, find a recent version of the NPM .zip archive from http://nodejs.org/dist/npm/. Unpack that to the same directory as the node.exe file, and you're done. You should now be able to run Node and NPM from anywhere you want.

Using a Package Installer

You can also use an automated package installer to install Node on Windows. Simply download the latest .msi package from http://nodejs.org/dist/latest/ and run the install wizard.

Checking Your Install

After your Node install finishes on Mac, Linux or Windows, you can verify the install by typing the following into the command line:

node --version

If everything went smoothly, you should see it return a version number—for instance 0.8.16. Now you're ready to dive in and build your first Node app.

If you want to manage multiple Node versions on the same system, install NVM: https://github.com/creationix/nvm.

Getting Started with Node

Although Node might seem intimidating at first, getting started is actually pretty easy. No intro to Node would be complete without the ubiquitous “Hello World” example, and I'm just as excited as you are.

Creating the Server

The first step is to open up a text editor and include the http module, which you use to create the HTTP server:

// load http module

var http = require('http'),

Here you use the require() method to include the module. The http module is one of the core Node modules. (Later this chapter you learn more about modules and how to include external modules in your app.) Next, use the http module to create the server:

// load http module

var http = require('http'),

// create http server

var server = http.createServer( function(request, response) {});

Here the createServer() method accepts a callback, which passes objects for the request and response.

You're going to see a lot of callbacks and closures when working with Node. They are the bread and butter of the asynchronous event loop.

Adding the Content

Next, use response to create your page content:

// create http server

var server = http.createServer( function(request, response) {

  // header

  response.writeHead(200, {

    'Content-type': 'text/plain'

  });

  

  // write content

  response.write('Hello World!'),

   

  // send the response

  response.end();

});

There are a few things going on in this code block:

1. The writeHead() method defines the HTTP headers for the page. In this case, it uses a response code of 200 (for OK) and passes the plain-text content-type.

Other response codes might be 404 (for file not found), and other headers might be ‘cache-control': ‘max-age=3600 , must-revalidate' (to cache the page for an hour).

2. The write() method writes the actual ‘Hello World' message to the page.

3. The end() method closes the response and sends the header and content to the client.

Wrapping Things Up

Now the Hello World script is creating the server and all the content for the page, but you're not done yet. You still need to create a path to access the script, which you can do using the listen() method:

// listen on port 8000

server.listen(8000);

Here the server you created earlier is set up on port 8000. The listen() method also accepts a second argument for the hostname, but you don't need that yet because you're just building this locally. Finally, it's a good idea to log what happened to the console:

// log it to the console

console.log('Server running on port 8000'),

Here the script uses the same console.log() method you're already familiar with from client-side development. However, instead of outputting this message in the browser, it will be logged in the terminal.

Now the Hello World example is complete. Putting all the code together:

// load http module

var http = require('http'),

// create http server

var server = http.createServer( function(request, response) {

  // header

  response.writeHead(200, {

    'Content-type': 'text/plain'

  });

  

  // write content

  response.write('Hello World!'),

  

  // send the response

  response.end();

});

// listen on port 8000

server.listen(8000);

// log it to the console

console.log('Server running on port 8000'),

Running the Script

The final step is actually running the script. To do so, save this file as helloworld.js and then type the following into the command line:

node helloworld.js

Now the server is running and you should see the message you logged to the console:

Server running on port 8000

Finally, navigate to http://localhost:8000 and you should see your first Node app in action, as shown in Figure 6-1.

Figure 6-1 The Hello World script is working in the browser.

Simplifying the Script

The Hello World script is working exactly as it should, but there are a couple ways that you can make it simpler. First, you can pass any content you want to write to the page into the response.end() method. That means you can eliminate the response.write() call and shorten the script a bit:

// load http module

var http = require('http'),

// create http server

var server = http.createServer( function(request, response) {

  // header

  response.writeHead(200, {

    'Content-type': 'text/plain'

  });

  

  // send the response with the content

  response.end('Hello World!'),

});

// listen on port 8000

server.listen(8000);

// log it to the console

console.log('Server running on port 8000'),

Additionally, most of the methods in Node are chainable, just like those in jQuery and Underscore. You can see this in action by chaining the listen() method to the createServer() method:

// load http module

var http = require('http'),

// create http server on port 8000

http.createServer( function(request, response) {

  // header

  response.writeHead(200, {

    'Content-type': 'text/plain'

  });

  

  // send the response with the content

  response.end('Hello World!'),

}).listen(8000);

// log it to the console

console.log('Server running on port 8000'),

As demonstrated here, listen() is chained to the previous method, allowing you to shorten the script a bit further. Now close the previous Node session by hitting Ctrl+C in your terminal window, and then reopen it with node helloworld.js. http://localhost:8000 shows the Hello World script working exactly as before.

Using the Node REPL

Before you dive deeper into Node, it's a good idea to get comfortable with the read-eval-print loop (REPL). Pronounced “repple,” it allows you to run Node code directly in the terminal. To access the REPL, simply call Node from the command line, passing in no variables:

node

Now you should see a prompt (>), where you can enter Node commands to be executed on the fly, as shown in Figure 6-2.

Figure 6-2 The REPL allows you to execute Node commands on the fly.

The REPL is useful for testing small bits of code. It's especially handy when you're first getting used to Node development, since you can quickly test code for the concepts you're learning.

REPL Features

The REPL functions largely like the normal command line: you can use the up arrow to access previous commands and the tab key to complete commands. Additionally, you can use the underscore (_) to pull the result of the last expression, which is shown in Figure 6-3.

Figure 6-3: The _ pulls the result of the previous expression in REPL.

The REPL even handles multi-line expressions gracefully. Simply start entering a function, or another unclosed tag, and the REPL automatically extends to multiple lines, as shown in Figure 6-4.

Figure 6-4: The REPL handles multi-line expressions automatically.

Additional REPL Commands

There are a few extra commands you can use in REPL, which you can see if you type the following in your REPL prompt:

.help

As shown in Figure 6-5, this outputs a list of REPL commands.

Figure 6-5: REPL also provides some additional handy commands.

Break

If you get lost in a multiline entry, you can start over again by typing:

.break

However, keep in mind that you lose everything you entered.

Save and Load

You can actually develop your entire application in the REPL. But be careful: if the terminal window crashes you'll lose everything. So when working in the REPL, remember to save early and save often. To save the current REPL session to a file, you can use the .save command—for instance:

.save ~/path-to/my-file.js

Likewise, you can load the contents of a file into the REPL session:

.load ~/path-to/my-file.js

Exit

Finally, to terminate your REPL session, use the exit command:

.exit

Alternatively, you can press Ctrl+D. But don't forget to save if there's anything you want to keep!

Node Modules

Rather than implement every possible feature into every build, the Node core is kept as lean and streamlined as possible using modules. Modules are just files that store encapsulated bits of JavaScript.

Node's module system is modeled after CommonJS.

Including Modules

You've already seen modules in action. For example, the Hello World example used the http module:

var http = require('http'),

As you can see here, including a module is as simple as using the require() method. You can also include a specific object from a module, rather than all objects:

var createServer = require('http').createServer;

It's important to remember that require is a synchronous method. Unlike most of the methods you'll use in Node, require blocks the main thread. You'll learn more about synchronous and asynchronous methods in the Node Patterns section.

External Modules and NPM

In addition to the core modules, you can install a vast number of external modules to speed up your development process. Although you can import these modules manually, it's a lot easier to use the Node Package Manager (NPM). NPM makes it easy to download, install and update various modules, similar to Ruby gems. Assuming you're using Node version 0.4 or higher, NPM is already bundled with the Node core, so you won't need to install it. But it's still a good idea to update NPM. You can do this with NPM from the command line:

npm update -g npm

This updates to the latest NPM that's supported by the version of Node you're running.

Installing Modules with NPM

Now, you can install a given module using npm install, for instance you can use the following command to install Underscore:

npm install underscore

This installs Underscore into the ./node_modules folder. But make sure to call this command from the root of whatever application is going to use the module. That way the module will install into that app's node_modules directory.

Now you can use Underscore in your app just like any other module:

var _ = require('underscore'),

var myArray = [ 1, 5, 3, 8, 7, 1, 4 ];

_.without(myArray, 1);

Likewise, you can also uninstall a module at any point, using npm uninstall:

npm uninstall underscore

Installing Modules Globally

You can also install modules globally using NPM. To do so, simply add the -g flag. For instance, if you want to make sure Underscore is available for any app you build in Node, you can write:

npm install underscore –g

Instead of installing this to your app's node_modules folder, this command installs it to ~/lib/node_modules, ensuring the module is available for any Node build.

Installing Dependencies

One of the best parts about using NPM is that it automatically installs any dependencies a module needs. To get a better idea of how dependencies are handled in Node, install the express module. Express is a very popular framework that you will learn more about in the next chapter. But before installing it, take a look at its dependencies using npm view:

npm view express dependencies

This command outputs a JSON object of dependencies:

npm http GET https://registry.npmjs.org/express

npm http 304 https://registry.npmjs.org/express

{ connect: '2.7.1',

  commander: '0.6.1',

  'range-parser': '0.0.4',

  mkdirp: '0.3.3',

  cookie: '0.0.5',

  'buffer-crc32': '0.1.1',

  fresh: '0.1.0',

  methods: '0.0.1',

  send: '0.1.0',

  'cookie-signature': '0.0.1',

  debug: '*' }

It's a good idea to take a look at a module's dependencies before installing it, so that you know what you're getting into.

Next, install Express using npm install:

npm install express

You already know that NPM installs dependencies, but take a look in node_modules just to be sure:

ls node_modules

While you should see express, along with any other modules you've installed, you won't see any of the dependencies. But don't worry, they're still installed.

Rather than install the dependencies to the node_modules folder at the root of your application, NPM installs them in a node_modules subfolder of the express module. Check this subfolder:

ls node_modules/express/node_modules/

You should then see a listing of all the dependencies that have been installed.

Node handles dependencies this way to avoid any collisions. Different modules might depend on different versions of the same module. If these were all installed to the top level node_modules directory, it could create a major problem.

Finding Modules

Just like with jQuery plugins, searching for Node modules can be a challenge. First, where do you look? And more importantly, how do you know if a module is any good? To find modules use npm search or peruse module directories. If you already have a pretty good idea of what you're looking for, using npm search from the command line is a good option. For instance, you can search for any modules related to Grunt (which you learned about in Chapter 1):

npm search grunt

npm search typically returns a lot of different modules, so you have to do some digging to find the one you want. Use npm view to take a look at any of the various options. There are also a number of module directories out there. Here are some good places to look:

• The Node module wiki (https://github.com/joyent/node/wiki/modules)

• NPM registry (https://npmjs.org/)

• Nipster! (http://eirikb.github.com/nipster/)

• The Node Toolbox (http://nodetoolbox.com/)

Even though Node is a fairly new technology, it is quite popular, and as such it already has many third-party contributions. While it's great to be able to find an out-of-the-box solution for pretty much any Node problem you may have, the flip side is that a lot of these solutions are not of a high enough quality. So, when you find multiple options for a given module, how do you determine which one to use?

The easiest test is popularity: a Google search shows you which modules people are talking about. The Node Toolbox also organizes modules by popularity on GitHub, as well as lists the most depended on modules. Beyond that, take a look at the star rating on Nipster. Finally, check on GitHub to make sure the module is actively supported and up to date with the current version of Node.

One module you should definitely install is Supervisor. Supervisor monitors the files in your Node app, and restarts the server whenever a change is made to a *.js file. It makes development much easier. Learn more here: https://github.com/isaacs/node-supervisor.

Node Patterns

The next three chapters discuss practical techniques for building apps in Node. For now I'll walk you through some general concepts, so that your app is built on a firm foundation of best practices.

Modules and Global Variables

You've already discovered a bit about including external modules from third parties. But did you know that you can create your own modules as well? Much like you might partition a portion of client-side JavaScript into your own library, it is a good idea to piece Node scripts into modules. Writing your own modules gives your app structure, and improves collaborative development efforts. It also makes it easier to reuse code across multiple projects.

Creating Your Own Modules

When creating your own modules, simply place them in the node_modules directory like any other module. To reference them later you use the require() method. For instance, if you have the module my-module.js in your node_modules directory, you can include it with the following statement:

var myModule = require('my-module'),

But if you'd rather use a different structure, feel free to include modules anywhere else on your system. Simply point to the correct path:

var myModule = require('~/path-to/my-module.js'),

If you think other developers can benefit from your module, please share it on GitHub or Bitbucket.

Global Scope Among Modules

One important thing to remember when writing your own modules is that global variables work a bit differently than you might expect. For example, in client-side JavaScript, you can define a global variable by omitting the var keyword:

globalVar = true;

var localVar = false;

On the front-end, this approach ensures that the globalVar is available in all the functions of the app, no matter which file calls that variable. However, Node handles globals differently. In Node, any variable you define globally will be available only in that module. Since modules map directly to files, each variable is available only within a given file.

Creating Global Variables

You can get around this scope issue by writing the following:

GLOBAL.globalVar = true; // try to avoid this

This defines globalVar such that it can be accessed across all the modules in your app. However, you should never define a global variable in Node, unless you have a very, very good reason. Keep in mind that the global context of modules is set up that way for a reason: it avoids variable collisions across different modules. Globals in Node will be used across your entire application, meaning all the modules and scripts in your Node server. Therefore the potential collision problems are exponentially greater than those in a client-side app.

The Global Object

To take a look at what has been defined on the global scope, open a REPL session and type global. That command lists everything in the global namespace object, a portion of which you can see in Figure 6-6.

The global object stores all of this information, much like window in client-side JavaScript. However, since there's no browser window, Node stores it in global. As you can see, there are a lot of variables defined on the global namespace. But that doesn't mean you should add to it. Due to potential collision problems, defining your own globals is a risky business.

Using Exports

In general you should avoid defining global variables in Node. Instead, whenever you need to share variable scope, do so manually. To share scope, you have to explicitly declare whatever you want to share from a module using exports. For example, your module might contain the following function:

var sayHello = function() {

  console.log('Hello World!'),

};

If you call sayHello() from the same module, it work just as you'd expect. However, if you require the module in your app.js, the function is no longer available:

var myModule = require('my-module'),

sayHello(); // this won't work

Figure 6-6 A portion of the global scope listed in a REPL session.

Instead, you have to explicitly expose the function you want to share, using exports:

exports.sayHello = function() {

  console.log('Hello World!'),

};

Now you can call this function from app.js:

var myModule = require('say-hello'),

myModule.sayHello(); // this works

Exporting the Entire Scope

Likewise, if your module is a “one trick pony,” you can expose the function a little differently. For instance, say that this module has only the sayHello() function. You could expose this a bit differently:

module.exports = function() {

  console.log('Hello World!'),

};

The only difference here is that you're using module.exports instead of exports.sayHello. That defines the export a little differently, which you can see when you reference the module in your app.js file:

var sayHello = require('say-hello'),

sayHello();

As you can see here, the function is defined as the entire extent of the module. This technique can also be useful for exporting an entire object at once, rather than using a number of exports calls. For example:

// define an object for the module

var myModule = {

  var1: true,

  var2: false,

  

  func1: function() {

    console.log('Function 1'),

  },

  

  func2: function() {

    console.log('Function 2'),

  }

};

// export the entire object

modules.exports = myModule;

Rather than explicitly exporting every variable in this module, this script exports an object of variables at once. The various components of the module can then be accessed in app.js:

var myModule = require('my-module'),

if ( myModule.var1 ) {

  myModule.func1();

}

else {

  myModule.func2();

}

Multiple Instances of a Module

This technique is also useful for exporting multiple instances of the same module. For example, create a module for users:

// user data module

modules.exports = function( name, age, gender ) {

  this.name = name;

  this.age = age;

  this.gender = gender;

  

  this.about = function() {

    return this.name + ' is a ' + this.age + ' year old ' + this.gender;

  }

}

Now you can create a new instance of User in app.js using the new keyword:

// include the module

var User = require('./user.js'),

// create a new instance of the user, passing in appropriate arguments

var user = new User( 'Jon', 30, 'man' );

// use the module's about() method

console.log( user.about() );

This script pulls in the context you set in the module, so user.about() returns “Jon is a 30 year old man.” Likewise, you can access other information defined in the module. For instance, to access the user's name directly you'd simply write user.name.

Asynchronous Patterns

Asynchronous requests are great for speeding up the Node server and improving the performance of your app. But they can also be more difficult to work with, especially when you are expecting functions to execute in a certain order. By their very nature, asynchronous methods are called as soon as possible, which means that even if you call function a before function b, you have no guarantee that it will be completed first. Without a guaranteed execution order, programming in Node can be challenging. That's especially true whenever you are depending on functions to execute sequentially.

Synchronous Calls

To avoid blocking execution, most of Node's native methods are asynchronous by default. That said, Node also provides synchronous versions of most of its methods. For example, you can read in the contents of a file using the fs (file system) module's readFile() method:

// include the fs module

var fs = require('fs'),

// read in the file

fs.readFile('./path-to/my-file.txt', 'utf8', function(err, data) {

  if (err) throw err;

  

  console.log(data);

});

Since you haven't flagged this as a synchronous method, Node uses its default asynchronous pattern. However, reading a file can take a little while, which can produce unexpected problems if you're waiting on the results of this function. To get around this, use the readFileSync() method:

// include the fs module

var fs = require('fs'),

// read in the file synchronously

var data = fs.readFileSync('./path-to/my-file.txt', 'utf8'),

console.log(data);

This script blocks the thread while the file is read, so that no dependent functions can execute before the file is ready. However, the fact that files can take a while to read is often the exact reason not to use this approach. Remember, the longer the synchronous call takes to complete, the longer your entire Node server will be blocked. Thus, while you can use synchronous methods in certain situations, it's a good idea to avoid them whenever possible.

Most Node methods come with a synchronous counterpart you can access by appending Sync—for example, readFile() becomes readFileSync().

Nested Callbacks

Fortunately, you can get all the benefits of synchronous calls, without blocking the main thread. The trick is to use nested callbacks. For example, you can read all the files in a directory using the following script:

var fs = require('fs'),

// get directory listing

fs.readdir('./my-dir', function(err, files) {

  var count = files.length,

      results = {};

  // loop through and read all the files

  files.forEach(function(filename) {

    fs.readFile('./my-dir/' + filename, 'utf8', function(err, data) {

      results[filename] = data;

      

      count--;

      if ( count <= 0 ) {

        console.log(results);

      }

    });

  });

});

This script first uses the fs module's readdir() to get a directory listing. It then loops through all the files in that directory, reading their contents. Since each of the individual calls is asynchronous, the files can all be read in parallel, and the script won't block the main thread from accomplishing other tasks. But the nesting ensures that the critical functions still execute serially.

However, you may have already noticed the downside of this approach: callback spaghetti, a.k.a. “the pyramid of doom.” Each nested callback indents the chain further, and makes it easier to get lost in your code. This example isn't too bad with only two nested callbacks, but they can really start to pile up. And when you get a lot of them, it can be difficult to track which closure ends where.

There are a number of ways to minimize the level of nesting in your function. One simple technique is to chunk off portions of the script and pass them as separate callbacks. Alternatively, you can use a module to handle execution order, such as Async: https://github.com/caolan/async.

Streams

You've already learned how to use the fs.readFile() to fetch the contents of files on your server. However, that's not always the best way to read in file data because the server has to wait for the entire file to buffer into memory before executing the callback. If the file is very large this technique can present two problems. First, it can create latency issues if you are relaying the file to the client, since users will have to wait for the entire file to buffer before they start receiving any contents. Second, this approach can create memory issues if the program is handling lots of concurrent requests for the file.

A better solution is to use streams to process the file data while it buffers:

var http = require('http'),

    fs = require('fs'),

var server = http.createServer(function(request, response) {

  // create the stream

  var stream = fs.createReadStream('my-file.txt'),

  

  // handle any errors

  stream.on('error', function(err) {

    response.statusCode = 500;

    response.end(String(err));

  });

  

  // pipe the response to the client

  stream.pipe(response);

});

server.listen(8000);

This script first uses fs.createReadStream() to create the stream and then sets up basic error handling. Next, pipe() relays the data in the stream to the client. To test this functionality, create an extremely long chunk of text in my-file.txt, and then navigate to http://localhost:8000. You should see the contents of the file stream into the page.

fs.createReadStream() is just the beginning when it comes to streaming in Node. There are a variety of other built in streams, and you can even create your own. For more information visit https://github.com/substack/stream-handbook.

Events

As Node's website puts it, “Node.js uses an event-driven, non-blocking I/O model.” Events are clearly a big part of Node, and the platform provides some useful techniques for setting up your own custom events. The main component for events in Node is EventEmitter. Whenever you see an event handled with on(), you're seeing the EventEmitter in action.

You can use the EventEmitter to create your own custom events and handlers throughout your app. First, include the events module, which is part of the Node core:

var events = require('events'),

Next, create a new instance of EventEmitter:

var em = new events.EventEmitter();

Now, you can use the EventEmitter to handle two important tasks: creating an event handler and emitting the actual event. First, create the event handler using on():

em.on( 'my-event', function(data) {

  // what to do when this event fires

});

Then, you can trigger this handler by firing the actual event. To do so, use emit():

em.emit('my-event'),

Using this technique, you can set up custom events and handlers throughout your app. For example, here's how to create an event that fires at every tick of an interval:

// include the events module

var events = require('events'),

// create an instance of the EventEmitter

var em = new events.EventEmitter();

var counter = 0;

// emit the event every 5 seconds

var timer = setInterval(function() {

  em.emit('tick'),

}, 5000);

// handle the event, logging either 'Tick' or 'Tock'

em.on('tick', function() {

  counter++;

  console.log( counter % 2 ? 'Tick' : 'Tock'),

});

Here, an interval fires the ‘tick' event every five seconds. Each time it fires, the ‘tick' handler increments the counter and then logs either ‘Tick' or ‘Tock'.

Fire up your Node server, and you will see something similar to Figure 6-7:

Figure 6-7 The custom event is logging every five seconds.

Child Processes

While client-side JavaScript is limited by whatever the browsers support, Node development is completely the opposite. In addition to the wide variety of modules available to Node, you can also access all of the functionality of the operating system that Node is running upon. That means you can build just about anything with Node.

Accessing the operating system commands is as simple as including the child_process module, which is part of the Node core. In particular, you'll want to use the spawn method, so call this as part of your require statement:

var spawn = require('child_process').spawn;

Note that the examples in this section will only work with a Linux or UNIX system (such as Mac). If you're using Windows, you'll need to use Windows commands.

Using Child Processes

Now you can use this to call any OS commands you'd like. For instance, you can get a listing of the current directory using ls. The first step is to spawn a child process for ls:

// include the child_process module

var spawn = require('child_process').spawn;

// spawn a child process for ls

var ls = spawn('ls'),

Next, set up a couple events. First, a handler for standard output (stdout):

// handle standard output

ls.stdout.on('data', function(data) {

  console.log(data.toString());

});

This outputs the result of ls to the Node console. However, that occurs only if there isn't an error. You should also set up a standard error (stderr) handler:

// handle error

ls.stderr.on('data', function(data) {

  console.log('Error: ' + data);

});

Finally, you should set up a handler for when the command exits. That way you can get a success or fail message depending on whether it crashes:

// handle exit

ls.on('exit', function(code) {

  console.log('child process exited with code ' + code);

});

Putting the script together:

// include the child_process module

var spawn = require('child_process').spawn;

// spawn a child process for ls

var ls = spawn('ls'),

// handle standard output

ls.stdout.on('data', function(data) {

  console.log(data.toString());

});

// handle error

ls.stderr.on('data', function(data) {

  console.log('Error: ' + data);

});

// handle exit

ls.on('exit', function(code) {

  console.log('child process exited with code ' + code);

});

When you run this script, it will output a listing of the working directory in the Node console, as you can see in Figure 6-8.

Figure 6-8 This directory listing has been output from Node using a child_process bridge to the operating system.

Passing Variables to a Child Process

You can also pass variables to your child process. For instance, to get a listing of all the files in the directory (including hidden files), enter the following into the command line:

ls -a

To execute the same command in Node, pass a second argument to spawn(). This second argument is an array of any arguments you want to pass with the command:

var ls = spawn('ls', ['-a']);

Now, when you restart your server, you see the expanded listing, like in Figure 6-9.

Figure 6-9 This directory listing now includes hidden files, such as the Mac cache file .DS_Store.

Summary

In this chapter you got your first taste of Node development. You learned what types of apps are the best fit for Node, and why Node works so well for these apps. After installing Node on your machine, you built your first Hello World app. Then you learned about the Node module system, and how to install external modules with NPM. Finally, you armed yourself with a firm foundation in Node best practices. You learned how globals are handled in Node modules, and how to expose context between modules. You also discovered how to set up sequential behaviors in asynchronous functions and take advantage of streams. Then you learned how to create your own custom events, and access the operating system's command line from Node.

In the coming chapters, you learn how to use the Express framework to set up routes and views for your application. You also learn how to use a NoSQL database, MongoDB, to complement the high-speed scalability of the Node application you're building. Then, you'll tie it all together, creating a real-time app, that communicates with the client using WebSockets.

Additional Resources

Node Documentation: http://nodejs.org/api/

Books

Learning Node by Shelley Powers, (978-1449323073) published by O'Reilly, 10/2012

Mastering Node.js an open source book created by TJ Holowaychuk: http://visionmedia.github.com/masteringnode/

Smashing Node.js by Guillermo Rauch, (978-1119962595) published by Wiley 9/2012

Node.js in Action by Mike Cantelon and TJ Holowaychuk, published by Manning Publications

Tutorials

Node.js Step by Step: http://net.tutsplus.com/tutorials/javascript-ajax/this-time-youll-learn-node-js/

Let's Make a Web App: Nodepad: http://dailyjs.com/2010/11/01/node-tutorial/

A variety of good Node Tutorials: http://howtonode.org/

Video Tutorials

Introduction to Node.js with Ryan Dahl: http://youtu.be/jo_B4LTHi3I

Node.js: JavaScript on the Server: http://youtu.be/F6k8lTrAE2g

Node.js First Look: http://www.lynda.com/Nodejs-tutorials/Nodejs-First-Look/101554-2.html

Module Directories

The Node module wiki: https://github.com/joyent/node/wiki/modules

NPM registry: https://npmjs.org/

Nipster!: http://eirikb.github.com/nipster/

The Node Toolbox: http://nodetoolbox.com/

Best Practices

Node.js Style Guide: http://nodeguide.com/style.html

Stream Handbook: https://github.com/substack/stream-handbook