Introduction

Welcome to Learning Node.js. Node.js is an exciting platform for writing applications of all sorts, ranging from powerful web applications to simple scripts you can run on your local computer. The project has grown from a reasonably small software package managed by one company into a production-quality system governed by a Technical Steering Committee (TSC) and has a sizeable following in the developer community. In this book, I teach you more about it and why it is special, then get you up and writing Node.js programs in short order. You’ll soon find that people are rather flexible with the name of Node.js and will refer to it frequently as just Node or even node. I certainly do a lot of that in this book as well.

It is a reasonably linear process, easy to understand and straightforward to code. There are, however, a couple of disadvantages that continue to plague the model:

1. Each of these threads or processes carries some overhead with it. On some machines, PHP and Apache can take up as much as 10–15MB per process. Even in environments where a large server runs constantly and forks threads to process the requests, each of these carries some overhead to create a new stack and execution environment, and you frequently run into the limits of the server’s available memory.

2. In most common usage scenarios where a web server communicates with a database, caching server, external server, or file system, it spends most of its time sitting around doing nothing and waits for these services to finish and return their responses. While it is sitting there doing nothing, this thread is effectively blocked from doing anything else. The resources it consumes and the process or thread in which it runs are entirely frozen waiting for those responses to come back.

Only after the external component has finally sent back its response will that process or thread be free to finish processing, send a response to the client, and then reset to prepare for another incoming request.

So, although it’s pretty easy to understand and work with, you do have a model that can be quite inefficient if your scripts spend most of their time waiting for database servers to finish running a query—an extremely common scenario for many modern web applications.

Many solutions to this problem have been developed and are in common use. You can buy ever bigger and more powerful web servers with more memory. You can replace more powerful and feature-rich HTTP servers such as Apache with smaller, lightweight ones such as lighttpd or nginx. You can build stripped-down or reduced versions of your favorite web programing language such as PHP or Python (indeed, for a time, Facebook took this one step further and built a system that converts PHP to native C++ code for maximal speed and optimal size). Or you can throw more servers at the problem to increase the number of simultaneous connections you can accommodate.

JavaScript, that old (meaning 1995 or so) language that came to be known mostly (and frequently reviled) for writing client-side scripts in the web browser, has become hugely popular. Modern versions of web browsers are cleaning up their implementations of it and adding new features to make it more powerful and less quirky. With the advent of client libraries for these browsers, such as jQuery, script.aculo.us, or Prototype, programming in JavaScript has become fun and productive. Unwieldy APIs have been cleaned up, and fun, dynamic effects have been added.

At the same time, a new generation of browser competition has erupted, with Google’s Chrome, Mozilla’s Firefox, Apple’s Safari, and Microsoft’s Edge all vying for the crown of browser king. As part of this, all these companies are investing heavily in the JavaScript portion of these systems as modern web applications continue to grow ever-more dynamic and script-based. In particular, Google Chrome’s V8 JavaScript runtime is particularly fast and also open sourced for use by anybody.

With all these things in place, the opportunity arose for somebody to come along with a new approach to network (web) application development. Thus, the birth of Node.js.

Inspired by some similar projects in the Ruby and Python communities, he eventually took the Chrome V8 runtime and an event-processing library called libev and came up with the first versions of a new system called Node.js. The primary methodology or innovation in Node.js is that it is built entirely around an event-driven, nonblocking model of programming. In short, you never (well, rarely) write code that blocks.

If your web application—in order to process a request and generate a response—needs to run a database query, it runs the request and then tells Node.js what to do when the response returns. In the meantime, your code is free to start processing other incoming requests or, indeed, do any other task it might need, such as cleaning up data or running analyses.

Through this simple change in the way the application handles requests and work, you are able to easily write web servers that can handle hundreds, if not thousands, of requests simultaneously on machines without much processing or memory resources. Node runs in a single process, and your code executes largely in a single thread, so the resource requirements are much lower than for many other platforms.

This speed and capacity come with a few caveats, however, and you need to be fully aware of them so you can start working with Node with your eyes wide open.

First and foremost, the new model is different from what you may have seen before and can sometimes be a bit confusing. Until you’ve wrapped your brain fully around some of the core concepts, some code you see written in Node.js can seem a bit strange. Much of this book is devoted to discussing the core patterns many programmers use to manage the challenges of the asynchronous, nonblocking way of programming that Node uses and how to develop your own.

Another limitation with this model of programming is that it really is centered around applications that are doing lots of different things with lots of different processes, servers, or services. Node.js truly shines when your web application is juggling connections to databases, caching servers, file systems, application servers, and more. The flip side of this, however, is that it’s actually not necessarily an optimal environment for writing compute servers that are doing serious, long-running computations. For these, Node’s model of a single thread in a single process can create problems if a given request is taking a ton of time to generate a complicated password digest or processing an image. In situations in which you’re doing more computationally intensive work, you need to be careful how your applications use resources or perhaps even consider farming those tasks out to other platforms and run them as a service for your Node.js programs to call.

Node.js’s path to adulthood has been a somewhat rocky one—the 0.x series of Node.js lingered for quite a while, releasing often but seemingly not making much progress, and some grew impatient with the governance of the project. This caused a schism in late 2014, with a group of people forking the open sourced code and creating io.js, a new version of node with the goals of being more open and transparent and responsive to the developer community. Fortunately, this break did not last long, and within nine months, Joyent agreed to hand over guidance of Node.js to the Technical Steering Committee (TSC) in autumn 2015.

Today, however, the platform is quite stable and predictable, and has adopted semantic versioning, where your versionsversion numbers have the format major.minor.patchlevel. In this model you only make breaking API changes with major version number changes, add features in minor version number changes, and can update and fix anything necessary in patch-level changes. Each major version is developed for 18 months and then supported for another 12 months after that, meaning you have 2.5 years of use for each version. After that, you’ll need (and definitely want) to migrate to the latest version to be sure you’re getting the latest features and most secure version of the software).

To help you keep track of and manage all of these updates, the developers have taken to labeling portions of the system with different degrees of stability, ranging from Unstable to Stable to Locked. Changes to Stable or Locked portions of the runtime are rare and involve much community discussion to determine whether the changes will generate too much pain. As you work your way through this book, we point out which areas are less stable than others and suggest ways you can mitigate the dangers of changing APIs. Newer versions of Node.js have introduced the concept of Deprecated APIs. If part of Node.js is becoming too difficult to maintain and is not heavily used, or otherwise doesn’t make sense to continue supporting, it will (again, after much community discussion) be marked as Deprecated and not included in the next major version update. This gives developers plenty of time to move to alternatives (of which there are always going to be dozens).

The good news is that Node.js already has a large and active user community and a bunch of mailing lists, forums, and user groups devoted to promoting the platform and providing help where needed. A simple Internet search will get you answers to 99 percent of your questions in a matter of seconds, so never be afraid to look!

If you’re like me, you have probably written some HTML/CSS/JavaScript and thus have “worked with” JavaScript, but you might not be intimately familiar with it and have just largely templated heavily off code found on blog posts or mailing lists. Indeed, because of its clunky UI and frustrating browser mismatches, you might even frown slightly at the mere mention of JavaScript. Fear not—by the end of the first section of this book, distasteful memories of the language will be a thing of the past and, I hope, you’ll be happily writing your first Node.js programs with ease and a smile on your face!

I also assume that you have a basic understanding of how web applications work: browsers send HTTP requests to a remote server; the server processes each request and sends a response with a code indicating success or failure, and then optionally some data along with that response (such as the HTML for the page to render or perhaps JavaScript Object Notation, or JSON, containing data for that request). You’ve probably connected to database servers in the past, run queries, and waited for the resulting rows, and so on. When I start to describe concepts beyond these in the samples and programs, I explain and refresh everybody’s memory on anything new or uncommon.

The book is divided into four major sections:

Part 1. Learning to Walk—You start installing and running Node, take another look at the JavaScript language and the extensions used in V8 and Node.js, and then write your first application.

Part 2. Learning to Run—You start developing more powerful and interesting application servers in this part of the book, and I start teaching you some of the core concepts and practices used in writing Node.js programs.

Part 3. Breaking Out the Big Guns—In this section, you look at some of the powerful tools and modules available to you for writing your web applications, such as help with web servers and communication with database servers.

Part 4. Getting the Most Out of Node.js—Finally, I close out the book by looking at a few other advanced topics such as ways in which you can run your applications on production servers, how you can test your code, and how you can use Node.js to write command-line utilities as well!

As you work your way through the book, take the time to fire up your text editor and enter the code, see how it works in your version of Node.js, and otherwise start writing and developing your own code as you go along. You develop your own little photo sharing application as you work through this book, which I hope provides you with some inspiration or ideas for things you can write.

The GitHub code has some fully functioning samples and has been tested to work on Mac, Linux, and Windows with the latest versions of Node.js. If new updates of Node require updates to the source code, I will put changes and notes there, so please be sure to pull down new versions every few months. Sadly, my code is not perfect, and I always welcome bug reports and pull requests!

If you have any questions or problems with the code in this book, feel free to go to github.com/marcwan/LearningNodeJS and add an issue; they’ll be monitored and answered reasonably quickly.