npm – the Node.js package management system

As described in Chapter 2, Setting up Node.js, npm is a package management and distribution system for Node.js. It has become the de-facto standard for distributing modules (packages) for use with Node.js. Conceptually, it's similar to tools such as apt-get (Debian), rpm/yum (Redhat/Fedora), MacPorts (Mac OS X), CPAN (Perl), or PEAR (PHP). Its' purpose is publishing and distributing Node.js packages over the Internet using a simple command-line interface. With npm, you can quickly find packages to serve specific purposes, download them, install them, and manage packages you've already installed.

The npm package defines a package format for Node.js largely based on the CommonJS package specification. It uses the same package.json file that's supported natively by Node.js, but with additional fields to build in additional functionality.

The npm package format

An npm package is a directory structure with a package.json file describing the package. This is exactly what we just referred to as a directory module except that npm recognizes many more package.json tags than Node.js does. The starting point for npm's package.json are the CommonJS Packages/1.0 specification. The documentation for npm's package.json implementation is accessed using the following command:

$  npm help json

A basic package.json file is as follows:

{ "name": "packageName",
   "version": "1.0",
   "main": "mainModuleName",
   "modules": {
    "mod1": "lib/mod1",
    "mod2": "lib/mod2"
  }
}

The file is in JSON format, which, as a JavaScript programmer, you should be familiar with.

The most important tags are name and version. The name will appear in URLs and command names, so choose one that's safe for both. If you desire to publish a package in the public npm repository, it's helpful to check whether a particular name is already being used at http://npmjs.com or with the following command:

$ npm search packageName

The main tag is treated the same as we discussed in the previous section on directory modules. It defines which file module will be returned when invoking require('packageName'). Packages can contain many modules within themselves and they can be listed in the modules list.

Packages can be bundled as tar-gzip (tarballs), especially to send them over the Internet.

A package can declare dependencies on other packages. That way, npm can automatically install other modules required by the module being installed. Dependencies are declared as follows:

"dependencies": {
    "foo" : "1.0.0 – 2.x.x", 
    "bar" : ">=1.0.2 <2.1.2"
 }

The description and keyword fields help people find the package when searching in an npm repository (http://search.npmjs.org). Ownership of a package can be documented in the homepage, author, or contributors fields:

"description": "My wonderful packages that walks dogs",
  "homepage": "http://npm.dogs.org/dogwalker/",
  "author": "[email protected]"

Some npm packages provide executable programs meant to be in the user's PATH. These are declared using the bin tag. It's a map of command names to the script which implements that command. The command scripts are installed into the directory containing the node executable using the name given:

bin: {
  'nodeload.js': './nodeload.js',
  'nl.js': './nl.js'
},

The directories tag describes the package directory structure. The lib directory is automatically scanned for modules to load. There are other directory tags for binaries, manuals, and documentation:

directories: { lib: './lib', bin: './bin' },

The script tags are script commands run at various events in the life cycle of the package. These events include install, activate, uninstall, update, and more. For more information about script commands, use the following command:

$ npm help scripts

We've already used the scripts feature when showing how to set up Babel. We'll use these later for automating the build, test, and execution processes.

This was only a taste of the npm package format; see the documentation (npm help json) for more.

Finding npm packages

By default, npm modules are retrieved over the Internet from the public package registry maintained on http://npmjs.com. If you know the module name, it can be installed simply by typing the following:

$ npm install moduleName

But what if you don't know the module name? How do you discover the interesting modules? The website http://npmjs.com publishes a searchable index of the modules in that registry.

The npm package also has a command-line search function to consult the same index:

Finding npm packages

Of course, upon finding a module, it's installed as follows:

$ npm install acoustid

After installing a module you may want to see the documentation, which would be on the module's website. The homepage tag in package.json lists that URL. The easiest way to look at the package.json file is with the npm view command, as follows:

$ npm view akashacms
...
{ name: 'akashacms',
  description: 'A content management system to generate static websites using latest HTML5/CSS3/JS goodness.',
  'dist-tags': { latest: '0.4.16' },
  versions: 
   [ '0.0.1',

  author: 'David Herron <[email protected]> (http://davidherron.com)',
  repository: { type: 'git', url: 'git://github.com/akashacms/akashacms.git' },
  homepage: 'http://akashacms.com/',

You can use npm view to extract any tag from package.json, like the following which lets you view just the homepage tag:

$ npm view akashacms homepage
http://akashacms.org/

Other fields in the package.json can be viewed by simply giving the desired tag name.

Other npm commands

The main npm command has a long list of subcommands for specific package management operations. These cover every aspect of the life cycle of publishing packages (as a package author), and downloading, using, or removing packages (as an npm consumer).

You can view the list of these commands just by typing npm (with no arguments). If you see one you want to learn more about, view the help information:

$ npm help <command>
The help text will be shown on your screen.  
Or, see the website: http://docs.npmjs.com

Installing an npm package

The npm install command makes it easy to install packages upon finding the one of your dreams, as follows:

$ npm install express
/home/david/projects/notes/
[email protected]

The named module is installed in node_modules in the current directory.

If you instead want to install a module globally, add the -g option:

$ npm install -g grunt-cli

If you get an error, and you're on a Unix-like system (Linux/Mac), you may need to run this with sudo:

$ sudo npm install -g grunt-cli

Initializing a new npm package

If you want to create a new package, you can create the package.json file by hand or you can get npm's help. The npm init command leads you through a little dialog to get starting values for the package.json file.

Initializing a new npm package

Once you get through the questions, the package.json file is written to disk.

Maintaining package dependencies with npm

In the preceding screenshot, you'll see that npm instructs you how to add package dependencies to package.json. What happens is npm looks in package.json for the dependencies or devDependencies field, and it will automatically install the mentioned npm packages.

Suppose you've run this:

$ npm install akashacms –-save

In response, npm will add a dependencies tag to package.json:

"dependencies": {
    "akashacms": "^0.4.16"
}

Now, when your application is installed, npm will automatically also install any packages listed in the dependencies.

The devDependencies are modules used during development. That field is initialized the same as above but with the –-save-dev flag.

By default, when an npm install is run, modules listed in both dependencies and devDependencies are installed. Of course, the purpose for having two lists is to not install the devDependencies in some cases:

$ npm install –-production

This installs only the modules listed in dependencies and none of the devDependencies modules.

In the Twelve-Factor application model, it's suggested that we explicitly identify the dependencies required by the application. On the Node.js platform, npm gives us this dependencies section, including a flexible mechanism to declare compatible package versions by their version number.

Throughout this book, we'll do the following:

$ npm install [email protected] --save

This combination installs the specified version, and adds the dependency declaration to the package.json file. Using a version number such as 4.x is a convenience because it will automatically install the latest of a given version number range.

For example, we know that Express 5.x is in its alpha phase at the time of writing this, with the full release some time in the future, and that 5.x will introduce changes that might break the code shown in this book. By instructing you to install 4.x, you'll be insulated from potential breaking changes.

Fixing bugs by updating package dependencies

Bugs exist in every piece of software. An update to the Node.js platform may break an existing package, as might an upgrade to packages used by the application, or your application may trigger a bug in a package it uses. In these and other cases, fixing the problem might be as simple as updating a package dependency to a later (or earlier) version.

You must first identify whether the problem exists in the package or in your code. Once you've determined that it's a problem in another package, it's time to investigate whether the package maintainers have already fixed the bug. Is the package hosted on Github or another service with a public issue queue? Look for an open issue on this problem. That investigation will tell you whether to update the package dependency to a later version. Sometimes, it will tell you to revert to an earlier version; for example, if the package maintainer introduced a bug that doesn't exist in an earlier version.

Sometimes, you will find that the package maintainers are unprepared to issue a new release. In such a case, you can fork their repository and create a patched version of their package. The package.json dependencies section lets you reference packages in many ways other than npm version numbers. For example, you can list a URL to a tarball or a reference to repositories on services such as Github. Refer to "npm help package.json" for more information on your options. Consider the following example:

"dependencies": {
    "express": "http://example.com/my/patched/express.tar.gz"
}

During the development of this book, an example of breakage due to a Node.js platform update occurred. Node.js 6.0 was released, bringing with it many great new features, including deeper compatibility with ES-2015 language features. It also introduced changes that broke some modules, including the sqlite3 module we will use starting in Chapter 7, Data Storage and Retrieval. An updated release was quickly made, but this begs a question of what to do when this happens.

A situation like this generally prevents us from moving to the newer platform version until all our dependent packages work with the new version. The specific action to take depends on if, or how badly, the new platform version breaks your application.

It is for these reasons that the Twelve-Factor application model recommends explicitly declaring specific dependencies to install. Listing an explicit version number in package.json means that the package maintainer can release a new version, but you're in control of if, or when, you adopt that version.

Declaring Node.js version compatibility

It's important that your Node.js software run on the correct version of Node.js. Breaking changes are introduced from time to time in the platform. It's either intentional when new features are added or unintentional when a bug is added. What's most common is to require a specific Node.js version, or later, because of which release in which feature was added.

For example, the ES2015 features started to be implemented in the 4.2.x release, and a large number of other ES2015 features were added in 6.0.

This dependency is declared in package.json using the engines tag:

"engines": {
    "node": ">= "5.x"
}

This, of course, uses the same version number matching that we just discussed.

Updating outdated packages you've installed

The coder codes, updating their package, leaving you in their dust unless you keep up.

To find out if your installed packages are out of date, use the following command:

$ npm outdated

The report shows the current npm packages, the currently installed version, as well as the current version in the npm repository. Updating the outdated packages is very simple:

$ npm update express
$ npm update

Installing packages from outside the npm repository

As awesome as the npm repository is, we don't want to push everything we do through their service. This is especially true for internal development teams who cannot publish their code for all the world to see. While you can rent a private npm repository, there's another way. You can install packages from other locations. Details about this are in package.json help.

Your package dependency can be any URL that downloads a tarball, that is, a .tar.gz file:

$ npm install http://dev.mydomain.com/path/to/v1.2.3/packageName.tar.gz --save

One way to use this from a Github repository is that each time a release is made, a new tarball is generated at a versioned URL. Simply navigate to the Releases tab of the Github repository.

There's also direct support for Git URLs' even ones using SSH authentication. Since Git URLs can include a hash code for a specific repository revision, branch, or tag, this is also versioned:

$ npm install git+ssh://user@hostname:project.git#tag --save

The #tag portion is where you can specify the release tag. For Git repositories on Github, there's a shortcut:

$ npm install strongloop/express --save

Publishing an npm package

All those packages in the npm repository came from people like you with an idea of a better way of doing something. It is very easy to get started with publishing packages. Online docs can be found at https://docs.npmjs.com/getting-started/publishing-npm-packages.

You first use the npm adduser command to register yourself with the npm repository. You can also sign up with the website.

Next, you log in using the npm login command.

Finally, while sitting in the package root directory, use the npm publish command. Then, stand back so that you don't get stampeded by the crush of thronging fans. Or, maybe not. There are over 250,000 packages in the repository, with well over 400 packages added every day. To get yours to stand out, you will require some marketing skill, which is another topic beyond the scope of this book.

Package version numbers

We've been showing version numbers all along without explaining how npm uses them.

Node.js doesn't know anything about version numbers. It knows about modules, and it can treat a directory structure as if it were a module. It has a fairly rich system of looking for modules, but it doesn't know version numbers.

The npm package, however, knows about version numbers. It uses the Semantic Versioning model (http://semver.org), and as we saw, you can install modules over the Internet, query for out-of-date modules, and update them with npm. All of this is version controlled, so let's take a closer look at the things npm can do with version numbers and version tags.

Earlier, we used npm list to list installed packages, and the listing includes version numbers of installed packages. If instead, you wish to see the version number of a specific module, type the following command:

$ npm view express version
4.13.4

Whenever npm commands take a package name, you can append a version number or version tag to the package name. This way, you can deal with specific package versions if needed. Sometimes, package maintainers introduce bugs that you want to avoid. The dependencies in your application can list a specific version number. You can list specific version numbers, changing the version dependency only when you're satisfied it is safe to do so. The npm install command, and several other npm commands, take version numbers as follows:

$ npm install [email protected]

The npm help package.json documentation has a full discussion of what you can do with version number strings, including these choices:

  • Exact version match: 1.2.3
  • At least version N: >1.2.3
  • Up to version N: <1.2.3
  • Between two releases: >=1.2.3 <1.3.0

One feature of the Twelve-Factor methodology is step 2, explicitly declaring your dependencies. We've already touched on this, but it's worth reiterating and to see how npm makes this easy to accomplish.

Step 1 of the Twelve-Factor methodology is ensuring that your application code is checked into a source code repository. You probably already know this, and even have the best of intentions to ensure that everything is checked in. With Node.js, each module should have its own repository rather than putting every single last piece of code in one repository.

Doing so ensures that each module progresses on its own timeline. For example, a breakage in one module is easy to back out by changing the version dependency in package.json.

This gets us to step 2. There are two aspects of this step, one of which is the package versioning we discussed previously. With npm, we can explicitly declare module version numbers in dependencies and devDependencies. It's extremely easy, then, to ensure that everyone on the team is on the same page, developing against the same versions of the same modules. When it's time to deploy to testing, staging, or production servers, and the deployment script runs npm install or npm update, the code will use a known version of the module.

The lazy way to declare dependencies is putting * in the version field. Doing so uses whatever is the latest version in the npm repository. Maybe this will work for you, until one day the maintainers of that package introduce a bug. You'll head over to the Github site for the package, look in the issue queue, and see that others have already reported the problem you're seeing. Some of them will say that they've pinned on the previous release until this bug is fixed. What that means is their package.json file does not depend on * for the latest version but on a specific version number before the bug was created.

Don't do the lazy thing, do the smart thing.

The other aspect of explicitly declaring dependencies is to not implicitly depend on global packages. Earlier, we said that some in the Node.js community caution against installing modules in the global directories. This might seem like an easy shortcut to sharing code between applications. Just install it globally, and you don't have to install the code in each application.

But, doesn't that make deployment harder? Will the new team member be instructed on all the special files to install here and there to make the application run? Will you remember to install that global module on all destination machines?

You don't want installation instructions to be the length of a small book full of arcane steps to ensure correct setup. That's error prone and hard to automate for modern cloud-based computing systems. Instead, you would want to automate as much of the installation and setup as possible.

For Node.js, that means listing all the module dependencies in package.json, and then the installation instructions are simply npm install, followed perhaps by editing a configuration file.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset