The Linux package management systems don't allow us to install two MySQL instances. We can implement some separation in the same MySQL instance by using separate databases with different usernames and access privileges for each database.

The first step is to ensure that MySQL is installed on your server. For Ubuntu, Digital Ocean has a fairly good tutorial: https://www.digitalocean.com/community/tutorials/how-to-install-mysql-on-ubuntu-14-04.

The MySQL server must support TCP connections from localhost. Edit the configuration file, /etc/mysql/my.cnf, to have the following line:

bind-address = 127.0.0.1

This limits MySQL server connections to the processes on the server. A miscreant would have to break into the server to access your database.

Now that our database server is available, let's set up two databases.

In the chap10/notes/models directory, create a file named mysql-create-db.sql containing the following:

CREATE DATABASE notes;
CREATE USER 'notes'@'localhost' IDENTIFIED BY 'notes';
GRANT ALL PRIVILEGES ON notes.* TO 'notes'@'localhost'
      WITH GRANT OPTION;

And in the chap10/users directory, create a file named mysql-create-db.sql containing the following:

CREATE DATABASE userauth;
CREATE USER 'userauth'@'localhost' IDENTIFIED BY 'userauth';
GRANT ALL PRIVILEGES ON userauth.* TO 'userauth'@'localhost'
      WITH GRANT OPTION;

When the Notes application source code is copied to the server, we'll run the scripts as:

$ mysql -u root -p <chap10/users/mysql-create-db.sql
$ mysql -u root -p <chap10/notes/models/mysql-create-db.sql

This will create the two databases, notes and userauth, with associated usernames and passwords. Each user can access only its associated database. Later we'll set up Notes and the User Authentication service with YAML configuration files to access these databases.

According to the Node.js documentation (https://nodejs.org/en/download/package-manager/), the recommended installation method for Debian or Ubuntu Linux distributions is the following:

$ curl -sL https://deb.nodesource.com/setup_5.x | sudo -E bash -
$ sudo apt-get install -y nodejs
$ sudo apt-get install -y build-essential

Installing this way means that as new Node.js releases are issued, upgrades are easily accomplished with the normal package management procedures. The build-essential package brings in all the compilers and other development tools required to build native code Node.js packages.

Before copying the Notes and User Authentication code to this server, let's do a little coding to prepare for the move. We know that the Notes and Authentication services must access the MySQL instance on localhost using the usernames and passwords given earlier.

Using the approach we've followed so far, this means a pair of YAML files for Sequelize parameters, and changing environment variables in the package.json files to match.

Create a chap10/notes/models/sequelize-server-mysql.yaml file containing:

dbname: notes
username: notes
password: notes
params:
    host: localhost
    port: 3306
    dialect: mysql

In chap10/notes/package.json, add the following line to the scripts section:

"on-server": "SEQUELIZE_CONNECT=models/sequelize-server-mysql.yaml NOTES_MODEL=models/notes-sequelize USERS_MODEL=models/users-rest USER_SERVICE_URL=http://localhost:3333 PORT=3000 node ./app",

Then create a chap10/users/sequelize-server-mysql.yaml file containing:

dbname: userauth
username: userauth
password: userauth
params:
    host: localhost
    port: 3306
    dialect: mysql

And in chap10/users/package.json, add the following line to the scripts section:

"on-server": "PORT=3333 SEQUELIZE_CONNECT=sequelize-server-mysql.yaml node ./user-server",

This configures the authentication service to access the databases just created.

Now we need to select a place on the server to install the application code:

# ls /opt

This empty directory looks to be as good a place as any. Simply upload chap10/notes and chap10/users to your preferred location, so you have the following:

# ls /opt
notes  users

Then, in each directory, run these commands:

# rm -rf node_modules
# npm install

When uploading the source to the server, you may have included the node_modules directory. That directory was built on your laptop and probably isn't suitable for the server you're running. It's unlikely your laptop is running the same OS as the server, so any native code modules will fail. The node_modules directory should be created on the target machine so that native code modules are compiled on the target machine.

The simplest method to do so is to just delete the whole node_modules directory and then let npm install do its job.

Remember that we set up the PATH environment variable the following way:

# export PATH=./node_modules/.bin:${PATH}

Also remember that in the chap10/notes directory, the npm install step will run bower install. This is likely to fail because bower insists on not being run with root privileges, for good reasons. If necessary, you can run bower the following way:

# apt-get install git
# bower --allow-root install

It's useful to record this in the package.json script as well:

"postinstall": "bower --allow-root install"

Finally, at this time, you can now run the SQL scripts written earlier to set up the database instances:

# mysql -u root -p <users/mysql-create-db.sql
# mysql -u root -p <notes/models/mysql-create-db.sql

Then you should be able to start up the services by hand to check that everything is working correctly. The MySQL instance has already been tested, so we just need to start the User Authentication and Notes services:

# cd /opt/users
# npm run on-server

> [email protected] on-server /opt/users
> PORT=3333 SEQUELIZE_CONNECT=sequelize-server-mysql.yaml node ./user-server

Then log in to the server on another terminal session and run the following:

# cd /opt/users/
# PORT=3333 node users-add.js 
Created { id: 1,
  username: 'me', password: 'w0rd', provider: 'local',
  familyName: 'Einarrsdottir',givenName: 'Ashildr',
  middleName: '', emails: '[]', photos: '[]',
  updatedAt: '2016-03-22T01:28:21.000Z',
  createdAt: '2016-03-22T01:28:21.000Z' }
# PORT=3333 node users-list.js 
List [ { id: 'me', username: 'me', provider: 'local',
   familyName: 'Einarrsdottir', givenName: 'Ashildr', middleName: '',
   emails: '[]', photos: '[]' } ]

The preceding command both tests the backend user authentication service is functioning and gives us a user account we can use to log in. The users-list command demonstrates that it works.

Now we can start the Notes service:

# cd ../notes
# npm run on-server

> [email protected] on-server /opt/notes
> SEQUELIZE_CONNECT=models/sequelize-server-mysql.yaml NOTES_MODEL=models/notes-sequelize USERS_MODEL=models/users-rest USER_SERVICE_URL=http://localhost:3333 PORT=3000 node ./app

And then we can use our web browser to connect to the application. Since you probably do not have a domain name associated with this server, Notes can be accessed via the IP address of the server, such as: http://104.131.55.236:3000.

The Twitter application we set up for Notes previously won't work because the authentication URL is incorrect for the server. For now, we can log in using the user profile created previously.

By now you know that the drill of verifying Notes is working. Create a few notes, open a few browser windows, see that real-time notifications work, and so on. Once you're satisfied that Notes is working on the server, kill the processes and move on to the next section, where we'll set this up to run when the server starts.

There are many ways to manage server processes, to ensure restarts if the process crashes, and so on. Supervisord (http://supervisord.org/) is a likely candidate for this purpose. However, we'll instead use PM2 (http://pm2.keymetrics.io/) because it's optimized for Node.js processes. It bundles process management and monitoring into one application whose purpose is managing background processes.

Install it as so (using sudo if needed):

# npm install -g [email protected]

In users/package.json, we can add the following line to the scripts section:

"on-server-pm2": "PORT=3333 SEQUELIZE_CONNECT=sequelize-server-mysql.yaml pm2 start --env PORT --env SEQUELIZE_CONNECT user-server.js",

In normal pm2 usage, we launch scripts with pm2 start script-name.js.

Because our services use environment variables for configuration, we have to add the --env options so that PM2 knows it should pay attention to these variables. We could go ahead and add a similar line in notes/package.json, but let's try something else instead. PM2 offers a better way for us to describe the processes it should start.

In the /opt directory, create a file named ecosystem.json containing the following:

{
  apps : [
    {
      name      : "User Authentication",
      script    : "user-server.js",
      "cwd"     : "/opt/users",
      env: {
        PORT: "3333",
        SEQUELIZE_CONNECT: "sequelize-server-mysql.yaml"
      },
      env_production : {
        NODE_ENV: "production"
      }
    },
    {
      name      : "Notes",
      script    : "app.js",
      "cwd"     : "/opt/notes",
      env: {
        PORT: "3000",
        SEQUELIZE_CONNECT: "models/sequelize-server-mysql.yaml",
        NOTES_MODEL: "models/notes-sequelize",
        USERS_MODEL: "models/users-rest",
        USER_SERVICE_URL: "http://localhost:3333"
      },
      env_production : {
        NODE_ENV: "production"
      }
    }
  ]
}

This file describes the directories containing both services, the script to run each service, and the environment variables to use.

We then start the services as so: pm2 start ecosystem.json, which looks like the following on the screen:

You can again navigate your browser to the URL for your server, such as http://104.131.55.236:3000, and check that the Notes application is working.

Once started, some useful commands are as follows:

# pm2 list
# pm2 describe 1
# pm2 logs 1

These commands let you query the status of the services.

The pm2 monit command gives you a pseudo-graphical monitor of system activity, as shown in the following screenshot:

As you access Notes, watch how those red bars grow and shrink showing the changes in CPU consumption. This is all cool, but if we restart the server, these processes don't start with the server. How do we handle that? It's very simple because PM2 can generate an init script for us:

# pm2 startup ubuntu
[PM2] Generating system init script in /etc/init.d/pm2-init.sh
[PM2] Making script booting at startup...
[PM2] -ubuntu- Using the command:
      su -c "chmod +x /etc/init.d/pm2-init.sh && update-rc.d pm2-init.sh defaults"

[PM2] Done.

Because we're using an Ubuntu server, we generated an Ubuntu startup script. Make sure to generate this script for the platform (Gentoo and so on) you are using. If you simply leave off the platform name, PM2 will autodetect the platform for you.

The script generated doesn't include specifics about the currently executing set of processes. Instead, it simply runs some of the PM2 commands needed to start or stop a list of processes it has been told to manage. PM2 keeps its own data about the processes you've asked it to manage.

It is necessary to run the following command:

# pm2 save

This saves the current process state so that the processes can be restarted later, such as when the system restarts.

Twitter support for the hosted Notes app

The only thing we're lacking is support for logging in with Twitter. When we registered the Notes application with Twitter, we registered a domain such as http://MacBook-Pro-2.local:3000 for our laptop. The Notes application is now deployed on the server and is used by a URL that Twitter doesn't recognize. We therefore have a couple of things to change.

On apps.twitter.com in the Notes application registration screen, we can enter the IP address of the website as so: http://104.131.55.236:3000.

Then, in notes/routes/users.js, we need to make a corresponding change to the TwitterStrategy definition:

passport.use(new TwitterStrategy({
    consumerKey: ".. KEY",
    consumerSecret: ".. SECRET",
    callbackURL: "http://104.131.55.236:3000/users/auth/twitter/callback"
  },

});

Upon making those changes, restart the services:

And then you'll be able to log in using your Twitter account.

We now have the Notes application under a fairly good management system. We can easily update its code on the server and restart the service. If the service crashes, PM2 will automatically restart it. Log files are automatically kept for our perusal.

PM2 also supports deployment from the source on our laptop, which we can push to staging or production environments. To support this, we must add deployment information to the ecosystem.json file and then run the pm2 deploy command to push the code to the server. See the PM2 website for more information.

While PM2 does a good job at managing server processes, the system we've developed is insufficient for an Internet-scale service. What if the Notes application were to become a viral hit and suddenly we need to deploy a million servers spread around the planet? Deploying and maintaining servers one at a time, like this, is not scalable.

We also skipped over implementing the architectural decisions at the beginning. Putting the user authentication data on the same server is a security risk. We want to deploy that data on a different server, under tighter security.

In the next section, we'll explore a new system, Docker, that solves these problems and more.