Chapter 4: Symmetric Encryption in Node.js
If you stopped a random person on the street and asked them what is the first thing that comes to their mind when they think of cryptography, chances are they'd talk about encrypting their data and protecting it with a password or PIN. This class of operations is called symmetric encryption, and it's arguably the most widely known, even among not particularly tech-savvy consumers.
In Chapter 3, File and Password Hashing with Node.js, we began looking at classes of cryptographic operations. starting with hashing functions, and we explained how they're different from encryption. While hashing is a one-way operation, encryption is reversible: you can obtain the plaintext back from an encrypted message (also called ciphertext).
In this chapter, we'll look at data encryption using symmetric algorithms, specifically AES and ChaCha20-Poly1305, by covering the following topics:
- The difference between symmetric and asymmetric ciphers
- Symmetric encryption and decryption with AES, with code samples for Node.js
- Using ChaCha20-Poly1305 with Node.js
- How to derive symmetric encryption keys from passphrases
- How (and why) to wrap symmetric keys with AES-KW using Node.js
Technical requirements
All the code samples for this chapter can be found in this book's GitHub repository at https://bit.ly/crypto-ch4.
Symmetric and asymmetric encryption
There are two main kinds of ciphers (and, consequentially, of encryption): symmetric and asymmetric, and the difference lies in the kind of keys that are used.
Symmetric encryption is probably the most well-known of the two, and the one that consumers are more likely to be familiar with as well. With symmetric encryption, you encrypt data using a key, and then use the very same key to decrypt it again. In many cases, the key can be derived from a password or passphrase, as we'll see at the end of this chapter.
Conceptually, algorithms such as AES (one of the most popular and widely used symmetric ciphers) and ChaCha20-Poly1305 work this way.
If you wanted to share the encrypted message with another person, you'd need to provide the other party with both the ciphertext (the encrypted data) and the key. Like many people, even consumers have experienced, in this case, the challenge of sharing the encryption key securely. Having the ciphertext and the key is both necessary and sufficient to decrypt the data, which means that great care must be put into protecting symmetric keys (in cryptography, it's assumed that the ciphertext doesn't necessarily need to be kept a secret). This can be especially problematic when you are transferring data over an untrusted channel, such as the Internet.
For example, consider a scenario in which you need to email a friend an encrypted ZIP file. If you send the key/passphrase together with the encrypted data (for example, you write the passphrase in the body of the email and add the ciphertext as an attachment), an attacker that's eavesdropping could steal both at the same time, voiding the protection offered by encryption.
To solve these issues, we can use asymmetric encryption. With that, there are two different, but related, keys: a private one and a public one.
With asymmetric encryption, you encrypt data with a public key and decrypt it with the corresponding private key.
This makes sharing data over an untrusted channel easier: if your friend wanted to send you an encrypted message, they'd only need the public part of your key to encrypt it. They'd then send the ciphertext to you, and you'd use your private key to decrypt it. The public key alone is not sufficient (nor necessary) to decrypt a ciphertext, and once your friend has encrypted what they wanted to send you, it can only be decrypted with your private key.
There are various algorithms for asymmetric encryption, sometimes called public-key cryptography, with RSA being one of the most popular ones.
Public key (asymmetric) cryptography is more complex than the symmetric one, both conceptually and in how it's used in practice. Going back to the previous example in which you were trying to send a ZIP file to a friend protected with encryption, you could safely do so with S/MIME or GPG, two standards and sets of tools that leverage public key cryptography to remove the issue of having to share the symmetric key. Both those standards have been around for a while, tracing their roots back to the 1990s. Yet, if you've never heard of them before, or if you're very confused by how they are set up and used, you are in very good company: the complexity of using them and the limited support in apps such as email clients makes their adoption fairly small, especially among the general public.
Even though asymmetric encryption may lack in how it's understood and leveraged with intention by the general public, it's still widely used, daily and transparently, by virtually every person on Earth. For example, public-key cryptography is what makes protocols such as TLS and HTTPS, which are foundational to the modern Internet, possible – among many other things that underpin common everyday activities.
Now that you understand the two different kinds of ciphers, we're going to dedicate this chapter to symmetric ones, starting with AES. We'll cover asymmetric ciphers in more detail in Chapter 5, Using Asymmetric and Hybrid Encryption in Node.js, and Chapter 6, Digital Signatures with Node.js and Trust, where we will cover data encryption and digital signatures, respectively.
Symmetric encryption with AES
The Advanced Encryption Standard (AES) is one of the most widely used symmetric ciphers, and it's been like that since its standardization in the early 2000s. It's safe (approved by the US government for encrypting "top secret" documents) and fast, with support available in all operating systems and programming languages.
Additionally, hardware acceleration for AES is available in all modern desktop and server CPUs (for example, the AES-NI instructions in Intel and AMD processors), as well as in a large number of mobile/embedded chips, making executing AES rather cheap in terms of computing power. A regular consumer-grade CPU can easily encrypt or decrypt AES streams at the speed of multiple gigabits per second. Many consumer operating systems are now encrypting hard drives by default, transparently to the user, using AES – that's the case of BitLocker on Windows and FileVault on macOS, for example.
To use AES, you need a key that is 128-, 192-, or 256-bit in length, and these keys should either be random sequences of bytes or derived from a passphrase using a Key Derivation Function (we'll look at how to derive a key from a passphrase at the end of this chapter).
Before we dive into using AES with code samples, however, we need to look at three important aspects, including two decisions we need to make (the length of the encryption key and the mode of operation) and the concept of initialization vectors.
Key length
The first thing we should discuss is the length of the key. We mentioned that the standard for AES defines three different key lengths: 128, 192, or 256 bits (16, 24, or 32 bytes, respectively), with 192-bit keys rarely used in practice. Shorter keys are not allowed in the standard, and even if they were, you'd be strongly advised against using them.
As we saw in Chapter 1, Cryptography for Developers, algorithms that haven't been broken – and AES has certainly not been broken yet – can only be cracked using brute force. This means an attacker would have to try every single combination that is possible and hope to find the right one. Although brute-force attacks are always possible in theory, the goal of the defending party is to make them impossible in practice, by making the effort required by an attacker (time and energy) too big to even attempt.
When deciding between the various sizes, you might think that 256-bit is safer, and you'd be correct in theory. By using twice as many bits for the key's length, a successful brute-force attack against AES-256 would require something significantly more effort: not double, but the square (power of two) of the time.
In practice, using a 128-bit key is generally safe at the time of writing and for the foreseeable future. As we saw in the Defining safe section of Chapter 1, Cryptography for Developers, even leveraging the entire computing power of every Bitcoin miner (which aren't optimized for brute-forcing AES, but we hypothetically assume they could be repurposed for that), it would take 1010 years to break AES with a 128-bit key, which is comparable with the age of the Universe. This means that even though AES-256 does provide significantly more strength than AES-128 in theory, in practical terms, the protection they offer can safely be considered equal. At the same time, AES-128 is faster due to performing fewer rounds of the algorithm (10 rounds instead of 14).
Yet, this is not the end of the argument.
After convincing you that AES-128 offers essentially the same security as AES-256 in practical terms, it's worth pointing out that unless your application is particularly sensitive to the performance loss, you may want to adopt 256-bit keys for AES regardless.
When 1Password (a popular password manager app) migrated from using 128-bit to 256-bit keys for AES, their "Chief Defender Against the Dark Arts" Jeffrey Goldberg wrote a well-thought and well-explained piece on that (https://bit.ly/crypto-aes256). The reasons for that choice can be applied to a lot of other applications too, and we can summarize them in three points:
- AES-256 provides double the key size but is only 40% slower than AES-128 (14 rounds instead of 10), rather than twice as slow. On modern systems, AES is really fast, especially when hardware acceleration is available, so the extra performance cost of using 256-bit keys is negligible for most applications.
- As we mentioned in Chapter 1, Cryptography for Developers, quantum computing can pose a threat to certain cryptographic algorithms, and AES is one of them. Against a quantum computer, AES key sizes are effectively halved, so a 128-bit key has the strength of a 64-bit one (which means that the time needed for a brute force is its square root).
At the time of writing, this is a purely theoretical threat: we're still far from quantum computers that could be used to mount practical attacks against AES keys even smaller than 64-bit. Virtually all quantum computers in existence today are experimental ones in advanced research labs, and it's unclear how long it will take before they will be able to break a 64-bit key in a practical amount of time.
- Lastly, there are psychological and marketing aspects involved. 256-bit keys sound better than 128-bit ones, so it can make people feel more comfortable to use the longer keys, even if, in practice, they're equally safe.
To summarize, in practice both 128-bit and 256-bit keys are equally safe.
If you need your code to run as fast as possible or are processing very large amounts of data, it's perfectly fine to use a 128-bit key.
However, if your system can tolerate the slightly longer processing required by a 256-bit key, you may want to use them to protect against the possible future threats of quantum computing, and because of the psychological effects.
Mode of operation
The other decision we need to make is the mode of operation. AES can be used with a variety of modes of operation; looking at each is beyond the scope of this book.
We can limit ourselves to two that we can recommend for most applications:
- AES-CBC (Cipher Block Chaining) when you don't need authentication
- AES-GCM (Galois/Counter Mode) when you need authentication
We will mostly ignore the other modes of operation in this book. There are situations where they can be useful, but those are much less common and are specific to certain applications or domains.
Something worth mentioning, given how often it's mentioned in articles online, is that it's very important not to use the AES-ECB mode (Electronic Code Book). This is the simplest of all modes, but it is not suitable for encrypting data larger than one block (16 to 32 bytes, depending on the size of the key) because each block of the message is encrypted independently. The result does not hide the plaintext well, as you can see with your own eyes:
Figure 4.1 – Encrypting an image (the Packt logo at the top) with AES-ECB (middle) and other AES modes (bottom)
Here, the original image at the top is encrypted with AES-EBC in the middle. Even though it's encrypted, it's possible to see the original image's "shape." Using AES in any other mode returns an image like the one at the bottom, which looks "random."
Going back to our two recommended modes of operation, the next thing we need to cover is authenticated encryption. All modes of operation ensure confidentiality, meaning that if you encrypt a message, no one should be able to understand it from the ciphertext.
Authenticated Encryption (AE) means that not only is the data encrypted (guaranteeing confidentiality), but also that a "tag" is added to guarantee the integrity of the message when it's decrypted. This is useful if you want to ensure that it's possible to determine whether anyone has altered the ciphertext. The tag works similarly to a hash/checksum, but it's calculated efficiently while the data is being encrypted.
When using AE, if someone were to alter the ciphertext, when you decrypt it, the authentication check would fail. Without AE, while the attacker would still be unable to decrypt your ciphertext, they could alter it and make you decrypt invalid data: this may corrupt your application's state or cause other unintended consequences.
Authenticated ciphers and decrypting data
Note that because the integrity of the message cannot be verified until the ciphertext has been decrypted in full, your application should not begin processing data until the decryption process is complete and the authentication tag has been validated. This means, for example, that if you're decrypting a stream that was encrypted with an authenticated cipher, you first need to decrypt it entirely and verify the authentication tag before you can begin processing it; otherwise, there's no guarantee that the data you're working with hadn't been tampered with.
In some cases, when fully decrypting a stream is not possible or not desirable, an option is to chunk the original message and encrypt each chunk separately with the authenticated cipher. An example of a scheme like that can be found in the DARE protocol used by the MinIO project, which you can read more about here: https://bit.ly/crypto-dare (it also includes a reference implementation in Go).
Ultimately, the choice of what mode of operation to use depends on your application, how the data is stored and/or transmitted, and whether you are hashing the data separately. If we were to make a very high-level recommendation, we would recommend the following:
- CBC should be your default mode of operation.
- GCM can be used when you're going to store the ciphertext in certain places, or transmit it through channels, where others could potentially alter your encrypted data.
Initialization vector
The last thing you need to know about AES is that in most modes of operation (including CBC and GCM), it requires an initialization vector (IV).
An IV is a random sequence of bytes that should be regenerated every time you're encrypting a file. As a rule, you should never reuse an IV.
The size of the IV is fixed and independent of the size of the key, and it is as follows:
- 16 bytes for AES-CBC
- 12 bytes for AES-GCM
The IV does not need to be kept secret, and it's common to store the IV, in plaintext form, alongside the encrypted data (ciphertext), such as at the beginning of the encrypted stream.
Using AES with Node.js
The good news for us is that Node.js includes built-in support for AES in the crypto module, including hardware acceleration where available. The two main methods are createCipheriv(algorithm, key, iv), for creating a cipher (for encrypting data), and createDecipheriv(algorithm, key, iv), for creating a decipher (for decrypting data).
Both methods accept an algorithm as the first argument, which includes the key size and the mode of operation. Considering 128- or 256-bit keys, and CBC or GCM as the mode of operation, you have four options to choose from for the algorithm parameter: aes-128-cbc, aes-256-cbc, aes-128-gcm, or aes-256-gcm.
You can also see the full list of supported ciphers in your system by using crypto.getCiphers(), which may contain over a hundred different options (depending on the version of Node.js and the OpenSSL library linked to it), including legacy ones.
Example – AES-256-CBC
Let's look at an example of using AES-CBC with a 256-bit key to encrypt and then decrypt a message using Node.js:
4.1: Encryption and decryption with AES-256-CBC (aes-256-cbc.js)
const crypto = require('crypto')
const randomBytes = require('util')
.promisify(crypto.randomBytes)
async function encrypt(key, plaintext) {
const iv = await randomBytes(16)
const cipher = crypto.createCipheriv('aes-256-cbc', key, iv)
const encrypted = Buffer.concat([
cipher.update(plaintext, 'utf8'),
cipher.final()
])
return Buffer.concat([iv, encrypted])
}
function decrypt(key, message) {
const iv = message.slice(0, 16)
const ciphertext = message.slice(16)
const decipher = crypto.createDecipheriv('aes-256-cbc', key, iv)
const decrypted = Buffer.concat([
decipher.update(ciphertext, 'utf8'),
decipher.final()
])
return decrypted.toString('utf8')
}
We start the code by importing the crypto module. Just like we did in the previous chapters, we are using util.promisify to "modernize" some older methods, such as crypto.randomBytes, so that they return a Promise (which can be used in an async function with the await keyword).
In the preceding code, we have defined two functions:
- As its name suggests, encrypt is used to encrypt a plaintext message (a string) with a given 256-bit symmetric key (in a Buffer object), returning the ciphertext.
The method begins by generating an IV as a random 16-byte sequence. Then, it creates a cipher (the object that will encrypt our data) with createCipheriv, using the key that was passed and the random IV.
Next, the code uses cipher.update to encrypt the message, and then signals that no more data is coming with cipher.final (you could invoke cipher.update to pass chunks of data as many times as you need, until you invoke cipher.final).
At the end of the function, we return the result, which is the concatenation of two buffers: the IV and the ciphertext. It's common practice to store the IV before the ciphertext.
- Conversely, the decrypt method performs the opposite operation, decrypting a ciphertext (in a Buffer object) with a given 256-bit symmetric key (again, as a Buffer argument), and returns the plaintext message as a string.
Because we are storing the IV at the beginning of the encrypted message, the first thing we need to do is slice the message argument into two parts. The first 16 bytes are the IV, while the rest is the actual ciphertext.
Next, we create a decipher with createDecipheriv, passing the key and the IV that was extracted from the beginning of the message. We then use the decipher object, just like we did with the cipher object in the previous method, invoking decipher.update with the ciphertext to decrypt and use decipher.final when we're done.
Because the message that was encrypted at the beginning (with the encrypt function) was a string, the decrypt function returns the result as a string too, in its original UTF-8 representation.
Let's look at how we can use these functions. In this example, we are generating a new, random 256-bit key every time using the crypto.randomBytes method, which we encountered in Chapter 2, Dealing with Binary and Random Data. At the end of this chapter, we'll learn how to derive a key from a passphrase as well:
4.2: Example of using the AES-256-CBC functions
;(async () => {
const plaintext = 'Hello world!'
const key = await randomBytes(32)
console.log('Key:', key.toString('base64'))
const encrypted = await encrypt(key, plaintext)
console.log('Encrypted message:', encrypted. toString('base64'))
const decrypted = decrypt(key, encrypted)
console.log('Decrypted message:', decrypted)
})()
In this case, we are wrapping our code in an async Immediately-Invoked Function Expression (IIFE) with ;(async () => { … })() so that we can use the await keyword.
After defining the plaintext we want to encrypt (in this simple example, the Hello world! string), we generate a random key of 32 bytes (256 bits).
Next, we encrypt the message and then decrypt it again to verify that the output matches the input.
In the console, you should see a result similar to this (but one that changes every time):
Key:
eydo/M0UBy62ipiqGn4bhUQsiA4HWJ0mVtdi4W72urQ=
Encrypted message: xwS/cTRCPgVoTeARmmbajlkUm4P2TC8mY9devEt+Kcw=
Decrypted message:
Hello world!
Example – AES-256-GCM
With Node.js, the code for encrypting and decrypting data with AES-GCM is similar to the one we saw moments ago for AES-CBC, but it has one important difference.
As we mentioned previously, AES-GCM is an authenticated cipher, which means that in addition to the ciphertext, it also returns an authentication tag that must be stored alongside the encrypted message and must be provided to the decipher. The decipher will need this tag to verify that the data that was decrypted is valid and was not tampered with.
Let's look at how the encrypt and decrypt functions we saw in the AES-256-CBC example need to be modified for AES-256-GCM, with the changes in bold:
4.3: Encryption and decryption with AES-256-GCM (aes-256-gcm.js)
const crypto = require('crypto')
const randomBytes = require('util')
.promisify(crypto.randomBytes)
async function encrypt(key, plaintext) {
const iv = await randomBytes(12)
const cipher = crypto.createCipheriv('aes-256-gcm', key, iv)
const encrypted = Buffer.concat([
cipher.update(plaintext, 'utf8'),
cipher.final()
])
const tag = cipher.getAuthTag()
return Buffer.concat([iv, tag, encrypted])
}
function decrypt(key, message) {
const iv = message.slice(0, 12)
const tag = message.slice(12, 28)
const ciphertext = message.slice(28)
const decipher = crypto.createDecipheriv('aes-256-gcm', key, iv)
decipher.setAuthTag(tag)
const decrypted = Buffer.concat([
decipher.update(ciphertext, 'utf8'),
decipher.final()
])
return decrypted.toString('utf8')
}
The main differences from the previous example are as follows:
- The IV for AES in GCM mode is 12 bytes long (rather than 16 in CBC mode).
- The name of the cipher for the crypto.createCipheriv and crypto.createDecipheriv methods is 'aes-256-gcm'.
- After invoking cipher.final in the encrypt method, the cipher object contains the 16-byte authentication tag, which can be retrieved with cipher.getAuthTag(). We are storing this value in the resulting message, after the IV but before the ciphertext.
- Because we're storing the authentication tag at the beginning of the message, the decipher function needs to slice the encrypted message buffer into three parts: 12 bytes for the IV, 16 bytes for the authentication tag, with the rest being the ciphertext. After creating the decipher object, before we start passing the ciphertext to it, we need to invoke decipher.setAuthTag with the authentication tag.
You can invoke these functions using the very same code as in example 4.2, which was for AES-CBC. This time, you'll see that your encrypted message is a little bit longer due to the presence of the authentication tag.
The main difference is that if you tried modifying the message encrypted with AES-CBC, the decrypt function would still succeed, but your output would look different from the input. However, with AES-GCM, you'd get a runtime exception, notifying you that the authentication tag does not match.
Using streams
Just like we saw with hashing in Chapter 3, File and Password Hashing with Node.js, the functions we have defined so far require loading the entire message in memory (in a string or Buffer object), so they're not suitable for encrypting large files.
Luckily, the Cipher and Decipher objects, which are returned by crypto.createCipheriv and crypto.createDecipheriv, respectively, can also be used with streams. This is particularly useful when you're trying to encrypt or decrypt large files (or messages of unknown length) because data will only be loaded in memory a chunk at a time.
When working with streams, using streaming ciphers such as AES with the GCM mode of operation is normally better. If you don't need authenticated encryption, the CTR (counter mode) is another streaming cipher that can be used as an option (note that this is not one of the two modes we encountered, and recommended, earlier).
The challenge with using authenticated ciphers and streams is that the authentication tag is generated by the cipher object once the entire file has been encrypted, yet it's needed by the decipher object before decryption can begin. This means that we won't be able to store the authentication tag at the beginning of the file, as we did previously (and like we're still doing for the IV) because we will have already flushed the data to the output stream. Instead, we need to keep the authentication tag separate from the file, such as in a database.
Let's look at an example of encrypting a file from disk, called photo.jpg, and writing the encrypted file in photo.jpg.enc, using AES-GCM. It then decrypts the file into photo.jpg.orig and uses the authentication tag to validate that the encrypted file was not tampered with. We'll start with the encrypt function:
4.4: Encrypting and decrypting a stream using AES-256-GCM (aes-stream.js)
const crypto = require('crypto')
const randomBytes = require('util')
.promisify(crypto.randomBytes)
async function encrypt(key, source, destination) {
const iv = await randomBytes(12)
return new Promise((resolve, reject) => {
const cipher = crypto.createCipheriv('aes-256-gcm', key, iv)
cipher.on('end', () => {
const tag = cipher.getAuthTag()
resolve(tag)
})
cipher.on('error', (err) => {
reject(err)
})
destination.write(iv)
source.pipe(cipher).pipe(destination)
})
}
Just like in the previous chapter's example, code that uses streams is a bit more convoluted, so let's go through this step by step:
- The function accepts three parameters: key is the 256-bit symmetric key (as a Buffer) object, just like in the previous examples; source and destination are a readable stream to the input and a writable stream to the output, respectively.
- We are wrapping the function's main code in a Promise object's callback so that we can use encrypt as an async function (and await on its result). This is similar to what we did in Chapter 3, File and Password Hashing with Node.js, when hashing a stream.
- Inside the promise's callback, we create the cipher object, just like we did in the previous examples.
- We attach a callback to when the cipher stream ends ( cipher.on('end', () => { … }) ). Inside that, we retrieve the authentication tag and then we resolve the promise with the tag as a value. As you'll see in the following invocation example, this allows you to do const tag = await encrypt(key, source, destination)and have the tag returned at the end of the processing.
- We also attach a callback to reject the promise when there's an error in the streams: cipher.on('error', (err) => { reject(err) }). This makes our encrypt function throw an exception in case of an error.
- Because we want the output to contain our IV (which we generated at the start of the function) at the beginning, we write that to the destination stream before anything else with destination.write(iv).
- Lastly, we read and process the data from the source stream, piping it into the cipher object first (which acts as a transform stream that encrypts the data) and then piping that into the destination stream so that it can be written to the destination (for example, a file).
Similarly, we can define a decrypt function that, still working with streams, does the inverse operation:
4.5: (Continued) Encrypting and decrypting a stream using AES-256-GCM (aes-stream.js)
async function decrypt(key, tag, source, destination) {
const iv = await new Promise((resolve) => {
const cb = () => {
const iv = source.read(12)
source.off('readable', cb)
return resolve(iv)
}
source.on('readable', cb)
})
if (!iv) {
throw Error('iv is null')
}
return new Promise((resolve, reject) => {
const decipher = crypto.createDecipheriv('aes-256-gcm', key, iv)
decipher.setAuthTag(tag)
decipher.on('end', () => {
resolve()
})
decipher.on('error', (err) => {
reject(err)
})
source.pipe(decipher).pipe(destination)
})
}
The decrypt function accepts four arguments: the key (the same 256-bit symmetric key that was used to encrypt the file), the authentication tag (as returned by the encrypt function), the source stream that reads the ciphertext (which begins with the IV), and the destination stream where the plaintext is written to.
Admittedly, this looks even scarier, but we can analyze it by splitting it into two parts:
- The first half of the decrypt function's goal is to extract the IV from the stream. Remember that our encrypt function puts the IV at the beginning of the stream, so the first Promise object we create and await on is meant to read the first 12 bytes of the stream (and no more than 12 bytes) and get the IV.
At a high level, the promise's callback works by attaching a handler that is invoked when the source stream has data available to read. That handler reads 12 bytes exactly (and nothing more, to not consume the ciphertext in the stream), which is then used to resolve the promise. The handler detaches itself from the stream once the IV has been read.
- The second half of the function should look similar to what we did with the encrypt function moments ago. The differences are that we are creating a Decipher object with createDecipheriv (rather than a Cipher) and passing the authentication tag to it before we begin piping the data to be decrypted. Once the stream ends (and all the data has been fully decrypted), the Promise is resolved with no value; in case of error, it's rejected instead.
To invoke these two functions, we can use the following code:
4.6: Invoking the functions that use streams to encrypt and decrypt data with AES
const fs = require('fs')
;(async function() {
const key = await randomBytes(32)
console.log('Key:', key.toString('base64'))
const testFile = 'photo.jpg'
let tag
{
const inFile = fs.createReadStream(testFile)
const outFile = fs.createWriteStream(testFile + '.enc')
tag = await encrypt(key, inFile, outFile)
console.log('File was encrypted; authentication tag:', tag.toString('base64'))
}
{
const inFile = fs.createReadStream(testFile + '.enc')
const outFile = fs.createWriteStream(testFile + '.orig')
await decrypt(key, tag, inFile, outFile)
console.log('File was decrypted successfully')
}
})()
In this example, we are encrypting the photo.jpg file using a randomly generated key every time. Let's take a look:
- First, we encrypt the file. We create a readable stream to photo.jpg (inFile) and a writable stream to photo.jpg.enc (outFile, where we want the ciphertext to be written to). Then, we invoke the encrypt function asynchronously by passing the key, the input stream, and the output stream. The result of the function is the authentication tag (a Buffer object) that we display on the terminal. In a real application, you'd want to store the tag somewhere alongside your encrypted file, for example: in a database, in an index file, in a separate metadata file, or other places.
- In the next step, we reverse the operation and decrypt the photo back into a file called photo.jpg.orig. This time, the readable stream, inFile, is reading photo.jpg.enc (the ciphertext), while the writable stream, outFile, points to our target file: photo.jpg.orig. Then, we invoke the decrypt function asynchronously, passing the key (the same as we did previously), the authentication tag returned by encrypt, and the input and output streams.
After running the preceding code, you'll see that you have two more files in your folder alongside photo.jpg: photo.jpg.enc and photo.jpg.orig. The photo.jpg.enc file is encrypted, so it will look like a blob of random data; however, if everything went well, photo.jpg.orig should look the same as the original file.
To check that the files are identical, you can hash them, as we saw in Chapter 3, File and Password Hashing with Node.js:
photo.jpg f14fe4c5fc04089cba31559e9ed88cf268fb61f43a15e9b3bb94 ff63ac57b5b2
photo.jpg.orig f14fe4c5fc04089cba31559e9ed88cf268fb61f43a15e9b3 bb94ff63ac57b5b2
The actual checksums will be different for your files, but the important part is that they match! This means that the file was encrypted and then decrypted successfully, and the preceding code is correct.
As we mentioned earlier, Node.js offers built-in support for many more symmetric ciphers besides AES. Many of them are legacy ones and are only offered for compatibility reasons, but others can be valid alternatives, depending on your scenario. In the next section, we'll look at one of those.
Symmetric encryption with ChaCha20-Poly1305
ChaCha20 is a more recent symmetric cipher designed by Daniel J. Bernstein in the mid-'00s. It's often used together with the Poly1305 hashing function (in this case, it's also called "message authentication code"), which was designed by the same cryptographer. The result of the combination of the two is the ChaCha20-Poly1305 authenticated stream cipher.
We talked about authenticated stream ciphers in the previous section when referring to AES-GCM: functionally, ChaCha20-Poly1305 serves the same purpose. Even in practice, they are used very similarly, as you'll see in the samples in this section.
ChaCha20-Poly1305 has been seeing an increase in interest and popularity in recent years since it's now implemented by a variety of applications and, more frequently, as a cipher for the TLS protocol (used by HTTPS). For example, Google offers support for it in all of its online services and in the Android operating system. Cloudflare, whose CDN protects a large number of websites, has enabled it for all of their customers too.
Let's be clear on one thing: the reasons behind the interest in ChaCha20-Poly1305 have nothing to do with security. At the time of writing, both ChaCha20-Poly1305 and AES-GCM offer the same level of security, given keys of the same size, with neither of the two having any publicly-known security vulnerability.
However, on devices where hardware-based AES acceleration is not available, ChaCha20-Poly1305 is known to be significantly faster. While most, if not all, modern PCs and laptops have CPUs with hardware acceleration, the same isn't always true for mobile and embedded devices, so offering those clients the choice of a faster software-based cipher can make a difference.
Example usage with Node.js
With Node.js, support for ChaCha20-Poly1305 is available in the crypto module starting with version 11.2, when compiled against a recent-enough version of OpenSSL. Technically speaking, the version of ChaCha20-Poly1305 that's implemented in Node.js is the one compatible with RFC 7539 (also called the IETF variant). To use it, we can implement our usual encrypt and decrypt functions, which look very similar to the ones we wrote for AES-GCM:
4.7: Encryption and decryption with ChaCha20-Poly1305 (chacha20-poly1305.js)
const crypto = require('crypto')
const randomBytes = require('util')
.promisify(crypto.randomBytes)
async function encrypt(key, plaintext) {
const nonce = await randomBytes(12)
const cipher = crypto.createCipheriv('chacha20-poly1305', key, nonce, {
authTagLength: 16
})
const encrypted = Buffer.concat([
cipher.update(plaintext, 'utf8'),
cipher.final()
])
const tag = cipher.getAuthTag()
return Buffer.concat([nonce, tag, encrypted])
}
function decrypt(key, message) {
const nonce = message.slice(0, 12)
const tag = message.slice(12, 28)
const ciphertext = message.slice(28)
const decipher = crypto.createDecipheriv('chacha20- poly1305', key, nonce, {
authTagLength: 16
})
decipher.setAuthTag(tag)
const decrypted = Buffer.concat([
decipher.update(ciphertext, 'utf8'),
decipher.final()
])
return decrypted.toString('utf8')
}
The differences between the functions that we used for AES-GCM are highlighted in the preceding code:
- The identifier for the algorithm in the crypto.createCipheriv and crypto.createDecipheriv methods is 'chacha20-poly1305' .
- While AES-GCM requires a 96-bit (12 bytes) IV, ChaCha20-Poly1305 requires a nonce of the same length. While conceptually different, nonces are 12 random bytes that are generated and used the same way as IVs are in our code.
- Lastly, the crypto.createCipheriv and crypto.createDecipheriv methods require a fourth argument, which is a JavaScript object containing {authTagLength: 16} to indicate that we want a 128-bit (16 bytes) authentication tag.
The rest of the code and the way it's invoked remains unchanged from the preceding examples for AES-GCM.
When to use ChaCha20-Poly1305 or AES-GCM
As we mentioned previously, AES-GCM and ChaCha20-Poly1305 offer the same security, but the latter is faster when hardware acceleration is not available.
This makes the ChaCha20-Poly1305 cipher very interesting to implement on HTTPS web servers when you want to support mobile clients, especially lower-end ones that may have CPUs without AES acceleration or embedded systems such as Internet of Things (IoT) devices.
However, these situations are generally in the infrastructure realm and not something that is of relevance to your application code. That is, to enable ChaCha20-Poly1305 as a cipher for TLS, you generally need to tweak your web server or proxy configuration (such as Nginx) rather than make a code change in your app. With Node.js, when your application is directly exposed to the Internet (without using reverse proxies, CDNs, and so on), ChaCha20-Poly1305 is already enabled by default by the built-in HTTPS server, so there's nothing you need to do there.
As a developer building an app that runs on top of Node.js, then, there are only a few situations when choosing ChaCha20-Poly1305 over AES may make sense:
- When you're adding encryption to a CLI app written in Node.js that may be run on a variety of clients, including those without hardware-accelerated AES.
- When you're building a server-side app that will run on a system without hardware acceleration for AES. While this is exceedingly rare with "regular" servers or cloud infrastructure, a notable exception is when you are running your app on a Raspberry Pi (at the time of writing, the latest model is 4 Model B, which does not support hardware-accelerated AES).
- When you need compatibility with applications that are running on systems that don't have hardware AES acceleration. For example, if your server-side Node.js app receives encrypted data from low-powered IoT devices, then encrypting that with ChaCha20-Poly1305 may give you better performance overall than using AES-GCM.
With this, we have learned about all the most important symmetric ciphers. Now, it's time to look into the other core part of symmetric cryptography: keys.
Key derivation
In all our examples so far, we've generated a new key every time by grabbing a random sequence of bytes from crypto.randomBytes. While a random key always gives the best security, in many situations we need to be able to have a memorable (or at least, human-readable) passphrase to derive the symmetric keys from.
As we mentioned previously, AES requires a 128-, 192-, or 256-bit key, which means 16, 24, or 32 bytes. You might be tempted to grab a string of 16 characters and call it a 128-bit key, such as thisismykey12345… however, that would be a really bad idea. Despite being 128 bits in length, it is only made up of lowercase letters and numbers, so its entropy is significantly lower than 128 bits: in fact, this has only about 60 bits of entropy, which means that it can be cracked relatively quickly with a brute-force attack (see Chapter 3, File and Password Hashing with Node.js, for an explanation on entropy).
However, all is not lost, and we can stretch passphrases into safe keys by using a Key Derivation Function (KDF), as we saw in Chapter 3, File and Password Hashing with Node.js (as I warned you, hashes are omnipresent!).
In particular, in that chapter, we mentioned that two hashing functions were well-suited to be used for key derivation: Argon2, or if that's not available, scrypt. As we explained there, KDFs are deliberately slow to compute, so they are able to significantly increase the cost (time and/or resources) for an attacker trying to perform a brute-force attack on a low-entropy input such as a passphrase.
Because we discussed hashing functions at length in the previous chapter, we will not cover them again here; instead, we'll just look at an example of using Argon2 (in the Argon2id variant) to derive a key from a passphrase.
This will be very similar to the code in the previous chapter, but this time, we're requesting the raw bytes returned by Argon2, rather than a base64-encoded hash that contains the parameters too. To ensure consistency and repeatability, we are also explicitly passing some salt and all the parameters for Argon2:
4.8: Deriving keys using Argon2 (argon2-kdf.js)
// From NPM: https://www.npmjs.com/package/argon2
const argon2 = require('argon2')
function deriveKey(passphrase, salt, length) {
try {
const params = {
raw: true,
hashLength: length,
salt: salt,
type: argon2.argon2id,
timeCost: 3,
memoryCost: 4096,
parallelism: 1,
version: 0x13,
}
const result = await argon2.hash(passphrase, params)
return result
}
catch (err) {
console.error('An internal error occurred: ', err)
}
}
The deriveKey function returns a Promise that resolves with a Buffer with a symmetric key, and it accepts three arguments:
- passphrase is a string with the passphrase to derive the key from.
- salt is a Buffer object that's used as salt and should be 16 bytes in length. The salt (which is not a secret) should be stored somewhere such as in a database and optimally it should be unique for each passphrase.
- length is the length of the key to return, in bytes. It should be 16 for a 128-bit key or 32 for a 256-bit key.
Note that in the parameters for the hash, we are specifying raw: true, which makes argon2.hash return just the hash as a Buffer object, without encoding it as base64 and without prepending the parameters. We need just the hash here!
Additionally, as you can see here, and unlike in the example shown in Chapter 3, File and Password Hashing with Node.js, this time we are being explicit about the parameters that are used in the argon2.hash invocation rather than accepting all the defaults. This is necessary because we want to make sure that for each invocation, with the same pair of inputs for passphrase and salt, the result is the same. We can't rely on the defaults that are baked into the library in case they change in the future (which is not a problem when we ask argon2.hash to return a string, because it stores the parameters it used at the beginning of the hash).
Tuning the Argon2 Parameters
The parameters we used here for timeCost, memoryCost, and parallelism control the "cost" of running the Argon2 KDF. The costlier each invocation of Argon2 is, the higher the effort (time and/or energy) is for an attacker to break your symmetric key, so the stronger the protection is against brute-force attacks. In your application, you may want to tune those parameters so that the function takes as much time as you're willing to tolerate for your solution.
You can run the preceding function asynchronously with the await keyword. For example, to generate a 256-bit key, you can use the following code:
const key = await deriveKey(passphrase, salt, 32)
The function is deterministic, so given the same passphrase, salt, and length, you should always get the same result. For example, let's say you have these inputs (and the parameters unchanged from code sample 4.8):
;(async function() {
const passphrase = 'correct horse battery staple'
const salt = Buffer.from('WiHmGLjgzYESy3eAW45W0Q==', 'base64')
const key128 = await deriveKey(passphrase, salt, 16)
console.log('128-bit key:', key128.toString('base64'))
const key256 = await deriveKey(passphrase, salt, 32)
console.log('256-bit key:', key256.toString('base64'))
})()
The result is always (and on every machine) as follows:
128-bit key: McvSLprU4zfh1kcVOeR40g==
256-bit key: oQumof86t+UlE6yBPCbblO6IcPmrL8qHj/jucYIxJFw=
Using scrypt as a KDF
In Chapter 3, File and Password Hashing with Node.js, we mentioned that scrypt is another valid KDF and that it can be used when Argon2 is not available. The crypto.scrypt function, which is available in Node.js already, returns a raw Buffer object that can be used as a symmetric key and has an optional parameter that can be used to tune its cost. You can read more about this in the Node.js documentation: https://bit.ly/crypto-scrypt
Reusing keys
So long as the IV is random and unique for each invocation, it's ok to reuse the same key more than once.
However, especially with AES in GCM and CTR modes, you should never use the same IV twice with the same key. Doing that may expose your encryption key to an attacker.
With GCM mode, the IV is relatively small at only 12 bytes, so the chances of a collision occurring are relatively high when the same key is reused many times. Because of that, if you're able to generate a new key every time, such as for every new file, it would be a good idea to do so.
When you use a KDF to derive a key, you can, for example, use a different random salt for each file, and then store that salt alongside your file (as we mentioned previously, it's ok for the salt to be public). Using a different salt allows you to get a different key every time, eliminating the risk of reusing an IV.
Deriving keys from passphrases is an incredibly common operation for applications that leverage cryptography. However, when encrypting data with keys derived from user-supplied passphrases, there's one more thing we need to consider, as we'll see in the next section.
Wrapping keys and best practices for encrypting large documents
In the previous section, we learned how symmetric keys are often derived from passphrases. Encrypting data with a passphrase or passcode that the user memorizes (or stores in a password wallet) is at the core of many, many solutions that leverage cryptography, such as to encrypt documents or files. The next time you unlock your laptop with a passphrase or your phone with a PIN, think about the key derivation functions and ciphers that are being executed!
By reading this chapter up to this point, you should already be able to build an application like the one we just described with Node.js. For example, you could use Argon2 to derive a key from a passphrase submitted by the user, and then use AES-GCM to encrypt and decrypt files.
However, passphrases are not static. That is to say that users do change their passphrases, sometimes because they want to rotate them, or sometimes because their previous one was compromised (a surprisingly frequent occurrence nowadays!).
Sadly, when a passphrase changes, the encryption key that's derived from it changes too, which means you'll need to re-encrypt every single document that used that key. If there are lots of those documents and/or they are large, that can take a lot of time!
To avoid having to re-encrypt everything when a user changes their passphrase, the usual scheme involves encrypting each file with a separate, random key, which is in turn encrypted with the key derived from the user's passphrase.
Terminology
In this section, you'll frequently read the expressions key wrapping and key unwrapping. These are just two fancier names that refer to the operations of encrypting and decrypting keys, respectively. When you encrypt another key, you are wrapping it; you can then unwrap it to get the original key.
As for the keys, there's no strict naming convention about what the one that's used to encrypt data is called, but in this section, will refer to it as a user key (UK). Instead, the key that's used to wrap the other user key is usually called a key encryption key (KEK) or a wrapping key (WK).
AES Key Wrap
When you're choosing what algorithm to use to wrap keys, any symmetric (or even asymmetric) cipher would work (ideally you'd want to use an authenticated cipher). In fact, at the byte level, keys are not unlike any other message your applications will encrypt!
Nevertheless, cryptographers have created a class of algorithms that are specifically optimized for wrapping and unwrapping other keys. Unlike the ciphers that we've seen earlier in this chapter, such as AES-GCM or ChaCha20-Poly1305, they aim to be able to offer integrity without the use of any nonce or random IV.
AES Key Wrap, also called AES-KW, is a mode of operation of the AES cipher that is optimized for wrapping symmetric keys. It's defined in RFC 3394 and it's widely available, including in the Node.js crypto module.
In the following code samples, we'll be using AES-KW for wrapping and unwrapping keys. You'll see that its use is almost identical to how we implemented solutions based on AES-CBC, with just two differences. First, the name of the cipher (for createCipheriv and createDecipheriv) is the odd-looking 'id-aes256-wrap'. Second, the IV that's used is defined by the RFC and it's a fixed value; that is, 0xA6A6A6A6A6A6A6A6 (hex-encoded).
Wrapping user keys
Let's start by looking at the high-level description of the algorithm.
When your application is started for the first time – or, in a multi-user system, when a new user account is created – you must do the following:
- Prompt the user for a passphrase, then derive a symmetric key from that using the KDF of your choice, such as Argon2id – this is the wrapping key (WK).
- Calculate the hash of the passphrase, which we'll use to verify that the user entered the correct passphrase when they try to encrypt or decrypt data.
If you're using a KDF such as Argon2id, a safe and convenient option is to ask the function to generate 32 more bytes of data, for a total of 64 bytes: the first 32 bytes are the WK, while the rest is used as a hash to verify the passphrase. This is done by setting hashLength: 64 in the parameters for the Argon2 function's invocation, as we saw previously.
- Generate a random key, such as a random 32-byte sequence, which will be used as the user key (UK).
- Using the WK, wrap the UK by encrypting it with the symmetric key encryption algorithm.
- Store the wrapped key and the password hash with the user's profile in your application.
Once you have generated the UK, then wrapped and stored it with the user's profile, you can use that to encrypt and decrypt data as needed:
- Prompt the user for the passphrase again.
- Rerun the KDF to obtain the WK and calculate the hash of the passphrase – as we saw previously, this can be done in the same step.
- Compare the passphrase hashes to ensure they match: the one you just generated and the one stored in the user's profile. If they don't match, stop here and return an error.
- Using the WK derived from the passphrase, unwrap the key that is stored in the user's profile. That key (which should only be kept in memory for the time it's needed) is the UK and can be used to encrypt and decrypt data.
You can find an example of implementing this in the key-wrap.js file in this book's GitHub repository, in the ch4-symmetric-encryption folder. While we won't be able to analyze it in detail here, the code is fully commented on and shows an implementation for all the steps described here.
With a solution like this, if the user decides to change their passphrase, then you need to repeat the steps you performed when creating a new user account but reusing the existing user key (UK) instead of generating a new (random) one. This means creating a new salt, deriving a new wrapping key (WK) and passphrase hash, and re-encrypting the original user key (UK). You will get a new wrapped key, but because the UK hasn't changed, you won't need to re-encrypt every single file that the user stored.
Using a Different Key for Each File
In the examples in this section, we've assumed that each user has one and only one UK, which is used to encrypt and decrypt every file the user owns. While this approach is perfectly fine for encrypting data that only one user can see, it makes it challenging to share files with others.
One common solution to this problem (whose implementation we're leaving as an exercise for you) is to encrypt each file with a third key, which we can call the file key (FK). Each file you encrypt uses a different, random FK. Let's assume that two users want to have access to that file, each one having their own UK: we'll call those UK1 and UK2 (and both those UKs are stored in a database wrapped with the respective user's wrapping key). You will then wrap the FK twice, with UK1 and UK2. This gives both users access to that shared file and that only, without having to know the other person's UK.
Summary
In this chapter, we learned about encrypting data with a symmetric cipher, starting with AES. After learning about how to use AES, including how to choose the size of the key, how to select a mode of operation (especially CBC and GCM), and how to generate an IV, we saw code samples for encrypting and decrypting data and streams with AES using Node.js. We then learned about ChaCha20-Poly1305, another symmetric stream cipher that's similar to AES-GCM.
Next, we explained how to derive encryption keys from a passphrase, stretching lower-entropy strings into safer keys for usage with symmetric ciphers. We saw examples of doing that with Argon2.
Finally, we learned how keys can be wrapped (encrypted), and why doing so can help solve real-world problems when applications use keys derived from passphrases to encrypt and decrypt users' data.
The next chapter will be the first one that covers the other kind of ciphers – asymmetric ones. We'll learn how to use public-key cryptography with algorithms such as RSA to encrypt and decrypt data.