Atomic Arithmetic and Bitwise Methods

The Atomics API offers a simple suite of methods for performing an in-place modification. In the ECMA specification, these methods are defined as AtomicReadModifyWrite operations. Under the hood, each of these methods is performing a read from a location in the SharedArrayBuffer, an arithmetic or bitwise operation, and a write to the same location. The atomic nature of these operators means that these three operations will be performed in sequence and without interruption by another thread.

All the arithmetic methods are demonstrated here:

// Create buffer of size 1
let sharedArrayBuffer = new SharedArrayBuffer(1);

// Create Uint8Array from buffer
let typedArray = new Uint8Array(sharedArrayBuffer);

// All ArrayBuffers are initialized to 0
console.log(typedArray); // Uint8Array[0]

const index = 0;
const increment = 5;

// Atomic add 5 to value at index 0
Atomics.add(typedArray, index, increment);

console.log(typedArray); // Uint8Array[5]

// Atomic subtract 5 to value at index 0
Atomics.sub(typedArray, index, increment);

console.log(typedArray); // Uint8Array[0]

All the bitwise methods are demonstrated here:

// Create buffer of size 1
let sharedArrayBuffer = new SharedArrayBuffer(1);

// Create Uint8Array from buffer
let typedArray = new Uint8Array(sharedArrayBuffer);

// All ArrayBuffers are initialized to 0
console.log(typedArray); // Uint8Array[0]

const index = 0;

// Atomic or 0b1111 to value at index 0
Atomics.or(typedArray, index, 0b1111);

console.log(typedArray); // Uint8Array[15]

// Atomic and 0b1100 to value at index 0
Atomics.and(typedArray, index, 0b1100);

console.log(typedArray); // Uint8Array[12]

// Atomic xor 0b1111 to value at index 0
Atomics.xor(typedArray, index, 0b1111);

console.log(typedArray); // Uint8Array[3]

The thread-unsafe example from earlier can be corrected as follows:

const workerScript = `
self.onmessage = ({data}) => {
 const view = new Uint32Array(data);

 // Perform 1000000 add operations
 for (let i = 0; i < 1E6; ++i) {
  // Thread-safe add operation
  Atomics.add(view, 0, 1);
 }

 self.postMessage(null);
};
`;

const workerScriptBlobUrl = URL.createObjectURL(new Blob([workerScript]));

// Create worker pool of size 4
const workers = [];
for (let i = 0; i < 4; ++i) {
 workers.push(new Worker(workerScriptBlobUrl));
}

// Log the final value after the last worker completes
let responseCount = 0;
for (const worker of workers) {
 worker.onmessage = () => {
  if (++responseCount == workers.length) {
   console.log(`Final buffer value: ${view[0]}`);
  }
 };
}

// Initialize the SharedArrayBuffer
const sharedArrayBuffer = new SharedArrayBuffer(4);
const view = new Uint32Array(sharedArrayBuffer);
view[0] = 1;

// Send the SharedArrayBuffer to each worker
for (const worker of workers) {
 worker.postMessage(sharedArrayBuffer);
}

// (Expected result is 4000001)
// Final buffer value: 4000001

Atomic Reads and Writes

Both the browser's JavaScript compiler and the CPU architecture itself are given license to reorder instructions if they detect it will increase the overall throughput of program execution. Normally, the single-threaded nature of JavaScript means this optimization should be welcomed with open arms. However, instruction reordering across multiple threads can yield race conditions that are extremely difficult to debug.

The Atomics API addresses this problem in two primary ways:

• All Atomics instructions are never reordered with respect to one another.
• Using an Atomic read or Atomic write guarantees that all instructions (both Atomic and non-Atomic) will never be reordered with respect to that Atomic read/write. This means that all instructions before an Atomic read/write will finish before the Atomic read/write occurs, and all instructions after the Atomic read/write will not begin until the Atomic read/write completes.

In addition to reading and writing values to a buffer, Atomics.load() and Atomics.store() behave as “code fences.” The JavaScript engine guarantees that, although non-Atomic instructions may be locally reordered relative to a load() or store(), the reordering will never violate the Atomic read/write boundary. The following code annotates this behavior:

const sharedArrayBuffer = new SharedArrayBuffer(4);
const view = new Uint32Array(sharedArrayBuffer);

// Perform non-Atomic write
view[0] = 1;

// Non-Atomic write is guaranteed to occur before this read,
// so this is guaranteed to read 1
console.log(Atomics.load(view, 0)); // 1

// Perform Atomic write
Atomics.store(view, 0, 2);

// Non-Atomic read is guaranteed to occur after Atomic write,
// so this is guaranteed to read 2
console.log(view[0]); // 2

Atomic Exchanges

The Atomics API offers two types of methods that guarantee a sequential and uninterrupted read-then-write: exchange() and compareExchange(). Atomics.exchange() performs a simple swap, guaranteeing that no other threads will interrupt the value swap:

const sharedArrayBuffer = new SharedArrayBuffer(4);
const view = new Uint32Array(sharedArrayBuffer);

// Write 3 to 0-index
Atomics.store(view, 0, 3);

// Read value out of 0-index and then write 4 to 0-index
console.log(Atomics.exchange(view, 0, 4)); // 3

// Read value at 0-index
console.log(Atomics.load(view, 0));        // 4

One thread in a multithreaded program might want to perform a write to a shared buffer only if another thread has not modified a specific value since it was last read. If the value has not changed, it can safely write the update value. If the value has changed, performing a write would trample the value calculated by another thread. For this task, the Atomics API features the compareExchange() method. This method only performs a write if the value at the intended index matches an expected value. Consider the following example:

const sharedArrayBuffer = new SharedArrayBuffer(4);
const view = new Uint32Array(sharedArrayBuffer);

// Write 5 to 0-index
Atomics.store(view, 0, 5);
// Read the value out of the buffer
let initial = Atomics.load(view, 0);

// Perform a non-atomic operation on that value
let result = initial ** 2;

// Write that value back into the buffer only if the buffer has not changed
Atomics.compareExchange(view, 0, initial, result);

// Check that the write succeeded
console.log(Atomics.load(view, 0)); // 25

If the value does not match, the compareExchange() call will simply behave as a passthrough:

const sharedArrayBuffer = new SharedArrayBuffer(4);
const view = new Uint32Array(sharedArrayBuffer);

// Write 5 to 0-index
Atomics.store(view, 0, 5);
// Read the value out of the buffer
let initial = Atomics.load(view, 0);

// Perform a non-atomic operation on that value
let result = initial ** 2;

// Write that value back into the buffer only if the buffer has not changed
Atomics.compareExchange(view, 0, -1, result);

// Check that the write failed
console.log(Atomics.load(view, 0)); // 5

Atomics Futex Operations and Locks

Multithreaded programs wouldn't amount to much without some sort of locking construct. To address this need, the Atomics API offers several methods modeled on the Linux futex (a portmanteau of fast user-space mutex). The methods are fairly rudimentary, but they are intended to be used as a fundamental building block for more elaborate locking constructs.

Atomics.wait() and Atomics.notify() are best understood by example. The following rudimentary example spawns four workers to operate on an Int32Array of length 1. The spawned workers will take turns obtaining the lock and performing their add operation:

const workerScript = `
self.onmessage = ({data}) => {
 const view = new Int32Array(data);

 console.log('Waiting to obtain lock');

 // Halt when encountering the initial value, timeout at 10000ms
 Atomics.wait(view, 0, 0, 1E5);
 
 console.log('Obtained lock');

 // Add 1 to data index
 Atomics.add(view, 0, 1);
  
 console.log('Releasing lock');
  
 // Allow exactly one worker to continue execution
 Atomics.notify(view, 0, 1);
  
 self.postMessage(null);
};
`;

const workerScriptBlobUrl = URL.createObjectURL(new Blob([workerScript]));

const workers = [];
for (let i = 0; i < 4; ++i) {
 workers.push(new Worker(workerScriptBlobUrl));
}

// Log the final value after the last worker completes
let responseCount = 0;
for (const worker of workers) {
 worker.onmessage = () => {
  if (++responseCount == workers.length) {
   console.log(`Final buffer value: ${view[0]}`);
  }
 };
}

// Initialize the SharedArrayBuffer
const sharedArrayBuffer = new SharedArrayBuffer(8);
const view = new Int32Array(sharedArrayBuffer);

// Send the SharedArrayBuffer to each worker
for (const worker of workers) {
 worker.postMessage(sharedArrayBuffer);
}

// Release first lock in 1000ms
setTimeout(() => Atomics.notify(view, 0, 1), 1000);

// Waiting to obtain lock
// Waiting to obtain lock
// Waiting to obtain lock
// Waiting to obtain lock
// Obtained lock
// Releasing lock
// Obtained lock
// Releasing lock
// Obtained lock
// Releasing lock
// Obtained lock
// Releasing lock
// Final buffer value: 4

Because the SharedArrayBuffer is initialized with 0s, each worker will arrive at the Atomics.wait() and halt execution. In the halted state, the thread of execution exists inside a wait queue, remaining paused until the specified timeout elapses or until Atomics.notify() is invoked for that index. After 1000 milliseconds, the top-level execution context will call Atomics.notify() to release exactly one of the waiting threads. This thread will finish execution and call Atomics.notify() once again, releasing yet another thread. This continues until all the threads have completed execution and transmitted their final postMessage().

The Atomics API also features the Atomics.isLockFree() method. It is almost certain that you will never need to use this method, as it is designed for high-performance algorithms to decide whether or not obtaining a lock is necessary. The specification offers this description:

Atomics.isLockFree() is an optimization primitive. The intuition is that if the atomic step of an atomic primitive (compareExchange, load, store, add, sub, and, or, xor, or exchange) on a datum of size n bytes will be performed without the calling agent acquiring a lock outside the n bytes comprising the datum, then Atomics.isLockFree(n) will return true. High-performance algorithms will use Atomics.isLockFree to determine whether to use locks or atomic operations in critical sections. If an atomic primitive is not lock-free then it is often more efficient for an algorithm to provide its own locking.

Atomics.isLockFree(4) always returns true as that can be supported on all known relevant hardware. Being able to assume this will generally simplify programs.

Bulk Encoding

The bulk designation means that the JavaScript engine will synchronously encode the entire string. For very long strings, this can be a costly operation. Bulk encoding is accomplished using an instance of a TextEncoder:

const textEncoder = new TextEncoder();

This instance exposes an encode() method, which accepts a string and returns each character's UTF-8 encoding inside a freshly created Uint8Array:

const textEncoder = new TextEncoder();
const decodedText = 'foo';
const encodedText = textEncoder.encode(decodedText);

// f encoded in utf-8 is 0x66 (102 in decimal)
// o encoded in utf-8 is 0x6F (111 in decimal)
console.log(encodedText); // Uint8Array(3) [102, 111, 111]

The encoder is equipped to handle characters, which will take up multiple indices in the eventual array, such as emojis:

const textEncoder = new TextEncoder();
const decodedText = '';
const encodedText = textEncoder.encode(decodedText);

//  encoded in UTF-8 is 0xF0 0x9F 0x98 0x8A (240, 159, 152, 138 in decimal)
console.log(encodedText); // Uint8Array(4) [240, 159, 152, 138]

The instance also exposes an encodeInto() method, which accepts a string and the destination Uint8Array. This method returns a dictionary containing read and written properties, indicating how many characters were successfully read from the source string and written to the destination array, respectively. If the typed array has insufficient space, the encoding will terminate early and the dictionary will indicate that result:

const textEncoder = new TextEncoder();
const fooArr = new Uint8Array(3); 
const barArr = new Uint8Array(2);
const fooResult = textEncoder.encodeInto('foo', fooArr); 
const barResult = textEncoder.encodeInto('bar', barArr);

console.log(fooArr);   // Uint8Array(3) [102, 111, 111] 
console.log(fooResult); // { read: 3, written: 3 }

console.log(barArr);   // Uint8Array(2) [98, 97] 
console.log(barResult); // { read: 2, written: 2 }

encode() must allocate a new Uint8Array, whereas encodeInto() does not. For performance-sensitive applications, this distinction may have significant implications.

Stream Encoding

A TextEncoderStream is merely a TextEncoder in the form of a TransformStream. Piping a decoded text stream through the stream encoder will yield a stream of encoded text chunks:

async function* chars() {
 const decodedText = 'foo';

 for (let char of decodedText) {
  yield await new Promise((resolve) => setTimeout(resolve, 1000, char));
 }
}

const decodedTextStream = new ReadableStream({
 async start(controller) {
  for await (let chunk of chars()) {
   controller.enqueue(chunk);
  }

  controller.close();
 }
});


const encodedTextStream = decodedTextStream.pipeThrough(new TextEncoderStream());

const readableStreamDefaultReader = encodedTextStream.getReader();

(async function() {
 while(true) {
  const { done, value } = await readableStreamDefaultReader.read();

  if (done) {
   break;
  } else {
   console.log(value);
  }
 }
})();

// Uint8Array[102]
// Uint8Array[111]
// Uint8Array[111] 

Bulk Decoding

The bulk designation means that the JavaScript engine will synchronously decode the entire string. For very long strings, this can be a costly operation. Bulk decoding is accomplished using an instance of a DecoderEncoder:

const textDecoder = new TextDecoder();

This instance exposes a decode() method, which accepts a typed array and returns the decoded string:

const textDecoder = new TextDecoder();

// f encoded in utf-8 is 0x66 (102 in decimal)
// o encoded in utf-8 is 0x6F (111 in decimal)
const encodedText = Uint8Array.of(102, 111, 111);
const decodedText = textDecoder.decode(encodedText);

console.log(decodedText); // foo

The decoder does not care which typed array it is passed, so it will dutifully decode the entire binary representation. In this example, 32-bit values only containing 8-bit characters are decoded as UTF-8, yielding extra empty characters:

const textDecoder = new TextDecoder();

// f encoded in utf-8 is 0x66 (102 in decimal)
// o encoded in utf-8 is 0x6F (111 in decimal)
const encodedText = Uint32Array.of(102, 111, 111);
const decodedText = textDecoder.decode(encodedText);

console.log(decodedText); // "f  o  o  "

The decoder is equipped to handle characters that span multiple indices in the typed array, such as emojis:

const textDecoder = new TextDecoder();

//  encoded in UTF-8 is 0xF0 0x9F 0x98 0x8A (240, 159, 152, 138 in decimal)
const encodedText = Uint8Array.of(240, 159, 152, 138);
const decodedText = textDecoder.decode(encodedText);

console.log(decodedText); // 

Unlike TextEncoder, TextDecoder is compatible with a wide number of character encodings. Consider the following example, which uses UTF-16 encoding instead of the default UTF-8:

const textDecoder = new TextDecoder('utf-16');

// f encoded in utf-8 is 0x0066 (102 in decimal)
// o encoded in utf-8 is 0x006F (111 in decimal)
const encodedText = Uint16Array.of(102, 111, 111);
const decodedText = textDecoder.decode(encodedText);

console.log(decodedText); // foo

Stream Decoding

A TextDecoderStream is merely a TextDecoder in the form of a TransformStream. Piping an encoded text stream through the stream decoder will yield a stream of decoded text chunks:

async function* chars() {
 // Each chunk must exist as a typed array
 const encodedText = [102, 111, 111].map((x) => Uint8Array.of(x));

 for (let char of encodedText) {
  yield await new Promise((resolve) => setTimeout(resolve, 1000, char));
 }
}

const encodedTextStream = new ReadableStream({
 async start(controller) {
  for await (let chunk of chars()) {
   controller.enqueue(chunk);
  }

  controller.close();
 }
});


const decodedTextStream = encodedTextStream.pipeThrough(new TextDecoderStream());

const readableStreamDefaultReader = decodedTextStream.getReader();

(async function() {
 while(true) {
  const { done, value } = await readableStreamDefaultReader.read();

  if (done) {
   break;
  } else {
    console.log(value);
  }
 }
})();

// f
// o
// o 

Text decoder streams implicitly understand that surrogate pairs may be split between chunks. The decoder stream will retain any fragmented chunks until a complete character is formed. Consider the following example, where the stream decoder will wait for all four chunks to be passed through before the decoded stream emits the single character:

async function* chars() {
 //  encoded in UTF-8 is 0xF0 0x9F 0x98 0x8A (240, 159, 152, 138 in decimal)
 const encodedText = [240, 159, 152, 138].map((x) => Uint8Array.of(x));

 for (let char of encodedText) {
  yield await new Promise((resolve) => setTimeout(resolve, 1000, char));
 }
}

const encodedTextStream = new ReadableStream({
 async start(controller) {
  for await (let chunk of chars()) {
   controller.enqueue(chunk);
  }

  controller.close();
 }
});


const decodedTextStream = encodedTextStream.pipeThrough(new TextDecoderStream());

const readableStreamDefaultReader = decodedTextStream.getReader();

(async function() {
 while(true) {
  const { done, value } = await readableStreamDefaultReader.read();

  if (done) {
    break;
  } else {
    console.log(value);
  }
 }
})();

// 

Text decoder streams will be most commonly used in conjunction with fetch(), as the response body can be processed as a ReadableStream. This might take the following form:

const response = await fetch(url);
const stream = response.body.pipeThrough(new TextDecoderStream());

for await (let decodedChunk of decodedStream) {
 console.log(decodedChunk);
}

Chunks, Internal Queues, and Backpressure

The fundamental unit in streams is the chunk. A chunk can be of any data type, but frequently it will take the form of a typed array. Each chunk is a discrete segment of the stream that can be handled in its entirety. Importantly, chunks do not have a fixed size or arrive at fixed intervals. In an ideal stream, chunks will generally be approximately the same size and arrive at approximately regular intervals, but any good stream implementation should be prepared to handle edge cases.

For all types of streams, there is a shared concept of an entrance and exit to the stream. There will sometimes be a mismatch between the rate of data entering and exiting the stream. This stream balance can take one of three forms:

• The exit of the stream can process data faster than the data is provided at the entrance. The stream exit will often be idle (which may indicate potential inefficiencies at the stream entrance), but there is little wasted memory or computation so this stream imbalance is acceptable.
• The stream entrance and exit are in equilibrium. This balance is ideal.
• The entrance of the stream can provide data faster than the exit can process it. This stream imbalance is inherently problematic. There will necessarily be a backlog of data somewhere, and streams must handle this accordingly.

Stream imbalance is a common problem, but streams are tooled to address it. All streams maintain an internal queue of chunks that have entered the stream but not yet exited. For a stream in equilibrium, the internal queue will consistently have zero or a small number of enqueued chunks because the stream exit is dequeuing chunks approximately as fast as they can be enqueued. The memory footprint of such a stream's internal queue will remain relatively small.

When chunks are enqueued faster than they can be dequeued, the size of the internal queue will grow. The stream cannot allow its internal queue to grow indefinitely, so it uses backpressure to signal the stream entrance to stop sending data until the queue size returns below a predetermined threshold. This threshold is governed by a queueing strategy that defines the high water mark, the maximum memory footprint of the internal queue.

Using the ReadableStreamDefaultController

Consider the following generator, which produces an incremented integer every 1000 milliseconds:

async function* ints() {
 // yield an incremented integer every 1000ms
 for (let i = 0; i < 5; ++i) {
  yield await new Promise((resolve) => setTimeout(resolve, 1000, i));
 }
}

These values can be passed into a readable stream via its controller. The simplest way to access a controller is by creating a new instance of a ReadableStream, defining a start() method inside the constructor's underlyingSource parameter, and using the controller parameter passed to that method. By default, the controller parameter is an instance of ReadableStreamDefaultController:

const readableStream = new ReadableStream({
 start(controller) {
  console.log(controller); // ReadableStreamDefaultController {}
 }
});

Use the enqueue() method to pass values into the controller. Once all the values have been passed, the stream is closed using close():

async function* ints() {
 // yield an incremented integer every 1000ms
 for (let i = 0; i < 5; ++i) {
  yield await new Promise((resolve) => setTimeout(resolve, 1000, i));
 }
}

const readableStream = new ReadableStream({
 async start(controller) {
  for await (let chunk of ints()) {
   controller.enqueue(chunk);
  }

  controller.close();
 }
});

Using the ReadableStreamDefaultReader

This example so far successfully enqueues five values in the stream, but there is nothing reading them out of that queue. To do so, a ReadableStreamDefaultReader instance can be acquired from the stream with getReader(). This will obtain a lock on the stream, ensuring only this reader can read values from that stream:

async function* ints() {
 // yield an incremented integer every 1000ms
 for (let i = 0; i < 5; ++i) {
  yield await new Promise((resolve) => setTimeout(resolve, 1000, i));
 }
}

const readableStream = new ReadableStream({
 async start(controller) {
  for await (let chunk of ints()) {
   controller.enqueue(chunk);
  }

  controller.close();
 }
});

console.log(readableStream.locked); // false
const readableStreamDefaultReader = readableStream.getReader();
console.log(readableStream.locked); // true

A consumer can get values from this reader instance using its read() method:

async function* ints() {
 // yield an incremented integer every 1000ms
 for (let i = 0; i < 5; ++i) {
  yield await new Promise((resolve) => setTimeout(resolve, 1000, i));
 }
}

const readableStream = new ReadableStream({
 async start(controller) {
  for await (let chunk of ints()) {
   controller.enqueue(chunk);
  }

  controller.close();
 }
});

console.log(readableStream.locked); // false
const readableStreamDefaultReader = readableStream.getReader();
console.log(readableStream.locked); // true

// Consumer
(async function() {
 while(true) {
  const { done, value } = await readableStreamDefaultReader.read();

  if (done) {
    break;
  } else {
    console.log(value);
  }
 }
})();

// 0
// 1
// 2
// 3
// 4

Creating a WriteableStream

Consider the following generator, which produces an incremented integer every 1000 milliseconds:

async function* ints() {
 // yield an incremented integer every 1000ms
 for (let i = 0; i < 5; ++i) {
  yield await new Promise((resolve) => setTimeout(resolve, 1000, i));
 }
}

These values can be written to a writable stream via its public interface. When the public write() method is invoked, the write() method defined on the underlyingSink object passed to the constructor is also called:

const readableStream = new ReadableStream({
 write(value) {
  console.log(value);
 }
});

Using a WritableStreamDefaultWriter

To write values to this stream, a WritableStreamDefaultWriter instance can be acquired from the stream with getWriter(). This will obtain a lock on the stream, ensuring only this writer can write values to that stream:

async function* ints() {
 // yield an incremented integer every 1000ms
 for (let i = 0; i < 5; ++i) {
  yield await new Promise((resolve) => setTimeout(resolve, 1000, i));
 }
}

const writableStream = new WritableStream({
 write(value) {
  console.log(value);
 }
});

console.log(writableStream.locked); // false
const writableStreamDefaultWriter = writableStream.getWriter();
console.log(writableStream.locked); // true

Before writing values to the stream, the producer must ensure the writer is capable of accepting values. WritableStreamDefaultWriter.ready returns a promise that resolves when the writer is ready for writing values to the stream. Following this, values can be passed with write() until the data stream is complete, at which point the stream can be terminated with close():

async function* ints() {
 // yield an incremented integer every 1000ms
 for (let i = 0; i < 5; ++i) {
  yield await new Promise((resolve) => setTimeout(resolve, 1000, i));
 }
}

const writableStream = new WritableStream({
 write(value) {
  console.log(value);
 }
});

console.log(writableStream.locked); // false
const writableStreamDefaultWriter = writableStream.getWriter();
console.log(writableStream.locked); // true


// Producer
(async function() {
 for await (let chunk of ints()) {
  await writableStreamDefaultWriter.ready;
  writableStreamDefaultWriter.write(chunk);
 }
   
 writableStreamDefaultWriter.close();
})();

User Timing API

The User Timing API allows you to record and analyze custom performance entries. Recording a custom performance entry is accomplished with performance.mark():

performance.mark('foo');

console.log(performance.getEntriesByType('mark')[0]);
// PerformanceMark {
//  name: "foo", 
//  entryType: "mark", 
//  startTime: 269.8800000362098, 
//  duration: 0
// }

Creating two performance entries on either side of the computation allows you to calculate the time delta. The newest marks are pushed onto the beginning of the array returned from getEntriesByType():

performance.mark('foo');
for (let i = 0; i < 1E6; ++i) {}
performance.mark('bar');

const [endMark, startMark] = performance.getEntriesByType('mark');
console.log(startMark.startTime - endMark.startTime); // 1.3299999991431832

It's also possible to generate a PerformanceMeasure entry that corresponds to the duration of time between two marks identified via their name. This is accomplished with performance.measure():

performance.mark('foo');
for (let i = 0; i < 1E6; ++i) {}
performance.mark('bar');

performance.measure('baz', 'foo', 'bar');

const [differenceMark] = performance.getEntriesByType('measure');

console.log(differenceMark);
// PerformanceMeasure {
//  name: "baz", 
//  entryType: "measure", 
//  startTime: 298.9800000214018, 
//  duration: 1.349999976810068
// }

Navigation Timing API

The Navigation Timing API offers high-precision timestamps for metrics covering how quickly the current page loaded. The browser automatically records a PerformanceNavigationTiming entry when a navigation event occurs. This object captures a broad range of timestamps describing how and when the page loaded.

The following example calculates the amount of time between the loadEventStart and loadEventEnd timestamps:

const [performanceNavigationTimingEntry] = performance.getEntriesByType('navigation');

console.log(performanceNavigationTimingEntry);
// PerformanceNavigationTiming {
//  connectEnd: 2.259999979287386
//  connectStart: 2.259999979287386
//  decodedBodySize: 122314
//  domComplete: 631.9899999652989
//  domContentLoadedEventEnd: 300.92499998863786
//  domContentLoadedEventStart: 298.8950000144541
//  domInteractive: 298.88499999651685
//  domainLookupEnd: 2.259999979287386
//  domainLookupStart: 2.259999979287386
//  duration: 632.819999998901
//  encodedBodySize: 21107
//  entryType: "navigation"
//  fetchStart: 2.259999979287386
//  initiatorType: "navigation"
//  loadEventEnd: 632.819999998901
//  loadEventStart: 632.0149999810383
//  name: "https://foo.com"
//  nextHopProtocol: "h2"
//  redirectCount: 0
//  redirectEnd: 0
//  redirectStart: 0
//  requestStart: 7.7099999762140214
//  responseEnd: 130.50999998813495
//  responseStart: 127.16999999247491
//  secureConnectionStart: 0
//  serverTiming: []
//  startTime: 0
//  transferSize: 21806
//  type: "navigate"
//  unloadEventEnd: 132.73999997181818
//  unloadEventStart: 132.41999997990206
//  workerStart: 0
// }


console.log(performanceNavigationTimingEntry.loadEventEnd – 
            performanceNavigationTimingEntry.loadEventStart);
// 0.805000017862767

Resource Timing API

The Resource Timing API offers high-precision timestamps for metrics covering how quickly resources requested for the current page loaded. The browser automatically records a PerformanceResourceTiming entry when an asset is loaded. This object captures a broad range of timestamps describing how quickly that resource loaded.

The following example calculates the amount of time it took to load a specific resource:

const performanceResourceTimingEntry = performance.getEntriesByType('resource')[0];

console.log(performanceResourceTimingEntry);// PerformanceResourceTiming {
//  connectEnd: 138.11499997973442
//  connectStart: 138.11499997973442
//  decodedBodySize: 33808
//  domainLookupEnd: 138.11499997973442
//  domainLookupStart: 138.11499997973442
//  duration: 0
//  encodedBodySize: 33808
//  entryType: "resource"
//  fetchStart: 138.11499997973442
//  initiatorType: "link"
//  name: "https://static.foo.com/bar.png",
//  nextHopProtocol: "h2"
//  redirectEnd: 0
//  redirectStart: 0
//  requestStart: 138.11499997973442
//  responseEnd: 138.11499997973442
//  responseStart: 138.11499997973442
//  secureConnectionStart: 0
//  serverTiming: []
//  startTime: 138.11499997973442
//  transferSize: 0
//  workerStart: 0
// }

console.log(performanceResourceTimingEntry.responseEnd – 
            performanceResourceTimingEntry.requestStart);
// 493.9600000507198

Using the difference between various times can give you a good idea about how a page is being loaded into the browser and where the potential bottlenecks are hiding.

Using a DocumentFragment

When rendered inside a browser, you will not see this text render on the page. Because the <template> content is not considered part of the active document, DOM matching methods such as document.querySelector() will be unable to find the <p> tag. This is because it exists inside a new Node subclass added as part of HTML templates: the DocumentFragment.

The DocumentFragment inside a <template> is visible when inspecting inside the browser:

<template id="foo">
 #document-fragment
  <p>I'm inside a template!</p>
</template>

A reference to this DocumentFragment can be retrieved via the content property of the <template> element:

console.log(document.querySelector('#foo').content); // #document-fragment

The DocumentFragment behaves as a minimal document object for that subtree. For example, DOM matching methods on a DocumentFragmentcan find nodes in its subtree:

const fragment = document.querySelector('#foo').content;

console.log(document.querySelector('p')); // null 
console.log(fragment.querySelector('p')); // <p>…<p>

A DocumentFragment is also incredibly useful for adding HTML in bulk. Consider a scenario where one wishes to add multiple children to an HTML element as efficiently as possible. Using consecutive document.appendChild() calls for each child is painstaking and can potentially incur multiple reflows. Using a DocumentFragment allows for you to batch these child additions, guaranteeing at most a single reflow:

// Start state:
// <div id="foo"></div>
//
// Desired end state:
// <div id="foo">
//  <p></p>
//  <p></p>
//  <p></p>
// </div>

// Also can use document.createDocumentFragment()
const fragment = new DocumentFragment();

const foo = document.querySelector('#foo');

// Adding children to a DocumentFragment incurs no reflow
fragment.appendChild(document.createElement('p'));
fragment.appendChild(document.createElement('p'));
fragment.appendChild(document.createElement('p'));

console.log(fragment.children.length); // 3

foo.appendChild(fragment);

console.log(fragment.children.length); // 0

console.log(document.body.innerHTML);
// <div id="foo">
//  <p></p>
//  <p></p>
//  <p></p>
// </div>

Using <template> tags

Notice in the previous example how the child nodes of the DocumentFragment are effectively transferred onto the foo element, leaving the DocumentFragment empty. This same procedure can be replicated using a <template>:

const fooElement = document.querySelector('#foo');
const barTemplate = document.querySelector('#bar');
const barFragment = barTemplate.content;

console.log(document.body.innerHTML);
// <div id="foo">
// </div>
// <template id="bar">
//  <p></p>
//  <p></p>
//  <p></p>
// </template>

fooElement.appendChild(barFragment);

console.log(document.body.innerHTML);
// <div id="foo">
//  <p></p>
//  <p></p>
//  <p></p>
// </div>
// <tempate id="bar"></template>

If you wish to instead copy the template, a simple importNode() can be used to clone the DocumentFragment:

const fooElement = document.querySelector('#foo');
const barTemplate = document.querySelector('#bar');
const barFragment = barTemplate.content;

console.log(document.body.innerHTML);
// <div id="foo">
// </div>
// <template id="bar">
//  <p></p>
//  <p></p>
//  <p></p>
// </template>

fooElement.appendChild(document.importNode(barFragment, true));

console.log(document.body.innerHTML);
// <div id="foo">
//  <p></p>
//  <p></p>
//  <p></p>
// </div>
// <template id="bar">
//  <p></p>
//  <p></p>
//  <p></p>
// </template>

Template Scripts

Script execution will be deferred until the DocumentFragment is added to the real DOM tree. This is demonstrated here:

// Page HTML:
//
// <div id="foo"></div>
// <template id="bar">
//  <script>console.log('Template script executed');</script>
// </template>

const fooElement = document.querySelector('#foo');
const barTemplate = document.querySelector('#bar');
const barFragment = barTemplate.content;

console.log('About to add template');
fooElement.appendChild(barFragment);
console.log('Added template');

// About to add template
// Template script executed
// Added template

This is useful in situations where addition of the new elements requires some initialization.

Introduction to Shadow DOM

Consider a scenario where you have multiple similarly structured DOM subtrees:

<div>
 <p>Make me red!</p>
</div>
<div>
 <p>Make me blue!</p>
</div>
<div>
 <p>Make me green!</p>
</div>

As you've likely surmised from the text nodes, each of these three DOM subtrees should be assigned different colors. Normally, in order to apply a style uniquely to each of these without resorting to the style attribute, you'd likely apply a unique class name to each subtree and define the styling inside a corresponding selector:

<div class="red-text">
 <p>Make me red!</p>
</div>
<div class="green-text">
 <p>Make me green!</p>
</div>
<div class="blue-text">
 <p>Make me blue!</p>
</div>

<style>
.red-text {
 color: red;
}
.green-text {
 color: green;
}
.blue-text {
 color: blue;
}
</style>

Of course, this is a less than ideal solution. This isn't much different than defining variables in a global namespace; this CSS will be applied to the entire DOM even though you know with certainty that these style definitions are not needed anywhere else. You could keep adding CSS selector specificity to prevent these styles from bleeding elsewhere, but this is little more than a half-measure. Ideally, you'd prefer to restrict CSS to only a portion of the DOM: therein lies the raw utility of the shadow DOM.

Creating a Shadow DOM

For reasons involving security or preventing shadow DOM collisions, not all element types can have a shadow DOM attached to them. Attempting to attach a shadow DOM to an invalid element type, or an element with a shadow DOM already attached, will throw an error.

The following is a list of elements capable of hosting a shadow DOM:

• Any autonomous custom element with a valid name (as defined in the HTML specification: https://html.spec.whatwg.org/multipage/custom-elements.html#valid-custom-element-name)
• <article>
• <aside>
• <blockquote>
• <body>
• <div>
• <footer>
• <h1>
• <h2>
• <h3>
• <h4>
• <h5>
• <h6>
• <header>
• <main>
• <nav>
• <p>
• <section>
• <span>

A shadow DOM is created by attaching it to a valid HTML element using the attachShadow() method. The element to which a shadow DOM is attached is referred to as the shadow host. The root node of the shadow DOM is referred to as the shadow root.

The attachShadow() method expects a required shadowRootInit object and returns the instance of the shadow DOM. The shadowRootInit object must contain a single mode property specifying either "open" or "closed". A reference to an open shadow DOM can be obtained on an HTML element via the shadowRoot property; this is not possible with a closed shadow DOM.

The mode difference is demonstrated here:

document.body.innerHTML = `
 <div id="foo"></div>
 <div id="bar"></div>
`;

const foo = document.querySelector('#foo'); 
const bar = document.querySelector('#bar');

const openShadowDOM = foo.attachShadow({ mode: 'open' }); 
const closedShadowDOM = bar.attachShadow({ mode: 'closed' });

console.log(openShadowDOM);     // #shadow-root (open) 
console.log(closedShadowDOM);   // #shadow-root (closed)

console.log(foo.shadowRoot);    // #shadow-root (open)
console.log(bar.shadowRoot);    // null

In general, there is rarely a situation in which creating a closed shadow DOM is necessary. Although it confers the ability to restrict programmatic access to a shadow DOM from the shadow host, there are plenty of ways for malicious code to circumnavigate this and regain access to the shadow DOM. In short, creating a closed shadow DOM should not be used for security purposes.

Using a Shadow DOM

Once attached to an element, a shadow DOM can be used as a normal DOM. Consider the following example, which recreates the red/green/blue example shown previously:

for (let color of ['red', 'green', 'blue']) {
 const div = document.createElement('div');
 const shadowDOM = div.attachShadow({ mode: 'open' });
 
 document.body.appendChild(div);
 
 shadowDOM.innerHTML = `
  <p>Make me ${color}</p>
  
  <style>
  p {
   color: ${color};
  }
  </style>
 `;
}

Although there are three identical selectors applying three different colors, the selectors will only be applied to the shadow DOM in which they are defined. As such, the three <p> elements will appear in three different colors.

You can verify that these elements exist inside their own shadow DOM as follows:

for (let color of ['red', 'green', 'blue']) {
 const div = document.createElement('div');
 const shadowDOM = div.attachShadow({ mode: 'open' });
 
 document.body.appendChild(div);
 
 shadowDOM.innerHTML = `
  <p>Make me ${color}</p>
  
  <style>
  p {
   color: ${color};
  }
  </style>
 `;
}

function countP(node) {
 console.log(node.querySelectorAll('p').length);
}

countP(document); // 0

for (let element of document.querySelectorAll('div')) {
 countP(element.shadowRoot);
}

// 1
// 1
// 1

Browser inspector tools will make it clear where a shadow DOM exists. For example, the preceding example will appear as the following in the browser inspector:

<body>
<div>
 #shadow-root (open)
  <p>Make me red!</p>

  <style>
  p {
    color: red;
  }
  </style>
</div>
<div>
 #shadow-root (open)
  <p>Make me green!</p>

  <style>
  p {
    color: green;
  }
  </style>
</div>
<div>
 #shadow-root (open)
  <p>Make me blue!</p>

  <style>
  p {
    color: blue;
  }
  </style>
</div>
</body>

Shadow DOMs are not an impermeable boundary. An HTML element can be moved between DOM trees without restriction:

document.body.innerHTML = `
<div></div>
<p id="foo">Move me</p>
`;

const divElement = document.querySelector('div');
const pElement = document.querySelector('p');

const shadowDOM = divElement.attachShadow({ mode: 'open' });

// Remove element from parent DOM
divElement.parentElement.removeChild(pElement);

// Add element to shadow DOM
shadowDOM.appendChild(pElement);

// Check to see that element was moved
console.log(shadowDOM.innerHTML); // <p id="foo">Move me</p>

Composition and Shadow DOM Slots

The shadow DOM is designed to enable customizable components, and this requires the ability to handle nested DOM fragments. Conceptually, this is relatively straightforward: HTML inside a shadow host element needs a way to render inside the shadow DOM without actually being part of the shadow DOM tree.

By default, nested content will be hidden. Consider the following example where the text becomes hidden after 1000 milliseconds:

document.body.innerHTML = `
<div>
 <p>Foo</p>
</div>
`;

setTimeout(() => document.querySelector('div').attachShadow({ mode: 'open' }), 1000);

Once the shadow DOM is attached, the browser gives priority to the shadow DOM and will render its contents instead of the text. In this example, the shadow DOM is empty, so the <div> will appear empty in turn.

To show this content, you can use a <slot> tag to instruct where the browser should place the HTML. In the code that follows, the previous example is reworked so the text reappears inside the shadow DOM:

document.body.innerHTML = `
<div id="foo">
 <p>Foo</p>
</div>
`;

document.querySelector('div')
  .attachShadow({ mode: 'open' })
  .innerHTML = `<div id="bar">
                  <slot></slot>
                <div>`

Now, the projected content will behave as if it exists inside the shadow DOM. Inspecting the page reveals that the content appears to actually replace the <slot>:

<body>
<div id="foo">
 #shadow-root (open)
  <div id="bar">
   <p>Foo</p>
  </div>
</div>
</body>

Note that, despite its appearance in the page inspector, this is only a projection of DOM content. The element remains attached to the outer DOM:

document.body.innerHTML = `
<div id="foo">
 <p>Foo</p>
</div>
`;

document.querySelector('div')
  .attachShadow({ mode: 'open' })
  .innerHTML = `
   <div id="bar">
    <slot></slot>
   </div>`

console.log(document.querySelector('p').parentElement);
// <div id="foo"></div>

The red-green-blue example from before can be reworked to use slots as follows:

for (let color of ['red', 'green', 'blue']) {
 const divElement = document.createElement('div');
 divElement.innerText = `Make me ${color}`;
 document.body.appendChild(divElement)
 
 divElement
   .attachShadow({ mode: 'open' })
   .innerHTML = `
   <p><slot></slot></p>
   
   <style>
    p {
      color: ${color};
    }
   </style>
   `;
}

It's also possible to use named slots to perform multiple projections. This is accomplished with matching slot/name attribute pairs. The element identified with a slot="foo" attribute will be projected into the <slot> with name="foo". The following example demonstrates this by switching the rendered order of the shadow host element's children:

 document.body.innerHTML = `
<div>
 <p slot="foo">Foo</p>
 <p slot="bar">Bar</p>
</div>
`;

document.querySelector('div')
  .attachShadow({ mode: 'open' })
  .innerHTML = `
  <slot name="bar"></slot>
  <slot name="foo"></slot>
  `;

// Renders:
// Bar
// Foo

Event Retargeting

If a browser event like click occurs inside a shadow DOM, the browser needs a way for the parent DOM to handle the event. However, the implementation must also respect the shadow DOM boundary. To address this, events that escape a shadow DOM and are handled outside undergo event retargeting. Once escaped, this event will appear to have been thrown by the shadow host itself instead of the true encapsulated element. This behavior is demonstrated here:

// Create element to be shadow host
document.body.innerHTML = `
<div onclick="console.log('Handled outside:', event.target)"></div>
`;

// Attach shadow DOM and insert HTML into it
document.querySelector('div')
 .attachShadow({ mode: 'open' })
 .innerHTML = `
<button onclick="console.log('Handled inside:', event.target)">Foo</button>
`;

// When clicking the button:
// Handled inside: <button onclick="…"></button>
// Handled outside: <div onclick="…"></div>

Note that retargeting only occurs for elements that actually exist inside a shadow DOM. Elements projected from outside using a <slot> tag will not have their events retargeted, as they still technically exist outside the shadow DOM.

Defining a Custom Element

Browsers already will attempt to incorporate unrecognized elements into the DOM as generic elements. Of course, by default they won't do anything special that a generic HTML element doesn't already do. Consider the following example, where a nonsense HTML tag becomes an instance of an HTMLElement:

document.body.innerHTML = `
<x-foo >I'm inside a nonsense element.</x-foo >
`;

console.log(document.querySelector('x-foo') instanceof HTMLElement); // true

Custom elements take this further. They allow you to define complex behavior whenever an <x-foo> tag appears, and also to tap into the element's lifecycle with respect to the DOM. Custom element definition is accomplished using the global customElements property, which returns the CustomElementRegistry object.

console.log(customElements); // CustomElementRegistry {}

Defining a custom element is accomplished with the define() method. The following creates a trivial custom element, which inherits from a vanilla HTMLElement:

class FooElement extends HTMLElement {}
customElements.define('x-foo', FooElement);

document.body.innerHTML = `
<x-foo>I'm inside a nonsense element.</x-foo>
`;

console.log(document.querySelector('x-foo') instanceof FooElement); // true

The power of custom elements is housed in the class definition. For example, now each instance of this class in the DOM will call the constructor that you control:

class FooElement extends HTMLElement {
 constructor() {
  super();
  console.log('x-foo')
 }
}
customElements.define('x-foo', FooElement);

document.body.innerHTML = `
<x-foo></x-foo>
<x-foo></x-foo>
<x-foo></x-foo>
`;

// x-foo
// x-foo
// x-foo

If a custom element inherits from an element class, the tag can be specified as an instance of that custom element using the is attribute and the extends option:

class FooElement extends HTMLDivElement {
 constructor() {
  super();
  console.log('x-foo')
 }
}
customElements.define('x-foo', FooElement, { extends: 'div' });

document.body.innerHTML = `
<div is="x-foo"></div>
<div is="x-foo"></div>
<div is="x-foo"></div>
`;

// x-foo
// x-foo
// x-foo

Adding Web Component Content

Because the custom element class constructor is called each time the element is added to the DOM, it is easy to automatically populate a custom element with child DOM content. Although you are forbidden from adding DOM children inside the constructor (a DOMException will be thrown), you can attach a shadow DOM and place content inside:

class FooElement extends HTMLElement {
 constructor() {
  super();
  
  // 'this' refers to the web component node
  this.attachShadow({ mode: 'open' });
  
  this.shadowRoot.innerHTML = `
   <p>I'm inside a custom element!</p>
  `;
 }
}
customElements.define('x-foo', FooElement);

document.body.innerHTML += `<x-foo></x-foo`;

// Resulting DOM:
// <body>
// <x-foo>
//    #shadow-root (open)
//      <p>I'm inside a custom element!</p>
// <x-foo>
// </body>

To avoid the nastiness of string templates and innerHTML, this example can be refactored to use HTML templates and document.createElement():

// (Initial HTML)
// <template id="x-foo-tpl">
//  <p>I'm inside a custom element template!</p>
// </template>

const template = document.querySelector('#x-foo-tpl');

class FooElement extends HTMLElement {
 constructor() {
  super();
  
  this.attachShadow({ mode: 'open' });
  
  this.shadowRoot.appendChild(template.content.cloneNode(true));
 }
}
customElements.define('x-foo', FooElement);

document.body.innerHTML += `<x-foo></x-foo`;

// Resulting DOM:
// <body>
// <template id="x-foo-tpl">
//  <p>I'm inside a custom element template!</p>
// </template>
// <x-foo>
//    #shadow-root (open)
//       <p>I'm inside a custom element template!</p>
// <x-foo>
// </body>

This practice allows for a high degree of HTML and code reuse as well as DOM encapsulation inside your custom element. With it, you are free to build reusable widgets without fear of outside CSS trampling your styling.

Using Custom Element Lifecycle Hooks

It is possible to execute code at various points in the lifecycle of a custom element. Instance methods on the custom element class with the corresponding name will be called during that lifecycle phase. There are five available hooks:

• constructor() is called when an element instance is created, or an existing DOM element is upgraded to a custom element.
• connectedCallback() is called each time this instance of the custom element is added into the DOM.
• disconnectedCallback() is called each time this instance of the custom element is removed from the DOM.
• attributeChangedCallback() is called each time the value of an observed attribute is changed. When the element instance is instantiated, definition of the initial value counts as a change.
• adoptedCallback() is called each time this instance is moved to a new document object with document.adoptNode().

The following example demonstrates the construction, connected, and disconnected callbacks:

class FooElement extends HTMLElement {
 constructor() {
  super();
  console.log('ctor');
 }
 
 connectedCallback() {
  console.log('connected');
 }
 
 disconnectedCallback() {
  console.log('disconnected');
 }
}
customElements.define('x-foo', FooElement);

const fooElement = document.createElement('x-foo');
// ctor

document.body.appendChild(fooElement);
// connected

document.body.removeChild(fooElement);
// disconnected

Reflecting Custom Element Attributes

Because an element exists as both a DOM entity and a JavaScript object, a common pattern is to reflect changes between the two. In other words, a change to the DOM should reflect a change in the object, and vice versa. To reflect from the object to the DOM, a common strategy is to use getters and setters. The following example reflects the bar property of the object up to the DOM:

document.body.innerHTML = `<x-foo></x-foo>`;

class FooElement extends HTMLElement {
 constructor() {
  super();
  
  this.bar = true;
 }
 
 get bar() {
  return this.getAttribute('bar');
 }
 
 set bar(value) {
  this.setAttribute('bar', value)
 }
}
customElements.define('x-foo', FooElement);

console.log(document.body.innerHTML);
// <x-foo bar="true"></x-foo>

Reflecting in the reverse direction—from the DOM to the object—requires setting a listener for that attribute. To accomplish this, you can instruct the custom element to call the attributeChangedCallback() each time the attribute's value changes by using the observedAttributes() getter:

class FooElement extends HTMLElement {
 static get observedAttributes() {
  // List attributes which should trigger attributeChangedCallback()
  return ['bar'];
 }
 
 get bar() {
  return this.getAttribute('bar');
 }
 
 set bar(value) {
  this.setAttribute('bar', value)
 }
 
 attributeChangedCallback(name, oldValue, newValue) {
    if (oldValue !== newValue) {
    console.log(`${oldValue} -> ${newValue}`);
   
    this[name] = newValue;
  }
 }
}
customElements.define('x-foo', FooElement);

document.body.innerHTML = `<x-foo bar="false"></x-foo>`;
// null -> false

document.querySelector('x-foo').setAttribute('bar', true);
// false -> true

Upgrading Custom Elements

It's not always possible to define a custom element before the custom element tag appears in the DOM. Web components address this ordering problem by exposing several additional methods on the CustomElementRegistry that allow you to detect when a custom element is eventually defined and upgrade existing elements.

The CustomElementRegistry.get() method returns the custom element class if it was already defined. In a similar vein, the CustomElementRegistry.whenDefined() method returns a promise that resolves when a custom element becomes defined:

customElements.whenDefined('x-foo').then(() => console.log('defined!'));

console.log(customElements.get('x-foo'));
// undefined

customElements.define('x-foo', class {});
// defined!

console.log(customElements.get('x-foo'));
// class FooElement {}

Elements connected to the DOM will be automatically upgraded when the custom element is defined. If you wish to forcibly upgrade an element before it is connected to the DOM, this can be accomplished with CustomElementRegistry.upgrade():

// Create HTMLUnknownElement object before custom element definition
const fooElement = document.createElement('x-foo');

// Define custom element
class FooElement extends HTMLElement {}
customElements.define('x-foo', FooElement);

console.log(fooElement instanceof FooElement); // false

// Force the upgrade
customElements.upgrade(fooElement);

console.log(fooElement instanceof FooElement); // true

Generating Cryptographic Digests

One extremely common cryptography operation is to calculate a cryptographic digest of data. The specification supports four algorithms for this, SHA-1 and three flavors of SHA-2:

• Secure Hash Algorithm 1 (SHA-1)—A hash function with an architecture similar to MD5. It takes an input of any size and generates a 160-bit message digest. This algorithm is no longer considered secure as it is vulnerable to collision attacks.
• Secure Hash Algorithm 2 (SHA-2)—A family of hash functions all built upon the same collision-resistant one-way compression function. The specification supports three members of this family: SHA-256, SHA-384, and SHA-512. The size of the generated message digest can be 256 bits (SHA-256), 384 bits (SHA-384), or 512 bits (SHA-512). This algorithm is considered secure and widely used in many applications and protocols, including TLS, PGP, and cryptocurrencies like Bitcoin.

The SubtleCrypto.digest() method is used to generate a message digest. The hash algorithm is specified using a string: SHA-1, SHA-256, SHA-384, or SHA-512. The following example demonstrates a simple application of SHA-256 to generate a message digest of the string foo:

(async function() {
 const textEncoder = new TextEncoder();
 const message = textEncoder.encode('foo');
 const messageDigest = await crypto.subtle.digest('SHA-256', message);

 console.log(new Uint32Array(messageDigest));
})();
 
// Uint32Array(8) [1806968364, 2412183400, 1011194873, 876687389, 
//                 1882014227, 2696905572, 2287897337, 2934400610]

Commonly, the message digest binary will be used in a hex string format. Converting an array buffer to this format is accomplished by splitting the buffer up into 8-bit pieces and converting using toString() in base 16:

(async function() {
 const textEncoder = new TextEncoder();
 const message = textEncoder.encode('foo');
 const messageDigest = await crypto.subtle.digest('SHA-256', message);

 const hexDigest = Array.from(new Uint8Array(messageDigest))
  .map((x) => x.toString(16).padStart(2, '0'))
  .join('');

 console.log(hexDigest);
})();

// 2c26b46b68ffc68ff99b453c1d30413413422d706483bfa0f98a5e886266e7ae

Software companies usually publish a digest of their install binaries so that people who wish to safely install their software can verify that the binary they download is the version that the company actually published (and not one with injected malware). The following example downloads v67.0 of Firefox, hashes it with SHA-512, downloads the SHA-512 binary verification, and checks that the two hex strings match:

(async function() {
 const mozillaCdnUrl = '//download-origin.cdn.mozilla.net/pub/firefox/releases/67.0/';
 const firefoxBinaryFilename = 'linux-x86_64/en-US/firefox-67.0.tar.bz2';
 const firefoxShaFilename = 'SHA512SUMS';
 
 console.log('Fetching Firefox binary…');
 const fileArrayBuffer = await (await fetch(mozillaCdnUrl + firefoxBinaryFilename))
   .arrayBuffer();
 
 console.log('Calculating Firefox digest…');
 const firefoxBinaryDigest = await crypto.subtle.digest('SHA-512', fileArrayBuffer);
 
 const firefoxHexDigest = Array.from(new Uint8Array(firefoxBinaryDigest))
  .map((x) => x.toString(16).padStart(2, '0'))
  .join('');
 
 console.log('Fetching published binary digests…');
 // The SHA file contains digests of every firefox binary in this release,
 // so there is some organization performed.
 const shaPairs = (await (await fetch(mozillaCdnUrl + firefoxShaFilename)).text())
   .split(/
/).map((x) => x.split(/s+/));
 
 let verified = false;
 
 console.log('Checking calculated digest against published digests…');
 for (const [sha, filename] of shaPairs) {
  if (filename === firefoxBinaryFilename) {
   if (sha === firefoxHexDigest) {
    verified = true;
    break;
   }
  }
 }
 
 console.log('Verified:', verified);
})();

// Fetching Firefox binary…
// Calculating Firefox digest…
// Fetching published binary digests…
// Checking calculated digest against published digests…
// Verified: true

CryptoKeys and Algorithms

Cryptography would be meaningless without secret keys, and the SubtleCrypto object uses instances of the CryptoKey class to house these secrets. The CryptoKey class supports multiple types of encryption algorithms and allows for control over key extraction and usage.

The CryptoKey class supports the following algorithms, categorized by their parent cryptosystem:

• RSA (Rivest-Shamir-Adleman)—A public-key cryptosystem in which two large prime numbers are used to derive a pair of public and private keys that can be used to sign/verify or encrypt/decrypt messages. The trapdoor function for RSA is called the factoring problem.
• RSASSA-PKCS1-v1_5—An application of RSA used to sign messages with the private key and allow that signature to be verified with the public key.
  • SSA stands for signature schemes with appendix, indicating the algorithm supports signature generation and verification operations.
  • PKCS1 stands for Public-Key Cryptography Standards #1, indicating the algorithm exhibits mathematical properties that RSA keys must have.
  • RSASSA-PKCS1-v1_5 is deterministic, meaning the same message and key will produce an identical signature each time it is performed.
• RSA-PSS—Another application of RSA used to sign and verify messages.
  • PSS stands for probabilistic signature scheme, indicating that the signature generation incorporates a salt to randomize the signature.
  • Unlike RSASSA-PKCS1-v1_5, the same message and key will produce a different signature each time it is performed.
  • Unlike RSASSA-PKCS1-v1_5, RSA-PSS is provably reducible to the hardness of the RSA factoring problem.
  • In general, although RSASSA-PKCS1-v1_5 is still considered secure, RSA-PSS should be used as a replacement for RSASSA-PKCS1-v1_5.
• RSA-OAEP—An application of RSA used to encrypt messages with the public key and decrypt them with the private key.
  • OAEP stands for Optimal Asymmetric Encryption Padding, indicating that the algorithm utilizes a Feistel network to process the unencrypted message prior to encryption.
  • OAEP serves to convert the deterministic RSA encryption scheme to a probabilistic encryption scheme.
• ECC (Elliptic-Curve Cryptography)—A public-key cryptosystem in which a prime number and an elliptic curve are used to derive a pair of public and private keys that can be used to sign/verify messages. The trapdoor function for ECC is called the elliptic curve discrete logarithm problem. ECC is considered to be superior to RSA: Although both RSA and ECC are cryptographically strong, ECC keys are shorter than RSA keys and ECC cryptographic operations are faster than RSA operations.
• ECDSA (Elliptic Curve Digital Signature Algorithm)—An application of ECC used to sign and verify messages. This algorithm is an elliptic curve–flavored variant of the Digital Signature Algorithm (DSA).
• ECDH (Elliptic Curve Diffie-Hellman)—A key-generation and key-agreement application of ECC that allows for two parties to establish a shared secret over a public communication channel. This algorithm is an elliptic curve-flavored variant of the Diffie-Hellman key exchange (DH) protocol.
• AES (Advanced Encryption Standard)—A symmetric-key cryptosystem that encrypts/decrypts data using a block cipher derived from a substitution-permutation network. AES is used in different modes, which change the algorithm's characteristics.
• AES-CTR—The counter mode of AES. This mode behaves as a stream cipher by using an incremented counter to generate its keystream. It must also be provided a nonce, which is effectively used as an initialization vector. AES-CTR encryption/decryption is able to be parallelized.
• AES-CBC—The cipher block chaining mode of AES. Before encrypting each block of plaintext, it is XORed with the previous block of ciphertext—hence the "chaining" name. An initialization vector is used as the XOR input for the first block.
• AES-GCM—The Galois/Counter mode of AES. This mode uses a counter and initialization vector to generate a value, which is XORed with the plaintext of each block. Unlike CBC, the XOR inputs do not have dependencies on the previous block's encryption and therefore the GCM mode can be parallelized. Because of its excellent performance characteristics, AES-GCM enjoys utilization in many networking security protocols.
• AES-KW—The key wrapping mode of AES. This algorithm wraps a secret key into a portable and encrypted format that is safe for transmission on an untrusted channel. Once transmitted, the receiving party can then unwrap the key. Unlike other AES modes, AES-KW does not require an initialization vector.
• HMAC (Hash-Based Message Authentication Code)—An algorithm that generates message authentication codes used to verify that a message arrives unaltered when sent over an untrusted network. Two parties use a hash function and a shared private key to sign and verify messages.
• KDF (Key Derivation Functions)—Algorithms that can derive one or many keys from a master key using a hash function. KDFs are capable of generating keys of a different length or converting keys to different formats.
• HKDF (HMAC-Based Key Derivation Function)—A key derivation function designed to be used with a high-entropy input such as an existing key.
• PBKDF2 (Password-Based Key Derivation Function 2)—A key derivation function designed to be used with a low-entropy input such as a password string.

Generating CryptoKeys

Generating a random CryptoKey is accomplished with the SubtleCrypto.generateKey() method, which returns a promise that resolves to one or many CryptoKey instances. This method is passed a params object specifying the target algorithm, a boolean indicating whether or not the key should be extractable from the CryptoKey object, and an array of strings—keyUsages—indicating which SubtleCrypto methods the key can be used with.

Because different cryptosystems require different inputs to generate keys, the params object provides the required inputs for each cryptosystem:

• The RSA cryptosystem uses a RsaHashedKeyGenParams object.
• The ECC cryptosystem uses an EcKeyGenParams object.
• The HMAC cryptosystem uses an HmacKeyGenParams object.
• The AES cryptosystem uses an AesKeyGenParams object.

The keyUsages object describes which algorithms the key can be used with. It expects at least one of the following strings:

• encrypt
• decrypt
• sign
• verify
• deriveKey
• deriveBits
• wrapKey
• unwrapKey

Suppose you want to generate a symmetric key with the following properties:

• Supports the AES-CTR algorithm
• Key length of 128 bits
• Cannot be extracted from the CryptoKey object
• Is able to be used with the encrypt() and decrypt() methods

Generating this key can be accomplished in the following fashion:

(async function() {
 const params = {
  name: 'AES-CTR', 
  length: 128
 };

 const keyUsages = ['encrypt', 'decrypt'];

 const key = await crypto.subtle.generateKey(params, false, keyUsages);

 console.log(key);
 // CryptoKey {type: "secret", extractable: true, algorithm: {…}, usages: Array(2)}
})();

Suppose you want to generate an asymmetric key pair with the following properties:

• Supports the ECDSA algorithm
• Uses the P-256 elliptic curve
• Can be extracted from the CryptoKey object
• Is able to be used with the sign() and verify() methods

Generating this key pair can be accomplished in the following fashion:

(async function() {
 const params = {
  name: 'ECDSA', 
  namedCurve: 'P-256'
 };

 const keyUsages = ['sign', 'verify'];

 const {publicKey, privateKey} = await crypto.subtle.generateKey(params, true, 
   keyUsages);

 console.log(publicKey);
 // CryptoKey {type: "public", extractable: true, algorithm: {…}, usages: Array(1)}

 console.log(privateKey);
 // CryptoKey {type: "private", extractable: true, algorithm: {…}, usages: Array(1)}
})();

Exporting and Importing Keys

If a key is extractable, it is possible to expose the raw key binary from inside the CryptoKey object. The exportKey() method allows you to do so while also specifying the target format (raw, pkcs8, spki, or jwk). The method returns a promise that resolves to an ArrayBuffer containing the key:

(async function() {
 const params = {
  name: 'AES-CTR', 
  length: 128
 };
 const keyUsages = ['encrypt', 'decrypt'];

 const key = await crypto.subtle.generateKey(params, true, keyUsages);

 const rawKey = await crypto.subtle.exportKey('raw', key);

 console.log(new Uint8Array(rawKey)); 
 // Uint8Array[93, 122, 66, 135, 144, 182, 119, 196, 234, 73, 84, 7, 139, 43, 238, // 110]
})();

The inverse operation of exportKey() is importKey(). This method's signature is essentially a combination of generateKey() and exportKey(). The following method generates a key, exports it, and imports it once again:

(async function() {
 const params = {
  name: 'AES-CTR', 
  length: 128
 };
 const keyUsages = ['encrypt', 'decrypt'];
 const keyFormat = 'raw';
 const isExtractable = true;

 const key = await crypto.subtle.generateKey(params, isExtractable, keyUsages);

 const rawKey = await crypto.subtle.exportKey(keyFormat, key);

 const importedKey = await crypto.subtle.importKey(keyFormat, rawKey, params.name, 
   isExtractable, keyUsages);

 console.log(importedKey);
 // CryptoKey {type: "secret", extractable: true, algorithm: {…}, usages: Array(2)}
})();

Deriving Keys from Master Keys

The SubtleCrypto object allows you to derive new keys with configurable properties from an existing secret. It supports a deriveKey() method that returns a promise resolving to a CryptoKey, and a deriveBits() method that returns a promise resolving to an ArrayBuffer.

The deriveBits() function accepts an algorithm params object, the master key, and the length in bits of the output. This can be used in situations where two people, each with their own key pairs, wish to obtain a shared secret key. The following example uses the ECDH algorithm to generate reciprocal keys from two keypairs and ensures that they derive the same key bits:

(async function() {
 const ellipticCurve = 'P-256';
 const algoIdentifier = 'ECDH';
 const derivedKeySize = 128;

 const params = {
  name: algoIdentifier, 
  namedCurve: ellipticCurve
 };

 const keyUsages = ['deriveBits'];

 const keyPairA = await crypto.subtle.generateKey(params, true, keyUsages);
 const keyPairB = await crypto.subtle.generateKey(params, true, keyUsages);

 // Derive key bits from A's public key and B's private key
 const derivedBitsAB = await crypto.subtle.deriveBits(
   Object.assign({ public: keyPairA.publicKey }, params), 
   keyPairB.privateKey, 
   derivedKeySize);
 
 // Derive key bits from B's public key and A's private key
 const derivedBitsBA = await crypto.subtle.deriveBits(
   Object.assign({ public: keyPairB.publicKey }, params), 
   keyPairA.privateKey, 
   derivedKeySize);
 
 const arrayAB = new Uint32Array(derivedBitsAB);
 const arrayBA = new Uint32Array(derivedBitsBA);
 
 // Ensure key arrays are identical
 console.log(
   arrayAB.length === arrayBA.length && 
   arrayAB.every((val, i) => val === arrayBA[i])); // true
})();

The deriveKey() method behaves similarly, returning an instance of a CryptoKey instead of an ArrayBuffer. The following example takes a raw string, applies the PBKDF2 algorithm to import it into a raw master key, and derives a new key in AES-GCM format:

(async function() {
 const password = 'foobar';
 const salt = crypto.getRandomValues(new Uint8Array(16));
 const algoIdentifier = 'PBKDF2';
 const keyFormat = 'raw';
 const isExtractable = false;
 
 const params = {
  name: algoIdentifier
 };
 
 const masterKey = await window.crypto.subtle.importKey(
  keyFormat,
  (new TextEncoder()).encode(password),
  params,
  isExtractable,
  ['deriveKey']
 );
 
 const deriveParams = {
  name: 'AES-GCM',
  length: 128
 };
 
 const derivedKey = await window.crypto.subtle.deriveKey(
  Object.assign({salt, iterations: 1E5, hash: 'SHA-256'}, params),
  masterKey,
  deriveParams,
  isExtractable,
  ['encrypt']
 );

 console.log(derivedKey);
 // CryptoKey {type: "secret", extractable: false, algorithm: {…}, usages: Array(1)}
})();

Signing and Verifying Messages with Asymmetric Keys

The SubtleCrypto object allows you to use public-key algorithms to generate signatures using a private key or to verify signatures using a public key. These are performed using the SubtleCrypto.sign() and SubtleCrypto.verify() methods, respectively.

Signing a message requires a params object to specify the algorithm and any necessary values, the private CryptoKey, and the ArrayBuffer or ArrayBufferView to be signed. The following example generates an elliptic curve key pair and uses the private key to sign a message:

(async function() {
 const keyParams = {
  name: 'ECDSA', 
  namedCurve: 'P-256'
 };

 const keyUsages = ['sign', 'verify'];

 const {publicKey, privateKey} = await crypto.subtle.generateKey(keyParams, true, 
   keyUsages);

 const message = (new TextEncoder()).encode('I am Satoshi Nakamoto');
 
 const signParams = {
  name: 'ECDSA',
  hash: 'SHA-256'
 };

 const signature = await crypto.subtle.sign(signParams, privateKey, message);

 console.log(new Uint32Array(signature)); 
 // Uint32Array(16) [2202267297, 698413658, 1501924384, 691450316, 778757775, … ]
})();

An individual wishing to verify this message against the signature could use the public key and the SubtleCrypto.verify() method. This method's signature is nearly identical to sign() with the exception that it must be provided the public key as well as the signature. The following example extends the previous example by verifying the generated signature:

(async function() {
 const keyParams = {
  name: 'ECDSA', 
  namedCurve: 'P-256'
 };

 const keyUsages = ['sign', 'verify'];

 const {publicKey, privateKey} = await crypto.subtle.generateKey(keyParams, true, 
   keyUsages);

 const message = (new TextEncoder()).encode('I am Satoshi Nakamoto');
 
 const signParams = {
  name: 'ECDSA',
  hash: 'SHA-256'
 };

 const signature = await crypto.subtle.sign(signParams, privateKey, message);

 const verified = await crypto.subtle.verify(signParams, publicKey, signature, 
   message);
 
 console.log(verified); // true
})();

Encrypting and Decrypting with Symmetric Keys

The SubtleCrypto object allows you to use both public-key and symmetric algorithms to encrypt and decrypt messages. These are performed using the SubtleCrypto.encrypt() and SubtleCrypto.decrypt() methods, respectively.

Encrypting a message requires a params object to specify the algorithm and any necessary values, the encryption key, and the data to be encrypted. The following example generates a symmetric AES-CBC key, encrypts it, and finally decrypts a message:

(async function() {
 const algoIdentifier = 'AES-CBC';

 const keyParams = {
  name: algoIdentifier, 
  length: 256
 };

 const keyUsages = ['encrypt', 'decrypt'];

 const key = await crypto.subtle.generateKey(keyParams, true, 
   keyUsages);

 const originalPlaintext = (new TextEncoder()).encode('I am Satoshi Nakamoto');

 const encryptDecryptParams = {
  name: algoIdentifier, 
  iv: crypto.getRandomValues(new Uint8Array(16))
 };
 
 const ciphertext = await crypto.subtle.encrypt(encryptDecryptParams, key, 
   originalPlaintext);

 console.log(ciphertext);
 // ArrayBuffer(32) {}

 const decryptedPlaintext = await crypto.subtle.decrypt(encryptDecryptParams, key, 
   ciphertext);
 
 console.log((new TextDecoder()).decode(decryptedPlaintext));
 // I am Satoshi Nakamoto
})();

Wrapping and Unwrapping a Key

The SubtleCrypto object allows you to wrap and unwrap keys to allow for transmission over an untrusted channel. These are performed using the SubtleCrypto.wrapKey() and SubtleCrypto.unwrapKey() methods, respectively.

Wrapping a key requires a format string, the CryptoKey instance to be wrapped, the CryptoKey to perform the wrapping, and a params object to specify the algorithm and any necessary values. The following example generates a symmetric AES-GCM key, wraps the key with AES-KW, and finally unwraps the key:

(async function() {
 const keyFormat = 'raw';
 const extractable = true;

 const wrappingKeyAlgoIdentifier = 'AES-KW';
 const wrappingKeyUsages = ['wrapKey', 'unwrapKey'];
 const wrappingKeyParams = {
  name: wrappingKeyAlgoIdentifier, 
  length: 256
 };
 
 const keyAlgoIdentifier = 'AES-GCM';
 const keyUsages = ['encrypt'];
 const keyParams = {
  name: keyAlgoIdentifier, 
  length: 256
 };

 const wrappingKey = await crypto.subtle.generateKey(wrappingKeyParams, extractable, 
   wrappingKeyUsages);
   
 console.log(wrappingKey);
 // CryptoKey {type: "secret", extractable: true, algorithm: {…}, usages: Array(2)}
 
 const key = await crypto.subtle.generateKey(keyParams, extractable, keyUsages);
 
 console.log(key);
 // CryptoKey {type: "secret", extractable: true, algorithm: {…}, usages: Array(1)}
 
 const wrappedKey = await crypto.subtle.wrapKey(keyFormat, key, wrappingKey, 
   wrappingKeyAlgoIdentifier);
 
 console.log(wrappedKey);
 // ArrayBuffer(40) {}
 
 const unwrappedKey = await crypto.subtle.unwrapKey(keyFormat, wrappedKey, 
   wrappingKey, wrappingKeyParams, keyParams, extractable, keyUsages);
 
 console.log(unwrappedKey);
 // CryptoKey {type: "secret", extractable: true, algorithm: {…}, usages: Array(1)}
})()