Each write head receives an erase vector, e_t, and an add vector, a_t, to reset and write to memory, just like an LSTM cell, as follows: M_t(i) ← M_t(i) [1- e_t ⁽ⁱ⁾ w_t (i) ] + w_t (i) a_t (i). 

Here is the pseudo code for the preceding operations:

mem_size = 128 #The size of memory 
mem_dim = 16 #The dimensionality for memory 
shift_range = 1 # defining shift[-1, 0, 1]

## last output layer from LSTM controller: last_output
## Previous memory state: M_prev
def Linear(input_, output_size, stddev=0.5):
  '''Applies a linear transformation to the input data: input_
  implements dense layer with tf.random_normal_initializer(stddev=stddev)
  as weight initializer
  ''''
def get_controller_head(M_prev, last_output, is_read=True):
  
  k = tf.tanh(Linear(last_output, mem_dim))
  # Interpolation gate
  g = tf.sigmoid(Linear(last_output, 1)
  
  # shift weighting
  w = Linear(last_output, 2 * shift_range + 1)
    s_w = softmax(w)
  
  # Cosine similarity
  similarity = smooth_cosine_similarity(M_prev, k) # [mem_size x 1]
  # Focusing by content
  content_focused_w = softmax(scalar_mul(similarity, beta))

  # Convolutional shifts
  conv_w = circular_convolution(gated_w, s_w)

  if is_read:
    read = matmul(tf.transpose(M_prev), w)
    return w, read
  else:
    erase = tf.sigmoid(Linear(last_output, mem_dim)
    add = tf.tanh(Linear(last_output, mem_dim))
    return w, add, erase

The full TensorFlow implementation of NTMs is available here: https://github.com/carpedm20/NTM-tensorflow. NTM algorithms can learn to copy—they can learn an algorithm to copy a sequence of random numbers. The following shows how NTMs use the memory read and write heads and shift them to implement a copy algorithm:

Similarly, NTMs can efficiently learn sorting algorithms from data, given a set of random sequences and corresponding sorted sequences.