Let's move on to create a neural network using symbolic programming:

1. We start with sampling data into training and validation datasets and creating respective iterators:

batch_size <- 3
train_ids <- 1:4
val_ids <- 5:6

## create data iterators
train_data <- mx.io.arrayiter(data = x_data[,,train_ids, drop = F],label = y_data[, train_ids], batch.size = batch_size,shuffle = TRUE)
val_data <- mx.io.arrayiter(data = x_data[,,val_ids, drop = F], label = y_data[, val_ids], batch.size = batch_size, shuffle = FALSE)

2. Now, let's create an RNN symbol with a one-to-one model configuration:

symbol <- rnn.graph(num_rnn_layer = 2,
 num_hidden = 30,
 input_size = NULL,
 num_embed = NULL,
 num_decode = 1,
 masking = F, 
 loss_output = "linear",
 ignore_label = -1, 
 cell_type = "lstm", 
 output_last_state = T,
 config = "one-to-one")

3. Next, we define our loss function:

seq_metric_mse <- mx.metric.custom("MSE", function(label, pred) {
 label = mx.nd.reshape(label, shape = -1)
 pred = mx.nd.reshape(pred, shape = -1)
 res <- mx.nd.mean(mx.nd.square(label - pred))
 return(as.array(res))
})

4. We set the device to use for training the model. Then, we define weight initialization and configure the optimizer:

ctx <- mx.cpu()
initializer <- mx.init.Xavier(rnd_type = "gaussian",
 factor_type = "avg", 
 magnitude = 1)
optimizer <- mx.opt.create("adadelta",
 rho = 0.9, 
 eps = 1e-06, 
 wd = 1e-06, 
 clip_gradient = 1, 
 rescale.grad = 1/batch_size)

5. Let's now train the network for 50 epochs with bucket support:

model <- mx.model.buckets(symbol = symbol, 
 train.data = train_data, 
 eval.data = val_data,
 num.round = 50, 
 ctx = ctx, 
 verbose = TRUE, 
 metric = seq_metric_mse, 
 initializer = initializer,
 optimizer = optimizer)

6. After training the network, we extract the state symbols from the trained model:

internals <- model$symbol$get.internals()
sym_state <- internals$get.output(which(internals$outputs %in% "RNN_state"))
sym_state_cell <- internals$get.output(which(internals$outputs %in% "RNN_state_cell"))
sym_output <- internals$get.output(which(internals$outputs %in% "loss_output"))
symbol <- mx.symbol.Group(sym_output, sym_state, sym_state_cell)

7. We use the symbol created in step 6 to create an inference of the RNN model. We also use the sixth data sample to get the initial values for RNN states, which will be used to initiate the prediction of future timestamps.

Note that the label is required only to create the iterator and will not be used in the inference:

data <- mx.nd.array(x_data[, , 6, drop = F])
label <- mx.nd.array(y_data[, 6, drop = F])

inference_data <- mx.io.arrayiter(data = data, 
 label = label, 
 batch.size = 1, 
 shuffle = FALSE)
infer <- mx.infer.rnn.one(infer.data = inference_data, 
 symbol = symbol, 
 arg.params = model$arg.params,
 aux.params = model$aux.params, 
 input.params = NULL, 
 ctx = ctx)

8. Now, we iterate over timesteps to generate predictions for three timesteps of our seventh sample. For predicting a timestep, we use the RNN states information generated from the actual values of the previous timestep and not the predicted one:

pred_length <- 3
predicted <- numeric()

for (i in 1:pred_length) {
 data <- mx.nd.array(x_data[, i, 7, drop = F])
 label <- mx.nd.array(y_data[i, 7, drop = F])
 infer.data <- mx.io.arrayiter(data = data, 
 label = label, 
 batch.size = 1, 
 shuffle = FALSE)
 ## use previous RNN state values
 infer <- mx.infer.rnn.one(infer.data = infer.data,
 symbol = symbol,
 ctx = ctx, 
 arg.params = model$arg.params,
 aux.params = model$aux.params, 
 input.params = 
 list(rnn.state=infer[[2]], 
 rnn.state.cell = infer[[3]]))
 pred <- infer[[1]]
 predicted <- c(predicted, as.numeric(as.array(pred)))
}

predicted

Next, we will understand the steps carried out in this section.