While constructing the output layer, make note of the nOut value of the preceding LSTM input layer. This will be taken as nIn for the output layer. nIn should be the same as nOut of the preceding LSTM input layer.

In steps 1 and step 2, we are essentially creating an LSTM neural network, an extended version of a regular RNN. We used gated cells to have some sort of internal memory to hold long-term dependencies. For a predictive model to make predictions (patient mortality), we need to have probability produced by the output layer. In step 2, we see that SOFTMAX is used at the output layer of a neural network. This activation function is very helpful for computing the probability for the specific label. MCXENT is the ND4J implementation for the negative loss likelihood error function. Since we use the negative loss likelihood loss function, it will push the results when the probability value is found to be high for a label on a particular iteration.

RnnOutputLayer is more like an extended version of regular output layers found in feed-forward networks. We can also use RnnOutputLayer for one-dimensional CNN layers. There is also another output layer, named RnnLossLayer, where the input and output activations are the same. In the case of RnnLossLayer, we have three dimensions with the [miniBatchSize,nIn,timeSeriesLength] and [miniBatchSize,nOut,timeSeriesLength] shape, respectively.

Note that we'll have to specify the input layer that is to be connected to the output layer. Take a look at this code again:

builder.addLayer("predictMortality", new RnnOutputLayer.Builder(LossFunctions.LossFunction.MCXENT)
 .activation(Activation.SOFTMAX)
 .nIn(LSTM_LAYER_SIZE).nOut(labelCount).build(),"L1")

We mentioned that the L1 layer is the input layer to the output layer.