To create a model, we are going to build a one-directional recurrent neural network that contains long short-term memory (LSTM) cells as shown in the following:
public static Function CreateModel(Variable input, int outDim, int LSTMDim, int cellDim, DeviceDescriptor device, string outputName)
{
Func<Variable, Function> pastValueRecurrenceHook = (x) => CNTKLib.PastValue(x);
Create a LSTM cell for each input variable, as follows:
Function LSTMFunction = LSTMPComponentWithSelfStabilization<float>(input, new[] { LSTMDim }, new[] { cellDim }, pastValueRecurrenceHook, pastValueRecurrenceHook, device)?.Item1;
After the LSTM sequence is created, return the last cell in order to continue generating the network, as follows:
pre> Function lastCell = CNTKLib.SequenceLast(LSTMFunction);
Implement dropout for 20%, as follows:
var dropOut = CNTKLib.Dropout(lastCell,0.2, 1);
Create the last dense layer before the output, as follows:
return FullyConnectedLinearLayer(dropOut, outDim, device, outputName);
}