To construct a LSTM, we simply call the MakeLstm() function of our NetworkBuilder. This function will ingest several input parameters and return to us a network object:
INetwork nn = NetworkBuilder.MakeLstm(inputDimension,
hiddenDimension,hiddenLayers,outputDimension,data.GetModelOutputUnitToUse(),
initParamsStdDev, rng);
As you can see, internally this calls our MakeLSTM() function inside the NetworkBuilder object. Here is a look at that code:
public static NeuralNetwork MakeLstm(int inputDimension, int hiddenDimension, int hiddenLayers, int outputDimension, INonlinearity decoderUnit, double initParamsStdDev, Random rng)
{
List<ILayer> layers = new List<ILayer>();
for (int h = 0; h<hiddenLayers; h++)
{
Add all of the hidden layers:
layers.Add(h == 0? new LstmLayer(inputDimension, hiddenDimension, initParamsStdDev, rng): new LstmLayer(hiddenDimension, hiddenDimension, initParamsStdDev, rng));
}
Add the feed forward layer:
layers.Add(new FeedForwardLayer(hiddenDimension, outputDimension, decoderUnit, initParamsStdDev, rng));
Create the network:
return new NeuralNetwork(layers);
}