Now, it is time to develop a neural network using OOP concepts and explain the related theory. The project presented in the previous chapter was adapted to implement the perceptron and adaline rules, as well as the Delta rule.
The NeuralNet class presented in the previous chapter has been updated to include the training dataset (input and target output), learning parameters, and activation function settings. The InputLayer function was also updated to include one method. We added to the project the Adaline, Perceptron, and Training classes. Details on the implementation of each class can be found in the codes. However, now, let's make the connection between the neural learning and the Java implementation of the Training class.
The Training class should be used for training neural networks. In this chapter, we are going to use this class to train Perceptron and Adaline classes. Also, the activation functions that are foreseen to be used in the neural networks in this chapter should be considered. So, now, let's define two enumeration sets that will handle these settings:
public enum TrainingTypesENUM {
PERCEPTRON, ADALINE;
}
public enum ActivationFncENUM {
STEP, LINEAR, SIGLOG, HYPERTAN;
}In addition to these parameters, we need to define the condition for stopping, the error, the MSE error, and the number of epochs, as shown in the following code:
private int epochs; private double error; private double mse;
The learning rate has already been defined in the NeuralNet class and will be used here.
Finally, we need a method to update the weights of a given neuron. So, let's take a look at the CalcNewWeight method:
private double calcNewWeight(TrainingTypesENUM trainType,
double inputWeightOld, NeuralNet n, double error,
double trainSample, double netValue) {
switch (trainType) {
case PERCEPTRON:
return inputWeightOld + n.getLearningRate() * error * trainSample;
case ADALINE:
return inputWeightOld + n.getLearningRate() * error * trainSample
* derivativeActivationFnc(n.getActivationFnc(), netValue);
default:
throw new IllegalArgumentException(trainType
+ " does not exist in TrainingTypesENUM");
}
}We see in this method a switch clause that selects the update procedure according to the training type (Adaline or Perceptron). We can also see the inputWeightOld (the old weights), n (neural network under training), error (difference between target and neural output), trainsample (input to the weight), and netValue (weighted sum before processing by activation function) parameters. The learning rate is retrieved by calling the getLearningRate() function of the NeuralNet class.
One interesting detail is the derivative of the activation function that is called for the Adaline training type, which is the Delta rule. All the activation functions are implemented as methods inside the Training class, and their respective derivatives are implemented as well. The derivativeActivationFnc method helps to call the derivative corresponding to the activation function passed in the argument.
Two special methods are implemented in the Training class: one for training the neural network and the other for training the neurons of some layer. Although this won't be necessary in this chapter, it is always good to have a code prepared for future examples or updates. Let's take a quick look at the implementation of the method train:
public NeuralNet train(NeuralNet n) {
ArrayList<Double> inputWeightIn = new ArrayList<Double>();
int rows = n.getTrainSet().length;
int cols = n.getTrainSet()[0].length;
while (this.getEpochs() < n.getMaxEpochs()) {
double estimatedOutput = 0.0;
double realOutput = 0.0;
for (int i = 0; i < rows; i++) {
double netValue = 0.0;
for (int j = 0; j < cols; j++) {
inputWeightIn = n.getInputLayer().getListOfNeurons().get(j)
.getListOfWeightIn();
double inputWeight = inputWeightIn.get(0);
netValue = netValue + inputWeight * n.getTrainSet()[i][j];
}
estimatedOutput = this.activationFnc(n.getActivationFnc(),
netValue);
realOutput = n.getRealOutputSet()[i];
this.setError(realOutput - estimatedOutput);
if (Math.abs(this.getError()) > n.getTargetError()) {
// fix weights
InputLayer inputLayer = new InputLayer();
inputLayer.setListOfNeurons(this.teachNeuronsOfLayer(cols,
i, n, netValue));
n.setInputLayer(inputLayer);
}
}
this.setMse(Math.pow(realOutput - estimatedOutput, 2.0));
n.getListOfMSE().add(this.getMse());
this.setEpochs(this.getEpochs() + 1);
}
n.setTrainingError(this.getError());
return n;
}This method receives a neural network in the parameter and produces another neural network with trained weights. Further, we see a while clause that loops while the number of epochs does not reach the maximum set out in the Training class. Inside this loop, there is a for clause that iterates over all the training samples that are presented to the network and so begins the process of calculating the neural output for the input in the current iteration.
When it gets the real output of the network, it compares it to the estimated output and calculates the error. This error is checked, and if it is higher than the minimum error, then it starts the update procedure by calling the teachNeuronsOfLayer method in the following line:
inputLayer.setListOfNeurons(this.teachNeuronsOfLayer(cols,
i, n, netValue));The implementation of this method is found in the codes attached with this chapter.
Then, this process is repeated iteratively until all the neural samples are passed to the neural network, and then, until the maximum number of epochs is reached.
The following table shows all the fields and methods for all the classes covered in this chapter:
The updated class diagram is shown in the following figure. Attributes and methods already explained in the previous chapter were omitted. Further, configuration methods of new attributes (setters and getters) were also omitted.
