In the previous sections, we understood the fundamental concepts underlying the use of a perceptron as a classifier. The time has come to put into practice what has been studied so far. We will do it by analyzing an example in which we will try to classify the floral species on the basis of the size of the petals and sepals of an Iris. As you will recall, the iris dataset has already been used in Chapter 3, Deep Learning Using Multilayer Neural Networks. The reason for its re-use is not only due to the quality of the data contained in it that allows the reader to easily understand the concepts outlined, but also, and more importantly, to be able to compare the different algorithms.
As you will recall, the dataset contains 50 samples from each of three species of Iris (Iris setosa, Iris virginica, and Iris versicolor). Four features were measured from each sample: the length and the width of the sepals and petals, in centimeters.
The following variables are contained:
- Sepal length in cm
- Sepal width in cm
- Petal length in cm
- Petal width in cm
- Class: setosa, versicolour, virginica
In the example presented, we will try to classify the setosa and versicolor species through linear separation.
Let us implement a perceptron function in R for the iris dataset. The code is presented next:
######################################################################
###Chapter 4 - Introduction to Neural Networks - using R ##########
###Simple Perceptron implementation function in R - iris dataset ####
######################################################################
data(iris)
head(iris, n=20)
iris_sub=iris[1:100, c(1, 3, 5)]
names(iris_sub)=c("sepal", "petal", "species")
head(iris_sub)
library(ggplot2)
ggplot(iris_sub, aes(x = sepal, y = petal)) +
geom_point(aes(colour=species, shape=species), size = 3) +
xlab("Sepal length") +
ylab("Petal length") +
ggtitle("Species vs Sepal and Petal lengths")
euclidean.norm = function(x) {sqrt(sum(x * x))}
distance.from.plane = function(z,w,b) {
sum(z*w) + b
}
classify.linear = function(x,w,b) {
distances = apply(x, 1, distance.from.plane, w, b)
return(ifelse(distances < 0, -1, +1))
}
perceptron = function(x, y, learning.rate=1) {
w = vector(length = ncol(x)) # initialize weights
b = 0 # Initialize bias
k = 0 # count updates
R = max(apply(x, 1, euclidean.norm))
mark.complete = TRUE
while (mark.complete) {
mark.complete=FALSE
yc = classify.linear(x,w,b)
for (i in 1:nrow(x)) {
if (y[i] != yc[i]) {
w = w + learning.rate * y[i]*x[i,]
b = b + learning.rate * y[i]*R^2
k = k+1
mark.complete=TRUE
}
}
}
s = euclidean.norm(w)
return(list(w=w/s,b=b/s,updates=k))
}
x = cbind(iris_sub$sepal, iris_sub$petal)
y = ifelse(iris_sub$species == "setosa", +1, -1)
p = perceptron(x,y)
plot(x,cex=0.2)
points(subset(x,Y==1),col="black",pch="+",cex=2)
points(subset(x,Y==-1),col="red",pch="-",cex=2)
intercept = - p$b / p$w[[2]]
slope = - p$w[[1]] /p$ w[[2]]
abline(intercept,slope,col="green")
Now, let us go through the code line-by-line. Following the style in the rest of this book, we will present a portion of the code first as follows and then explain it in detail:
data(iris)
head(iris, n=20)
The first command loads the iris dataset, which is contained in the datasets library, and saves it in a given dataframe. Then we use the head function to display the first 20 lines of the dataset. Remember, the head function returns the first or last parts of a vector, matrix, table, dataframe, or function. In this case, we specify the number of lines that must be displayed (n=20). The following is the result:
> head(iris, n=20)
Sepal.Length Sepal.Width Petal.Length Petal.Width Species
1 5.1 3.5 1.4 0.2 setosa
2 4.9 3.0 1.4 0.2 setosa
3 4.7 3.2 1.3 0.2 setosa
4 4.6 3.1 1.5 0.2 setosa
5 5.0 3.6 1.4 0.2 setosa
6 5.4 3.9 1.7 0.4 setosa
7 4.6 3.4 1.4 0.3 setosa
8 5.0 3.4 1.5 0.2 setosa
9 4.4 2.9 1.4 0.2 setosa
10 4.9 3.1 1.5 0.1 setosa
11 5.4 3.7 1.5 0.2 setosa
12 4.8 3.4 1.6 0.2 setosa
13 4.8 3.0 1.4 0.1 setosa
14 4.3 3.0 1.1 0.1 setosa
15 5.8 4.0 1.2 0.2 setosa
16 5.7 4.4 1.5 0.4 setosa
17 5.4 3.9 1.3 0.4 setosa
18 5.1 3.5 1.4 0.3 setosa
19 5.7 3.8 1.7 0.3 setosa
20 5.1 3.8 1.5 0.3 setosa
Let's go back to the code. We will get the binary output by extracting only 100 rows of the iris dataset, and extracting only sepal length and petal length with species:
iris_sub=iris[1:100, c(1, 3, 5)]
names(iris_sub)=c("sepal", "petal", "species")
head(iris_sub)
Here, only the first 100 rows of the iris dataset are taken and columns 1,3, and 5 are chosen. This is because the first 100 lines contain the data for the two species (setosa and versicolor) we are interested in, in the following example. The three columns are sepal.length(x1), petal.length(x2), and species(y - output).
library(ggplot2)
ggplot(iris_sub, aes(x = sepal, y = petal)) +
geom_point(aes(colour=species, shape=species), size = 3) +
xlab("Sepal length") +
ylab("Petal length") +
ggtitle("Species vs Sepal and Petal lengths")
First we load the ggplot2 library, and then we use ggplot() to get the scatterplot of the distribution of species with respect to sepal.length and petal.length. Of course, the library should have been installed beforehand.
The objective of the perceptron function is to find a linear separation of the setosa and versicolor species. The following figure shows Sepal length versus Petal length for the two species of Iris flower:
As can be seen, the two species are placed in distinct areas of the plane, so linear separation is possible. At this point, we need to define functions to do the perceptron processing:
euclidean.norm = function(x) {sqrt(sum(x * x))}
distance.from.plane = function(z,w,b) {
sum(z*w) + b
}
classify.linear = function(x,w,b) {
distances = apply(x, 1, distance.from.plane, w, b)
return(ifelse(distances < 0, -1, +1))
}
perceptron = function(x, y, learning.rate=1) {
w = vector(length = ncol(x)) # initialize weights
b = 0 # Initialize bias
k = 0 # count updates
R = max(apply(x, 1, euclidean.norm))
mark.complete = TRUE
while (mark.complete) {
mark.complete=FALSE
yc = classify.linear(x,w,b)
for (i in 1:nrow(x)) {
if (y[i] != yc[i]) {
w = w + learning.rate * y[i]*x[i,]
b = b + learning.rate * y[i]*R^2
k = k+1
mark.complete=TRUE
}
}
}
s = euclidean.norm(w)
return(list(w=w/s,b=b/s,updates=k))
}We define the perceptron function as discussed in the algorithm for perceptron training. We apply learning.rate as 1 and try to update the weights in each loop. Once we have the output and the function (weights*inputs) equal, we stop the training and move out. The updated weights are returned by the function. The objective of the function is to get a set of optimal weights needed for the model as follows:
x = cbind(iris_sub$sepal, iris_sub$petal)
y = ifelse(iris_sub$species == "setosa", +1, -1)
p = perceptron(x,y)
With the first line, we set the x inputs as the sepal and petal lengths. sepal.length and petal.length form the input matrix. In the second line, we set label output as positive for setosa and the rest as negative. The output is either setosa or not (+1 or -1). In the third line, we run the perceptron function.
We call the perceptron function with x and y, which gives the optimal weights for the perceptron as shown in the following code sample:
plot(x,cex=0.2)
points(subset(x,Y==1),col="black",pch="+",cex=2)
points(subset(x,Y==-1),col="red",pch="*",cex=2)
intercept = - p$b / p$w[[2]]
slope = - p$w[[1]] /p$ w[[2]]
abline(intercept,slope,col="green")
The previous lines of code plot x and y, highlighting setosa and versicolor as + and * points in the graph. We then find the intercept and slope of the p variable (perceptron), returned by the perceptron. Plotting the linear separation line gives the following graph:
To summarize, we have implemented the perceptron using R code and found optimal weights. The linear separation has been achieved using the perceptron.