A perceptron can be understood as anything that takes multiple inputs and produces one output. It is the simplest form of a neural network. The perceptron was proposed by Frank Rosenblatt in 1958 as an entity with an input and output layer and a learning rule based on minimizing the error. This learning function called error backpropagation alters connective weights (synapses) based on the actual output of the network with respect to a given input, as the difference between the actual output and the desired output.

The enthusiasm was enormous and the cybernetics industry was born. But later, scientists Marvin Minsky and Seymour Papert (1969) demonstrated the limits of the perceptron. Indeed, a perceptron is able to recognize, after a suitable training, only linearly separable functions. For example, the XOR logic function cannot be implemented by a perceptron.

The following image showns Frank Rosenblatt at the Cornell Aeronautical Laboratory (1957-1959), while working on the Mark I Perceptron classifier:

Potentially, a multilevel network of percetters could solve more complex problems, but the increasing computational complexity of training made this path impracticable. Only in recent times have we started to consider the utility of this operational entity.

In the single form, a perceptron has one neuron unit accepting inputs and producing a set of outputs.

For example, let us take a look at the following image:

Here x₁, x₂,.., x_n are the set of inputs and x₀ is the bias. x₀ is set to 1. The output y is the sum product of w_ix_i. The signum function is applied after the sum product has been executed.

It separates the output as:

• If y>0, the output is 1
• If y<=0, the output is -1

The bias is constant and is associated with weight w₀. This perceptron functions as a linear separator, splitting the output into one category, -1 or +1.

Note that here this is no backpropagation and the weight update updates through steps we will soon see. There is a threshold setup which determines the value of the output. The output here is binary (either -1 or +1), which can be set as zero or one.

Hence, a perceptron is a simple classification function that directly makes its prediction. The core of the functionality lives in the weights and how we update the weights to the best possible prediction of y.

This case is a simple perceptron or basic perceptron, and the outputs are binary in nature: 0/1 true/false +1/-1.

There is another type of perceptron called the multi-class perceptron, which can classify many possible labels for an animal, such as dog, cat, or bird.

In the following figure is shown a simple perceptron architecture versus multi-class perceptron architecture:

By modifying the weighting vector, we can modify the output of a perceptron to improve the learning or storage properties. For example, we can try to instruct a perceptron such that given an input x, output y is as close as possible to a given a priori chosen y actual value. The computational capabilities of a single perceptron are, however, limited, and the performance that can be obtained depends heavily on both the input choice and the choice of function that you want to implement.

In fact, inputs can be limited to a subset of all the possible inputs, or be randomly extracted according to a certain predetermined probability distribution. To a lesser extent, the performance of such a system also depends on how the distance between the actual outputs and the expected outputs is quantified.

Once you have identified the problem of learning, you can try to find the optimal weight assignment for the given problem.