What is a Perceptron?

The Perceptron is an algorithm for supervised learning of binary classifiers. A binary classifier is a function which can decide whether or not an input, represented by a vector of numbers, belongs to some specific class.

Biological Neuron
The Perceptron is a simplified model of a biological neuron. Most neurons receive signals via the dendrites and send out signals down the axon. At the majority of synapses, signals cross from the axon of one neuron to a dendrite of another. The following figure demonstrate how a biological neuron in terms of input and output is modeled.

Neuron - Math Model
As mentioned before, the Perceptron is an algorithm for learning a binary classifier called a threshold function: a function that maps its input  (a real-valued vector) to an output value  (a single binary value):


Perceptron Learning Algorithm
Perceptron learning algorithm can be written as the following steps: 
   1. Initialize the weights and the threshold. 
   2. For each example 
       a. Calculate the actual output
            {\displaystyle {\begin{aligned}y_{j}(t)&=f[\mathbf {w} (t)\cdot \mathbf {x} _{j}]\\&=f[w_{0}(t)x_{j,0}+w_{1}(t)x_{j,1}+w_{2}(t)x_{j,2}+\dotsb +w_{n}(t)x_{j,n}]\end{aligned}}}
        b. Update the weights as followoing
            {\displaystyle w_{i}(t+1)=w_{i}(t)+r\cdot (d_{j}-y_{j}(t))x_{j,i}}
    3. The second step may be repeated until the iteration error is less than a user-specified error threshold  or a predetermined number of iterations have been completed, where error is defined as followin.

Multiclass Perceptron (MCP)
So far, we have used the Perceptron as a binary classifier, telling us the probability p that a point x belongs to one of two classes. The probability of x  belonging to the respective other class is then given by 1−p. Generally, however, we could have more than two classes which means more than one neuron should be used in the output. Also, Perceptron can have more than one layer which is called multi-layered Perceptron.

Multi Layer Perceptron usually uses BackPropagation Algorithm for learning.

Show me some Code


Further Readings


Popular Posts