The biological neuron : A biological neuron receives incoming signals from other neurons through its dendrites. The cell body (soma) integrates and processes these signals. If the combined input reaches a certain threshold, the neuron generates an electrical impulse called an action potential, which travels along a long projection known as the axon. The signal then reaches the branching axon terminals, where it is transmitted to neighboring neurons through connections called synapses.

McCulloch–Pitts (MCP) neuron

McCulloch and Pitts [(1943) “A Logical Calculus of the Ideas Immanent in Nervous Activity”] proposed a simple mathematical model of a biological neuron, inspired by how the brain could perform computations using logical operations (If X is true AND Y is true, then Z is true.)

McCulloch and Pitts turned this into a simple computational model, an artificial neuron. They showed how by using one such artificial neuron one could implement certain basic Boolean logical operations such as AND, OR, NOT which are the building blocks of digital computation.

The McCulloch-Pitts model represents a neuron as a binary unit that receives inputs and produces an output (0 or 1) based on whether a threshold is reached.

In the simple version of the McCulloch-Pitts model, x1 and x2 can be either 0 or 1

The neurode’s (“neuron” + “node”) output y is calculated by first summing the inputs and then checking to see if that sum is greater than or equal to some threshold, theta (θ). If so, y equals 1; if not, y equals 0.

The formal mathematical description of the simple neurode is :

For an arbitrary sequence of inputs, x1, x2, x3,…, xn.

g(x) : sums up the inputs f(g(x)) : takes the summation and performs the thresholding

We can design some of the basic Boolean logic gates.

  • : Implements an AND gate for two binary inputs.
  • : Implements an OR gate for two binary inputs.

Inhibition in MCP

To distinguish between excitatory and inhibitory inputs.

  • = Sum of excitatory inputs.
  • = Inhibitory input (0 or 1).
  • = Threshold.

NOT gate with MCP

The NOT gate has one variable binary input , which acts as an inhibitory input, and one constant excitatory input of .

Choose:

The MCP neuron fires only when the excitatory sum reaches the threshold and no inhibitory input is active.

Since and :

  • : Inhibition inactive, .
  • : Inhibition active, neuron cannot fire .

The McCulloch-Pitts (MCP) neuron is a unit of computation, and you can use combinations of it to create any type of Boolean logic. But it could not learn.

  • It could perform logical operations but its threshold had to be manually chosen.

Perceptron

Frank Rosenblatt invented the perceptron in late 1950’s as a brain inspired algorithm that could learn about patterns in data by looking at them.

Rosenblatt’s perceptron could learn its weights from data. The weights encoded some knowledge about patterns in the data

Perceptron will always find the pattern hidden in the data in a finite amount of time, it will converge upon a solution without fail if the data is linearly separable.