In 1943, neurophysiologist Warren Mc Culloch and mathematician Walter Pitts wrote a paper on how neurons might work.
In order to describe how neurons in the brain might work, they modeled a simple neural network using electrical circuits.
In the same time period, a paper was written that suggested there could not be an extension from the single layered neural network to a multiple layered neural network.
In addition, many people in the field were using a learning function that was fundamentally flawed because it was not differentiable across the entire line.
As a result, research and funding went drastically down.
This was coupled with the fact that the early successes of some neural networks led to an exaggeration of the potential of neural networks, especially considering the practical technology at the time.The first multilayered network was developed in 1975, an unsupervised network.An artificial neural network is an interconnected group of nodes, inspired by a simplification of neurons in a brain.Here, each circular node represents an artificial neuron and an arrow represents a connection from the output of one artificial neuron to the input of another.Artificial neural networks (ANN) or connectionist systems are computing systems that are inspired by, but not identical to, biological neural networks that constitute animal brains.They do this without any prior knowledge of cats, for example, that they have fur, tails, whiskers and cat-like faces.Instead, they automatically generate identifying characteristics from the examples that they process.There were a few advances in the field, but for the most part research was few and far between.In 1972, Kohonen and Anderson developed a similar network independently of one another, which we will discuss more about later.Such systems "learn" to perform tasks by considering examples, generally without being programmed with task-specific rules.For example, in image recognition, they might learn to identify images that contain cats by analyzing example images that have been manually labeled as "cat" or "no cat" and using the results to identify cats in other images.