Ellen Moons, the chair of the
Nobel Committee for Physics and physicist at Karlstad University lauded the
work of these two laureates which used “fundamental concepts from statistical
physics to design artificial neural networks” that can “find patterns in large
data sets”.
This announcement however stirred debate
across academia. Their argument is: How could research associated with Machine
learning and Artificial Intelligence, which come under Computer Sciences/
mathematics be awarded under Physics?
Another section however argues
that Dr Hopfield, having developed a type of artificial network—Hopfield
network— which behaves like a physical structure called a spin glass, appears
to have offered a tenuous reason to the Academy to reward their research under
Physics.
But Noah Giansiracusa, an
associate maths professor at Bentley University said: “… Even if there's
inspiration from physics, they're not developing a new theory in physics or
solving a longstanding problem in physics." Thus, many opine that though their
work deserves recognition, the lack of a Nobel Prize for mathematics /computer
science has distorted the outcome.
That said, let us move on to
learn how these two laureates have made computers that cannot of course think,
to mimic functions such as memory learning. They created a technology—Artificial
Neural Networks (ANN) and deep learning— that became the intellectual
foundation for AI systems.
ANNs are networks of neurons (or
processing centres) designed to function as a system similar to the human brain
in learning and processing information. The foundations of ANN rest in various
branches of science such as statistical physics, neurobiology, cognitive
psychology, etc.
ANNs are computer programs designed
to act like our brains. They loosely model the way biological networks of nerve
cells called neurons connected by axons are believed to work. The basic units
of the brain called neurons have limited intelligence. Each neuron has a number
of input wires called dendrites through which it receives inputs from other
locations. A neuron also has an output wire called an axon through which it
sends signals to other neurons.
In short, a neuron is a
computational unit that gets a number of inputs through dendrites, does some
computation, and then sends its output via axon to other neurons. Such billions
of densely connected neurons, spread over many layers in the brain have
tremendous processing power and thus can solve complex problems.
Led by this understanding, ANNs
are created with many simple machines—nodes/neurons— distributed over many
layers: an input layer, an output layer, and other hidden layers. These
machines are connected through unidirectional links which can carry electric
current. Each machine/node performs a simple processing based on its inputs. If
the result exceeds a threshold value, it gets activated, just like a neuron in
the brain fires. The activated node transmits an electrical impulse to the next
machine, which may or may not be activated. We thus ultimately get a pattern of
1s and 0s and based on this pattern of 1s and 0s in the input and output
layers, one can train the network to respond in a particular way.
In 1982 Dr Hopefield, a physicist,
who later became a professor of molecular biology at Princeton University, inspired
by associative memory in the brain, wanted to build a network with 100 neurons
but that was beyond the capacity of the then-prevailing computing capabilities.
Hence, he finally settled on a network consisting of 30 neurons and
demonstrated the idea of machine learning— a system by which a computer can
learn from examples of data instead of learning from a programmer.
Hopfield’s network consists of interconnected
neurons or nodes. It is a single-layered and recurrent network. It has binary
threshold nodes, with the states +1 and -1 (or 1 and 0) respectively. Each
neuron stands for a component of memory. Each neuron is connected to every
other neuron except diagonally.
The network has an energy
function. It has the ability to transform itself through transitions to
different states until it stabilizes. When the network reaches a stable state, which
corresponds to a stored pattern, its energy is decreased. This is the key to
associative memory. It thus retrieves a stored pattern even when presented with
incomplete or noisy versions of the patterns.
It is the simplest mathematical
model with built-in feedback loops. Hence, the Hopfield network is supposed to work
similarly to our brain. But Hopfield’s network by virtue of having a limited
number of neurons, had very limited memory storage.
Using the Hopfield network as the
foundation, Geoffrey Hinton— often referred to as the “Godfather of AI”—
came up with a new network using a different method: the Boltzman machine.
His network consists of nodes (neurons) organized in layers: an input layer,
hidden layers, and an output layer. Each neuron receives input data, processes
it, and passes it on to the next layer.
Relying on the energy dynamics of
statistical physics, Hinton showed that his generative computer model could
learn to store data over time by training it using examples of things that it
should remember. It can thus be used to classify images or create new examples
of the patterns on which it was trained.
Hinton—along with psychologist David
Rumelhart and computer scientist Ronald J. Williams—also developed the
backpropagation algorithm. It helps in minimizing errors in the output. It is
this pioneering research of Hinton that worked as the “fundamental to give
‘depth’ to deep learning.
The two Laureates’ research has thus
transformed all areas of AI, from image processing and data science to natural
language processing, with advances that are already impacting our daily lives.
Indeed, the rapid progress in AI and its applications is causing anxiety,
anxiety even to its grandfather, Hinton. But the better way to handle this
anxiety is for businesses to propel forward.
**
No comments:
Post a Comment