Artificial neural networks have become the cornerstone of modern artificial intelligence, which is now touching all of our lives. Work on such networks, figuring out what they are capable of doing and setting the foundations for what we see today, started many decades ago.

One of the key steps was the development of a type of network known as a Hopfield neural network. These networks are named after physicist John Hopfield, who laid out their basics in two landmark papers, written in 1982 and 1984 when whe was at the California Institute of Technology and at Bell Labs in New Jersey.

Hopfield was working on the question of how many individually simple units (each of the “neurons” in an artificial neural network) could come to act collectively and achieve complex computations. For example, if many individual ants can coordinate with each other to build an anthill, can we get many individual neurons to work together to recollect a memory?

A key step in Hopfield’s thinking was realizing that under certain conditions on the patterns of connections between the neurons, the entire system could be descibed as having an overall “energy”, and that the system, as a whole, will settle into low “energy” states. (The quote marks indicate that it isn’t physical energy, but an analog to it.) We can think of that settling as what the whole set of neurons will do collectively. Hopfield moreover showed how many interesting and difficult computations could be mapped onto settling into low energy states. For example, imagine there’s a name you’re struggling to remember and someone reminds you of the first letter. That hint can often be enough to trigger recall of the full name. Such full memory recall from partial information can be very quickly and efficiently accomplished by Hopfield networks.

The “energy” formulation helped many researchers link artificial neural networks to physics, unleashed a wave of further developments and papers, and became one of the key founding concepts that led to modern artifical neural networks. For his contributions, John Hopfield was awarded the 2024 Nobel Prize in Physics (together with Geoffrey Hinton, another artificial neural network pioneer).