ARTIFICIAL intelligence has come a long way. In recent years, smart machines inspired by the human brain have demonstrated superhuman abilities in games like chess and Go, proved uncannily adept at mimicking some of our language skills and mastered protein folding, a task too fiendishly difficult even for us.
But with various other aspects of what we might reasonably call human intelligence – reasoning, understanding causality, applying knowledge flexibly, to name a few – AIs still struggle. They are also woefully inefficient learners, requiring reams of data where humans need only a few examples.
Advertisement
Some researchers think all we need to bridge the chasm is ever larger AIs, while others want to turn back to nature’s blueprint. One path is to double down on efforts to copy the brain, better replicating the intricacies of real brain cells and the ways their activity is choreographed. But the brain is the most complex object in the known universe and it is far from clear how much of its complexity we need to replicate to reproduce its capabilities.
That’s why some believe more abstract ideas about how intelligence works can provide shortcuts. Their claim is that to really accelerate the progress of AI towards something that we can justifiably say thinks like a human, we need to emulate not the brain – but the mind.
“In some sense, they’re just different ways of looking at the same thing, but sometimes it’s profitable to do that,” says . “You don’t want a replica, what you want is to learn the principles that allow the brain to be as effective as it is.”
Whether the mind and the brain can even be thought of as separate is controversial, and neither philosophers nor scientists can pinpoint where one might draw the line. But exactly what point on that spectrum AI researchers should be focused on for inspiration is currently a big debate in the field.
There can be no doubt that the brain has been a handy crib sheet. The artificial neural networks powering today’s leading AIs, such as the impressive language model GPT-3, consist of highly interconnected webs of simple computational units analogous to biological neurons. Like the brain, the behaviour of the network is governed by the strength of its connections, which are adjusted as the AI learns from experience.
“AIs can struggle to apply their skills outside highly specific niches”
This simple principle has proved incredibly powerful and today’s AIs can learn to spot cancer in X-rays, navigate flying drones or produce compelling prose. But they require mountains of data and most struggle to apply their skills outside highly specific niches. They lack the flexible intelligence that allows humans to learn from a single example, adapt experiences to new contexts or use common sense to reason about unfamiliar situations.
One reason might be that the similarities between real brains and AIs are only skin deep. One disparity that has recently come to the fore is in the processing power of artificial neurons. The “point neurons” used in artificial neural networks are a shadow of their biological counterparts, doing little more than totting up inputs to work out what their output should be. “It’s a vast simplification,” says , a computational neuroscientist at the Institute of Molecular Biology and Biotechnology in Greece. “In the brain, an individual neuron is much more complicated.”
There is evidence that neurons in the cortex – the brain region associated with high-level cognitive functions like decision-making, language and memory – carry out complex computations all by themselves. The secret appears to lie in dendrites, the branch-like structures around a neuron that carry signals from other neurons to the cell’s main body. The dendrites are studded with synapses, the contact points between neurons, which pepper them with incoming signals.
We had known for some time that dendrites can modify incoming signals before passing them on. But in a 2020 study, Poirazi and her colleagues at Humboldt University of Berlin found that . Moreover, when a group from the Hebrew University of Jerusalem tried to train an AI to mimic all the computations of a single biological neuron, it required an artificial neural network five to eight layers deep to reproduce all of its complexity.
Could these insights point the way to more powerful, flexible AIs? Both Poirazi’s group and researchers from AI company Numenta published studies in October 2021 suggesting that the properties of dendrites could help tackle one of deep learning’s most debilitating problems – catastrophic forgetting. This is the tendency of artificial neural networks to forget previously learned information when they learn something new. Using more complex artificial neurons seems to get around this by allowing different regions of the network to specialise at different tasks, says Poirazi.
Flexible thinking
“You have smarter, smaller units, so you don’t really need the entire network to learn,” she says. That means previously learned information in other areas of the network doesn’t get overwritten. Poirazi suspects this could also make AIs more flexible. By breaking problems down into smaller chunks that are stored in different parts of the network, it may be easier to recombine them to solve new challenges that an AI hasn’t seen before.
Not everyone is convinced this is the best way forward. When , they saw no performance gains. Richards has a hunch that dendrites are simply evolution’s answer to connecting billions of neurons within the space and energy constraints of the brain, which is less of a concern for AIs running on computers.
For Richards, the key thing we need to tease out is the “loss function” used by specialised circuits in biological brains. In AI, a loss function is the measure that an artificial neural network uses to assess how well it performs on a task during training. Essentially, it is a measure of error. For instance, a language AI might measure how good it is at predicting the next word in a sentence.
If we can determine what a particular brain circuit is striving towards, we could establish a relevant loss function and use this to train a neural network to aim for the same goal, which should, in theory, replicate the brain function. Richards has tentative evidence of how this might work. In June 2021, he and colleagues from McGill University and AI company DeepMind showed that .
By repeating this process for the many specialised networks in the brain, Richards thinks we could piece together the key components that make humans such versatile thinkers. “I suspect it’ll have to be more modular,” he says. “We’ll want something that doesn’t look radically different from the brain in some ways.”
One brain area that could be crucial to advancing AI is the hippocampus, says . Stachenfeld is trying to understand how neurons in this region help the brain organise knowledge in a structured way so it can be reused for new tasks. “It allows us to make analogies with the past, to reuse information in new settings and be very dynamic and flexible and adaptive,” she says.
It is possible to pull more general insights out of neuroscience to advance AI too, says Jeff Clune at the University of British Columbia, Canada, and Californian firm OpenAI. Thinking about catastrophic forgetting, Clune became fascinated by the brain’s neuromodulatory system, in which certain neurons release chemicals that modulate the activity of other neurons, often in distant brain regions.
He and his colleagues realised that the ability to turn learning up or down in separate parts of their artificial neural network could help with continual learning, by allowing different regions to specialise at different tasks. They didn’t try to build a replica of the neuromodulatory system. Instead, they trained one neural network to modulate the activity of another, switching regions of the second network on and off so it could learn a series of tasks without forgetting previous ones. “We weren’t terribly faithful to the biology, we just took the abstract idea,” says Clune. “When you’re trying to do bio-inspired work, you want to get all, and only, the ingredients that will really move the needle.”
But some insist we should take abstraction further, focusing not on replicating the brain’s nuts and bolts, but the higher-level mental processes involved in gaining knowledge and reasoning about the world: a top-down rather than a bottom-up approach. Gary Marcus is the standard-bearer for this perspective.
Despite impressive progress, says Marcus, neuroscience can tell us very little about how the brain achieves higher-level cognitive capabilities. More importantly, the brain is an ad hoc solution, cobbled together by aeons of haphazard evolutionary experiments. “The brain is actually really flawed,” he says. What we want is to emulate what it does, regardless of how it is put together. “In some ways, psychology might be more useful for that.”
Psychology has some clear and well-validated models of the cognitive processes behind intelligence. Take the principle of compositionality, the idea that we understand things in terms of their parts and the relationships between those parts. This underpins reasoning in humans, says Marcus, but has proven difficult to implement in artificial neural networks.
There are ways to implement such principles in machines. The basic idea, known as symbolic AI, was the dominant approach to AI in the second half of the 20th century. It builds on cognitive theories describing how humans think by manipulating symbols. We use the word “dog” to refer to a real-world animal, for instance, and we know that the + sign means add two values together.
“We want to emulate what the brain does. Psychology might be more useful for that”
For engineers, creating symbolic AIs involves generating structured ways to represent real-world concepts and their relationships as well as rules about how a computer can process this information to solve problems. With chess, for instance, engineers encode possible configurations of pieces, the moves each can make and rules about which moves will help win the game.
Chess is one thing. Unpicking all the variables and relationships that govern most real-world problems is a different matter. That is why symbolic AI fell out of favour in the late 1980s, setting the stage for the rise of data-driven deep learning. And yet it turns out that many of symbolic AI’s strengths overlap with the weaknesses we have discovered in deep learning. Now, there is growing interest in combining the two.
Hybrid intelligence
So-called neuro-symbolic systems attempt to retain deep learning’s ability to learn from new experiences, while introducing symbolic AI’s ability to do complex reasoning and draw on pre-existing knowledge. “There must be some way of bringing the insights from these two traditions together,” says Marcus.
One possibility was outlined at a conference in January 2021 by IBM’s . Their proposal builds on the idea outlined by Daniel Kahneman in his best-selling book Thinking, Fast and Slow, which splits the human mind into two broad modes of thought. System 1 is fast, automatic and intuitive, and responsible for rapidly making sense of the world around us. System 2 is slow, analytical and logical, and controls our ability to reason through complex problems.
The group combined this idea with AI pioneer Marvin Minsky’s “society of mind” theory, which postulates that the mind consists of many specialised cognitive processes that interact to create a coherent whole. The result is a conceptual system made up of multiple components specialised for different system 1 and system 2 tasks.
As in the human mind, system 1 agents kick in automatically as soon as the AI is set a task. But an overarching “metacognitive” module then assesses their solutions, and if they don’t work, it pulls in a more deliberative system 2 agent. It doesn’t necessarily matter which technology is used for individual components, says Rossi, but in their early experiments, the system 1 agents are often data-driven, while system 2 agents and the metacognitive module rely on symbolic approaches.
There is considerable resistance to the revival of symbolic approaches. In a recent paper, the three pioneers of deep learning – Geoffrey Hinton, Yoshua Bengio and Yann LeCun – made it clear they think system 2 capabilities should be learned by neural networks, not built by hand.
The argument, says Richards, is that humans aren’t smart enough to build symbol systems that capture the complexity of the real world. The focus therefore should be on working out how to encourage a network to develop in ways that mimic the brain’s development of high-level cognitive abilities. “We are not smart enough to hand-engineer this stuff,” he says. “And you don’t have to be. You can just let the neural network discover the solution.”
We still don’t know how to steer one to do so, though. says a promising approach is to build symbolic models that replicate aspects of human intelligence and then try to replace as many components as possible with data-driven machine learning. “You can take symbolic models that have been really successful and then see what are the minimal, critical symbolic pieces that you need in order to explain its abilities,” he says.
Ultimately, there are probably benefits to both the top-down and bottom-up approaches, says Konrad Kording at the University of Pennsylvania. Studying human behaviour can give us clues about the abstract cognitive processes we need to replicate in thinking machines, he says, while fundamental neuroscience can tell us about the building blocks required to build them efficiently.
But perhaps the biggest contribution either approach can make to AI is cultural, says Kording. AI research today is driven by benchmark challenges and competitions, which promote an incrementalist approach. Most advances are achieved by simply tweaking the previous state-of-the-art model or training it on more data or on ever bigger computers.
Those who study human intelligence bring a different perspective to the field. “They’re driven by a will to understand instead of a will to compete,” says Kording. In the long run, that attitude may prove more valuable than any details about how our brains and minds work.
New Scientist audio
You can now listen to many articles – look for the headphones icon in our app newscientist.com/app
