Artificial intelligence is becoming ever more powerful. From deciding who gets hired, to locking you in a social media bubble and picking your next romantic partner, it is changing society in countless ways. As machines become ever-smarter, who will ensure they and their creators behave themselves? Iyad Rahwan鈥檚 Scalable Cooperation group at MIT Media Lab is taking on the challenge.
Why do we need to keep tabs on algorithms?
Advertisement
We have a new kind of agent in the world. It鈥檚 not an animal or a human, it鈥檚 a machine. Today, only computer scientists explore the behaviours of algorithms. My team is studying machine behaviour as we would a human, animal or corporation 鈥� in the wild.
Are you saying algorithms behave like animals?
Animals are much more complicated, yet we have a long history of studying their behaviour. It鈥檚 the same for human psychology: we are far from understanding the brain, but we know a lot about human behaviour. Even if we could see behind corporate walls and access algorithmic source codes, we wouldn鈥檛 understand their impact on society. The crucial thing is that algorithms act in the world, so we should look at their behaviour in the world.
Algorithms already direct many aspects of our lives. Aren鈥檛 we a little late to the party?
I鈥檓 not as pessimistic as that, but we do need to catch up because we are frequently taken by surprise by the consequences of algorithmic technologies. A good example is Facebook鈥檚 newsfeed. It鈥檚 just a ranking algorithm that serves news, and posts from your friends. People thought it would usher in an era of democratic sophistication, connecting people with views outside their own, but what happened is exactly the opposite.
There will be more unpleasant surprises, and my role is to try to shed light on these.
You recently created a psychopathic AI. Why?
The AI we created, dubbed Norman, sees and horror in every image it looks at. It is far from what a human psychopath might be like, but we launched it to popularise an important notion in machine learning: that if an algorithm exhibits undesirable bias, the culprit is often the data used to train it. Norman was an evocative way to explain that point to a layperson.
What is appropriate behaviour for an AI system?
Part of my work on machine ethics explores what people think is appropriate. What do humans expect a machine to do, and how do humans evaluate mistakes and moral violations by machines?
Do people perceive machine mistakes differently to human ones?
They do. We take machines to be systematic, so when an algorithm makes a mistake, we distrust it more than a similar mistake by a human. But, crucially, when there is shared control between a human and a machine and a mishap occurs, people throw more blame on the human. I have found this by looking at autonomous vehicle accidents.
Such as the self-driving Uber car that hit a woman crossing the highway at night?
Yes. With the Uber accident, people were very quick to blame the human in the driver seat.
Is there such a thing as optimal ethical behaviour for a self-driving car?
Consider Isaac Asimov鈥檚 laws of robotics, designed to prevent robots harming humans. Thinking about AI has been dominated by the idea that once we find perfect rules for ethical conduct, a machine can derive the correct behaviour in any given situation. That鈥檚 a misguided approach. My Moral Machine project has quizzed more than 4 million people on a wide range of road-accident scenarios, and that work suggests there can be no universal rules for automated vehicles.
So what do you propose?
I鈥檓 pushing for a negotiated social-contract approach. As a society we want to get along well, but to do it we need property rights, free speech, protection from violence and so on. We need to think about machine ethics in the same way. People are happy, in the abstract, to endorse a car that might sacrifice its driver to save multiple pedestrians, but they certainly don鈥檛 want to buy that car. If you leave it up to consumers, they will buy cars that prioritise them at all cost. And that鈥檚 what an 鈥渆very AI for itself鈥� approach could bring about: AI that caters only for the preferences of consumers who can afford them, rather than to the public good.
Are you confident that AI won鈥檛 outpace society鈥檚 ability to control it?
It鈥檚 very tempting to think of AI as this monolithic entity that will somehow, suddenly, get too powerful and become unstoppable, like Skynet in the Terminator movies. That鈥檚 a very unlikely scenario. More likely is what we have today: different groups who struggle for financial success and power, and complex social processes of competition and regulation. I think they will evolve to the next level and be AI-augmented, but they won鈥檛 fundamentally change.
We want AI to invent new drugs and power social media, but we also want to protect the weak and make sure AI, and those who control it, don鈥檛 amass too much power. That鈥檚 a political process and we need to acknowledge that.
What worries you most about where this technology is headed?
Technology can be a force for good, but also a force for evil. This goes all the way back to the invention of fire. I worry about its politicisation. Cultural wars that are happening today about how we run society, whether we should have more progressive values or more conservative values, and how you negotiate those things鈥� I worry that these fights will slip to algorithms.
鈥淵ou would never buy a car that sacrificed the driver to save multiple pedestrians鈥�
What are the risks of an algorithmic culture war?
A politician eventually moves on, retires or you can embarrass them into changing tack. But algorithms that, say, allocate resources and influence hiring decisions are more entrenched and hidden. That becomes dangerous because government systems may get locked into a particular set of values and become unchangeable as society evolves.
So when it comes to societies, one size should not fit all, so to speak?
That鈥檚 right. I was born in Syria, but I lived for many years in the United Arab Emirates, Australia and the US, and I鈥檝e seen various ways of running society. These experiences have always coloured my reaction to this tendency of analytically minded people to pursue universal goals of ethics for AI. I鈥檓 very suspicious of that sort of thinking.
But still, which way is best?
(Laughs) It鈥檚 impossible to say.
This article appeared in print under the headline 鈥淭reat AI like a wild animal鈥�
