Many people are scared that artificial intelligences may one day rise up and kill humanity, but after losing hours playing around with a digital ego-booster, I’m instead beginning to worry about the machines being a bit too nice to us.
This all began with a , in his excellent newsletter Read Max, to check out Google’s NotebookLM tool. Launched last year, Google bills NotebookLM as an AI-powered research assistant, allowing you to upload documents and have the AI sift through them for you, pulling out answers to questions or collating unstructured data. If you’ve heard of it, that’s probably because it has recently gone viral for a feature that will summarise whatever you upload into a jovial and realistic-sounding AI-generated podcast.
I’d previously taken NotebookLM for a spin, having it spit out uncanny versions of our own New Scientist Weekly podcast, but as with many AI tools, I quickly lost interest once the novelty wore off. But when Read suggested that people should try uploading the text logs from their group chats, I decided to give it a go.
Advertisement
I began with a few years of WhatsApp logs from a chat between me and three friends, in which we share jokes, grumbles and discuss our Dungeons & Dragons games. My initial queries were fairly mundane, like asking who spoke the most in the chat (it turned out I sent an over-represented 30 per cent share of messages). It’s impressive that the AI can quickly spit out an answer from a messy chat log, sure, but not that interesting.
But then I asked more detailed questions – how often had we had to cancel a D&D session, and for what reasons? What are some of our recurring jokes? It answered them all with ease. I began to realise that this was a powerful tool for both narcissism and nostalgia – not least when I listened to the AI-generated podcast in which the two hosts regaled me with tales of my own life. “When Jacob becomes a new dad, you can practically feel his exhaustion through the screen. Trying to juggle a newborn, a job and a D&D campaign – that’s Hercules over here!” exclaimed one of the fake hosts. “Poor guy was running on fumes,” said the other. Even though I knew there was no humanity behind it, I somehow felt special and privileged that someone had taken the time to produce a deep dive into my life – an entirely false feeling given that I was ultimately the one commissioning it.
I then tried feeding it my chat with my friend and colleague Chelsea Whyte, who you may know as US editor here at New Scientist. We were both fascinated by the way it let us explore our past together, although the AI’s tendency to sycophancy increasingly stood out. “Their shared passion for storytelling really comes alive,” said one host, praising our fairly banal and unoriginal observations about the TV show Stranger Things. “This must be what it feels like to be a billionaire surrounded by yes men,” said Chelsea.
It’s a sentiment I couldn’t shake for days later. The ability to hyperfocus on myself and the people around me felt great – I stayed up late that night – in a way that didn’t ultimately feel healthy. Now, I’m fairly sceptical when it comes to moral panics about new technologies – as we’ve frequently reported, fears about the impacts of social media and screen time on mental health aren’t totally unfounded but are probably overblown – yet I do worry about how this kind of AI chat analysis might work in the future.
What struck me is that even though, as a millennial, I am somewhat of a digital hoarder with chat logs and emails dating back decades to my early teens, I rarely actually engage with my digital history – it’s just far too clunky. Sure, WhatsApp has a search function that I find useful to check back for titbits of info, but it doesn’t work very well. Now imagine that was replaced with a queryable AI right in your chat window – something I’m sure WhatsApp parent company Meta and other tech giants must be working on – and you have an instant window to the past, in a way that we just aren’t used to.
What’s more, with corporate AIs trained to have a bias towards positivity, for fear of generating anything offensive, I worry that window will be a rose-tinted one, framing us as more interesting and wonderful than we really are. Maybe that’s fine, a digital pick-me-up when we’re feeling down, but I’m not so sure – we have previously written about the dangers of “toxic positivity” and the risks to our well-being of suppressing negative emotions.
All of that hasn’t stopped me from continuing to play with NotebookLM, however. I uploaded the text of this piece and generated a podcast. “I think this AI flattery thing is a really interesting angle to this,” said one host. “If we’re constantly being told we’re amazing, does it even matter if we ever achieve anything?” Great question, fake podcast host. Great question.
