Did artificial intelligence begin to plateau in 2024, following a boom year in 2023? It depends on who you ask. Some say AI models that were released this year and can apparently reason more effectively than their predecessors show that the lofty goal of artificial general intelligence (AGI) is still on track, but not everyone is convinced.
Certainly, tech firms have continued to talk up the hype. When it launched GPT-4 in 2023, OpenAI boasted that the model had 鈥渉uman-level鈥� performance on professional tests in fields like law and medicine, while researchers at Microsoft (a significant investor in OpenAI) claimed it showed 鈥�sparks鈥� of AGI. The next iteration, GPT-5, would perform like a PhD student, according to OpenAI executives.
However, OpenAI has yet to release GPT-5, while known issues with this type of large language model (LLM) persist, says at the Santa Fe Institute in New Mexico.
Advertisement
鈥淭here鈥檚 still obviously quite a lot of problems with these models,鈥� says Mitchell. 鈥淭hey鈥檙e not very reliable, and they do seem to be brittle in certain ways, meaning that they don鈥檛 perform well on certain things that are sufficiently different from what they鈥檝e seen in their training data.鈥�
While newer models have achieved higher scores on certain measures of chatbot capabilities, these jumps in performance have become increasingly small. In April, Gary Marcus at New York University highlighted this as one of the reasons he believed that LLMs were entering a new 鈥減eriod of diminishing returns鈥�.
However, the release of OpenAI鈥檚 o1 model in September is a 鈥渓arge step鈥� towards more capable models, says at OpenAI. 鈥淭here were legitimate reasons to be concerned about AI plateauing,鈥� he says, such as the increasing cost required to train larger and larger models. But o1 could change this paradigm, he says, because of the way it calculates its answers.
Unlike GPT-4, which is fed data and trained once, often taking vast amounts of computing power for months at a time, o1 instead uses more computing power to answer individual questions, which Brown compares to the way humans take time to carefully think through their answers to problems.
鈥淧revious models would do all their computation upfront, and when it came time to actually query them, they would respond basically instantly,鈥� he says. 鈥淲ith o1, you see this difference where it鈥檚 spending a lot more time thinking about how to respond to the question before actually responding.鈥�
Google DeepMind has also been experimenting with AI that can tackle tasks requiring reasoning, such as its AlphaProof system, which achieved a silver medal score at this year鈥檚 International Mathematical Olympiad, the first time any AI has attained a medal-level performance.
These advances are promising, says Mitchell, but as companies don鈥檛 share full information on how their models work, it is difficult to test the claim that the models are reasoning in the same ways humans do.
鈥淭here鈥檚 been a lot of debate about what we should consider to be reasoning,鈥� says Mitchell. 鈥淏eing able to take what you have learned and generalise it to new situations in a robust way is still something that AI has not really conquered.鈥�
