Doctors, as a whole, are a pretty clever bunch, but they can be resistant to change. The most famous example is probably the 19th-century surgeons who refused to wash their hands when moving from mortuary to labour ward, spreading as-yet-undiscovered microbes and leading to infant deaths. Hungarian physician Ignaz Semmelweis, who collected statistics to make the case that soap and water could save lives, was ridiculed and ostracised.

Today, we live in more enlightened times, and medical practice is generally backed by evidence 鈥� but are we always getting the right evidence to bring about change? For example, there are signs that bringing artificial intelligence into clinical use could also save lives. As we report in 鈥淎I helps radiologists spot breast cancer in real-world tests鈥�, radiologists who chose to use an image-classifying AI to help spot breast cancer picked up an extra case per 1000 people screened. Across healthcare systems, the effect could be big.

Does that mean we should encourage doctors to hang up their scrubs and let the machines take over? Far from it. While large language model AI systems like ChatGPT can ace multiple-choice medical tests, they do less well on conversational diagnoses (see 鈥淎I chatbots fail to diagnose patients by talking with them鈥�). A medic with a good bedside manner and listening ear is still vital, for now.

We should be bolder in testing medical AI systems in real-world settings

Instead, there are two conclusions we can draw from these studies. The first is that we should be careful about using the generic term 鈥渁rtificial intelligence鈥�. Although the two systems we report on share an underlying neural network technology, image classification is a very different task to text generation, and the latter has a much higher risk of the AI spitting out plausible but incorrect results. In other words, not all AIs are made equal.

The second conclusion is that we should be bolder in testing medical AI systems in real-world settings, rather than just in the lab or simulations. The breast cancer study, by giving radiologists control over when to use AI, shows it can be a useful tool. With a push to get more evidence like this, lives could be saved, just as after Semmelweis, who is now considered a medical hero.

Advanced artificial intelligence models score well on professional medical exams but still flunk one of the most crucial physician tasks: talking with patients to gather relevant medical information and deliver an accurate diagnosis.

鈥淲hile large language models show impressive results on multiple-choice tests, their accuracy drops significantly in dynamic conversations,鈥� says at Harvard University. 鈥淭he models particularly struggle with open-ended diagnostic reasoning.鈥�

That became evident when researchers developed a method for evaluating a clinical AI model鈥檚 reasoning capabilities based on simulated doctor-patient conversations. The 鈥減atients鈥� were based on 2000 medical cases primarily drawn from professional US medical board exams.

鈥淪imulating patient interactions enables the evaluation of medical history-taking skills, a critical component of clinical practice that cannot be assessed using case vignettes,鈥� says , also at Harvard University. The new evaluation benchmark, called CRAFT-MD, also 鈥渕irrors real-life scenarios, where patients may not know which details are crucial to share and may only disclose important information when prompted by specific questions鈥�, she says.

The CRAFT-MD benchmark itself relies on AI. OpenAI鈥檚 GPT-4 model played the role of a 鈥減atient AI鈥� in conversation with the 鈥渃linical AI鈥� being tested. GPT-4 also helped grade the results by comparing the clinical AI鈥檚 diagnosis with the correct answer for each case. Human medical experts double-checked these evaluations. They also reviewed the conversations to check the patient AI鈥檚 accuracy and see if the clinical AI managed to gather the relevant medical information.

Multiple experiments showed that four leading large language models 鈥� OpenAI鈥檚 GPT-3.5 and GPT-4 models, Meta鈥檚 Llama-2-7b model and Mistral AI鈥檚 Mistral-v2-7b model 鈥� performed considerably worse on the conversation-based benchmark than they did when making diagnoses based on written summaries of the cases. OpenAI, Meta and Mistral AI did not respond to requests for comment.

Free newsletter

Sign up to The Weekly

Sign up to newsletter

For example, GPT-4鈥檚 diagnostic accuracy was an impressive 82 per cent when it was presented with structured case summaries and allowed to select the diagnosis from a multiple-choice list of answers, falling to just under 49 per cent when it did not have the multiple-choice options. When it had to make diagnoses from simulated patient conversations, however, its accuracy dropped to just 26 per cent.

Are you truly healthy? These new tests provide the ultimate check-up

Conventional measures like blood pressure and body mass index only tell you so much. Testing your microbiome and metabolites, or even discovering your 鈥渋mmune grade鈥�, can offer a clearer picture of your health

And GPT-4 was the best-performing AI model tested in the study, with GPT-3.5 often coming in second, the Mistral AI model sometimes coming in second or third and Meta鈥檚 Llama model generally scoring lowest.

The AI models also failed to gather complete medical histories a significant proportion of the time, with leading model GPT-4 only doing so in 71 per cent of simulated patient conversations. Even when the AI models did gather a patient鈥檚 relevant medical history, they did not always produce the correct diagnoses.

Such simulated patient conversations represent a 鈥渇ar more useful鈥� way to evaluate AI clinical reasoning capabilities than medical exams, says at the Scripps Research Translational Institute in California.

If an AI model eventually passes this benchmark, consistently making accurate diagnoses based on simulated patient conversations, this would not necessarily make it superior to human physicians, says Rajpurkar. He points out that medical practice in the real world is 鈥渕essier鈥� than in simulations. It involves managing multiple patients, coordinating with healthcare teams, performing physical exams and understanding 鈥渃omplex social and systemic factors鈥� in local healthcare situations.

鈥淪trong performance on our benchmark would suggest AI could be a powerful tool for supporting clinical work 鈥� but not necessarily a replacement for the holistic judgement of experienced physicians,鈥� says Rajpurkar.