Artificial Intelligence Gets 70 Percent of Medical Answers Wrong

A huge waste of people's health, those who think they can heal themselves through chatbot responses. This is the result of a thorough study by Humanitas University in Rozzano.

di |
Artificial Intelligence Gets 70 Percent of Medical Answers Wrong

Artificial intelligence can cause serious harm if we rely on chatbots to guide us in diagnosing and treating any disease, especially the most delicate ones: in 70 percent of cases, its answers to medical questions are incorrect, while a third of bibliographical references are inaccurate or invented. These data, surprising and disturbing, and which must be interpreted carefully to avoid wasting our own health and that of others, are the result of a scientific study coordinated by Vincenzo Guastafierro, at the Humanitas Clinical Institute and Humanitas University in Rozzano, in the province of Milan.

Therefore, the errors linked to an uncontrolled and passive use of artificial intelligence are even more serious than those triggered by the bad habit of having all the answers (from diagnosis to treatment) for any disease, through the word of the doctor Google.

The search forHumanitas University, published on European Journal of Pathology, analyzed the reliability of generative artificial intelligence models as a diagnostic aid. The researchers administered 200 clinical questions in five simulated scenarios, calibrated according to current diagnostic guidelines and validated by expert pathologists. ChatGPT provided useful answers 62% of the time and actually correct answers only 32%. Of the 214 references produced, 18% were completely fabricated..

There are two additional things to note that emerge from this research. The first: Some conditions for which the answers provided by AI chatbots are completely incorrect are very serious and delicate, such as skin cancer and breast cancer. The second consideration: according to a 2024 survey conducted by KFF Health Misinformation Tracking Poll, about 1 in 6 adults He claims to use AI-powered medical chatbots at least once a month to search for "health information or advice." And so the risks of wasting it become very high and widespread.

Read also

Want to see a selection of our news?

Don't Waste