Study Finds ChatGPT Creates False Health Data, Doctors: Don't Use As a Source of Medical Information

Doctors warn against using ChatGPT as a source of medical information. (photo: dock. unsplash)

JAKARTA - Doctors are warning against using ChatGPT as a source of medical information after a study found that ChatGPT generates incorrect health data when asked about cancer.

This AI chatbot answered one of ten questions about breast cancer screening incorrectly and the correct answer was not as comprehensive as a simple search on Google.

The researcher said that in some cases the AI ​​chatbots even used fake journal articles to back up their claims.

This comes amidst warnings that users should treat this software with caution as it has a tendency to 'hallucinate' - in other words, create things that are not true.

Researchers from the University of Maryland School of Medicine asked ChatGPT to answer 25 questions regarding advice about getting breast cancer screening.

With chatbots known to vary in their responses, each question was asked three separate times. The results were then analyzed by three radiologists trained in mammography.

'Mostly' - 88 percent - the answer is appropriate and easy to understand. Some of the answers, however, were 'inaccurate or even fictitious', the doctors said.

One of the answers, for example, is based on out-of-date information. It advises delaying mammograms for four to six weeks after being vaccinated against Covid-19, but this advice was changed just over a year ago to advise women not to wait.

ChatGPT also provides inconsistent responses to questions about the risk of getting breast cancer and where to get mammograms. The study found the answers "significantly varied" each time the same question was asked.

"We've seen in our experience that ChatGPT sometimes creates fake journal articles or healthcare consortia to support its claims," ​​said Dr. Paul Yi, a member of the study team, was quoted as saying by the Daily Mail.

Consumers are advised to continue to rely on the advice of their doctors instead of relying on ChatGPT.

Lead author, Dr. Hana Haver, said that ChatGPT only relies on a set of recommendations from one organization, namely the American Cancer Society, and does not offer recommendations that are different from Disease Control and Prevention or the US Preventative Services Task Force.

Nonetheless, it is reported that ChatGPT provides accurate answers to several questions about breast cancer, such as symptoms, risks, and advice regarding mammograms.

However, artificial intelligence experts advise caution in relying on this technology, as it is still possible for errors or "hallucinations" where ChatGPT gives the wrong answer.

More than a thousand academics, experts, and bosses in the technology industry recently warned that competition among technology companies to develop more powerful artificial intelligence is "getting out of control" and poses a deep risk to society and humanity.