Study Discovers ChatGPT Inadequate for Recommending Cancer Treatment

Cancer treatment regimens created by artificial intelligence tools may seem promising and alluring, but they are currently plagued with errors and are unlikely to be available for several more years.

Computer-designed cancer treatment plans created by OpenAI Inc.’s ChatGPT suffered from a familiar problem, Brigham and Women’s Hospital researchers found: According to a published study, inappropriate treatment recommendations were mixed up with correct ones, making them particularly difficult to distinguish. Thursday in JAMA Oncology.

ChatGPT “often speaks in a very confident way that seems to make sense, and the way it can mix incorrect and correct information is potentially dangerous,” said Danielle Bitterman, an oncologist and co-author of the study in Mass’s Artificial Intelligence Program. Brigham General Health Care System. “Even an expert has a hard time identifying what is a false recommendation.”

Video: Is artificial intelligence good for health?

While nearly all ChatGPT responses included at least one recommendation in accordance with National Comprehensive Cancer Network guidelines, the researchers found that about a third also included incorrect suggestions. About 12 percent of the responses were “hallucinations” — recommendations that are not in any of the guidelines, the researchers said.

Although generative AI tools may not be accurate enough to create cancer treatment plans, the techniques are much more widely discussed because they can help detect cancer at an early stage, when it is more likely to be treated successfully. OpenAI has highlighted that ChatGPT can be unreliable, generates information and requires “great care”, especially in “high-stakes contexts”.

Clinicians are hopeful that AI can help ease their administrative burdens, Bitterman said, but concerns about accuracy and privacy mean big language models like ChatGPT are years away from widespread adoption in doctors’ offices and hospitals.

“It is impressive that in almost all cases it included a guideline requirement recommendation,” he said. Given its broad knowledge base, it’s almost like ChatGPT went to medical school, he said, “but it didn’t go to residency,” the advanced clinical training that doctors receive after graduation.