Abstract
Background There are many myths regarding Alzheimer’s disease (AD) that have been circulated on the Internet, each exhibiting varying degrees of accuracy, inaccuracy, and misinformation. Large language models such as ChatGPT, may be a useful tool to help assess these myths for veracity and inaccuracy. However, they can induce misinformation as well. The objective of this study is to assess ChatGPT’s ability to identify and address AD myths with reliable information.
Methods We conducted a cross-sectional study of clinicians’ evaluation of ChatGPT (GPT 4.0)’s responses to 20 selected AD myths. We prompted ChatGPT to express its opinion on each myth and then requested it to rephrase its explanation using a simplified language that could be more readily understood by individuals with a middle school education. We implemented a survey using Redcap to determine the degree to which clinicians agreed with the accuracy of each ChatGPT’s explanation and the degree to which the simplified rewriting was readable and retained the message of the original. We also collected their explanation on any disagreement with ChatGPT’s responses. We used five Likert-type scale with a score ranging from -2 to 2 to quantify clinicians’ agreement in each aspect of the evaluation.
Results The clinicians (n=11) were generally satisfied with ChatGPT’s explanations, with a mean (SD) score of 1.0(±0.3) across the 20 myths. While ChatGPT correctly identified that all the 20 myths were inaccurate, some clinicians disagreed with its explanations on 7 of the myths.
Overall, 9 of the 11 professionals either agreed or strongly agreed that ChatGPT has the potential to provide meaningful explanations of certain myths.
Conclusions The majority of surveyed healthcare professionals acknowledged the potential value of ChatGPT in mitigating AD misinformation. However, the need for more refined and detailed explanations of the disease’s mechanisms and treatments was highlighted.
Impact Statement There are many statements regarding Alzheimer’s disease (AD) diagnosis, management, and treatment circulating on the Internet, each exhibiting varying degrees of accuracy, inaccuracy, and misinformation. Large language models are a popular topic currently, and many patients and caregivers may turn to LLMs such as ChatGPT to learn more about the disease. This study aims to assess ChatGPT’s ability to identify and address AD myths with reliable information. We certify that this work is novel.
Key Points
- Geriatricians acknowledged the potential value of ChatGPT in mitigating misinformation in Alzheimer’s Disease
- There remain nuanced cases where ChatGPT explanations are not as refined or appropriate.
- Why does this matter? Large language models such as ChatGPT are very popular nowadays and patients and caregivers often may use them to learn about their disease. The paper seeks to determine whether ChatGPT does an appropriate job in moderating understanding of Alzheimer’s Disease myths.
Competing Interest Statement
The authors have declared no competing interest.
Funding Statement
Research reported in this paper was supported by the National Institutes of Health under award number U54HG012510.
Author Declarations
I confirm all relevant ethical guidelines have been followed, and any necessary IRB and/or ethics committee approvals have been obtained.
Yes
The details of the IRB/oversight body that provided approval or exemption for the research described are given below:
IRB of Vanderbilt University gave ethical approval for this work.
I confirm that all necessary patient/participant consent has been obtained and the appropriate institutional forms have been archived, and that any patient/participant/sample identifiers included were not known to anyone (e.g., hospital staff, patients or participants themselves) outside the research group so cannot be used to identify individuals.
Yes
I understand that all clinical trials and any other prospective interventional studies must be registered with an ICMJE-approved registry, such as ClinicalTrials.gov. I confirm that any such study reported in the manuscript has been registered and the trial registration ID is provided (note: if posting a prospective study registered retrospectively, please provide a statement in the trial ID field explaining why the study was not registered in advance).
Yes
I have followed all appropriate research reporting guidelines, such as any relevant EQUATOR Network research reporting checklist(s) and other pertinent material, if applicable.
Yes
Footnotes
Funding Disclosure: Research reported in this paper was supported by the National Institutes of Health under award number U54HG012510.
Dr. Beiting is supported by the Health Resources and Services Administration (HRSA) of the U.S. Department of Health and Human Services (HHS) under grant number K01HP49070. This information or content and conclusions are those of the author and should not be construed as the official position or policy of, nor should any endorsements be inferred by HRSA, HHS or the U.S. Government.
Data Availability
All data produced in the present study are available upon reasonable request to the authors.
