ChatGPT may have a future use in glaucoma
Key Takeaways
- Large language models (LLMs) such as ChatGPT show great promise in the realm of glaucoma with additional capabilities of self-correction.
- However, the application of LLMs in glaucoma is still in its infancy, and requires further research and validation.
Large language models (LLMs) such as ChatGPT show great promise in the realm of glaucoma with additional capabilities of self-correction, a recent study published in the journal Frontiers in Medicine found. However, use of the technology in glaucoma is still in its infancy, and further research and validation are needed, according to first author Darren Ngiap Hao Tan, MD, a researcher from the Department of Ophthalmology, National University Hospital, Singapore, Singapore.
Tan and his colleagues wanted to determine if LLMs were useful in medicine. “Most LLMs available for public use are based on a general model and are not trained nor fine-tuned specifically for the medical field, let alone a specialty such as ophthalmology,” they explained.
Therefore, they evaluated the responses of an artificial intelligence chatbot ChatGPT (version GPT-3.5, OpenAI), which is based on a LLM and was trained on a massive dataset of text. The investigators curated 24 clinically relevant questions on 4 categories in glaucoma: diagnosis, treatment, surgeries, and ocular emergencies. An expert grader panel of 3 glaucoma specialists with combined experience of more than 30 years in the field then graded the responses of the LLM to each question. When the responses were poor, the LLM was prompted to self-correct, and the expert panel then re-evaluated the subsequent responses. The main outcome measures were the accuracy, comprehensiveness, and safety of the responses of ChatGPT.
The expert panel concluded that ChatGPT gave appropriate answers to 17 out of 24 questions and gave inappropriate answers to 7 out of 24 questions. According to the authors, “there was one response in particular with gross factual errors with regards to the pathophysiology of disease”. Use of ChatGPT may therefore pose a “potential risk of misinformation,” though in the case found in the study, the misinformation would be unlikely to cause direct patient harm. Tan and colleagues concluded, “LLMs show great promise in the realm of glaucoma with additional capabilities of self-correction, with the caveat that the application of LLMs in glaucoma is still in its infancy and requires further research and validation.”
Edited by Miriam Kaplan, PhD
Sources:
Lynda Charters, Ophthalmology Times, July 26, 2024; see source article
Tan et al, Frontiers in Medicine, July 8, 2024; see https://doi.org/10.3389/fmed.2024.1359073