Overlord or underachiever: AI poised to disrupt eye care

Key Takeaways

• Two recent studies demonstrated the ability of artificial intelligence (AI) to match ophthalmologists’ answers to patients’ questions about eye disease.
• Chatbot AI is poised to assist eye doctors to manage patient workflow and overcome shortages in the eye care workforce.  

Ophthalmologists, like most other medical specialists, might be looking warily over their shoulders as AI continues to evolve. But should they be concerned, or is it time to embrace the new technology? Attendees at the American Glaucoma Society meeting in Huntington Beach, CA, on March 2, 2024 were told that two recent studies have demonstrated the ability of AI to match ophthalmologists’ answers to patients’ questions about eye disease. 

A study at the Icahn School of Medicine at Mount Sinai in New York City found chatbots matched the proficiency of fellowship-trained ophthalmologists in diagnostic accuracy and completeness in handling questions about eye disease and real patient cases. For the study, 15 clinicians reviewed answers to patient questions by fellowship-trained glaucoma and retina specialists and those generated by ChatGPT-4, the chatbot model released by OpenAI in the spring of 2023. “The specialists themselves didn’t rate their answers as good as they rated the chatbot answers,” said Robert Chang, MD, a glaucoma specialist and associate professor of ophthalmology at Stanford University California who was not involved in the study. “So it’s really showing that what we can come up with generative AI (is) so human-like that it’s difficult for us to tell the difference on accuracy and completeness.” 

Another study, coauthored by Chang, found a similar result by handling 200 eye care questions from an online chat forum. The study used questions patients submitted to an online medical forum that received responses from ophthalmologists, then presented those responses plus answers generated by ChatGPT to a panel of eight ophthalmologists and asked them to distinguish between the two. He said that the answers produced by ChatGPT were so realistic that specialist physicians had difficulty telling the difference between human- and machine-produced responses. The study also found that the likelihood of chatbot answers containing incorrect or inappropriate material was less than 1%. 

Chang noted that chatbots still have some kinks to work out with factual errors, difficulty referencing reliable sources, and hallucinations — fabricated material that may not be accurate. “We’re not quite there yet, but it’s getting close,” Chang added. 

His clinic at Stanford is testing an AI platform to perform virtual scribing of patient encounters in real time. “That can save a lot of time on documentation,” he said. “I think this is a direction moving forward to increase our productivity because as we know, we all have workforce problems whether it’s the doctors or having enough technicians.” Chatbots also are being tested for scheduling appointments and generating letters. “Because of the unique needs of each ophthalmologist, AI agents that augment the existing workforce on specific administrative tasks will be the most likely early use case rather than autonomous disease screening or clinical decision support tools, which will take longer to validate prospectively in specific cohorts,” he said. 

Edited by Dawn Wilcox, BSN, RN

Source:

Richard Mark Kirkner, Medscape Medical News, April 01, 2024; see source article