ChatGPT scored higher than human candidates in a mock obstetrics and gynecology test. The artificial intelligence chatbot even outperformed human counterparts in areas of empathy, communication and exhibiting specialist knowledge in the new study. But the artificial intelligence tool can’t fully replace the value of a human physician — it says so itself.
“While I am a language model that has been trained on a vast amount of knowledge, I am not a licensed medical professional and I am not capable of providing medical diagnoses, treatments or advice,” the chatbot wrote in response to a question from CNN.
ChatGPT was developed by OpenAI to interact in a conversational way by answering questions, following instructions, and providing detailed responses And it has performed pretty darn well in medical studies, such as the latest one from Singapore.
According to the New York Post, the study conducted at the National University of Singapore and published in the American Journal of Obstetrics & Gynecology, found that ChatGPT scored an average of 77.2% on the ob-gyn specialist exam, while human candidates scored an average of 73.7%. ChatGPT took an average of under three minutes to complete each test station, which was well under the 10-minute time limit allotted.
The stations consisted of clinical scenarios that required formulating a care plan by the candidates who had to grasp a variety of issues regarding labor and pregnancy, gynecologic oncology, and postoperative care of the patient. The examiners were surprised not only by the fact that ChatGPT scored higher than humans, but it also scored very well in empathic communication and had answers to complex questions quickly.
One downside of ChatGPT was that it lacked “local ethnic knowledge,” said the study authors, and had difficulty navigating complex language dialects.
“The arrival and increased use of ChatGPT has proven that it can be a viable resource in guiding medical education, possibly provide adjunct support for clinical care in real time, and even support the monitoring of medical treatment in patients,” said Dr. Mahesh Choolani, associate professor of obstetrics and gynecology at National University of Singapore, in a news release. Choolani, who ran the study, added that the bot was less able to handle complex scenarios, however, and lacked up-to-date medical references and data.
“In an era where accurate knowledge and information is instantly accessible, and these capabilities could be imbedded within appropriate context by generative AI in the foreseeable future, the need for future generations of medical doctors to clearly demonstrate the value and important of human touch is now saliently obvious. As doctors and medical educators, we need to strongly emphasize and exemplify the use of soft skills, compassionate communication and knowledge application in medical training and clinical care,” he said.