ChatGPT has passed a tough test to become a doctor in the US. Researchers hailed the AI chatbot’s success as a milestone in medical science
Chatbot ChatGBT succeeded between 52.4 percent and 75 percent in various sections of the three-part US Medical Licensing Examination (USMLE). In order to pass the exam, an average of 60 percent is required.
MILESTONE
“Achieving a passing score for this notoriously difficult specialist exam, and doing so without any human augmentation, marks a remarkable milestone in the development of clinical AI,” researchers from AnsibleHealth, the technology company that conducted the research, said in a statement.
PREVIOUSLY PASSED BUSINESS ADMINISTRATION AND LAW EXAMS
The results of the study were published in PLOS Digital Health, a peer-reviewed scientific journal. ChatGBT has also previously been tested and passed exams in business (at the University of Pennsylvania’s Wharton School of Business) and law (at the University of Minnesota).
In the latest study, researchers tested the software on 350 questions from the June 2022 USMLE.
Two doctors evaluated the results and discrepancies were reviewed by a third expert.
The exam assesses the knowledge of medical students and doctors in training in medical fields and has been in use since 1992.
Medical students in the US usually take the first step of the USMLE at the end of their second year of medical school, the second step is taken in their fourth year, and the third step is taken in their first year of residency after finishing medical school. More than one hundred thousand students and graduate students take the exam each year.
On the other hand, the researchers emphasized that ChatGPT also produced at least one important insight that was “novel and clinically relevant” for 88.9 percent of its responses.
The results also surpassed the performance of PubMedGPT, a different AI robot trained specifically on biomedical literature, which passed 50.8 percent of the time on an exam consisting of USMLE-style questions.
The authors of the study stated that the research findings show that ChatGPT can become a valuable tool in medical education, “The AI bot has a partial ability to teach medicine by revealing new and non-obvious concepts that may not be in the students’ awareness area. Artificial intelligence technology is now positioned to soon become ubiquitous in clinical practice with various applications in all health sectors.”
WHAT IS CHATGBT?
ChatGPT, recently acquired by Microsoft, is a large language model that uses a variant of the GPT (Generative Pre-training Transformer) architecture to generate human-like text. The chatbot is trained on a variety of internet texts and can produce consistent responses to a wide range of commands. The model can be fine-tuned for various tasks such as language translation, question answering and conversation. ChatGBT, created by the research team of OpenAI in the US, continues to be updated based on new research and developments.
WHAT ARE THE FEATURES OF CHATGPT?
- Question and answer
- Writing texts (basic academic papers, literary texts, movie scripts, etc.)
- Solving math equations
- Debugging and correcting (e.g. detecting and correcting errors in a block of code)
- Translation between languages
- Summarizing the text and identifying keywords in the text
- Making recommendations
- Classification
- Explain what something does (e.g. explain what a block of code does)
