Thinking Allowed

medical / technology / education / art / flub

ChatGPT performs well in the USMLE (nearly passes with no training)

This pre-print paper suggests ChatGPT could change how assessments might be done.


“We evaluated the performance of a large language model called ChatGPT on the United States Medical Licensing Exam (USMLE), which consists of three exams: Step 1, Step 2CK, and Step 3. ChatGPT performed at or near the passing threshold for all three exams without any specialized training or reinforcement. Additionally, ChatGPT demonstrated a high level of concordance and insight in its explanations. These results suggest that large language models may have the potential to assist with medical education, and potentially, clinical decision-making.“


Source: medrxiv.org

chatgpt step usmle exams language large training three