The rapid ascension of generative artificial intelligence has sparked a global conversation about the future of professional expertise, particularly within the high-stakes corridors of modern medicine. As platforms like ChatGPT and Gemini become household names, the medical community has begun to grapple with a fundamental question: can these general-purpose "large language models" (LLMs) outperform the specialized, expert-driven systems that have quietly assisted clinicians for decades? New research conducted by computer scientists and medical professionals at Massachusetts General Hospital (MGH) suggests that while the new frontier of AI is impressive, the specialized "expert systems" of the past still maintain a critical advantage in diagnostic accuracy.
For those outside the medical informatics field, it may come as a surprise that artificial intelligence is not a newcomer to the hospital setting. Medical professionals have been utilizing various forms of AI to streamline and verify diagnoses for nearly forty years. These tools, known as diagnostic decision support systems (DDSSs), were designed specifically to act as a digital safety net for clinicians. Among the most venerable of these systems is DXplain, a platform first developed in 1984 by the Laboratory of Computer Science (LCS) at MGH. As a founding member of the Mass General Brigham healthcare system, MGH has long been at the forefront of integrating technology into patient care, and DXplain represents one of the earliest and most enduring examples of this mission.
DXplain was built upon a foundation of structured medical knowledge, relying on a massive database that includes thousands of disease profiles, clinical findings, and specific data points. Its primary function is to take a set of clinical observations and generate a ranked list of potential diagnoses, providing clinicians with a comprehensive "differential diagnosis" that helps ensure no rare or easily overlooked condition is ignored. Over the past four decades, the system has evolved, absorbing new medical research and refining its algorithms to remain a relevant tool in the modern clinical environment. However, the sudden explosion of generative AI and LLMs—which are trained on vast swaths of the internet rather than curated medical databases—prompted the investigators at MGH’s LCS to see how their legacy system measured up against the modern giants of Silicon Valley.
The study, recently published in the journal JAMA Network Open, provides a direct comparison between the veteran DXplain system and two of the most prominent LLMs available today: OpenAI’s ChatGPT and Google’s Gemini. The goal was to determine which approach—specialized medical logic or generalized linguistic intelligence—offered the most reliable support for physicians during the diagnostic process. The findings revealed a nuanced picture of the current technological landscape. While DXplain performed somewhat better across the board, the LLMs demonstrated a level of proficiency that suggests they are rapidly closing the gap, albeit with different strengths and weaknesses.
The researchers at MGH believe that the conversation should not necessarily be about which system is "better," but rather how they can be integrated to create a more robust diagnostic framework. The investigators envision a future where the rigid, evidence-based logic of a system like DXplain is paired with the fluid, conversational capabilities of an LLM. This synergy, they argue, represents the optimal way forward for clinical AI, potentially enhancing the efficacy of both systems while significantly improving patient outcomes.
