AI Outperforms Doctors in Diagnostic Tests, Major Study Finds

A new wave of artificial intelligence is shaking up healthcare, and the latest evidence suggests it’s not just hype. Researchers have found that advanced AI models can diagnose medical cases with surprising accuracy — in some instances outperforming trained physicians.

The study, published in Science, tested a reasoning-focused AI model developed by OpenAI against both doctors and earlier AI systems. The results point to a clear leap forward in how machines handle complex medical decisions.

At the center of the research was OpenAI’s “o1” model, designed to process problems step by step rather than generate quick responses. When compared with GPT-4, the newer system showed a marked improvement in diagnosing clinical cases.

Researchers didn’t rely on simple examples. They used unfamiliar, real-world medical scenarios, including emergency department records. In these tests, the o1 model correctly identified diagnoses more than two-thirds of the time during initial triage. By comparison, experienced physicians reached the correct answer roughly half the time.

That gap has caught the attention of experts. Robert Wachter, a leading figure in digital health, described the findings as a turning point. He noted that modern AI systems are now clearly outperforming older models — and in certain tasks, even doctors — when it comes to identifying diagnoses and recommending next steps.

Still, the results come with important caveats. The study focused entirely on text-based scenarios, which strips away much of what doctors rely on in real clinical settings. Visual cues, patient behavior, tone of voice, and medical imaging all play a critical role in diagnosis — and these elements were not part of the testing.

That limitation matters. Real hospital environments are unpredictable, fast-paced, and often chaotic. Translating strong performance in controlled conditions into everyday clinical practice remains a major challenge.

Researchers behind the study acknowledge this gap and are calling for urgent follow-up work. They stress the need for large-scale clinical trials to assess how AI can safely improve patient outcomes in real-world settings.

Experts are also pushing back against the idea of replacing doctors outright. Instead, the emerging view is one of collaboration — where AI supports medical professionals by offering faster insights, while humans provide context, judgment, and accountability.

The pace of progress is undeniable. As AI systems continue to evolve, their role in healthcare is shifting rapidly. The real question now isn’t if AI will become part of medicine — it’s how far its influence will reach, and how safely it can be integrated into patient care.