A new Stanford study revealed that diagnostic variations are largely due to differences in radiologist skill levels (not work styles/preferences, etc.), suggesting that physician skill gaps might represent a major source of healthcare waste, and warning that efforts to standardize care could lead to even worse results.
The researchers analyzed 4.67M CXR interpretations from patients with suspected pneumonia, finding that radiologist skill level accounted for 39% of variations in positive diagnoses (both true & false) and 78% of variations in missed diagnoses. Those variations had a major impact on patient care:
- Reassigning a patient from a radiologist in the 10th to 90th percentile for positive diagnostic rates would increase their probability of receiving a positive diagnosis from 8.9% to 12.3%.
- Reassigning a patient from a radiologist in the 10th to 90th percentile for missed diagnosis rates would increase their probability of receiving a false negative from 0.2% to 1.8%.
Perhaps counterintuitively, they found that the radiologists who were more likely to diagnose patients with pneumonia were also more likely to submit false negative diagnoses, suggesting that less skilled radiologists are responsible for an outsized share of unnecessary, delayed, and inconsistent care.
Skill can be hard to define, but the researchers found that the “most skilled radiologists” were generally older and more experienced, wrote shorter reports, and spent more time on each report.
The researchers weren’t specifically trying to understand radiologist skill variations with this study, and their main takeaway is that we might have to change our assumptions about how to fix the U.S. healthcare system:
- Healthcare inefficiency might have more to do with physician performance, and less to do with other commonly cited issues (e.g. misaligned payor/provider incentives)
- Relying on standardized approaches to equalize patient care and address cost variations might actually lead to worse care and higher costs
The Takeaway
Most readers probably aren’t surprised to hear that some radiologists are way more accurate than others, and that diagnostic skill increases with age/experience. However, this study gives new evidence supporting the value of quality improvement efforts, and could make it easier to demonstrate how radiology products/processes that reduce variability but don’t generate revenue (like AI…) might deliver clearer ROI than some might think.
