Breast screening is becoming one of the most promising use cases for AI, but there’s still a lot we’re learning about it. A new study in Radiology: Artificial Intelligence revealed new insights into how well mammography AI performs in a screening environment.
As we’ve reported in the past, mammography is one of radiology’s most challenging cancer screening exams, with radiologists sorting through large volumes of normal images before encountering a case that might be cancer.
- AI is being touted as an mammography aid that can help, and recent studies like the MASAI trial show it can reduce workload while boosting radiologist accuracy.
In the new study, researchers applied Lunit’s Insight MMG algorithm to mammograms in a retrospective study of 136.7k women screened in British Columbia from 2019 to 2020.
- Canada uses single reading for mammography, unlike the double-reading protocols employed in the U.K. and Europe.
AI’s performance was compared to single-reading radiologists using various metrics and follow-up periods, finding …
- At one-year follow-up, AI had slightly lower sensitivity (89% vs. 93%) and specificity (79% vs. 92%) compared to radiologists.
- At two-year follow-up, there was no statistically significant difference in sensitivity between the two (83.5% vs. 84.3%, p=0.69).
- AI’s overall AUC at one year was 0.93, but this varied based on mammographic and demographic features, with AI performing better in cases with fatty versus dense breasts (0.96 vs. 0.84) and cases with architectural distortion (0.96 vs. 0.92) but worse in cases with calcifications (0.87 vs. 0.92).
The researchers then constructed hypothetical scenarios in which AI might be used to assist radiologists, finding …
- If radiologists only read cases ruled abnormal by AI, it would reduce workload by 78%, but at a price of reduced sensitivity (86% vs. 93%) and 59 missed cancers across the cohort.
It’s worth noting that Insight MMG is designed to analyze 2D digital mammography exams.
- In the U.S., Lunit is emphasizing its recently cleared Insight DBT algorithm, which may be better suited for the country’s large installed base of 3D mammography systems.
The Takeaway
While the new findings aren’t a slam dunk for mammography AI, they do provide valuable insight into its performance that can inform future research, especially into areas where AI could use improvement.
