AI has shown in research studies it can help radiologists interpret breast screening exams, but for routine clinical use many questions remain about the optimal AI parameters to catch the most cancers while generating the fewest callbacks. Fortunately, a massive new study out of Norway in Radiology: Artificial Intelligence provides some guidance.
Recent research such as the MASAI trial has already demonstrated that AI can help reduce the number of screening mammograms radiologists have to review, and for many low-risk cases eliminate the need for double-reading, which is commonplace in Europe.
- But growing interest in breast screening AI is tempered by the field’s experience with computer-aided detection, which was introduced over 20 years ago but generated many false alarms that slowed radiologists down.
Fast forward to 2024. The new generation of breast AI algorithms seems to have addressed CAD’s shortcomings, but it’s still not clear exactly how they can best be used.
- Researchers from Norway’s national breast screening program tested one mammography AI tool – Lunit’s Insight MMG – in a study with data obtained from 662k women screened with 2D mammography from 2004 to 2018.
Researchers tested AI with a variety of specificity and sensitivity settings based on AI risk scores; in one scenario, 50% of the highest risk scores were classified as positive for cancer, while in another that threshold was set to 10%. The group found …
- At the 50% cutoff, AI would correctly identify 99% of screen-detected cancers and 85% of interval cancers.
- At the 10% cutoff, AI would detect 92% of screen-detected cancers and 45% of interval cancers
- AI understandably performed better in identifying false-positive cases as negative at the 10% threshold than 50% (69% vs. 17%)
- AI had a higher AUC than double-reading for screen-detected cancers (0.97 vs. 0.88)
How generalizable is the study? It’s worth noting that the research relied on AI of 2D mammography, which is prevalent in Europe (most mammography in the US employs DBT). In fact, Lunit is targeting the US with its recently cleared Insight DBT algorithm rather than Insight MMG.
The Takeaway
As with MASAI, the new study offers an exciting look at AI’s potential for breast screening. Ultimately, it may turn out that there’s no single sensitivity and specificity threshold at which mammography AI should be set; instead, each breast imaging facility might choose the parameters they feel best suit the characteristics of their radiologists and patient population.
