The Mammography AI Generalizability Gap

The “radiologists with AI beat radiologists without AI” trend might have achieved mainstream status in Spring 2020, when the DM DREAM Challenge developed an ensemble of mammography AI solutions that allowed radiologists to outperform rads who weren’t using AI.

The DM DREAM Challenge had plenty of credibility. It was produced by a team of respected experts, combined eight top-performing AI models, and used massive training and validation datasets (144k & 166k exams) from geographically distant regions (Washington state, USA & Stockholm, Sweden).

However, a new external validation study highlighted one problem that many weren’t thinking about back then. Ethnic diversity can have a major impact on AI performance, and the majority of women in the two datasets were White.

The new study used an ensemble of 11 mammography AI models from the DREAM study (the Challenge Ensemble Model; CEM) to analyze 37k mammography exams from UCLA’s diverse screening program, finding that:

  • The CEM model’s UCLA performance declined from the previous Washington and Sweden validations (AUROCs: 0.85 vs. 0.90 & 0.92)
  • The CEM model improved when combined with UCLA radiologist assessments, but still fell short of the Sweden AI+rads validation (AUROCs: 0.935 vs. 0.942)
  • The CEM + radiologists model also achieved slightly lower sensitivity (0.813 vs. 0.826) and specificity (0.925 vs. 0.930) than UCLA rads without AI 
  • The CEM + radiologists method performed particularly poorly with Hispanic women and women with a history of breast cancer

The Takeaway

Although generalization challenges and the importance of data diversity are everyday AI topics in late 2022, this follow-up study highlights how big of a challenge they can be (regardless of training size, ensemble approach, or validation track record), and underscores the need for local validation and fine-tuning before clinical adoption. 

It also underscores how much we’ve learned in the last three years, as neither the 2020 DREAM study’s limitations statement nor critical follow-up editorials mentioned data diversity among the study’s potential challenges.

Developing the Eighth Modality

Radiology has adopted seven mainstream modalities over its 127 years, and 4DMedical is determined to create the eighth imaging modality with its new XV Scanner.

The XV Scanner would be the first dedicated lung imaging system, giving radiologists four-dimensional and color-coded visibility into patients’ lung airflow and blood flow, and potentially a new way to assess lung diseases. 

  • The XV Scanner integrates fluoroscopy with advanced analytics software, producing qualitative and quantitative 4D lung function metrics 
  • It simultaneously acquires images from different angles, then measures lung tissue motion, and calculates ventilation at each breathing stage and every lung location
  • XV scans take 5 seconds to perform and deliver less radiation than a typical chest X-ray

4DMedical’s XV technology is also backed by a growing number of positive clinical studies, solid post-IPO funding, and an impressive expansion across Australian imaging giant I-Med Radiology’s 250 locations.

Although the XV Scanner hardware is still forthcoming, 4DMedical will initially launch XV software that can be installed on existing fluoroscopy systems (FDA cleared for ventilation, later adding perfusion) and will also support existing CTs in the future. 

  • Software-only might prove to be a logical starting point, providing 4DMedical with a low-friction way to demonstrate XV’s impact on patient care and test whether this impact is great enough to entice imaging departments to add a whole new scanner to their fleets.

The Takeaway

Creating medical imaging’s eighth mainstream modality might be among the most ambitious goals you’ll hear at RSNA 2022, but if the XV Scanner proves to be much better than existing lung imaging techniques, radiology might have to make room for one more.

AWS Targets Storage and Speed with HealthLake Imaging

AWS took a major step to bolster its cloud value proposition with the launch of Amazon HealthLake Imaging, a new HIPAA-eligible capability that addresses some of cloud imaging’s most common pain points. We sat down with AWS AI leader, Dr. Taha Kass-Hout, at HLTH 2022 last week to explore Amazon HealthLake Imaging’s potential impact on radiology.

Amazon HealthLake Imaging allows healthcare organizations to run multiple applications from a single authoritative copy of an image’s data that’s stored in the cloud, while giving each on-site application customizable metadata-level image access (e.g., patient ID, modality), and returning specially-encoded/compressed images to facilitate faster transfer. As a result…

  • Healthcare providers can cut their image storage TCO by 40% by eliminating the storage creep that comes from saving the same images to the cloud multiple times
  • Radiologists can retrieve and load imaging data from the cloud with sub-second latencies
  • Image viewers and AI algorithms can present or analyze the contents of a DICOM study faster, because they don’t have to load unnecessary image data
  • Researchers and developers can create de-identified image copies, without copying pixel data (and having to store that extra data)
  • AI development teams can access DICOM metadata in a developer-friendly format

Although AWS already plays a major role in radiology, this is one of very few imaging-targeted launch announcements that we’ve seen from the cloud giant. It also comes one month after Google Cloud similarly made its most public cloud imaging announcement in recent memory. 

  • Considering that medical imaging is responsible for roughly 90% of healthcare data, the recent surge in cloud imaging announcements suggests that the cloud leaders are increasing their focus on imaging as a way to add, keep, and grow their healthcare cloud accounts.

The Takeaway

It’s not every day that a storage provider launches a solution specifically intended to cut their clients’ storage costs nearly in half, but this seems like a logical move for AWS, considering that storage costs and performance lag are two of cloud imaging’s biggest challenges. It makes even more sense considering imaging’s role in overall healthcare cloud adoption, where we are in the healthcare cloud landgrab, and the fact that Amazon’s core principals start with “Customer Obsessed.”

Siemens Healthineers Targets MRI at Shape 2023

Siemens Healthineers kicked-off RSNA announcements season with its Shape 23 event, highlighted by a pair of forthcoming MRIs that should serve as the cornerstones of its high-end lineup for years to come.

Magnetom Cima.X – Siemens reinforced its already-solid 3T MR lineup with its new Magnetom Cima.X, calling it the company’s “strongest 3T MRI system ever.”

The Magnetom Cima.X owes that “strongest 3T” title to its new Gemini Gradients, which achieve 200 mT/m amplitude and 200 T/m/s slew rate performance. That’s a 2.5x increase from Siemens’ previous 3T MRIs and it’s higher than any other clinically released whole-body MRI. 

The Magnetom Cima.X also features Siemens Healthineers’ …

  • Benchmark 3T magnet
  • Deep Resolve AI image reconstruction for up to 50% faster scans
  • Open Recon for integrating custom reconstruction and post-processing solutions
  • BioMatrix Technology to automatically adjust exams based on patient biovariability
  • myExam Companion for streamlining technologist workflows

Magnetom Terra.X – Siemens’ new Magnetom Terra.X 7T MR is the long-awaited successor to the Magnetom Terra (the first FDA-cleared 7T MR), bringing improved clinical and research performance. The Magnetom Terra.X leverages Siemens’ new Ultra IQ Technology to achieve even greater image quality and visualization of small structures, Deep Resolve for image reconstruction-based speed and image enhancements, and Open Recon to support custom reconstructions.

The Takeaway

Although both MRIs are still under development, their starring role in Siemens Healthineers’ big RSNA event underscores their significance to Siemens’ high-end MRI lineup, and gives a glimpse of features to expect in future 1.5T and 3T MR launches. That’s especially notable given that Siemens’ last two RSNA announcements focused on its new low-field 0.55T MRIs, and it hasn’t launched any high-field systems in over three years.

The Medical Imaging Economy

With economic warning signs flashing brighter by the day, and hospitals continuing to struggle, it’s hard not to be concerned about medical imaging’s economic situation. However, the major imaging companies’ latest round of earnings suggest that there might be more reasons to remain confident. 

  • Agfa – Agfa’s two imaging divisions had very different Q3s, as HealthCare IT posted solid revenue and earnings growth (+25.7% to $64M; +63.4% to $4.1M EBIT), and Radiology Solutions saw modest revenue growth and a big earnings decline (+1.5% to $121M; -69.3% to $2.9M EBIT).
  • Canon – Canon Medical Systems continued its upswing, posting solid revenue (+9% to $908.5M) and operating profit (+7.5% to $46M) growth amid rising orders and strong post-COVID demand.
  • Fujifilm – Fujifilm’s Healthcare unit posted yet another positive quarter, as imaging drove big increases to revenue (+17.1% to $1.7B) and operating income (+24.4% to $236M). 
  • GE HealthCare – GE HealthCare posted its third straight quarter of revenue growth (+10% to $4.6B), while inflation led to slightly lower profit ($700M).
  • Hologic – The semiconductor shortage caused Hologic’s breast imaging revenue to fall yet again (-20.2% to $212M), while the company’s overall net income plummeted (-63.9% to $118.7M).
  • Konica Minolta – Konica Minolta’s Healthcare revenue increased for the second straight quarter (+14% to $254M), although the division continued to operate at a loss (-$18M).  
  • Philips – Philips’ Diagnosis & Treatment division’s comparable sales fell for the third straight quarter (-2% to $2.37B) due to component shortages, while division profit also declined (Adjusted EBITA -31.6% to $216M).
  • RadNet – RadNet posted another quarter of rising revenues (+5.2% to $350M), although the labor shortage and related payroll inflation cut into its profitability (Adjusted EBITDA -16.1% to $45.8M).
  • Siemens Healthineers – Siemens’ imaging business remained the company’s (and industry’s) top performer, as strong MRI and CT sales drove yet another quarter of revenue growth (+8.1% to $3.35B) and solid margins (Adjusted EBIT +22.4% to $776M).

Although several companies noted economic and inflation headwinds, nearly every earnings report forecasted positive Q4s and 2023s, as supply chain challenges subside and the post-COVID demand surge continues. 

The Takeaway

There are plenty of reasons to be concerned about the economy. However, most companies still reported solid healthcare/imaging financials, and most factors that hurt Q3 performances are likely to improve throughout 2023. Plus, healthcare is historically insulated from economic downturns. 

That doesn’t mean that the next year (or two) will be easy, but it does suggest that medical imaging could fare better than many sectors of the overall economy.

The Case for 18F-NaF PET/CT Bone Metastases Detection

Results from the MITNEC-A1 trial are in, and they further support using 18F-NaF PET/CT to detect bone metastases in patients with prostate and breast cancer, while bolstering its case for replacing 99mTc-MDP as the “bone imaging radiopharmaceutical of choice.”

The prospective, multicenter, single-cohort, phase 3 trial enrolled 261 breast and prostate cancer patients (57 & 204) who had high risk or suspected bone metastasis, scanning each participant with 18F-NaF PET/CT and 99mTc-MDP SPECT. 

Two experts interpreted the scans, which were later compared to 24-month follow-up results, revealing that 42% of the patients had bone metastases (109), and finding that 18F-NaF PET/CT diagnosed bone metastases with far higher…

  • Accuracy – 84.3% vs. 77.4%
  • Sensitivity – 78.9% vs. 63.3%
  • Negative Predictive Value – 85.4% vs. 76.9%

The MITNEC-A1 trial stands on the shoulders of a growing list of studies that support 18F-NaF PET/CT for bone metastases detection, and these latest results make the transition to 18F-NaF PET/CT “appealing” to this study’s authors. 

The next step in that transition process will likely be exploring 18F-NaF PET/CT’s cost-effectiveness versus bone scintigraphy with 99mTc-MDP SPECT, potentially leading to more widespread adoption.

The Takeaway

It’s historically been a challenge to detect prostate and breast cancer bone metastases. Although there’s more research to be done, it appears that 18F-NaF PET/CT might help overcome that challenge, and become bone imaging’s new radiopharmaceutical of choice.

RadNet’s UK Lung Cancer Screening Acquisition

RadNet advanced its AI-led cancer screening strategy, acquiring a 75% stake in Heart & Lung Health, a UK-based teleradiology network with a direct connection to the NHS’ lung cancer screening program.

Heart & Lung Health (HLH) has a network of over 70 cardiothoracic radiologists, and provides teleradiology reporting services for the NHS and a variety of UK hospitals and academic institutions.

Acquiring a UK telerad company might seem out of character for RadNet, which has historically focused its M&A on US-based imaging centers (and more recently global AI developers), only mentioned Europe once in its 2021 annual report, and exited the teleradiology business in 2020. However…

  • HLH is the leading reporting provider for NHS England Targeted Lung Health Check (TLHC), an AI-enabled lung cancer screening pilot program that might pave the way for a UK-wide program. 
  • TLHC requires all radiologists to use AI with their LDCT screening interpretations, suggesting that AI might also be required in a future UK-wide program.
  • HLH uses RadNet’s Aidence subsidiary’s lung cancer AI tools, and HLH will work with Aidence to further develop its solutions.

The Takeaway

RadNet started 2022 by acquiring two major cancer screening AI companies (Aidence and Quantib), which combined with its DeepHealth breast cancer AI business to support its ambitious new strategy to become a population-scale cancer screening leader. 

That goal might have seemed like a longshot to some, given AI’s uncertain path forward and RadNet’s geographic concentration in just seven US states. However, last week’s HLH acquisition showed that RadNet remains very committed to AI-driven cancer screening leadership, and its strategy might not be as geographically-challenged as some initially thought.

Multimodal AI Virtual Breast Biopsies

Radiology Journal detailed a multimodal AI solution that can classify breast lesion subtypes using mammograms, potentially reducing unnecessary biopsies and improving biopsy interpretations. 

Researchers from Israel and IBM/Merative first pretrained a deep learning model with 26k digital mammograms to classify images (malignant, benign, or normal), and used these pretraining weights to develop a lesion subtype classification model trained with mammograms and clinical data. Finally, they trained a pair of lesion classification models using digital mammograms linked to biopsy results from 2,120 women in Israel and 1,642 women in the US. 

When the Israel AI model was tested against mammograms from 441 Israeli women it…

  • Predicted malignancy with an 0.88 AUC
  • Classified ductal carcinoma in situ, invasive carcinomas, or benign lesions with 0.76, 0.85, and 0.82 AUCs
  • Correctly interpreted 98.7% of malignant mammographic examinations and 74.6% of invasive carcinomas (matching three radiologists)
  • Would have prevented 13% of unnecessary biopsies and missed 1.3% of malignancies (at 99% sensitivity)

When the US AI model was tested against mammograms from 344 US women it…

  • Predicted malignancy with a lower 0.80 AUC
  • Classified ductal carcinoma in situ, invasive carcinomas, or benign lesions with lower 0.74, 0.83, and 0.72 AUCs 
  • Correctly interpreted 96.8% of malignant mammographic examinations and 63% of invasive carcinomas (matching three radiologists)

The authors attributed the US model’s lower accuracy to its smaller training dataset, and noted that the two models’ also had worse performance when tested against data from the other country (US model w/Israel data, Israel model w/ US data) or when classifying rare lesion types. 

However, they were still bullish about this approach with enough training data, and noted the future potential to add other imaging modalities and genetic information to further enhance multimodal breast cancer assessments.

The Takeaway 

We’ve historically relied on biopsy results to classify breast lesion subtypes, and that will remain true for quite a while. However, this study shows that multimodal-trained AI can extract far more information from mammograms, while potentially reducing unnecessary biopsies and improving the accuracy of the biopsies that are performed.

Get every issue of The Imaging Wire, delivered right to your inbox.

You might also like..

Select All

You're signed up!

It's great to have you as a reader. Check your inbox for a welcome email.

-- The Imaging Wire team

You're all set!