The conclusion of a new JAMA study highlighting the importance of local validation and tuning, even if the AI’s training, design, and previous validations were all promising.

Imaging Wire Sponsors

annalise.ai  •  Arterys  •  Bayer  •  Blackford  •  Canon Medical  •  CARPL.ai  •  Change Healthcare  •  Enlitic  •  GE Healthcare  •  Hyperfine  •  Intelerad  •  Novarad  •  Riverain Technologies  •  Siemens Healthineers  •  United Imaging  •  Us2.ai

Artificial Intelligence

The Mammography AI Generalizability Gap

The “radiologists with AI beat radiologists without AI” trend might have achieved mainstream status in Spring 2020, when the DM DREAM Challenge developed an ensemble of mammography AI solutions that allowed radiologists to outperform rads who weren’t using AI.

The DM DREAM Challenge had plenty of credibility. It was produced by a team of respected experts, combined eight top-performing AI models, and used massive training and validation datasets (144k & 166k exams) from geographically distant regions (Washington state, USA & Stockholm, Sweden).

However, a new external validation study highlighted one problem that many weren’t thinking about back then. Ethnic diversity can have a major impact on AI performance, and the majority of women in the two datasets were White.

The new study used an ensemble of 11 mammography AI models from the DREAM study (the Challenge Ensemble Model; CEM) to analyze 37k mammography exams from UCLA’s diverse screening program, finding that:

• The CEM model’s UCLA performance declined from the previous Washington and Sweden validations (AUROCs: 0.85 vs. 0.90 & 0.92)
• The CEM model improved when combined with UCLA radiologist assessments, but still fell short of the Sweden AI+rads validation (AUROCs: 0.935 vs. 0.942)
• The CEM + radiologists model also achieved slightly lower sensitivity (0.813 vs. 0.826) and specificity (0.925 vs. 0.930) than UCLA rads without AI 
• The CEM + radiologists method performed particularly poorly with Hispanic women and women with a history of breast cancer

The Takeaway

Although generalization challenges and the importance of data diversity are everyday AI topics in late 2022, this follow-up study highlights how big of a challenge they can be (regardless of training size, ensemble approach, or validation track record), and underscores the need for local validation and fine-tuning before clinical adoption. 

It also underscores how much we’ve learned in the last three years, as neither the 2020 DREAM study’s limitations statement nor critical follow-up editorials mentioned data diversity among the study’s potential challenges.

Exploring Calantic Digital Solutions

Bayer’s cloud-based Calantic Digital Solutions AI platform features a suite of disease-specific AI apps that integrate into radiologist workflows, helping radiology teams scale AI deployment and improve efficiency and quality of care.

The Intelerad Cloud

Intelerad just launched its Intelerad Cloud suite of imaging solutions, marking the culmination of over four years of cloud investments and acquisitions. The new Intelerad Cloud allows imaging organizations to adopt a variety of hybrid, public, or private cloud solutions based on their specific needs (including: PACS, VNA, image exchange storage, long-term archiving, disaster recovery, patient portal).

The Wire

• CARPL.ai’s AI Expansion: CARPL.ai significantly expanded the list of companies and solutions available through its end-to-end AI platform, launching distribution alliances with Lunit, Avicenna.AI, ImageBiopsy Lab, and CoLumbo. The alliances give CARPL new AI solutions for CXR and mammography (Lunit), stroke and ICH (Avicenna.AI), MSK (ImageBiopsy Lab), and lumbar spine MRI (CoLumbo), while giving these AI software companies a uniquely positioned platform partner.

• O-RADS’ Pelvic MRI Performance: A study review in Radiology Journal (12 studies, 3.7k women, 4.5k adnexal lesions) highlighted the Ovarian-Adnexal Reporting and Data System (O-RADS) and related ADNEX system’s strong performance characterizing US-indeterminate adnexal lesions in pelvic MRI exams. The O-RADS and ADNEX reports characterized US-indeterminate adnexal lesions with 92% sensitivity and 91% specificity, while exams with O-RADS MRI 4 and 5 scores had higher-than-predicted malignancy rates (60% & 96% vs. 49% and 89%).

• Arterys’ Partner Additions: Arterys continued its AI platform expansion, partnering with InferVision to offer its lung oncology AI workflow (CT-based detection, segmentation, follow-up) and DeepLook Medical to offer its DL Precise automated tumor segmentation tool (all modalities; one-click segmentations and measurements). Arterys also announced a collaboration with AWS, using Amazon HealthLake Imaging to enhance the speed and responsiveness of solutions on its cloud platform. 

• KA Imaging Goes Mobile: KA Imaging unveiled its new Reveal Mobi Lite portable X-ray system, built on the company’s SpectralDR technology and Reveal 35C dual-energy X-ray detector. KA Imaging’s SpectralDR technology enables dual-energy subtraction, providing bone and tissue differentiation with a single standard X-ray exposure by acquiring three images simultaneously (DR, bone, and soft tissue). The Reveal Mobi Lite gives KA Imaging a new scanner-based path to market, in addition to offering the Reveal 35C detector as a retrofit solution for other X-ray systems.

• Viz.ai & Illuminate’s Aortic Alliance: Viz.ai brought its AI platform further beyond imaging, announcing a partnership with Illuminate to add patient records-based aortic aneurysm detection and monitoring. Illuminate uses NLP and AI to screen EHR data and radiology reports for signs of aortic aneurysm, and then assesses disease severity and facilitates follow-up surveillance. Illuminate joins Viz.ai’s CT-based Aortic AI module to create a uniquely comprehensive aortic aneurysm solution. 

• MRIguidance BoneMRI Adds Lumbar Spine: MRIguidance’s BoneMRI solution gained FDA clearance for lumbar spine exams, joining BoneMRI’s recent pelvic imaging FDA clearance. BoneMRI transforms MRI images into CT-quality 3D bone scans, allowing patients with degenerative orthopedic conditions to be examined without the radiation risks. BoneMRI also has a European CE Mark for cervical spine exams, suggesting that a cervical FDA clearance might be forthcoming.

• Rad AI & 3M: 3M Health Information Systems and RAD AI announced a reseller agreement that will make RAD AI’s Continuity incidental findings management solution and Omni impression generation software available through 3M’s M*Modal Fluency division. This seems like one of the more logical alliances, providing RAD AI with a complementary and well-connected channel partner, and giving 3M an answer to dictation rival Nuance’s incidental follow-up and report impression capabilities.

• SpinTech MRI’s STAGE 2.0 FDA: SpinTech MRI announced the FDA clearance and launch of its STAGE 2.0 with CROWN (Constrained Reduction Of White Noise) MRI acquisition and post-processing software. Unlike AI solutions that reconstruct under-sampled MRI data, STAGE 2.0 uses fully sampled data, allowing 30% faster scans and significantly improved image quality (brings 1.5T MRI output close to 3T MRI quality).

• TeraRecon Adds Imaging Biometrics: TeraRecon continued its Eureka Clinical AI platform expansion, adding Imaging Biometrics’ IB Clinic suite. The neuro MR post-processing solutions included in IB Clinic provide quantitative insights and standardized metrics to support tumor diagnosis, biopsy guidance, and post-surgical monitoring. The alliance comes several weeks after TeraRecon released a new version of the Eureka AI platform and launched its new Neuro AI solution for head CT-based ICH and LVO detection.

• Centaur Labs & Dandelion’s AI Alliance: Healthcare data annotation platform company, Centaur Labs, and AI training/testing data provider, Dandelion Health, announced an alliance to provide AI developers with access to de-identified and labeled clinical data (e.g. images, waveforms, health records). The collaborative dataset would initially include over 4M patients from two major US health systems, with more coming in the future.

• Oxipit Expands Quality Suite: Oxipit announced the expansion of its ‘Quality’ AI product lineup, adding musculoskeletal X-ray, mammography, and lung CT Quality solutions, which join its original chest X-ray solution. Oxipit’s Quality solutions are used as a double reader, notifying radiologists of potential missed findings when it identifies a mismatch between images and radiologist reports.

Portable Patient Identification

Did you know that portable chest X-ray exams are responsible for 69% of patient misidentification errors in radiology? That’s why GE Healthcare’s AMX Navigate portable X-ray system features a handheld barcode reader, allowing technologists to scan a patient’s wristband and automatically match the patient to the worklist.

Working on Your Organization’s AI Strategy?

This Blackford Analysis post outlines the key considerations for creating your AI goals and strategy, including some you might not have considered.

The Resource Wire

• Check out this Imaging Wire Show featuring Arterys’ Director of Product Management, Maya Khalifé, PhD, discussing how to deliver clinical value with AI, Arterys’ platform approach to neuro AI, and how AI can serve radiologists today and into the future.

• You know you’ve been making an impact, when the VA adopts your technology nationwide. That’s exactly what’s happening with Riverain Technologies’ ClearRead CT, which will be implemented across the VA Lung Precision Oncology Program (22 hub and 87 spoke locations).

• Deciding between public and private cloud? Watch this brief Change Healthcare animation detailing the operational benefits of public cloud hosting for imaging.

• Check out this Imaging Wire Show featuring Us2.ai’s co-founders – James Hare and Dr. Carolyn Lam – for a great discussion about Us2.ai’s continued clinical and commercial expansion, and their efforts to improve echocardiography accuracy, efficiency, and accessibility.

• See how Novarad’s new VisAR augmented reality surgical navigation system enables physicians to find and reach their target destination faster – without the expense, footprint, and setup time of conventional navigation systems and robots.

• Siemens Healthineers’ AI-powered MR image reconstruction technology, Deep Resolve, allows faster scans without compromising imaging resolution. Check out how Deep Resolve improves workflow efficiency and the patient experience, and view their portfolio of reconstructed scans.

• Annalise.ai doubled-down on its comprehensive AI strategy with the launch of its Annalise Enterprise CTB solution, which identifies a whopping 130 different non-contrast brain CT findings. Annalise Enterprise CTB analyzes brain CTs as they are acquired, prioritizes urgent cases, and provides radiologists with details on each finding (types, locations, likelihood).

• Check out this Imaging Wire Show with United Imaging’s Jeffrey Bundy and Mike Coulter, who detail their unique approach to medical imaging innovations. If you’re trying to figure out a simpler and more scalable way to run your imaging organization, this interview is a great way to start.

• How patients interact with a medical device can have as much impact as the device itself. Check out this Q&A with Hyperfine Lead Product Designer Corinne Hay to learn how human factors influence the design of everything from prescription containers to portable MRI systems.

• Imaging AI is evolving fast, but radiology leaders’ expectations for their AI technologies might be evolving even faster. In this Imaging Wire Show with Dr. Charlene Liew of SingHealth and Dr. Nina Kottler of Radiology Partners, we explore radiology leaders’ current and future expectations for AI, and the central role platforms play in their AI roadmaps.

• Canon’s Cartesion Prime is the only air-cooled digital PET/CT system, which means big reductions in service, chiller, and infrastructure costs. Check out Canon’s case for the air-cooled Cartesion Prime.

• Contrary to popular belief, imaging’s interoperability problem might actually be a data governance problem. The good news is, Enlitic’s Curie|ENDEX helps solve this problem, allowing you to get the most out of our modalities, PACS, and AI without needing your other vendors to intervene.