|
Predicting AI Performance | NY Times Probes Prenuvo September 25, 2023
|
|
|
|
Together with
|
|
|
“Gosh I have a scary number of friends right now who have found out very important, actionable things from a Prenuvo scan. I was skeptical but now more open minded. The major issue seems to be price.”
|
Healthcare investor Christina Farr, commenting on whole-body MRI screening firm Prenuvo.
|
|
|
How can you predict whether an AI algorithm will fall short for a particular clinical use case such as detecting cancer? Researchers in Radiology took a crack at this conundrum by developing what they call an “uncertainty quantification” metric to predict when an AI algorithm might be less accurate.
AI is rapidly moving into wider clinical use, with a number of exciting studies published in just the last few months showing how AI can help radiologists interpret screening mammograms or direct which women should get supplemental breast MRI.
But AI isn’t infallible. And unlike a human radiologist who might be less confident in a particular diagnosis, an AI algorithm doesn’t have a built-in hedging mechanism.
So researchers from Denmark and the Netherlands decided to build one. They took publicly available AI algorithms and tweaked their code so they produced “uncertainty quantification” scores with their predictions.
They then tested how well the scores predicted AI performance in a dataset of 13k images for three common tasks covering some of the deadliest types of cancer:
1) detecting pancreatic ductal adenocarcinoma on CT 2) detecting clinically significant prostate cancer on MRI 3) predicting pulmonary nodule malignancy on low-dose CT
Researchers classified the highest 80% of the AI predictions as “certain,” and the remaining 20% as “uncertain,” and compared AI’s accuracy in both groups, finding …
- AI led to significant accuracy improvements in the “certain” group for pancreatic cancer (80% vs. 59%), prostate cancer (90% vs. 63%), and pulmonary nodule malignancy prediction (80% vs. 51%)
- AI accuracy was comparable to clinicians when its predictions were “certain” (80% vs. 78%, P=0.07), but much worse when “uncertain” (50% vs. 68%, P<0.001)
- Using AI to triage “uncertain” cases produced overall accuracy improvements for pancreatic and prostate cancer (+5%) and lung nodule malignancy prediction (+6%) compared to a no-triage scenario
How would uncertainty quantification be used in clinical practice? It could play a triage role, deprioritizing radiologist review of easier cases while helping them focus on more challenging studies. It’s a concept similar to the MASAI study of mammography AI.
The Takeaway
Like MASAI, the new findings present exciting new possibilities for AI implementation. They also present a framework within which AI can be implemented more safely by alerting clinicians to cases in which AI’s analysis might fall short – and enabling humans to step in and pick up the slack.
|
|
|
The Clinical Value of Soft-Tissue Chest X-Ray
Soft-tissue techniques can improve the visibility and accuracy of chest X-ray. Learn about two important soft-tissue methods – bone suppression and dual-energy subtraction – in this white paper from Riverain Technologies.
|
|
A Better Way to Deploy AI
CARPL.ai’s end-to-end modular platform for AI deployment is changing how University Hospitals in Cleveland uses AI to enhance patient care. Hear from UH clinicians and administrators as they share their experiences.
|
|
- NY Times Probes Prenuvo: The New York Times posted a feature story examining full-body MRI screening company Prenuvo. The article focused on the company’s marketing tactics, such as its use of celebrities like Kim Kardashian to tout its offerings, but also dove deeper into the question of whether full-body screening is clinically justified (spoiler alert – most clinicians said it isn’t). The article mostly takes a skeptical view of the company, but we suspect the folks at Prenuvo believe that any press is good press.
- FTC Sues over Anesthesiology Roll-Up: In a move that could signal a federal crackdown on private equity in healthcare, the FTC is suing Welsh, Carson, Anderson & Stowe, charging the PE firm with monopolizing the anesthesiology market in Texas. The feds claim that the firm executed a roll-up scheme to buy large anesthesiology practices in the state, drove up prices, and struck deals to keep out competitors. Welsh Carson also owns US Radiology Specialists, which recently launched US Radiology Connexia, notes RadTweeter Ben White, MD.
- 1.5T vs. 3T MRI for Cardiac Imaging: And you thought the field-strength wars in MRI were over. A new point-counterpoint in AJR poses the question, which is best for cardiac MRI, 1.5T or 3T? Singing 3T’s praises are Sophie You, MD, and Albert Hsiao, MD, PhD, who argue that 3T’s higher SNR is invaluable for contrast MRA and 4D-flow sequences. But Seth Kligerman, MD, maintains that 1.5T MRI can scan a wider range of patients with implants, has fewer and less complex artifacts, and lower cost.
- Bracco Partners with Subtle: Bracco Imaging has signed a partnership with Subtle Medical covering MRI contrast in the US. The companies will work together to harness AI and leverage each firm’s intellectual property for more efficient interpretation of contrast-enhanced MRI scans. Subtle has developed AI-based technology that enables faster acquisition of MRI and PET scans that can then be reconstructed into higher-quality images. The company just received an NIH grant to further develop its SubtleSYNTH application to create synthetic MR images.
- Myocarditis after COVID Vax: People who received their second COVID-19 vaccination showed signs of myocarditis on PET/CT scans up to 180 days after getting the shot … even if they didn’t have symptoms. In Radiology, researchers found that in 1k people, those who were vaccinated had higher FDG uptake from heart inflammation as measured by median SUVmax compared to the unvaccinated (4.8 vs. 3.3). With vaccination season approaching, the findings show that radiology practices should be on the lookout for vaccination side effects visible on imaging.
- Bayer Signs CEM Collab with Hologic: Bayer and Hologic are collaborating to offer contrast-enhanced mammography (CEM) solutions to clinicians in Europe, Asia-Pacific, and Canada. The partnership unites Bayer’s contrast injectors for CEM with Hologic’s mammography gantries to enable the companies to offer an integrated solution. CEM serves as an adjunct to screening mammography to investigate inconclusive findings or perform preoperative disease assessment; it’s also a relatively low-cost procedure compared to other tools like breast MRI.
- GE Issues Sustainability Report: GE HealthCare issued its first report since becoming an independent company on its progress toward achieving environmental, social, and governance goals. GE noted that it has reduced greenhouse gas emissions by 27% since 2019, and is targeting a 50% reduction by 2030, with net-zero emissions achieved by 2050. The company has also joined a federal government pledge to decarbonize the healthcare sector, and has established principles and guidelines for social impact and governance and accountability.
- Black-Blood CT of Carotids: Detecting atherosclerotic plaque and determining if it’s at risk of rupture is one of imaging’s great challenges. In European Radiology, researchers from China describe black-blood CT, a CT image subtraction method in which the vascular lumen appears black. In 110 patients, black-blood CT had high sensitivity and specificity detecting stenosis >50% (97% and 84%), with AUC of 0.93. Black-blood CT also was comparable to vessel-wall MRI for plaque burden analysis, while being cheaper and easier to perform.
- Cardiac Imaging’s Uneven Rebound: Cardiovascular imaging procedure volume rebounded unevenly around the world from the declines seen during the COVID-19 pandemic. In Radiology: Cardiothoracic Imaging, researchers found that volumes in April 2021 had recovered compared to pre-pandemic levels in the US (+4%) and were down slightly in non-US high-income countries (-6%). But volumes were down markedly in lower-middle and low-income countries (-41% and -50%). Recovery within the US also varied by region: Midwest (+11%), Northeast (+9%), South (+1%), and West (-7%).
- Cardiac Imaging Kickbacks: New York City cardiologist Klaus Rentrop, MD will pay $6.5M in fines and give up ownership of his cardiac imaging practice Gramercy Cardiac Diagnostic Services after settling a False Claims Act case related to a 2010-2021 kickback scheme. Rentrop maintained kickback agreements disguised as “rental payments” with referring physician practices and other cardiologists. The practices referred “tens of thousands” of patients to the partner cardiologists, who referred those same patients to Gramercy Cardiac for cardiac PET and SPECT exams.
- A Board of Their Own for Cardiology: A group of cardiovascular organizations plans to form a new independent board to certify cardiologists. The new group would split from the ABIM, which currently oversees cardiologist certification, and has the backing of the ACC, HFSA, HRS, and SCAI, while the AHA is reviewing its support. A major reason for the move reportedly has been dissatisfaction with the maintenance of certification (MOC) process for cardiologists; the new group said it will replace MOC with “a pathway to continuous certification and competency.” Should the ABR take notice?
|
|
The Multitenant Cloud Advantage
Check out this Change Healthcare video explaining the difference between single-tenant and multitenant cloud architecture, and how multitenant solutions can improve your efficiency and flexibility.
|
|
Fast. Flexible. Modern.
United Imaging’s service organization is called Customer Success for a reason. Their mission is to think ahead, understand their customers’ goals, and proactively help customers achieve them. They also store critical service parts in the U.S. and back up their promises with the United Performance Guarantee.
|
|
- Join us on September 25 for the final installment in a series of #RadEqual webinars hosted by Intelerad on overcoming the mid-career stall. Hear from Jennifer Harvey, MD, and Maureen Kohli, MD, as they discuss strategies to guide your career and find your circle of influence.
- Optellum has collaborated with GE HealthCare to get its software into more clinical settings around the world, and eventually integrate its precision lung cancer care platform with existing GE HealthCare imaging devices and cloud platforms. Read more on how Optellum is helping catch cancer at the earliest and most treatable stage.
- Radiology is leading healthcare’s AI revolution, and yet many people in radiology are just starting to build their understanding of AI. That’s why Bayer published its truly Complete Guide to Artificial Intelligence in Radiology, detailing how AI can address radiology’s challenges, AI’s core use cases, and AI’s path towards adoption.
- What are the advantages of CloudPACS? Speed, reliability, and security, with no latency. Learn how Visage Imaging’s Visage 7 PACS was built around the cloud from the ground up in this white paper.
- Teleradiology can be a force multiplier in radiology. Join Daniel Corbett and Daniel Arnold on September 27 for a webinar exploring the pros, cons, and essential factors for private practice leaders considering teleradiology integration, hosted by Medality.
- Radiology faces numerous challenges to more efficient workflow, from the siloed nature of healthcare enterprises to mundane tasks that are ripe for automation. In this Imaging Wire Show, we talked to Dr. Matthew Lungren and Calum Cunningham of Nuance Communications.
- Strain imaging is the most sensitive parameter for determining myocardial deformation and systolic left ventricular function. In this video, see how echo AI from Us2.ai was used to analyze myocardial global longitudinal strain.
- What are some of the major innovations happening now in the medical image acquisition chain? In this Imaging Wire Show, we talked to Josh Gurewitz and Ajit Shankaranarayanan of Subtle Medical about new developments in image acquisition and enhancement.
- I-MED Radiology Network in Australia has been using AI solutions from Annalise.ai for the last few years. Hear the comments of Chief Medical Officer Dr. Ron Shnier on how they complement the network’s radiologists to enhance accuracy, consistency, and efficiency in reading medical images.
- How is fast, quantitative SPECT/CT supporting quicker assessment of treatment response in patients with metastatic prostate cancer? Find out in this case study from New York-Presbyterian Hospital/Weill Cornell Medical Center, provided by Siemens Healthineers.
- What tools are available to help radiologists work remotely? In this case study, teleradiology provider 4ways Healthcare of the UK describes how they used Merative’s Merge PACS 8.0 platform to improve their service to clients while supporting remote radiologists.
- Medical data has emerged as a critical resource for driving better patient outcomes. But how can we improve data collection? This article from Enlitic describes how its Curie|ENDEX technology can help.
- Adopting a platform strategy can simplify the deployment and management of imaging applications and AI algorithms, but there’s a lot to consider. In this eBook, Blackford Analysis and its clients detail how AI platforms can benefit clinical and IT teams, and share guidelines to consider when selecting a platform.
- Looking for a better way to get patients to comply with follow-up instructions? PocketHealth now offers Follow-Up Navigator, a new solution that puts follow-ups directly in the patient’s hands. Learn more in this article.
- Creating your AI adoption plan? This Tempus Radiology report details what clinical, efficiency, and regulatory factors to look for in radiology AI vendors.
|
|
|
|
|