Tag: radiology AI

How Should AI Be Monitored?

Posted on by Brian Casey

Once an AI algorithm has been approved and moves into clinical use, how should its performance be monitored? This question was top of mind at last week’s meeting of the FDA’s new Digital Health Advisory Committee.

AI has the potential to radically reshape healthcare and help clinicians manage more patients with fewer staff and other resources. 

  • But AI also represents a regulatory challenge because it’s constantly learning, such that after a few years an AI algorithm might be operating much differently from the version first approved by the FDA – especially with generative AI. 

This conundrum was a point of discussion at last week’s DHAC meeting, which was called specifically to focus on regulation of generative AI, and could result in new rules covering all AI algorithms. (An executive summary that outlines the FDA’s thinking is available for download.)

Radiology was well-represented at DHAC, understandable given it has the lion’s share of authorized algorithms (73% of 950 devices at last count). 

  • A half-dozen radiology AI experts gave presentations over two days, including Parminder Bhatia of GE HealthCare; Nina Kottler, MD, of Radiology Partners; Pranav Rajpurkar, PhD, of Harvard; and Keith Dreyer, DO, PhD, and Bernardo Bizzo, MD, PhD, both of Mass General Brigham and the ACR’s Data Science Institute.  

Dreyer and Bizzo directly addressed the question of post-market AI surveillance, discussing ongoing efforts to track AI performance, including … 

The Takeaway

Last week’s DHAC meeting offers a fascinating glimpse at the issues the FDA is wrestling with as it contemplates stronger regulation of generative AI. Fortunately, radiology has blazed a trail in setting up structures like ARCH-AI and Assess-AI to monitor AI performance, and the FDA is likely to follow the specialty’s lead as it develops a regulatory framework.

Better Prostate MRI with AI

Posted on by Brian Casey

A homegrown AI algorithm was able to detect clinically significant prostate cancer on MRI scans with the same accuracy as experienced radiologists. In a new study in Radiology, researchers say the algorithm could improve radiologists’ ability to detect prostate cancer on MRI, with fewer false positives.

In past issues of The Imaging Wire, we’ve discussed the need to improve on existing tools like PSA tests to make prostate cancer screening more precise with fewer false positives and less need for patient work-up.

  • Adding MRI to prostate screening protocols is a step forward, but MRI is an expensive technology that requires experienced radiologists to interpret.

Could AI help? In the new study, researchers tested a deep learning algorithm developed at the Mayo Clinic to detect clinically significant prostate cancer on multiparametric (mpMRI) scans.

  • In an interesting wrinkle, the Mayo algorithm does not indicate tumor location, so a second algorithm – called Grad-CAM – was employed to localize tumors.

The Mayo algorithm was trained on a population of 5k patients with a cancer prevalence similar to a screening population, then tested in an external test set of 204 patients, finding …

  • No statistically significant difference in performance between the Mayo algorithm and radiologists based on AUC (0.86 vs. 0.84, p=0.68)
  • The highest AUC was with the combination of AI and radiologists (0.89, p<0.001)
  • The Grad-CAM algorithm was accurate in localizing 56 of 58 true-positive exams

An editorial noted that the study employed the Mayo algorithm on multiparametric MRI exams.

  • Prostate cancer imaging is moving from mpMRI toward biparametric MRI (bpMRI) due to its faster scan times and lack of contrast, and if validated on bpMRI, AI’s impact could be even more dramatic.

The Takeaway
The current study illustrates the exciting developments underway to make prostate imaging more accurate and easier to perform. They also support the technology evolution that could one day make prostate cancer screening a more widely accepted test.

MASAI Gets Even Better at ECR 2024

Posted on by Brian Casey

One of the biggest radiology stories of 2023 was the release of impressive interim results from the MASAI study, a large-scale trial of AI for breast screening in Sweden. At ECR 2024, MASAI researchers put an emphatic cap on the conference by presenting final data indicating that AI could have an even bigger impact on mammography screening than we thought. 

If you remember, MASAI’s interim results were published in August in Lancet Oncology and showed that ScreenPoint Medical’s Transpara AI algorithm was able to reduce radiologist workload by 44% when used as part of the kind of double-reading screening program that’s common in Europe.

  • Another MASAI finding was that AI-aided screening had a 20% higher cancer detection rate than conventional double-reading with human radiologists, but the difference was not statistically significant. 

That’s all changed with the final MASAI results, presented at ECR on March 2 by senior author Kristina Lång, MD, of Lund University.

  • Lång presented data from 106k participants who were randomized to either screening with Transpara V. 1.7 or conventional double reading without AI.

Transpara triaged mammograms by giving them a risk score of 1-10, and only those classified as high risk received double reading; lower-risk mammograms got a single human reader. In the final analysis, AI-aided screening … 

  • Had a 28% higher cancer detection rate per 1k women (6.4 vs. 5.0), a difference that was statistically significant (p=0.002)
  • Detected more cancers 10-20 mm (122 vs. 79)
  • Detected more cancers of non-specific histologic type (204 vs. 155)
  • Detected 20 more non-luminal A invasive cancers and 12 more DCIS grade 3 lesions

The Takeaway

When combined with the Lancet Oncology data, the new MASAI results indicate that AI could enable breast radiologists to have their cake and eat it too: a lower workload with higher cancer detection rates. 

Real-World AI Experiences

Posted on by Brian Casey

Clinical studies showing that AI helps radiologists interpret medical images are great, but how well does AI work in the real world – and what do radiologists think about it? These questions are addressed in a new study in Applied Ergonomics that takes a deep dive into the real-world implementation of a commercially available AI algorithm at a German hospital. 

A slew of clinical studies supporting AI were published in 2023, from the MASAI study on AI for breast screening to smaller studies on applications like opportunistic screening or predicting who should get lung cancer screening. 

  • But even an AI algorithm with the best clinical evidence behind it could fall flat if it’s difficult to use and doesn’t integrate well with existing radiology workflow.

To gain insight into this issue, the new study tracked University Hospital Bonn’s implementation of Quantib’s Prostate software for interpreting and documenting prostate MRI scans (Quantib was acquired by RadNet in January 2022). 

  • Researchers described the solution as providing partial automation of prostate MRI workflow, such as helping segment the prostate, generating heat maps of areas of interest, and automatically producing patient reports based on lesions it identifies. 

Prostate was installed at the hospital in the spring of 2022, with nine radiology residents and three attending physicians interviewed before and after implementation, finding…

  • All but one radiologist had a positive attitude toward AI before implementation and one was undecided 
  • After implementation, seven said their attitudes were unchanged, one was disappointed, and one saw their opinion shift positively
  • Use of the AI was inconsistent, with radiologists adopting different workflows and some using it all the time with others only using it occasionally
  • Major concerns cited included workflow delays due to AI use, additional steps required such as sending images to a server, and unstable performance

The findings prompted the researchers to conclude that AI is likely to be implemented and used in the real world differently than in clinical trials. Radiologists should be included in AI algorithm development to provide insights into workflow where the tools will be used.

The Takeaway

The new study is unique in that – rather than focusing on AI algorithm performance – it concentrated on the experiences of radiologists using the software and how they changed following implementation. Such studies can be illuminating as AI developers seek broader clinical use of their tools. 

AI Models Go Head-to-Head in Project AIR Study

Posted on by Brian Casey

One of the biggest challenges in assessing the performance of different AI algorithms is the varying conditions under which AI research studies are conducted. A new study from the Netherlands published this week in Radiology aims to correct that by testing a variety of AI algorithms head-to-head under similar conditions. 

There are over 200 AI algorithms on the European market (and even more in the US), many of which address the same clinical condition. 

  • Therefore, hospitals looking to acquire AI can find it difficult to assess the diagnostic performance of different models. 

The Project AIR initiative was launched to fill the gap in accurate assessment of AI algorithms by creating a Consumer Reports-style testing environment that’s consistent and transparent.

  • Project AIR researchers have assembled a validated database of medical images for different clinical applications, against which multiple AI algorithms can be tested; to ensure generalizability, images have come from different institutions and were acquired on equipment from different vendors. 

In the first test of the Project AIR concept, a team led by Kicky van Leeuwen of Radboud University Medical Centre in the Netherlands invited AI developers to participate, with nine products from eight vendors validated from June 2022 to January 2023: two models for bone age prediction and seven algorithms for lung nodule assessment (one vendor participated in both tests). Results included:

  • For bone age analysis, both of the tested algorithms (Visiana and Vuno) showed “excellent correlation” with the reference standard, with an r correlation coefficient of 0.987-0.989 (1 = perfect agreement)
  • For lung nodule analysis, there was a wider spread in AUC between the algorithms and human readers, with humans posting a mean AUC of 0.81
  • Researchers found superior performance for Annalise.ai (0.90), Lunit (0.93), Milvue (0.86), and Oxipit (0.88)

What’s next on Project AIR’s testing agenda? Van Leeuwen told The Imaging Wire that the next study will involve fracture detection. Meanwhile, interested parties can follow along on leaderboards for both bone age and lung nodule use cases. 

The Takeaway

Head-to-head studies like the one conducted by Project AIR may make many AI developers squirm (several that were invited declined to participate), but they are a necessary step toward building clinician confidence in the performance of AI algorithms that needs to take place to support the widespread adoption of AI. 

Lunit’s Deal for Volpara and AI Consolidation

Posted on by Brian Casey

Is the long-awaited consolidation of the healthcare AI sector gaining steam? In a deal valued at close to $200M, South Korean AI developer Lunit announced a bid to acquire Volpara Health, a developer of software for calculating breast density and cancer risk. 

At first glance, the alliance seems to be a match made in heaven. Lunit is a well-regarded AI developer that has seen impressive results in clinical trials of its Insight family of algorithms for indications ranging from mammography to chest imaging. 

  • Most recently, Lunit received FDA clearance for its Insight DBT software, marking its entry into the US breast screening market, and it also raised $150M in a public stock offering. 

Volpara has a long pedigree as a developer of breast imaging software, although it has shied away from image analysis applications to instead focus on breast center operations and risk assessment, in particular by calculating breast density. 

  • Thus, combining Lunit’s concentration in image analysis with Volpara’s focus on operations and risk assessment enables the combined company to offer a wider breadth of products to breast centers.

Lunit will also be able to take advantage of the marketing and sales structure that Volpara has built in the US mammography sector (97% of Volpara’s sales come from the US, where it has an installed base of 2k sites). Volpara expects 2024 sales of $30M and is cash-flow positive.

The question is whether the acquisition is a sign of things to come in the AI market. 

  • As commercial AI sales have been slow to develop, AI firms have largely funded their operations through venture capital firms – which are notoriously impatient in their quest for returns.

In fact, observers at the recent RSNA 2023 meeting noted that there were very few new start-up entrants into the AI space, and many AI vendors had smaller booths. 

  • And previous research has documented a slowdown in VC funding for AI developers that is prompting start-up firms to seek partners to provide more comprehensive offerings while also focusing on developing a road to profitability. 

The Takeaway

It’s not clear yet whether the Lunit/Volpara deal is a one-off combination or the start of a renewed consolidation trend in healthcare AI. Regardless of what happens, this alliance unites two of the stronger players in the field and has exciting potential for the years to come. 

AI’s Incremental Revolution

Posted on by Brian Casey

So AI dominated the discussion at last week’s RSNA 2023 meeting. But does that mean it’s finally on the path to widespread clinical use? 

Maybe not so much. For a technology that’s supposed to have a revolutionary impact on medicine, AI is taking a frustratingly long time to arrive. 

Indeed, there was plenty of skepticism about AI in the halls of McCormick Place last week. (For two interesting looks at AI at RSNA 2023, also see Hugh Harvey, MD’s list of takeaways in a post on X/Twitter and Herman Oosterwijk’s post on LinkedIn.) 

But as one executive we talked to pointed out, AI’s advance to routine clinical use in radiology is likely to be more incremental than all at once. 

  • And from that perspective, last week’s RSNA meeting was undoubtedly positive for AI. Scientific sessions were full of talks on practical clinical applications of AI, from breast AI to CT lung screening. 

Researchers also discussed the use of AI apart from image interpretation, with generative AI and large language models taking on tasks from answering patient questions about their reports to helping radiologists with dictation.

It’s fine to be a skeptic (especially when it comes to things you hear at RSNA), but for perspective look at many of the past arguments casting doubt on AI: 

  • AI algorithms don’t have FDA clearance (the FDA authorized 171 algorithms in just the past year)
  • You can’t get paid for using AI clinically (16 algorithms have CPT codes, with more on the way) 
  • There isn’t enough clinical evidence backing the use of AI (tell that to the authors of MASAI, PERFORMS, and a number of other recent studies with positive findings)
  • The AI market is overcrowded with companies and ripe for consolidation (what exciting new growth market isn’t?)

The Takeaway

Sure, it’s taking longer than expected for AI to take hold in radiology. But last week’s conference showed that AI’s incremental revolution is not only advancing but expanding in ways no one expected when IBM Watson was unveiled to an RSNA audience a mere 6-7 years ago. One can only imagine what the field will look like at RSNA 2030.

Looking for more coverage of RSNA 2023? Be sure to check out our videos from the technical exhibit floor, which you can find on our new Shows page.

Reimbursement Drives AI Adoption

Posted on by Brian Casey

It’s no secret that insurance reimbursement drives adoption of new medical technology. But a new analysis in NEJM AI shows exactly how reimbursement is affecting the diffusion into clinical practice of perhaps the newest medical technology – artificial intelligence. 

Researchers analyzed a database of over 11B CPT claims from January 2018 to June 2023 to find out how often reimbursement claims are being submitted for the use of the over 500 AI devices that had been approved by the FDA at the time the paper was finalized. 

  • The authors chose to focus on CPT claims rather than claims under the NTAP program for new technologies because CPT codes are used by both public and private payors in inpatient and outpatient settings, while NTAP only applies to Medicare inpatient payments. 

They found 16 medical AI procedures billable under CPT codes; of these, 15 codes were created since 2021 and the median age of a CPT code was about 374 days, indicating the novelty of medical AI.

  • Also, only four of the 16 had more than 1k claims submitted, leading the authors to state “overall utilization of medical AI products is still limited and focused on a few leading procedures,” such as coronary artery disease and diabetic retinopathy.

The top 10 AI products and number of CPT claims submitted are as follows:

  1. HeartFlow Analysis for coronary artery disease (67,306)
  2. LumineticsCore for diabetic retinopathy (15,097)
  3. Cleerly for coronary atherosclerosis (4,459)
  4. Perspectum LiverMultiScan for liver MRI (2,428)
  5. Perspectum CoverScan for multiorgan MRI (591)
  6. Koios DS for breast ultrasound (552)
  7. Anumana for ECG cardiac dysfunction (435)
  8. CADScor for cardiac acoustic waveform recording (331)
  9. Perspectum MRCP for quantitative MR cholangiopancreatography (237)
  10. CompuFlo for epidural infusion (67)

While radiology may rule in terms of the sheer number of FDA-approved AI products (79% in a recent analysis), the list shows that cardiology is king when it comes to paying the bills. 

The Takeaway

Amid the breathless hype around medical AI, the NEJM AI study comes as a bit of a wake-up call, showing how the cold reality of healthcare economics can limit technology diffusion – a finding also indicated in other studies of economic barriers to AI. 

On the positive side, it shows that a rosy future lies ahead for those AI algorithms – like HeartFlow Analysis – that can make the leap.

FDA Data Show AI Approval Boom

Posted on by Brian Casey

In the previous issue of The Imaging Wire, we discovered how venture capital investment in AI developers is fueling rapid growth in new AI applications for radiologists (despite a slowdown this year). 

This trend was underscored late last week with new data from the FDA showing strong growth in the number of regulatory authorizations of AI and machine learning-enabled devices in calendar 2023 compared to the year before. The findings show:

  • A resurgence of AI/ML authorizations this year, with over 30% growth compared to 14% in 2022 and 15% in 2021 – The last time authorizations grew this fast was in 2020 (+39%)
  • The FDA authorized 171 AI/ML-enabled devices in the past year. Of the total, 155 had final decision dates between August 1, 2022 to July 30, 2023, while 16 were reclassifications from prior periods 
  • Devices intended for radiology made up 79% of the total (122/155), an impressive number but down slightly compared to 87% in 2022 
  • Other medical specialities include cardiology (9%), neurology (5%), and gastroenterology/urology (4%)

One interesting wrinkle in the report was the fact that despite all the buzz around large language models for generative AI, the FDA has yet to authorize a device that uses generative AI or that is powered by LLMs. 

The Takeaway

The FDA’s new report confirms that radiology AI shows no sign of slowing down, despite a drop in AI investment this year. 

The data also offer perspective on a JACR report last week predicting that by 2035 radiology could be seeing 350 new AI/ML product approvals for the year. Product approvals would only have to grow at about a 10% annual rate to hit that number – a figure that seems perfectly achievable given the new FDA report.

What’s Fueling AI’s Growth

Posted on by Brian Casey

It’s no secret that the rapid growth of AI in radiology is being fueled by venture capital firms eager to see a payoff for early investments in startup AI developers. But are there signs that VCs’ appetite for radiology AI is starting to wane?

Maybe. And maybe not. While one new analysis shows that AI investments slowed in 2023 compared to the year before, another predicts that over the long term, VC investing will spur a boom in AI development that is likely to transform radiology. 

First up is an update by Signify Research to its ongoing analysis of VC funding. The new numbers show that through Q3 2023, the number of medical imaging AI deals has fallen compared to Q3 2022 (24 vs. 40). 

  • Total funding has also fallen for the second straight year, to $501M year-to-date in 2023. That compares to $771M through the third quarter of 2022, and $1.1B through the corresponding quarter of 2021. 

On the other hand, the average deal size has grown to an all-time high of $20.9M, compared to 2022 ($15.4M) and 2021 ($18M). 

  • And one company – Rapid AI – joined the exclusive club of just 14 AI vendors that have raised over $100M with a $75M Series C round in July 2023. 

In a look forward at AI’s future, a new analysis in JACR by researchers from the ACR Data Science Institute (DSI) directly ties VC funding to healthcare AI software development, predicting that every $1B in funding translates into 11 new product approvals, with a six-year lag between funding and approval. 

  • And the authors forecast long-term growth: In 2022 there were 69 FDA-approved products, but by 2035, funding is expected to reach $31B for the year, resulting in the release of a staggering 350 new AI products that year.

Further, the ACR DSI authors see a virtuous cycle developing, as increasing AI adoption spurs more investment that creates more products available to help radiologists with their workloads. 

The Takeaway

The numbers from Signify and ACR DSI don’t match up exactly, but together they paint a picture of a market segment that continues to enjoy massive VC investment. While the precise numbers may fluctuate year to year, investor interest in medical imaging AI will fuel innovation that promises to transform how radiology is practiced in years to come.

Get every issue of The Imaging Wire, delivered right to your inbox.

You might also like..

Select All

You're signed up!

It's great to have you as a reader. Check your inbox for a welcome email.

-- The Imaging Wire team

You're all set!