Tag: imaging AI

Teleradiology AI’s Mixed Bag

Posted on by Brian Casey

An AI algorithm that examined teleradiology studies for signs of intracranial hemorrhage had mixed performance in a new study in Radiology: Artificial Intelligence. AI helped detect ICH cases that might have been missed, but false positives slowed radiologists down. 

AI is being touted as a tool that can detect unseen pathology and speed up the workflow of radiologists facing an environment of limited resources and growing image volume.

  • This dynamic is particularly evident at teleradiology practices, which frequently see high volumes during off-hour shifts; indeed, a recent study found that telerad cases had higher rates of patient death and more malpractice claims than cases read by traditional radiology practices.

So teleradiologists could use a bit more help. In the new study, researchers from the VA’s National Teleradiology Program assessed Avicenna.ai’s CINA v1.0 algorithm for detecting ICH on STAT non-contrast head CT studies.

  • AI was used to analyze 58.3k CT exams processed by the teleradiology service from January 2023 to February 2024, with a 2.7% prevalence of ICH.

Results were as follows…

  • AI flagged 5.7k studies as positive for acute ICH and 52.7k as negative
  • Final radiology reports confirmed that 1.2k exams were true positives for a sensitivity of 76% and a positive predictive value of 21%
  • There were 384 false negatives (missed ICH cases), for a specificity of 92% and a negative predictive value of 99.3%
  • The algorithm’s performance at the VA was a bit lower than in previously published literature
  • Cases that the algorithm falsely flagged as positive took over a minute longer to interpret than prior to AI deployment
  • Overall, case interpretation times were slightly lower after AI than before

One issue to note is that the CINA algorithm is not intended for small hemorrhages with volumes < 3 mL; the researchers did not exclude these cases from their analysis, which could have reduced its performance.

  • Also, at 2.7% the VA’s teleradiology program ICH prevalence was lower than the 10% prevalence Avicenna has used to rate its performance.

The Takeaway

The new findings aren’t exactly a slam dunk for AI in the teleradiology setting, but in terms of real-world results they are exactly what’s needed to assess the true value of the technology compared to outcomes in more tightly controlled environments.

AI Detects Incidental PE

Posted on by Brian Casey

In one of the most famous quotes about radiology and artificial intelligence, Curtis Langlotz, MD, PhD, once said that AI will not replace radiologists, but radiologists with AI will replace those without it. A new study in AJR illustrates his point, showing that radiologists using a commercially available AI algorithm had higher rates of detecting incidental pulmonary embolism on CT scans. 

AI is being applied to many clinical use cases in radiology, but one of the more promising is for detecting and triaging emergent conditions that might have escaped the radiologist’s attention on initial interpretations.

  • Pulmonary embolism is one such condition. PE can be life-threatening and occurs in 1.3-2.6% of routine contrast-enhanced CT exams, but radiologist miss rates range from 10-75% depending on patient population.

AI can help by automatically analyzing CT scans and alerting radiologists to PEs when they can be treated quickly; the FDA has authorized several algorithms for this clinical use. 

  • In the new paper, researchers conducted a prospective real-world study of Aidoc’s BriefCase for iPE Triage at the University of Alabama at Birmingham. 

Researchers tracked rates of PE detection in 4.3k patients before and after AI implementation in 2021, finding … 

  • Radiologists saw their sensitivity for PE detection go up after AI implementation (80% vs. 96%) 
  • Specificity was unchanged (99.1% vs. 99.9%, p=0.58)
  • The PE incidence rate went up (1.4% vs. 1.6%)
  • There was no statistically significant difference in report turnaround time before and after AI (65 vs. 78 minutes, p=0.26)

The study echoes findings from 2023, when researchers from UT Southwestern also used the Aidoc algorithm for PE detection, in that case finding that AI cut times for report turnaround and patient waits. 

The Takeaway

While studies showing AI’s value to radiologists are commonplace, many of them are performed under controlled conditions that don’t translate to the real world. The current study is significant because it shows that with AI, radiologists can achieve near-perfect detection of a potentially life-threatening condition without a negative impact on workflow.

Better Prostate MRI Tools

Posted on by Brian Casey

In past issues of The Imaging Wire, we’ve discussed some of the challenges to prostate cancer screening that have limited its wider adoption. But researchers continue to develop new tools for prostate imaging – particularly with MRI – that could flip the script. 

Three new studies were published in just the last week focusing on prostate MRI, two involving AI image analysis.

In a new study in The Lancet Oncology, researchers presented results from AI algorithms developed for the Prostate Imaging—Cancer Artificial Intelligence (PI-CAI) Challenge.

  • PI-CAI pitted teams from around the world in a competition to develop the best prostate AI algorithms, with results presented at recent RSNA and ECR conferences. 

Researchers measured the ensemble performance of top-performing PI-CAI algorithms for detecting clinically significant prostate cancer against 62 radiologists who used the PI-RADS system in a population of 400 cases, finding that AI …

  • Had performance superior to radiologists (AUROC=0.91 vs. 0.86)
  • Generated 50% fewer false-positive results
  • Detected 20% fewer low-grade cases 

Broader use of prostate AI could reduce inter-reader variability and need for experienced radiologists to diagnose prostate cancer.

In the next study, in the Journal of Urology, researchers tested Avenda Health’s Unfold AI cancer mapping algorithm to measure the extent of tumors by analyzing their margins on MRI scans, finding that compared to physicians, AI … 

  • Had higher accuracy for defining tumor margins compared to two manual methods (85% vs. 67% and 76%)
  • Reduced underestimations of cancer extent with a significantly higher negative margin rate (73% vs. 1.6%)

AI wasn’t used in the final study, but this one could be the most important of the three due to its potential economic impact on prostate MRI.

  • Canadian researchers in Radiology tested a biparametric prostate MRI protocol that avoids the use of gadolinium contrast against multiparametric contrast-based MRI for guiding prostate biopsy. 

They compared the protocols in 1.5k patients with prostate lesions undergoing biopsy, finding…

  • No statistically significant difference in PPV between bpMRI and mpMRI for all prostate cancer (55% vs. 56%, p=0.61) 
  • No difference for clinically significant prostate cancer (34% vs. 34%, p=0.97). 

They concluded that bpMRI offers lower costs and could improve access to prostate MRI by making the scans easier to perform.

The Takeaway

The advances in AI and MRI protocols shown in the new studies could easily be applied to prostate cancer screening, making it more economical, accessible, and clinically effective.  

Is Radiology’s AI Edge Fading?

Posted on by Brian Casey

Is radiology’s AI edge fading, at least when it comes to its share of AI-enabled medical devices being granted regulatory authorization by the FDA? The latest year-to-date figures from the agency suggest that radiology’s AI dominance could be declining. 

Radiology was one of the first medical specialties to go digital, and software developers have targeted the field for AI applications like image analysis and data reconstruction.

  • Indeed, FDA data from recent years shows that radiology makes up the vast majority of agency authorizations for AI- and machine learning-enabled medical devices, ranging from 86% in 2020 and 2022 to 79% in 2023. 

But in the new data, radiology devices made up only 73% of authorizations from January-March 2024. Other data points indicate that the FDA …

  • Authorized 151 new devices since August 2023
  • Reclassified as AI/ML-enabled 40 devices that were previously authorized 
  • Authorized a total of 882 devices since it began tracking the field 

      In an interesting wrinkle, many of the devices on the updated list are big-iron scanners that the FDA has decided to classify as AI/ML-enabled devices. 

      • These include CT and MRI scanners from Siemens Healthineers, ultrasound scanners from Philips and Canon Medical Systems, an MRI scanner from United Imaging, and the recently launched Butterfly iQ3 POCUS scanner. 

      The additions could be a sign that imaging OEMs increasingly are baking AI functionality into their products at a basic level, blurring the line between hardware and software. 

      The Takeaway

      It should be no cause for panic that radiology’s share of AI/ML authorizations is declining as other medical specialties catch up to the discipline’s head start. The good news is that the FDA’s latest figures show how AI is becoming an integral part of medicine, in ways that clinicians may not even notice.

      Fine-Tuning AI for Breast Screening

      Posted on by Brian Casey

      AI has shown in research studies it can help radiologists interpret breast screening exams, but for routine clinical use many questions remain about the optimal AI parameters to catch the most cancers while generating the fewest callbacks. Fortunately, a massive new study out of Norway in Radiology: Artificial Intelligence provides some guidance. 

      Recent research such as the MASAI trial has already demonstrated that AI can help reduce the number of screening mammograms radiologists have to review, and for many low-risk cases eliminate the need for double-reading, which is commonplace in Europe. 

      • But growing interest in breast screening AI is tempered by the field’s experience with computer-aided detection, which was introduced over 20 years ago but generated many false alarms that slowed radiologists down. 

      Fast forward to 2024. The new generation of breast AI algorithms seems to have addressed CAD’s shortcomings, but it’s still not clear exactly how they can best be used. 

      • Researchers from Norway’s national breast screening program tested one mammography AI tool – Lunit’s Insight MMG – in a study with data obtained from 662k women screened with 2D mammography from 2004 to 2018. 

      Researchers tested AI with a variety of specificity and sensitivity settings based on AI risk scores; in one scenario, 50% of the highest risk scores were classified as positive for cancer, while in another that threshold was set to 10%. The group found …

      • At the 50% cutoff, AI would correctly identify 99% of screen-detected cancers and 85% of interval cancers. 
      • At the 10% cutoff, AI would detect 92% of screen-detected cancers and 45% of interval cancers 
      • AI understandably performed better in identifying false-positive cases as negative at the 10% threshold than 50% (69% vs. 17%)
      • AI had a higher AUC than double-reading for screen-detected cancers (0.97 vs. 0.88)

      How generalizable is the study? It’s worth noting that the research relied on AI of 2D mammography, which is prevalent in Europe (most mammography in the US employs DBT). In fact, Lunit is targeting the US with its recently cleared Insight DBT algorithm rather than Insight MMG. 

      The Takeaway

      As with MASAI, the new study offers an exciting look at AI’s potential for breast screening. Ultimately, it may turn out that there’s no single sensitivity and specificity threshold at which mammography AI should be set; instead, each breast imaging facility might choose the parameters they feel best suit the characteristics of their radiologists and patient population. 

      Headwinds Slow AI Funding

      Posted on by Brian Casey

      Venture capital funding of medical imaging AI developers continues to slow. A new report from Signify Research shows that funding declined 19% in 2023, and is off to a slow start in 2024 as well. 

      Signify tracks VC funding on an annual basis, and previous reports from the UK firm showed that AI investment peaked in 2021 and has been declining ever since. 

      • The report’s author, Signify analyst Ellie Baker, sees a variety of factors behind the decline, chief among them macroeconomic headwinds such as tighter access to capital due to higher interest rates. 

      Total Funding Value Drops – Total funding for 2023 came in at $627M, down 19% from $771M in 2022. Funding hit a peak in 2021 at $1.1B.

      Deal Volume Declines – The number of deals in 2023 fell to 35, down 30% from 50 the year before. Deal volume peaked in 2021 at 63. And 2024 isn’t off to a great start, with only five deals recorded in the first quarter. 

      Deals Are Getting Bigger – Despite the declines, the average deal size grew last year, to $19M, up 23% versus $15M in 2022. 

      HeartFlow Rules the Roost – HeartFlow raised the most in 2023, fueled by a massive $215M funding round in April 2023, while Cleerly held the crown in 2022.

      US Funding Dominates – On a geographic basis, funding is shifting away from Europe (-46%) and Asia-Pacific (no 2023 deals) and back to the Americas, which generated over 70% of the funding raised last year. This may be due to the US providing faster technology uptake and more routes to reimbursement.

      Early Bird Gets the Worm – Unlike past years in which later-stage funding dominated, 2024 has seen a shift to early-stage deals with seed funding and Series A rounds, such as AZmed’s $16M deal in February 2024. 

      $100M Club Admits New Members – Signify’s exclusive “$100M Club” of AI developers has expanded to include Elucid and RapidAI. 

      The Takeaway

      Despite the funding drop, Signify still sees a healthy funding environment for AI developers ($627M is definitely a lot of money). That said, AI software developers are going to have to make a stronger case to investors regarding revenue potential and a path to ROI. 

      AI Speeds Up MRI Scans

      Posted on by Brian Casey

      In our last issue, we reported on a new study underscoring the positive return on investment when deploying radiology AI at the hospital level. This week, we’re bringing you additional research that confirms AI’s economic value, this time when used to speed up MRI data reconstruction. 

      While AI for medical image analysis has garnered the lion’s share of attention, AI algorithms are also being developed for behind-the-scenes applications like facilitating staff workflow or reconstructing image data. 

      • For example, software developers have created solutions that enable scans to be acquired faster and with less input data (such as radiation dose) and then upscaled to resemble full-resolution images. 

      In the new study in European Journal of Radiology, researchers from Finland focused on whether accelerated data reconstruction could help their hospital avoid the need to buy a new MRI scanner. 

      • Six MRI scanners currently serve their hospital, but the radiology department will be losing access to one of them by the end of the year, leaving them with five. 

      They calculated that a 20% increase in capacity per remaining scanner could help them achieve the same MRI throughput at a lower cost; to test that hypothesis they evaluated Siemens Healthineers’ Deep Resolve Boost algorithm. 

      • Deep Resolve Boost uses raw-data-to-image deep learning reconstruction to denoise images and enable rapid acceleration of scan times; a total knee MRI exam can be performed in just two minutes. 

      Deep Resolve Boost was applied to 3T MRI scans of 78 patients acquired in fall of 2023, with the researchers finding that deep learning reconstruction… 

      • Reduced annual exam costs by 399k euros compared to acquiring a new scanner
      • Enabled an overall increase in scanner capacity of 20-32%
      • Had an acquisition cost 10% of the price of a new MRI scanner, leading to a cost reduction of 19 euros per scan
      • Was a lower-cost option than operating five scanners and adding a Saturday shift

      The Takeaway

      As with last week’s study, the new research demonstrates that AI’s real value comes from helping radiologists work more efficiently and do more with less, rather than from direct reimbursement for AI use. It’s the same argument that was made to promote the adoption of PACS some 30 years ago – and we all know how that turned out.

      Study Shows AI’s Economic Value

      Posted on by Brian Casey

      One of the biggest criticisms of AI for radiology is that it hasn’t demonstrated its return on investment. Well, a new study in JACR tackles that argument head on, demonstrating AI’s ability to both improve radiologist efficiency and also drive new revenues for imaging facilities. 

      AI adoption into radiology workflow on a broad scale will require significant investment, both in financial cost and IT resources. 

      • So far, there have been few studies showing that imaging facilities will get a payback for these investments, especially as Medicare and private insurance reimbursement for AI under CPT codes is limited to fewer than 20 algorithms. 

      The new paper analyzes the use of an ROI calculator developed for Bayer’s Calantic platform, a centralized architecture for radiology AI integration and deployment. 

      • The calculator provides an estimate of AI’s value to an enterprise – such as by generating downstream procedures – by comparing workflow without AI to a scenario in which AI is integrated into operations.

      The study included inputs for 14 AI algorithms covering thoracic and neurology applications on the Calantic platform, with researchers finding that over five years … 

      • The use of AI generated $3.6M in revenue versus $1.8M in costs, representing payback of $4.51 for every $1 invested
      • Use of the platform generated 1.5k additional diagnoses, resulting in more follow-up scans, hospitalizations, and downstream procedures
      • AI’s ROI jumped to 791% when radiologist time savings were considered
      • These time savings included a reduction of 15 eight-hour working days of waiting time, 78 days in triage time, 10 days in reading time, and 41 days in reporting time  

      Although AI led to additional hospitalizations, it’s possible that length of stay was shorter: for example, reprioritization of stroke cases resulted in 264 fewer hospital days for patients with intracerebral hemorrhage. 

      • Executives with Bayer told The Imaging Wire that while the calculator is not publicly available, the company does use it in consultations with health systems about new AI deployments. 

      The Takeaway

      This study suggests that examining AI through the lens of direct reimbursement for AI-aided imaging services might not be the right way to assess the technology’s real economic value. Although it won’t settle the debate over AI’s economic benefits, the research is a step in the right direction.

      Real-World AI Experiences

      Posted on by Brian Casey

      Clinical studies showing that AI helps radiologists interpret medical images are great, but how well does AI work in the real world – and what do radiologists think about it? These questions are addressed in a new study in Applied Ergonomics that takes a deep dive into the real-world implementation of a commercially available AI algorithm at a German hospital. 

      A slew of clinical studies supporting AI were published in 2023, from the MASAI study on AI for breast screening to smaller studies on applications like opportunistic screening or predicting who should get lung cancer screening. 

      • But even an AI algorithm with the best clinical evidence behind it could fall flat if it’s difficult to use and doesn’t integrate well with existing radiology workflow.

      To gain insight into this issue, the new study tracked University Hospital Bonn’s implementation of Quantib’s Prostate software for interpreting and documenting prostate MRI scans (Quantib was acquired by RadNet in January 2022). 

      • Researchers described the solution as providing partial automation of prostate MRI workflow, such as helping segment the prostate, generating heat maps of areas of interest, and automatically producing patient reports based on lesions it identifies. 

      Prostate was installed at the hospital in the spring of 2022, with nine radiology residents and three attending physicians interviewed before and after implementation, finding…

      • All but one radiologist had a positive attitude toward AI before implementation and one was undecided 
      • After implementation, seven said their attitudes were unchanged, one was disappointed, and one saw their opinion shift positively
      • Use of the AI was inconsistent, with radiologists adopting different workflows and some using it all the time with others only using it occasionally
      • Major concerns cited included workflow delays due to AI use, additional steps required such as sending images to a server, and unstable performance

      The findings prompted the researchers to conclude that AI is likely to be implemented and used in the real world differently than in clinical trials. Radiologists should be included in AI algorithm development to provide insights into workflow where the tools will be used.

      The Takeaway

      The new study is unique in that – rather than focusing on AI algorithm performance – it concentrated on the experiences of radiologists using the software and how they changed following implementation. Such studies can be illuminating as AI developers seek broader clinical use of their tools. 

      AI Models Go Head-to-Head in Project AIR Study

      Posted on by Brian Casey

      One of the biggest challenges in assessing the performance of different AI algorithms is the varying conditions under which AI research studies are conducted. A new study from the Netherlands published this week in Radiology aims to correct that by testing a variety of AI algorithms head-to-head under similar conditions. 

      There are over 200 AI algorithms on the European market (and even more in the US), many of which address the same clinical condition. 

      • Therefore, hospitals looking to acquire AI can find it difficult to assess the diagnostic performance of different models. 

      The Project AIR initiative was launched to fill the gap in accurate assessment of AI algorithms by creating a Consumer Reports-style testing environment that’s consistent and transparent.

      • Project AIR researchers have assembled a validated database of medical images for different clinical applications, against which multiple AI algorithms can be tested; to ensure generalizability, images have come from different institutions and were acquired on equipment from different vendors. 

      In the first test of the Project AIR concept, a team led by Kicky van Leeuwen of Radboud University Medical Centre in the Netherlands invited AI developers to participate, with nine products from eight vendors validated from June 2022 to January 2023: two models for bone age prediction and seven algorithms for lung nodule assessment (one vendor participated in both tests). Results included:

      • For bone age analysis, both of the tested algorithms (Visiana and Vuno) showed “excellent correlation” with the reference standard, with an r correlation coefficient of 0.987-0.989 (1 = perfect agreement)
      • For lung nodule analysis, there was a wider spread in AUC between the algorithms and human readers, with humans posting a mean AUC of 0.81
      • Researchers found superior performance for Annalise.ai (0.90), Lunit (0.93), Milvue (0.86), and Oxipit (0.88)

      What’s next on Project AIR’s testing agenda? Van Leeuwen told The Imaging Wire that the next study will involve fracture detection. Meanwhile, interested parties can follow along on leaderboards for both bone age and lung nodule use cases. 

      The Takeaway

      Head-to-head studies like the one conducted by Project AIR may make many AI developers squirm (several that were invited declined to participate), but they are a necessary step toward building clinician confidence in the performance of AI algorithms that needs to take place to support the widespread adoption of AI. 

      Get every issue of The Imaging Wire, delivered right to your inbox.

      You might also like..

      Select All

      You're signed up!

      It's great to have you as a reader. Check your inbox for a welcome email.

      -- The Imaging Wire team

      You're all set!