Measuring AI’s Impact in Real-World Radiology

One of the criticisms of AI in radiology is that there isn’t much data on its effectiveness in real-world environments. Well, a new study in JACR aims to fill that gap by detailing how AI adoption at a large imaging network in Europe affected key radiology performance metrics. 

Plenty of studies have been published on AI’s positive contribution to radiologists’ diagnostic performance. 

  • But in most cases this research was performed under controlled conditions, or involved a single AI algorithm working on a particular task. That’s left many wondering how well AI would perform under actual conditions. 

In the new study, researchers tracked the performance of 10 AI algorithms from seven vendors that were implemented over the Incepto Medical AI orchestration platform at 3R Swiss Imaging Network, a 20-center outpatient radiology network in Switzerland. 

  • Over 4.5 years, researchers tracked the network’s performance before and after AI implementation for the major radiology key performance indicator, report turnaround time (TAT). They also tracked AI adoption rates and infrastructure latency, or how delays in data transfer between PACS networks affected AI usage.

In all, 389k AI-assisted imaging exams were processed over the implementation period from 2021 to 2025, with researchers finding…

  • AI’s biggest impact was in high-volume modalities, reducing TAT for trauma radiography (-26%) and knee MRI (-18%).
  • AI improved the network’s diagnostic capacity, performing the work of almost half a full-time-equivalent radiologist for trauma radiography alone (0.46 FTE). 
  • AI usage rates were high, with 91% reporting active AI adoption and 66% reporting regular use.

On the downside, latency in data transfer between PACS networks slowed down AI performance, and in some cases AI results arrived after reports were finalized – too late to make a clinical impact…

  • Median total latency was 2.06 minutes per exam, with 72% attributable to data routing and tasks like fetching studies from PACS.
  • AI’s “too late rate” was 7.2% overall, ranging from 13% for chest CT to 3% for knee MRI, with 6.8% for trauma X-ray.

How did the network’s radiologists feel about AI? Researchers surveyed over 50 radiologists working for the network, finding that most of the differences in before-and-after opinions weren’t statistically significant. 

  • The only difference was a modest gain in the perception that AI made radiologists more productive (from 2.57 to 2.94 on a five-point Likert scale).

The Takeaway

The new study fills in a major gap in real-world experience regarding radiology AI implementation. It also reveals areas for improvement, as slow data transfer rates between networks can prevent AI results from reaching radiologists in time. 

Radiology Report Turnaround Slows

Radiologists are taking longer to interpret medical images than they did 10 years ago … and it’s a problem that appears to be getting worse. New data from the ACR’s Neiman HPI group, published in JACR, show that report turnaround times rose 27% between 2023 and 2024.

Report turnaround time (commonly referred to as TAT) is a closely watched barometer of radiologist productivity, and longer TAT could indicate radiologists are struggling to keep up with growing imaging volumes.

  • Another Neiman HPI study on TAT released earlier this year discovered a “hockey stick” effect, with turnaround times jumping sharply starting in 2022. 

But that study only tracked TAT through 2023. Meanwhile, anecdotal reports have surfaced of a “rapid increase” in turnaround times in late 2024.

  • So the updated research adds more recent data, analyzing newly available Medicare fee-for-service claims from 2024.

Researchers analyzed data from 2.9M office and hospital outpatient claims from 2014 to 2024, finding that TAT…

  • Grew just 14% in seven years from 2014 to 2021 (0.091 to 0.104 days).
  • But then rose at double-digit annual rates in 2023 (60%) and 2024 (27%).
  • Reached 0.251 days by the end of 2024, a 177% increase over 10 years. 
  • Increased at different rates by modality over the 10-year period, including CT (381%), MRI (278%), ultrasound (267%), and X-ray (126%). 

Why did TAT growth decelerate in 2024 over the year before? The Neiman HPI authors weren’t sure, but they said practice leaders may have taken steps to meet higher demand, like moonlighting and reactivating retired staff.

  • But these aren’t long-term measures that can meet the mismatch between supply and demand, suggesting instead that “the radiology workforce may be operating at or near maximum capacity.”

The Takeaway

The new data on slower radiology report turnaround times confirm fears that radiologists are struggling under rising imaging volumes and a stagnant workforce. While slower TAT growth in 2024 is a bit of a silver lining, it does little to hide the growing storm clouds on the horizon.

Get every issue of The Imaging Wire, delivered right to your inbox.