AI for PE Detection: ‘Selective but Meaningful’

AI made a “selective but meaningful” contribution to radiologist interpretations of CT pulmonary angiography scans for pulmonary embolism. The study, published in Radiology: Artificial Intelligence, offers valuable insights into real-world implementation of AI on a large scale. 

One of the major criticisms of AI is that algorithms used in real-world clinical situations don’t perform as well as they do in the controlled environments that vendors use to acquire data for regulatory submissions.

  • AI performance can drop off as much as 20 to 30 percentage points for important metrics like sensitivity and specificity. 

The new study sought to investigate this phenomenon by analyzing a real-world implementation of Aidoc’s AI algorithm for PE detection. 

  • Researchers assessed the algorithm’s performance for analyzing CTPA exams across a variety of clinical environments in an integrated health network, including the emergency department and inpatient and outpatient settings. 

Scans of 29.5k patients acquired from 2021 to 2023 were included. AI analyzed images in real time, after which exams were interpreted by radiologists who knew the AI findings. Researchers found…

  • Radiologists using AI had higher sensitivity than the algorithm on its own (99% vs. 85%).
  • Specificity was more or less the same (99.8% vs. 99.5%).
  • Agreement between radiologists and AI was high (98%).
  • Agreement was higher when AI assessed cases as negative rather than positive (98% vs. 94%).
  • Radiologists disagreed with AI in 2.2% of cases. The final determination by a panel of expert thoracic radiologists strongly favored radiologists (89%).
  • Of the 3.3k cases positive for PE, 0.81% were detected only by AI – or 26 cases.

In analyzing the results, the researchers characterized AI’s contribution as “selective but meaningful.”

  • AI-positive results meant scans might require more scrutiny from radiologists, while an AI-negative call might be supportive – but not definitive – for negative PE.

The Takeaway

The new study of AI for PE detection is a fascinating look at real-world AI deployment. While the sensitivity, specificity, and agreement numbers are interesting, what draws our attention is the 26 PE cases caught only by AI over 18 months of use. That boils down to 26 patients whose clinical condition wasn’t missed, and 26 potential malpractice lawsuits that were never filed.

Are CT Scans Too Slow?

Is your imaging practice paying attention to the acquisition speed of your CT scans? A new study in AJR suggests that CT pulmonary angiography patients are being scanned at an average speed that’s 30% slower than optimal, resulting in unnecessary imaging artifacts. 

CT is radiology’s workhorse modality, but recent studies have found wide variation in CT scanning parameters, sometimes between scanners in the same health system or even the same facility. 

  • This can result in patients getting different levels of radiation dose for the same type of study, or differences in image quality that can make image comparisons more difficult. 

To even out the variation, imaging groups have proposed best-practice scanning protocols for different exams and anatomical regions. 

  • Most of these protocols focus on reducing radiation dose, but there are other elements – such as scan acquisition speed –  that are also important.

For CT pulmonary angiography exams, scans that are performed too slowly run the risk of motion artifacts caused by patients breathing.

  • Imaging sites, therefore, should try to optimize scan speed, which is determined by a mix of collimation, pitch, and gantry rotation time. 

Researchers in the new study — from UC San Francisco and University of Wisconsin — found that many sites were using a fixed scan speed for CTPA regardless of patient characteristics. 

  • They analyzed data for 167k CTPA scans acquired from 2016 to 2021, with data coming from 121 sites using 28 different scanner models from various manufacturers.

Researchers compared scan speeds to best-practice levels, revealing … 

  • Scans on average were 30% slower than the best-practice speed.
  • 87% of acquisitions were slower.
  • 62% were more than 20% slower.
  • Scan speeds varied widely by vendor and by scanner model.

Researchers concluded that CT sites weren’t optimizing their equipment and not implementing the fastest scan speeds available.  

  • As a result, they could be generating widespread motion artifacts, especially for smaller patients. 

The Takeaway

The new findings show that radiation dose isn’t the only CT scanning parameter that needs optimization. Imaging practices should take a look at their CT protocols to ensure they have the optimal settings, especially for CTPA scans of smaller patients. 

Did Malpractice Risk Kill V/Q Exams?

CT perfusion angiography exams have largely replaced nuclear medicine-based ventilation/perfusion (V/Q) studies for detecting pulmonary embolism. But a new article in Academic Radiology suggests that CT’s rise wasn’t entirely based on clinical efficacy – fears of malpractice risk may have played a role. 

V/Q studies can help diagnose PE by enabling clinicians to visualize lung perfusion, showing defects such as blockages in pulmonary vessels. The scans are typically performed in three phases … 

  1. An albumin injection to show pulmonary vasculature.
  2. A radiopharmaceutical that’s inhaled and imaged with a gamma camera.
  3. A chest radiograph to correlate findings. 

The scans dominated PE imaging in the 1980s, but the rise of CT saw radiology facilities begin to shift.

  • CTPA was seen as having higher spatial resolution and was easier to perform than nuclear medicine exams. 

But the new article suggests that there were other forces at work as well – in particular, fear of malpractice risk from PEs that weren’t adequately followed after inconclusive V/Q exams.

  • The problem originated with clinical guidelines for V/Q reporting that classified some 20% of V/Q studies as “low probability” for PE when they probably would have better been classified as “inconclusive” or “non-diagnostic.”

As a result, a number of “low probability” patients weren’t followed up adequately, with tragic results that later figured into medical malpractice cases …

  • A patient who was diagnosed with pneumonia after an inconclusive V/Q exam, sent home, and died one day later of a “massive” PE.
  • A patient with leg and chest pain who was given heparin after a negative V/Q scan and later suffered internal hemorrhage; fortunately she survived.
  • A patient with “vague symptoms” who had an inconclusive V/Q scan and later died of an undiagnosed PE that some claimed would have been detected on CTPA.

Indeed, the theme of PE malpractice cases began to shift over time, from failure to diagnose V/Q scans to failure to order CTPA exams – which were soon seen as the clinical gold standard.

The Takeaway

Given the fast pace of development in radiology, it’s inevitable that some technologies that were once clinical staples fall by the wayside. But the new article offers a fascinating look at how clinical language can lead to medico-legal concerns that influence physician behavior – often in ways that are impossible to detect as they happen.

Get every issue of The Imaging Wire, delivered right to your inbox.