How Should AI Be Monitored?

Once an AI algorithm has been approved and moves into clinical use, how should its performance be monitored? This question was top of mind at last week’s meeting of the FDA’s new Digital Health Advisory Committee.

AI has the potential to radically reshape healthcare and help clinicians manage more patients with fewer staff and other resources. 

  • But AI also represents a regulatory challenge because it’s constantly learning, such that after a few years an AI algorithm might be operating much differently from the version first approved by the FDA – especially with generative AI. 

This conundrum was a point of discussion at last week’s DHAC meeting, which was called specifically to focus on regulation of generative AI, and could result in new rules covering all AI algorithms. (An executive summary that outlines the FDA’s thinking is available for download.)

Radiology was well-represented at DHAC, understandable given it has the lion’s share of authorized algorithms (73% of 950 devices at last count). 

  • A half-dozen radiology AI experts gave presentations over two days, including Parminder Bhatia of GE HealthCare; Nina Kottler, MD, of Radiology Partners; Pranav Rajpurkar, PhD, of Harvard; and Keith Dreyer, DO, PhD, and Bernardo Bizzo, MD, PhD, both of Mass General Brigham and the ACR’s Data Science Institute.  

Dreyer and Bizzo directly addressed the question of post-market AI surveillance, discussing ongoing efforts to track AI performance, including … 

The Takeaway

Last week’s DHAC meeting offers a fascinating glimpse at the issues the FDA is wrestling with as it contemplates stronger regulation of generative AI. Fortunately, radiology has blazed a trail in setting up structures like ARCH-AI and Assess-AI to monitor AI performance, and the FDA is likely to follow the specialty’s lead as it develops a regulatory framework.

Non-Physicians Are Reading More Medical Images

Non-physician practitioners are reading more medical images in U.S. medical offices. That’s according to a new study in JACR by researchers who found that the share of images interpreted by NPPs has doubled in the last 10 years. 

In the U.S., radiologists consider themselves to be the primary interpreters of medical images, but inroads have been made not only by other physicians but also by non-physician practitioners like nurses and physician assistants. 

  • NPPs are supposed to receive specialized training in image interpretation, but radiologists question whether such training is adequate, especially compared to the years of training that radiologists receive. 

Previous research has documented the rise in NPP image interpretation, but the new study takes a longer view, examining the period 2013-2022. 

  • It also specifically focuses on the medical office setting, where it’s believed NPP interpretation is growing faster than in hospitals, where radiologists still dominate interpretations. 

In their analysis of Medicare claims, researchers from the ACR’s Harvey L. Neiman Health Policy Institute found … 

  • NPPs’ share of office-based image interpretations grew 9% annually (2.5% to 5.5%).
  • Growth rates varied by modality, with MRI growing at 9.9% annually, followed by CT (9.4%), ultrasound (9.4%), radiography (8.9%), and nuclear medicine (7.2%).
  • Despite the growth, just 5.6% of NPPs were interpreting images.
  • By specialty, the share of NPP interpretation was most common with primary care (40%) and orthopedic offices (34%).

The researchers also tracked variability in NPP interpretation rates by state, finding the highest rates ( ~13%) in Western states with large rural areas like Montana, Alaska, and Idaho, where presumably there are fewer radiologists available to read images. 

The Takeaway

The findings provide a good news/bad news look at non-physician image interpretation. The good news for radiologists is that NPP interpretation is still pretty rare; the bad news is that rates are growing quickly. And given the ongoing radiologist shortage, there is sure to be continuing pressure to allow allied health staff to read images on their own.

Will Congress Stop Medicare Cuts?

Radiologists find themselves once again in a familiar position, facing CMS cuts in Medicare and Medicaid physician payments for 2025. A new analysis by revenue cycle management company Healthcare Administrative Partners details the impact of the reductions, as well as other reimbursement changes set to take effect next year. 

CMS has been driving down radiology reimbursement for years, a trend widely seen as part of the agency’s effort to shift funding from medical specialties to primary care. 

  • That’s having an impact on physician pay, as a study last week found that private-practice diagnostic radiologists have seen inflation-adjusted salaries decline at a -1% annual rate since 2014. 

That trend is set to continue in 2025, with CMS publishing its final rule for the Medicare Physician Fee Schedule that affirms most of the changes it proposed in July. In the new article, HAP’s Sandy Coffta unpacks the changes, which include … 

  • A new conversion factor of $32.3465 (down from $33.2875).
  • Payment reductions of -2.8% for radiology and nuclear medicine, and -4.8% for interventional radiology.

But not all of the changes are negative. Other 2025 policies that affect radiology include …

  • Reimbursement for CT colonography for Medicare beneficiaries at a rate of $108.68 for the professional component.
  • New codes for reporting MRI safety procedures.
  • New quality category measures in the Merit-based Incentive Payment System.

CMS proposed similar cuts last year, but Congress swooped in at the last minute to roll them back with the Consolidated Appropriations Act, which applied a positive 2.93% upward adjustment. 

  • Several bills in Congress now would likewise stave off the 2025 reductions (H.R. 2474 and H.R. 10073), but time is running out to pass them before the current Congressional session expires on January 3, 2025. 

The Takeaway

Will Congress once again ride to the rescue and stave off Medicare reimbursement cuts, as it did a year ago? Or will things be different this time, given the political turbulence that’s shaking Washington, DC? We’ll find out in a few weeks.

Radiologist Salaries Lag Inflation

A new study in JACR confirms what many radiologists have suspected: salary growth for private-practice radiologists has lagged inflation over the last 10 years. While there were a few bright spots, the study mostly shows that radiologists are working harder for less pay. 

Radiology has long been one of the better-compensated medical specialties, often landing in the top 10 of disciplines with the highest average annual compensation. 

  • But radiology has also been a target for reimbursement cuts by the U.S. government as it tries to shift more Medicare and Medicaid payments to primary care practitioners.

As a result, previous studies have found that payments per Medicare beneficiary in radiology have actually declined. 

  • And another 2.83% cut is on the docket for 2025 unless Congress steps in before the end of the current legislative session to prevent cuts in the 2025 Medicare Physician Fee Schedule.

The new study analyzes radiologist compensation based on MGMA salary survey data from 2014 to 2023. 

  • Researchers compared salaries for both diagnostic and interventional radiologists, and also between private-practice and academic radiologists. 

Based on the data, they found …

  • Diagnostic radiologists saw median total compensation grow over the survey period, but at a faster rate for academic radiologists (32% vs. 18%). 
  • Academic radiologists enjoyed faster annualized salary growth (3.2% vs. 1.9%) and had an edge after adjustment for inflation (+0.3% vs. -1%).
  • Work RVUs (a measure of productivity) also grew but at a slightly higher rate for academic radiologists (21% vs. 20%). 
  • Interventional radiologists saw higher salary growth for both non-academic and academic physicians (41% and 35%). 

The findings indicate that the traditional salary gap between private-practice and academic radiologists may be narrowing.

  • The growth in wRVUs in a time of stagnant or declining salaries after inflation adjustment may confirm the suspicions of both types of radiologists: that they are working harder for less pay. 

The Takeaway

The findings could be a gut punch for private-practice diagnostic radiologists, who are finding that their salary gains aren’t keeping pace with inflation (sound familiar?). They also suggest that academic radiology could offer a refuge from the market and government forces that are reshaping the private sector.

Real-World Stroke AI Implementation

Time is brain. That simple saying encapsulates the urgency in diagnosing and treating stroke, when just a few hours can mean a huge difference in a patient’s recovery. A new study in Clinical Radiology shows the potential for Nicolab’s StrokeViewer AI software to improve stroke diagnosis, but also underscores the challenges of real-world AI implementation.

Early stroke research recommended that patients receive treatment – such as with mechanical thrombectomy – within 6-8 hours of stroke onset. 

  • CT is a favored modality to diagnose patients, and the time element is so crucial that some health networks have implemented mobile stroke units with ambulances outfitted with on-board CT scanners. 

AI is another technology that can help speed time to diagnosis. 

  • AI analysis of CT angiography scans can help identify cases of acute ischemic stroke missed by radiologists, in particular cases of large vessel occlusion, for which one study found a 20% miss rate. 

The U.K.’s National Health Service has been looking closely at AI to provide 24/7 LVO detection and improve accuracy in an era of workforce shortages.

  • StrokeView is a cloud-based AI solution that analyzes non-contrast CT, CT angiography, and CT perfusion scans and notifies clinicians when a suspected LVO is detected. Reports can be viewed via PACS or with a smartphone.  

In the study, NHS researchers shared their experiences with StrokeView, which included difficulties with its initial implementation but ultimately improved performance after tweaks to the software.  

  • For example, researchers encountered what they called “technical failures” in the first phase of implementation, mostly related to issues like different protocol names radiographers used for CTA scans that weren’t recognized by the software. 

Nicolab was notified of the issue, and the company performed training sessions with radiographers. A second implementation took place, and researchers found that across 125 suspected stroke cases  … 

  • Sensitivity was 93% in both phases of the study.
  • Specificity rose from the first to second implementation (91% to 94%).
  • The technical failure rate dropped (25% to 17%).
  • Only two cases of technical failure occurred in the last month of the study.

The Takeaway

The new study is a warts-and-all description of a real-world AI implementation. It shows the potential of AI to improve clinical care for a debilitating condition, but also that success may require additional work on the part of both clinicians and AI developers.

Time to Embrace X-Ray AI for Early Lung Cancer Detection

Each year approximately 2 billion chest X-rays are performed globally. They are fast, noninvasive, and a relatively inexpensive radiological examination for front-line diagnostics in outpatient, emergency, or community settings. 

  • But beyond the simplicity of CXR lies a secret weapon in the fight against lung cancer: artificial intelligence. 

Be it serendipitous screening, opportunistic detection, or incidental identification, there is potential for AI incorporated into CXR to screen patients for disease when they are getting an unrelated medical examination. 

  • This could include the patient in the ER undergoing a CXR for suspected broken ribs after a fall, or an individual referred by their doctor for a CXR with suspected pneumonia. These people, without symptoms, may unknowingly have small yet growing pulmonary nodules. 

AI can find these abnormalities and flag them to clinicians as a suspicious finding for further investigation. 

  • This has the potential to find nodules earlier, in the very early stages of lung cancer when it is easier to biopsy or treat. 

Indeed, only 5.8% of eligible ex-smoking Americans undergo CT-based lung cancer screening. 

  • So the ability to cast the detection net wider through incidental pulmonary nodule detection has significant merits. 

Early global studies into the power of AI for incidental pulmonary nodules (IPNs) shows exciting promise.

  • The latest evidence shows one lung cancer detected for every 1,120 CXRs has major implications to diagnose and treat people earlier – and potentially save lives. 

The qXR-LN chest X-ray AI algorithm from Qure.ai is raising the bar for incidental pulmonary nodule detection. In a retrospective study performed on missed or mislabelled US CXR data, qXR-LN achieved an impressive negative predictive value of 96% and an AUC score of 0.99 for detection of pulmonary nodules. 

  • By acting as a second pair of eyes for radiologists, qXR-LN can help detect subtle anatomical anomalies that may otherwise go unnoticed, particularly in asymptomatic patients.

The FDA-cleared solution serves as a crucial second reader, assisting in the review of chest radiographs on the frontal projection. 

  • In another multicenter study involving 40 sites from across the U.S., the qXR-LN algorithm demonstrated an impressive AUC of 94% for scan-level nodule detection, highlighting its potential to significantly impact patient outcomes by identifying early signs of lung cancer that can be easily missed. 

The Takeaway 

By harnessing the power of AI for opportunistic lung cancer surveillance, healthcare providers can adopt a proactive approach to early detection, without significant new investment, and ultimately improving patient survival rates.

Qure.ai will be exhibiting at RSNA 2024, December 1-4. Visit booth #4941 for discussion, debate, and demonstrations.

Sources

AI-based radiodiagnosis using Chest X-rays: A review. Big Data Analytics for Social Impact, Volume 6 – 2023

Results from a feasibility study for integrated TB & lung cancer screening in Vietnam, Abstract presentation UNION CONF 2024: 2560   

Performance of a Chest Radiography AI Algorithm for Detection of Missed or Mislabelled Findings: A Multicenter Study. Diagnostics 12, no. 9 (2022): 2086

Qure.ai. Qure.ai’s AI-Driven Chest X-ray Solution Receives FDA Clearance for Enhanced Lung Nodule Detection. Qure.ai, January 7, 2024

Studies Support Breast Ultrasound for Screening

A pair of new research studies offers guidance on when and where to use ultrasound for breast screening. The publications highlight the important advances being made in one of radiology’s most versatile modalities. 

Ultrasound is used in developed countries for supplementary breast cancer screening in women who may not be suitable for X-ray-based mammography due to issues like dense breast tissue.

  • Ultrasound is also being examined as a primary screening tool in developing regions like China and Africa, where access to mammography may be limited.

But despite growing use, there are still many questions about exactly when and where ultrasound is best employed in a breast screening role – and this week’s studies shed some light. 

First up is a study in Academic Radiology in which researchers compared second-look ultrasound to mammography in women with suspicious lesions found on breast MRI. 

  • Their goal was to find the best clinical path for working up MRI-detected lesions without performing too many unnecessary biopsies. 

In a group of 221 women, second-look ultrasound was largely superior to mammography with… 

  • Higher detection rates for mass lesions (56% vs. 17%).
  • A much higher detection rate for malignant mass lesions > 10 mm (89%).
  • But worse performance with malignant non-mass lesions (22% vs. 38%).

They concluded second-look ultrasound is a great tool for assessment and biopsy of MRI-detected lesions > 10 mm without calcifications. 

  • It’s not so great for suspicious non-mass lesions, which might be better sent to mammography for further workup. 

Breast ultrasound of non-mass lesions was also the focus of a second study, this one published in Radiology

  • Non-mass lesions are becoming more frequent as more women with dense breast tissue get supplemental screening, but incidence and malignancy rates are low. 

So how should they be managed? In a study of 993 women with non-mass lesions found on whole-breast handheld screening ultrasound, researchers classified by odds ratios the factors indicating malignancy…

  • Associated calcifications (OR=21.6).
  • Posterior shadowing (OR=6.9).
  • Segmental distribution (OR=6.2).
  • Mixed echogenicity (OR=5.0).
  • Larger size (2.6 vs. 1.9 mm).
  • Negative mammography (2.8% vs. 29%).

The Takeaway

Ultrasound’s value comes from its high prevalence, low cost, and ease of use, but in many ways clinicians are still exploring its optimal role in breast cancer screening. This week’s research studies should help.

CT Lung Screening’s Weak Link

CT lung cancer screening rates in the U.S. remain abysmally low, over a decade after the exam was recommended. Is part of lung screening’s problem its reliance on provider referrals? A new research letter in JAMA Network Open examines this question. 

Unlike breast screening, in which eligible women are able to self-refer themselves for exams, CT lung screening revolves around provider referrals to start the process. 

  • CMS requires a shared decision-making session that results in a written order from a practitioner for a CT lung screening exam in order to pay for screening through Medicare and Medicaid. 

When CMS created the rules in 2015, provider referrals and shared decision-making were seen as ways to get patients involved in their own care by making choices in coordination with their caregivers.

  • But many are starting to see the requirements as a barrier, especially given low CT lung screening rates in the U.S.

In the new article, researchers investigated how easy it would be for an eligible individual to secure a CT lung screening appointment by just calling hospitals – without a provider referral. 

  • They note that one-third of Americans don’t have primary care clinicians, and are often told to call hospitals directly to set up appointments.

So they did just that, placing phone calls to 527 hospitals asking to arrange CT lung screening appointments, finding …

  • 317 calls (60%) failed because the caller did not have a primary care provider’s order.
  • Only 51 hospitals (9.7%) were able to connect callers to any component of a lung cancer screening process. 

The study authors note that the provider referral requirement isn’t the only thing holding CT lung cancer screening back, as even patients with primary care providers aren’t getting screened, and managing nodule follow-up can also be challenging. 

  • But Medicare’s cumbersome reimbursement rules certainly don’t help bring new people into the fold.

The Takeaway

Given CT lung cancer screening’s undisputed life-saving value, there’s no reason to put unnecessary barriers in its way. The provider referral and shared decision-making requirements are lung screening’s weak link to securing greater adoption, and CMS should rescind them to put CT lung cancer screening on the path to greater adoption.

Get every issue of The Imaging Wire, delivered right to your inbox.

You might also like..

Select All

You're signed up!

It's great to have you as a reader. Check your inbox for a welcome email.

-- The Imaging Wire team

You're all set!