Agentic AI has quickly become one of the hottest topics in radiology. But what is it really good for? Texas researchers offer one possible use case in a new study in NEJM Catalyst: scouring radiology reports to identify patients who require follow-up.
Agentic AI is a new flavor of artificial intelligence that’s capable of working autonomously to complete tasks with minimal human supervision.
- In healthcare, it’s being applied to a wide range of tasks, from improving health system operations to clinical and administrative jobs.
In the current study, researchers from Parkland Health in Dallas assigned agentic AI to one of the trickiest tasks in radiology: making sure patients with suspicious findings comply with recommendations for follow-up procedures.
- Previous studies have documented low rates of adherence to radiologist recommendations for follow-up imaging (possibly as low as 50%), creating the uncomfortable possibility of missed opportunities that could have major patient-care ramifications.
The dilemma can be compounded with the use of structured note templates in EHRs, as improper use or modification of these macros can lead to missed notifications.
- To address the problem, Parkland clinicians developed an AI agent based on a pretrained open-source large language model (Meta’s LLM Llama 3 70B) that reviews clinical impressions, extracts important details for follow-up, and integrates its findings into departmental workflow to enable patient outreach.
In tests on 10k radiologist notes, Parkland researchers found that their AI agent…
- Had an overall detection rate of ~5.1%, slightly lower than other published studies (8% to 12%).
- Had far higher sensitivity than Parkland’s previous macro-based follow-up notification system (99% vs. 16%), correctly flagging 6X more cases (513 vs. 83).
- Achieved higher accuracy (99% vs. 58%), and 94% accuracy for characterizing follow-up timing, recommended procedure, and underlying abnormality.
Considering Parkland’s annual volume of 500k imaging studies, the AI agent could identify 21.5k follow-up cases a year.
- Many of these could be serious issues, such as new cancer diagnoses or pathologies that require surgical intervention.
The Takeaway
The new study shows that agentic AI isn’t some technogeek’s far-off dream – it’s a useful tool on the verge of real-world implementation, with the potential to improve patient care without overburdening radiology staff.

