I spend most of my working life building machine learning systems for medical imaging, and the question I get asked most often — at conferences, by clinicians, occasionally by relatives at weddings — is some version of "so when do the radiologists get replaced?"
The honest answer is that this is the wrong question, and the way it gets asked usually reveals a misunderstanding of what these systems actually do.
A useful reference point is the 2020 Nature paper on AI for breast cancer screening. A Google Health team trained a model on mammograms from the UK and the US and evaluated it retrospectively against the readings that had actually been recorded for those women. On those datasets the model produced fewer false positives and fewer false negatives than the original single-reader interpretations, and in a smaller reader study it outperformed the six radiologists who took part. That study got headlines. What the headlines mostly skipped is that this was a retrospective evaluation on curated historical data, that the six readers were working without the clinical context or prior imaging they would normally have, and that the authors themselves framed the result as an argument for AI-assisted reading rather than AI-replaced reading. A later commentary in the same journal pointed out that the code and models were not released in a form that let anyone reproduce the work.
That gap — between a strong retrospective result and a system that improves care in a real department — is the whole story of AI in radiology, and it is where I would focus your attention.
What These Systems Actually Do in a Workflow
When people picture AI in radiology they picture a computer looking at a scan and producing a diagnosis. Almost nothing deployed today works like that.
The dominant pattern is triage and flagging. An emergency department might accumulate a few hundred chest radiographs overnight. A triage model scans each one as it lands in the archive and marks the subset that look like they contain a pneumothorax or a large consolidation. Those studies move up the worklist. The radiologist still reads every single one of them — nothing is skipped, nothing is auto-reported — but the ones most likely to be time-critical get read first instead of in arrival order.
Read that again, because it is the part that matters: the clinical value is in reordering the queue, not in making the call. Nobody's diagnosis is being replaced. What changes is how long a critical finding sits unread.
The second common pattern is measurement. Quantifying ventricle volume, segmenting a liver, tracking a lesion's longest diameter across four prior studies, counting vertebral levels. These are tasks where a human is accurate but slow and inconsistent, and where a model is fast and repeatable. The output goes into the report as a number the radiologist confirms or corrects.
The third is reporting support — structured templates pre-filled from the measurements above, speech recognition that understands radiological vocabulary, automatic comparison against priors. Unglamorous, widely deployed, and probably responsible for more minutes saved per day than anything involving deep learning on pixels.
Where the Evidence Is Strongest
Stroke triage is where I think the case for deployment is clearest, and it is worth understanding why: the bottleneck there is speed, not diagnostic difficulty. When someone presents with stroke symptoms, every minute before reperfusion costs neurons. Software that reads the CT angiogram as soon as it reconstructs and pushes an alert to the interventional team's phones can shave meaningful time off the door-to-groin interval, not because it is a better reader than the neuroradiologist but because it does not have to wait for anyone to open the study. Viz.ai's LVO software received FDA clearance through the De Novo pathway in 2018 specifically as a notification tool — the clearance language is explicit that it does not replace review of the images. Published evaluations from hospitals using these systems have generally reported reductions in notification and transfer times. Whether that translates into better functional outcomes at ninety days is a harder question and the evidence there is thinner.
Oncology imaging is the area with the most research activity and the most inflated claims. Lung nodule detection and characterization, breast screening, prostate MRI — all have credible published models. Most of the reported accuracy figures come from retrospective evaluation on datasets assembled by the same group that built the model. When those models get tested on data from a different scanner, a different hospital, or a different patient population, performance drops, sometimes a lot. This is not a scandal; it is a normal property of supervised learning on medical data. It just means you should treat any single-institution accuracy number as an upper bound rather than an expectation.
What Actually Breaks
Having built and shipped these things, here is my honest ranking of what causes trouble, which is close to the inverse of what gets discussed at conferences.
Distribution shift is the fundamental technical problem. A model learns the joint distribution of the images it was trained on, including things nobody intended it to learn — the scanner manufacturer, the reconstruction kernel, the positioning conventions of a particular technologist, the fact that sicker patients at that site got imaged with a specific protocol. Move it to a new site and some of those correlations no longer hold. The only real defences are diverse training data, external validation before deployment, and monitoring after deployment. The third one is the one most teams skip.
Integration is where good models die. A tool that requires the radiologist to leave PACS, log into a separate viewer, and manually associate a result with a study will not get used, no matter how good it is. Alerts that fire too often produce the same fatigue that pulse oximeter alarms do in an ICU: people learn to dismiss them without looking. I have seen more value destroyed by bad workflow design than by bad model performance.
Explainability matters, but not in the way the literature suggests. Saliency maps are the standard offering, and they are weak evidence — they show where the network's output was sensitive to input perturbation, which is not the same as showing the reasoning, and they can look entirely plausible for a model that is right for the wrong reason. What clinicians actually seem to want is not a heatmap but calibration: a system that reliably says "I am unsure about this one," and is correct about when it is unsure. That is a solvable problem and an underinvested one.
Regulation is genuinely hard for a reason. A model whose weights are frozen at clearance is safe to evaluate but goes stale as scanners and protocols change. A model that keeps learning after deployment is more useful and much harder to guarantee. The FDA's predetermined change control approach is an attempt to square this, and it is real progress, but it puts a substantial monitoring burden on the manufacturer.
What This Means If You Work in the Field
The radiologists I work with who are most effective with these tools are not the ones who trust them and not the ones who dismiss them. They are the ones who have developed a calibrated sense of where a particular tool is reliable and where it is not — which is exactly the skill they already apply to a junior trainee's preliminary read.
The workload argument for adoption is stronger than the accuracy argument and gets discussed less. Imaging volume per radiologist has been climbing for years. A department that can shave time off routine measurement and get critical findings read sooner has solved an operational problem, whether or not any model ever beats a human on a benchmark.
Most tools in clinical use today are narrow: one finding, one modality, one body part. The path from that to something general is not obvious, and I would be sceptical of anyone who tells you it is imminent. But the direction is clear enough, and understanding what these systems really do — and where they quietly fail — is now part of the job.
Tags
Taresh Sharan