Diagnostic Error Is Not Mostly a Perception Problem

Diagnostic error is a large, measurable and stubbornly persistent problem. One widely cited synthesis of three large observational studies put the rate of diagnostic error in US outpatient care at around 5 percent — roughly 12 million adults a year, with the authors estimating that about half of those errors have the potential to cause harm.

It is natural to look at a number like that and see a machine learning problem. Humans are inconsistent, get tired, anchor on first impressions and cannot hold the whole literature in their heads. Algorithms are consistent, do not tire, and can be trained on more cases than any individual will ever see.

I work on these systems, and I think that framing gets the diagnosis of the problem wrong in a way that leads to building the wrong things. Most diagnostic error is not a perception failure that a better classifier fixes. This piece is about where the errors actually come from, and about the one design decision that determines more than anything else whether a clinical model helps or harms.

Where diagnostic errors actually occur

Break down a missed or delayed diagnosis and it rarely reduces to "the clinician looked at the finding and misread it."

A great deal of it is process. The test was never ordered because the differential did not include the condition. The test was ordered and the result came back after the patient was discharged, into an inbox nobody triaged. The referral was made and the patient never got an appointment. The abnormal result was seen, noted, and then lost between two teams at a handover.

Some of it is information. The clinician did not have the prior imaging, or the records from the other hospital, or the medication list, and made a reasonable decision on incomplete data.

Some of it is cognitive in the way the literature describes: premature closure, anchoring on the first plausible explanation, availability bias after a recent similar case. This is real and well documented.

And some of it is perception — the finding was there in the image and was not seen. This is the slice that detection models address directly, and it is a genuine slice. It is not the majority.

The implication is not that imaging AI is useless. It is that measuring a model's sensitivity against a reference standard tells you how well it addresses one contributing cause of one part of the problem. Improvements in follow-up tracking, result routing and handover communication plausibly prevent more harm per unit of effort than any classifier, and they are unglamorous enough that nobody writes press releases about them.

The target variable problem

Here is the decision I care about most when I look at a clinical model, and the one that is most often made carelessly: what is the thing the model was trained to predict?

Almost never is the label the clinical outcome you care about. It is a proxy for it — something recorded in the data that correlates with the thing you want. Choosing that proxy is a modelling decision with ethical consequences, and it is usually made on grounds of data availability.

The definitive demonstration of this is a 2019 Science paper analysing a commercial algorithm used by US health systems to identify patients who would benefit from extra care management. The researchers found substantial racial bias: at any given algorithm risk score, Black patients were considerably sicker than White patients. Correcting the disparity would have raised the proportion of Black patients identified for additional help from about 17.7 percent to about 46.5 percent.

The algorithm did not use race as an input. It was not badly fit. It was trained to predict future healthcare costs, on the reasonable-sounding theory that sicker people generate more cost. But unequal access means less money is spent on Black patients at the same level of illness. The model learned that relationship faithfully and reproduced it. It was an accurate model of the wrong quantity.

I find this the most important paper in the field for people who build these systems, because nothing about it would have been caught by better validation methodology. The model would have passed every accuracy check. The error was upstream of the modelling, in the sentence that said "we will use cost as a proxy for need."

The equivalents in imaging are everywhere once you look. Training a model on radiology report text means learning what radiologists wrote, including their systematic omissions. Training on biopsy-confirmed labels means learning about the population who got biopsied, which is not the population you will screen. Training on follow-up-confirmed outcomes means excluding the patients who never came back, who are not a random sample.

Ask what the label is, and how it was generated, before you ask how accurate the model is.

Why accuracy improvements do not automatically become outcome improvements

Suppose the model genuinely is better at detection. Several things still have to be true before a patient is better off.

The finding has to be actionable. Detecting something earlier only helps if earlier treatment changes the trajectory. For some cancers it clearly does; for others, earlier detection mostly extends the period during which the patient knows they are ill. Screening research has a long and painful history with lead-time bias and overdiagnosis, and imaging AI walks straight into it by making detection cheaper.

The pathway has to absorb the output. More detections means more follow-up imaging, more biopsies, more clinic slots. A system that increases detections by a fifth in a service already at capacity creates a queue, and queues have their own mortality.

The reader behaviour has to not degrade. If the presence of a reliable model makes readers less thorough on unflagged studies, the net effect can be negative even with a good model. This is measurable and rarely measured.

And the improvement has to survive the local population. Performance established on one case mix does not transfer automatically.

None of this argues against building the models. It argues that "improves diagnostic accuracy" and "improves patient outcomes" are separate claims requiring separate evidence, and that the literature is heavily weighted towards the first while the marketing talks about the second.

What good practice looks like

A few things I would hold any clinical AI deployment to.

State the target variable and defend it. If the label is a proxy, say what it is a proxy for and what could drive a wedge between them.

Report performance by subgroup, always. Aggregate metrics conceal exactly the disparities that matter, and the groups at risk of being concealed are the ones already worst served.

Validate locally before go-live and monitor continuously after. Vendor numbers are a starting hypothesis about your site, not a finding about it.

Keep a human accountable for the decision, and design the interface so that disagreeing with the model is easy and gets recorded. That record is both a safety mechanism and the feedback data the next version needs.

Measure the downstream consequences, not just the detections. Follow-up volume, biopsy rate, time to treatment, and the yield of those extra procedures.

The part I actually believe

The realistic contribution of AI to diagnostic accuracy is narrower than the promise and still worth pursuing. Consistency is a genuine gift: a model applies the same criteria at 4pm on a Friday as at 9am on a Monday, and inter-reader variability is a real source of harm. Never getting tired is worth something. Flagging the urgent study so it is read first is worth a great deal.

What these systems will not do is fix a diagnostic process whose failures are mostly organisational. If a health system is losing abnormal results in an inbox, buying a better classifier is solving the wrong problem expensively.

The honest version of the pitch is: this addresses one contributing cause, it addresses it reasonably well within a narrow scope, and whether it improves outcomes depends almost entirely on the system it is dropped into. That is a less exciting claim. It also happens to be the one supported by the evidence.