Psychiatry is the only branch of medicine with essentially no diagnostic tests. There is no blood panel for depression, no scan for bipolar disorder, no biomarker that separates generalised anxiety from an entirely reasonable response to a difficult life. Diagnosis is a structured conversation, and the categories being applied are consensus constructs rather than discovered natural kinds.
That is the gap machine learning is being aimed at, and it explains both the genuine enthusiasm and the specific ways this field goes wrong.
Start With the Label Problem
I build medical machine learning systems, and the first question I ask about any clinical model is not how accurate it is. It is: accurate against what?
In radiology there is usually something solid underneath. Biopsy results. Surgical findings. A follow-up scan that settles the question. The label is imperfect but it is anchored to something outside the model.
In mental health, the label is almost always a questionnaire score. A model trained to detect depression from speech is, in nearly every published study, trained to predict whether someone scores above a cutoff on the PHQ-9 or a similar instrument. This has an unavoidable consequence: the model can never be more valid than the questionnaire. If the PHQ-9 misclassifies someone, a model trained on PHQ-9 labels learns to misclassify them the same way. It is not detecting depression. It is predicting a screening score from a different input modality.
That is not useless — a screening score you can obtain from twenty-five seconds of speech is operationally very different from one requiring a clinic visit. But it is a much narrower claim than "AI diagnoses depression", and the distinction gets lost constantly.
It is worth adding that the reference standard itself is shakier than outsiders assume. When the DSM-5 criteria were tested in field trials, inter-rater agreement between trained clinicians was mediocre for several common conditions, major depressive disorder among them. Two psychiatrists interviewing the same patient often disagree. Any model is being trained against that noise.
What the Evidence Actually Shows
Here is a real number from a real study, which I offer as a corrective to the accuracy figures that circulate in this space.
A 2025 study in Annals of Family Medicine evaluated a voice biomarker tool for detecting signals consistent with moderate to severe depression. Participants answered "How was your day?" and completed a standard depression questionnaire; the analysis covered more than fourteen thousand recordings. From roughly twenty-five seconds of speech, the model achieved sensitivity of 71.3 percent and specificity of 73.5 percent.
Sit with those numbers, because they are what a large, properly conducted study of this technology looks like. Not 93 percent. Around seventy percent on both axes.
That is not a failure. For a screening tool that costs nothing and takes half a minute, in a primary care system where the great majority of patients are never screened for depression at all, seventy percent is potentially worth having. The authors are explicit that it is an adjunct to screening, not a diagnostic replacement.
But now do the arithmetic that almost never appears alongside these figures. Suppose ten percent of the screened population has moderate to severe depression. Out of a thousand people, a hundred have it and the model correctly flags about seventy-one. Of the nine hundred who do not, it incorrectly flags about two hundred and thirty-nine. So of roughly three hundred and ten people flagged, fewer than one in four actually meets the threshold.
This is base rate arithmetic, it applies to every screening test ever built, and it is the single most important thing to understand about AI screening claims. A test with respectable sensitivity and specificity produces mostly false positives when the condition is uncommon. That does not make the test bad. It makes the downstream pathway — what happens to someone who gets flagged — the part that determines whether the system helps or harms.
Where the Signals Come From
The underlying science is more interesting than the product claims.
Speech carries a lot. Depression is associated with slower speech, reduced pitch variation, longer pauses; psychomotor retardation is a diagnostic criterion, and it shows up acoustically. Models pick up prosodic features, spectral features, timing. The signal is real and has been replicated across research groups.
Language content is a separate channel from acoustics. Increased first-person singular pronoun use, more absolutist vocabulary, shifts in temporal focus — these correlate with depressive states in study populations. The effect sizes are modest at the individual level even when they are robust in aggregate, which is a distinction that gets flattened when a group finding becomes a personal prediction.
Digital phenotyping — passive sensing from a phone — tracks movement, sleep timing, communication frequency, screen patterns. The appeal is that it observes behaviour continuously without anyone having to report anything. The problem is that these signals are enormously confounded by ordinary life. Reduced movement and fewer outgoing messages might indicate a depressive episode, or a deadline, or a head cold, or a holiday. The research here is genuinely promising and the replication record across independent cohorts is uneven.
Suicide Prediction Deserves Its Own Warning
This is where I would push back hardest against the prevailing optimism.
A large meta-analysis of decades of suicide risk research found that our predictive accuracy is only marginally better than chance, and that this had not meaningfully improved over the fifty years surveyed. Machine learning has improved on the classic clinical risk factors somewhat, which is a low bar and a real result.
The problem is the base rate. Suicide is, statistically, extremely rare. Combine a rare outcome with an imperfect classifier and the positive predictive value collapses — you generate an enormous number of false alarms for every case identified. If the response to a flag is a gentle check-in and an offer of support, that may be an acceptable trade. If it is an involuntary hold, a police welfare visit, or a note in a record that follows someone, the harm from false positives is severe and falls on people who were never at risk.
I am not arguing against building these systems. I am arguing that the consequence attached to a positive flag has to be proportionate to how often that flag is wrong, and that is a design decision, not a modelling one.
The Chatbot Evidence Is Softer Than Advertised
Conversational agents delivering CBT techniques are the consumer face of this field, and the marketing tends to cite "clinical evidence" without characterising it.
A representative example: a 2024 randomised controlled trial in JMIR Formative Research compared a CBT-based therapy chatbot against a self-help book in young adults with subclinical anxiety or depression. Symptoms improved in both groups and remained improved at one-month follow-up. There was no significant difference between the chatbot and the book on most outcomes. One interesting secondary finding: frequent users of the chatbot reported reduced loneliness, which the book group did not.
That is a useful, honest result, and note what it implies. The chatbot worked. It also did not outperform a control that costs nothing and requires no technology. Many chatbot trials use waitlist controls, against which almost any active intervention looks effective. When the comparator is something real, the advantage tends to shrink.
The sample was small and the population subclinical, so this is one study rather than a verdict. But it is the kind of study that should shape expectations more than a vendor's landing page does.
Regulatory Reality
To be precise, because this is widely misstated: as far as I am aware, no AI system is authorised to diagnose a psychiatric disorder. The FDA has cleared or authorised various digital health products in mental health — including a prescription video-game-based treatment for paediatric ADHD — but authorisation as a treatment device, or as a screening adjunct, is not authorisation to diagnose.
Most tools on the market operate as wellness products, which sit largely outside device regulation, or as clinical decision support that stays inside the exemption by presenting information for a clinician to evaluate rather than issuing a conclusion. When you see "FDA cleared" attached to a mental health AI product, read what exactly was cleared and for what indication. The gap between the cleared indication and the marketing claim is often wide.
What Would Make This Genuinely Useful
A few things, none of which are model improvements.
Validate against something other than the questionnaire you trained on. Longitudinal outcomes, treatment response, structured clinical interview by raters blinded to the model — anything that breaks the circularity.
Test across populations before deployment, not after. Speech models are sensitive to language, accent, age and recording device. Expression of psychological distress varies substantially across cultures, and a model trained on one population encodes that population's presentation as the norm. This is the same distribution-shift problem I deal with in imaging, and it is worse here because the phenotype itself is culturally shaped.
Report calibration, not just accuracy. A screening tool that reliably communicates how uncertain it is on a given case is far more useful clinically than one that is slightly more accurate on average and equally confident about everything.
Design the pathway before the model. What happens to a flagged person? Who reviews it, how fast, and what can they offer? A screening system that generates flags into a service with a six-month waiting list has not helped anyone and may have harmed people by telling them something alarming they cannot act on.
The Honest Summary
Access is the actual crisis in mental health care. Most people who need help do not get it, most primary care patients are never screened, and waiting lists are long everywhere. A cheap, fast, scalable screening signal is a real contribution to that problem, and voice and behavioural markers are legitimate science, not hype.
What they are not is objective measurement of an internal state. These systems detect correlates of a questionnaire score, imperfectly, in populations that may not resemble yours. Used as triage into human care, with proportionate responses and honest communication of uncertainty, that is valuable. Used as a verdict, it will produce a great many confident, incorrect statements about people's minds.
If you are struggling, please speak to a professional. These tools are supplements to human care, not substitutes for it.
Tags
Taresh Sharan
support@sharaninitiatives.com