XAI in High-Stakes Domains
Applying explainability in healthcare, finance, and criminal justice — where decisions have real consequences
The Stakes Are Different
In most ML applications, a wrong prediction costs some revenue or causes some friction. In high-stakes domains, wrong predictions affect human lives, freedoms, and health outcomes. The explainability requirements aren't just ethical — they're practical. A radiologist who sees 'high risk' from an AI with no explanation has no basis to accept or reject the finding. An explanation that points to specific regions of the scan gives them something to work with clinically.
Healthcare: Grad-CAM for medical imaging
Gradient-weighted Class Activation Mapping (Grad-CAM) produces heatmaps highlighting which regions of an image drove the model's prediction. For chest X-ray analysis, Grad-CAM maps should highlight the clinically relevant region — if they highlight the corner where a hospital ID tag is visible, the model is using spurious correlations.
Finance: Counterfactual explanations for credit
'Your loan was denied because your debt-to-income ratio is 0.52' is a fact. 'Your loan would have been approved if your debt-to-income ratio were below 0.45' is actionable. Counterfactual explanations tell people what they could change to get a different outcome.
Criminal justice: The COMPAS case revisited
Risk scores used in sentencing require explanations that pass legal scrutiny. The factors used must be legally permissible (no race, gender), and the relative weights must be defensible. COMPAS's failure was partly an explainability failure: defendants and their lawyers could not understand or challenge the score.