Adverse events from marketed AI/ML devices (Babic et al., 2025)

Boris Babic, I. Glenn Cohen, Ariel Dov Stern, Yiwen Li and Maxime Ouellet, A general framework for governing marketed AI/ML medical devices, npj Digital Medicine 8:328 (2025), doi:10.1038/s41746-025-01717-9. Read via PubMed Central. Brought in by the research pass for growth edge 6a — the spoke's standing question of whether any field has both a harm count and something to divide it by.

Medical AI is the one place where the answer looked likely to be yes. The FDA runs a mandatory reporting duty, publishes the roster of devices it has authorised, and publishes the reports themselves in MAUDE. So this is the field's best case, and the paper is what the best case yields.

The numbers

• 950 AI/ML devices authorised by the FDA as of August 2024. A public roster, updated.
• The authors' final dataset is 823 unique 510(k)-cleared devices linked to 943 adverse events (medical device reports, MDRs) between 2010 and 2023.
• The reports are thin where it matters. Event Location is missing for every MDR in the sample — 369 blank, 574 marked "No information". Reporter health-professional status is missing in 73%, event date in 32%, reporter occupation in 30%.

The sentence the edge was hunting

"Without an overall 'denominator' for the frequency of a medical device's use, it is not possible to talk about the relative safety of one product versus another."

That is the wall, stated by the people who did the work, inside the one field with a mandatory duty and a published device roster. The FDA knows what it authorised; nobody knows how often any of it ran. 943 reports over fourteen years is a numerator with nothing under it. A device used ten times and a device used ten million times produce the same kind of row.

What the authors propose instead

Not a rate. Two things, both reporting rather than counting. MDRs extended with the three failure modes the current form cannot express (concept drift, covariate shift, algorithmic stability), plus quarterly updates on deployment conditions. And nutrition-style labels for a device's operating envelope, read alongside the reporting rather than instead of it.

The proposal is an admission. If a rate were reachable through this system, the fix would be to compute it. They are asking for richer descriptions of individual events because the arithmetic is unavailable.

Where this leaves the question

A half close of edge 6a, and the informative half. The spoke can now say something stronger than "no dataset carries deployments": in the field with the strongest legal duty to report, a peer-reviewed team went looking and reported the denominator missing. ai-incident-database fails as a rate for its own reasons — press attention as coverage, no severity scale, namespaced taxonomies. Those looked like the failings of a volunteer project. This paper shows the same hole in a federal database built under statute, which makes it a property of the instrument rather than of the funding.

One caution the paper is explicit about, and it cuts against reading 943 as small: MAUDE is passive-surveillance and under-reported, and (the AI-specific part) "many problems caused by medical AI/ML devices may not even occur as a result of malfunction, in the ordinary sense of that word." A model that quietly drifts wrong on a new patient population does not look like a broken device to the person filling in the form.

Related

ai-incident-database · risk-based-regulation · eu-ai-act · nist-ai-rmf · ai-governance · synthesis

Denominator, partly found (2026-09-27). ai-device-cpt-claims (Wu et al., NEJM AI 2024) counts billed uses of AI devices through CPT codes. Only AI for coronary artery disease and for diabetic retinopathy pass 10,000 claims, so a rate is now possible for those two classes and no others.