Researchers from Imperial College London, the Technical University of Munich, and the Hasso Plattner Institute have published findings in Nature showing that artificial intelligence models trained on medical data can be exploited to identify specific individual patients — sometimes with near-perfect accuracy, even when those patients' records have been de-identified.
The research represents one of the first patient-level privacy audits of medical AI, and its findings challenge a foundational assumption in healthcare AI development: that removing names and other identifiable information from medical records is sufficient to protect patient privacy when that data is used to train AI models.
The patients most at risk of being identified are those who are most different from the average in a dataset — people with rare diseases, unusual medical presentations, Medicaid patients, and members of racial minority groups who are underrepresented in large clinical datasets.
Why This Matters
Millions of Americans have had their de-identified medical records incorporated into datasets used to train AI systems for tasks like cancer detection, disease prediction, and diagnostic imaging analysis. Many of these patients never knew their data was being used. Many assumed that de-identification — the removal of names, birthdates, and Social Security numbers — made their records effectively anonymous.
This study demonstrates that an assumption can be wrong. An attacker who uses a "membership inference attack" — a method to determine whether a specific individual's data was used to train a model — can successfully identify certain patients with near-perfect accuracy, even when the overall performance of such an attack looks close to random guessing at the aggregate level.
The practical implication is that your de-identified medical record in a hospital's research dataset may still be linkable to you if you have an unusual disease, a distinctive combination of conditions, or if you belong to a group that is statistically underrepresented in large datasets.
What We Know So Far
The researchers conducted one of the first patient-level privacy audits of AI models for medical diagnostic applications, focusing on membership inference attacks — which seek to determine whether the data of a given individual was used to train a model.
The team studied seven large, real-world clinical datasets including medical images, electrocardiograms, and electronic health records. They trained around 200 versions of an AI model for each dataset, then quantified, for every single record and patient, how accurately an attack would be at guessing whether a patient was part of the training set. They showed that membership inference attacks can be almost perfectly successful for some individual patients — such as those with an unusual disease or presentation — even though the average attack performance across the whole training set looked close to random guessing.
The paper was published in Nature in 2026, with DOI: 10.1038/s41586-026-10688-0.
Who Is Most at Risk
The researchers highlight that standard privacy evaluations can overlook serious risks to individual patients, particularly those from underrepresented groups — and that these risks may grow as models become more powerful.
The groups facing the highest re-identification risk include:
- Patients with rare diseases — whose records are statistically distinctive in large datasets
- Patients with unusual symptom combinations or imaging findings
- Racial and ethnic minorities who are underrepresented in the training data, making their records statistically stand out
- Medicaid patients in datasets dominated by commercially insured patients
- People who contributed multiple records (visits, scans, labs) to the same dataset over time
Conversely, patients whose records are similar to many others in a dataset — for example, a middle-aged white male with a common chronic condition in a large hospital system — face lower individual re-identification risk from the same attack.
What Doctors and Experts Say
Researchers from Imperial College's Department of Computing, led by PhD student Moritz Knolle and professors Ben Glocker and Daniel Rueckert, conducted the core analysis. Professor Stefanos Zafeiriou, Head of the Department of Computing at Imperial College London, noted: "This work shows that standard privacy evaluations can overlook serious risks to individual patients, particularly those from underrepresented groups, and that these risks may grow as models become more powerful."
Professor Georg Kaissis of the Hasso Plattner Institute noted that effective safeguards exist: "Differential privacy introduces small modifications into the training data that do not affect the model's calculations but make [membership inference attacks] significantly more difficult." The researchers advocate for patient-level privacy auditing of AI models before their release — a step that is not currently standard practice.
What the Evidence Shows — and What It Does Not
MedicalDaily Evidence Check
- Study type: Patient-level privacy audit using membership inference attacks across seven clinical datasets
- Datasets examined: Medical images, electrocardiograms, and electronic health records (seven large real-world clinical datasets)
- Models trained: Approximately 200 AI model versions per dataset
- What it found: Near-perfect re-identification success rates for individual patients with unusual or distinctive health records, even when aggregate attack performance appeared near-random
- What it does not prove: It does not demonstrate that any specific hospital's AI model has already been exploited to identify patients — it shows the vulnerability exists and can be demonstrated, not that it has been exploited in practice
- Published in: Nature (2026); DOI: 10.1038/s41586-026-10688-0
- What readers should know: De-identification is not the same as anonymization for patients with distinctive health characteristics
What You Can Do Now
- Ask about data use when you sign hospital or research consent forms. Many patients sign broad consent forms that include data use for research or AI development. Ask specifically whether your de-identified records may be used to train AI models and what privacy protections are in place.
- Request a privacy opt-out. Under HIPAA, patients have the right to request restrictions on certain uses of their health information, though exceptions apply for research. Ask your health system's privacy officer what options exist.
- If you have a rare condition, ask directly. Patients with rare diseases face the highest re-identification risk in this study. Asking your care team or hospital research office about differential privacy safeguards is a reasonable step.
- Support policy accountability. Advocacy for stronger AI privacy standards — including mandatory patient-level auditing before model deployment — is the systemic response this research calls for.
Cost and Access: What Patients Should Know
The privacy risks described in this study pertain to how hospitals and research institutions handle patient data when building or licensing AI systems. Patients who want to understand how their health information is used for AI training can request their Notice of Privacy Practices from their health system. Under HIPAA, institutions must provide this document and describe research-related uses of patient data.
What Happens Next
The Nature publication is expected to increase pressure on AI developers and health systems to conduct patient-level privacy audits before deploying medical AI models. Regulatory bodies, including the FDA — which oversees AI-based medical devices — and the Office for Civil Rights within HHS, which enforces HIPAA, may consider updated guidance on privacy requirements for AI training data. Researchers are continuing to develop and validate differential privacy techniques that can be applied during AI training to reduce re-identification risk without significantly impairing model accuracy.
The Bottom Line
A rigorously conducted study published in Nature has demonstrated that de-identified medical data used to train AI systems can still be used to identify individual patients — and that patients with rare diseases, unusual medical presentations, or underrepresented demographic characteristics face the highest risk. Current standard approaches to de-identification are insufficient protection for the most vulnerable patients. Healthcare AI developers should conduct patient-level privacy audits before model deployment.
