What that means for medical AI models is that any patient whose data is used to educate the bot could be exposed, leading to details about their medical history and diagnoses being leaked. In an analysis of seven medical AI datasets consisting of images, ECG records, and general electronic health records, the team determined that individual patients targeted by such attacks can be identified with “near-perfect attack success,” which they explain flies in the face of how such models are evaluated for safety. 

“The fact that MIAs can achieve near-perfect success rates for individual patients is not adequately captured by the standard evaluation protocol, which measures attack success in aggregate across records,” the researchers said. Based on their findings, they conclude, reporting standards for AI privacy audits need to change. 

It gets worse, too: Patients in the dataset are generally easy to identify and, unsurprisingly, those underrepresented in medical AI training data are even easier to finger than those whose data doesn’t stand out.