Anonymizing Faces, Preserving Anomalies
Deep-Learning Based Privacy Protection for Data of Patients with Cleft Lip
Karen Rosero, Lucas M. Harrison, Alex A. Kane, Carlos Busso, Rami R. Hallac
15th International Congress on Cleft Lip/Palate and Related Craniofacial Anomalies (CLEFT 2025), Kyoto, Japan
In collaboration with UT Southwestern Medical Center and Children’s Health.
Why anonymize clinical photos?
Facial photographs support treatment planning for patients with cleft lip: lip marking, assessment of surgical outcomes, facial symmetry analysis, and automatic detection of lip abnormalities. Using facial images in medical contexts also raises critical ethical and legal concerns about patients’ privacy, which limits how these images can be shared for research and education.
Traditional anonymization falls short
Black bars over the eyes, pixelation and blurring are the usual ways to de-identify a face. Generative AI can reconstruct a highly similar face from any of them, so they do not reliably protect the patient.
Our approach: replace the face, keep the anomaly
We preserve the orofacial anatomy of the patient and use a generative AI system to synthesize the rest of the face. The result is a realistic portrait of a person who does not exist, with the clinically relevant region left exactly as it was.
- Landmarks. Facial landmarks are detected automatically on the original image, using our landmark detector for patients with repaired cleft lip and palate (FG 2024).
- Mask. A mask isolates the orofacial region of interest. It can be adjusted manually or personalized for other facial regions.
- Synthesis. A diffusion model, guided by the orofacial region and a text prompt (for example, “light skin girl, green eyes”), generates the new face by progressive denoising. The region of interest stays unaltered, and the text controls the appearance of the synthetic face.
Results
The approach generalizes to both pre- and post-operative cleft lip cases and produces realistic, anatomically consistent synthetic faces.
We anonymized 140 images of patients with cleft lip, evenly distributed between pre- and post-operative cases, and asked two questions.
Is the generated face recognized as the same person? Each original image was compared with 8 synthesized variants using a deep-learning identity verification tool, for a total of 1,120 generated images. 98.9% were classified as distinct individuals. The remaining 1.1% were flagged as potential matches, all with low confidence (below 70%).
Is the orofacial region preserved? A clinical evaluator rated 50 images on a 1–5 scale, where 5 means high anatomical preservation after anonymization. 48 images received a 5 and the other 2 received a 4.
Takeaways
- A privacy-preserving method for facial portraits of patients with cleft lip.
- It retains the clinical features that matter for evaluation.
- It makes it possible to build and share more diverse clinical datasets.
Images on this page are illustrations, images generated by our method, or original images blurred as in the conference presentation.
Related work: DiffusionCleft: facial anomaly synthesis guided by text (ICMI 2025) · Enhanced facial landmarks detection for patients with repaired cleft lip and palate (FG 2024)
I am a Ph.D. candidate at the Language Technologies Institute, Carnegie Mellon University, where I am a member of the Multimodal Speech Processing (MSP) Lab led by Prof. Carlos Busso. My research sits at the intersection of multimodal speech processing and controllable generation: I model atypical speech and speech-grounded facial dynamics, and use generative models to make speech and facial technologies work for the people they most often fail, including children with speech disorders, individuals with craniofacial anomalies, and second-language learners.
My work spans generative reconstruction of disordered speech, synthetic data for low-resource clinical ASR, anatomically consistent facial analysis and synthesis, and interpretable multimodal biomarkers of speech disorders. In summer 2026 I interned with the Apple Accessibility team in Seattle, working on atypical speech generation.
My Ph.D. research is developed in collaboration with Children’s Health Dallas and UT Southwestern Medical Center, with Dr. Rami Hallac, Director of Research at the Analytical Imaging and Modeling (AIM) Center at Children’s Medical Center and Associate Professor at UT Southwestern. I communicate my work to both the AI and the medical communities: beyond AI venues such as ICMI and Interspeech, I have presented at clinical meetings, including the American Cleft Palate–Craniofacial Association annual meeting (ACPA 2024) and the International Congress on Cleft Lip/Palate (CLEFT 2025), and published in The Cleft Palate Craniofacial Journal.