Face De-Identification: A Domain-Centric Survey from Capture to Processing

AI summary

Overview

Research area: Computer vision and privacy-preserving machine learning, specifically face de-identification (De-ID) — the concealment of personally identifiable facial features in images and videos.

Technical level: Advanced. The survey assumes familiarity with face recognition architectures, adversarial perturbations, optical imaging (point spread functions, phase masks), generative models, and privacy metrics such as k-anonymity.

Scope: The paper is the authors' stated first unified survey of face De-ID spanning the full acquisition pipeline — physical, sensor, and digital domains — together with an analysis of evaluation protocols and open problems.

Authors and affiliation: Hui Wei, Hao Yu, and Guoying Zhao (Fellow, IEEE) are with the ELLIS Institute Finland, Espoo, 02150, Finland, and the Center for Machine Vision and Signal Analysis, University of Oulu, 90570 Oulu, Finland. Guoying Zhao is the corresponding author. The paper is arXiv:2607.25926v1 [cs.CV], dated 28 Jul 2026, licensed CC BY 4.0, and is accompanied by a project page at https://github.com/CV-AC/Awesome-FaceDe-ID that organizes literature, datasets, and open source code.

What This Paper Is About

Face de-identification aims to remove or conceal personally identifiable facial features so that a subject cannot be recognized, by either an automated face recognizer or a human observer, while non-identity information needed by downstream tasks (such as medical analysis or attribute recognition) remains usable. Existing surveys of this area are, in the authors' account, fragmented and narrow — most cover only digital post-capture techniques and rarely analyze evaluation protocols. This survey builds a single taxonomy organized by where in the imaging pipeline the privacy transformation happens: in the physical world before capture, inside the sensor during capture, or in software after capture.

Key Contributions

1. First domain-centric taxonomy of face De-ID. Methods are categorized according to the point at which the privacy transformation occurs — before, during, or after image capture — with representative methodologies summarized for each domain.

2. In-depth analysis of methodologies and trends across physical, sensor-based, and digital face De-ID, covering design principles, advantages, and limitations of each.

3. Review and organization of evaluation protocols, emphasizing the need for unified assessment frameworks that jointly measure privacy protection, utility preservation, and visual quality.

4. Identification of open challenges and future directions, including cross-modal privacy preservation, foundation-model-level face De-ID, verifiable and reversible De-ID, and fairness-aware privacy guarantees.

A supporting comparison table (TABLE I) contrasts this survey with prior surveys from 2015 through 2025, noting that earlier works predominantly cover digital-domain methods, with limited exploration of sensor or physical De-ID, and that this survey includes evaluation protocol analysis that the listed prior surveys do not.

Main Findings

• The field has expanded beyond post-capture software. Early De-ID operated in the digital domain using handcrafted filters (blurring, pixelation) or statistical mechanisms such as the k-Same family. Research has since shifted into sensor domain designs that suppress identity cues at capture time (for example, PrivacyOptics) and physical domain strategies that modify a subject's real-world appearance through adversarial wearables, cosmetics, or projected light patterns.

• Historical anchoring. The authors trace the field from Eigenfaces (Turk and Pentland, 1991), the first successful demonstration of automatic face recognition, to EFVAP (1996), the first dedicated De-ID technique. The timeline includes a work if it introduced a methodological primitive later adopted by others (k-Same, AdvEyeglass, DefocusOptics), was the first reported method in its sub-category (EFVAP), represents the state of the art or a distinct subsequent generation in a sub-category (AT3D, G²Face), or has high scholarly impact (citations ≥ 200 for works published before 2022 and ≥ 50 from 2022 onwards). The survey covers 112 methods in total (TABLE IV).

• Three physical-domain classes. Physical De-ID divides into wearable adversarial accessories (AdvEyeglass, AdvHat, AdvMakeup, Adversarial Mask, AdvSticker, ALPA, AT3D, and others), projected perturbations (ALPA, ProjAttacker, Agile), and adversarial illumination (Optical De-ID, ARA). AT3D is described as the first method using Adversarial Textured 3D Meshes optimized in the low-dimensional coefficient space of 3D Morphable Models rather than high-dimensional mesh space; physical tests were reported to evade four commercial anti-spoofing APIs, two mobile systems, and two access control systems.

• Three generations of sensor-domain optics. Fixed designs with hand-specified degradation (DefocusOptics, and DefocusOptics+ with programmable optics), learnable designs jointly optimized with downstream networks (PrivHPE, PrivHAR, PrivPDE, OpticalDR, PrivacyOptics), and dynamic designs that randomize the optical configuration per capture (DyPP, implemented via spatial light modulators). Extreme low-resolution imaging is a separate sensor paradigm, capturing at resolutions such as 16 × 12 pixels where faces are reduced to 2 × 2 or smaller representations, with downstream networks recovering task utility (ISR, LRPrivacy).

• Handcrafted digital methods fail the privacy–utility trade-off. EFVAP and De-ID Filter found that moderate blur conceals identity while preserving awareness cues such as posture, activity, and people count, with blur outperforming pixelization (which showed abrupt jumps in information disclosure across levels). BFPP showed that in home telecommuting, blur weak enough to preserve awareness cues failed to de-identify because motion and context still enabled recognition. This motivated spatially adaptive methods such as PUI (Eigen-space filtering) and EmotionPreserve (variational adaptive filtering with Total Variation regularization).

• k-anonymity provides a formal bound. k-Same replaces each face with its cluster average, guaranteeing recognition accuracy ≤ 1/k. Refinements include k-Same-Select (utility constraints preserving gender and expression), k-Same-M (active appearance models for alignment and photometric fidelity), and the ε-map framework, which formalized three privacy targets: ε-map ("like no one"), wrong-map ("like someone else"), and (ε, k)-map ("like everyone"). Multi-Factor DeID disentangled identity from nuisance factors, and k-Same-furthest averaged over maximally distant faces. These works exposed weaknesses in ad-hoc pixelation, including susceptibility to parrot recognition and resolution enhancement attacks.

• Domain choice is a trust-model choice. The authors argue that the three domains differ in where raw identity-bearing signal first exists in digital form: physical De-ID prevents identifiable signal from ever being produced; sensor De-ID ensures identifiable signal never leaves the camera; digital De-ID accepts a fully identifiable signal and removes identity in software. Earlier-stage interventions can make later ones redundant, and interventions can compound (sensor phase mask plus digital generative refiner in PrivacyOptics) or conflict (a physical adversarial patch followed by a digital denoiser may be neutralized as removable noise).

• The recognition adversary defines the field. Most De-ID approaches assume a strong face recognition adversary. The survey names FaceNet (triplet-loss deep metric learning), SphereFace (angular softmax for hypersphere embedding), ArcFace (additive angular margin loss, described as the most prevalent in De-ID research), and AdaFace (quality-adaptive margins), with faces typically localized by RetinaFace.

Methodology in Plain English

This is a literature survey rather than an experimental paper. The authors collected work using Google Scholar with the keywords "face de-identification," "face privacy preserving," and "face adversarial attack," and report the cumulative growth of methods over time. They then sorted the literature by a single organizing question — at which point in the pipeline does the privacy transformation happen — producing three domains: physical (on the subject or environment before capture), sensor (inside the camera during capture), and digital (on recorded imagery afterward). Within each domain they identify sub-classes (for example, eyeglasses, patches and stickers, makeup and masks, 3D meshes, projected perturbations, and adversarial illumination for the physical domain) and review representative methods with their design principles, advantages, and limitations. A separate section systematizes evaluation protocols, and the final sections lay out open problems.

The paper also formalizes the task: given an input face image, a De-ID method seeks a transformation producing an output in which identity cues are concealed while non-identity information is preserved. This is written as a joint maximization of privacy, utility, visual-quality, and reversibility scores with trade-off weights, plus an application-level switch that enables reversibility when authorized recovery (for example, forensic investigation, medical audit, or legal disclosure) is required. The framework states four criteria for a solution: privacy protection (the de-identified image should not match any other identity, preventing both re-identification to the original and misattribution to others), utility preservation (task-specific predictors should maintain consistent outputs before and after De-ID), visual fidelity and naturalness, and reversibility (recovery succeeds only for the legitimate key). The authors note that reversibility is orthogonal to privacy: a method may be strongly de-identifying and reversible, strongly de-identifying and irreversible, or neither.

Why This Matters

Impact on research. The paper argues that prior surveys concentrate on digital-domain techniques and overlook the evolution of sensor-based and physical De-ID methods, and that surveys rarely provide evaluation protocol analysis. By structuring the field around the acquisition pipeline rather than around algorithms, the authors aim to bridge fragmented literature across domains and to highlight emerging convergence between optical design, adversarial learning, and generative synthesis for privacy preservation. The paper also documents the growth trajectory of the field, including the shift from early digital methods to sensor-based and physical solutions.

The authors place face De-ID within a broader family of privacy-preserving methodologies that also includes differential privacy, federated learning, cryptographic protocols, synthetic data generation, and de-identification, noting that face De-ID is a vision-centric technique that complements but does not replace these paradigms. They further note that recognition in video and surveillance exploits complementary modalities such as body shape, gait, and other behavioral biometrics (voice, eye gaze, hand motions), making face De-ID one component of a broader person De-ID pipeline.

Real-world applications mentioned in the paper:

• Medical analysis and medical audit, where identity must be concealed but clinical content preserved.
• Attribute recognition, where non-identity attributes are the downstream target.
• Forensic investigation and legal disclosure, which the formal framework treats as cases requiring authorized, key-conditioned recovery.
• Workplace and home video (awareness cues such as posture, activity, and people count in the EFVAP, De-ID Filter, and BFPP studies), and surveillance, where gait and body shape may still permit recognition.

Industry relevance. The paper cites the EU AI Act (Regulation (EU) 2024/1689), stating that misuse of facial data constitutes a severe infringement of fundamental rights including the right to privacy. It reports physical testing of AT3D against commercial anti-spoofing APIs, mobile systems, and access control systems, and discusses the importance of black-box transferability against unknown commercial systems and of generalization under policy and threshold changes, including open-set verification. The authors argue that standardized benchmarks and reporting standards are needed to move claims from single-device demonstrations to deployable guarantees.

Future Directions

• Cross-modal privacy preservation — one of the four explicitly named directions, motivated by the observation that face De-ID does not address gait, body shape, voice, eye gaze, or hand motions.

• Foundation-model-level face De-ID — extending De-ID to large pretrained models rather than the task-specific architectures currently used.

• Verifiable and reversible De-ID — the paper notes that reversibility is orthogonal to privacy and that claims of irreversibility in the sensor domain should be stress-tested against optical inversion, deconvolution/deblurring, super-resolution, face restoration, and ISP-level side channels.

• Fairness-aware privacy guarantees — the paper notes that privacy–utility objectives remain task- and population-dependent, that preserving low-frequency geometry can leak soft-biometrics, and that fairness across demographics under optical distortions is underexplored.

Additional gaps the paper flags in its domain discussions: genuinely reusable, physics-grounded testbeds and shared calibration protocols; standardized fabrication tolerances and material-aware optimization for 3D-printed meshes; benchmarks covering multi-view video, multi-sensor (RGB-IR-depth) fusion, and liveness under varied ambient conditions; quantifying stealth in terms of human perception and social norms rather than area or total-variation proxies; quantifying calibration, latency, and energy costs of dynamic optics on mobile and embedded platforms; co-design between De-ID and defenses that jointly perform spoof detection, quality assessment, and identity inference; and long-term robustness to hardware drift and manufacturing tolerances.

Note on completeness: the supplied content is truncated partway through the digital-domain section on k-anonymity methods. The detailed digital-domain coverage of adversarial and generative approaches, the full evaluation protocol systematization in Section IV, and the complete Section V discussion are referenced in the abstract, introduction, and section outline but are not present in the text above, so specific datasets, metrics, and results from those sections are not reported here.

Target Audience

Researchers and graduate students in computer vision, biometrics, and privacy-enhancing technologies who need a single entry point to face De-ID across hardware and software settings; privacy engineers and compliance-oriented practitioners evaluating where in a capture pipeline to place a protection mechanism; and designers of camera, sensor, or wearable hardware interested in the physical and sensor-domain constraints (manufacturability, real-world robustness, conspicuity and social acceptability, and propagation through sensor capture) that the survey organizes. Readers seeking hands-on material will find the linked project page (https://github.com/CV-AC/Awesome-FaceDe-ID) useful for literature, datasets, and open source code. Because the paper does not report new experiments, readers looking for benchmark numbers or head-to-head quantitative comparisons will not find them in this work.

Authors’ abstract

Face de-identification (De-ID) aims to remove or conceal personally identifiable facial features in images or videos to prevent identity recognition while preserving utility for downstream tasks. With the rising emphasis on data privacy and responsible AI, face De-ID has emerged as an active research area spanning computer vision and privacy-preserving communities. Early approaches, and many contemporary ones, operate in the digital domain by modifying pixel-level or appearance-level features through post-capture processing. Recent advances extend face De-ID beyond post-processing by integrating privacy mechanisms directly into sensors during image acquisition, bridging sensing systems and downstream vision algorithms. In parallel, physical-domain methods explore wearable accessories and materials that conceal identity information in real-world environments prior to capture. In this survey, we present the first unified overview that spans the full data acquisition pipeline, encompassing the physical, sensor, and digital domains. Through this domain-centric lens, we systematically analyze current methodologies, technical progress, and the distinct challenges inherent to each stage. We then review and organize existing evaluation protocols, examining current practices and highlighting the critical need for standardized, comprehensive benchmarks. Finally, we identify key open problems and outline emerging research directions to guide future work in this rapidly evolving field. To support ongoing research, we maintain a project page that organizes relevant literature with collected datasets and open source code: https://github.com/CV-AC/Awesome-FaceDe-ID.