Face Recognition, Liveness, and Biometrics

Security depends on capture, identity binding, comparison, policy, and fallback. Five stages run in order. Enroll the identity, binding a verified person to one or more protected templates. Assess the capture for face quality, pose, occlusion, device integrity and presentation attacks. Produce a template under a versioned model and preprocessing pipeline. Compare or search, calculating similarity to a claimed template or to gallery candidates. Apply policy: accept, reject, step up authentication, or route to manual recovery.

CBP's Biometric Air Exit programme is a deployed system where those stages were measured separately. They did not fail together. The programme's Key Performance Parameters require a true accept rate of at least 97 percent of in-scope travelers and a false accept rate no greater than 0.1 percent. DHS's 2025 final rule on the collection of biometric data from aliens entering and departing the United States, effective 26 December 2025, reports internal CBP estimates of at least 98 percent TAR and 0.02 percent FAR.

Stage four held up under test. An independent test agent inside CBP ran an operational test and evaluation in May–June 2019, and GAO reported the findings in September 2020 and again in July 2022. Air exit correctly matched 98 percent of travelers' photos against galleries built from passenger manifests. It incorrectly matched a traveler to a gallery photo less than 0.1 percent of the time. Stage two is where the programme missed its own bar. Rebecca Gambler of the U.S. Government Accountability Office told a House subcommittee on 27 July 2022: “Specifically, the test agent found that air exit successfully captured the photos of approximately 80 percent of in-scope travelers on participating flights, short of the 97 percent minimum requirement.” The matcher was not the constraint. The camera was. CBP then moved to remove the 97 percent requirement.

A vendor accuracy figure describes stage four alone. It says nothing about the stage that decided how many people reached stage four at all.

The candidate set and the decision question change the error surface.

One-to-one verification asks whether this capture matches the claimed enrolled identity. Small comparison set; the claimant supplies the identity; a threshold controls accept and reject. An account login.

One-to-many identification asks which gallery identity, if any, best matches the capture. Large search space; open-set rejection required, because the right answer may be "nobody"; candidate ranking precedes any decision. A duplicate-enrollment check.

Watchlist search asks whether the capture resembles anyone in a sensitive target list. Low base rates, a high false-alert burden, and human confirmation as the only thing between a ranked list and someone's front door. A controlled-access alert.

That last line is usually written as advice. What it costs when it is skipped is documented. Detroit police wrongfully arrested Robert Williams in January 2020, after a face-recognition search on a blurry surveillance still returned his expired driver's licence photo. The ACLU, the ACLU of Michigan and the University of Michigan Law School's Civil Rights Litigation Initiative sued in April 2021, and the parties settled Williams v. City of Detroit on 28 June 2024. The ACLU, counsel of record for Williams, records what the city agreed to: “Under the agreement, police will be required to back up face recognition results with independent and reliable evidence linking a suspect to a crime before making any arrest.”

The settlement also mandates training. And it requires an audit of every case since 2017 in which the department used face recognition to obtain an arrest warrant. Human confirmation is now written policy in Detroit: DPD Directive No. 203.11 states that a lineup “may never be conducted based solely on a facial recognition investigative lead”. The U.S. Commission on Civil Rights, which quoted that directive in September 2024, names seven known U.S. wrongful arrests from face recognition misidentification.

Genuine and impostor similarity distributions overlap. Lowering the threshold admits more genuine users and more impostors; raising it reduces false accepts while rejecting more legitimate users.

Choose operating points for the workflow, population, gallery size, and fallback; report false match and false non-match rates under the actual comparison protocol.

NIST measured what those thresholds do to different groups. The demographic-effects volume of its Face Recognition Vendor Test, NISTIR 8280, appeared in December 2019. Its reporting advice is blunt, and it is about the threshold you actually run: since “most systems are configured with a fixed threshold, it is necessary to report both false negative and false positive rates for each demographic group at that threshold. This is rarely done - most reports are concerned only with false negatives.”

One operating mode abandons the threshold entirely. NISTIR 8280 calls investigative search “a special-case application of identification algorithms where the threshold is set to zero so that all searches will produce a fixed number of candidates. In such cases, the false positive identification rate is 100% because any search of someone not in the database will still yield candidates”. Then it names the safeguard that is left: “what matters then is the human response”. GAO described the same output in law-enforcement terms in September 2023, where “identification searches compare a photo of a single unknown individual against a gallery of photos to determine if there is a potential match (i.e., an investigative lead)”.

The FBI gives that shortlist its exact dimensions. Its Privacy Impact Assessment for the Next Generation Identification Interstate Photo System, approved on 7 November 2024, states: “Specifically, a single photo will never be returned, but a gallery of two to fifty photos, with the LEA choosing the size of the gallery. If no choice is made, a default of twenty photos is returned.” Every response carries the caveat “This response is an INVESTIGATIVE LEAD ONLY. It is NOT positive identification of the subject.” The U.S. Commission on Civil Rights reported 34,014 investigative face recognition searches of the NGI-IPS in FY 2023.

That is what "no threshold" looks like as a product: a fixed-length list of ranked strangers, returned tens of thousands of times a year, with the accuracy question moved off the algorithm and onto whoever reads the list.

Aggregate error can hide systematic differences and brittle capture conditions.

NIST IR 8491 is what slicing one component looks like when someone actually does it. Published on 20 September 2023, it is the tenth part of NIST's Face Analysis Technology Evaluation and it covers passive, software-based presentation-attack detection: 82 passive software-only PAD algorithms from 45 developers, roughly 20,000 attack presentations and 21,000 bona fide presentations, across 9 attack categories. The summary of what it found is not reassuring: “PAD performance varied widely across algorithms, use cases, and PA types, with only a small percentage of algorithms evaluated demonstrating notable detection capabilities.” The report is also explicit about its own boundary. It is silent on injection attacks. And it reports that zooming a print or replay attack, so that the photo frame and the hands are out of view, raises error rates across all algorithms.

Attack resistance extends beyond the recognition network. Each stage of the channel has its own documented failure.

Secure enrollment verifies identity binding and prevents duplicate or fraudulent enrollment. Every weakness in the document check behind enrollment is inherited for the life of the trait.

Protect capture by authenticating devices, detecting replay or injection, and preserving provenance. The route FinCEN names is a third-party webcam plugin feeding previously generated video into the check. NIST IR 8491 is the largest public PAD benchmark available, and it says outright that it is silent on injection attacks. No passive PAD score covers that route.

Protect templates through encryption, separated identifiers, restricted access and a revocation plan. The Dutch decision of 16 May 2024 is what an unprotected, unconsented template store looks like when a regulator reads Article 9(1) GDPR against it.

Limit decisions with least privilege, rate limits, thresholds and step-up authentication. 34,014 investigative searches in one fiscal year is a decision volume, not an incident.

Provide recovery: accessible non-biometric alternatives and a route of appeal for false rejection. CBP's air exit captured approximately 80 percent of in-scope travelers against a 97 percent minimum requirement, so the alternative path is the normal path for a substantial share of people. And Robert Williams needed a federal settlement, formalized 28 June 2024, to obtain the written rule that a face-recognition result is not on its own a reason to arrest anyone.

A bank considers face verification when customers lose their device, a situation already attractive to attackers.

The attack typology is published, so design against it rather than against a hypothetical. FinCEN issued an alert on deepfake fraud schemes against financial institutions on 13 November 2024. Since 2023, it reports, suspicious activity reports describing deepfake media used to defeat financial institutions' identity verification have increased. The alert names the injection route: “Some identity verification solutions may also flag possible attempts to circumvent verification checks, such as the use of third-party webcam plugins, which can let a customer display previously generated video rather than live video.” The FBI's Internet Crime Complaint Center issued a parallel public service announcement on 3 December 2024, on criminals using generative artificial intelligence to produce identification documents and real-time video for fraud.