Information technology — A study of the differential impact of demographic factors in biometric recognition system performance
Introduction: Automated systems (including biometrics) are increasingly used in decision-making processes. In recent years, systemic performance differentials reflected in several automated decision systems have been reported and hotly debated. In the context of this report, an algorithm exhibiting performance differentials produces statistically different outcomes or decisions for different groups of individuals, for example, based on gender, age and race/ethnicity. In the context of biometric recognition, this means that probabilities of false positives and/or false negatives can differ among the demographic groups. The impacts of such performance differentials on the affected individuals can range from mere inconvenience in cooperative access control systems, to consequential harms such as varying arrest rates for certain demographic groups based on decisions produced by facial recognition systems. Although such systems are almost certainly not designed to be explicitly differential against any group, implicit differences can occur independently of the intentions of the system designers. They can be exhibited and propagated at many stages of the decision-making pipeline, including but not limited to training data itself as well as the data processing. Due to the scalability of such systems, a higher quantity of erroneous or inaccurate decisions can be generated than in the typical, human-based processes. Consequently, in recent years, measuring and ensuring the fairness (i.e. lack of di
Scope: This document introduces the effects of population demographics on biometric functions. It: — establishes terms and definitions relevant to the study of demographic factors in biometric recognition system performance; — identifies areas where biometric systems can exhibit different performances based on different demographic factors of the individuals submitting the biometric samples; — explains how different demographic factors can influence the biometric characteristics captured by different biometric modalities and how these influences can affect biometric performance measures; — presents a case study on existing scientific material that explores the impact of demographic factors on biometric system performance. Only biometric modalities where quantitative information is available on the impact of demographic factors are considered. Outside of the scope of this document are: — effects of disease and injury on biometric performance; and — how religious and cultural norms can affect biometric operations.