Information technology — Biometric performance testing and reporting — Part 1: Principles and framework
Introduction: This document is concerned solely with the scientific technical performance testing of biometric systems and devices. Technical performance testing seeks to determine error and throughput rates, with the goal of understanding and predicting the real-world error and throughput performance of biometric systems. The error rates include both false-positive and false-negative rates, as well as failure-to-enrol and failure- to-acquire rates across the test population. Throughput rates refer to the number of individuals processed per unit of time based both on computational speed and human–machine interaction. These measures are generally applicable to all biometric systems and devices. Technical performance tests that are modality- specific, for example, fingerprint scanner image quality, are not considered in this document. The purpose of this document is to present the requirements and best scientific practices for conducting and reporting technical performance testing. It is acknowledged that technical performance testing is only one form of biometric testing. Other types of testing not considered in this document include: — reliability, availability and maintainability; — security, including vulnerability; — conformance; — safety; — human factors, including user acceptance; — cost/benefit; — privacy regulation conformance. Biometric technical performance testing can be of three types: technology, scenario and operational evaluation. Each type of test requires a different protoc
Scope: This document: — establishes general principles for testing the performance of biometrics systems in terms of error rates and throughput rates for purposes including measurement of performance, prediction of performance, comparison of performance, and verifying conformance with specified performance requirements; — specifies performance metrics for biometric systems; — specifies requirements on the recording of test data and reporting of test results; and — specifies requirements on test protocols in order to: — reduce bias due to inappropriate data collection or analytic procedures; — help achieve the best estimate of field performance for the expended effort; — improve understanding of the limits of applicability of the test results. This document is applicable to empirical performance testing of biometric systems and algorithms through analysis of the comparison scores and decisions output by the system, without requiring detailed knowledge of the system’s algorithms or of the underlying distribution of biometric characteristics in the population of interest. Not within the scope of this document is the measurement of error and throughput rates for people deliberately trying to subvert the intended operation of the biometric system (e.g. by presentation attacks).