Information technology — Artificial Intelligence — Performance measurement for AI classification, regression, clustering and recommendation tasks
Introduction: As academic, commercial and governmental researchers continue to improve AI models, a common framework of measures has evolved to quantify this improvement. Some types of advances in artificial intelligence are reported in terms of improved functional correctness (also referred to as performance by AI practitioners) relative to the state of the art or a reasonable baseline. Functional correctness can be measured for AI classification, regression, clustering and recommendation tasks. For these tasks, the choice of an appropriate metric depends on the use case and domain constraints. Further, for machine learning (ML) systems, the chosen metric can differ from the metric used during training. For example, the functional correctness of an AI model for classification can be represented through the following examples: — A new AI model achieves 97,8 % classification accuracy on a dataset, where the state-of-the-art AI model achieves 96,2 % accuracy. — A new AI model achieves classification accuracy equivalent to the state of the art, but it operates with much less training data than state-of-the-art approaches. — A new AI model generates inferences 100x faster than state-of-the-art AI models while maintaining equivalent accuracy. This document describes approaches and methods for AI classification, regression, clustering and recommendation tasks, mapping tasks to relevant metrics. Various AI stakeholder roles as defined in ISO/IEC 22989:2022 can apply the approaches and methods to
Scope: This document specifies methodologies for measuring the performance of AI models for classification, regression, clustering and recommendation tasks.