New ISO Technical Specification for AI Testing Emerges
A new technical specification, ISO-IEC TS 42119-2, is gaining attention as a potentially game-changing standard for AI system testing. The specification is expected to standardize test protocols, methodologies, and reporting for AI systems. This development addresses a growing industry and regulatory need for robust, consistent frameworks to substantiate claims about AI performance, safety, and bias mitigation.
- The specification is developed by ISO/IEC JTC 1/SC 42, the key joint technical committee for AI standardization, chaired by Wael William Diab. This committee, which held its inaugural meeting in Beijing in April 2018, serves as the central hub for creating foundational AI standards and guiding other ISO and IEC committees on AI applications. - It is explicitly designed to complement a suite of other foundational AI standards, including ISO/IEC 42001 (the first AI management system standard), ISO/IEC 23894 on AI risk management, and ISO/IEC TR 24028, which provides a framework for AI trustworthiness. This integrated approach allows organizations to align testing protocols with broader governance and risk management strategies. - The technical specification adapts the existing ISO/IEC/IEEE 29119 software testing standards for the unique challenges of AI. It provides guidance on applying established testing processes and documentation to AI-specific issues like data quality, model validation, algorithmic bias, and the probabilistic nature of AI outputs. - This document is the first part of a broader ISO/IEC 42119 series on AI testing. Future planned installments will address more advanced topics, including verification and validation analysis (Part 3), structured adversarial testing or "red teaming" (Part 7), and quality assessment for prompt-based generative AI (Part 8). - The standard's risk-based approach is crucial for regulatory alignment, providing a methodology for organizations to generate evidence and documentation to comply with emerging frameworks like the EU's Artificial Intelligence Act. - The concepts of trustworthiness that underpin this testing specification are drawn from ISO/IEC TR 24028. This technical report defines the core attributes to be tested, such as reliability, robustness, transparency, fairness, and accountability in AI systems.
