AI Assurance

The current artificial intelligence (AI) renaissance, fueled by major breakthroughs in machine learning (ML), promises to automate and make more efficient our homes, transportation networks, industrial processes, and national security, among many other things. As safety-critical decisions are handed over to AI, there needs to be confidence in the underlying ML to ensure its decisions are testable, repeatable, and auditable. We need to understand failure modes and be able to mitigate them.
Current research into deep learning is creating systems that approach or exceed human-level performance on a wide range of tasks, but for the most part deep learning is a black box. Deep learning has improved predictive capability beyond previous AI constructs, but when the algorithms make mistakes, we often cannot explain them nor find ways to correct them. This is problematic when a self-driving car makes an unexplainable unsafe driving decision. It also opens up an entirely new frontier for hackers who target AI algorithms directly to cause unexpected behavior, rather than more traditional computer systems, networks, and data.
Traditional approaches to software assurance – how confident we can be in software to function as intended – are complicated by the introduction of AI algorithms into software. AI algorithms can change in real time as they are used. They are ‘black box’, and the underlying code can’t be reviewed in traditional ways. They are inherently vulnerable to slight deviations in input data. They might embed policies and tradeoffs in ways that can’t be observed and changed. And even if the algorithm is functionally perfect, user distrust and misunderstanding can undermine their performance. AI Assurance requires that we account for all of these challenges in certifying how confident we are that ML and/or AI algorithms function as intended and are free of vulnerabilities, either intentionally or unintentionally designed or inserted as part of the data/algorithm.
CCI will build an AI assurance testbed and software factory that will support research in applications and technologies requiring AI across the network. Research will include methods for quantifying confidence in ML and/or AI algorithms ensuring they function as intended and perform reasonably under a variety of circumstances and contexts. Researchers will investigate evaluation metrics and test designs for AI, test and evaluation of algorithm transparency and explainability, and threat portrayal for ML/AI algorithms. The testbed will enable improvements in designing and testing robust, verifiable, and unbiased ML/AI algorithms.