search menu icon-carat-right cmu-wordmark

Building Coherent Use into the DevOps Lifecycle for High-Stakes AI

March 2022 Conference Paper
Kelsey Rassmann (University of Maryland at College Park), Jana Schwartz (University of Maryland), Julie Marble (University of Maryland), William Regli (University of Maryland at College Park)

This paper was presented at the 2022 AAAI Spring Symposium on AI Engineering.

Publisher:

Software Engineering Institute

Abstract

The creation of high-stakes artificial intelligence (AI) has the potential to positively impact users and their surrounding environment. However, today’s development practices introduce the risk of serious errors of misuse, disuse, and abuse that can lead to loss of life and liberty. Software developers must focus on mitigating these potential errors of use while also ensuring the timely delivery of AI-enabled technology. In this paper, we lay out the human-machine interaction in terms of a cost-benefit analysis and thus highlight the need for coherency between the human and AI. We use this reference model to inform development of a Goal, Question, Metric approach to achieving a Coherent Use of AI-enabled software that can be applied throughout the DevOps lifecycle.