search menu icon-carat-right cmu-wordmark

Deep System Instrumentation for In Situ Human-AI Interaction Measurement Within Complex Information Systems

March 2022 Conference Paper
Joshua C. Poore (University of Maryland at College Park), Alex Veerasammy (University of Maryland at College Park), Amir Ghaemi (University of Maryland at College Park), Grant Tamrakar (University of Maryland at College Park), Kelsey Rassmann (University of Maryland at College Park), Craig Lawrence (University of Maryland at College Park)

This paper was presented at the 2022 AAAI Spring Symposium on AI Engineering.

Publisher:

Software Engineering Institute

Abstract

In this report, we explore key objectives for Human Systems Integration (HSI) Test Evaluation Validation and Verification (TEVV) for human-Artificial Intelligence (AI) interactions within analytical and information systems. Owing to their features and function, these systems pose challenges to the test community in evaluating the reciprocal influence of human and AI elements on one another. We argue that software system instrumentation provides the necessary capabilities to resolve measurement gaps. We further report our own efforts in adapting this technology for HSI TEVV and making these capabilities accessible to the larger test community.