Deep System Instrumentation for In Situ Human-AI Interaction Measurement Within Complex Information Systems

March 21, 2022 • Conference Paper

By

Joshua C. Poore (University of Maryland at College Park), Alex Veerasammy (University of Maryland at College Park), Amir Ghaemi (University of Maryland at College Park), Grant Tamrakar (University of Maryland at College Park), Kelsey Rassmann (University of Maryland at College Park), and Craig Lawrence (University of Maryland at College Park)

This paper was presented at the 2022 AAAI Spring Symposium on AI Engineering.

Publisher

Software Engineering Institute

Subjects

Artificial Intelligence Engineering

Abstract

In this report, we explore key objectives for Human Systems Integration (HSI) Test Evaluation Validation and Verification (TEVV) for human-Artificial Intelligence (AI) interactions within analytical and information systems. Owing to their features and function, these systems pose challenges to the test community in evaluating the reciprocal influence of human and AI elements on one another. We argue that software system instrumentation provides the necessary capabilities to resolve measurement gaps. We further report our own efforts in adapting this technology for HSI TEVV and making these capabilities accessible to the larger test community.

Software Engineering Institute