search menu icon-carat-right cmu-wordmark

Poster - Human Decision Making with AI Support

November 2020 Poster
Rotem D. Guttman

This poster details the development of the Human-AI Decision Evaluation System (HADES), a test harness allowing the collection of human decision-making data on an arbitrarily large set of possible AI interfaces.


Software Engineering Institute


Time and again we’ve seen humans making poor choices while relying on (or ignoring) existing artificial intelligence (AI) decision support systems. These failures have led several systems to be abandoned. Preliminary research indicates that a failure to communicate model output understandably may contribute to this problem, but it is currently unknown what the best practices in AI system design are that would alleviate it.

If you want to know what humans will do, you usually need to check what a human will do. Our goal in developing the Human-AI Decision Evaluation System (HADES) was to collect data on real human decision making and use that data to determine appropriate best practices for AI system interface design within a chosen domain. HADES has the ability to slot in an actual AI-enabled decision support system and simulate not-yet-implemented AI systems, supporting multiple experimental designs to find the one that is most intuitive for its end users.