AI Engineering Assets
Publications and resources for the SEI's AI Engineering effort
Abstract
AI Engineering is an emergent discipline focused on developing tools, systems, and processes to enable the application of artificial intelligence in real-world contexts. This collection includes papers, videos, presentations, and other publications related to the SEI's AI Engineering work.
Collection Contents
-
Deep System Instrumentation for In Situ Human-AI Interaction Measurement Within Complex Information Systems
March 21, 2022 • Conference Paper
By Joshua C. Poore (University of Maryland at College Park), Alex Veerasammy (University of Maryland at College Park), Amir Ghaemi (University of Maryland at College Park), Grant Tamrakar (University of Maryland at College Park), Kelsey Rassmann (University of Maryland at College Park), Craig Lawrence (University of Maryland at College Park)
This paper was presented at the 2022 AAAI Spring Symposium on AI Engineering.
read -
Building Coherent Use into the DevOps Lifecycle for High-Stakes AI
March 21, 2022 • Conference Paper
By Kelsey Rassmann (University of Maryland at College Park), Jana Schwartz (University of Maryland), Julie Marble (University of Maryland), William Regli (University of Maryland at College Park)
This paper was presented at the 2022 AAAI Spring Symposium on AI Engineering.
read -
Measuring Beyond Accuracy
March 21, 2022 • Conference Paper
By Violet Turri, Rachel Dzombak, Eric Heim, Nathan M. VanHoudnos, Jay Palat, Anusha Sinha
This paper was presented at the 2022 AAAI Spring Symposium on AI Engineering.
read -
Lessons Learned in Human-Artificial Intelligence Teaming in Business Processes
March 21, 2022 • Conference Paper
By Michael J. Mendenhall (Air Force Research Laboratory), Gilbert L. Peterson (Air Force Research Laboratory), Alexander Graves (Air Force Research Laboratory), Jonathan W. Butler (Air Force Research Laboratory)
This paper was presented at the 2022 AAAI Spring Symposium on AI Engineering.
read -
Exploring Opportunities in Usable Hazard Analysis Processes for AI Engineering
March 21, 2022 • Conference Paper
By Nikolas Martelaro (Carnegie Mellon University), Carol J. Smith, Tamara Zilovic (Carnegie Mellon University)
This paper was presented at the 2022 AAAI Spring Symposium on AI Engineering.
read -
Experience with Using Synthetic Training Images for Wearable Cognitive Assistance
March 21, 2022 • Conference Paper
By Roger Iyengar (Carnegie Mellon University), Emily Zhang (Carnegie Mellon University), Mahadev Satyanarayanan (Carnegie Mellon University)
This paper was presented at the 2022 AAAI Spring Symposium on AI Engineering.
read -
Adaptive Autonomy as a Means for Implementing Shared Ethics in Human-AI Teams
March 21, 2022 • Conference Paper
By Allyson I. Hauptman (Clemson University), Beau G. Schelble (Clemson University), Nathan J. McNeese (Clemson University)
This paper was presented at the 2022 AAAI Spring Symposium on AI Engineering.
read -
Kernel Density Decision Trees
March 21, 2022 • Conference Paper
By Jack H. Good (Carnegie Mellon University, Robotics Institute, Kyle Miller (Carnegie Mellon University, Robotics Institute), Artur Dubrawski (Carnegie Mellon University, Robotics Institute)
This paper was presented at the 2022 AAAI Spring Symposium on AI Engineering.
read -
Juneberry - Tutorial
March 22, 2022 • Presentation
By Andrew O. Mellinger, Nathan M. VanHoudnos, Nick Winski
Presented at Naval Applications of Machine Learning 2022, this tutorial reviews Juneberry, a reproducible research framework to build, maintain, and evaluate ML with declarative configs.
read -
AI at the SEI
May 19, 2022 • Video
By Michael Mattarock, Matthew J. Butkovic
In this episode, Matt Butkovic, talked with Michael Mattarock, about the SEI’s efforts to apply AI techniques to address national security mission needs while leading a national initiative to build a new discipline of AI Engineering.
watch -
Explainable AI Explained
May 19, 2022 • Podcast
By Violet Turri
Violet Turri discusses explainable AI, which encompasses all the techniques that make the decision-making processes of AI systems understandable to humans.
learn more -
Human-Centered AI
June 25, 2021 • White Paper
By Hollen Barmer, Rachel Dzombak, Matt Gaston, Jay Palat, Frank Redner, Carol J. Smith, Tanisha Smith
This white paper discusses Human-Centered AI: systems that are designed to work with, and for, people.
read -
Scalable AI
June 25, 2021 • White Paper
By Hollen Barmer, Rachel Dzombak, Matt Gaston, Jay Palat, Frank Redner, Tanisha Smith, John Wohlbier
This white paper discusses Scalable AI: the ability of AI algorithms, data, models, and infrastructure to operate at the size, speed, and complexity required for the mission.
read -
Robust and Secure AI
June 25, 2021 • White Paper
By Hollen Barmer, Rachel Dzombak, Matt Gaston, Eric Heim, Jay Palat, Frank Redner, Tanisha Smith, Nathan M. VanHoudnos
This white paper discusses Robust and Secure AI systems: AI systems that reliably operate at expected levels of performance, even when faced with uncertainty and in the presence of danger or threat.
read -
AI Engineering: 11 Foundational Practices
September 12, 2019 • White Paper
This initial set of recommendations can help organizations that are beginning to build, acquire, and integrate artificial intelligence capabilities into business and mission systems.
read -
AI Engineering for Defense and National Security: A Report from the October 2019 Community of Interest Workshop
October 29, 2020 • Special Report
Based on a workshop with thought leaders in the field, this report identifies recommended areas of focus for AI Engineering for Defense and National Security.
read -
Future Reach Conversation: Countering Adversarial Operations Made Possible by AI
November 22, 2019 • Video
Watch as Dr. Matt Gaston, Director of SEI Emerging Technology Center, moderates discussion on countering adversarial operations made possible by AI
watch -
Defining AI Engineering
February 14, 2020 • Video
By Matt Gaston,
Watch Dr. Matt Gaston, Director of the SEI Emerging Technology Center, and Professor Martial Hebert, Dean of CMU School of Computer Science and recognized world expert in computer vision, discuss the emerging field of AI Engineering.
watch -
Ethics in AI Engineering
December 15, 2020 • Video
By Carol J Smith
The presentation discusses how to reduce unintended/harmful bias and prevent the inevitable harm that comes from "unknowable" systems.
watch -
AI at the SEI
January 29, 2019 • Fact Sheet
The SEI's history as a federally funded research and development center (FFRDC) dedicated to software engineering means that we know what it takes to lay a foundation for confident, rapid adoption of AI for national defense and security.
read -
Train, but Verify: Towards Practical AI Robustness
November 3, 2020 • Presentation
By Nathan M. VanHoudnos, Jon Helland
This presentation describes efforts to train AI systems to enforce at least two security policies and verify security by testing against realistic threat models.
read -
Train, but Verify: Towards Practical AI Robustness
December 15, 2020 • Video
By Nathan M. VanHoudnos, Jon Helland
This presentation describes efforts to train AI systems to enforce at least two security policies and verify security by testing against realistic threat models.
watch -
Poster - Train, but Verify: Towards Practical AI Robustness
November 3, 2020 • Poster
By Nathan M. VanHoudnos, Jon Helland
This presentation describes efforts to train AI systems to enforce at least two security policies and verify security by testing against realistic threat models.
read -
Designing Trustworthy AI
April 29, 2020 • Podcast
By Carol J Smith
Carol Smith discusses a framework that builds upon the importance of diverse teams and ethical standards to ensure that AI systems are trustworthy and able to effectively augment warfighters.
learn more -
Designing Trustworthy AI: A Human-Machine Teaming Framework to Guide Development
December 10, 2019 • Conference Paper
By Carol J Smith
The Human-Machine Teaming (HMT) Framework for Designing Ethical AI Experiences, when used with a set of technical ethics, will guide AI development teams to create AI systems that are accountable, de-risked, respectful, secure, honest, and usable
read -
Learning by Observing via Inverse Reinforcement Learning
March 22, 2019 • Video
By Ritwik Gupta, Eric Heim
This SEI Cyber Talk episode explains how inverse reinforcement learning can be effective for teaching agents to perform complex tasks with many states and actions.
watch -
A Series of Unlikely Events: Learning Patterns by Observing Sequential Behavior (video)
November 11, 2019 • Video
By Eric Heim
Watch SEI principal investigator Eric Heim discuss research to develop novel Inverse Reinforcement Learning (IRL) techniques as efficient and effective means for DoD/IC to perform activity-based intelligence or to teach novices how to perform tasks.
watch -
A Series of Unlikely Events: Learning Patterns by Observing Sequential Behavior
October 28, 2019 • Poster
By Eric Heim
This poster represents research to apply Inverse Reinforcement Learning techniques to model sequential behavior.
read