Key Message: Research on how expert cybersecurity incident handlers make decisions raises the bar for aspiring incident analysts.
In this podcast, Dr. Richard Young, a professor with Carnegie Mellon’s Tepper School of Business teams with Sam Perl, a member of CERT’s Enterprise Threat and Vulnerability Management team to discuss their research on how expert cybersecurity incident handlers think, learn, and act when faced with an incident.
The research study focuses on critical cognitive factors that such experts use to make decisions when faced with a complex incident, including how to deal with critical information that is missing. Study results may be used to enhance the knowledge and skills of less experienced responders.
Definition and Purpose
For this study, the research team wanted to
Research Study Approach
The research team used the following approach:
The team recorded, transcribed, coded, and analyzed each interview. They hoped to discover if the experts shared the same process and schema for making decisions.
Experts were selected based on the following criteria:
Incidents were selected from real samples contributed by participating organizations. Experience has shown that experts are easily able to sniff out problems with dummy incident reports, which could affect study results.
A schema is a mental model - knowledge in an experts’ head that they use to make decisions. It is used to help identify patterns that would assist a new or inexperienced incident handler in becoming more skilled. If an incident handler does not have such a schema, they are not considered to be an expert.
Expertise in other fields of study (e.g. accounting, business, science, or medicine) has been shown to be dependent on schemas to make consistent, repeatable, and reliable decisions.
Having a mental model is a great way to teach novices to become experts in their respective fields.Identifying Schemas
The research team found that all four experts used similar schemas. They shared a common understanding of what was important, what to look for, and how to make their decision.
Specifically, the team identified two schemas that the experts used:
The study confirmed that an experienced mental model, or schema, is critical to decision –making for incident handling.Surprises
For incident handling experts to reach the same decisions, an incident report needs to match their schema and the information has to appear in the order they expect.
If the incident report did not match their schema and information did not appear as expected:
The team thought that if you put the right data in front of an expert, regardless of how it’s presented, they will be able to make sense of it. It turns out that providing incident data in the right way and in the right order has a big influence on the consistency of decisions.
The research team recommends the following actions:
Perl, Sam & Young, Richard. “A Cognitive Study of Incident Handling Expertise.” 27th Annual FIRST Conference, Berlin, Germany, June 2015.
Young, Richard. How Audiences Decide: A Cognitive Approach to Business Communication. Routledge, December 2010.