CERT PODCAST SERIES: SECURITY FOR BUSINESS LEADERS: SHOW NOTES

Capturing the Expertise of Cybersecurity Incident Handlers

Key Message: Research on how expert cybersecurity incident handlers make decisions raises the bar for aspiring incident analysts.

Executive Summary

In this podcast, Dr. Richard Young, a professor with Carnegie Mellon’s Tepper School of Business teams with Sam Perl, a member of CERT’s Enterprise Threat and Vulnerability Management team to discuss their research on how expert cybersecurity incident handlers think, learn, and act when faced with an incident.

The research study focuses on critical cognitive factors that such experts use to make decisions when faced with a complex incident, including how to deal with critical information that is missing. Study results may be used to enhance the knowledge and skills of less experienced responders.


PART 1: PURPOSE, APPROACH, EXPERTS, AND INCIDENTS

Definition and Purpose

Cognition is:

For this study, the research team wanted to

Research Study Approach

The research team used the following approach:

The team recorded, transcribed, coded, and analyzed each interview. They hoped to discover if the experts shared the same process and schema for making decisions.

Experts

Experts were selected based on the following criteria:

Incidents

Incidents were selected from real samples contributed by participating organizations. Experience has shown that experts are easily able to sniff out problems with dummy incident reports, which could affect study results.


PART 2: SCHEMAS, MENTAL MODELS, AND SURPRISES

Definition

A schema is a mental model - knowledge in an experts’ head that they use to make decisions. It is used to help identify patterns that would assist a new or inexperienced incident handler in becoming more skilled. If an incident handler does not have such a schema, they are not considered to be an expert.

Expertise in other fields of study (e.g. accounting, business, science, or medicine) has been shown to be dependent on schemas to make consistent, repeatable, and reliable decisions.

Having a mental model is a great way to teach novices to become experts in their respective fields.

Identifying Schemas

The research team found that all four experts used similar schemas. They shared a common understanding of what was important, what to look for, and how to make their decision.

Specifically, the team identified two schemas that the experts used:

  1. Look for a certain set of attack attributes within each of the incident reports.
  2. Understand how the gaps in the data impacted the decision of what action to take and what additional pieces of information were needed to fill in the blanks.

The study confirmed that an experienced mental model, or schema, is critical to decision –making for incident handling.

Surprises

For incident handling experts to reach the same decisions, an incident report needs to match their schema and the information has to appear in the order they expect.

If the incident report did not match their schema and information did not appear as expected:

The team thought that if you put the right data in front of an expert, regardless of how it’s presented, they will be able to make sense of it. It turns out that providing incident data in the right way and in the right order has a big influence on the consistency of decisions.


PART 3: RECOMMENDATIONS

The research team recommends the following actions:


Resources

Perl, Sam & Young, Richard. “A Cognitive Study of Incident Handling Expertise.” 27th Annual FIRST Conference, Berlin, Germany, June 2015.

Young, Richard. How Audiences Decide: A Cognitive Approach to Business Communication. Routledge, December 2010.