search menu icon-carat-right cmu-wordmark

Methods for Testing and Qualifying Analytics

August 2020 Presentation
Timothy J. Shimeall

This presentation describes a process for testing analytics and qualifying them to be used to inform ongoing network defense.


Software Engineering Institute


The presenter started the presention with a brief discussion of what analytics are and describes some ways in which they fail. A sample analytic is introduced as a running example. Based on this foundation, the talk covers principles for testing analytics in general and the sample analytic in particular. As the principles are introduced, sample testing and results from that testing are provided. The presentation concludes with information on what it means to qualify an analytic for use and why such qualification is useful to network defenders.