Software Engineering Institute | Carnegie Mellon University
Software Engineering Institute | Carnegie Mellon University

Digital Library

Javascript is currently disabled for your browser. For an optimal search experience, please enable javascript.

Advanced Search

Basic Search

Content Type

Topics

Publication Date

Technical Report

Smart Collection and Storage Method for Network Traffic Data

  • September 2014
  • By Angela Horneman, Nathan Dell
  • This report discusses considerations and decisions to be made when designing a tiered network data storage solution.
  • Network Situational Awareness
  • Publisher: Software Engineering Institute
    CMU/SEI Report Number: CMU/SEI-2014-TR-011
  • Abstract

    Captured network data enables an organization to perform routine tasks such as network situational awareness and incident response to security alerts. The process of capturing, storing, and evaluating network traffic as part of monitoring is an increasingly complex and critical problem. With high-speed networks and ever-increasing network traffic volumes, full-packet traffic capture solutions can require petabytes of storage for a single day. The capacity needed to store full-packet captures for a time frame that permits the needed analysis is unattainable for many organizations. A tiered network storage solution, which stores only the most critical or effective types of traffic in full-packet captures and the rest as summary data, can help organizations mitigate the storage issues while providing the detailed information they need. This report discusses considerations and decisions to be made when designing a tiered network data storage solution. It includes a method, based on a cost-effectiveness model, that can help organizations decide what types of network traffic to store at each storage tier. The report also uses real-world network measurements to show how storage requirements change based on what traffic is stored in which storage tier.

  • Download

Cite This Report

SEI

Horneman, Angela; & Dell, Nathan. Smart Collection and Storage Method for Network Traffic Data. CMU/SEI-2014-TR-011. Software Engineering Institute, Carnegie Mellon University. 2014. http://resources.sei.cmu.edu/library/asset-view.cfm?AssetID=304860

IEEE

Horneman. Angela, and Dell. Nathan, "Smart Collection and Storage Method for Network Traffic Data," Software Engineering Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, Technical Report CMU/SEI-2014-TR-011, 2014. http://resources.sei.cmu.edu/library/asset-view.cfm?AssetID=304860

APA

Horneman, Angela., & Dell, Nathan. (2014). Smart Collection and Storage Method for Network Traffic Data (CMU/SEI-2014-TR-011). Retrieved March 30, 2017, from the Software Engineering Institute, Carnegie Mellon University website: http://resources.sei.cmu.edu/library/asset-view.cfm?AssetID=304860

CHI

Angela Horneman, & Nathan Dell. Smart Collection and Storage Method for Network Traffic Data (CMU/SEI-2014-TR-011). Pittsburgh, PA: Software Engineering Institute, Carnegie Mellon University, 2014. http://resources.sei.cmu.edu/library/asset-view.cfm?AssetID=304860

MLA

Horneman, Angela., & Dell, Nathan. 2014. Smart Collection and Storage Method for Network Traffic Data (Technical Report CMU/SEI-2014-TR-011). Pittsburgh: Software Engineering Institute, Carnegie Mellon University. http://resources.sei.cmu.edu/library/asset-view.cfm?AssetID=304860

BibTex

@techreport{HornemanSmartCollection2014,
title={Smart Collection and Storage Method for Network Traffic Data},
author={Angela Horneman and Nathan Dell},
year={2014},
number={CMU/SEI-2014-TR-011},
institution={Software Engineering Institute, Carnegie Mellon University},
address={Pittsburgh, PA},
url={http://resources.sei.cmu.edu/library/asset-view.cfm?AssetID=304860} }