search menu icon-carat-right cmu-wordmark

SysFlow: Scalable System Telemetry for Improved Security Analytics

August 2020 • Presentation
Federico Araujo (IBM Research), Teryl Taylor (IBM Research)

This presentation introduces SysFlow as a new data representation for system behavior introspection for scalable security, compliance, and performance analytics.




SysFlow is a compact open data format that lifts the representation of system activities into a flow-centric, object-relational mapping that records how applications interact with their environment—analogous to how NetFlow summarizes network communications. However, unlike NetFlow, which only captures network interactions, SysFlow connects network behaviors to processes and file access information, providing a richer context for analysis. This additional context facilitates deeper introspection into attack kill chains, resulting in analyses that yield lower false positives, and higher detection rates than traditional network-based approaches. SysFlow supports single-event and volumetric flow representations of process control flows, file interactions, and network communications. The new telemetry format drastically reduces storage requirements as compared to existing system telemetry sources, thereby enabling feature-filled analytics, process-level provenance tracking, and long-term data archival for threat hunting and forensics.