Software Engineering Institute | Carnegie Mellon University
Software Engineering Institute | Carnegie Mellon University

Digital Library

Javascript is currently disabled for your browser. For an optimal search experience, please enable javascript.

Advanced Search

Basic Search

Content Type

Topics

Publication Date

Presentation

Evolution of a Data-Streaming Solution

  • May 2016
  • By Joseph Paulchel (Capital One)
  • This talk describes a business need for data collection and streaming that led to a data-streaming solution for delivering messages from many clients to many end points.
  • Publisher: Software Engineering Institute
  • This presentation was created for a conference series or symposium and does not necessarily reflect the positions and views of the Software Engineering Institute.
  • Abstract

    In this talk, we will describe a business need for data collection and streaming that led to implementation of a data-streaming solution for delivering messages from many clients to various end points including Kafka, HDFS, Mongo DB, and Splunk. The talk will walk attendees through the evolution of the solution from local use of Active MQ and relational databases to a real-time data-streaming solution: Pivotal's Spring Extreme Data (XD).

    We will cover scalability considerations, observed performance, and security considerations. We will describe why the tool was selected, the various solution patterns implemented and discarded, and the final solution topology. We will also discuss deployment considerations for the public cloud, including the secure transport of data between data centers.

  • Download

Part of a Collection

SATURN 2016 Presentations