Advanced Search

Content Type

Topics

Publication Date

Distribution, Data, Deployment: Software Architecture Convergence in Big Data Systems

  • Author(s): Ian Gorton, John Klein
  • Publish Date:
  • Publisher: IEEE
  • Type: White Paper
  • Topics: Software Architecture
  • Description: This paper describes the challenges of big data systems for software architects, including harmonizing designs across the software, data, and deployment architectures.

Abstract

This article will appear in an upcoming issue of IEEE Software.

Exponential data growth from the internet, low-cost sensors, and high-fidelity instruments has fueled the development of advanced analytics operating on vast data repositories. These analytics bring business benefits ranging from web content personalization to predictive maintenance of aircraft components. To construct the data repositories that underpin these systems, there has been rapid innovation in distributed data-management technologies, employing schema-less data models and relaxing consistency guarantees to satisfy scalability and availability requirements. This paper describes the challenges of these "big data" systems that confront software architects. We show how distributed software architecture quality attributes are tightly linked to the both the data and deployment architectures. This causes a consolidation of concerns, and designs must be closely harmonized across these three structures to satisfy quality requirements.