search menu icon-carat-right cmu-wordmark

Distribution, Data, Deployment: Software Architecture Convergence in Big Data Systems

May 2014 Article
Ian Gorton, John Klein

This paper describes the challenges of big data systems for software architects, including harmonizing designs across the software, data, and deployment architectures.

Publisher:

IEEE Computer Society Press

Abstract

This article appears in the May/June 2015 issue of IEEE Software, Volume 32, Number 3, pages 78-85.

Exponential data growth from the internet, low-cost sensors, and high-fidelity instruments has fueled the development of advanced analytics operating on vast data repositories. These analytics bring business benefits ranging from web content personalization to predictive maintenance of aircraft components. To construct the data repositories that underpin these systems, there has been rapid innovation in distributed data-management technologies, employing schema-less data models and relaxing consistency guarantees to satisfy scalability and availability requirements. This paper describes the challenges of these "big data" systems that confront software architects. We show how distributed software architecture quality attributes are tightly linked to the both the data and deployment architectures. This causes a consolidation of concerns, and designs must be closely harmonized across these three structures to satisfy quality requirements.