Four Principles for Engineering Scalable, Big Data Systems
“The more nodes, the more hardware you have, the more software you have, the law of averages is going to dictate that things will fail. You have to handle this. The bigger your system, the more things will fail. So, failures become common.
In this podcast, Ian Gorton describes four general principles that hold for any scalable, big data system. These principles can help architects continually validate major design decisions across development iterations, and hence provide a guide through the complex collection of design trade-offs all big data systems require.
About the Speaker
In his work at the SEI, Ian Gorton investigates issues related to software architecture at scale. This includes designing large-scale data management and analytic systems and understanding the inherent connections and tensions among software, data, and deployment architectures in cloud-based systems.