Preface
Five years ago, Pini Reznik and Jamie Dobson founded Container Solutions, a professional services firm that helps early adopter companies figure out the best ways to make use of the emerging cloud computing paradigm. Jamie had been working with many companies to promote Agile development practices, and Pini was the one to identify containers as something that would change the world of software development. Containers were pretty new, and a powerful way to develop and deliver software in standalone, platform-agnostic packages—more lightweight than virtual machines and infinitely more portable. They decided to work together, forming a company dedicated to helping enterprises to harness this powerful new technology.
In 2014, companies moving to containerized applications were using Apache Mesos for cluster management, since it was the only tool capable of scheduling containers at that point. Skipping ahead a few years, Mesos has become outdated, and so now these companies want to move to Kubernetes, which they absolutely should do. But they are coming to us again for help in doing it. This repeat business is flattering but led us to a moment of realization.
Three years ago, when we helped these companies move to Mesos, it was a jump. Now they are coming to us to help them jump again, this time to K8s—meaning that they didn’t build in innovation capabilities so they would be ready and able to evolve to the next step whenever it came. They were still operating under the old mindset of “Buy a major tool and install it; the process is disruptive for a few weeks but then you’re good again for the next five or ten years.”
Don’t get us wrong, it isn’t trivial to migrate from Mesos to Kubernetes if you’re running 100+ microservices and using continuous integration/continuous delivery (CI/CD) with configuration management pretty heavily specialized for Mesos. But moving to a new way of managing the containerized apps you’re already running should not feel like another major jump. It should feel more like a reasonably straightforward step. To us the process was fairly obvious: get the new platform in place, do some test migrations. You don’t need to do everything all at once, or move all the services together—you have time. Start experimenting with simple stateless services and meanwhile learn about K8s in the process. You’ll get there. What was self-evident to us, however, was not so obvious to our clients, who were still of the old mindset that adopting new tooling should be a one-and-done move.
And that was our “a-ha!” moment of realization: our job is no longer simply guiding clients onto the cloud, helping them choose and adopt the best tools and technologies for their migration. Our true job now is to help them achieve complete digital transformation—not just adopting new tech, but also evolving a completely new way of thinking. Because technology is only getting faster and faster. Being cloud native means delivering a distributed system of microservices that are constantly iterated and newly deployed every day, if not many times each day. Now we are changing everything in every part of the organization all the time, and all at the same time, which is a completely different way of thinking. You will always be doing this, innovating and moving to the next thing, and the next. Innovation now needs to be simply another piece of your workflow—one that is built right in.
We didn’t invent this. We didn’t even invent how to do it. We decided to write this book to describe how to do it, to map the cloud native trail that we and other early adopters uncovered before this way of working even really had a name. Patterns are the most effective way we see for doing it. However, if you’re looking for a book of highly technical software design patterns, this is not the book for you.
On the other hand, if you’re a middle manager, a project leader, or maybe even an executive-level decision maker seriously contemplating cloud native for your company, then this is the book you’ve been waiting for.
There are many very good technical pattern books out there that can tell you the ten best ways to Terraform a Kubernetes cluster. We are here to explain what Kubernetes is, how it fits into cloud native, and why you need it in the first place. You don’t have to build the system, just understand it.
These are high-level patterns for strategy and risk reduction to guide decision making as you transition your organization and culture to this new way of thinking and working. We hope engineers will read this book too, because they are key to helping their companies execute a transformation and then excel at delivering afterward via optimized processes on their new cloud native platform. After reading this book, they can hand it to their managers and say, “This is what we need to do.”
About This Book
At first we started writing a fairly straightforward and academic book: here are some patterns, here is how this all works.
Eventually, though, it became clear that many, if not most, of the problems we see arising during real-world transformations have little to do with the technology itself. The tech and tools are concrete—you can wrap your head around infrastructure, even if it’s virtual—as well as fairly well supported. Instead, culture, process, and the other human-centered changes are, we have observed, the most difficult aspect of a transformation. Many migrations stumble, or even fail, by making common mistakes rooted in cognitive bias. Patterns can help them avoid these pitfalls.
And so we decided to present this book in a more human-centered way, through telling the story of a typical transformation. The tale is drawn from actual projects we have worked on with client enterprises, though we will not be naming names, to tell the story of a typical enterprise as it undertakes a cloud migration—including failures along the way.
We created a fictional company called WealthGrid to represent this “typical enterprise,” which is very similar to the companies taking a serious look at going cloud native now that the tech is entering the early edge of mainstream adoption. WealthGrid’s transformation story is presented as a series of interludes with informational chapters in between. The first five chapters describe essential cloud native background knowledge and concepts and then introduce tools, approaches, and strategies for working with both the tech and the patterns. We provide our own cloud native Maturity Matrix tool for helping you figure out your company’s current situation and select your cloud native destination. The first half of the book ends with WealthGrid’s crisis moment: why, after trying so hard and investing so much, is their attempted cloud native transformation simply not succeeding?
The second half of the book brings us to the patterns themselves, organized into chapters for strategy, organization/culture, development/process, and infrastructure. After that comes our favorite part: we demonstrate a successful cloud native transformation path for a typical company, told in patterns. Basically this is the story of what WealthGrid should have done.
Patterns are revealed in context as the narrative unfolds, along with biases that commonly arise in these situations to slow down or even derail an initiative. We show you how to use some of these biases in a positive way, to nudge things forward, while also avoiding common mistakes. By the end, a full pattern language is established. The story functions as a transformation design that can be used as a solid starting point for a real-world cloud native initiative by a small or mid-size company.
The transformation design (and, spoiler alert, happy ending for WealthGrid) is followed by a chapter where we break out common transformation pitfalls and challenges and show you how to avoid them. The second half of the book concludes with two real-world cloud native transformation case studies, illustrated by patterns. Starling Bank’s founding CTO Greg Hawkins describes how he and his team built a cloud native challenger bank in a year, and Senior Cloud Platform Strategist Daniel Eichten tells us how he helped lead Adidas, the second largest sports apparel manufacturer and retailer in the world, through the IT jungle and onto the cloud.
All of this is rooted in our own experience, not from any kind of formal research (and, realistically, this is such new knowledge that formal research doesn’t really exist yet). These are our opinions and observations, and as such not offered as any kind of definitive Ten Commandments of cloud native.
Our hope is that showing, rather than simply telling, how all of these complicated pieces fit together makes it easier to grasp how a functional and efficient cloud native system is made. Committing to a full organizational transformation is anxiety-provoking because you are stepping off the known path into a completely new world up there in the clouds. Patterns give us a series of small steps to follow along this new path, so as to slowly and gradually shift in the right direction. And the story of those who went before you, facing some challenges commonly found along the way but still successfully completing the journey, lends the confidence that you can do it too.
More importantly, though, you will learn that you can also use everything you learn from this book to do it again and again. We aren’t trying to teach you how to “do” cloud native. We are trying to teach you how to become a responsive and adaptive organization, able to move fast to whatever comes next—without breaking things.
Latest Pattern Developments
New cloud native patterns are emerging constantly. To continue sharing them and extending the cloud native pattern language we have established here, please visit www.CNpatterns.org.
This is where to find the latest pattern developments, but also an online community for discussing and creating new patterns. We are inviting people from across the industry, thought leaders and influencers but most importantly everyday engineers and managers—those out there working elbows-deep in cloud native code and architecture—to contribute and participate. Hope to see you there!
Conventions Used in This Book
The following typographical conventions are used in this book:
This element signifies a tip or suggestion.
This element signifies a general note.
This element indicates a warning or caution.
O’Reilly Online Learning
For more than 40 years, O’Reilly Media has provided technology and business training, knowledge, and insight to help companies succeed.
Our unique network of experts and innovators share their knowledge and expertise through books, articles, conferences, and our online learning platform. O’Reilly’s online learning platform gives you on-demand access to live training courses, in-depth learning paths, interactive coding environments, and a vast collection of text and video from O’Reilly and 200+ other publishers. For more information, please visit http://oreilly.com.
How to Contact Us
Please address comments and questions concerning this book to the publisher:
- O’Reilly Media, Inc.
- 1005 Gravenstein Highway North
- Sebastopol, CA 95472
- 800-998-9938 (in the United States or Canada)
- 707-829-0515 (international or local)
- 707-829-0104 (fax)
We have a web page for this book, where we list errata, examples, and any additional information. You can access this page at https://oreil.ly/cloud-nat-tr.
Email [email protected] to comment or ask technical questions about this book.
For more information about our books, courses, conferences, and news, see our website at http://www.oreilly.com.
Find us on Facebook: http://facebook.com/oreilly
Follow us on Twitter: http://twitter.com/oreillymedia
Watch us on YouTube: http://www.youtube.com/oreillymedia
Acknowledgments
The authors have many people to thank for helping this book happen.
First, the whole Container Solutions team has contributed to us actually bringing this book into being. We would especially like to shout out to the engineers who contributed their knowledge and experience to the creation of these patterns and all the concepts surrounding them. The CS design team worked both creatively and proficiently to deliver the drawings and diagrams that bring this material to life, and so extra special thanks go to Svitlana Chunyayeva, Mickey Houlder, and Wijnand Lustof. Heather Joslyn’s expert editing and Productive Feedback as the chapters emerged improved the text immensely.
We would also like to thank the editorial and production staff at O’Reilly, including Christopher Guzikowski for his willingness to take a chance on our slightly radical storytelling approach to pattern design. Particular thanks go to Michele Cronin, whose deft guidance helped this book take shape—she always knows just when to offer encouragement (and chocolate).
Additional thanks to the many people who generously gave their time and insight to make the book better. Anne Currie, who was there from the beginning, gets special credit for inspiring a lively tone throughout. Adrian Mouat, for his honest and conscientious review and comments throughout the project, which significantly helped the authors clarify and improve the material. Hans Wegener, who shepherded and then helped create Pini’s first PLoP (Pattern Languages of Programs) conference paper on cloud native patterns, which was the genesis for this book. As well as Kyle Brown, who as head of the PLoP organizing committee encouraged Pini to improve and refine the paper and continue investing time into patterns. We also thank Kyle for serving as a technical reviewer and especially for lending his particular expertise in reviewing and revising the patterns.
Jamie would like to thank all of Container Solutions’ earliest customers who were willing to take risks with new ideas, new technologies, and, back then, a new company. They are Henk Kolk, of ING Bank; Max Schöfmann, then of HolidayCheck; and Ken Owens, then of Cisco. Without you, Container Solutions wouldn’t be here, and this book wouldn’t have been written. Thanks to the team at Container Solutions who continue to amaze on a daily basis with their utterly insane appetite for experimentation and learning. To Adrian Mouat, for his friendship for the last 17 years and his continued compassionate and critical support. To Andrew Hesketh, Ian Ravenscroft, and Andrew Clark simply for putting up with Jamie. To Andrea Dobson, Jamie’s wife, and constant source of love, compassion, and honesty. Finally, and most importantly, Jamie would like to thank Pini, his partner for half a decade, and Michelle, without whom this book would never have been started let alone finished.
Pini would like to thank his wife, Sarit, for her unwavering support over the past 20 years. She had to listen for endless hours about patterns and encouraged him to invest many evenings and weekends into writing this book, all while trying to build Container Solutions into the amazing company it has become. He would also like to thank Sarit for being the best mom for our daughter, Shachar, and son, Yonatan, who always give us the inspiration to become the best possible role models we could ever be.
Michelle would like to thank her family, and especially her sons, Jack and Cole, for enduring all the evenings and weekends she disappeared into her office to work on this book. She is grateful to her husband, Jeff, for his support throughout the project and patience in listening to endless talk about cloud native in general and patterns in particular. She would like to thank Sally Neustadt for providing invaluable encouragement and insight in non-work areas of life, all of which also flowed into this project. Most of all, Michelle would like to thank her mother, Marlene Gienow, for teaching by example the value of hard work, persistence, and a glass of wine at the end of a productive day.
And, finally, thanks is due to all the other countless folks who contributed their support and ideas to this project. We may not have had room to include your names here, but we are grateful to you nonetheless!