front matter

foreword

A service mesh can maximize the development speed of your whole organization by enabling thousands of independent microservices that are automatically compliant with a wide range of evolving policies. This book discusses many other benefits of Istio, but they largely follow from this premise.

This brings us to the central question, “What is a service mesh, and why do I need one?” I’m asked this question frequently, and the answer is not trivial. It is not about security or telemetry or most of the benefits some people claim. Nor is it automatically the case that you need a service mesh for your application, especially if it is a monolith.

The real answer has to do with decoupling applications from infrastructure. Istio is the third major step in that direction. First, Docker provided a way to package an application (and its library choices) separately from the machine on which it runs. Next, Kubernetes made it easy to create a service with automation to help with autoscaling and management. Together, Docker and Kubernetes enabled the practical movement to fine-grain services, often called microservices. This book guides you through implementing a service mesh with Istio to achieve this third step: application decoupling.

Microservices allow greater overall velocity by enabling teams to be more autonomous. Ideally, your team can update its microservice(s) without deep interactions with other teams. The top-level goal for Istio is to enable this at scale—to make it easy to have thousands of microservices (Google has more than a million!).

But enabling velocity for a service is not just about decoupling it from the machine; the service must also be decoupled from shared policies. Every enterprise has policies that apply to all services, and we must be able to change those policies quickly if needed. Traditionally, such policies are embedded inside services as part of the code or as libraries that services are expected to use. Either way, such policies are hard to update and enforce.

Istio moves a wide range of such policies (primarily those involving an API) out of the service and into the service mesh, which is essentially a proxy that sits in front of the service and implements the policies. When this is done correctly, all services meet the policies with no work—and, conversely, policy changes do not require updating the services. This is the decoupling we are after.

In Istio in Action, Christian and Rinor present a clear-headed vision of how to achieve the goal of decoupling applications from infrastructure. I hope you’ll enjoy this book as much as I have.

—Eric Brewer, VP Infrastructure and Google Fellow

preface

Building software is hard. Connecting different services across a network is harder. Any time you put a packet, message, or request on the network, there are no guarantees about its outcome. Will the request make it? How long will it take? Will anyone know if the communication fails?

Docker and Kubernetes have done a lot to support distributed services architectures like microservices, but they exacerbate the existing communication problems. One misbehaving service might take down everything.

While working with organizations worldwide that are trying to adopt microservices, I find that getting teams to consistently think about and solve these communication problems is very difficult. There are many questions: How will they do service discovery? Timeouts? Retries? Circuit breaking? Tracing? Authentication? Large cloud companies like Netflix, Twitter, and Google pioneered some of the early, successful microservices architectures. These companies had to build a lot of their own developer tooling and infrastructure to solve these problems, and fortunately, they open sourced much of it. Could other organizations use the NetflixOSS stack or Twitter Finagle? They could, and some did, but doing so created a new operational nightmare.

For example, the NetflixOSS stack was primarily written for Java developers. What about NodeJS, Golang, and Python teams? Teams had to either build libraries themselves or hack together the functionality with various bits they found on the internet. They also had to intermingle this “networking” code into their business logic. This approach added transitive dependencies, cluttered the code, and made revisions more difficult. Operating a service architecture with these application networking libraries, upgrading, patching, and doing this consistently across many different languages was extremely complex and error prone.

A service mesh is a cleaner solution to this application-networking problem. With a service mesh, we abstract away the application-networking logic into a dedicated piece of infrastructure and apply it to all services regardless of what languages they are written in.

Istio is a scalable, mature, powerful service-mesh implementation that originally came out of a project from IBM and Google. I was introduced to the Istio team in January 2017 and began working on the project very early. At the end of 2018, I went to work as global field CTO at a startup, Solo.io, to focus full-time on service mesh technology and advancing the state of application networking.

Building a startup from the ground up, pushing the boundaries on this technology, and writing an in-depth book on this topic is not an easy combination. I needed someone with dedication and passion to help me move the book forward; so, when I was halfway through, the Manning team and I invited Rinor Maloku to join the effort. Thanks to the time we both spend in the community and working with our customers at Solo.io, some of which are the largest deployments of Istio in the world, Rinor and I have been able to compile an excellent resource for Istio based on real-world experience. We hope this book will show you the value and power of Istio and make you comfortable adopting this technology into production, as many others have.

acknowledgments

Writing this book has been possible only due to the support of many people.

Special thanks to our personal friends Gentrina Gashi, Dimal Zeqiri, and Taulant Mehmeti, who provided such valuable feedback.

Thank you to the Manning Early Access Program (MEAP) readers who posted comments in the online forum—Takahiko Suzuki, George Tseres, Amol Nayak, Mark O’Crally—and the forum moderator, Ayush Singh.

Sincere gratitude to our editor, Elesha Hyde, who was always patient when responding to our questions and concerns. Most important, thank you for being understanding and supportive when we missed deadlines, and for keeping our focus on writing a better book for our readers.

A big thank you also goes to our technical editors, Gregor Zurowski and Brent Stains, as well as our technical proofreader, Gregory Reshetniak.

Thank you to all the reviewers: Alceu Rodrigues de Freitas Junior, Alessandro Campeis, Allan Makura, Amitabh Cheekoth, Andrea Cosentino, Andrea Tarocchi, Andres Sacco, Borko Djurkovic, Christoph Schubert, Dinkar Gupta, Eriks Zelenka, Ernesto Cardenas, Fotis Stamatelopoulos, Giuseppe Catalano, James Liu, Javier Muñoz, Jeff Hajewski, Karthikeyan Mohan, Kelum Prabath Senanayake, Kent R. Spillner, Leonardo Jose Gomes da Silva, Maciej Droz.dz.owski,Michael Bright, Michael J Haller, Morgan Nelson, Paolo Antinori, Salvatore Campagna, Satadru Roy, Stanley Anozie, Taylor Dolezal, Vijay Thakorlal, and Yogesh Shetty. Your excellent suggestions made this a better book.

Thank you to key people in the Istio community, including Louis Ryan (Google), Shriram Rajagopalan (Google), and Sven Mawson (Google) who are the three founders of the Istio project; along with Dan Berg (Digital.ai), Lin Sun (Solo.io), Dan Ciruli (Zuora), Idit Levine (Solo.io), John Howard (Google), Kevin Connor (Red Hat), Jason McGee (IBM), Zack Butcher (Tetrate), Ram Vennam (Solo.io), and Neeraj Poddar (Solo.io).

Finally, we thank the entire Istio community, who are working to build an amazing technology that is a joy to work on and write about on a daily basis.

I am extremely fortunate to work on technology, which has been my passion since I was young. I wouldn’t have been able to get here without the love and support of my family. My father, Cask Posta, immigrated to the United States in the early 1970s; he provided me and my sister with a strong foundation and taught me the value of hard work. My beautiful wife Jackie has been by my side through all the unknowns and uncertainties and has shown nothing but unwavering support and love. Thank you, Jackie: without you, we wouldn’t have been able to accomplish all we’ve done.

Finally, I’m thankful for my two lovely daughters, Maddie and Claire, who can put a smile on my face no matter how the day has been and for whom my wife and I work so hard.

—Christian E. Posta

I’d like to thank my parents, Sahadi and Sheride, and my brothers, Aurel and Drilon, for giving me a worry-less childhood that allowed me to explore my many hobbies. One of those, programming, turned into a joyful career; and another, writing, contributed to this book. I’d also like to thank my girlfriend, Rinora, for her endless love and support. To Christian Posta, I’d like to express my eternal gratitude for the trust you put in me. It was crucial in enabling me to do my best work as a first-time writer.

—Rinor Maloku

about this book

Who should read this book

This book is meant for developers, architects, and or service operators who operate or are planning to operate distributed services such as user-facing web applications, APIs, and backend services and want to provide highly available services to their end users. If you are a member of a platform engineering team and provide infrastructure and other supporting components such as log management, monitoring, container orchestration, and so on to many development teams within your organization, this book will show you how to give your users the tools to make their apps resilient, secure, and observable as well as reduce the risk of shipping new features.

If you are already using Istio in a testing or staging environment but many of its workings are a mystery to you, this book will demystify Istio’s components. The latter chapters, in particular, will show you how to scale the service mesh in your organization, troubleshoot it when its behavior doesn’t match your expectations, and customize it to meet your enterprise’s needs.

If you are already an Istio expert, you may still find this book useful, as we took great care to ingrain into this book what we’ve learned from working in the field over the last three years.

If building a container is new to you, or you are unsure what a Kubernetes deployment, pod, or service is, this book may not be for you—yet. There are a lot of resources to get you started. We highly recommend Kubernetes in Action by Marko Lukša (Manning, 2017); in addition to being a thorough introduction to this topic, the book is a real page-turner. Once you understand the foundation of Kubernetes and its resources and how Kubernetes controllers work, you can return here and dive into the Istio service mesh.

You should also have a basic understanding of networking, and we do mean basic. If you are familiar with the network layer (layer 3), the transport layer (layer 4), and how they differ from the application (layer 7) according to the Open Systems Interconnection model, you are ready for this book.

How this book is organized: A roadmap

This book has four parts and 14 chapters. Part 1 of the book introduces the concept of a service mesh and explains how Istio implements it. These three chapters cover the architecture of Istio, how Envoy fits into it, and how it can benefit your organization:

• Chapter 1 introduces the benefits of Istio and the value that adopting service meshes can bring to an organization.

• Chapter 2 is a hands-on tutorial for installing Istio in a Kubernetes cluster. You deploy and integrate your first application into the mesh and configure it with Istio’s custom resources. Using the demo application, this chapter provides an overview of what you get out of the box with Istio and covers traffic management, observability, and security.

• Chapter 3 is all about Envoy: how it came to be, what problems it solves, and how it fits within the service mesh architecture.

Part 2 is a deep dive into Istio. The focus switches to practical examples, and we answer key operational questions: how to secure traffic coming into your cluster, make services more resilient, and make your system observable using the telemetry generated by the service proxies. This part contains six chapters:

• Chapter 4 teaches you how to use and configure the Istio ingress gateway to route traffic securely from the public network to your services (what we call north-south traffic).

• Chapter 5 proceeds after traffic is admitted into the cluster. It shows how VirtualService. and DestinationRule. are used to route traffic in a fine-grained manner, enabling complex deployment patterns to reduce risk when you release new software.

• Chapter 6 explores how Istio benefits application teams. We discuss making services robust by implementing retries, circuit-breaking, load balancing across regions, and locality-aware load balancing right in the service mesh.

• Chapter 7 teaches you how Istio makes services observable by generating metrics, traces, and logs. Here we dive deeper into the metrics generated by service proxies, what information the metrics record, and how the recorded information can be customized.

• Chapter 8 shows you how to use telemetry visualization tools to make sense of the collected data. You use Prometheus to collect metrics and Grafana to visualize them. You use Jaeger to stitch together the traces of a request traveling through your services. And we show how Kiali intertwines this information to make troubleshooting services in the mesh a breeze.

• Chapter 9 elaborates on how Istio secures service-to-service traffic, how services receive their identity, and how the identity is used to implement access control and reduce the potential attack scope.

Part 3 is all about day-2 operations. It shows you how to troubleshoot issues in the data plane and maintain the control plane’s stability and performance. By the end of this part, you will have a firm understanding of Istio’s internals, and you will be able to discover and fix issues on your own:

• Chapter 10 shows you how to troubleshoot issues in the data plane using tools such as Istioctl, Kiali, and telemetry that is collected and visualized.

• Chapter 11 discusses Istio’s performance factors. It shows how Istio can be configured to make the control plane more performant—the foundation of a robust service mesh.

The fourth and final part of the book shows you how to make Istio yours. Enterprises have services running across boundaries, such as different clusters, different networks, or a mixture of cloud-native and legacy workloads. By the end of part 4, you will know how to join your workloads into a single mesh and customize the mesh’s behavior using WebAssembly to meet your unique requirements:

• Chapter 12 shows you how to connect workloads in different Kubernetes clusters wherever they are running, such as different cloud providers, on premises, or in a hybrid cloud.

• Chapter 13 shows how to integrate legacy workloads running in virtual machines into the mesh and extend to those workloads the mesh’s capabilities of resiliency and high availability.

• Chapter 14 teaches you how to extend and customize Istio’s capabilities with existing Envoy functionality or your code using Lua scripting and WebAssembly.

About the code

This book contains many examples of source code in numbered listings and in line with normal text. In both cases, source code is formatted in a fixed-width font like this to separate it from ordinary text. Sometimes code is also in bold to highlight code that has changed from previous steps in the chapter, such as when a new feature adds to an existing line of code.

In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. In rare cases, even this was not enough, and listings include line-continuation markers (➥). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany many of the listings, highlighting important concepts.

You can get executable snippets of code from the liveBook (online) version of this book at https://livebook.manning.com/book/istio-in-action. The complete code for the examples in the book is available for download from the Manning website at www.manning.com, and from GitHub at https://github.com/istioinaction/book-source-code.

liveBook discussion forum

Purchase of Istio in Action includes free access to liveBook, Manning’s online reading platform. Using liveBook’s exclusive discussion features, you can attach comments to the book globally or to specific sections or paragraphs. It’s a snap to make notes for yourself, ask and answer technical questions, and receive help from the authors and other users. To access the forum, go to https://livebook.manning.com/book/istio-in-action/discussion. You can also learn more about Manning’s forums and the rules of conduct at https://livebook.manning.com/discussion.

Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the authors can take place. It is not a commitment to any specific amount of participation on the part of the authors, whose contribution to the forum remains voluntary (and unpaid). We suggest you try asking the authors some challenging questions lest their interests stray! The forum and the archives of previous discussions will be accessible from the publisher’s website as long as the book is in print.

about the authors

Christian Posta (@christianposta) is VP, Global Field CTO at Solo.io. He is well known in the cloud-native community for being an author, blogger (https://blog.christianposta.com), speaker, and contributor to various open-source projects in the service mesh and cloud-native ecosystem. Christian has spent time at enterprises and web-scale companies and now helps organizations create and deploy large-scale, cloud-native, resilient, distributed architectures. He enjoys mentoring, training, and leading teams to be successful with distributed systems concepts, microservices, DevOps, and cloud-native application design.

Rinor Maloku (@rinormaloku) is an engineer at Solo.io, where he consults clients adopting application networking solutions, such as service meshes. Previously, he worked at Red Hat, where he built middleware software that enabled teams to ensure the high availability of their services. As a freelancer, he consulted multiple DAX 30 members in their endeavor to fully utilize the potential of cloud computing technologies.

about the cover illustration

The figure on the cover of Istio in Action is “Femme Islandoise,” or “Icelandic woman,” taken from a collection by Jacques Grasset de Saint-Sauveur, published in 1797. Each illustration is finely drawn and colored by hand.

In those days, it was easy to identify where people lived and what their trade or station in life was just by their dress. Manning celebrates the inventiveness and initiative of the computer business with book covers based on the rich diversity of regional culture centuries ago, brought back to life by pictures from collections such as this one.