front matter

preface

APIs and microservices have taken the software industry by storm. Under the pressure of increasing software complexity and the need to scale, more and more organizations are migrating from monolithic to microservices architecture. O’Reilly’s “Microservices Adoption in 2020” report found that 77% of respondents had adopted microservices, a trend that is expected to continue growing in the coming years.

Using microservices poses the challenge of driving service integrations through APIs. According to Nordic APIs, 90% of developers work with APIs and they spend 30% of their time building APIs.¹ The growth of the API economy has transformed the way we build applications. Today, it’s more and more common to build products and services that are delivered entirely over APIs, such as Twilio and Stripe. Even traditional sectors like banking and insurance are finding new lines of business by opening their APIs and integrating within the Open Banking ecosystem. The wide availability of API-first products means that we can focus on our core business capabilities when building our own applications, while using external APIs to handle common tasks such as authenticating users and sending emails.

It’s exciting to be part of this growing ecosystem. However, before we embrace microservices and APIs, we need to know how to architect microservices, how to design APIs, how to define an API strategy, how to make sure we deliver reliable integrations, how to choose a deployment model, and how to protect our systems. In my experience, most organizations struggle with one or more of these questions, and a recent report by IBM found that 31% of businesses haven’t adopted microservices due to lack of internal expertise.² Equally, Postman’s 2022 State of the API Report found that 14% of respondents experience API integration failures 11%–25% of the time (http://mng.bz/Xa9v), and according to Salt Security, 94% of organizations experienced API security incidents in 2022.³

Many books address the problems mentioned in the previous paragraph, but they typically do it from a highly specific point of view: some focus on architecture, others on APIs, and yet others on security. I felt there’s a gap for a book that brings all these questions together and addresses them with a practical approach: essentially, a book that can get an average developer up and running quickly with the best practices, principles, and patterns for designing and building microservice APIs. I wrote this book with that goal in mind.

Over the past years, I’ve had the opportunity to work with different clients helping them to architect microservices and deliver API integrations. Working on those projects gave me a vantage view into the major hurdles that development teams face when working with microservices and APIs. As it turns out, both technologies are deceivingly simple. A well-designed API is easy to navigate and consume, while well-architected microservices boost developer productivity and are easily scalable. On the other side of the spectrum, badly designed APIs are error prone and difficult to use, and badly architected microservices result in so-called distributed monoliths.

The obvious questions arise: How do you design good APIs? And how do you architect loosely coupled microservices? This book will help you answer these questions and more. You’ll also get your hands dirty building APIs and services, and you’ll learn how to secure them, test them, and deploy them. The methods, patterns, and principles that I teach in this book are the outcome of many years of trials and experimentation, and I’m very excited about sharing them with you. I hope you find this book a valuable resource in your journey towards becoming a better software developer and architect.

acknowledgments

Writing this book has been one of the most fascinating journeys in my career, and I couldn’t have done it without the help and support of my family and an amazing team of colleagues. The book is dedicated to my wonderful wife, Jiwon, without whose constant encouragement and understanding I wouldn’t have been able to complete this book, and to our daughter, Ivy, who made sure I never had a dull moment in my schedule.

I have benefited enormously from the people who contributed ideas for the book, helped me better understand the tools and protocols I use in it, and provided feedback on various chapters and drafts. Special thanks go to Dmitry Dygalo, Kelvin Meeks, Sebastián Ramírez Montaño, Chris Richardson, Jean Yang, Gajendra Deshpande, Oscar Islas, Mehdi Medjaoui, Ben Hutton, Andrej Baranovskij, Alex Mystridis, Roope Hakulinen, Steve Ardagh-Walter, Kathrin Björkelund, Thomas Dean, Marco Antonio Sanz, Vincent Vandenborne, and the amazing maintainers of Ariadne at Mirumee.

Since 2020, I’ve presented drafts and ideas from the book at various conferences, including EuroPython, PyCon India, API World, API Specifications Conference, and various podcasts and meetups. I want to thank everyone who attended my presentations and gave me valuable feedback. I also want to thank the attendants to my workshops at microapis.io for their thoughtful comments on the book.

I want to thank my acquisitions editor, Andy Waldron. Andy did a brilliant job helping me get my book proposal in good shape and keeping the book focused on relevant topics. He also supported me tirelessly to promote the book and helped me to reach a wider audience.

The book you now have in your hands is readable and understandable thanks to the invaluable work of my editor, Marina Michaels, who went far and beyond to help me write a better book. She did an outstanding job helping me improve my writing style, and keeping me on track and motivated.

I want to thank my technical editor, Nick Watts, who rightly pointed out many inaccuracies and always challenged me to provide better explanations and illustrations, and my technical proofreader, Al Krinker, who diligently checked all the code listings and the GitHub repository for this book, making sure the code is correct and executes without issues.

I also want to thank the rest of the Manning team who was involved in the production of this book, including Candace Gillhoolley, Gloria Lukos, Stjepan Jureković, Christopher Kaufmann, Radmila Ercegovac, Mihaela Batinić, Ana Romac, Aira Dučić, Melissa Ice, Eleonor Gardner, Breckyn Ely, Paul Wells, Andy Marinkovich, Katie Tennant, Michele Mitchell, Sam Wood, Paul Spratley, Nick Nason, and Rebecca Rinehart. Thanks also go to Marjan Bace for betting on me and giving this book a chance.

While working on this book, I had the opportunity to receive detailed and outstanding feedback from the most amazing group of reviewers, including Alain Lompo, Björn Neuhaus, Bryan Miller, Clifford Thurber, David Paccoud, Debmalya Jash, Gaurav Sood, George Haines, Glenn Leo Swonk, Hartmut Palm, Ikechukwu Okonkwo, Jan Pieter Herweijer, Joey Smith, Juan Jimenez, Justin Baur, Krzysztof Kamyczek, Manish Jain, Marcus Young, Mathijs Affourtit, Matthieu Evrin, Michael Bright, Michael Rybintsev, Michal Rutka, Miguel Montalvo, Ninoslav Cerkez, Pierre-Michel Ansel, Rafael Aiquel, Robert Kulagowski, Rodney Weis, Sambasiva Andaluri, Satej Kumar Sahu, Simeon Leyzerzon, Steven K Makunzva, Stuart Woodward, Stuti Verma, and William Jamir Silva. I credit them all with much of the good content that made its way into the book.

Since the book went into MEAP, I’ve been blessed by the words of encouragement and feedback that many of my readers sent me through various channels, such as LinkedIn and Twitter. I was also lucky to converse with a brilliant community of readers who actively participated in the book’s forum in Manning’s liveBook platform. I’m heartily grateful to all of you.

This book wouldn’t have been possible without the tireless work of thousands of open source contributors who created and maintain the amazing libraries that I use in this book. I’m very thankful to all of you, and I hope my book helps to make your amazing work more visible.

Finally, thank you, the reader, for acquiring a copy of my book. I can only hope that you find this book useful and informative and that you enjoy reading it as much as I enjoyed writing it. I love to hear from my readers, and I’d be delighted if you share your thoughts on the book with me.

about this book

The goal of this book is to teach you how to build microservices and drive their integrations using APIs. You’ll learn to design a microservices platform and to build REST and GraphQL APIs to enable communication between microservices. You’ll also learn to test and validate your microservice APIs, to secure them, and to deploy and operate them in the cloud.

Who should read this book?

This book is helpful for software developers who work with microservices and APIs. The book uses a very practical approach, and nearly every chapter illustrates the explanations with full coding examples. Therefore, hands-on developers who work directly with microservice APIs will find the book’s contents valuable.

The coding examples are in Python; however, knowledge of the language isn’t necessary to be able to follow along with them. Before introducing new code, every concept is explained thoroughly.

The book contains a lot of emphasis on design strategies, best practices, and development workflows, and therefore it’s also useful for CTOs, architects, and VPs of engineering who need to decide whether microservices are the right architectural solution for them, or who need to choose between different API strategies and how to make the integrations work.

How this book is organized: A roadmap

The book is divided into four sections with a total of 14 chapters.

Part 1 introduces the concepts of microservices and APIs, shows how to build a simple API, and explains how to design a microservices platform:

• Chapter 1 introduces the main concepts of the book: microservices and APIs. It explains how microservices differ from monolithic architecture, and when it makes sense to use monoliths versus microservices. It also explains what APIs are and how they help us drive integrations between microservices.

• Chapter 2 offers a step-by-step guide for implementing APIs using Python’s popular FastAPI framework. You’ll learn to read an API specification and understand its requirements. You’ll also learn to build APIs in gradual steps, and how to test your data validation models.

• Chapter 3 explains how to design a microservices platform. It introduces three fundamental microservice design principles, and it explains how to decompose a system into microservices, using decomposition by business capability and decomposition by subdomains.

Part 2 explains how to design, document, and build REST APIs, and how to build a microservice:

• Chapter 4 explains the design principles of REST APIs. It introduces the six constraints of REST architecture and the Richardson Maturity Model, and then moves on to explain how we leverage the HTTP protocol to design well-structured and highly expressive REST APIs.

• Chapter 5 explains how to document a REST API using the OpenAPI specification standard. You’ll learn the basics of JSON Schema syntax, how to define endpoints, how to model your data, and how to refactor your documentation with reusable schemas.

• Chapter 6 explains how to build REST APIs using two popular Python frameworks: FastAPI and Flask. You’ll learn about the differences between the two frameworks, but you’ll also learn how the principles and patterns for building APIs remain the same and transcend the implementation details of any technical stack.

• Chapter 7 explains fundamental principles and patterns for building microservices. It introduces the concept of hexagonal architecture, and it explains how to enforce loose coupling between the layers of an application. It also explains how to implement database models using SQLAlchemy and how to manage database migrations using Alembic.

Part 3 explains how to design, consume, and build GraphQL APIs:

• Chapter 8 explains how to design GraphQL APIs and how the Schema Definition Language works. It introduces GraphQL’s built-in types, and it explains how to define custom types. You’ll learn how to create relationships between types, and how to define queries and mutations.

• Chapter 9 explains how to consume GraphQL APIs. You’ll learn to run a mock server and how to explore GraphQL documentation using GraphiQL. You’ll learn to run queries and mutations against a GraphQL server and how to parametrize your operations.

• Chapter 10 explains how to build GraphQL APIs using Python’s Ariadne framework. You’ll learn to leverage the API documentation to automatically load data validation models, and also to implement resolvers for custom types, queries, and mutations.

Part 4 explains how to test, secure, and deploy your microservice APIs:

• Chapter 11 explains how to add authentication and authorization to your APIs using standard protocols such as OpenID Connect (OIDC) and Open Authorization (OAuth) 2.1. You’ll learn how to produce and validate JSON Web Tokens (JWTs) and how to create an authorization middleware for your APIs.

• Chapter 12 explains how to test and validate your APIs. You’ll learn what property-based testing is and how to use it to test your APIs, and you’ll also learn to use API testing automation frameworks like Dredd and schemathesis.

• Chapter 13 explains how to Dockerize your microservice APIs, how to run them locally using Docker Compose, and how to publish your Docker builds to AWS Elastic Container Registry (ECR).

• Chapter 14 explains how to deploy your microservice APIs to AWS using Kubernetes. You’ll learn to create and operate a Kubernetes cluster using AWS EKS, how to launch an Aurora serverless database into a secure network, how to inject application configuration securely using envelope encryption, and how to set up your services to operate at scale.

All chapters have a common theme: building components of a fictitious, on-demand coffee delivery platform called CoffeeMesh. We introduce CoffeeMesh in chapter 1, and in chapter 3, we break the platform down into microservices. Therefore, I recommend reading chapters 1 and 3 to get a better understanding of the examples introduced in later chapters. Otherwise, every part of the book is fairly independent, and each chapter is pretty self-contained. For example, if you want to learn how to design and build REST APIs, you can jump straight to part 2, and if your interest lies with GraphQL APIs, you can focus on part 3. Equally, if you want to learn to add authentication and authorization to your APIs, you can jump straight into chapter 11, or if you want to learn how to test APIs, you can go directly to chapter 12.

There’re some cross-references between chapters: for example, chapter 12 references the API implementations from parts 2 and 3, but if you’re comfortable building APIs, you should be able to skip directly to chapter 12. The same is true for the other chapters in part 4.

About the code

This book contains many examples of source code both in numbered listings and in line with normal text. In both cases, source code is formatted in a fixed-width font like this to separate it from ordinary text. Sometimes code is also in bold to highlight code that has changed from previous steps in the chapter, such as when a new feature adds to an existing line of code.

In many cases, the original source code has been reformatted; we’ve added line breaks and reworked indentation to accommodate the available page space in the book. In some cases, even this was not enough, and listings include line-continuation markers (➥). Additionally, comments in the source code have often been removed from the listings when the code is described in the text. Code annotations accompany many of the listings, highlighting important concepts.

Except for chapters 1, 3, and 4, every chapter of the book is full of coding examples that illustrate every new concept and pattern introduced to the reader. Most of the coding examples are in Python, except in chapters 5, 8, and 9, which focus on API design, and therefore contain examples in OpenAPI/JSON Schema (chapter 5) and the Schema Definition Language (chapters 8 and 9). All the code is thoroughly explained, and therefore it should be accessible to all readers, including those who don’t know Python.

You can get executable snippets of code from the liveBook (online) version of this book at https://livebook.manning.com/book/microservice-apis. The complete code for the examples in the book is available for download from the Manning website at www.manning.com, and from a GitHub repository dedicated to this book at: https://github.com/abunuwas/microservice-apis. Every chapter has a corresponding folder in the GitHub repo, such as ch02 for chapter 2. Unless otherwise specified, all file references in each chapter are relative to their corresponding folder in GitHub. For example, in chapter 2, orders/app.py refers to the ch02/orders/app.py file in GitHub.

The GitHub repository for this book shows the final state of the code in every chapter. Some chapters show how to build features progressively, in iterative steps. In those cases, the version of the code you’ll find on GitHub matches the final version of the code in the chapter.

The Python code examples in the book have been tested with Python 3.10, although any version of Python upwards of 3.7 should work just the same. The code and the commands that I use throughout the book have been tested on a Mac machine, but they should work without problems on Windows and Linux as well. If you work on Windows, I recommend you use a POSIX-compatible terminal, such as Cygwin.

I’ve used Pipenv to manage dependencies in every chapter. In each chapter’s folder, you’ll find Pipfile and Pipfile.lock files that describe the exact dependencies that I used to run the code examples. To avoid problems running the code, I recommend you download those files at the start of every chapter, and install the dependencies from them.

liveBook discussion forum

Purchase of Microservice APIs includes free access to liveBook, Manning’s online reading platform. Using liveBook’s exclusive discussion features, you can attach comments to the book globally or to specific sections or paragraphs. It’s a snap to make notes for yourself, ask and answer technical questions, and receive help from the author and other users. To access the forum, go to https://livebook.manning.com/book/microservice-apis/discussion. You can also learn more about Manning’s forums and the rules of conduct at https://livebook.manning.com/discussion.

Manning’s commitment to our readers is to provide a venue where a meaningful dialogue between individual readers and between readers and the author can take place. It is not a commitment to any specific amount of participation on the part of the author, whose contribution to the forum remains voluntary (and unpaid). We suggest you try asking him some challenging questions lest his interest stray! The forum and the archives of previous discussions will be accessible from the publisher’s website for as long as the book is in print.

Other online resources

If you want to learn more about microservice APIs, you can check out my blog, https://microapis.io/blog, which contains additional resources that complement the lessons of this book. On the same website, I also keep an up-to-date list of workshops and seminars that I organize frequently, which also complement this book.

about the author

José Haro Peralta is a software and architecture consultant. With over 10 years of experience, José has helped organizations big and small to build complex systems, architect microservice platforms, and deliver API integrations. He’s also the founder of microapis.io, a company that provides software consulting and training services. Recognized as a thought leader in the fields of cloud computing, DevOps, and software automation, José speaks regularly at international conferences and frequently organizes public workshops and seminars.

about the cover illustration

The figure on the cover of Microservice APIs is captioned “L’invalide,” or “The Disabled,” and depicts a wounded French soldier who was a resident at the Hôtel national des Invalides, or National House of the Disabled. This image is taken from a collection by Jacques Grasset de Saint-Sauveur, published in 1797. Each illustration is finely drawn and colored by hand.

In those days, it was easy to identify where people lived and what their trade or station in life was just by their dress. Manning celebrates the inventiveness and initiative of the computer business with book covers based on the rich diversity of regional culture centuries ago, brought back to life by pictures from collections such as this one.

---

¹ J. Simpson, “20 Impressive API Economy Statistics” (https://nordicapis.com/20-impressive-api-economy-statistics/ [accessed May 26, 2022]).

² “Microservices in the enterprise, 2021: Real benefits, worth the challenges,” (https://www.ibm.com/downloads/cas/OQG4AJAM [accessed 26th May 2022]).

³ Salt Security, “State of API Security Q3 2022”, p. 4 (https://content.salt.security/state-api-report.html).