Building a continuous integration pipeline

Originally, working in a CI environment meant that developers had to commit their code in a common branch as frequently as possible (as opposed to working off a separate branch or not committing changes for weeks). This allowed for better visibility of the ongoing work and encouraged communication to avoid integration problems commonly known as integration hell. As the different tooling around source control and build and release management matured, so did the vision of how code integration should look in an ideal world.

Nowadays, most effective engineering organizations will continue down the path of integrating early and often, but with a more modern development process where developers are required to edit the code and, at the same time, add or edit the different relevant tests to validate the change, this change drastically increases the overall productivity as it is now easier to find new bugs as the amount of code changes between merges is fairly small.

To adopt such a workflow, using Git for example, you can proceed in the following way:

  1. When, as a developer, you want to make changes, you start by creating a new Git branch branching off of the HEAD of the master branch.
  2. Edit the code and, at the same time, add or edit the different relevant tests to validate the change.
  3. Test the code locally.
  4. When the code is ready, rebase the branch to integrate new eventual changes from other developers. If needed, resolve conflicts and test the code again.
  5. If everything went well, the next step consists of creating a pull request. In this process, you tell other developers that your code is ready to be reviewed.
  6. Once the pull request is created, an automated testing system, such as the one we will build in this chapter, will pick up the change and run the entire test suite available to make sure nothing is failing.
  7. In addition, other interested parties will review the code and the different tests that were added into the branch. If they are satisfied with the proposed change, they will approve it, giving the developers the green light to merge their change.
  8. In the last step, the developers will merge their pull requests, which will translate into merging their new code and tests into the master branch. Other developers will now integrate this change when they rebase or create new branches.

In the following section, we will create a CI server using Jenkins running on top of an EC2 instance and GitHub.

As projects get bigger, so does the number of tests and the time it takes to run them. While certain advanced build systems such as Bazel (bazel.io) have the ability to run only the tests relevant to a particular change, it is usually easier to start simply and create a CI system that runs all the tests available every time a new pull request is proposed. Having an external test infrastructure with the elasticity of AWS becomes a huge time-saver for the developers who don't have to wait minutes or even sometimes hours to see all the tests being executed. This book focuses on web application development, but you may face a more challenging environment where you need to build software for a specific hardware and OS. Having a dedicated CI system will allow you to run your tests on the hardware and software you are ultimately targeting.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset