We put in place a basic, yet functional CI pipeline. While it is a good starting point, you will typically want to perfect certain details of this system.
As previously mentioned, our Ansible recipe for Jenkins can be improved to include the configuration of the jobs such as the helloworld job we created manually.
We only created a single functional test to illustrate using a TDD approach and integrating a testing step in our pipeline. The success of a CI pipeline depends highly on the quality and quantity of the tests produced. Tests will typically be broken up into functional and nonfunctional tests. In order to take best advantage of your pipeline, you will want to catch possible errors as early as possible. This means focusing on the functional tests and in particular the unit tests, which are used to validate small units of code such as a method in a class.
After this, you can focus on the integration testing, which covers a bit more ground and usually interacts with data stores and other functions in the code. Finally, you will want to add some acceptance testing to verify that all the requirements for your stories are complete:
In terms of nonfunctional testing, you will usually want to look at performance, security, usability, and compatibility testing.
Finally, you can complement your own tests with code analyzer tools to get a sense of the code coverage (how many lines of code are executed by your automated tests).
As always with DevOps, it is important to collect metrics. In a CI pipeline, you will typically want to monitor the number of builds that go through the CI pipeline and the quality of the pull requests.
Like any other system, you will need to spend some time setting up backups and monitoring. You may decide to back up the Jenkins home directory if you haven't moved to a model where your jobs and the Jenkins configuration are managed by your configuration management system (Ansible). In terms of metrics, keeping an eye on the system performance, its availability, and health are paramount. A build pipeline breakage should be considered a critical issue as it impacts all the developers' and operators' productivity.
Finally, it is expected to have to scale up your CI infrastructure over time. As code and tests get added, it will take longer and longer to run the tests. You can decide to add more Jenkins slaves, which will allow you to run tests in parallel or/and use bigger instances. In the current format, Jenkins will run the helloworld pipeline every time a change is pushed to a branch. You may also decide only to run the pipeline once the pull requests are created.
In this initial section of the chapter, we adopted a new workflow where developers can commit code and tests to individual branches and send frequent pull requests to share the proposed changes with the rest of the engineering organization. In addition, we made sure that the new code is fully tested by creating a continuous integration pipeline. To do so, we created a Jenkins server and hooked it to GitHub. Thanks to that system, all the tests committed with the project will get automatically executed and the results will be sent back to GitHub. We are now in an ideal situation to take our workflow to the next level and build some automation around deployment.
Yes and no. In an effective DevOps organization, nontechnical QA jobs are usually gone. If everything is fully automated and the developers wrote enough tests to cover all aspects of the code, the organization doesn't need to task anyone to write and execute test plans. Instead of that, DevOps-focused organizations will have engineers sometimes called QA engineers focusing on quality but with automation in mind. This means working on tooling and processes to improve the ability to automatically test code.