When we talk about CI/CD, it infers that we want to enhance our product continuously without compromising on quality. The frequency of delivery may vary from product to product. It also depends on how robust the pipeline is for CI/CD. Budget is another factor where organizations may slow down on delivery as they are not comfortable with the cost of more frequent deployments, but let’s not discuss this here as we should only focus on the solution we can offer to deliver continuous quality.
Test automation merely plays a small part in continuous delivery as this is just one part of the picture yet the importance of it cannot be denied. Let’s first understand about continuous delivery and then we will talk about where test automation comes into the picture.
For creating a delivery pipeline of a software application, you need a distributed version control system for the code management which enables you to create the differentiation between new versus old code. Code Management tools are necessary so that we can fall back to the older version of the code in case of any issues. Developers can work with their own branches and merge the code as needed. GIT is a well know system to manage all such needs. Now when it comes to deployment, we have to execute multiple jobs in chronological order such that when different components put together they behave in a planned way. Out of all the integrations serves available, Jenkins stands out when it comes to robustness and cost-effectiveness. It has been there in the building the continuous pipeline for a while now.
Consider the continuous delivery system as a factory which is producing products on a line. The production line is as good as if all the supporting processes are running is a perfect sequential way, maintaining the same speed. If anyone of the processes slows down, it is going to turn the whole production line down. The same case applies to automate testing in continuous delivery model. We cannot just put a manual tester to the production line and slow the whole thing down.
An automation framework is a must which can be integrated with the CI/CD pipeline. If we talk about Jenkins, the edge we get using this automation server is that it supports all types of testing frameworks, may it be off the shelf or a custom built. The first step is to identify the layers of testing on which the automation needs to be performed. It can start from the infrastructure level testing to functional UI testing. As CI/CD allows us to fail fast and fix fast, the first level of testing at infrastructure can help determine if we want to proceed further with the code deployment or not. If the first layer of testing passes, we may move to API/DB level testing given us more confident to proceed further with the deployment. And once that test pass as well, we run the subset of the smoke test also called BVTs to further gain confidence in the environment to start the large scale testing.
The question is why we are domain so many tests before starting the testing on a larger scale. The answer is simple, once the testing begins on a larger scale, it starts provision the resources which are required for testing and hence start incurring the costs. Introducing layers of testing help us improve the chances that the functional testing will run smoothly after the testing infrastructure level or API/DB level is completed.
Large organizations do not really stick to a single product of platforms. Their product portfolio may have been a result of continuous acquisitions or mergers. In the end, the software development team ends up with a collage of tools and technologies to work with. Creating multiple pipelines on a single tool bring up its challenges. The same challenge is then transferred to building a single automation framework. Languages like Python are increasingly become popular due to this capability. They work really well when it comes to integration with any tool or language. The automation framework must to be robust and resilient so that it can be integrated with all types of tests that may run on the infrastructure level or on the UI level. In fact, the test automation framework and its capabilities will impact greatly on the CI/CD pipeline and therefore all the necessary dependencies should be addressed before building a framework for CI/CD pipeline.
As it’s said “You are as strong as the weakest link” continuous delivery is not achievable without continuous testing and continuous automation. It’s not as simple as it seems, it has challenges. There will be areas which cannot be automated what so ever, so targeting for an ideal case will not be a good idea. In the end, it all boils down to the deployment strategy, the strategy you create around your CI/CD pipeline and then for the test automation. Known to all, but worth mention here again is that taking a step-by-step approach is the only way to go. So decide on what to automate first, which automation layer will yield the best outcome and what to cover for automation. Remember, automation required skills, time and resources, only better planning can help enhance the skills, reduce the costs and at the end, gain great results.
