One key devops best practice is instrumenting a continuous integration/continuousdelivery (CI/CD) pipeline that automates the process of building software, packaging applications, deploying them to target environments, and instrumenting service calls to enable the application. This automation requires scripting individual procedures and orchestrating the steps from code checkin to running application. Once matured, devops teams use the automation to drive process change and strive to do smaller, more frequent deployments that deliver new functionality to users and improve quality.
But there’s a significant assumption that the automation coupled with frequent deployments will drive quality. Here’s what test automation has to test for: Have the application and code changes been tested thoroughly? Does the release meet the minimal acceptance criteria for deployment? Will the new release introduce production defects that affect users, are difficult to debug, and may be disruptive to the organization to resolve? Has performance of the application been evaluated sufficiently? Has the application been tested for known security vulnerabilities?
Defined: What is continuous testing?
To be truly running CI/CD, testing must be automated and integrated into the CI/CD pipeline. In other words, the development teams that target and achieve this goal are implementing continuous testing.
Continuous testing requires teams to have already automated a set of tests that can be plugged into the CI/CD pipeline. Although testing applications is done in most organizations developing software, having a technical practice in place that evaluate risks, prioritizes quality assurance implementations, and automates the most critical application tests requires people, practice, and technology to establish.
