Imagine this all-too-common scenario: Software testers are asked to check new application functionality after developers have deemed it “done.” They have a list of actions to perform, and they manually click through each and every step—trying to faithfully follow all instructions and methodically document the outcome of each step, in case something goes awry. They don’t really understand the business or the application’s users, and they rarely interact with the developers or stakeholders. Their primary goal is entering X and checking that the outcome is Y and not Z.
This is undeniably boring, but I don’t think it’s truly testing. It’s certainly not the type of software testing that’s required to deliver exceptional customer experiences faster than your competitors.
The digital transformation sea change is impacting virtually every department at every business today, and few QA departments will remain untouched. The resulting wave will likely wipe out the “boring as shit” manual verification I described above. But it could also elevate testing to a sexy discipline where testers become the primary stewards of the customer experience.
That’s probably the most memorable takeaway from “The Great Software Testing Debate,” a recent panel I participated on with Jeff Wilkinson, managing director at Accenture, and Anders Wallgren, CTO at Electric Cloud. As we debated the evolution of testing, we touched upon topics like the emerging role of SDETs and the future of TCoEs. We all agreed that the disruption we’re facing today presents a great opportunity to reposition testing as a discipline that’s driving innovation rather than dragging it down. However, the devil is in the details. What exactly needs to change, and how do we get there?
Here are insights gained from this discussion.
One team, one fight
Wallgren has a great mantra that captures how quality has become everyone’s responsibility: “‘One team, one fight.’ When customers see a bug, they don’t blame the testers or the developers. They blame the company that released the crappy software.”
Continuing on this theme, Wallgren has an interesting analogy for how to approach quality as a process: “Think of the School House Rock on how a bill becomes a law. This is how we need to think about our code. We need to consider all the different things that happen to our code as it passes along the conveyer belt of our software delivery factory, all the different people that are involved in the process. Once we have this mapped out, that’s when we can really start optimizing the process. If a bug escapes through that process and reaches the customer, this means that there’s a larger organizational failure that must be identified, analyzed, and addressed.”
The need for diversity
Wilkinson is a passionate champion of diversity. In past conferences, I’ve witnessed his moving presentations about how including employees from a broad mix of backgrounds not only benefits specific testers, it also enhances the quality of your overall testing.
In this panel, Wilkinson focuses on how the shift to devops is opening up a variety of options to accommodate different skill sets. For example, former manual testers can take a number of different paths to contribute to the automation effort. Those who want to continue along the testing path might learn model-based test automation and/or Selenium to remain marketable in the software testing field. Some might decide to focus on test data management. Others will be more inclined to branch off and master devops practices and platforms. There are 40,000 testers at Accenture. They don’t all have the same personalities and skill sets, and Accenture is a much stronger company as a result.
The importance of unit testing
Wallgren is a staunch proponent of unit testing. He requires engineers to write unit tests for their code because it’s so much faster and cheaper to expose errors at that level. In his experience, many issues found in production would have been stopped with better unit tests.
I agree that unit testing is both valuable and necessary. As Wallgren believes, it forms the stable foundation of the agile test pyramid. However, I’m less confident that unit testing will catch the issues that really impact the user experience. Especially in large enterprises, I’ve seen unit tests decay over time to the point where nobody pays attention to them. I’ve also found that most of the issues reported by users could not be found at the unit level; they would have required integration testing, system testing, or end-to-end testing by a professional tester who really understood the domain.
This is one belief where we agree to disagree.
Developers test, but aren’t testers
I believe that unless you’re a Google, Apple, Facebook, or Amazon, you probably won’t have the luxury of hiring SDETs: developers who will devote themselves to testing. However, even if you could get developers to test, they’re probably not your best option for protecting the end user experience.
Yes, of course developers should be checking their code with static analysis, unit testing, peer code reviews, and so on. As Wallgren points out, the earlier and cheaper you can find a problem, the better. However, also consider that the person who’s great at creating things isn’t always the best at breaking things. These are distinctly different disciplines. If you’re moving to a new skyscraper, would you want the final building inspection to be completed by the architect or by a professional building inspector?
Wilkinson puts it this way: “QA is a mindset. To be really good at QA, you need to be passionate about driving quality in.”
Wallgren’s view is: “The testing mindset is ‘I’m going to figure out how to break this.’ This is great because it gets developers thinking: ‘I’m not going to let them break this’—and the software is much stronger as a result.”
Shift (even farther) left
More and more organizations—including Accenture, Electric Cloud, and many customers of my company, Tricentis—are reinventing testing as quality engineering. The change is more than skin-deep. Testing is reactive, while quality engineering is proactive. You don’t just catch defects, you prevent them from entering the code base in the first place.
We all agree that quality engineering means addressing quality much earlier in the process than most organizations do today. But the solution isn’t simply to adopt the techniques commonly branded as “shift left testing.” It could also involve more early stage reviews, with testers participating in those reviews. Even farther left, it could involve inviting testers to the table at design sessions—where they can strategize on ways to build in quality and testability from the start.
With this shift, testers are elevated to the sexy role of trusted advisors, ambassadors of quality and the “boring as shit” work becomes a relic of the past.
This article is published as part of the IDG Contributor Network. Want to Join?
