Effective Quality Assurance poses a serious challenge with Agile teams. Agile’s characteristic fast pace means traditional QA approaches do not often scale affordably and are rarely sustainable: writing and maintaining test plans, bug models, and even the most strategic regression testing assumes a near equal amount of time for QA process and testing as there is for feature development.

Without the resources for a traditional approach to Quality, our processes can be corrupted and devalued —and often QA is abandoned altogether. As Quality Professionals, we know the value of our expertise and want to reverse this trend. To achieve this, though, we need to accept that, for most organizations, solutions dependent on bigger ratios of QA to developers, larger budgets, and longer testing cycles won’t work; such solutions only reinforce the bias against Quality best practices as being slow and expensive. Consider this: our best practices are fine; it’s our approach to them that requires an update. We need to understand what it means to be a high-performing Agile team and reimagine the role of Quality in such teams.

Instead of simply following tradition, let’s focus on how we work with others to embody our values and apply our expertise in ways that everyone in our organizations can recognize and benefit from. Let’s do less quality checking with QA and rather apply our practice toward the common goal of increasing team velocity. There is more to “Engineering” Quality than test automation. To thrive in an Agile organization, we need better teamwork and Behavior Driven Development.

Framing the problem

By their nature, organizations successful with Agile development inevitably focus on increasing their velocity and overall productivity. While manual testing scope increases with each new feature, the testing resources rarely scale along with it. Regression testing before a release, a best practice for assuring quality, takes significant effort and time. Most organizations regard regression as a drag on velocity and conclude automated testing is the ideal means to full testing coverage with fewer resources and time. Like a lot of Quality Engineering Managers, when I am asked to lead a QA team, I’m primarily given the directive to convert the existing manual testers to automation engineers.

What is often not stated, but quickly becomes obvious, is the need to address the myriad quality issues that threaten every release. They are a huge distraction to any conversion effort. Common wisdom suggests that converting the Quality team to automated testing is near impossible without replacing all of the manual testers. But without the manual testers, handling the bulk and range of quality issues is an even bigger challenge given that some things can’t be automated and existing quality issues can impede the development and reliability of automated tests. When the organization buys into the myth that automated testing will magically “fix everything,” Quality Managers are put in an unenviable position: racing to teach QA how to code, recruit skilled technical talent, and devising a reasonable plan to get a test framework in place and catch the first bugs before patience and funding run out.

Those of us who have ventured on this quest will tell you it is exhausting, expensive, and incredibly difficult to get the results we really want. How many of us have spent more than a year building test automation solutions for our team, only to be disappointed by test coverage, sustainability, and overall impact on the business?

Traditional approaches to manual and automated testing seem to be incompatible with the rapid release cycles demanded by Agile. As a result, some companies opt out of QA altogether, choosing instead to invest in efficient deployment pipelines with the belief that faster recovery from issues is better than slow prevention of them. I hear statements like, “If you want quality software, fire all your QA! Make developers 100% responsible for the quality and they’ll hold themselves accountable for it.” The reality is these ideas are also idealistic fixes that will not work for every organization, and may be short-lived even when they do. Not all developers know how or what to test. “Move fast and break things” can lead to poorly calculated risks, catastrophic failure or worse: a user base that wearies of using a product in perpetual beta.

So, before we start toying with shiny new test Automation tools, let’s bring manual QA back into the fold of the engineering team. Let’s build a culture of quality and be strategic about our use of tools — not as “QA” teams in Agile organizations, but as QA members of “Agile” teams ourselves.

How QA stopped being part of the team

Having worked with numerous teams in various stages of adoption, I know the rituals of Scrum may come quickly. However, the important central philosophy and best practices of self-managed teams take time to implement. How and why we work together is culturally entrenched, and Agile expects multidisciplinary teams to be successful via a collaborative culture. Scrum advocates shared responsibility and couples this with the idea that team members work faster when they work across disciplines and solve problems together.

Now, this is a beautiful image of unity. But on the surface, most Quality professionals don’t appear to have the technical skills to work across disciplines with developers. We don’t produce a tangible deliverable like graphics or even written copy. Being at the end of the development phase, just checking for defects, frankly, does not carry the same weight as adding something we ship to the user, and we aren’t frequently solving problems as a team. With Quality designated to the end of the development process, QA is limited to a supportive role. We might even view ourselves as tangential to the engineering team. If you ever question that, see how often developers are willing to run regression test plans with QA for a release or help the automation engineers get 100% feature coverage.

What we tend to forget is that devs’ performance is measured by the complexity of what they produce and how much can be produced in a given period. Without a QA contribution to the development effort, the concept that we are “all one team” quickly breaks down — especially when testing looks like a negative impact on the team’s ability to ship on schedule or the effort of making UI visible to automation tools does not seem worth the yield of defects found by automation. After all, if the devs do a great job, there are no defects for Quality to find, right? I frequently tell my QA team that the least value we bring to the organization is our ability to confirm the software works as designed. Our value comes from our ability to assume the user’s perspective and our expertise in discovery of issues Product and Development did not account for.

Know what is important to your organization

I have fought more than my share of battles to earn respect for QA. I have proven half a sprint more of manual regression assured zero hotfixes after release. I assembled and converted testing teams to Automation Engineers and even when stated goals were achieved and quality metrics are the best predictors of release dates, in my view, it never amounted to quite enough. QA needs to contribute to the success of the engineering team, not in our eyes, but in the eyes of the devs and the larger organization. This means helping the devs ship features. Pointing out a problem is not the same as solving or preventing it. If you have experienced a dev getting upset with you for opening a bug in their feature, arguing with you over the validity of the bug you opened, or even telling you to stop testing, it’s easy to feel like your work and contribution to the team are not appreciated. The problem is that, while your bug, if fixed, would improve the user experience, the effort to implement it may mean they can’t deliver as many features that sprint or the release may be further delayed. Having to review the bug late in the sprint in itself can be a disruption as it forces them to context switch.

I believe this reality of building software, more than actual coding skills, is why white box testers and Software Development Engineers in Test (SDET) tend to be valued more by the engineering team. When we think of Agile teams and the ability to work across disciplines, the potential of an SDET is clear: They fix small bugs, improve the CodePipeline, and help out with code reviews. Think of how often an SDET or QE from your team is pulled to a tools team or DevOps, if not promoted to a dev role soon after their skill is recognized. A lot of manual testers take this as a sign they must learn code. Coding is great, coding has value, but the real lesson here is not that only coding ability is valued but that the SDET actively contributes to the effort to ship features. Exactly how they do that, and what skills they need, will vary depending on the team.

As QA Managers we often evangelize broad test regression, metrics on the dashboard, and test coverage percentages as proof of the ROI of Quality. Does this really help? We think we are educating others on what is important for Quality — but what if all we are doing is showing how the sausage is made? No one outside of QA cares about that. Everyone cares about how we ship better features more often.

More than ever before, managers should look for gaps in skills on the Agile team. We should seek to address problems that hold the team back or slow them down. QA can fill many technical skill gaps that don’t require coding. In fact, we’re uniquely positioned to solve a wide range of problems for development and Product Management. Enabling QA to contribute to the success of the product in this way brings QA back into the fold of the engineering team.

From Assuring Quality To Engineering It

It’s important to remember that Agile development is typically characterized by rapid development and frequent releases, and it’s imperative that assuring quality, whether through manual or automated testing, doesn’t impede this. The good news is that faster velocity with maintained quality is a natural consequence of holistic, healthy engineering practices.

Engineering best practices cover a wide range of technical areas, many not specific to programming. One of the best things QA can do is research them. Propose adoptions which may help address existing challenges. Serve as stewards of best practices to help the team reinforce behaviors that benefit them.

As a Quality professional don’t be afraid of becoming more of a generalist for your team. You don’t have to write code to help your dev team get more value out of Git hooks or improve Jira workflows. Many cloud provider certifications require little to no coding knowledge and are very helpful when designing infrastructure and alerting and monitoring systems. Docker containers, security scanners, and a huge number of technical areas are relevant to your team’s success. In many cases, the root cause of issues found by QA can be traced to underlying systems, code complexity, or problems with process and procedures. In addition, a world of new problems are both addressed (and created) by new Machine Learning Systems that don’t make use of traditional programming models.

Be a relevant contributor to the team by working with developers and pointing out problems with proposed solutions or possible implementations ready to go. Learn. Grow. Fill gaps. Use your knowledge and expertise in quality to prevent issues both early —when it is cheaper and easier to do so — and then throughout the development process. Be mindful of the team’s needs and always look for ways to improve velocity. A QE is an essential part of the Agile team and development process, not something left to the end “if we have time.” The objective is that completion of feature development meets a high enough bar of quality that dropping manual test regression is not a significant risk. That is the difference between assuring quality and engineering it!

In my next post, I will focus on the methodology of Behavior Driven Development and how it enables Quality Engineering and Effective Test Automation.