Test automation comes of age

If you work in a software development organization, no doubt you’ve heard quite a bit about test automation in recent years. That’s because test automation, which involves using software tools to automate the execution of tests and record results, has grown in popularity along with agile development and devops. These three trends seem to go together, according to a survey my company QASymphony conducted in partnership with TechWell. We collected data from software testers and QA decision-makers at midsize and large enterprises and found that 72 percent of organizations are currently automating testing, 76 percent have adopted agile methodologies, and 55 percent are using devops practices.

Companies are adopting these practices due to marketplace changes and user expectations. Most companies use software as a primary means for interacting with their customers, and, to meet their expectations, they must be more proactive about what users want and need, which means developing, testing, and releasing much more frequently than ever before. Global competition demands it. There’s been ample research showing the positive impact that continuous software delivery has in helping organizations reach these goals, but not as much in the area of test automation, an essential component of maintaining high quality in continuous delivery environments.

Implementing test automation isn’t easy, but the benefits are clear

Our survey found that organizations that automate 50 percent or more of their tests experience faster testing cycles (88 percent), improved test coverage (71 percent), and the ability to catch bugs earlier (68 percent). That means faster time to market without sacrificing quality. Other top benefits include improved efficiencies in testing and higher morale of the QA team. When we can be productive and deliver excellent goods and services that people love, work is fun and meaningful.

Still, test automation is an emerging practice. Even though most of the organizations that we surveyed are automating at least some tests today, the majority (76 percent) are automating less than half of all testing. This speaks to some challenges in the transition from manual to automated testing:

• Implementing test automation requires QA teams to develop new testing strategies, implement new tools, and learn new skills.
• There are new processes and workflows that affect testers and other key team members, including developers and business analysts.
• Organizations may need to hire skilled test automation engineers who can hit the ground running and lead others.

All of this takes time, money, and patience.

Those surveyed say their companies struggle to create teams with the appropriate skill sets as well as gaining buy-in for change, because testers must work faster and in continual collaboration with others. Some testers may resent or fear the need to become more technical in their jobs.

Technology adoption also plays a significant role in test automation. New testing tools and frameworks that support test automation and complementary devops practices such as continuous integration are necessary. This can introduce more complexities for traditional QA teams. There are dozens of test automation tools and platforms available today, both open source and commercial. Choice is great, but selection can be confusing. Many times, organizations wind up with too many repetitive tools. Nearly one third of the organizations we surveyed say it’s hard to keep track of automation work across multiple tools. Roughly the same percentage expressed difficulty finding tools that meet their needs.

In addition to selecting the best tools for the job, testers will also need to get leadership on board—and that often means combating unrealistic expectations and setting new ones. Common misconceptions include that test automation is quick and easy to implement, that it will require fewer resources than manual testing, and that it’s free or has minimal costs. The truth is that it takes time to build a test automation strategy and skill to write the scripts that will power it. And while there are many effective open source tools out there, resources will be required to implement and maintain them. It’s best to explain this upfront, then define timelines together based on what’s realistic and make sure sufficient resources are set aside for the project.

These are growing pains, indicative of any major transformation in the workplace. The upside is that organizations that successfully adopt test automation and expand it over time should be able to deliver more value to the business and make better use of their time.

Test automation complements manual testing

Many of the manual tests that automation replaces are tests that are repeated over and again from sprint to sprint. Automation reduces errors that can occur in repetitive tests and also saves testers from getting bored. With automation in place to take care of these somewhat rudimentary tasks, manual testers can spend time on more creative testing and problem solving to fill in the gaps and deliver a truly awesome user experience.

Survey respondents said that automation allows them to spend their time more strategically—both planning and executing automation strategies and performing the exploratory testing that cannot be replaced by automation. One respondent summed it up this way: “We don’t use automation to reduce manual testing; we use it to augment it and allow our human testers to focus on new functionality and high-value testing work.” QA teams should view test automation as a complementary activity to manual testing versus replacing invaluable, human insight with tools.

If your organization is not automating tests yet, chances are you will be before long.

Nearly half of respondents in our survey say they expect to have automated 50 percent or more of all testing one year from now. Preparing early for the shift can set up your team for less pain and quicker success as you embrace these new testing skills and methods.