Software testing using Artificial Intelligence

As the complexity of software continues to increase and release delivery cycles shorten, software testers need to provide quality feedback to developers nearly instantaneously. In this age of continuous testing, companies are adopting a new mantra: Test smarter, not harder. There’s simply no other choice, given the torrential pace of new software launches.
 
Today, releases that happened once a month now happen on a week weekly basis, and updates are often made several times a day. By incorporating Artificial Intelligence (AI) into testing practices, users can move beyond traditional, manual testing models and experience truly automated continuous testing.

We chatted with Dr. Gerd Weishaar, Chief Product Officer at Tricentis, about continuous testing and how AI is the key to making software testing smarter and more efficient.

ADM: Why aren’t traditional ways of developing, testing, and releasing software equipped to support today’s demands?

Weishaar: Software has moved from being a process enabler to a business differentiator. What this means is that for an organization to achieve its goals, there is a dependency on the frequency and quality of software releases. Long ago - in software years - nobody could have imagined the pace at which today’s new software features, products, and digital services would be released. And, in the future, we’re going to be talking about how today, none of us could have known just how fast those things would be released. Thanks to shifts to Agile and DevOps, we’ve seen software development and deployment become completely modernized to meet these new demands for speed. What we’re only just now seeing is software testing start to catch up and join this modernization revolution as well. None of the ways we used to do these things were expected to be performed at the pace that today’s business demands, and organizations have to adopt the principles, methodologies, and transformations that enable development, testing, and deployment at today’s, or better yet, tomorrow’s speeds.

ADM: What is continuous testing, and how does it differ from traditional testing methods?

Weishaar: At Tricentis, we define continuous testing as “the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release candidate as rapidly as possible.” Continuous testing evolves and extends test automation to address increased software complexity and a modern pace of application development and delivery. As for how it’s different, continuous testing allows organizations to determine - in real time - the business risk of a release candidate, because the generic pass/fail nature of testing is no longer sufficient. True continuous testing goes beyond test automation alone to not only include end-to-end tests that realistically assess the end-user experience across all associated technologies, but to also maximize business risk coverage. 

ADM: How can organizations implement automated continuous testing into their software development processes?

Weishaar: There are several “requirements” for successfully implementing continuous testing, all of which are designed to enable the modernization of testing required for organizations to truly be able to deliver agility at scale. Organizations need to begin making the following improvements:

- Align testing with business risk to optimize test execution, implement automated quality gates, and provide critical insight for release decisions

- Minimize manual testing and shift it to session-based exploratory testing

- Shift testing efforts to the API layer when feasible

- Use test data management and service virtualization to enable continuous execution of realistic end-to-end tests—without false positives and timeouts

- Integrate functional testing into CI/CD so it’s a seamless part of the delivery pipeline

All of these steps are leaps forward for any organization looking to control risk amidst increased software complexity and today’s pace of application delivery. Continuous Testing evolves and extends traditional test automation in risk, breadth, and testing time/speed.

ADM: If demands for innovation continue to accelerate, will today’s continuous testing be sufficient in the future?

Weishaar: As mentioned earlier, it’s foolish to believe that everything we are doing today, and the pace at which we’re doing it, will be the ideal pace and methods of tomorrow. It’s like agile development and testing - you never “become” agile. You’re always striving to go faster, and reach even higher levels of quality and customer satisfaction. The journey never ends. So, while we may still be doing continuous testing in the future, we know at Tricentis that we will always need to make sure our platform leverages, supports, and integrates with the current most innovative technologies and processes around. For instance, we’re already looking at AI, and the role it plays in software testing today, and what that role might look like in the future.

ADM: What is Artificial Intelligence, and how are organizations applying AI to the software development life cycle (SDLC)?

Weishaar: We’ve been using Forrester’s definition of AI lately. It reads, “A system, built through coding, business rules, and increasingly self-learning capabilities, that is able to supplement human cognition and activities and interacts with humans naturally, but also understands the environment, solves human problems, and performs human tasks.” As for how AI can be applied to the SDLC, think about it this way:  Any complex task that a human can solve without consciously making a decision is a candidate for AI. We’re currently seeing AI be leveraged for things like facial or voice recognition - things that machine learning can be applied to, where the more data that is fed to these systems, the more opportunities arise for these systems to become “better” or more accurate over time. In the software testing industry in particular, we’re seeing opportunities to explore where AI can help testers test smarter, not harder. AI will be tool that testers can utilize in order to achieve this, but AI will never replace testers and their immense contributions to their organizations.

ADM: What areas of software testing can AI benefit?

Weishaar: AI can help us identify clear and repetitive patterns. For those generic patterns, AI can observe them and then process them into tests. However, you’ll still need human testers because new development introduces new patterns and new intentions for an application. This is where software testers who can interpret business expectations and validate whether those expectations are represented in new software are critical. Hence why AI will never replace human software testers.

Today, we’ve seen AI help save testers from having to constantly update test cases - something that takes them away from actually testing their release candidate. Just as intelligent test automation is designed to automate actions that provide no value by being done manually, AI today is being used in image recognition to improve UI testing. Dynamic UI controls can be automatically recognized with AI - in any of their various shapes and forms. AI-supported systems can also recognize and learn new patters that helps test execution remain repeatable and stable, even as software complexity grows.

ADM: What challenges of incorporating AI into software testing should organizations be prepared for?

Weishaar: When looking at introducing AI into your software testing processes, it’s important to know two things. Firstly, AI and machine learning are only going to take place after these technologies can observe repeating actions. AI and machine learning don’t just get “plugged in” and start delivering pure magic. There is a learning process within these technologies. Organizations need to understand that there needs to be a set of observable actions that will ultimately be expressed in a series of business decisions. Those aren’t going to simply show up on day one.

And, secondly, as I’d just started to mention, AI elements need to be optimized toward a business outcome. These elements need to first observe, and then be able to be directed toward a specific outcome so you have feedback that is not only actionable, but also geared toward an existing business goal that the entire organization is supporting. People sometimes think along the lines of, “We’ll just plug in Watson, and then get everything we need to know.” That’s really not how AI works.

About Dr. Gerd Weishaar

Dr. Gerd Weishaar has been Chief Product Officer at Tricentis since September 2017, and previously served as its Vice President of Product Management since August 2014. Dr. Weishaar has over 20 years of experience in product development and product management from various software companies, including Segue Software, Rational Software and IBM. Prior to joining Tricentis, Dr. Weishaar was responsible for the product management of Update Software, a leading vendor in the field of Customer Relationship Management (CRM) software. Dr. Weishaar is a computer science graduate of the Johannes Kepler University in Linz, where he also obtained his Ph.D. He is also an MBA graduate of the prestigious MIT Sloan School of Management in Cambridge, Massachusetts, USA.