Lets talk about QA automation

Automation is making its rounds through the tech industries it seems in recent times. And each time it reaches a new destination we see all new innovation and a breaking down of traditional models. We sat down with Metova's quality assurance experts to learn more from the experts about how automation is making changes in the field and what we can we expect in the future of QA.

Trevor Bennett is a Quality Assurance Engineer and Skylar Fisher is a quality assurance analyst, both of whom work for Metova.

ADM: What is an overlooked area in development that often contains bugs?

Fisher: There is no answer across the board, but it's usually in areas that are not thought to be connected to a new update. For example, if you add a new field that is to appear on multiple pages within a CMS there is a good chance the functionality on the pages that are unrelated to fields will be buggy somehow. This is why smoke and regression testing is very important.

Bennett: Emulation vs Physical devices. While the theory of using emulated devices to develop and test against ‘should’ mimic real life devices, often-time bugs only happen on real devices.

ADM: What's QA automation all about, how might this be different than how people think of QA in the traditional sense?

Bennett: QA automation is most commonly found in taking the most tedious, time consuming tasks and automating them to reduce test time, improve test consistency and broaden environmental variables performed throughout the application.

Fisher: Automated QA is simply a tool in the QA process. It's a part of the big QA picture. What automation does is repeat the tedious steps that never or rarely change and are truly made for the regression testing stage of the QA cycle.

ADM: Is it more difficult to build products with the intention of having QA automation? If so, what's the key benefit to doing so?

Bennett: Automation relies heavily on the ability to identify elements within the application via means of attributes or xpaths. Referred to by some developers as “best practices”, so as long as an application is designed with unique tags declared for elements throughout the product, the design shouldn’t be any different.

Fisher: Yes and No. This is due to the fact that for code and features to be candidates for QA automation, they should not change much. So it may require additional planning and communication for the long term.

ADM: Can QA automation replace manual QA entirely?

Fisher: No. At least not until AI is heavily implemented and a part of everyday life!  Automation in QA only covers parts of code that do not change much and are easily repeatable. It does not cover edge case scenarios or features that change often.

Bennett: Automation is a tool utilized by QA. While automation is great at testing controlled situations and data driven results, manual testing is still the staple to ensure edge cases and ‘Quality’ is met from a user perspective.

ADM: How hard is it to build in QA automation if a product wasn't initially built that way?

Bennett: Automation is an external tool that looks at the application from the outside as a user would, black box testing, so implementing automation may simply come in as requests to update current code base to include unique identifiers within the code structure (i.g. Id’s or name’s).

Fisher: Technically anything can be automated at least in part, assuming the product has parts that are not constantly updated.

ADM: What's the difference between QA automation resources and developers that engineer the products?

Bennett: QA automation takes the approach of a user standpoint whereas developers look at code programmatically. While something may seem correct code-wise, ultimately the end user experience can alter desired test cases.

Fisher: At the end of the day they’re both still coders, but I think the biggest difference is intent. Developers that engineer the products are looking to create and implement features, whereas automation engineers are looking to verify these implementations and create a validating system.

ADM: How many ‘user stories’ are typically covered in automation testing for a mobile app? What about code coverage, what's a meaningful metric there to shoot for?

Fisher: This is dependent on the app and what it does. Ideally you’d automate as much as possible. The more that is automated the more a manual tester can dig deeper into the app to find the truly strange bugs.

Bennett: Automation only makes sense when the scope of the project allows for it. In some cases this number might be 0, in others it might be 90%. In reality the coverage alters based on what is consumer-facing. Behind the curtain stories to alter implemented libraries do not require additional tests.

ADM: What are some interesting “saves” that you have made through QA work?

Fisher: I’ve made all kinds of “interesting saves”, but my favorite kind are the ones where you misunderstand a bug that has already been reported and find a new bug in the process. This is another great example of why manual QA will always be required. There needs to be a human element in the testing process.

Bennett: I feel the most interesting ‘saves’ come in the form of design. Like mentioned earlier, the ultimate goal is user functionality so utilizing the ‘Quality’ aspect of QA allows constructive feedback for design and user functionality.

ADM: Do you find that a particular platform (i.e. Android, iOS, Windows) is more prone to bugs?

Fisher: I find that obscure OSes/browsers, older versions of OSes/browsers, and/or brand new OSes/browsers are more prone to bugs than simply one platform or another. This is mostly due to market share and the software life cycle. If the demand for a product does not exist or the technology is changed/gets phased out, developers will not have the manpower to ensure stability on everything.

Bennett: I find features to be more prone. Simple applications that are static are less likely to exhibit bugs while complex and dynamic designs introduce numerous environmental variables increasing bug count.

ADM: Have you discovered bugs in applications that you use in your spare time?

Fisher: All the time! When you look for bugs for 8 hours a day it's hard not to notice or critique other apps in your off time.

Bennett: All the time. Some bugs are comical and fun to show off normally in the form of graphic glitches while others are feature breaking and have been reported to forum support lines.

About Trevor Bennett

A Nashville transplant from California's city of angels, Los Angeles, Trevor is a Quality Assurance Engineer at Metova. In his free time he enjoys a mix of the outdoor lifestyle experience by hiking, kayaking, trail driving and the nostalgia of playing retro video games.

About Skylar Fisher

Skylar has been a quality assurance analyst for over four years. In her free time she enjoys hiking, finding out about new music from artist social media  (The weirder or more niche it is, the better) and pouring over new technology - especially AI.