UI and UX testing report from Applitools

Applitools announced the 2022 State of UI/UX Testing Report. Testers, developers, QA managers, and UI/UX designers from nearly 1,000 companies were surveyed.

As the first-ever industry report on the user interface (UI) and user experience (UX) testing, the data shows that most organizations' current testing techniques are unable to meet the quality engineering demands for their digital products and services.

The report identifies critical factors that inhibit the scalability of modern UI/UX testing and shows that while the demand for techniques like frontend testing is increasing rapidly, only 30% of respondents are actively testing for an application’s ease-of-use and visual correctness on each new deployment to production.

Here are the top trends and challenges identified in the State of UI / UX Testing Report:

• 38% of companies are deploying changes to production daily. This results in a significant increase in the amount of test coverage required in a shorter amount of time and expands coverage gaps. An even greater percentage of Applitools customers, nearly 2x than non-customers, deployed daily due to the confidence and coverage that they get from Applitools Visual AI.
   
• Manual testing is still rampant, most organizations automated less than half of their testing despite test coverage for every device being critical.
   
• More than 40% of test suites take more than an hour to run, creating a bottleneck for software delivery. However, respondents from companies that are using Applitools Ultrafast Test Cloud reported an average of 4x faster execution times.
   
• Across the industry, companies are seeing less than 50% test coverage and test suites that take more than an hour to run. However, based on survey data, Applitools customers report 75% test coverage on average and test suite execution takes less than 15-20 minutes on average (or 4x faster than non-customers using traditional methods).
   
• Another challenge reported by nearly 70% of respondents is that the UI is constantly changing across applications and device types causing constant test maintenance and stability issues for engineering teams.
  
   

"Achieving higher rates of automation for testing digital experiences is vital for companies to stay ahead of the competition and ensure that their users are getting a great experience on every device. Automating a single test case initially is fairly simple, but maintaining it over time in a CD environment and implementing cross-device testing and improving test coverage to cover all the critical user flows is more challenging and important than ever today," said Moshe Milman, co-founder, and COO of Applitools.

State of UI/UX Testing: priorities and opportunities

Nearly 80% of survey respondents reported their team is responsible for testing two or more web applications. Surprisingly, more than 50% of respondents are responsible for testing five apps or more with at least five localized languages.

• Most organizations deem "increasing test coverage" as the number one reason for test automation.
   
• The priority of the types of testing is changing, while unit, functional, and API testing remain the most important, usability and UI testing are now more important than load, performance, and security testing, according to respondents.
   
• Visual testing and test execution are two key areas where the majority of organizations are looking to implement AI-augmented technology.
  
   

Applitools customers are better equipped to handle the challenges of UI/UX testing and quality engineering. Marianne Epstein, Director of UX design at EVERFI explains: 

"The number of permutations is overwhelming and was impossible for the UX team to navigate. Applitools provides instant access and validation to the massive inventory of screens and the peace of mind knowing that every regression is immediately spotted across all screen variations ensuring the user experience is exactly as it was intended to be. I feel like it is a transformative tool for our UX team and how we partner with developers to drive user experience."

Applitools is able to visually test five applications, in five languages, across five browser types, for three different devices, with just a single line of code, in minutes, not hours.