Modern cross browser testing helps engineers test faster

Applitools announced an industry report empirically proving that modern cross browser testing powered by Visual AI helps engineers test 18.2x faster across the full test cycle including writing, running, analyzing, reporting, and maintaining tests.

Data for the report was sourced from 203 engineers across 75 countries who spent 3,112 hours completing the Ultrafast Cross Browser Testing Hackathon in June 2020. The data shows Applitools Ultrafast Test Cloud Powered by Visual AI allowed engineers to complete a full test cycle on a modern, responsive e-commerce application in just 1.6 hours versus 29.2 hours when using traditional cloud testing environments.

Additional highlights from the report include:

A detailed breakdown of the 18.2x time savings across the end-to-end test cycle including writing, running, analyzing, reporting, and maintaining tests. Because Applitools runs tests once locally, then renders across all browsers, viewports, and devices, it eliminates the need for slow VMs and multiple connections that traditional test cloud solutions rely on.

99% reduction in the number of lines of test code virtually eliminating the maintenance overhead that plagues traditional cloud testing solutions.

91% reduction in the use of locators due to the use of Applitools Visual AI because it relies on images instead of brittle, flaky locators to support the test scenario.  

77% increase in software engineer satisfaction vs. traditional cloud testing approaches. Applitools received a Net Promoter Score (NPS)* of 70 compared to -54 for traditional approaches to cross browser testing.

Engineers of any skill level learned Applitools Ultrafast Test Cloud in less than 90 minutes and upskilled to modern cross browser testing as a result.

“The incredible challenges brought on by digital transformation require a modern approach to cross environment testing. Applitools Ultrafast Test Cloud combines Visual AI with container-based rendering to address them,” said Gil Sever, CEO and co-founder of Applitools. “Hundreds of software engineers have shown us how Applitools Ultrafast Test Cloud is an incredible efficiency multiplier for cross browser testing. It delivers significant time and money savings while helping teams release faster than ever before. It represents the future of testing where continuous quality is key to successful digital transformation for modern engineering teams.”

"Cross browser testing is ultra fast using Applitools Ultrafast Test Cloud. Bugs that I missed (and there were lots!) on different browsers and viewports were easily caught without affecting the time it takes to run my tests," said Marie Drake, Principal Test Engineer at News UK. "Functional testing frameworks automate browsers, but they are not built to catch the visual changes now incredibly common, cross environment. Applitools Ultrafast Test Cloud helps engineers succeed when doing modern cross browser testing."

Data from the report was sourced from the Ultrafast Cross Browser Testing Hackathon which challenged participants to author cross browser and cross device tests against a modern, responsive, ecommerce app. The event challenged engineers to test a new release candidate across seven different browsers and three viewport sizes and ensure the new version is bug free. Participants used both their preferred legacy cloud testing solution and Applitools Ultrafast Test Cloud Powered by Visual AI integrated into their preferred test framework and desired programming language using over 50 SDKs.

Engineering leaders that participated in the study commented:

“With Applitools, setting up configurations related to viewports, devices and browsers is very easy and saves a lot of time,” said Himanshu Soni, Lead QA at Amadeus Labs. “Frankly speaking, I spent a heck of a lot of time setting the same configuration for traditional tests, whereas using the Ultrafast Grid it was in a blink of an eye.”

“The Ultrafast Grid is really the right tool when it comes to cross-browser and responsive design testing,” said Roman Lymariev, Senior Quality Assurance Automation Engineer at Mobiquity. “You can quickly set up your tests and start catching the related bugs really fast. Traditional approaches do not work quite as well in this department.”

“I loved how intuitive Applitools Ultrafast Grid is,” said Kerry McKeever, Senior Software Test Engineer at Paylocity. “The speed of learning the API and the dashboard, as well as the speed of developing the tests without the need for dozens of locators, conditional waits, management of drivers/browsers, etc., made testing a breeze.”