Testing your app for the holidays

It’s almost the most wonderful time of the year and retailers of all sizes have already been busy preparing - and in some cases bracing - for the upcoming surge in traffic that is expected to hit their websites and mobile shopping apps this holiday season. Salesforce is predicting that mobile devices will drive 68 percent of all holiday shopping traffic this year, while Cyber Week is expected to account for 40 percent of holiday sales.

As consumers around the world gear up to shop on their apps and grab the latest online deals, behind the scenes, DevOps teams are hard at work to ensure their websites and apps don’t encounter major technical issues. A retailer’s worst nightmare is their website crashing on Black Friday or customers receiving error messages as they finalize their purchases.

With holiday music starting on the radio, decorations being sold before November 1, and the industry competition rising, let’s explore the top testing priorities for developers leading up to the holiday season.

Running Your Tests, and Then Checking Them Twice

As we enter the hustle and bustle of holiday shopping, continuous testing is the gift that keeps on giving. For teams to be effective today, they need to be testing every day, multiple times a day. By ensuring all tests - including functional, non-functional, performance, accessibility and user experience - are running as part of continuous testing workflows, DevOps teams can quickly identify any hiccups during production. During heavy traffic times - such as Cyber Monday and Cyber Week - continuous testing allows DevOps teams to ensure the quality of user experience and the reliability of mobile and web apps no matter how big the sale. In addition, features like Apple Pay and location-based notifications should also be prioritized during testing.  

During the holiday season, servers will be targeted and inundated with requests more than any other time of the year. To account for this, DevOps teams should test at scale and consider availability and a robust continuous testing solution that includes quality dashboards with smart insights for fast feedback. This is a vital component as it enables DevOps teams to quickly respond and resolve an issue.

Ensuring it’s the Most Wonderful Time of Year for New Devices

Every holiday season, DevOps teams also find themselves up against an almost constant stream of new devices and operating systems releases. And this year is no different - already Apple has announced new phones and features ahead of the holiday season and there’s a new Google Pixel on the way. While consumers eagerly await the release of these new toys, it’s a race against time for developers who are under strict deadlines to ensure their apps and websites remain operational regardless of which device or OS the consumer is using. Labs that aren’t configured well or do not cover the new device/OS combination will likely face increased quality risks.

With so many new devices slated for distribution around the same time, determining which devices and OSs to test on can be overwhelming. To start, teams should first explore the new devices that are slated for release closet to the holidays. In this scenario, the new iPhones, iPad, and new Google Pixel are three musts for testing this year. However, while developing and testing for new devices is crucial, developers must do this without neglecting those old gifts - legacy and -1 versions and devices that are likely to remain popular and in use. Deciding what devices to retire should be based on customer web traffic analytics and risk management decisions.

AR Capabilities Are Coming to Town

As retailers look to remain competitive this holiday season, many have turned to incorporating augmented reality (AR) capabilities into their mobile apps with the goal of making the shopping experience as seamless as possible for customers. Take Lowes for example - its Lowe’s Measured app gives users instant measurements based off images from the device’s camera.

For dev teams, the addition of these new features means they need to consider important factors such as the variety of smartphones - both iOS and Android - and their maturity and support for the Apple iOS ARKit and the Android ARCore APIs. Since mobile users are always on-to-go, DevOps should also test apps in different environments including degraded network connection and incoming events such as calls, texts, use of the camera outside of the retail app, and more. For apps that use the AR functionality through the camera, various outputs will be presented to the user. Because of this, DevOps should test across all screen sizes, smartphones, tablets while also ensuring that users can engage with the buttons and visuals on the output. If not, the entire shopping experience will be ruined.

As retailers look to come out on top this holiday shopping season, the availability, user experience, and performance of their apps and websites will dictate their success. Now is the time for developers to start preparing for the holidays. With sufficient test coverage on the right platforms and the right scenarios, all will be merry and bright.