A recent
story by
Pulse.io compares testing and monitoring mobile app performance, and why both are important aspects of mobile app development.
There are two stages associated with optimizing app performance; one stage is before the app is released (i.e. testing performance), and the other after it is released (i.e. monitoring performance).
The bulk of app developers only test performance before releasing an app, while in development or QA. What these developers are missing is the all-important monitoring of live users that can reveal issues missed in testing.
The downside of only testing performance include:
- Handsets used during tests could be faster or slower than ones used by live app users
- The available resources on handsets used during tests could be different than actual. That includes varying memory, CPU, or network resources
- The volume of user data downloaded or uploaded from handsets could be different in tests than actual
A good practice therefore is to use always-on lightweight instruments to monitor performance of production users. Doing so can ensure that no glaring issues are missed during testing.
It can be expensive to acquire users to download an app and so every measure should be taken to keep those users. If the app is frustratingly slow at times then it is essentially a leaking bucket.
Examples of app performance issues include:
- Long waits on a spinners
- Freezes causing jittery experiences
- Apps being shut down due to out-of-memory situations
By including testing and monitoring as part of the development cycle, developers can reduce their churn rate.
Also as mobile teams add new features, OS versions, or developers, having performance testing and monitoring in place can help teams stay on top of unintended performance issues.
Unlike the mobile app performance testing market, which has been around for years, the market for mobile app performance monitoring services is only about a year old. Nonetheless these services are now starting to become popular among leading app developers, for example Trulia, Yellow Pages, and Saks Fifth Avenue. These companies make watching live app users an integral part of their performance optimization process.