SwiftKey, founded in London in 2008, is an intelligent keyboard app which is currently the number one selling app on Google Play in over 50 countries. Its contextual predictions are available in over 60 languages making it a regular on top 10 must-have app lists. Its language technology is also available to license and comes pre-installed on a number of leading smartphones.

In response to the growing popularity of smartphones and the difficulties users have in typing on their tiny smartphone keyboards, the mobile team at SwiftKey needed to ensure that their virtual keyboard could always wok with different types of smartphones from well-known manufacturers through intensive mobile app testing.

Challenges in testing virtual keyboards across hundreds of real devices

In Android, keyboards need to be both enabled and selected manually by the user. Also, the keyboard is a system service with a UI which means that it is both used by other apps and also displays its own UI inside the calling app UI. "Automating tests for virtual keyboards is a bit more challenging due to its nature of being a system service," said Michele Sama. "Also in search of an mobile test automation solution, we wanted to keep using original Android instrumentation tests rathen than taking in use third-party library or frameworks."

Since the software supports over 240 languages for different Android devices, the mobile team at SwiftKey had large suites of Android instrumentation tests which they wished to run across various devices. "We needed a solution to allow us to automate tests across as many real devices as possible to ensure our software is supported different Android configurations," explained Michele. Instead of running a test for each different configuration, SwiftKey was looking for a automated testing cloud solution that scales up test efficiency and improves the testing speed.

Kickstarting test automation for scalable mobile device testing

While most cloud systems offer a record and replay solution, the company wanted to avoid using a third-party library or testing framework in order for their tests to remain purely Android. The company turned to Bitbar Testing, the best Android and iOS test automation solution for scalable mobile device testing. SwiftKey took full advantage of Bitbar's API by making it part of its test infrastructure through Bitbar's Jenkins plugin.

"Software testing is a critical activity for a company from both commercial and operational perspectives. Consumers will stop using a piece of software due to bugs and slow performance and a company’s costs to detect and fix bugs increases the longer a bug exists in the code. The combination of continuous integration and automated testing is one of the most powerful instruments which quality engineers have to improve the quality of their software during any stage of development," said Michele.

"Without Bitbar Testing and the ability to run scalable automated tests on every available Android device, we would not have been able to find, fix and validate a critical SSL-encryption issue on very few Android devices. Owners of these devices who tried to use SwiftKey would have performance issues that could have potentially made them stop using our app which, in the long run, translates into a loss of users and revenue".

Michele Sama, Test Architect at SwiftKey

Spotting problems before getting an impact on user retention and revenue generation

The adoption of Bitbar Testing enabled the SwiftKey team to run its own functional test suites that spanned over 1000 test cases on any devices in Bitbar Cloud and avoid the need of implementing third-party testing frameworks. "Adopting new tools and libraries has not been an issue since we started leverage Bitbar's solution technology. Built with open standards, Bitbar Testing runs and generates normal Android instrumentation tests which means that there are no additional adoption and learning costs," said Michele. The team also created comprehensive tests to check the SwiftKey keyboard layout using any languages and ran performance tests all a large amount of devices in Bitbar Cloud to find performance bottlenecks.

By using Bitbar Testing, SwiftKey was able to find an issue that only appeared on a few specific devices. "Without Bitbar Testing and the ability to run scalable automated tests on every available Android device, we would not have been able to find, fix and validate a critical SSL-encryption issue on very few Android devices," emphasized Mchiele, "Owners of these devices who tried to use SwiftKey would have faced performance issues that could have potentially made them want to stop using our app which, in the long run, translates into a loss of users and revenue."