To enable the QA to do the best job possible, test resources must be made available. As an organisation, you want to facilitate these means in the best way possible. The more test appliances the QA has at his/her disposal, the smaller the chance of bugs on particular devices or screens.
There’s a number of choices that can be made:
- Real or virtual devices
- Different OS-versions
- Tablets vs smartphones
- cover of related smart devices
Real or virtual devices
Given the ever increasing market of all kinds of smart devices, it is not realistic as a company to think that all products released can be tested. Specifically in the case of Android, there are many different manufacturers (and therefore many different devices). In Android’s Playstore, no distinction can be made among these manufacturers, which means that all of them have to be supported.
That is why online applications by which devices can be controlled via a web browser are becoming more and more popular. Often, these are backed by a real device, located at the company concerned, but this does not have to be the case. It can be a way to save costs and create a larger test area.
Egeniq explicitly does NOT work with virtual or online devices. This has several reasons, the most important one being the following: The user does not have a virtual device either. The user has a real smartphone or tablet in his or her hands, uses a finger rather than a mouse, and can experience discomfort from rounded sides or a notch. The only way in which the QA is able to know how a device really works is by holding it in his or her own hands.
But this is not the only reason. Another very important aspect in our choice of working with real devices has to do with security. It happens often that we want to keep our app design secret for as long as possible. Or that we do not want the used code or information to become known outside our organisation. Working with a tool based on virtual devices makes you dependent upon the clean-up process of the company involved or its security systems. But if you actually have the real device lying on your desk, you are much more in control.
A virtual or online device can be a suitable test tool in case of a bug occurring in an app that is still in production on a specific device. Here, security issues hardly play a role. And let’s be honest, there is no company with a budget so large that it can provide the QA with test models of all devices produced by the manufacturers.
Which OS-versions should you choose to support as app? This is not only important with respect to the development of the app but also for its test-procedure. For companies that have already been working with a QA for a long time, this is probably not a big issue, but try to find an Android 4.4. device these days, or an iOS 9 device. These are even hard to obtain on a second hand market. As a QA, I often see that the most modern, fancy libraries used in the app tend to cause problems in older OS-versions. That is why it is always important to test major features on the oldest supported OS-versions. But then they do have to be available of course.
Not only the oldest versions can produce bugs. All kinds of in-between OS-versions can also cause these problems. That is why we as Egeniq always make sure that our QA-s have at least 1 device at their disposal with a number of major in-between OS-versions. Roughly, we use the following rules of thumb:
the QA with a smartphone and a tablet that use the newest OS-version (currently Android 11 and iOS-14) sure you have a device (smartphone or tablet) for each OS-version that you support. Depending on the app, smartphones may be more useful to the QA than tablets
Because most people are in the habit of updating their OS-version regularly anyway, or buy a brand new device, it is important to be able to test the newest OS-version on both a smartphone and on a tablet.
Tablets vs smartphones
For each app that is developed, a choice has to be made between tablet support or smartphone support. The choice made largely depends of the type of app and the (potential) users. A news app, for example like RTL Nieuws, is quite often used on tablets, which means that the tablet support must be adequate. And there are also apps, like the HRA-app of NBA (Handleiding Regelgeving Accountancy van de Nederlandse Beroepsorganisatie van Accountants (Instruction Manual Accountancy Regulations of the Professional Association of Accountants) that are work on tablets. Or apps that are particularly suitable for use on the mobile phone. As an example you can think of the FeedCalculator app.
The type of app and the type of users determine which devices are going to be supported. Apps that are specifically used during travel, for example, are likely to be made for smartphones. And for apps that contain a lot of content, the choice can be made to only support tablets.
When the decision has been made to support tablets and/or smartphones, the QA’s test device kit must of course be equipped accordingly. In other words, it makes no sense if the QA only has access to smartphones if the app is only suitable for tablets.
App facilitation for related smart devices
The creation of apps for smart appliances does not end with tablets or smartphones. Specifically within Egeniq, we develop apps for far more devices than these two types. You can think of Chromecast, Airplay, Google Assistant, iWatch, AndroidTV, AppleTV, SmartTV, Google Hub, CarPlay, Android Auto and much more. And in the future, this collection will only increase.
In selecting the most suitable of these devices for testing the apps, here the same applies as to the choice between tablets and smartphones, the app and the user are the most important point of departure. A radio app, such as the NPO Radio, can obviously also be connected with both Chromecast and Airplay. And for an app such as the Pathe Cinema app, we support iWatch for displaying the sold tickets.
It can be decided, however, to not make all related smart devices available to the QA. For example, giving the QA 2 cars for testing CarPlay and Android Auto may just be a bit over-the-top. For this kind of smart devices, it usually suffices to give only one person within the organisation access to it, especially if there is not so much development involved in that particular item.