One of our previous blogs already gave you some insight into what a QA actually is and what characteristics he or she generally has. Now it is time for the next step: What is exactly required of a QA?

Every business or team has a different interpretation of a QA’s activities. And all these task descriptions include different tools. This blog tells you what Egeniq considers to be the activities of a QA and what resources are necessary to perform them successfully. A number of specific tools and terms are therefore additionally explained.

Within an organisation or team, the QA can have various functions. From “just performing a brief check” to “being the supervisor of all PR-s”. For each role that the QA is assigned, a certain capacity is needed. Here, it is no matter of right or wrong. It particularly depends on how the organisation views the role of the QA and the quality of the developers.

Egeniq is a company that has been creating apps for smart devices for over 10 years now. And through the course of time, we have considerably grown, both as regards the number of colleagues and in terms of the different types of apps that we produce. With this growth, the role of the QA has increased too. At the moment, we consider the role of the QA as follows: within the team, the QA is THE person in charge of monitoring the quality of the app and its added value for the user. This definition has deliberately been kept broad because every project requires a different level of intensity. Roughly, the duties of Egeniq’s QA are the following:

• Testing the tickets developed
• Tracking down and registering bugs
• Making comparisons and levelling the different platforms
• Asking questions, questions, and even more questions
• Being the first user
• Performing regression tests
• Monitoring crashes

Egeniq works almost entirely with Jira boards. On a Jira board, a product’s development process is mapped out with the aid of tickets. A Jira board has various lanes that represent the progress and different stages of a ticket. One of these lanes is the QA lane.

Virtually every ticket eventually ends up in this QA lane. Here, it is the QA’s task to approve the product developed. An important step because development is done by people, so there is always a chance of error. In addition, the QA will also check the status of other processes or issues. And, depending on the type of ticket, whether the product runs smoothly on each device or OS-version.

An important aspect of maintaining the quality of an app is to minimise the number of bugs. This is where the QA plays a significant role. In order to solve a bug successfully, it must – firstly – be identified, and -secondly – be reproducible. The QA is engaged in both tasks. By regularly looking just a bit further than the ticket, or by just scrolling through the app, bugs are detected. Next, of each bug a ticket is made on the Jira board, so that the developers are able to find them.

In this context, reproducibility is very important. Not only in case of fixing a bug, but also for testing whether the bug is actually gone. Sometimes, this is simple, for example by opening a particular page or by clicking on a picture. But it can also be more complicated when multiple specific steps have to be executed one after another. By assessing these steps and registering them, the QA can start doing what he or she is are really good at: further improving the app.

We develop apps for many different platforms. For example, the player that we designed for Mediahuis. It can be used on web, Android and iOS apps. Or the Lotto app of the Dutch Lottery, which is a specific app for the Huawei platform HarmonyOS.

Because most developers focus their activities on only one platform, differences may sometimes occur with other platforms in terms of hardware standards. Different interpretations of tickets, for example, or differences in the standard icons (such as the share icons of iOS and Android). Or because certain elements function differently (‘go back to’ works differently in a browser than in a mobile app). Since the QA tests all platforms, he or she will detect these differences. And if those differences deviate from “those particularly related to the OS”, it is the task of the QA to report this. Usually, the first step is  then to discuss the difference with the client, who can decide upon the best way to proceed. After that, it is up to the developers to solve the issue.

This is what every QA commonly does. Asking multiple questions. Not only to check whether all components are working as they should, but also to determine how they have to function. These questions can be directed at anyone. From developer to project manager, and from client to designer.

Particularly the type of question is important in this context. For example, whether the design has been implemented properly is an issue which – to a certain extent – the QA is able to assess him or herself. On the other hand, however, a topic such as ‘the sequence of an app’s user flow’ is not always really clear in advance. It is something that developers may approach and interpret differently. In the levelling process, where information is gathered from multiple parties involved, it is the task of the QA to find out which interpretation is the correct one.

The QA is actually the very first user of the new functionality, and can therefore reveal important insights. Whenever possible, Egeniq considers the perspective of the user, and here the QA is an important addition. Because if there is unclarity with respect to a particular matter as far as the QA is concerned, there is a good chance that the user will not understand it either.
This means that for certain issues, the tickets and their testing will be dealt with in a less conventional manner than is usually the case. Namely: what solution will really work for the user? This means that there is less emphasis on the requirements stipulated in the ticket.

A ticket is always formulated on the basis of a particular situation. But it can always happen that although its contents make sense on paper, they do not work for the actual application itself. It is therefore essential to maintain a critical attitude. Because in the end, not the client but the user has to work with the app. And if the ticket is not functional for the user, the added value for the client will also be unsatisfactory.

By looking at each new functionality through the eyes of the user, the app’s added value is mapped out and can be guaranteed. And the clients are often surprised by the outcomes. The fact of the matter is: what works on the drawing board does not always work in practice.

Before going live with an app, it is important to be absolutely certain that everything works properly. To check whether this is the case, we perform a regression test.

In a regression test, all major functionalities are checked to be sure that they do what they have to do. To give an example, in case of the BankGiro Millionares app, it is important that the user can answer the questions and use the helplines, and that when he/she answers a question incorrectly, he/she is back to square one. Every app has its own specific set of functionalities. The QA is the one who exactly knows what these functionalities are and how to check them properly.

It is sometimes also wise to check less important functionalities. For example because a new one has been added, or because an old one has suddenly ceased to work. This too is an important task of the QA.

A final important activity in the execution of a regression test is to see “whether the app as a whole” is still working as and when it should. Here, we particularly refer to the issue whether all supported OS-versions are sufficiently capable of accurately carrying out user flows. In this context, it is also important to monitor the connected smart devices, such as Chromecast and Airplay.

However, which version is the best suitable for performing a regression test, and in which test environment should it be done? The answer is brief, but sometimes also a bit complicated. A regression test should be executed at a location closest to production. For iOS this is obvious: regression tests are done via a Testflight build because the next step is to actually put the app online (after Apple’s review). So, this is the closest you can get to production.

For Android there a number of choices. Developers can add an APK-file. This is a file that also has to be uploaded to the Google Playstore. Another possibility for Android is to make use of the Alfa- or Beta-track within the Google Playstore. Here, you can choose among an internal, a closed or an open testing environment. Depending upon the type of app, what it is that you want to test, and with whom, you can make your choice. Since the creation of an APK-file is often quicker, whereas an Alfa-or Beta-track first has to be installed within the Playstore, it is within Egeniq standard procedure to perform regression tests via an APK file. The Playstore’ s Alfa- or Beta-tracks, however, are certainly also a good option.

Often, the simplest solution for web- or API-platforms is to install a beta-environment, which must then be filled with the production data. This is of course crucial, because without production data not much can be said about the performance of the functionalities.

Let’s start by breaking the myth that apps don’t crash; they all do. No matter how smart and simple the app, there is always someone who will do something that makes the app quit working or produce an error.

Still, it is particularly desirable to monitor the occurrence of crashes straight after the app’s release. Because if the number of crashes increases all of a sudden, it says something about the app’s stability. In an ideal world, the number of crashes would decrease with each release. In the real world, one should be happy when the number of crashes after each release remains the same.

It is true, however, that crashes can be dealt with by taking measures. Here, the QA again plays an important role. In practically every app that we develop, we integrate Firebase. This is an online tool of Google that has many options. One of them is keeping track of crashes via Crashlytics. During the first 24 hours after an app release, the QA monitors possible crashes in Firebase. If the number of crashes exceeds the average, the QA reports it to the team, after which it can then be decided to release a hotfix.

If the number of crashes no longer increases, the QA will create the relevant tickets within the Jira board after three or four days. In this way, the app will get a bit better after each release.

One of the methods within Egeniq to make sure that the design is implemented as effectively as possible, is organising Design Review Meetings. Egeniq is engaged in all kinds of projects. And all these projects have in common that sooner or later a new design has to be implemented. After the app’s implementation, it is the design what the user sees. Therefore, it is not surprising that – as far as our clients are concerned – the implementation of the app is an important aspect in the development process.

Here, the QA plays a significant role, being the first one to see the implementation of the app after the developers have completed its design. On various devices and screen sizes, for that matter, of which particularly the latter quite frequently tend to pose a challenge. This is because designs are often not suitable for all screen sizes. And given the very extensive range of devices offered by Android, this can be a problem.

In addition, a design is a somewhat static concept; a bunch of separate screens that may or may not be linked to one another. But it is quite difficult to imagine in advance how it will function in the actual device for which it has been created.

For these reasons, Egeniq introduced the Design Review Meetings. When a page or part of an application is ready, and the largest design bugs have been removed, a screen sharing session is planned with the client and the designer. During this meeting, the emphasis is not as much on the design but on the user. By going through the app or page together without focussing on the design, both the designer and the client are given the opportunity to propose objective adaptations. And these are then recorded in a new ticket.

This way of working makes the process easier to oversee and to handle, given that tickets are less likely to come back with additional final instructions. And it gives the client and the designer peace of mind because after the completion of the design they still have the possibility to make adjustments.

These practices are a reflection of how we at Egeniq view the role of a QA. As explained earlier, there is no right or wrong here. What is most important is that this approach has been carefully considered, and that on this basis, conscious choices have been made with respect to its execution.

Part two

Part three