Black and white Betta

This post is part of a monthly series of articles on software testing. We intend to cover testing in general, but especially targeted at mobile apps. Developers and testers alike will find useful testing tips and techniques.

We explain the difference between white box and black box testing, followed by some helpful and efficient testing tricks and techniques. Along the way we also define some names and word.

What color is the box?
A common distinction between two vastly different types of software testing, is that between white box and black box. You may associate the latter term with plane crashes, but in this case it denotes that you are testing the software without any inside knowledge about source code, architecture or internal design. This is most likely the case with any app you may test on The Beta Family.

White – the insider
White box testing, on the other hand, is mainly used by developers with access to the source code itself. Distinct code units can be separately tested by writing unit tests that assert expected output for given input data. These tests often strive to cover as many if statements and code branches as possible. The measurement of this is called code coverage, often defined as the percentage of code lines covered by test cases.

While the general consensus is that developers theoretically would like 100 % code coverage, it’s also recognized that it’s realistically very hard to achieve. It would require defining the expected results for all combinations of input, variables and states. An advantage however is the re-usability of unit tests, which makes them very suitable for automation. At this point I should also point out that there are other types of white box tests, even those that are based on access to documentation rather than actual source code.

Black – testing the functionality
Let us move the focus back to black box testing, as this is more relevant to tests and testers on The Beta Family. When testing from a black box perspective much revolves around the functionality itself. As Wikipedia defines the technique: “The tester is aware of what the software is supposed to do but is not aware of how it does it”.

So if you’re an app developer publishing your app for testing on The Beta Family, it’s extremely helpful for the tester if you attach some kind of documentation over expected functionality. Without it you’re essentially relying on the tester’s personal opinion of how the app should work. This may be good or bad: it could produce more “bug” reports than necessary, but also give you unbiased first impressions.

Techniques and tricks within black box testing
Depending on the size and complexity of the app you are testing, it may not be viable to evaluate the expected outcome of every possible scenario. Luckily there are a number of techniques and tricks that with high probabilities help to identify possible bugs and errors.