Introduction:

In today’s world, computers and software are highly involved in different aspects of our daily life, systems handling different tasks are used daily in different fields, systems that are usually formed through a combination of a set of hardware and software working together. In order for these systems to perform properly, different sets of hardware and software forming the system need to be configured together correctly.

However, hardware and software forming different system can have multiple versions and specifications, in order to select the combination specs of both software and hardware that performs better than any other combination, Configuration Testing is done.

What is Configuration Testing?

Configuration testing is testing the performance of the system under development against various combinations of software and hardware to find out the best configurations under which the system can work with any flaws or issues while matching its functional requirements.

Usually, Configuration Testing is a time-consuming process; since usually sets of software and hardware forming the system have many variables which result in a large number of combinations, also installing and uninstalling software and hardware can be a very time-consuming task. That’s why planning is usually an essential phase of Configuration Testing process.

Planning of Configuration Testing:

Planning phase usually includes two main processes, the first phase is about collecting info and building a matrix of different available specs or versions of software and hardware forming the system.  The second phase is about prioritizing combinations in order to select a less number of combinations to be tested so that it matches the timeline of the project.

Now to give an example about planning activities performed in Configuration Testing, consider a simple system  that consists of client-server applications, for this simple system there can be a very large number of combinations, this combinations will consider factors like hardware specs of the device on which client and server  application is running, like size of memory used and hard drives, operation system version on both devices as well

All these different factors can form up to thousands of combinations, which is almost impossible to test all of them considering budget and time limit. This comes in conformance with one of the 7 testing principles which state that exhaustive testing is impossible and all combinations can’t be tested.

So in order to select a reasonable number of combinations for the system to be tested against, a coverage message is created containing all varying factors, here’s an example of a simple coverage matrix considering the only variance in operation system on both client and server sides:

From this simple matrix, we can see that if only operation system versions are the only considered factors, there can be around 70 possible combinations to be tested against, this number increases as we introduce different factors that can result in new combinations. That’s why in the planning phase to select a less number of combinations that match time and budget constraints while ensuring an acceptable coverage and quality.

Configuration Testing Types:

Configuration testing is usually classified into hardware configuration testing and software configuration testing.

Hardware configuration testing is usually executed in test labs containing a big number of all machines variations that can be used in the system, once a build is released, it’s tested on different machines, this can be a very time-consuming task if the number of hardware combinations is big, in order to avoid that, automation is usually used to run released build against selected hardware combinations.

Setting up hardware labs that match the scope of different systems under testing can be a very expensive procedure, usually outsources companies are hired to perform this process.

Software configuration testing is mainly about the different installing and uninstalling different versions of different software used in order to come up with the best combinations, this can be time-consuming as well, that’s why the virtual machines approach is usually used in software configuration testing, a virtual machine can simulate a specific real software configuration. so instead of installing and uninstalling different software configurations, multiple virtual machines can be used, each will simulate a real different software configuration.

Conclusion:

Configuration testing is an important testing approach that shouldn’t be ignored or considered as an unnecessary wasted effort, it can help in defining the conditions under which the system can provide optimal performance with the least flaws or problems.