Software Testing Class

Pairwise Testing in Software Testing

We need test cases with 100% test coverage in order to test any software application. This will ensure that all possible combination of input and output parameters are well tested. Now a quick question, is that possible to achieve 100% test coverage without writing verbose test cases? Well, it’s possible if we practice “Pairwise Testing” or “All-Pairs testing” test design. Pairwise Testing can be defined as a “black box testing” test design technique that ensures almost hundred percent test coverage without writing verbose test cases and their execution.

With this test design technique, the testing team can achieve maximum test coverage ensuring high defect yield ratio and there is no need to test the application behavior against all possible combinations of input and output parameters. Since a number of test cases are less, therefore, test cases execution requires very less time and covers almost all possible scenarios.

In the black box testing approach, we test the application behavior for input parameters versus output results obtained. Again, the application output depends on input parameters, static values, state transitions and variables, output values, etc. Other approaches like boundary value analysis, equivalence partitioning, etc. are beneficial but these techniques do not provide maximum test coverage. Therefore “Pairwise Testing” approach is practiced to achieve maximum test coverage which we are going to discuss below in this article with an example.

Pairwise Testing

Advantages of Pairwise Testing:

Disadvantages of Pairwise Testing:

Example of Pairwise Testing:

Suppose we have to test an application known as “Phone Ordering System” which has following input parameters along with their set of permissible values.

Automated “Public Library System”:

Rule I: First of all, we need to identify the variables and its permissible set of values.

  1. Member Residence: If library member is resident of Toronto of other cities of Ontario such as Hamilton, Mississauga, and Brampton. Total 4 cities in scope.
  2. Book Bar Code: Total 80,000 books are present in the library with associated status as reserve and non-reserve.
  3. Book Language: Books are either available in French or English. Total 2 Languages.
  4. Book Reserve Method: As discussed above there are total 2 ways such as online or in-house.
  5. Book Categories: There are total 3 categories of books as discussed above. These 3 categories are encyclopedia, fiction, and academics.

In order to test all possible valid combinations, here is the mathematics.

=> 4 X 80,000 X 2 X 2 X 3

=> 3,840,000 (Valid tests)

Also, if consider the negative tests and invalid test combinations, the total test number may bump up to some indefinite value.

Rule II: Smart Representative Simplification

New valid combinations will look like as below:

=> 2 X 2 X 2 X 2 X 3

=> 48 (Valid tests)

Rule III: Arrangement of variables and corresponding values as input test data.

After arranging variables and values, test data table looks like this.

Phone Ordering System Specifications:

Rule I: Identify the input variables required for testing and permissible set of values.

  1. Purchase Type: Permissible values are Buy and Sell. Total number of values is 2.
  2. Booking Type: Permissible value is In-store. Total number of value is 1.
  3. Deal Time: Permissible values are ‘Working hours’ and ‘Non-Working hours’. Total number of values is 2.
  4. Location: Permissible values are Bangalore and Mumbai. Total number of values is 2.
  5. Phone Brand: Permissible values are Apple, Samsung, and LG. Total number of values is 3.
  6. Contract Number: Permissible values are Active (6000) and Inactive (2000). Total number of values is 8000.

In order to test all possible valid combinations, here is the mathematics.

=> 2 X 2 X 3 X (6000 + 2000) X 1 X 2

=> 192,000. (Valid tests)

Also, if consider the negative tests and invalid test combinations, the total test number may bump up to some indefinite value.

Rule II: Smart Representative Simplification

Let’s reduce contract number variable into Active and Inactive, i.e. total 2 value. Also, try to use groups and boundary conditions even though data may not be non-discrete.

New valid combinations will look like as below:

=> 2 X 2 X 3 X 2 X 1 X 2

=> 48 (Valid tests)

Rule III: Arrangement of variables and corresponding values as input test data.

After arranging variables and values, test data table looks like this.

Sorting the columns of the table in descending order, as given below.

Rule IV: Arrange the table to prepare the test suite data as guided below:

Choose the column with a maximum number of permissible values first and place the values of variables as given below. It is actually multiple of first two column numbers (here 3 X 2 = 6). It will look like this.

Based on the first two column arranged in descending order, prepare the table as given below. Such that each book categories should be made available to both Toronto and non-Toronto residents.

Let’s try to place the next column ‘Book reserve method, such that it provides maximum test coverage, after placing online and in-house values it will look like this.

After placing online and in-house values, we can observe the pairs (Toronto, online) and (non-Toronto, in-house). Using such data these two cities could never be tested for in-house and online book reserve method respectively. The problem can be solved when we swap the online and in-house vales as indicated in the above picture and it will look like as given below in the picture.

Let’s place the values for next variable ‘Book Bar Code’ as reserve or non-reserve. Again we can identify pairs as (Toronto, reserve) and (non-Toronto, non-reserve). In order to increase the test coverage, we can ‘Book Bar Code’ values for 5th and 6th rows as demonstrated below.

After swapping values for ‘Book Bar Code’, the table will look like as below.

In a similar way, I have included values for variable ‘Book Language’ as French and English such that each of the row pairs is unique and covering all possible scenarios.

Let’s include the last variable ‘Book Language’ value. It has two value, therefore, we can place French and English as shown in the picture below.

Thus, we have left with only 6 valid combinations to test for all valid possible scenarios which we calculated as 48 before. This is what made possible by “Pairwise Testing” technique.

Conclusion:

Pairwise Testing” requires initial effort to understand and select the possible input data combination. When this is selected efficiently it may result in high defect yield ratio and may provide 100 percent test coverage. It is one of the best design which results in high test efficiency in considerably less effort and time. There are number tools available in the market that can be used to achieve “Pairwise Testing” technique as follows.

S. No.Pairwise Tools NameVendor
1ACTS [formerly: FireEye]NIST
2AETGTelecordia
3AllPairsSatisfice
4AllPairs[McDowell]
5AllPairsMetaCommunications
6ATDAtYourSide Consulting
7Bender RBT Inc.BenderRBT
8CaseMakerDíaz & Hilterscheid
9CATS (Constrained Array Test System) *)[Sherwood] Bell Labs.
10Combo-TestThe Australian eHealth Research Centre
11CombTestWeb Universidad de Castilla-La Mancha
12CTE-XLBerner & Mattner
13CTS (Combinatorial Test Services)IBM
14DDA *)[Colburn/Cohen/Turban]
15EXACT *)[Yan/Zhang]
16HexawiseHexawise
17IBM Functional Coverage Unified SolutionIBM
18Intelligent Test Case Handler (replaces CTS)IBM
19IPO (PairTest) *)[Tai/Lei]
20IPO-s *)[Calvagna/Gargantini]
21Jenny[Jenkins]
22NTestCaseBuilder[Murphy]
23OA1k sharp technology
24OATS (Orthogonal Array Test System) *)[Phadke] ATT
25OATSGen *)Motorola
26Pairwise Test Case GeneratorTestersDesk
27PairwiserInductive AS
28PICTMicrosoft Corp.
29PictMasterIWATSU System & Software
30Pro-TestSigmaZone
31rdExpertPhadke Associates, Inc.
32ReduceArray2STSC, U.S. Air Force
33SmartTestSmartware Technologies Inc.
34SpecExplorerMicrosoft Corp.
35tcases[Kimbrough]
36TCG (Test Case Generator) *)NASA
37TConfig[Williams]
38Test Vector Generator 
39TestCoverTestcover.com
40VPTAG [Robert Vanderwall]

⇓ Subscribe Us ⇓


If you are not regular reader of this website then highly recommends you to Sign up for our free email newsletter!! Sign up just providing your email address below:


 

Check email in your inbox for confirmation to get latest updates Software Testing for free.


  Happy Testing!!!
 
Exit mobile version