One of the main reasons behind poor software delivery is lack of time in delivering the product, which results in lesser time in software testing and impacts software quality. Risk based testing can help in identifying which scenarios are business critical and allocates more time to such scenarios. Risk based testing is based on test prioritization.
Risk based testing approach:
Risk Analysis need to be done initially which helps to take control over the issues in an effective manner. The first step in Risk based testing is Test Planning. Potential business risks need to be recognized and Risk Strategy need to be developed. Risk can be categorized on the basis of intricacy in the AUT, different categories of resources and tools. Risk strategy is followed by Test planning activities.
Thereafter, Risk Mitigation plan need to be created. Here, solution for corresponding plan is described. For instance, for intricate applications risk mitigation plan would be like dividing the AUT into smaller units and test them thoroughly with more potential resources.
Risk mitigation plan is followed by Risk reporting. It offers clear and wide clarity for the stakeholders of the project to measure and function on the risk. This helps to determine the risk by making use of techniques like inspection based on test metrics. Output from this stage is fetched into risk identification area and it continues as a cyclic process, till the AUT is free from high risks.
Example for risk based testing:
Consider that there is approximately 6000 -8,000 test cases for the entire set of Applications. Imagine that approximately 1000 to 2000 test cases needs to be executed per release for end to end testing. This can result in delay and is too expensive to function like this. However, in real time applications, among this 1000 to 2000 cases, may be executing 100-200 test cases would result in defects as they are linked to risk prone zones. Statistical method of study will help to get maximum number of defects.
Statistical model is based on the categorization of numerical data. It also helps in measuring the probability with respect to the system behavior. Here testing is based on probability of having a specific issue or a segment to fail on a specific environment. This helps in determining which testing is the best and to evaluate focus areas in risk based testing. This also helps in evaluation of best fit criteria by understanding critical path for the defects. Risk based testing will help to understand which areas demand testing in the AUT.
Risk exposure of the system is a crucial parameter in statistical modelling which is dependent on the probability of occurrence of defects and the after-effect of a defect. This would be dependent on the quality of code to a large extent. It would be caused by factors like poor design or coding by novice programmer. Code quality can also suffer due to complex functionality.
Let us define parameters as follows:
Probability of occurrence of defect – P(f)
After-effect of an issue for the customer – C(c) After-effect of an issue to the vendor – C(v).
After-effect of an issue for the customer will include the following:
- Defects with high risk may result in legal threat
- May cause the customer to lose the market
- Violation of FDI regulation
The two parameters C (c) and C (v) combined with probability of failure results in risk exposure, Re(f)
Re(f) = P(f) * (C(c) + C (v))/2
Consequence parameters for customer and vendor are usually weighed between 1-3. Probability of failure is weighted between 0-1.
The parameter probability of failure is based on following aspects:
- Amended functionality
- New functionality
- Quality of design
- Size of the project
- Skill of programmer
Different types of testing approaches like System testing, core system testing, business specific areas, Integration testing etc. need to be done. QA team also need to concentrate on regression testing, adhoc testing, with special focus on data integrity and volume testing.
What are the input parameter required for designing statistical model?
- Count of issues
- Nature of defect: DB defect, Web area defect
- Categorization of issues: This is mostly pertaining to origin of the issue- if it is from DB, application server
- Effort involved in getting issues and fixing them
- Weightage assigned for probability function
Procedure for Risk based testing:
- Identified Defects need to be classified
- This has to be followed by checking of probability of multiple factors that could hamper quality of the product
- Get the value for risk exposure coefficient.
- Risk exposure coefficient need to be fed into iterative algorithms
- Identify type and count of test cases that are required for Risk Based Testing.
In real time application, imagine that there are 1000-2000 test cases. Suppose that QA team understands based on current analysis that 21st or 31st test case will result in bugs in a particular area which comes under the sampling techniques. Also imagine that there are other bugs from a different zone. QA team can sample them on a common algorithm and fetch them into an algorithm. This helps to determine the value of Risk exposure coefficient. For areas where Re(f) is high, such areas need to be given complete coverage.
What all testing need to be done for complete coverage?
- Cross browser testing
- Compatibility testing
- Verification and validation
- Positive and negative scenarios
- BR validation
- Interface testing
- End to end testing.