Testing is the backbone of the SDLC lifecycle, therefore any software application or product developed by an organization can be put into use only after that product has achieved testing sign off. Therefore in order to evaluate the testing effort in SDLC, below are the definitions of two terms, Test Effectiveness And Test Efficiency.
What is Test Effectiveness?
Test Effectiveness can be defined as how effectively testing is done or goal is achieved that meets the customer requirement. In SDLC (Software development Life Cycle), we have requirements gathering phase where SRS (Software Requirements Specification) and FRD (Functional Requirements Document) are prepared and based on that development team starts building the software application, at the same time test cases are carved out of SRS and FRD documents by the testing team. Test effectiveness starts right at the beginning of the development and execution of test cases and after development is completed to count the number of defects. Defects can be valid or invalid. Valid defects are required to be fixed in the application or product and invalid ones need to be closed or ignored. Thus, mathematically it is calculated as a percentage of a number of valid defects fixed in software application divided by the sum of a total number of defects injected and a total number of defects escaped.
Test Effectiveness Formula:
Test Effectiveness = ((Defects removed in a phase) / (Defect injected + Defect escaped)) * 100
What is Test Efficiency?
Test Efficiency can be defined as the most efficient way in which the resources associated with the software project are utilized to achieve an organization goal (for example, number of projects to be completed in that particular year). Test efficiency is an internal process for an organization that evaluates the utilization of resources which could be a timeline, hardware, manpower, expertise of test team, etc. to develop a software product. Mathematically test efficiency is calculated as a percentage of the number of alpha testing (in-house or on-site) defects divided by sum of a number of alpha testing and a number of beta testing (off-site) defects. Alpha testing is the testing done by the expertise project test team on-site and this test team is expected to test the product thoroughly before the product is available to the customer or end user in the market. Once internal alpha testing is completed the product is made available to end users to test and look for the defects and provide their valuable feedback. Ideally, there should not be any defect observed by the end user during beta testing as any valid bug identified during beta testing, it will directly deteriorate the test efficiency of the on-site project team.
Test Efficiency Formula:
Test Efficiency = (Test Defects (Alpha) / (Test Defects (Alpha) + Acceptance Defects (Beta))) * 100
Test defects = Unit testing defects + Integration testing defects + System testing defects.
Acceptance Defects = Defects or bugs found by the customer or end user during the acceptance testing.
Factors Affecting Test Efficiency:
As explained above, the expectation from test team is 100% efficiency which is quite challenging. Onsite testing demands efficient project management, testing expert professionals, the best training to resources at technical as well as business level. Below are the key points to achieve almost 100% test efficiency.
- Efficient Project Management: Project management plays a key role in running the project smoothly. From a testing point of view, project management should make sure that the project has ample resources at headcount as well as expertise level. Project management should take care of the smooth transition from requirement phase to development and testing phase or in order words project requirements should be crystal clear. Project management should facilitate regular project meetings and track the project progress.
- Expertise Test Professional: In order to achieve 100% efficiency, one cannot deny on the fact that such test team demands testing professionals who are expert not only at a technical level but should have project domain knowledge and ability to think rationally on unforeseen test scenarios. For example, if the software product is related to the banking industry then a test professional having the knowledge of banking concepts suits best for this testing team.
- Project Training: Not only best management and expertise test professional are sufficient to drive a software project efficiently but a software project demands regular project-related training to test professional so that they may sharpen their skills and contribute to the best testability of the software product.
- Technologies and Test automation: Testing team should leverage the use of latest testing tools and technologies to automate the testing which will save the human effort and provide the ample time to the test team to test for various unforeseen test scenarios which can be later automated.
Over to you:
Test effectiveness and test efficiency are very important to count for a software product on the market value or an asset to the customer or end user. Better the test efficiency the best is the test effectiveness. 100% test effectiveness allows the customer to gain confidence on product manufactured by an organization whereas 100% test efficiency on the other side allows an organization to rate the efficiency of its project test team which is directly proportional to the quality of the software product manufactured by it and hence the market reputation of an organization.