What is performance testing?
We build a software application which enters into various testing phases such as unit testing, integration testing, system testing and acceptance testing. Here we are covering most of the aspects of the white box testing and black box testing. What about the software application performance? Can we say that an application which is functionally sound after testing is fit for the business purpose and use? Definitely not. There is another important aspect of testing which is known as performance testing. It is a type of test which evaluates the responsiveness and the stability of a software application in terms of its load capacity, behavior under stress conditions, reliability, scalability, etc. When we develop any software application, we always capture the requirements in Functional requirement document such as volume of request per unit time to the application from end user, service level agreement, permitted delay in response from application server, load condition, etc. If an application is functionally sound and the performance criteria is satisfied after carrying out performance testing then we can say that the application is ready to use i.e. eligible for production.
What are baseline and benchmark testing?
When an application undergoes performance testing then we record the performance testing metrics with various application aspects. After certain period of time when the same application undergoes the change such as operating system upgrade, hardware upgrade, patch, etc. then the performance testing team again execute the test to determine the new performance testing metrics and compare the collected metrics against the baseline metrics which was collected before. This kind of testing is known as baseline testing.
Benchmark testing on the other hand is the company wide standard for the performance testing metrics which every software application that belongs to such an organization should pass. Ideally, the baseline performance metrics values should not exceed benchmarked performance standard.
What are performance Metrics?
When we conduct the performance testing for a software application, we record the following performance aspects.
1. Web Server Performance Metrics: Most of the applications which are used by various organizations are web based. Any web based application uses the web server such as Apache, IIS, Nginx, etc. The following are the performance metrics which are determined under the performance test.
- Busy and Idle Threads: Web application serves the request by assigning them to the server threads which have the capability of executing the request in parallel i.e. multithreading environment. To evaluate this performance meter, we load the server with required number of requests and then test the performance after calculating the number of worker thread idle threads, etc. against the delay in the server response.
- Throughput: It is the calculation of the number of requests served per unit time i.e. transactions per second or minute by the web server. It defines the scalability of the web application.
- Bandwidth requirements: Under this performance parameter, we evaluate the bottleneck in the network, weight of the web pages while loading, content to CDNs, etc. Accordingly, we decide the network bandwidth under which the application is going to operate in an efficient way.
2. App Server Performance Metrics: A web application is built in the following platforms:Java, .NET, PHP, etc., each of these platform has the parser or the environment set up on the application server. The following are the key performance metrics that determine the deployment or configuration problems on these application servers.
- Load Distribution: It is the measure of the volume of the transactions which are handled by each JVM (in JAVA), CLR (in .NET) and PHP engine. Next we check if these transactions or the requests to the app server are properly load balanced or more application servers are required to handle the load (clustered).
- CPU Hotspots: Here we evaluate the application performance based on the available system CPU (Central processing unit) on the application server. We check for the CPU utilization on the application server and If CPU utilization is more all the time and the delays are caused in the responses then we might have to consider to increase the CPU power. Sometimes more CPU consumption’s are because of the bad programming such as unnecessary loops in the code, dead code or evaluated values which are not in use, unnecessary waits in program, etc. Such loop holes in the code should come up in code reviews.
- Worker Threads: It is the measure of the volume of the worker threads which are correctly configured. How often these threads are busy due to web server unavailability under load or stress condition and identifying the web server modules that block these threads.
- Memory Issues: This performance parameter deals with the identification of the bad memory patterns, identification of the memory leak, impact of the garbage collection on the CPU and Transaction Throughput as these threads are continuously operating in the background while worker threads are operating, etc.
3. Host Health Performance Metrics: Both Web and Application Servers run on the physical or the virtualized hosts. Therefore, it is very important to do a performance sanity check on all hosts that are involved in the application infrastructure. It includes the following key metrics.
- CPU, Memory, Disk, Input/Output: It is the measure of the CPU, memory, disk and I/O usage when the application operates in a production like environment. It should not happen that under the normal load condition any of these resources get exhausted. We also check for the volume of the logs that the application is writing to the disk to avoid the flooding the disk.
- Key Processes: It is the measure of the types of processes that run on the box. Which all processes are consuming the number of resources or creating the deadlock, etc.? At this performance testing step, we make the decision either to increase the number of resources or move the processes to other box.
4. Software Application Performance Metrics: We are also required to test the performance of the application code under execution. The following are the key metrics to test the application code performance.
- Time Spent in Logical Tier or Layer: The enterprise applications are multi-tier e.g. MVC (Model View and Controller) application. Here, we check which tier is taking more time for execution than other with the increasing load on the application. This helps in understanding the need to scale that tier of the application which is the sole creator of the hurdle in the application operation.
- Number of Calls into a Logical Tier or Layer: It is the measure of the callsto the internal Web Services and other critical APIs such as Hibernate, spring, Struts, etc.
In the article, we have covered the key performance metrics along with the justification what they are importance to test for a software application. There could be other performance metrics which depends on the nature of an application, kits platform, end user, etc.