There are plenty of tools available in market which supports performance testing of web application. Many of them are licensed tools and few of them are open source as well. Few are supporting only web applications few can handle only web API’s. However, VSTS stands tall among them with having support to both of them.
Unlike the Load UI and Jmeter it has lots of flexibility because of support of C# as backend scripting language. Of course it has that UI support to enable record and playback functionality to novice users. Let’s have high level overview of different processes which can be performed by VSTS to achieve complete goal of performance testing of a web application.
Just for easiness of understanding we will take an example of a sample e-commerce website (http://demo.nopcommerce.com/) throughout the series of this article.
Performance Testing Requirement
First phase of any performance testing plan is to identify the actual performance testing requirement and mould it into the format which can be better understand by the tool and the technical person going to use that tool. So here also let’s start identifying requirement of performance testing of any standard e-commerce web application. Here, we will be using our sample application as reference.
Nopcommerce is the e-commerce web application which provides users an easy interface to search for various products, log in, add to cart and pay for the product and all basic functionality specific for the e-commerce web application.
Performance testing is required to get following information related to application.
To validate applications behavior from end user’s perspective when high no of concurrent users are accessing the application. Parameters to be captured for analysis will be……
- Page response time
- Transaction response time
- To find slowest responsive pages
To validate application server, DB server’s behavior when high no of concurrent users accessing the application. Parameters to be captured for analysis will be……
- Memory usage (RAM)
- CPU consumption
- Disk space utilization
For capacity planning. My current hardware is supporting n number of concurrent users. Now firm is anticipating potential increase in user load due to some business requirement. So performance testing is required for capacity planning. Basically to identify…..
- X size of hardware is required to support Y no of concurrent users
Load Test and Workload mix creation with VSTS:
Nopcommerce is the web application and for performance testing of the application Web performance test functionality of the VSTS can be used. Along with the accurate recording of the requests VSTS also provides well-formed UI for generating the load test on top of that. It also provides various options for creating the real life scenario by creating different workload mix.
For this performance test experiment we have considered below information for creating workload mix.
Login -> Search Product(s)-> Log Out
- P % of total user will perform this action
Open Application -> Search Products ( As guest users )
- Q % of total user will perform this action
Login -> Search Product(s)->Add To Cart -> Check out
- R % of total user will perform this action
Search Product(s) -> Add To Cart -> Check out
- S % of total user will perform this action
* Numbers P, Q, R and S will be derived from the historical usage of the application / or from the standard usage of similar existing application
To test the above mentioned scenario first we need to create web test for each of the scenario. Once these web tests are ready you can add them into your load test. There are various options which can be used for the generating most realistic tests.
Multiple options for browser mix and network mix can be used along with various load patterns for creating various permutation combinations of load using same set of tests.
Sample load test is generated using the above information. In VSTS load test would be look like ….
Performance Test Execution:
Based on the requirement of the performance test method of execution can be varied. Generally instead of standalone execution of performance test, it gets executed in some pattern to get some statistical data for analysis. This is called a performance test experiments.
Here, we will execute same test for defined time duration with different concurrent user load (10,50,100,175,250 and 500). We will get different statistical data for analysis out of all this execution cycles.
For example one of the data set for web application is page response time…
Above table is extracted from standard summary report generated by VSTS during one of the execution. This data will be captured for each execution cycle (with 10, 50, 100, 175, 250 and 500 users) for the further result analysis.
Performance Test Result Analysis and Reporting:
Performance test result analysis is very generic term. Process for analyzing result will be varied based on the experiment we are carrying out. Successful completion of this task is really depending upon the accuracy of data collected during earlier phase of performance test execution.
Here for our experiment we have selected the data of page response time during the performance test execution. Based on data captured during each execution we can generate various graphs.
What we need to identify is that with increase in number of concurrent user page response time can be increased but it should be increased in smooth way. If we notice any spike in this chart, then there is an issue which needs to be investigated.
Users Vs Time Chart
This graph shows the condition of page response time along with increase in number of concurrent users.
Current situation in the graph is the ideal condition where response time is not being hampered with increased load.
If the line in the graph inclines with increase in the number of concurrent users than we need to identify the threshold value for acceptable response time for page and based on that value we will have information of supported number of concurrent user with current set up of hardware.
Users Vs Failures Chart
This graph shows number failures with increase in concurrent user load. Failures increases with increase in number of concurrent user load. However, graph should be straight line which is shown in below chart.
If you see a spike here than that is the issue and that needs to be investigated with other experiments.
Users Vs Transactions
This graph shows the number transactions with increase in concurrent user load. Ideally transaction should increase in parallel with user load which is shown in below graph.
If you see this as straight line instead of inclining then there is an issue. Transactions are not increasing with users and that issue needs to be investigated with other experiments
*all this charts are generated with custom written code with excel based on data exported from VSTS summary report
So far we have seen very high level overview of web performance testing with VSTS. Now next article in the series would be taking you into details of each and every activity mentioned in this article so that you will be able to generate your first web performance test using your VSTS.
So basically this was all about what we can do with VSTS with respect to web performance testing and next set of articles would be all about how we can use VSTS to do all these activities described above.
If you are not regular reader of this website then highly recommends you to Sign up for our free email newsletter!! Sign up just providing your email address below: