Load & Spike Testing with JMeter
Any internet user can attest to how frustrating it is to experience website delays or crashes. It can be annoying and frequently costs internet businesses money. Performance problems can, in a nutshell, be very disappointing. As a result, teams need frequently spend money on performance testing. Performance testing makes ensuring that the application is as fast, responsive, stable, and reliable as needed. Additionally, it increases user confidence in the program in question, which leads to better execution and requires less upkeep.
There are several tools to carry out performance tests, and that too of various types. One such tool is JMeter, through which one can easily carry out Load and Stress testing, two common performance testing types.
Load Test
A type of performance test, load testing determines how the system under test behaves when concurrent users act. In simple words, it is used to determine the user capacity the system can handle. This can be done by gradually increasing the number of users to hit the network request, to simulate real-time multiple users’ behavior, over a period of time.
A way to perform load testing is to use a Concurrency Thread Group, which is a specialty thread group for load testing by Blazemeter. It uses concurrent users to perform a specific action at the same time. In other words, the same number of concurrent threads would be created which will execute the desired request at the same time for a specific period of time.
To use it, first of all, you will need to install it using the Plugins Manager. Once that’s done, it will be visible under Thread groups like so:
Here,
- Target concurrency refers to the total number of virtual users.
- Ramp-up time refers to the total time to gradually achieve target concurrency.
- Ramp-up steps refers to the increments to reach the target concurrency.
- Hold Target Rate Time refers to the additional time the target concurrent users will continue to execute at the same time all at once.
Optional values:
- Thread Iteration Limit refers to the loop count of the thread.
- Log Threads Status into File refers to the file path where the logs should be stored.
This means that every 6 seconds, 20 users will be added until we reach the target concurrency of 100 users. This we calculate by dividing the Ramp-up time 30 sec by steps 5, resulting in 6 second per step of ramp up. Also divide the total users 100 by 5, giving us 20, so 20 users every 3 minutes. Once 100 threads are reached, all of them will continue to execute for another 30 seconds, which is the Hold Target Rate. This is also evident in the preview graph generated by the values added. To view the results, you can add listener of choice.
Stress Test
As the name suggests, this type of performance test checks the limits of the system under test, by placing it under extreme load or stress. There are two main types of Stress tests: Soak test and Spike test.
In Soak test, we place the system under stress for a long period of time. On the other hand, in Spike test, we add a large sudden spike in load in a short period of time. The core difference between the two is the way they ramp up and down the total number of users.
Using the Ultimate Thread Group is valuable when it comes to stress testing. Similar to the previous one, in order to use one you will need to download it with the plugins manager and then add it to your test plan.
Here, for each schedule added, the expected behaviour is depicted on the graph.
- Start threads count refers to the total number of virtual users for a schedule
- Initial delay is the gap or delay in time before the test starts
- Startup time is the ramp up time taken to reach the target number of virtual users
- Hold load time means how long the threads will continue hitting the server
- Shutdown time refers to the time to shut down the threads or reduce the threads to 0.
In this example, for the first schedule, since the initial delay is 0, so it will start the test from 0. The startup time is 20 sec and the start threads count is 10, so it will take 20 seconds to reach 10 concurrent threads. The hold load is 5 seconds in this case, so the concurrent threads will execute for 5 seconds. The shutdown time is 0, so these threads will not decrease to 0, but instead move to the second schedule.
The second schedule is where the spike comes in. There will be a spike of 400 users and it occurs after 25 seconds, which is the total time taken by the previous schedule. The start up time is 0, so the spike will be sudden and 400 concurrent threads will execute simultaneously, rather than ramp up. This will continue for another 10 seconds, the hold load time, and will not decrease to 0 as the shutdown time is 0.
The third and last schedule is when the test returns back to its original number of concurrent users and gradually shuts down.
These values can be modified as per the need of your system’s performance requirements, to simulate the expected behavior of multiple users. For instance, in the flash sales period, applications should expect a sudden increase in concurrent users. In that situation, prior performance testing would be a wise decision to anticipate how the system handles the load.
The above two scenarios explain how to cover load and stress testing scenarios using JMeter. When tasked with performance testing for a project, you will most likely also be required to submit a report at the end of the performance test with your findings. There is an easy way to generate a report using JMeter.
Report Generation
JMeter offers generating a report for your performance test flows. A simple HTML report generation requires 3 things:
- Simple Data Writer: A listener where you need to specify the path to an empty CSV file which would contain test results.
- User Properties: The user.properties file can be found in the bin location of installation of JMeter, this would contain any user specified variables used in the jmx file.
- Output directory: An empty folder directory where the report should be generated.
To generate a report, simply have the above mentioned items ready, then, in your jmx file, go to Tools > Generate HTML Report > Enter the paths to the above 3 items, and click Generate Report. This will generate a report.
Another way to do this is through the command line. To generate a report through command line, you must run your tests through command line first.
Run the following command to execute tests through command line and generate a report with a single command:
sh jmeter -n -t <JMX file path> -l <CSV file for test results> -e -o <Folder path to store report>
Here,
- -n to denote non-GUI mode
- -t followed by Location for jmeter test script
- -l followed by Location of the result file
- -e To generate HTML Reports
- -o followed by Location of the Output folder to store report
