Whether you have a continuous integration setup or even a continuous deployment chain, it is necessary to integrate the performance testing phase into your chain to ensure that the level of quality and performance is always acceptable for the tightening of your SLAs. Tools like JMeter vs LoadRunner often come into play during this stage, offering different approaches to performance testing. Today’s learning is to introduce you to this stage, which is between continuous integration and continuous deployment, continuous performance, or continuous performance testing.
What is a performance test?
Load testing is the art of creating artificially generated work that mimics real work created by real users. Performance tests allow you to expose a service to representative and reproducible activity (load model) determined from observations or predictions of real user activity (usage model) by injecting load.
In other words, performance tests allow you to identify poor service performance, quality degradation or slow response time, inability to meet demand, unavailability, etc.
Why should you conduct performance tests?
It is important to conduct performance testing because all these indicators directly affect the company’s business. If they are bad, it can lead to loss of profit, drop in sales, customers fleeing to competitors, loss of audience, or even a deterioration of the company’s image!
Different metrics to consider and monitor
To check the performance of the program, it is important to track the metrics of both the last and the main machine (if possible). This is a crucial step when conducting a JMeter and LoadRunner comparison, as both tools offer different capabilities for monitoring and analyzing these metrics effectively.
We can note several metrics that should be monitored during the load test:
- CPU
- RAM
- Disk
- Network
- Response time
- Number of hits per second
- Number of running threads
- Re-access to the database
- and many others
Different types of performance tests
Now that you know the metrics you’re going to monitor, it’s important to examine the context in which the shot will be taken and what type of load will be applied.
In most cases, it is recommended to apply at least these 4 types of shots to ensure that the results are tangible as well as the quality of the application.
- Unit test: The unit test allows us to test all the endpoints of the application, like a functional test, and allows us to record the response time, which allows us to calculate the number of virtual users to launch later.
- Nominal: this type of test is a test that will apply a standard scope to an application, or production type, or to meet an SLA.
- Endurance: An endurance shoot is a shoot that lasts for several hours, a minimum of 8 hours. This type of test allows you to highlight problems that would not be detected by a level or nominal test, such as a memory leak from the JVM.
- At Level: This type of injection allows you to apply different levels of load to an application, which can range from +100% to +200% or +300%, to determine the limits of the application or to test the application in degraded mode.
Different performance testing tools
You have numerous tools available to perform performance tests. The choice of injection tool will depend on various factors, such as your application’s runtime environment (IaaS, CaaS, PaaS?), the protocols you intend to test, your hardware, and the cost of the license. The most popular injection tools are:
- JMeter, developed by the Apache Foundation
- Neoload, developed by Neotys.
- LoadRunner, developed by Mercury in 1999, purchased by HP in 2006, and finally acquired by MicroFocus in 2016.
Let’s choose one example and take a closer look at JMeter, a desktop program developed by the Apache Software Foundation that allows you to test program downloads. JMeter was originally designed to test web applications, but it has since been expanded to allow testing of any type of software, from a very simple piece of code to a complete program. In a LoadRunner vs JMeter pros and cons analysis, JMeter stands out for its open-source nature and flexibility. You can even simulate parallelism using threads, which allows you to measure performance as the load changes. The results obtained by JMeter provide statistical values such as mean, median, variance, and throughput. Let’s learn how to test a web application using HTTP requests with functions provided by JMeter.
Option 1. Create a test plan
First, you need to create a test plan describing the steps JMeter should take to execute the test. You can configure the following options:
- Name: Each test plan is identified by a name that appears in the menu on the left;
- User-defined variables: You can associate n variables with a test plan that can be called in the test;
- Run Thread Group consecutively: you can set whether the threads (virtual users) should be started in sequential mode (at the end of a thread, the next one starts) or in parallel mode (the first one begins after k seconds). the second one starts; after 2k seconds, the third one starts, and so on);
- Library: You can add classes or jar packages to the class path.
You can see how the name of our test plan has been inserted into the tree menu on the left, as well as add items to our test plan by right-clicking on the name.
The main element of the test plan is the thread group, which can contain all the other elements. The thread group allows you to set the number of threads that simulate simultaneous access by a variable number of users, the maximum time that will elapse from the start of the first thread to the start of the last thread (ramp-up period), and how many times the test should be executed. Then you add a group of threads that represents the users of our web application.
Option 2. Adding an HTTP request
A window opens where you can set the following basic parameters:
- Name: a name by which to identify the request in the menu;
- Server name or IP: the name of the server or IP to connect to;
- Method: the GET or POST method used to send the request;
- Path: the server resource to be retrieved;
- Parameters: parameters to be passed in the request.
To view the test results, you need to insert Listeners. JMeter provides different types of listeners, each of which displays results in a different way.
Conclusion
Performance testing is vital for ensuring application stability and meeting SLAs in continuous integration and deployment pipelines. Tools like JMeter, an open-source option, offer flexibility and cost-effectiveness for testing various applications, while LoadRunner provides enterprise-grade scalability and protocol support for large systems. Companies like PFLB, specializing in performance testing solutions, can help organizations choose the right tools and strategies for their needs. The choice depends on your project’s requirements, budget, and environment. By simulating real-world load and monitoring key metrics, you can identify performance bottlenecks, enhance user experience, and safeguard business outcomes. Integrating performance testing ensures reliable applications that meet user demands, prevent downtime, and maintain business reputation.
