From the early stages of planning to the final stages of deployment, ensuring core functionality is met is crucial to delivering what is expected from clients. However, nonfunctional aspects like performance are often overlooked until significant issues arise. By then, without proper performance testing, the consequences can include financial losses, damage to reputation, or even risks to human safety.
Identifying and addressing performance issues early in the software development lifecycle can make the difference between a product that launches successfully and one that leaves a bad first impression on customers. Whether preparing to scale your product for a wider audience or pinpointing the cause of poor performance, performance testing provides invaluable insights that can set the stage for a successful release.
We were tasked with auditing the performance of a product we had previously consulted on. The module under review functioned similarly to a checkout system, where users search for items, add them to their cart, and process payments.
Performance Testing Strategy
While performance targets and objectives are typically set by stakeholders (both business and technical), in this case, we were tasked with identifying potential critical performance issues that could arise during the software’s initial public deployment.
Our strategy involved ramping up the system with a high volume of virtual users to observe how the software’s performance changed under stress. This approach allowed us to establish a benchmark, helping us provide our clients with insights on how much user activity the system could manage before performance began to degrade.
The first step was to identify key scenarios that represented the product’s typical usage. We then scripted these behaviors for our virtual users. We utilized JMeter, a popular load-testing tool, to create operation profiles for our tests. Our scripts were parameterized to ensure that factors like system caching and concurrent user issues would not affect the results, simulating real-world usage.
We set up the tests to measure network response times and the throughput of API calls during execution. These metrics were essential for performing an in-depth analysis.
Although JMeter provides default reports, we chose to use PowerBI for data transformation and visual presentation. This tool gave us greater flexibility to analyze the data and uncover any underlying performance issues that might not be immediately apparent from default reports.
Testing Challenges and Solutions
During the POC, we encountered a challenge due to the unavailability of application usage and monitoring logs. These logs would have provided a more accurate understanding of the use scenarios that represented the application’s core functionality.
To address this, we turned to the available requirement documents and interviewed the developers. These resources allowed us to identify and shortlist key use cases to script our virtual users for load testing. This proactive approach helped us overcome the lack of direct usage data, ensuring we could still effectively simulate the product’s real-world usage during testing.
Key Findings from the Performance Test
Through assertions in our scripts, we identified the breaking point where API calls began to fail. In the test environment, we discovered that approximately 90 concurrent users were able to use the application before a specific API method started returning failed responses.
This issue stemmed from an API call made when users searched for items before adding them to their cart. The problem was confirmed at the front-end level, where we were unable to search for items with over 90 virtual users from our load generation tool.
Further investigation of the IIS logs revealed an inefficient code module. We promptly relayed this finding to the development team for in-depth analysis and resolution.
Final Thoughts
Performance is often an afterthought when developing a software product, only addressed when problems become apparent in the development or early release stages. By incorporating performance testing into the development cycle, we can identify potential issues early, preventing costly delays or performance failures later on.
Performance testing provides insights into the technical and architectural aspects of a software product, often overlooked in standard functional testing. Thus, it is a crucial addition to any software consulting service.
