Medical Software Firm Boosts Quality With Eggplant Performance
Performance testing delivers benefits beyond expectations for clinical trial software.
Our client is the software development division of a premier provider to the bio-pharmaceutical and medical device industries. It provides technological innovations that make the drug development process faster, more precise, easier to track, and more productive. One of its products is a clinical trial management solution that provides a single, centralized system for bio-pharmaceutical companies to intelligently manage the complexity of clinical trials.
Service assurance demands
The company was asked by its customers, including many of the world’s largest pharmaceutical companies, to provide assurance that its system would function as expected when in use.
“They wanted us to be able to guarantee that when they scaled up to their 'real-life' usage of the product there wouldn’t be problems.”
The company initially tried some open source performance testing solutions, including JMeter, but then decided to implement Eggplant Performance, which had already been used by another department in the company.
Eggplant Performance is a sophisticated, yet easy-to-use, performance and load testing tool that can test the widest range of technologies and can scale up to simulate any load.
From the outset, our client found that Eggplant Performance was easy to use and well supported.
“We have had good support from Eggplant throughout, even when we faced challenges such as how to navigate through the complex authentication of our systems.”
Mapping user scenarios
Clinical trial management is a complex process, and the software developed by our client is used by several different types of users.
These users range from trial managers, who need occasional access to a wide range of information, through to clinical research associates, who require frequent, highly responsive access to be able to record information relating to patients’ experiences during a trial.
“Using some of our largest customers as examples, we mapped out what a typical usage of the system might look like. We created a performance criteria document that outlined the expectations we had of our application, mainly focused on its ability to respond given certain conditions.”
Criteria range from how swiftly a page should display or update, through to hardware configurations, memory usage or leaks over longer periods. All of these criteria then need to be tested under expected loads.
“In a day, we calculated there might be 500–1000 concurrent research associates on the system or 10–20 managers at a given time. We use Eggplant Performance to run the appropriate tests accordingly.”
Our client typically sets a standard single-user test to establish a response time benchmark and then follows that up with load tests with larger numbers of virtual users, depending on the predicted real-life usage.
The company also uses Eggplant to conduct endurance tests, over a 48-hour period, to see if performance remains steady under load over time, as well as stress tests, where they push the system beyond its limits to see how quickly it recovers.
Eggplant Performance’s Analyzer feature is used to generate reports over a period of time, where the team identifies maximum, minimum and median response times for specific transactions. Outside of the application itself, our client is also load testing its application and database servers using Eggplant Performance.
“There are some significant advantages to using Eggplant From scripting and execution control to result analysis, Eggplant is much more powerful than what we used before. Other important features for us include, auto-correlation, which allowed us to test with tokenization turned on, and the fact that its data can be parameterized easily.”
Fitting into agile
Our client’s product is being developed in an agile environment, and it is the company’s objective to bring performance testing into this methodology.
“Our goal is certainly to performance test more frequently throughout the development process. And the data that we get from Eggplant Performance is already feeding directly back into the product development lifecycle.”
“In addition, we will have a need to look at performance testing on mobile devices in future, as that is the way things are going.”
Benefits beyond expectations
For our client, the primary objective of Eggplant Performance was to provide its customers with service assurance of its software product. And, while this objective has been achieved, the actual benefit of the product has been much greater.
“A lot of what we discovered through the use of Eggplant Performance has been instances of high memory consumption or memory leaks, which would not have previously been identified until customer implementation. We have actually held up releases based on the information we’ve collected at this stage, so it’s also having a positive effect on our software quality.”
“Our initial goal was simply to be able to assure our customers that our product meets certain performance standards but, since we’ve been using it, Eggplant Performance has increasingly become a vital part of our development process.”