Retail Giant and Eggplant Get Creative With Test Automation for POS
Innovative approach generates a 92% reduction in test cycle effort.
Our client is one of the world’s largest retailers, with millions of customers visiting thousands of its stores in countries across the globe each year.
Retail point of sale (POS) systems are a critical technology component for any modern retailer and our client is no exception. Every U.S. store is equipped with some form of POS system, ranging from traditional systems where the customer and cashier use different screens, to self-service system where the customer scans and weighs items themselves and uses a touch screen to complete their purchase.
Increasingly, there is also a mobile component to POS systems — whether offering customers mobile payment options like Apple Pay and Google Wallet, or enabling staff to use mobile devices to help speed up checkout — an approach that is growing in popularity at its bulk retail locations.
The complexity of POS
For any large retailer, any problem with its POS systems is potentially disastrous. Like any other software-driven technology, POS systems have to be rigorously tested to ensure they remain stable when changes and updates are made.
But POS systems are expensive and retailers do not often replace them, choosing instead to build new features and technologies on top of well-established, trusted platforms, such as the IBM 4690 in our client’s case. As a result, its POS systems are built on a very thick technology stack, comprising a multitude of legacy technologies, and are also worked on by several generations of development and testing staff, meaning that no one has complete knowledge of the system.
These complexities meant that the retailer historically conducted all of its POS testing manually, not believing that there was a solution that could automate such a complex scenario.
As time went by, our client noticed it was taking more time to manually conduct regression testing of its POS applications ahead of releasing a new build. Its POS team releases updates using an agile methodology, which was generating builds too quickly for the manual test team to conduct regression testing. Time for manual testing eventually grew to around 125 man days per test cycle, predominantly due to the growing number of test cases to cover increasingly more complex customer journeys.
In addition to delaying the rollout of updates, this also resulted in increasing use of resources, including significant outsourced IT consulting, leading to growth in the overall cost of testing. Our client was particularly keen to reduce its reliance on consulting firms and to take more control over its POS testing processes.
The POS team realised that automation might be the solution to their increasing testing demands.
Other divisions within the company were using UFT from HP to conduct test automation. The challenge with using an object-based tool like UFT is that it could not effectively penetrate the complex layers of the retailer’s POS system, and the same was true of any other automation tool they tried.
After spending extensive time and effort looking for a solution, our client’s executives were beginning to believe that they were stuck with their automation problem. Until they discovered Eggplant Functional.
Eggplant Functional helps companies test their applications better, faster, and with less effort by automating the execution of functional testing. Eggplant Functional uses a patented, image-based approach to GUI testing that allows it to interact with any device by looking at the screen, in the same way a user does.
Implementing with innovation
While Eggplant's image-based approach to testing presented a potential solution to the automation challenge, there was still a lot of work to be done. The POS systems had to be tested in a production state, which meant that a range of KVM switches would need to be used to control the terminals during testing.
The complexity of the POS stack meant that it was still necessary to understand which parts controlled certain functions in the terminal before Eggplant could automate.
For several months, Eggplant worked closely with the client to solve every challenge the POS presented. Eggplant's engineers refused to be beaten, even working with the manufacturer of the KVM switches to build custom firmware to allow the retailer to overcome some of the hurdles they faced.
While Eggplant's engineering team was working to address the challenges of automating testing, the company was simultaneously working with our client’s manual testing consultants, to develop the core test scripts that would enable automated regression testing.
As different retail locations were liable to require different scripts, a lot of time was also spent training the consulting staff on how to use Eggplant to build their own scripts for specific circumstances. Eggplant’s SenseTalk language allows even non-technical users to become productive with Eggplant within a few hours, as it provides them with the ability to write test scripts without having any programming knowledge.
Card reader testing, Terminator-style
One challenge of automating POS testing is how to test some of the more manual customer processes that do not involve screens, such as swiping credit or debit cards, entering PIN numbers, or making an on-screen signature. Once again, innovation was the order of the day, as a solution was found far outside the box: a robotic arm from Rethink Robotics.
Now, during any testing that requires a customer to swipe a card, Eggplant controls the robotic arm, which selects a card from a selection, swipes it and completes the transaction, whether by entering a PIN on a keypad or even signing on a signature reader.
Following the successful automation of the core regression tests, the scope of automation was quickly expanded to include additional areas, including self-service and mobile payment elements.
Automating for results
In total, the retailer’s POS team works with a set of around 5,000 regression test cases, of which they eventually hoped to automate approximately 80 percent. Since implementing Eggplant Functional, our client has automated over 2,500 test cases and counting.
Eggplant has worked well within the company’s agile development environment, allowing the test team to cope well with increasingly frequent development sprints, and ensuring release delays are avoided.
While previously requiring 125 man days for each test cycle using purely manual testing, our client has been able to reduce this to just 10 man days per cycle using Eggplant Functional. An effort savings of more than 90 percent. Through these savings, the company has seen improvements in quality, due to its ability to run tests more frequently, as well as allowing resources to be reallocated to more exploratory testing on data that emerges from the automated testing.
Looking to the future
Following the success of Eggplant Functional within its POS group, our client is beginning to explore uses of the tool in other areas of the business. It is also exploring potential uses of Eggplant Automation Cloud for management of mobile devices under test and Eggplant Network to test potential different network conditions.