BT Drives Development Transformation With Eggplant
Telecom giant increases test automation by 150%, driving productivity and quality improvements.
BT is one of the world’s leading providers of communications services and solutions, serving customers in more than 170 countries and with annual revenues of £18 billion. One of its divisions, BT Consumer, is the largest provider of consumer fixed-voice and broadband services in the UK and provides broadband, TV, exclusive sports channels and mobile services to more than 10 million customers across the country.
BT spends tens of millions of pounds on the business support systems (BSS) it uses to support customers of its consumer division. These systems are used by customers who visit the company’s web site or call in to its call centres, and also by the BT staff who support them. Typical support provided includes ordering products and services like telephone lines, TV, mobile and broadband, as well as “Trouble to Resolve” services to help fix problems. Increasingly, these support services include self service tools that help customers to diagnose and solve issues on their own.
Increasing scale and complexity
Ensuring the quality and excellence of customer service is critical to BT. Testing of its consumer-focused BSS had been predominantly conducted manually (with some limited automation using HP’s UFT in certain areas), but as the number and complexity of systems and features increased, the company began to search for an automation tool that would help ease the testing burden on its staff and improve testing coverage.
“It takes time to test every change manually, so it’s quite typical to see three-month development cycles,” said Andrew Hodgson, Head of BT Consumer E2E Test. “Obviously the business would like that time to be reduced significantly, and we are also looking to expand the breadth of testing to improve the quality of our systems.”
The company considered a variety of tools, including UFT, Selenium and Perfecto Mobile, but found that Eggplant Functional was the product that best suited its requirements.
“Eggplant provides the best overall mix of capabilities from an end-to-end test perspective,” said Hodgson. “The ability to automate cross-browser testing as well as testing on multiple devices was a key factor. Devices are a growing area for us, including mobile handsets and tablets, but also increasingly additional hardware such as set top boxes, so Eggplant ’s technology agnostic approach to testing was well-suited to our needs.”
To prove to the organisation that Eggplant Functional would deliver the results expected, BT engaged in an initial six-month project, with the aim of automating the majority of regression testing and improving quality.
Test automation with Eggplant and UFT was implemented across almost every aspect of BT’s consumer-focused BSS. This involved testing everything from a customer’s initial engagement on the company website and setting up payment details, through to fault-checking and validation that the various backend databases and systems (Oracle and Siebel) are being updated correctly.
Eggplant's ease-of-use was recognised as a key factor in the deployment, largely attributed to the simplicity of its SenseTalk programming language.
“SenseTalk made it very easy for our team to quickly adopt Eggplant Functional,” said Hodgson. “We didn’t have a high level of automation adoption before Eggplant because you had to be an ‘automation expert’ to work with UFT. Our strategy is now to roll out automation to everyone involved in testing and SenseTalk allows them to do this.”
In addition, BT was impressed by the support that it received from Eggplant during the project.
“Eggplant didn’t just want to sell us a product,” said Hodgson. “They showed willingness and interest in fully engaging in our solution and this was a key factor in our decision to expand the use of Eggplant across the organisation.”
“Within six months of using Eggplant , we automated 75% of our regression testing,” said Hodgson. “In addition to reducing costs, this has helped us quickly achieve quality improvements by being able to increase the number of times tests were run on each development cycle. We anticipate that we will be running at least 2.5 times the number of test executions by the end of this financial year.”
Increasing the number of times regression tests are run during a development cycle has a direct impact on improving quality by ensuring more issues are caught when changes and updates are made to an application in development.
“Regression test automation will also lead to more flexibility in our delivery, and ultimately will help us significantly reduce our development cycle time,” added Hodgson.
Return on investment
“Automation has been win-win for everyone involved in the development process,” said Hodgson. “The improvements in quality lead to productivity improvements because, by finding defects sooner, we are helping take the pressure off the development teams, allowing them to focus on the next set of changes, rather than the ones they did last month.”
Rather than using the efficiencies gained as a reason to cut back on staff, the increased productivity of BT’s testing community has freed up resources to spend more time investigating why tests might fail, and increasing testing scope and coverage.
“We are already starting to see the benefits of a cultural change initiated by test automation, where our testing community is moving away from the more mundane, repetitive testing activities to higher-value input into the delivery cycle,” said Hodgson. “And with the cost savings we’ve realised through automation, we’ve basically been able to achieve these benefits for free.”
BT is only at the beginning of a transformational approach to its development methodology and is aiming to achieve 100% automation of previously manual regression testing by the end of the financial year. However, the work will continue beyond that point to expand the breadth and depth of testing.
“Not only will we be looking to increase the number of tests being executed on each cycle, we also plan to expand the breadth of testing to include more kinds of test cases,” said Hodgson. “Our strategy is to increase the use of Eggplant Functional over UFT because of its flexibility and strengths in cross-browser and cross-device testing.”
Looking forward, the company is also looking at automation of new functionality testing, as well as testing in production.
“It’s more challenging to automate new functionality, but if we are able to drive development from test automation, then we will be able to look at adopting a Continuous Integration model, and that is really exciting for us!” said Hodgson. “And we can also see huge benefit in automating testing in production environments, which we are also pursuing.”