Successful Automation of Acceptance Testing at Air Traffic Controller

Eggplant - Case Study - NATS

Acceptance Testing


Previously known as the UK’s National Air Traffic Services, NATS is responsible for providing safe air traffic navigation services to aircraft flying through UK controlled airspace and at several UK and international airports.

NATS is a public private partnership between the Airline Group, a consortium of seven airlines, which holds 42 percent, NATS staff who hold 5 percent, UK airport operator BAA with 4 percent, and the government, which holds 49 percent and a golden share.

The company provides air traffic navigation services for the airspace over the whole UK, including London, one of the busiest and most complex areas of airspace anywhere in the world. It is important that the organization remains agile and is able to redesign airspace in reaction to key events. For example, the London Olympics required new temporary controlled airspace to be designed and implemented to accommodate the increase in air traffic during the games. All its underlying systems must therefore be designed and prepared to cope with flexible demands.

The need to automate testing

NATS’ Asset Infrastructure team is responsible for all infrastructure systems (voice, radar, navaids, etc.), and the voice communications system that enables air traffic controllers to talk to aircraft via radio systems. The infrastructure and equipment for the NATS voice communications systems is delivered from approved suppliers. All equipment submitted by these suppliers goes through formal acceptance testing and then undergoes further confirmation testing and user-acceptance testing on delivery by NATS using various methods and approaches, which includes using Eggplant. NATS also conducts additional testing of specific software builds using Eggplant. So Eggplant is used to test the highest level of boundary interface testing, testing the user interfaces of the software and builds. This need has evolved as the company has adopted new software and new software builds.

Stuart J. Watson, senior systems engineer at NATS, holds the responsibility for selecting the right testing methodologies and software for NATS’ Voice System as part of the Asset Infrastructure Engineering division. This includes the underlying technologies and processes for NATS’ voice-based and human-machine-interface systems.

The Asset Engineering Team came across Eggplant through its use in other parts of the business. Watson explains: “I was aware of the purpose of it l elsewhere in our business — what it did and how it did it, and I thought it was interesting and relevant to us and to our needs.”

The Asset Infrastructure Engineering team conducted a set of parallel evaluations of Eggplant together with another automated software testing solution: “Eggplant came out the clear winner,” says Watson.

The challenges

Eggplant is used by NATS as a boundary test, which Watson and his team call “a replacement for the human finger.” The system is a touch-based voice communication system used to make radio calls and telephone calls. The testing NATS completes with Eggplant is at the boundary or the interface, because it is testing the human interface to the underlying system.

Watson continues: “Before we had Eggplant there was a limited amount of manual testing, but not a great deal as we weren’t dealing with the same number of software builds. Now we are dealing with a number of new software builds and we either needed to scale up our manual testing team or towand a better way to do it and move to an automated solution — which is what we did and why we decided to work with Eggplant and use Eggplant.”

He adds: “Speed to market is a key factor for us. Before we were using Eggplant, we were sending teams of our testers out to our suppliers, which could be teams of ten or 15 people leaving at a time for software testing. But now with Eggplant we can set up, manage and execute this testing from our site in Fareham, with effectively one person writing scripts and the Eggplant software doing the rest.”

Eggplant - Case Study - NATS2 - Control Interface

The need for a non-invasive testing system

A feature of Eggplant crucial to NATS is its unique non-invasive testing, achieved by controlling or testing a system from another computer using a KVM with a Virtual Network Computing (VNC) communications link. This remote connection allows Eggplant to see the screen on the system under test. It recognizes icons, buttons, message boxes, prompts, text, backgrounds, colors, and search and compare screens, and then signals test success or failure.

Watson adds: “We wanted something which could interface with the system without being resident on the system itself so there would be no software installed on our actual system or any of our servers. We use the VNC capabilities of the Eggplant testing software in combination with keyboard, video, and mouse over internet protocol (KVM-over-IP) capabilities to plug into the touch-screen interfaces, visual interfaces and keyboard interfaces that we want to test.”

“The non-invasive aspect was important to us because we wanted to ensure that there was never anything resident on the system, and that there was no possible interference. This also eliminates potential challenges around process loads which could present complications when looking at software testing results.”

Benefits and results

In terms of outcomes, the repeatability and the automation provided by Eggplant have been key benefits for Watson and his team. “If we need to, we can simply keep repeating and analyzing, with the software presenting its search-and-compare results without us having to employ additional manual testers to repeat tests. It’s important that we have cost-efficient and scalable testing methods. Eggplant enables both.”

Eggplant has also led NATS to consider new approaches to software testing. “With Eggplant we’ve gone down software testing avenues that we’ve never had the ability to explore before due to manpower. It allows us to conduct long-term Simple Object Access Protocol (SOAP) testing for specific changes caused by external influences that we have no control over within the business. It also allows us to test things in ways we’ve never been able to before, as it allows us to stand back and think about how we want to do testing. The very act of planning scripts has made us consider our approach. This has led to some very small but very important changes in the HMI layout which makes user interaction better and flow better. In that respect, Eggplant has helped with the design of the system. It may seem a strange benefit but that’s what has happened.”

Watson continues: “We can conduct long-term testing which is something we otherwise just can’t do without having to employ hundreds of people, it’s an enormous benefit.”

Working with Eggplant

NATS found the implementation of Eggplant to be very straightforward. “I thought the biggest challenge would be how to get Eggplant to interface with the system,” says Watson. “But as soon as we saw it could work remotely via the KVM approach, I knew it would be really easy, and it was.”

How about working with Eggplant? “As a supplier, they are very open with us and we’re looking at how we can expand its use across applications and other uses. The tool does exactly what we need, and the company is there to support us and our needs.”

All images are copyright NAS and used with kind permission of NATS Press Office.