Software securityEnabling more comprehensive tests on high-risk software

Published 25 April 2019

We entrust our lives to software every time we step aboard a high-tech aircraft or modern car. A long-term research effort has developed new tools to make this type of safety-critical software even safer.

We entrust our lives to software every time we step aboard a high-tech aircraft or modern car. A long-term research effort guided by two researchers at the National Institute of Standards and Technology (NIST) and their collaborators has developed new tools to make this type of safety-critical software even safer.

Augmenting an existing software toolkit, the research team’s new creation can strengthen the safety tests that software companies conduct on the programs that help control our vehicles, operate our power plants and manage other demanding technology. While these tests are often costly and time-consuming, they reduce the likelihood this complex code will glitch because it received some unexpected combination of input data. This source of trouble can plague any sophisticated software package that must reliably monitor and respond to multiple streams of data flowing in from sensors and human operators at every moment.

With the research toolkit called Automated Combinatorial Testing for Software, or ACTS, software companies can make sure that there are no simultaneous input combinations that might inadvertently cause a dangerous error. As a rough parallel, think of a keyboard shortcut, such as pressing CTRL-ALT-DELETE to reset a system intentionally. The risk with safety-critical software is that combinations that create unintentional consequences might exist.

NIST says that until now, there was no way to be certain that all the significant combinations in very large systems had been tested: a risky situation. Now, with the help of advances made by the research team, even software that has thousands of input variables, each one of which can have a range of values, can be tested thoroughly.

NIST’s ACTS toolkit now includes an updated version of Combinatorial Coverage Measurement (CCM), a tool that should help improve safety as well as reduce software costs. The software industry often spends seven to 20 times as much money rendering safety-critical software reliable as it does on more conventional code.

The peer-reviewed findings of the research team appear in two papers the team will present on April 23 at the 2019 IEEE International Conference on Software Testing, Verification and Validation in Xi’an, China.