Software helps detect nuclear tests

The Vertically Integrated Seismic Analysis (VISA) project began in 2007, when the CTBTO was looking for an upgrade for its software system. The older software was making lots of mistakes, Sudderth said. It was wasting analysts’ time with false positives. But most critically, it was missing lots of smaller events and making errors in triangulating the exact position of events. “It was the job of the analysts to clean up the results of the automated system to some acceptable level of accuracy,” Sudderth said. For him and his colleagues, the raw automated data, combined with data that had been cleaned up by experts, was a goldmine.

“This is what we in machine learning think of as training data,” he said. “We can look at an event that we have reasonable confidence was real and look at its relationship to what the station actually measured. What do the errors look like? What are the biases? We use this historical data to get calibrated models of these things.”

Out of that came a system that models the multiple layers of uncertainty that occur in the processes of generating seismic events and in detecting them. Seismic waves travel in different ways through different types of rock. That introduces uncertainty because there is no high-resolution map of rock types across the entire surface of the Earth. On the detection side, each sensor works a little differently, and all of them are subject to many types of random noise — from wave activity in marine sensors to vehicle traffic on land.

Combining these statistical analyses leads to a comprehensive generative model of the rates at which seismic events of various sizes occur in various locations and the ways that energy from these events propagates to seismic sensors. Sudderth and his team devised an efficient inference algorithm that can scan incoming data to find events that likely represent an actual seismic signal.

VISA has been up and running at the CTBTO’s headquarters in Vienna for the last four years or so. It currently runs as a kind of backstop for the organization’s original automated system, and the CTBTO has asked the member states to approve it officially as a replacement.

“As it is now, the analysts have kind of a VISA button [on their computers],” Sudderth said. “They can press the VISA button and it says, ‘Here’s a bunch of stuff we think you missed.’” The analysts can then decide whether or not those events should be included in the daily seismic bulletin.

Brown U notes that Sudderth and his colleagues have shown that VISA can reduce the number of missed events by 60 percent compared to the original system. It can also provide more accurate location information in many cases. For example, VISA did a better job than the older system in pinpointing the location of a prior North Korean nuclear test in 2013, Sudderth said. He and his colleagues at the University of California recently published a paper on their work in the Bulletin of the Seismological Society of America. The International Society for Bayesian Analysis awarded this paper the Mitchell Prize, which recognizes an outstanding paper that uses Bayesian analysis to solve an important applied problem.

Sudderth was able to confirm that VISA did detect the recent North Korean test, but that blast was large enough that most traditional systems would have caught it as well.

“If all you cared about was finding events in North Korea, there are simpler, more targeted things you could do,” Sudderth said. “Where this system would potentially be a lot more powerful would be catching someone trying to do a clandestine test of a smaller device somewhere that you don’t know is a test site.”

Sudderth hopes VISA’s ability to detect more events might eventually aid in gaining full ratification of the Comprehensive Nuclear-Test-Ban Treaty. To be enforced by international law, the world’s forty-four nuclear nations must ratify the treaty. Eight of those nations, including the United States, have yet to do so. President Clinton signed the treaty in 1996, but Congress refused to ratify it.

“One of the things that was raised as a concern about ratifying it was the difficulty in verification,” Sudderth said. “If the technical systems for validation aren’t good enough, then countries aren’t going to be willing to ratify. So that’s one thing this work is trying to address and remove as an obstacle.”

— Read more in Nimar S. Arora et al., “NET-VISA: Network Processing Vertically Integrated Seismic Analysis,” Bulletin of the Seismological Society of America 103, no. 2a (April 2013): 709-29 (doi: 10.1785/0120120107); and Nimar S. Arora et al., “Global Seismic Monitoring: A Bayesian Approach,” Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence (2011)