Big dataManaging the data deluge for national security analysts

Published 1 December 2015

In this information age, national security analysts often find themselves searching for a needle in a haystack. The available data is growing much faster than analysts’ ability to observe and process it. Sometimes they cannot make key connections and often they are overwhelmed struggling to use data for predictions and forensics. A Sandia National Laboratories’ team has made a number of breakthroughs that could help solve these problems.

After a disaster or national tragedy, bits of information often are found afterward among vast amounts of available data that might have mitigated or even prevented what happened, had they been recognized ahead of time.

In this information age, national security analysts often find themselves searching for a needle in a haystack. The available data is growing much faster than analysts’ ability to observe and process it. Sometimes they cannot make key connections and often they are overwhelmed struggling to use data for predictions and forensics.

Sandia National Laboratories’ Pattern Analytics to Support High-Performance Exploitation and Reasoning (PANTHER) team has made a number of breakthroughs that could help solve these problems. Sandia Lab says that they are developing solutions that will enable analysts to work smarter, faster and more effectively when looking at huge, complex amounts of data in real-time, stressful environments where the consequences might be life or death.

PANTHER’s accomplishments include rethinking how to compare motion and trajectories; developing software that can represent remote sensor images, couple them with additional information and present them in a searchable form; and conducting fundamental research on visual cognition, said Kristina Czuchlewski, PANTHER’s principal investigator and manager of Sandia’s Intelligence Surveillance and Reconnaissance Systems Engineering and Decision Support.

The PANTHER team looked at raw data and ways to pre-process and analyze it to make it searchable and more meaningful. The project’s fundamental research in cognitive science will inform the design of software and tools to help those viewing the data and make information of interest or trends easier to uncover.

PANTHER, which was funded by Sandia’s Laboratory Directed Research & Development program, is gleaning deeper insights from complex data sets in minutes instead of months, and covering hundreds of square miles instead of dozens.

PANTHER developed the foundation for transforming how massive, complex data sets can be quickly analyzed to provide the nation’s decision-makers with new perspectives on situations and circumstances,” said Anthony Medina, director of Sandia’s Radio Frequency & Electronic Systems Center. “If an analyst is collecting information on a specific location over time and learns that something of interest might be occurring there, they probably don’t have the tools they need to quickly gather and analyze information from all relevant data sets that might corroborate the forecast. But PANTHER is probably the nation’s best bet right now to get to that point quickly.”