AIExplainable AI: A Must for Nuclear Nonproliferation, National Security

Published 10 March 2021

As it is with raw human intelligence, so it is with artificial intelligence (AI). We may not know exactly what’s going on inside that elaborate black box built by humans, but its decisions can be so accurate that it earns our trust, if not our comprehension. But the need for understanding escalates when the stakes are higher. For national security concerns, it’s not good enough to know that a system works; scientists demand to know how and why. That’s the foundation for a field of study known as “explainable AI.”

We’ve all met people so smart and informed that we don’t understand what they’re talking about. The investment advisor discussing derivatives, the physician elaborating about B cells and T cells, the auto mechanic talking about today’s computerized engines—we trust their decisions, even though we do not completely grasp the meaning of their words.

As it is with raw human intelligence, so it is with artificial intelligence (AI). We may not know exactly what’s going on inside that elaborate black box built by humans, but its decisions can be so accurate that it earns our trust, if not our comprehension.

That’s no problem when the decisions are of little consequence. Do we really need to understand, for instance, the inner workings of an AI system that sorts hundreds of photos of cats and dogs flawlessly in the time it takes to say the words “cats and dogs”?

Probably not. Human lives and the fate of nations do not rest on those decisions.

But the need for understanding escalates when the stakes are higher. For national security concerns under study at Pacific Northwest National Laboratory (PNNL), it’s not good enough to know that a system works; scientists demand to know how and why.

Explainable AI: A Path to Understanding
That’s the foundation for a field of study known as “explainable AI.” The goal is to understand and explain the reasoning of the system—to untangle the threads of information that an AI system uses to make choices or recommendations.

“In the national security space, decisions are made by people who demand transparency with the technologies they are working with,” saidAngie Sheffield, a senior program manager with the Department of Energy’s National Nuclear Security Administration (NNSA).

Sheffield manages the data science portfolio in NNSA’s Office of Defense Nuclear Nonproliferation Research and Development, also known as DNN R&D. The office oversees and improves the nation’s ability to detect and monitor nuclear material production and movement, weapons development, and nuclear detonations across the globe. The office is supporting work by a team of PNNL scientists who are exploring how to make AI explainable in new ways.

AI is everywhere these days, from drug design to online purchasing, reservation systems, and health risk checklists. The national security realm, with vast data analysis challenges and powerful computing capabilities, is no exception. The stakes are exceedingly high when it comes to nuclear nonproliferation issues, and knowing exactly how an AI system reaches its conclusions is crucial.