AIResearchers Unveil First-Ever Defense Against Cryptanalytic Attacks on AI

By Matt Shipman

Security researchers have developed the first functional defense mechanism capable of protecting against “cryptanalytic” attacks used to “steal” the model parameters that define how an AI system works.

“AI systems are valuable intellectual property, and cryptanalytic parameter extraction attacks are the most efficient, effective, and accurate way to ‘steal’ that intellectual property,” says Ashley Kurian, first author of a paper on the work and a Ph.D. student at North Carolina State University. “Until now, there has been no way to defend against those attacks. Our technique effectively protects against these attacks.”

“Cryptanalytic attacks are already happening, and they’re becoming more frequent and more efficient,” says Aydin Aysu, corresponding author of the paper and an associate professor of electrical and computer engineering at NC State. “We need to implement defense mechanisms now, because implementing them after an AI model’s parameters have been extracted is too late.”

At issue are cryptanalytic parameter extraction attacks. Parameters are the essential information used to describe an AI model. Essentially, parameters are how AI systems perform tasks. Cryptanalytic parameter extraction attacks are a purely mathematical way of determining what a given AI model’s parameters are, allowing a third party to recreate the AI system.

“In a cryptanalytic attack, someone submits inputs and looks at outputs,” Aysu says. “They then use a mathematical function to determine what the parameters are. So far, these attacks have only worked against a type of AI model called a neural network. However, many – if not most – commercial AI systems are neural networks, including large language models such as ChatGPT.”

So, how do you defend against a mathematical attack?

The new defense mechanism relies on a key insight the researchers had regarding cryptanalytic parameter extraction attacks. While analyzing these attacks, the researchers identified a core principle that every attack relied on. To understand what they learned, you have to understand the basic architecture of a neural network.

The fundamental building block of a neural network model is called a “neuron.” Neurons are arranged in layers and are used in sequence to assess and respond to input data. Once the data has been processed by the neurons in the first layer, the outputs of that layer are passed to a second layer. This process continues until the data has been processed by the entire system, at which point the system determines how to respond to the input data.