How Poisoned Data Can Trick AI − and How to Stop It
A famous example of data poisoning in the field of computer science came in 2016, when Microsoft debuted a chatbot known as Tay. Within hours of its public release, malicious users online began feeding the bot reams of inappropriate comments. Tay soon began parroting the same inappropriate terms as users on X (then Twitter), and horrifying millions of onlookers. Within 24 hours, Microsoft had disabled the tool and issued a public apology soon after.
The social media data poisoning of the Microsoft Tay model underlines the vast distance that lies between artificial and actual human intelligence. It also highlights the degree to which data poisoning can make or break a technology and its intended use.
Data poisoning might not be entirely preventable. But there are commonsense measures that can help guard against it, such as placing limits on data processing volume and vetting data inputs against a strict checklist to keep control of the training process. Mechanisms that can help to detect poisonous attacks before they become too powerful are also critical for reducing their effects.
Fighting Back with the Blockchain
At Florida International University’s solid lab, we are working to defend against data poisoning attacks by focusing on decentralized approaches to building technology. One such approach, known as federated learning, allows AI models to learn from decentralized data sources without collecting raw data in one place. Centralized systems have a single point of failure vulnerability, but decentralized ones cannot be brought down by way of a single target.
Federated learning offers a valuable layer of protection, because poisoned data from one device doesn’t immediately affect the model as a whole. However, damage can still occur if the process the model uses to aggregate data is compromised.
This is where another more popular potential solution – blockchain – comes into play. A blockchain is a shared, unalterable digital ledger for recording transactions and tracking assets. Blockchains provide secure and transparent records of how data and updates to AI models are shared and verified.
By using automated consensus mechanisms, AI systems with blockchain-protected training can validate updates more reliably and help identify the kinds of anomalies that sometimes indicate data poisoning before it spreads.
Blockchains also have a time-stamped structure that allows practitioners to trace poisoned inputs back to their origins, making it easier to reverse damage and strengthen future defenses. Blockchains are also interoperable – in other words, they can “talk” to each other. This means that if one network detects a poisoned data pattern, it can send a warning to others.
At solid lab, we have built a new tool that leverages both federated learning and blockchain as a bulwark against data poisoning. Other solutions are coming from researchers who are using prescreening filters to vet data before it reaches the training process, or simply training their machine learning systems to be extra sensitive to potential cyberattacks.
Ultimately, AI systems that rely on data from the real world will always be vulnerable to manipulation. Whether it’s a red laser pointer or misleading social media content, the threat is real. Using defense tools such as federated learning and blockchain can help researchers and developers build more resilient, accountable AI systems that can detect when they’re being deceived and alert system administrators to intervene.
M. Hadi Amini is Associate Professor of Computing and Information Sciences, Florida International University.
This article is published courtesy of The Conversation.
