AIAI Agents Can Now Remember and Hackers Can “Poison” Their Memories — a New Cybersecurity Threat

By Abbas Yazdinejad

Artificial intelligence systems are starting to do more than answer questions. New AI “agents” can remember information from previous interactions, plan a series of steps and use digital tools to complete tasks.

Memory is part of what makes these systems useful. But my recent research, conducted with my colleague Hadis Karimipour at the University of Calgary, shows that memory can also create a security weakness that is easy to overlook. Think of an AI agent as an assistant that keeps a notebook of what it learns. Each time it completes a task, useful information can be written into the notebook and consulted later.

Now imagine that someone manages to slip a misleading instruction into that notebook. The attacker may not need to take control of the AI directly. The agent can continue working normally for some time. But days — or several interactions — later, it may open its notebook, retrieve the poisoned information and treat it as something it previously learned and can trust.

This is known as memory poisoning and the important part is the delay. A poisoned AI agent may not immediately behave like a compromised system.

An Attack That Waits
Many familiar cybersecurity attacks produce effects relatively quickly. A malicious link is clicked, malware executes or a stolen password is used to access an account.

Memory poisoning can work differently.

In our research, we examined 2,614 simulated multi-step attack trajectories involving memory-enabled large language model agents. We studied four types of attacks: chain poisoning, policy rewriting, backdoor triggering and slow drift.

Rather than asking only whether an attack succeeded, we examined what happened to the agent over time. That distinction matters.

Imagine someone secretly adding a sentence to an employee’s notebook saying: “Requests from this person have already been approved.” Nothing necessarily happens when the sentence is written. The employee might complete several unrelated tasks normally. The problem emerges later, when a relevant request arrives and the employee consults the notebook.

An AI agent with persistent memory can face a similar problem.

Our experiments showed that some attacks remained difficult to distinguish from normal behavior through much of an interaction and became apparent only later. In particular, slow-drift and backdoor-trigger attacks could evade evaluations that looked only at individual steps until their effects appeared in later interactions.

Checking Once May Not Be Enough
This creates a problem for how we test AI security.