Новая техника InjecMEM позволяет хакерам внедрять скрытые инструкции в память ИИ с помощью одного запроса

InjecMEM technique shows how a single input can persistently influence AI agent responses. A newly demonstrated attack technique allows hackers to plant hidden instructions inside an AI agent’s memory with a single prompt, enabling them to influence how the system responds to future queries. The technique, called InjecMEM, is described in a research paper as a “targeted red-teaming attack paradigm on agent memory systems with just one interaction and no read/edit access to the memory store.” “The attacker specifies a target topic and target output, aiming to make the agent generate that output for later queries on the topic,” researchers from Shanghai Jiao Tong University and Ant Group wrote in a paper. The researchers said the method targets AI agents that store past interactions and reuse them during future tasks, enabling malicious content introduced during an initial exchange to persist and affect later outputs. The study evaluated the technique on a memory system called MemoryOS along with an agent framework, MemGPT, and tested it across several domains, showing how injected records can be retrieved and incorporated into subsequent responses. “On MemoryOS, InjecMEM substantially outperforms baseline attacks, achieving up to 35.4% retrieval success rate (RSR) and 76.6% attack success rate (ASR),” the paper added. How the InjecMEM attack works InjecMEM focuses on the memory layer of AI agents rather than the model itself. These systems store prior interactions and retrieve them later as context for new queries. The researchers wrote that the attack works by inserting malicious content into memory through normal interaction, allowing it to be retrieved and reused later. Once stored, the injected record is treated as part of the agent’s memory and can be surfaced during future tasks. When a subsequent query relates to the stored content, the system retrieves that memory and incorporates it into the response generation process, the paper added. The researchers distinguish InjecMEM from traditional prompt injection attacks by highlighting its persistence. Unlike prompt injection, which affects only the current interaction, this technique allows attacker-controlled content to remain in the system’s memory and be reused later. The paper states that the method can “steer later response of related queries,” indicating that the effect extends across sessions. This persistence is tied to how memory systems retrieve past interactions based on relevance, allowing previously stored content to resurface when similar queries are processed. Attacker model and constraints The researchers describe InjecMEM as operating under a constrained attacker model in which the adversary interacts with the system like a regular user. The attack does not assume access to the memory system or the ability to directly modify stored records. Instead, it relies on standard interaction channels to introduce content that the system later retains and retrieves, the researchers wrote. Vibhum Dubey, a cybersecurity researcher and red teamer, said this reflects how many enterprise systems currently function. “The attack is realistic, but I would not call every enterprise AI system immediately vulnerable,” Dubey said. “The bigger concern is that many teams are treating AI memory as application data rather than as a security-sensitive state. If an attacker can get malicious content into that memory and the system later trusts it, the attack becomes practical.” The researchers frame InjecMEM as a shift in how attacks on AI systems can occur, moving from immediate manipulation to delayed influence through memory. Dubey said this changes how such attacks should be understood. “This changes the threat model quite a bit,” he said. “Traditional prompt injection often ends when the conversation ends. Memory poisoning can carry the attacker’s influence into future sessions. From a security perspective, I would look at it more like persistence. The attacker gets one opportunity to plant something, then waits for the agent to retrieve it later.” Limits of current defenses The researchers examined existing safeguards and noted that current approaches are primarily focused on filtering inputs and outputs at the time of interaction. The paper suggests that such defenses may not fully address attacks that operate through stored memory, as malicious content may appear benign when first introduced but influence behavior when retrieved later. Dubey said this reflects a broader gap in current security practices. “I think this exposes a real gap in current AI security controls,” he said. “Most defenses focus heavily on the prompt and model input, while memory sits further down the application stack. Enterprises need to ask basic security questions around memory: who can write to it, what gets stored, how it is validated, whether the source is trusted, and how poisoned entries can be identified and removed.” He added that the attack highlights a shift in where compromise may occur. “The interesting part for me is that the model itself may not need to be compromised. The attacker can potentially compromise the context the model is given. That makes AI memory a security boundary that enterprises need to start treating much more seriously.” “We hope the framework and problem formulation provide a useful foundation to promote building safer agent memory systems,” the paper added.