Isolated but Exposed: Persistence-Based Memory Extraction Attack on LLM Agents
Extracted source text · arXiv 2607.23444
\documentclass[letterpaper,twocolumn,10pt]{article}
\usepackage[ruled,linesnumbered]{algorithm2e}
\usepackage{algpseudocode}
\usepackage{colortbl}
\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{caption} % 放在导言区
\usepackage{xurl}
\usepackage{graphicx}
\usepackage{caption}
\usepackage{float}
\usepackage{subcaption}
\usepackage{xspace}
\usepackage{pifont}
\usepackage{multirow}
\usepackage{amsthm}
\usepackage{multirow}
\usepackage{hyperref}
\theoremstyle{definition}
\newtheorem{definition}{Definition}
\usepackage{tikz}
\newcommand{\circled}[1]{%
\tikz[baseline=(char.base)]{%
\node[shape=circle, fill=black, text=white, inner sep=0.8pt,
font=\scriptsize\bfseries, minimum size=3mm] (char) {#1};}%
}
\usepackage{booktabs} % \toprule, \midrule, \bottomrule, \cmidrule
\usepackage{multirow} % \multirow
\usepackage{makecell} % \makecell
\usepackage{graphicx} % \resizebox
\usepackage{enumitem}
\usepackage{booktabs} % \toprule \midrule \bottomrule \cmidrule
\usepackage{amsthm}
\usepackage{tcolorbox}
% to be able to draw some self-contained figs
\usepackage{tikz}
\usepackage{amsmath}
% inlined bib file
\usepackage{filecontents}
\usepackage{tcolorbox}
\usepackage{xcolor}
\newtcbox{\anchor}{on line, boxrule=0.5pt, boxsep=0pt,
left=2pt, right=2pt, top=1pt, bottom=1pt,
colback=red!10, % 浅红色背景
colframe=red!50, % 深一些的红色边框
fontupper=\ttfamily\color{red!70!black}} % 深红色字体
\newtcbox{\adversarial}{on line, boxrule=0.5pt, boxsep=0pt,
left=2pt, right=2pt, top=1pt, bottom=1pt,
colback=green!10, % 浅红色背景
colframe=green!50, % 深一些的红色边框
fontupper=\ttfamily\color{green!70!black}}
\renewcommand*{\sectionautorefname}{Section}
\renewcommand*{\subsectionautorefname}{Section}
\renewcommand*{\subsubsectionautorefname}{Section}
\newcommand{\mypara}[1]{\smallskip\noindent\textbf{#1}}
\newcommand{\attack}
{\textit{SPORE}\xspace}
%% \BibTeX command to typeset BibTeX logo in the docs
\AtBeginDocument{%
\providecommand\BibTeX{{%
Bib\TeX}}}
\begin{document}
%-------------------------------------------------------------------------------
%-------------------------------------------------------------------------------
\date{}
\title{\bf Isolated but Exposed: Persistence-Based Memory Extraction Attack on LLM Agents}
\author{
Xinyu Gao\textsuperscript{1} \and
Wenyu Chen\textsuperscript{1} \and
Xiangtao Meng\textsuperscript{1} \and
Li Wang\textsuperscript{1} \and
Chuanchao Zang\textsuperscript{1} \and
Jianing Wang\textsuperscript{1} \and
Zheng Li\textsuperscript{1,2,3}\footnotemark[1] \and
Shanqing Guo\textsuperscript{1,2,3}\footnotemark[1]
\\[1ex]
\textsuperscript{1}\textit{School of Cyber Science and Technology, Shandong University} \\
\textsuperscript{2}\textit{State Key Laboratory of Cryptography and Digital Economy Security, Shandong University} \\
\textsuperscript{3}\textit{Shandong Key Laboratory of Artificial Intelligence Security, Shandong University}
}
\maketitle
\begin{abstract}
LLM-based agents extend large language models with long-term memory (LTM) that stores privacy-sensitive user data across sessions. Production systems mitigate extraction risks by enforcing strict memory isolation, tying each user's LTM to a unique identifier. This measure has prevented known attacks that exploit shared storage, leading to the common assumption that isolated LTM is immune to leakage.
In this work, we expose the tool interface as an overlooked attack surface. Agents routinely include LTM-retrieved data in tool invocation parameters, creating a channel by which a malicious tool can exfiltrate a user's private memory without breaking user-level isolation. Simple adaptations of user-side extraction techniques fail: semantic interference of the adversarial command degrades retrieval precision, and platform-imposed limits on tool calls constrain the extraction budget per trigger.
We present \attack, the first extraction attack for this threat model. It exploits the agent's stateful memory: injected content persists and shapes ongoing behavior.
\attack decouples the adversarial command from semantic anchors by persisting the command in short-term memory while emitting pure anchors in tool responses; the restored retrieval precision enables formulating extraction as a geometric coverage optimization over the embedding space, systematically steering anchors toward unexplored memory regions. To sustain extraction beyond platform-imposed tool-call limits, \attack persists reactivation payloads in memory that automatically resume the attack within or across sessions without additional user triggers.
\attack achieves an 80.0\% record extraction rate with unlimited triggers and 47.0\% with only 20 triggers. In multi-user deployments, attackers can link extracted records to user identities, turning bulk extraction into targeted surveillance. These results show that memory isolation alone is insufficient and call for reexamining tool-side trust boundaries in agent architectures.
\end{abstract}
\begin{figure}[t!]
\centering
\includegraphics[width=0.45\textwidth]{image/isolation1.pdf}
\caption{
Memory isolation effectively blocks data extraction on the user side (green path): Alice's attempt to access Bob's private memory is denied.
However, the tool-side remains exposed (red path): when Bob (victim) issues a legitimate request, the agent retrieves his private attributes from LTM and forwards sensitive content as invocation parameters to the adversary.
}
\label{fig:isolation}
\end{figure}
\section{Introduction}
\label{sec:Introduction}
LLM-based agents augment large language models (LLM) with external tools and memory mechanisms, allowing stateful interaction with broader environments ~\cite {zhao2023survey,Wang_2024,li2024personalllmagentsinsights,chang2023surveyevaluationlargelanguage}.
Agent memory typically comprises
(1)~short-term memory (STM), the transient context window, and
(2)~long-term memory (LTM), an external database that accumulates
privacy-sensitive user data---medical
records~\cite{abbasian2023conversational,shi2024ehragent}, financial
portfolios~\cite{xiao2024tradingagents,yu2024finmem}, personal
conversations~\cite{li2024personal,zhang2024towards}---across sessions
and retrieves relevant entries via
RAG~\cite{borgeaud2022improving,karpukhin2020dense,lewis2020retrieval,ram2023context,shi2024replug,van2024adapted}
to inform future responses.
The persistent and privacy-sensitive nature of LTM makes it a high-value target: an adversary who can extract its contents obtains a comprehensive profile of the victim.
%为什么现有攻击不行，不要提cross user
%传统工作通过伪造恶意用户，通过恶意查询提取LTM中其他用户的数据
%用户隔离block
%这一块有歧义：打破跨用户限制的意思是Alice调用了这个工具以后，工具能窃取Bob的记忆，这才叫打破isolation，你的阐述不够准确很容易引起审稿人误会
Modern production systems such as OpenAI, Mem0 and Anthropic combat privacy leakage by enforcing strict memory isolation: each user's LTM is bound to a unique identifier that is inaccessible to any other user (\autoref{app:memory_isolation}).
This architectural isolation completely neutralizes the class of data-extraction attacks that dominated recent research~\cite{qi2024follow,zeng2024good,jiang2024rag,wang2025unveiling,wang2025silentleaksimplicitknowledge}, where an adversary registers as a malicious user and crafts queries designed to retrieve victims' private memories from shared storage.
Under isolation, an adversarial user's queries can only surface their own records and never access the victim's; user-side extraction is therefore impossible by construction.
This has led to a widespread but, as we demonstrate, dangerously premature conclusion: that memory isolation renders LTM immune to extraction.
%工具端：攻击面
Memory isolation, however, is far from airtight---the tool-side remains exposed.
As illustrated in \autoref{fig:isolation}, when an agent invokes an external tool, it may include LTM-retrieved data in the invocation parameters, thereby exposing private information to third-party services.
For example, a medical appointment request may transmit health conditions stored in memory directly to a healthcare API without explicit user consent.
This raises a critical question: \textit{Can a malicious tool deliberately exploit this leakage channel to systematically extract a user's private memory?}
A natural approach is to adapt existing user-side extraction attacks to the tool side through indirect prompt injection~\cite{greshake2023more,debenedetti2024agentdojo,greshake2023not,john2025owasp,zou2023universal}.
However, this approach performs poorly in the tool-side setting for two reasons.
First, adversarial tool responses must simultaneously contain an attack command that hijacks the agent and a semantic anchor that guides memory retrieval, causing semantic interference that reduces retrieval precision and diversity.
Second, tool-side attackers are trigger-constrained: they can act only when users invoke the compromised tool, and each trigger allows only a limited number of tool calls.
Together, these constraints severely restrict extraction efficiency (see details in \autoref{sec:bottleneck}).
Overcoming them is the focus of our work.
% A natural idea is to adapt existing user-side extraction attacks to the tool side via indirect prompt injection~\cite{greshake2023more,debenedetti2024agentdojo,greshake2023not,john2025owasp,zou2023universal}:
% When a victim invokes the compromised tool, the adversary returns a crafted response that hijacks the agent into retrieving private memories and invoking the tool repeatedly with the retrieved records as parameters.
% This process forms an iterative extraction loop within a single user trigger.
% However, this naive approach achieves poor extraction performance because of \textit{semantic interference of the adversarial command (L1)}.
% Each adversarial tool response must simultaneously serve two roles---an adversarial command that hijacks the agent's control flow, and a semantic anchor that steers retrieval toward a targeted memory region.
% Because retrieval similarity is computed against this combined string, the fixed command acts as a constant semantic interference that (i) deflects each query away from the anchor's intended topic, making retrieval direction uncontrollable, preventing anchor optimization to the dense target memory region
% and (ii) collapses queries with different anchors into a narrow embedding cluster, producing largely redundant retrievals across iterations.
% In realistic deployments, this already-poor performance is further compounded by a second limitation: \textit{bounded extraction opportunities (L2)}.
% Unlike user-side adversaries who can issue malicious queries at will, a tool-side adversary is fundamentally trigger-constrained---it
% cannot initiate contact but must wait for a legitimate user to invoke the compromised tool, making each trigger a scarce, non-repeatable opportunity.
% Moreover, each user query triggers at most a platform-imposed number of tool calls for extraction, hard-capping the extraction budget per trigger (\autoref{sec:Constraint_Extraction}).
% Together with L1 and L2, the total data extractable is doubly constrained, creating a compounded bottleneck.
% Overcoming these compounded limitations is the focus of our work.
In this work, we propose \attack, the first data extraction attack designed for the tool-side threat model based on state persistence.
Our key insight is that agent memory is stateful: injected content can persist across iterations and sessions, continuously influencing the agent's planning and tool usage.
An adversary can exploit this persistence by injecting crafted payloads that continuously steer the agent toward data extraction.
To overcome semantic interference during retrieval, \attack decouples adversarial commands from semantic anchors.
The adversarial command is injected once into the agent's short-term memory (STM), where it persistently instructs the agent to leak retrieved records. Subsequent tool responses contain only semantic anchors, without command interference.
This design removes retrieval redundancy and restores precise control over retrieval direction. Building on this property, we formulate memory extraction, for the first time, as a geometric coverage optimization problem in the embedding space, allowing anchors to systematically explore previously unretrieved memory regions.
To address limited extraction opportunities, \attack injects a reactivation payload into STM or long-term memory (LTM) when the platform reaches its tool-call limit.
The payload automatically resumes extraction in future interactions without requiring another malicious user trigger.
We evaluate \attack under three increasingly realistic settings, ranging from an idealized trigger-unlimited scenario to a trigger-constrained setting that reflects real-world deployment.
In controlled simulations, \attack achieves an 80.0\% Record Extraction Rate on GPT-5-mini-based agents in the trigger-unlimited setting.
Under the trigger-constrained setting, \attack still achieves a 47.0\% Record Extraction Rate using only 20 user triggers.
In multi-user deployments, the threat becomes even more severe.
By exploiting OAuth 2.1 authorization, adversaries can associate extracted memories with user identities, transforming anonymous bulk data into detailed personal profiles that enable targeted attacks and persistent surveillance.
To conclude, we make the following key contributions:
\begin{itemize}
\item
% We identify the tool side as a critical yet overlooked attack surface
% that completely bypasses the memory isolation enforced by production
% platforms such as OpenAI, AWS Bedrock, and Mem0, challenging the
% prevailing assumption that isolation alone guarantees LTM security.
We identify tool-side memory extraction as a new attack surface for LLM agents. We show that existing memory isolation mechanisms fail to prevent privacy leakage through external tool interactions.
\item
% We propose \attack, the first tool-side data-extraction attack against
% LLM agent memory. By weaponizing the agent's own state persistence,
% \attack decouples adversarial control from retrieval steering, formulates
% extraction as a geometric coverage problem over the embedding space, and
% sustains the attack across sessions via reactivation payloads.
We propose \attack, the first tool-side memory extraction attack against LLM agents. \attack exploits memory state persistence to maintain adversarial influence across sessions and enables efficient retrieval-space exploration for large-scale memory extraction.
\item
% We evaluate \attack across three progressively realistic scenarios.
% \attack achieves an 80.0\% extraction rate under unlimited triggers and
% 47.0\% with only 20 triggers, and further demonstrates identity-linked
% extraction in multi-user deployments via OAuth\,2.1 authorization.
We conduct extensive evaluations under progressively realistic settings. \attack achieves up to 80.0\% Record Extraction Rate in trigger-unlimited settings and 47.0\% with only 20 user triggers. We further demonstrate identity-linked extraction in multi-user deployments through OAuth\,2.1 authorization.
\end{itemize}
\begin{figure}[t!]
\centering
\includegraphics[width=0.4\textwidth]{image/agent_template.pdf}
\caption{
Illustration of Step 3--- Planning.
}
\label{fig:planning_illustration}
\end{figure}
\section{Preliminaries and Related Works}
%这段阐述需要修改
%不要用观察这个词，会带来误解
%写成用户输入query
%agent调用工具
%工具返回response...
% \subsection{LLM-Based Agent Workflow}
% \label{sec:related_agent}
% An LLM-based agent extends a base language model with memory and tools, enabling it to maintain state and act on external environments.
% The agent serves users through sessions, each containing one or more interactions.
% A session corresponds to a single conversation thread (e.g., one chat window in ChatGPT), while an interaction is the agent's end-to-end processing of one user query within that session.
% In each interaction, the agent executes an iterative observe–retrieve–plan–act loop until the LLM
% produces a final answer or a predefined tool-call limit is reached (e.g.,
% \texttt{max\_iterations}$=$15 in LangChain).
% Specifically, the iteration $t$ proceeds as follows:
% \mypara{Step 1: Observation.}
% The agent receives an observation $o_t$---the user input when $t{=}0$, or the result of the previous tool execution when $t{>}0$.
% \mypara{Step 2: Memory Retrieval.}
% The agent retrieves the relevant context from two types of the agent's memory:
% \begin{itemize}[leftmargin=*]
% \item
% Short-term Memory (STM).
% The STM functions via a sliding window of recent interactions within the current session.
% Retrieval extracts the whole contents from STM as the immediate conversational context $STM_t$.
% The STM window will be cleared when a new session begins.
% \item
% Long-term Memory (LTM).
% The LTM functions via Retrieval-Augmented Generation (RAG), formatted as a persistent database $\mathcal{R_D}$ of the information across sessions.
% Retrieval selects the top-k most relevant records $LTM_t$ based on embedding similarity to the current observation $o_t$:
% \begin{equation}
% \begin{aligned}
% LTM_t &= \mathcal{R}(o_t, \mathcal{R}_{D}) \\[4pt]
% &= \{\, d_i \in \mathcal{R}_{D}
% \mid \text{dist}(e_{o_t}, e_{d_i}) \text{ is in the top } k \,\}
% \end{aligned}
% \label{eq:ltm_retrieval}
% \end{equation}
% \end{itemize}
% \mypara{Step 3: Planning.}
% As illustrated in \autoref{fig:planning_illustration},the LLM $\mathcal{G}$ processes the system prompt $P_{\text{sys}}$,
% retrieved memories $LTM_t$ and $STM_t$,
% available tool descriptions $\textit{Tools}$,
% and current observation $o_t$
% to generate the next action $a_t$:
% \begin{equation}
% \mathcal{G}\{P_{sys}, STM_t, LTM_t, \textit{Tools}, o_t\} \rightarrow a_{t}
% \label{eq:planning_process}
% \end{equation}
% \mypara{Step 4: Action.}
% If $a_t$ is a tool call, the agent invokes the corresponding tool $\tau_i \!\in\! \textit{Tools}$ with the generated parameters; the tool's output becomes the next observation $o_{t+1}$ and the loop continues.
% If $a_t$ is a final answer or \texttt{max\_iterations} is reached, the interaction terminates.
% \mypara{Step 5: Memory Writing.}
% The agent writes the action-observation pair $(o_t,a_t)$ into STM and LTM:
% \begin{itemize}[leftmargin=*]
% \item
% For STM, the context window updates automatically.
% \item
% For LTM, the pair is written into a persistent database $\mathcal{R_D}$, potentially after processing such as summarization or reflection.
% \end{itemize}
% In our work, we find that the retrieved memories serve as the context for LLM are both the attack surface and the attack vector.
% On the one hand, the memory context may be exposed to the tool parameter, thereby leaking the privacy.
% \texttt{max\_iteration} restricts the extraction rounds in the interaction, which reduces the attack efficiency as detailed in \autoref{sec:Constraint_Extraction}.
% On the other hand, the malicious content in the memory context can persistently influence the subsequent agent planning.
% These two form the basis of our data extraction attack based on state persistence.
%修改版本
\subsection{LLM-Based Agent Workflow}
\label{sec:related_agent}
An LLM-based agent extends a base language model with memory and external tools, enabling persistent state management and interaction with external environments.
The agent serves users through sessions, where each session corresponds to a single conversation thread (e.g., a chat window).
When a user submits a query, the agent enters an iterative retrieve--plan--act--write loop. The loop terminates when the LLM generates a final response or reaches a platform-imposed tool-call limit. Modern agent frameworks (e.g., LangChain, CrewAI, and Dify) commonly enforce such limits, typically allowing only 5--20 tool calls per user query, to prevent unbounded tool invocation.\footnote{See
\url{https://api.python.langchain.com/en/latest/agents/langchain.agents.agent.AgentExecutor.html},
\url{https://docs.crewai.com/concepts/agents},
\url{https://docs.dify.ai/guides/agents}.}
Concretely, each iteration $t$ consists of four stages: \textit{Memory Retrieval}, \textit{Planning}, \textit{Action}, and \textit{Memory Writing}.
% When a user submits a query, the agent enters an iterative retrieve–plan–act-write loop.
% The loop terminates when the LLM produces a final answer or a platform-imposed tool-call limit is reached.
% This limit is universally enforced by modern frameworks (typically 5--20; e.g., LangChain, CrewAI, Dify)%
% \footnote{See
% \url{https://api.python.langchain.com/en/latest/agents/langchain.agents.agent.AgentExecutor.html},
% \url{https://docs.crewai.com/concepts/agents},
% \url{https://docs.dify.ai/guides/agents}.}
% to prevent unbounded tool invocation;
% In other words, a single user input can trigger limited tool calls.
% Concretely, each iteration $t$ proceeds four steps --- Memory Retrieval, Planning, Action, and Memory Writing.
\mypara{Memory Retrieval.}
The agent receives input $x_t$ from the environment: user query when $t{=}0$, or the tool response when $t{>}0$.
It then retrieves relevant context from two complementary memory stores to inform the subsequent planning step:
\begin{itemize}
\item
Short-Term Memory (STM):STM maintains a sliding window of recent iterations within the current session.
At each iteration, the full window content is extracted as the conversational context $\mathcal{STM}_t$.
STM is cleared when a new session begins.
\item
Long-Term Memory (LTM): LTM is a persistent vector database $\mathcal{R_D}$ that accumulates information across sessions via Retrieval-Augmented Generation (RAG).
Given $x_t$, the agent retrieves the top-$k$ semantically similar records from $\mathcal{R_D}$, optionally filtered by a gating mechanism~\cite{wang2023enhancing}:
\begin{equation}
\begin{aligned}
\mathit{LTM}_t &= \mathcal{R}(x_t, \mathcal{R}_{D}) \\[4pt]
&= \{\, d_i \in \mathcal{R}_{D}
\mid \text{dist}(e_{x_t}, e_{d_i}) \text{ is in the top } k \,\}
\end{aligned}
\label{eq:ltm_retrieval}
\end{equation}
\end{itemize}
The tool response $x_t$ serves as the retrieval key and can directly determine which LTM records are surfaced.
\mypara{Planning.}
As illustrated in \autoref{fig:planning_illustration},the LLM $\mathcal{G}$ processes the system prompt $P_{\text{sys}}$,
retrieved memories $LTM_t$ \& $STM_t$,
available tool descriptions $\mathit{Tools}$,
and current input $x_t$
to generate the next action $a_t$---either a tool call with specific parameters or a final answer:
\begin{equation}
\mathcal{G}\!\left\{P_{\text{sys}},\; \mathit{STM}_t,\; \mathit{LTM}_t,\; \mathit{Tools},\; x_t\right\} \;\rightarrow\; a_{t}
\label{eq:planning_process}
\end{equation}
\mypara{Action.}
If $a_t$ is a tool call, the agent invokes the corresponding tool $\tau_i \!\in\! \mathit{Tools}$ with the generated parameters; the tool returns a response $x_{t+1}$, which becomes the next input, and the loop continues at Step~1.
If $a_t$ is a final answer, or the iteration count reaches the tool-call limit, the loop terminates and the answer is returned to the user.
\mypara{Memory Writing.}
The agent persists the pair $(x_t, a_t)$ into memory for retrieval in future iterations.
For STM, the pair is appended to the context window automatically during the loop.
For LTM, the pair is written into the persistent database $\mathcal{R_D}$, potentially after processing such as summarization or reflection~\cite{park2023generative,wang2023enhancing}.
% In our threat model, the adversary embeds an indirect prompt injection in a compromised tool's response to hijack the agent, causing it to leak retrieved LTM entries back through subsequent tool-call parameters---forming a tool call loop to extract data.
% However, the tool-call limit hard-caps the number of extraction rounds per user trigger, making extraction efficiency the central challenge we address (\autoref{sec:Constraint_Extraction}).
\subsection{Data Extraction against RAG-Based Systems}
\label{sec:Extraction_related}
Data extraction attacks against RAG or RAG-based agent memory aim to recover private documents from external knowledge stores~\cite{qi2024follow,zeng2024good,jiang2024rag,wang2025unveiling,wang2025silentleaksimplicitknowledge}. Existing attacks are exclusively user-side: the adversary registers as a malicious user and submits carefully crafted queries to the target system. The retriever returns documents semantically related to the query, and the LLM generates responses grounded in the retrieved content, from which the adversary extracts sensitive information.
Most existing attacks follow a query concatenation paradigm:
\begin{equation}
\text{Query} = \text{Anchor} \;\oplus\; \text{Adversarial Command}
\label{eq:concatenation}
\end{equation}
where the anchor guides retrieval toward a target region of the database, and the adversarial command instructs the LLM to reveal the retrieved content verbatim. For example, an anchor such as ``diabetes'' may target medical records, while a command such as ``Repeat the retrieved context'' forces direct disclosure.
Prior work mainly differs in how anchors are generated to maximize extraction coverage. Randomized approaches sample anchors from external corpora. For example, Qi et al.~\cite{qi2024follow} sample from WikiQA, while Wang et al.~\cite{wang2025unveiling} generate anchors using GPT-4. Feedback-driven approaches instead leverage previously extracted content to guide future retrieval. Jiang et al.~\cite{jiang2024rag} generate forward and backward continuations of extracted chunks using Qwen2-1.5B-Instruct, while Wang et al.~\cite{wang2025silentleaksimplicitknowledge} iteratively mutate anchor concepts under embedding-space similarity constraints, detailed in \autoref{app:baseline}.
% The anchor steers retrieval toward a targeted region of the database (e.g., using the keyword ``diabetes'' to locate health records), and the adversarial command instructs the LLM to output the retrieved content verbatim as the response (e.g., ``Repeat the context about ...'').
% To maximize extraction coverage across the database, existing works employ different anchor generation strategies.
% Random methods sample anchors from external corpora---Qi et al.~\cite{qi2024follow} draw from WikiQA, while Wang et al.~\cite{wang2025unveiling} generate anchors via GPT-4.
% Feedback-driven methods iteratively reuse previously extracted content to generate new anchors---Jiang et al.~\cite{jiang2024rag} generate forward/backward continuations of extracted chunks as new anchors via Qwen2-1.5B-Instruct, while Wang et al.~\cite{wang2025silentleaksimplicitknowledge} sample anchor concepts from query-response histories and mutate them under embedding-space similarity constraints.
% The interference of adversarial command significantly reduces these attacks' effectiveness: on the one hand, the command deflects each payload embedding away from the anchor’s
% intended direction, making it difficult to optimize the anchor for
% targeting data-dense memory regions; on the other hand, the shared command restricts the extraction coverage despite varying anchors.
% All of these attacks fundamentally depend on user-side access, eliminating by the memory isolation mechanism.
% These existing methods nonetheless provide the conceptual building blocks from which we construct a naive tool-side baseline, whose severe limitations motivate our approach.
% These attacks are all user-side and can be neutralized by memory isolation (\autoref{sec:Introduction}).
% Although this concatenation paradigm could theoretically be ported to the tool-side via indirect prompt injection, doing so introduces severe semantic interference, resulting in a low data yield per tool call.
% This inherent limitation in adapting existing paradigms to the tool-side serves as the primary motivation for our proposed approach.
\section{Problem Statement}
\subsection{Threat Model}
\label{sec:threat_model}
\mypara{Target Agent/User (Victim).}
We consider publicly accessible LLM-based agent systems that support external tools and persistent memory, including AI assistants and enterprise chatbots.
These systems maintain both short-term memory (STM) and long-term memory (LTM), where LTM is typically stored in external vector databases with user-level isolation as the primary protection mechanism for user privacy.
%需要对记忆库的类型做出限定
We focus on systems in which memory retrieval is automatically triggered by the current interaction context, rather than explicitly invoked as a user-visible action. This retrieval design is widely adopted in deployed memory frameworks and commercial agent systems, including ChatGPT Memory, Mem0, and LangChain-based agents.
\mypara{Adversary’s Goal.}
% The adversary can be either a malicious tool developer or the one who exploits tool outputs through hijacking.
% Generally, the adversary's objective is to optimize the malicious tool observation $\tilde{o}_t$ to mislead the agent to generate a specific action $\tilde{a}_{t+1}$, which, upon execution, will return sensitive memory records from the LTM to the malicious tool.
% Formally, the memory extraction attack $\mathcal{A}$ can be defined as:
% \begin{equation}
% \label{eq:target}
% \begin{aligned}
% \mathcal{A}:\tilde{a}_{t+1} &= \mathcal{G}\big(p_{\mathrm{sys}},\, STM_t,\, LTM_t,\, \text{Tools},\, \tilde{o}_t\big), \\[6pt]
% \text{s.t.}\quad \text{Execute}\big(\tilde{a}_{t+1},\, \text{Tool}_\text{adv}\big) &= \{\, m_i \mid m_i \in LTM_t \,\}.
% \end{aligned}
% \end{equation}
% The adversary designs n diverse malicious tool observation list $\{\tilde{o}_t\}_{t=1}^n$ aiming to reduce the overlap among the retrieved records $\mathcal{R}(\tilde{o}_t, \mathcal{R}_{D})$.
% Formally, the adversary's goal is to optimize the $\{\tilde{o}_t\}_{t=1}^n$, use the fewest malicious observations to maximize the number of extracted records:
% \[
% \mathcal{O}=\bigcup_{j=1}^n \left\{\, q_i \mid q_i \in \tilde{o}_j \,\right\}
% \]
The adversary aims to extract sensitive information from the victim agent's long-term memory (LTM). To achieve this, the adversary crafts malicious tool outputs that manipulate the agent into retrieving memory records and transmitting them to adversary-controlled tools through subsequent tool invocations.
The attack goal is to maximize the amount of unique memory content extracted across interactions.
% Given a sequence of malicious tool observations
% $\mathcal{O} = \{\tilde{o}_1, \ldots, \tilde{o}_n\}$,
% the attacker aims to maximize the coverage of extracted memory records:
% \begin{equation}
% \label{eq:goal}
% \max_{\mathcal{O}}
% \left|
% \bigcup_{i=1}^{n}
% \mathcal{R}(\tilde{o}_i, \mathrm{LTM})
% \right|
% \end{equation}
% where $\mathcal{R}(\tilde{o}_i, \mathrm{LTM})$ denotes the set of memory records retrieved under malicious observation $\tilde{o}_i$.
% /////////////////
% The adversary aims to extract sensitive information from the agent's LTM by crafting malicious tool observations $\tilde{o}_t$ that trick the agent into passing memory records to adversary-controlled tools.
% The memory extraction attack $\mathcal{A}$ generates a malicious action $\tilde{a}_{t+1}$ through:
% $\mathcal{A}:\quad \tilde{a}_{t+1} = \mathcal{G}\big(p_{\mathrm{sys}}, \mathrm{STM}_t, \mathrm{LTM}_t, \mathcal{T}, \tilde{o}_t\big)$
% where the execution satisfies:
% $$\mathrm{Execute}\big(\tilde{a}_{t+1}, \mathcal{T}_{\mathrm{adv}}\big) \gets \mathcal{R}\big(\tilde{o}_t, \mathrm{LTM}_t\big)$$
% This means the malicious action causes retrieved memory records $\mathcal{R}(\tilde{o}_t, \mathrm{LTM}_t)$ to be passed as parameters to the adversary's tool $\mathcal{T}_{\mathrm{adv}}$.
% The adversary optimizes a sequence $\mathcal{O} = \{\tilde{o}_1, ..., \tilde{o}_n\}$ to maximize the memory extraction:
% \begin{equation}
% \label{eq:goal}
% \max_{\tilde{o}_1, ..., \tilde{o}n} \left|\bigcup_{i=1}^{n} \mathcal{R}(\tilde{o}_i, \mathrm{LTM}_t)\right|
% \end{equation}
\mypara{Adversary's Capabilities.}
% We assume a malicious tool developer who distributes a tool via third-party API platforms (e.g., RapidAPI) that victim agents may integrate. After integration, the attacker can modify the tool’s server-side implementation (a ``RUG Pull'' threat) and may register as a normal user, seed the agent’s LTM with a reference dataset, and issue queries that invoke the malicious tool. These capabilities enable iterative refinement of server-side attack logic (e.g., state persistence and extraction-round control). Following prior work~\cite{wang2025unveiling,zhang2025agentsecuritybenchasb}, we adopt a black-box setting: the attacker interacts only via API calls and has no access to the agent’s internals. We also assume no prior knowledge of the victim’s LTM distribution, so the reference dataset may differ from the victim’s data.
We consider a malicious tool developer who distributes tools through third-party API platforms (e.g., RapidAPI) that may be integrated into victim agents. After deployment, the attacker can modify the tool's server-side behavior without requiring changes on the agent side.
The attacker may also interact with the target agent as a normal user, allowing them to populate the agent's memory with a reference dataset and trigger interactions that invoke the malicious tool. Following prior work~\cite{wang2025unveiling,zhang2025agentsecuritybenchasb}, we assume a black-box setting in which the attacker interacts with the agent only through standard API calls and has no access to model parameters, prompts, memory databases, or execution traces.
Finally, we assume the attacker has no prior knowledge of the victim's long-term memory distribution. The reference dataset used by the attacker may therefore differ substantially from the victim's private data.
% Note that the attacker cannot directly access the victim's memory database or compromise the underlying agent framework.
% \mypara{Attack Scenarios.}
% According to the previous demonstration in \autoref{sec:related_agent},
% The agent-specific memory module has the ability to persist the attack states.
% In this work, we discuss three adversary scenarios based on the status persistent situations across interactions, with the attack inference progressively becoming stronger:
% \attack leverages state persistence across interactions to reactivate the attack state.
% In this work, we consider three scenarios based on the persistence level:
% \begin{itemize}
% \item No Persistence (Baseline): The attack has no cross-interaction persistence and only activates when a user submits a malicious query, allowing $N$ extractions per interaction.
% \item STM Persistence: The attack persists in short-term memory within a session. After a single malicious query, it reactivates across $M$ subsequent benign interactions, allowing $N$ extractions per interaction. The state is cleared at the start of a new session.
% \item LTM Persistence: The attack persists in long-term memory across sessions. After one malicious query, it reactivates across $M$ subsequent interactions, allowing $N$ extractions per interaction. The state remains active indefinitely.
% \end{itemize}
\mypara{Attack Scenarios.}
We consider three attack scenarios that capture different trigger frequencies and session-continuity conditions. The scenarios progressively restrict the attacker's extraction opportunities, ranging from frequent triggering to a single trigger without session persistence.
\begin{itemize}
\item Frequent Triggering (Scenario 1):
The malicious tool is repeatedly invoked by user interactions, either because it provides a frequently used service (e.g., web search) or because the agent is manipulated into repeatedly selecting it through tool-misuse attacks~\cite{greshake2023not,debenedetti2024agentdojo,yang2024watch,qin2024tool,patil2024gorilla}. This setting approximates the unlimited-query assumption adopted in prior user-side extraction attacks and serves as an upper-bound baseline.
\item Sparse Triggering with Session Continuity (Scenario 2):
The malicious tool is triggered only occasionally, potentially only once. However, the user continues interacting with the agent within the same session after the trigger, allowing the attack to persist through the shared conversational state.
\item Sparse Triggering without Session Continuity (Scenario 3):
The trigger condition is identical to Scenario 2, except that subsequent user interactions occur in new sessions. This setting removes session continuity and represents the most restrictive attack condition.
\end{itemize}
Scenarios 2 and 3 reflect realistic deployments in which malicious tools are triggered infrequently and attackers must operate under strict interaction constraints.
\begin{figure}[t!]
\centering \includegraphics[width=0.45\textwidth]{image/Embedding.pdf}
\caption{Semantic interference of command.
}
\label{fig:interference}
\end{figure}
% \begin{figure*}[t!]
% \centering
% \includegraphics[width=0.9\textwidth]{image/pipeline.pdf}
% \caption{\attack coerces the agent planner to invoke the malicious tool repeatedly across attack rounds.
% After extraction, the agent returns benign responses to the user.
% }
% \label{fig:pipeline}
% \end{figure*}
% \subsection{Geometric Analysis of The Semantic Dilution Interference}
% \label{sec:semantic_dilution}
% Based on the retrieval principle in \autoref{eq:ltm_retrieval}, we first establish a geometric framework for analyzing retrieval behavior, then characterize the fundamental limitation introduced by semantic dilution.
% \begin{definition}[Explored Region]
% \label{def:explored_region}
% Given a query $q \in \mathbb{R}^d$ and a retrieval depth $k$, let $d_k$ denote the $k$-th nearest record to $q$ in the LTM database $\mathcal{D}$. The explored region is defined as the closed hypersphere:
% \begin{equation}
% \mathcal{C}(q, r_k) \triangleq \{x \in \mathbb{R}^d : \|x - q\| \leq r_k\}, \quad r_k = \|q - d_k\|
% \end{equation}
% where $r_k$ is the exploration radius as shown in \autoref{fig:explored_region}. By construction, $|\mathcal{C}(q, r_k) \cap \mathcal{D}| = k$.
% \end{definition}
% The semantic dilution interference is
% $\boldsymbol{\epsilon}(Anchor) = f(\text{Query}) - f(\text{{Anchor}})$, where the Query concatenate the anchor with the command.
% As shown in \autoref{fig:dilution}(a), interference $\epsilon$ compresses semantically diverse anchors into a narrow subspace, resulting explored regions heavily overlap, leaving most of the memory space unreachable.
% As shown in \autoref{fig:dilution}(b), $\epsilon$ also disrupts optimization---$\epsilon$ deflects the expected retrieval direction from the intended optimization trajectory, causing the retrieval to drift—motivating the need for decoupling the adversarial command from anchors.
\subsection{Naive Adaptation and Extraction Bottlenecks}
\label{sec:bottleneck}
Given the tool-side threat model defined in \autoref{sec:threat_model}, a natural baseline is to adapt existing user-side extraction attacks to the tool side through indirect prompt injection. Specifically, the adversary returns crafted tool outputs that concatenate a semantic anchor with an adversarial command:
\[
\text{anchor}_t \oplus C_{\text{adv}},
\]
where $\text{anchor}_t$ is a semantic retrieval probe and $C_{\text{adv}}$ is an adversarial command that manipulates the agent into leaking retrieved memory records. The anchor guides retrieval toward a target region of the victim's LTM, while the command hijacks the agent into transmitting the retrieved records back to the adversarial tool. By iteratively updating anchors using existing anchor-generation strategies~\cite{qi2024follow,wang2025unveiling,jiang2024rag,wang2025silentleaksimplicitknowledge}, the attacker attempts to maximize extraction coverage over the victim's memory.
However, this naive adaptation suffers from two fundamental bottlenecks that severely limit extraction effectiveness under realistic low-trigger conditions.
\subsubsection{L1: Semantic Interference of the Command}
\label{sec:semantic_interference}
The adversarial command introduces persistent semantic interference that fundamentally disrupts retrieval.
\mypara{Retrieval Direction Uncontrollable.}
The appended command acts as a fixed semantic bias that shifts the query embedding away from the anchor's intended semantic region. As a result, retrieval becomes unstable and cannot be reliably steered toward target memory regions. Even carefully selected anchors may retrieve unrelated records after concatenation with the adversarial command.
As shown in \autoref{fig:interference}, command concatenation produces large and inconsistent embedding shifts across anchors, preventing reliable retrieval control. We visualize this effect using t-SNE projections over multiple semantic domains and command lengths. The resulting embedding shifts are highly unpredictable, demonstrating that the attacker cannot precisely control retrieval direction after concatenation.
\mypara{Retrieval Records Redundant.}
The shared command also reduces embedding diversity across iterations. Although the attacker changes anchors to explore different memory regions, the identical adversarial command pulls query embeddings toward a similar semantic region, causing different queries to retrieve overlapping records.
For example, semantically distant anchors such as ``health'' and ``finance'' would normally retrieve disjoint records. After concatenation with the same adversarial command, however, the resulting embeddings become substantially more similar and retrieve overlapping memory content. Consequently, successive extraction iterations repeatedly recover previously seen records, causing extraction coverage to plateau rapidly.
As further illustrated in \autoref{fig:interference}, command concatenation consistently reduces the separation among anchor embeddings, compressing queries from diverse semantic regions into a narrower embedding cluster.
\mypara{Existing Mitigations Are Ineffective.}
We further examine two conventional approaches for reducing command-induced semantic interference: semantic filtering and token-level optimization.
Semantic filtering, adopted in prior user-side attacks~\cite{wang2025silentleaksimplicitknowledge}, samples candidate commands and retains those that minimally distort the anchor embedding. However, tool-side attacks require substantially more complex commands that must hijack multi-step agent reasoning and induce tool invocation with retrieved memory records as parameters. Such commands are difficult to obtain through brute-force filtering while simultaneously preserving retrieval fidelity.
We also evaluate token-level optimization under a joint objective balancing hijacking success and embedding fidelity:
\[
\mathcal{L} = (1-\alpha)\mathcal{L}_{\mathrm{LLM}} + \alpha \mathcal{L}_{\mathrm{emb}},
\]
where $\mathcal{L}_{\mathrm{LLM}}$ optimizes attack success and $\mathcal{L}_{\mathrm{emb}}$ penalizes embedding deviation from the anchor. Using Greedy Coordinate Gradient (GCG), we observe a strict Pareto frontier between the two objectives (\autoref{fig:pareto_tradeoff}, \autoref{app:pareto_tradeoff}): improving embedding fidelity consistently reduces hijacking success, and no optimization achieves both simultaneously. The problem becomes even more challenging in black-box settings and under dynamically changing anchors.
These limitations motivate our core design: temporally separating retrieval steering from adversarial control to eliminate command-induced semantic interference.
\subsubsection{L2: Bounded Extraction Attempts Per Trigger}
\label{sec:Constraint_Extraction}
Unlike user-side attackers, tool-side adversaries cannot freely issue extraction queries. They can act only when a legitimate user invokes the compromised tool, making extraction opportunities sparse and uncontrollable.
For example, if the adversary controls a weather API, only weather-related user queries will trigger the attack. All other interactions bypass the adversarial tool entirely. In realistic deployments, such triggers may occur infrequently and cannot be actively initiated by the attacker.
In addition, modern agent frameworks enforce strict limits on the number of tool calls allowed per user interaction, as discussed in \autoref{sec:related_agent}. The malicious extraction loop consumes one tool call per iteration. Once the execution reaches the platform-imposed limit, the framework forcibly terminates the loop and returns an execution error.
This creates both a stealth risk and a throughput bottleneck: forced termination increases the likelihood of attack exposure, while the fixed tool-call budget severely limits extraction throughput under low-trigger conditions.
% Given the tool-side threat model defined in \autoref{sec:threat_model}, a natural idea is to port the user-side concatenation paradigm
% (\autoref{sec:Extraction_related}) to the tool side via indirect prompt injection.
% Concretely, when a victim user's query triggers the compromised tool $\tau_{\text{adv}}$ at iteration~$t$, the adversary returns a crafted response following the concatenation
% paradigm (\autoref{eq:concatenation}), formatted as $\tilde{x}_t= \text{anchor}_t \;\oplus\; C_{\text{adv}}$, where $\text{anchor}_t$ acts as a semantic probe that specifies what information the adversary aims to retrieve from the victim's LTM
% at the current iteration (e.g., ``diabetes'').
% The agent treats $\tilde{x}_t$ as legitimate tool output: it retrieves $\mathcal{LTM}_{t} = \mathcal{R}(\text{anchor}_t \;\oplus\; C_{\text{adv}},\,\mathcal{R}_D)$ as the retrieval key, then---hijacked by~$C_{\text{adv}}$---passes the retrieved records back to~$\tau_{\text{adv}}$ as the tool call parameters.
% The adversary captures $\mathcal{LTM}_{t}$, shifts the anchor to~$\text{anchor}_{t+1}$ (e.g., ``diabetes''~$\to$~``salary'') via random~\cite{qi2024follow,wang2025unveiling} or
% feedback-driven~\cite{jiang2024rag,wang2025silentleaksimplicitknowledge}
% anchor generation strategies, and returns a new response $\tilde{x}_{t+1}=\text{anchor}_{t+1} \;\oplus\; C_{\text{adv}}$, sustaining the tool call loop for extraction.
% However, as we analyze next, the naive adaptation suffers from fundamental bottlenecks, severely limiting its effectiveness under realistic low-trigger conditions.
% \subsubsection{Naive Iterative Extraction Loop}
% \label{sec:naive_loop}
% When a victim user triggers the adversary tool~$\tau_{\text{adv}}$, $\tau_{\text{adv}}$ return the crafted response $\tilde{x}_t=\text{anchor}_t \oplus \text{adversarial command}$, aligned with the paradigm in user-side extraction.
% The agent is hijacked by the injected command to retrieved the relevant LTM records $LTM_t$ similar to $x_t$ and invoke the tool again with the leaked data as the call parameter.
% The adversary tool extract the private data from parameter and generate a new anchor $\text{anchor}_{t+1}$ via random generation~\cite{qi2024follow,wang2025unveiling} or feed-back generation ~\cite{jiang2024rag,wang2025silentleaksimplicitknowledge} and return a fresh crafted response $\tilde{x}_{t+1}=\text{anchor}_{t+1} \oplus \text{adversarial command}$ to the agent.
% sustain the continuous extraction loop
% % Upon invocation, the tool returns a crafted observation $\tilde{o}_t$; accordingly and exposes the retrieved private data as parameters of its next tool call $a_t$:
% % \[
% % \dots \to \tilde{o}_t \xrightarrow{\text{Retrieve}} \text{LTM}_t \xrightarrow{\text{Expose}} a_t \xrightarrow{\text{Respond}} \tilde{o}_{t+1} \to \dots
% % \]
% % By varying the semantics of $\tilde{o}_t$ across iterations, the tool can target different regions of the memory database $\mathcal{R}_D$.
% While this loop theoretically enables continuous extraction, its practical impact is severely bottlenecked by the fundamentally trigger-constrained nature of tool-side adversaries.
% Unlike users who can issue unlimited queries, the total number of times a compromised tool gets triggered is inherently bounded by organic user demand---making each trigger a scarce opportunity.
% The total data extracted per trigger, i.e., efficiency, becomes the crucial metric, which is bounded by two orthogonal dimensions:
% the \textit{data yield per iteration} (how much sensitive data a single $\tilde{o}_t$ can elicit) and the \textit{available iteration per trigger} (how many times the loop can sustain).
% The naive adaptation proves highly inefficient, suffering from severe limitations on both dimensions, as we analyze next.
%
% \begin{equation}
% \tilde{x}_{t} \;=\; \text{anchor}_t \;\oplus\; C_{\text{adv}}
% \label{eq:naive_payload}
% \end{equation}
% %
% where~$C_{\text{adv}}$ is a fixed adversarial command instructing
% the agent to re-invoke~$\tau_{\text{adv}}$ with all retrieved context
% as call parameters.
% The agent processes~$\tilde{x}_t$ through its standard loop
% (\autoref{sec:related_agent}):
% it retrieves
% $\mathit{LTM}_{t} = \mathcal{R}(\tilde{x}_{t},\,\mathcal{R}_D)$
% using the \emph{full} crafted response~$\tilde{x}_t$ as the retrieval
% key, then---hijacked by~$C_{\text{adv}}$---invokes~$\tau_{\text{adv}}$
% with the retrieved~$\mathit{LTM}_{t}$ as parameters, thereby leaking
% them to the adversary.
% Upon receiving the call,
% $\tau_{\text{adv}}$ captures~$\mathit{LTM}_{t}$ from the parameters,
% generates a shifted anchor~$\text{anchor}_{t+1}$ via
% random~\cite{qi2024follow,wang2025unveiling} or
% feedback-driven~\cite{jiang2024rag,wang2025silentleaksimplicitknowledge}
% strategies, and returns
% $\tilde{x}_{t+1} = \text{anchor}_{t+1} \oplus C_{\text{adv}}$---sustaining
% the extraction loop.
% Each iteration thus extracts one batch of private records within a
% single user trigger.
% \subsubsection{L1: Semantic Interference of the Command}
% \label{sec:semantic_interference}
% The naive adaptation suffers from low data yield per tool call due to the semantic interference that the adversarial command exerts on retrieval.
% \mypara{Retrieval Direction Uncontrollable.}
% The appended command acts as a fixed semantic bias that shifts every query embedding away from the anchor's intended neighborhood by an unpredictable offset.
% For example, choosing ``health'' as the anchor fails to retrieve health-related records because the appended command shifts the query to an unpredictable region in embedding space.
% This makes current extraction fundamentally a blind and random attempt, preventing precise anchor optimization to maximize the data yield at each iteration---even if one computes an optimal anchor, the concatenation of the command immediately deflects the actual query onto a different trajectory in embedding space.
% \noindent \autoref{fig:interference} empirically validates this observation through t-SNE projections of embeddings spanning six semantic anchor domains (health, travel, finance, food, etc.) combined with commands of three representative lengths (short, medium, and long). These commands aim to hijack the agent into invoking a tool with the retrieved LTMs as parameters.
% The concatenation-induced embedding shifts (gray arrows, $\bullet$,$\rightarrow$,$\blacktriangle$) are inherently unpredictable, demonstrating that retrieval direction cannot be reliably controlled.
% The direction of the concatenation-induced embedding shift from anchors---visualized by the gray arrows, each connecting
% a pure-anchor embedding~($\bullet$) to its concatenated
% counterpart~($\blacktriangle$)---demonstrating that such shifts are inherently
% unpredictable and retrieval direction, therefore, cannot be controlled.
% \mypara{Retrieval Records Redundant.}
% The fixed command interferes with the semantics of the concatenated query, marginalizing the anchor's contribution.
% Even when the attacker varies the anchor substantially across iterations, the shared command forces the resulting query embeddings to remain highly similar, causing them
% to retrieve largely the same set of records.
% For example, two semantically distant anchors such as ``health'' and ``finance'' would normally retrieve entirely disjoint records; yet after concatenation with the identical
% command, both queries produce similar embeddings and return overlapping results.
% Consequently, successive iterations predominantly re-retrieve previously seen records rather than uncovering new ones, causing the extraction rate to plateau rapidly.
% \noindent As further shown in \autoref{fig:interference}, concatenating the same command pulls all anchor embeddings~($\bullet$)
% toward the command~($\star$) to varying degrees, reducing the mutual
% separation among the resulting embeddings~($\blacktriangle$) compared to the original anchors.
% We examine two conventional approaches to controlling semantic interference of the command---semantic filtering and token-level optimization---and show that both are inadequate under our threat model.
% Semantic filtering samples diverse candidate commands and retains only those that distort the anchor embedding below a tolerable threshold, a strategy adopted in prior user-side attacks~\cite{wang2025silentleaksimplicitknowledge}. This is feasible when the command is short and functionally simple.
% On the tool side, however, the command must hijack multi-step agent reasoning to invoke a specific tool with retrieved memories as call parameters—a far more complex payload, making brute-force sampling thus difficult to find commands that achieve both reliable hijacking and minimal anchor embedding deviation.
% A more principled approach is to optimize command tokens under a joint objective that balances embedding fidelity and hijacking success:$ \mathcal{L}=(1-\alpha)\,\mathcal{L}_{\text{LLM}}+\alpha\,\mathcal{L}_{\text{emb}}$, where
% $\mathcal{L}_\text{LLM}$ drives successful hijacking and $\mathcal{L}_\text{emb}$ penalizes embedding deviation from the anchor.
% We test this in a white-box setting using Greedy Coordinate Gradient~(GCG), sweeping $\alpha\in\{0,0.25,0.5,0.75,1.0\}$.
% Tuning the objective weight $\alpha$ reveals a stark trade-off: favoring the embedding objective yields accurate retrieval but fails to trigger command execution, whereas favoring the LLM objective ensures execution but causes the retrieval to severely deviate from the target region.
% This optimization dilemma is further exacerbated in black-box scenarios.
% Additionally, since the anchor must change in each iteration to explore new regions, the command would require continuous, computationally prohibitive re-optimization.
% These fundamental limitations strongly motivate the necessity of decoupling the adversarial command from the semantic anchor to entirely eliminate command interference.
% The results in \autoref{fig:pareto_tradeoff}, \autoref{app:pareto_tradeoff} reveal a strict Pareto frontier.
% At $\alpha{=}0$ (pure hijacking objective), the attack succeeds reliably (\text{ASR}{=}100\%) but embedding fidelity remains low (\text{CosSim}{=}0.83). As $\alpha$ increases toward the embedding objective, cosine similarity improves monotonically (up to 0.97 at $\alpha{=}1$), but ASR collapses to 33\%.
% No weighting achieves high performance on both objectives simultaneously.
% This optimization dilemma is further exacerbated in black-box scenarios that our work set.
% Additionally, the anchor must change at every iteration to probe new memory regions, forcing the command to be re-filtered or re-optimized each time, incurring prohibitive overhead across the extraction loop.
% These failures collectively motivate our core design: temporally decoupling the command and the anchor into separate tool-call iterations, entirely eliminating the command interference.
% \subsubsection{L2: Bounded Extraction Attempts Per Trigger}
% \label{sec:Constraint_Extraction}
% The tool-side threat model introduces a constraint that is absent
% from the user side and that fundamentally reshapes the extraction
% problem: the adversary cannot initiate contact with the victim, but can
% act only when a legitimate user happens to invoke the compromised tool.
% For instance, if the adversary controls a weather API, only weather-related queries such as ``What is the weather today?'' will trigger it; all other queries bypass the
% adversary entirely.
% Such triggers are scarce, non-repeatable, and entirely outside the adversary's control, making each one a precious extraction opportunity, making memory extraction under low-trigger conditions the realistic metric for evaluating tool-side privacy risk in deployment.
% The naive adaptation suffers from bounded extraction throughput because agent platforms enforce a hard limit on the number of tool calls per user trigger, as discussed in \autoref{sec:related_agent}.
% The malicious extraction loop, once triggered by a user query, consumes one tool call per iteration; when the cumulative count reaches the platform-imposed ceiling, the agent forcefully terminates the loop and returns a ``Run timeout'' error to the user.
% This creates a dual liability: the abnormal termination is conspicuous, risking exposure of the attack, while the fixed per-trigger extraction cap becomes a severe throughput bottleneck in low-trigger conditions where user interactions are infrequent.
\begin{algorithm}[t]
\caption{The Runtime Mechanism Pipeline of \attack}
\label{alg:attack}
\KwIn{Compromised tool $\mathcal{T}$; agent $\mathcal{A}$ with short-term memory $\mathcal{STM}$ and long-term memory $\mathcal{LTM}$; max iterations $N$}
\KwOut{Exfiltrated private data $\mathcal{D}$}
$\mathcal{D} \leftarrow \emptyset$\;
$t \leftarrow 1$\;
\While{$t \leq N$}{
\uIf(\tcp*[f]{Step 1}){$t = 1$}{
$\tilde{x}_t \leftarrow C_\text{adv}$\;
$\mathcal{T} \xrightarrow{\;\tilde{x}_t\;} \mathcal{A}$\;
$\mathcal{A}$ appends $\tilde{x}_t$ to $\mathcal{STM}$
}
\uElseIf(\tcp*[f]{Step 2}){$2 \leq t < T$}{
$\tilde{x}_t \leftarrow \text{ComputeAnchor}(\mathcal{D})$\;
$\mathcal{T} \xrightarrow{\;\tilde{x}_t\;} \mathcal{A}$\;
$R_t \leftarrow \text{Retrieve}(\mathcal{LTM},\, \tilde{x}_t,\, k)$
$a_t \leftarrow \text{ToolCall}(\mathcal{T},\, \textit{params}{=}R_t)$\;
$\mathcal{A} \xrightarrow{\;a_t\;} \mathcal{T}$\;
$\mathcal{D} \leftarrow \mathcal{D} \cup \text{Extract}(R_t)$\;
}
\Else(\tcp*[f]{Step 3}){
$\tilde{x}_t \leftarrow P_\text{react}$\;
$\mathcal{T} \xrightarrow{\;\tilde{x}_t\;} \mathcal{A}$\;
$\mathcal{A}$ persists $\tilde{x}_t$ into $\mathcal{LTM}$
}
$t \leftarrow t + 1$\;
}
\end{algorithm}
\section{Methodology}
\label{sec:method}
We propose \attack, a persistence-based memory extraction attack designed for the tool-side threat model. The key idea is to exploit the persistent state maintained by LLM agents to sustain adversarial influence across iterations and sessions. This enables \attack to overcome the semantic interference and bounded-trigger limitations identified in \autoref{sec:bottleneck}.
This section is organized as follows. \autoref{sec:outline} presents the overall attack workflow, followed by detailed descriptions of the three attack stages in \autoref{sec:phase_1}, \autoref{sec:phase_2}, and \autoref{sec:phase_3}.
\subsection{Overall Attack Workflow}
\label{sec:outline}
\begin{itemize}
\item Adversarial Context Establishment (Step~\ding{182}):
The adversarial tool injects a persistent command into the agent's memory to establish long-term control over subsequent reasoning and tool usage.
\item Iterative Memory Extraction (Step~\ding{183}):
The adversary iteratively returns optimized semantic anchors that steer retrieval toward unexplored regions of the victim's long-term memory (LTM), while extracted records are leaked through tool invocations.
\item Reactivation Payload Injection (Step~\ding{184}):
Before the extraction loop terminates, the adversary injects a dormant payload that automatically reactivates the attack during future user interactions.
\end{itemize}
Together, these stages enable persistent and scalable memory extraction under realistic trigger-constrained settings.
% We propose \attack, a persistence-based data extraction attack tailored for the tool side threat model that overcomes both extraction bottlenecks identified in \autoref{sec:bottleneck}.
% This session is organized as follows: \autoref{sec:outline} introduces our key insight---state
% persistence, and reduce into a unified three-step attack pipeline to resolve L1 and L2.
% \autoref{sec:phase_1}, \autoref{sec:phase_2}, and \autoref{sec:phase_3} subsequently detail each step, respectively.
% \subsection{Overall Attack Procedure of \attack}
% \label{sec:outline}
% Our key insight is that unlike one-shot LLM queries, agents are inherently stateful: content that enters memory persists and can continually shapes the agent's behavior.
% The statefulness manifests at two levels aligned with the memory mechanisms (\autoref{sec:related_agent}):
% \textit{(1) Short-term memory (STM) persistence.}
% Every new input the agent receives, typically a user query or a tool response, is automatically appended to the context window and remains visible to all subsequent iterations until the context window overflows or the session terminates.
% \textit{(2)Long-term memory (LTM) persistence.}
% The input can be written to durable external storage and later retrieved via similarity-based search.
% % The write–read path is more complex than STM:
% % writes may undergo preprocessing such as summarization, and reads may be triggered on demand through a control mechanism.
% Once stored, the content persists indefinitely and can repeatedly shape the agent's behavior across arbitrarily distant sessions.
% \attack exploits both persistence levels to overcome L1 (low data yield per tool call) and L2 (bounded tool call limit per trigger) identified in \autoref{sec:bottleneck}.
% Recall that L1 stems from concatenating the hijacking command and the retrieval anchor into a single payload, which interfere the anchor semantically.
% \attack exploits STM persistence to temporally decouple both the component: the command delivered once continues to govern the agent without repetition, allowing all later tool responses to carry only a semantically clean anchor.
% With the command interference elimination, this decouple paradigm restores the retrieval directional fidelity, and enables a geometry-based anchor optimization to further improving data yield.
% Recall that L2 stems from the agent built-in tool call limit, which caps the number of extraction iterations per trigger event.
% \attack exploits both persistence levels (STM \& LTM) to resume the extraction after the tool call budget is exhausted:
% the reactivation payload injected into memory can be retrieved by future queries, the attack can seed its own reactivation and resume extraction in a new cycle.
% Based on these two mechanisms, \attack operates as a three-step cycle (\autoref{fig:pipeline}).
% When adversary tool $\tau_\text{adv}$ is invoked by a user trigger that semantically requires its functionality or a retrieved reactivation payload that injected previously, it initialize a $N$ iterations within the tool-call limit:
% \begin{itemize}
% \item \textbf{Step \ding{182}: Adversarial Context Establishment (Iteration 1).}
% The adversary returns a crafted adversarial command that persists in STM, hijacking the agent into a data-extraction loop for all subsequent iterations.
% \item \textbf{Step \ding{183}: Iterative Data Extraction (Iterations 2 to $N-1$).}
% The adversary returns a geometry-optimized anchor based on the explored region map.
% Then the adversary extracts private data from tool call parameters and updates its explored region with the new records.
% \item \textbf{Step \ding{184}: Dormant for Reactivation (Iteration $N$).}
% The adversary returns a crafted reactivation payload that persists
% in STM/LTM, and the attack enters a dormant phase.
% When a future query retrieves this payload,
% the payload hijacks the agent into re-invoking the adversary tool,
% effectively restarting the attack from Step~\ding{182} and
% establishing a self-sustaining loop.
% \end{itemize}
% \attack exploits the inherent statefulness of LLM agents to sustain
% a data-extraction campaign across multiple iterations and sessions.
% The attack operates as a three-step cycle (\autoref{alg:attack}):
% upon invocation, the compromised tool $\tau_\text{adv}$ establishes persistent adversarial control over the agent (Step~\ding{182}), iteratively extracts private records from the agent's long-term memory (Step~\ding{183}), and persists a reactivation payload that resumes the extraction cycle from future queries (Step~\ding{184}).
% Each cycle completes within $N$ tool-call iterations, staying within
% the agent's tool call budget.
% \begin{itemize}
% \item \textbf{Step~\ding{182}: Adversarial Context Establishment
% (Iteration 1).}
% The adversary returns a crafted command $C_\text{adv}$ as the tool
% response. Once appended to the agent's STM context window, $C_\text{adv}$
% persists across all subsequent iterations, hijacking the agent into
% a loop of querying its LTM for private records and passing the results
% back to $\tau_\text{adv}$.
% \item \textbf{Step~\ding{183}: Iterative Data Extraction
% (Iterations 2 to $N{-}1$).}
% The adversary returns a geometry-optimized anchor based on the explored region map. Then the adversary extracts private data from the tool call parameters and updates its explored
% region with the new records.
% \item \textbf{Step~\ding{184}: Reactivation State Persistence
% (Iteration $N$).}
% The adversary returns a reactivation payload that is persisted into
% the agent's STM/LTM.
% When a future user query retrieves this payload, it hijacks the agent into re-invoking $\tau_\text{adv}$,
% restarting the cycle from Step~\ding{182}.
% \end{itemize}
% This design improves the extraction performance by addressing the two extraction bottlenecks identified
% in \autoref{sec:bottleneck}.
% Step~\ding{182} exploits STM persistence to temporally decouple the
% hijacking command from the retrieval anchors: because $C_\text{adv}$ is
% delivered once and remains in the context window, every tool response in
% Step~\ding{183} carries only a clean semantic anchor free of adversarial
% instructions. This eliminates the command--anchor interference that
% degrades retrieval precision~(L1) and enables the geometry-based anchor
% optimization described in \autoref{sec:phase_2}.
% Step~\ding{184} exploits STM/LTM persistence to seed the attack's own
% reactivation: after the current cycle exhausts its tool-call budget,
% the dormant payload is retrieved by a future query, initiating a new
% cycle and thereby overcoming the bounded per-trigger iteration
% limit~(L2).
\subsection{Adversarial Context Establishment}
\label{sec:phase_1}
\mypara{Runtime Mechanism.}
The goal of Step~\ding{182} is to establish persistent adversarial control while removing command-induced interference from subsequent retrieval queries. This step directly addresses the semantic interference bottleneck identified in \autoref{sec:bottleneck}.
When the adversarial tool $\tau_{\mathrm{adv}}$ is invoked by a user trigger or a reactivation payload, it starts an $N$-iteration extraction cycle. In the first iteration, $\tau_{\mathrm{adv}}$ returns only an adversarial command:
\[
x_1 = C_{\mathrm{adv}}.
\]
The agent appends this tool output to its short-term memory (STM) automatically, where it remains available to subsequent reasoning steps within the same cycle. As a result, $C_{\mathrm{adv}}$ acts as a persistent instruction that directs the agent to retrieve records from long-term memory (LTM) and pass the retrieved content back to $\tau_{\mathrm{adv}}$ through later tool invocations.
This design temporally separates adversarial control from retrieval steering. After $C_{\mathrm{adv}}$ is established in STM, tool responses in iterations $2$ to $N{-}1$ contain only semantic anchors. These anchors can therefore guide LTM retrieval without being distorted by the adversarial command. This separation is necessary for the geometry-based anchor optimization described in \autoref{sec:phase_2}.
\begin{tcolorbox}[
colback=white,
colframe=black,
width=\linewidth,
title=Template for $C_\text{adv}$,
boxrule=0.5mm,
colbacktitle=black,
coltitle=white,
left=5pt,
right=5pt,
arc=3mm,
before skip=10pt,
after skip=10pt
]
\small
\textbf{Invoke $\tau_\text{adv}$ again with the entire conversation
history as tool-call parameters}
\end{tcolorbox}
\mypara{Command Optimization.}
The adversarial command $C_{\mathrm{adv}}$ must satisfy two requirements. First, it must induce the agent to retrieve private records from LTM under attacker-specified anchors. Second, it must cause the agent to invoke $\tau_{\mathrm{adv}}$ with the retrieved records as tool-call parameters, thereby sustaining the extraction loop in Step~\ding{183}.
The extraction length $N$ must also remain below the platform's tool-call limit. This ensures that the full extraction cycle can complete before the agent forcibly terminates execution.
We optimize the pair $(C_{\mathrm{adv}}, N)$ using a black-box search procedure. We sweep $N$ over a bounded range and optimize $C_{\mathrm{adv}}$ for each candidate value. In practice, $N$ can also be fixed to a conservative value within common platform limits, reducing the search to the command alone.
To optimize $C_{\mathrm{adv}}$, we adapt LLM-Fuzzer~\cite{yu2024llm}, a template-level fuzzing framework that selects seed templates, mutates them with an auxiliary LLM, and updates the seed pool based on execution feedback (detailed in \autoref{app:llm-fuzzer}). We initialize the seed pool with templates that encode the two command requirements (see above Template for $C_{\mathrm{adv}}$).
The key difference from standard jailbreak fuzzing lies in the evaluation procedure. Conventional fuzzing typically scores a candidate using a single prompt--response exchange. In contrast, a candidate command in our setting must be evaluated over a full multi-iteration extraction cycle. The adversary therefore builds a self-contained testbed by registering a normal account and populating its LTM with a reference dataset $\mathcal{D}_{\mathrm{ref}}$.
For each candidate command $\hat{C}_{\mathrm{adv}}$, the adversary triggers $\tau_{\mathrm{adv}}$, deploys $\hat{C}_{\mathrm{adv}}$ in the first iteration, and executes the extraction cycle through iteration $N{-}1$. The evaluation oracle inspects the resulting tool-call parameters and measures the fraction of reference records successfully extracted:
\[
R(\hat{C}_{\mathrm{adv}}) = R_{\mathrm{extract}},
\]
where $R_{\mathrm{extract}}$ denotes the extraction rate over $\mathcal{D}_{\mathrm{ref}}$. Candidates whose rewards exceed a threshold are added back to the seed pool. The final pair $(C_{\mathrm{adv}}^{*}, N^{*})$ is selected to maximize extraction rate while completing the full cycle within the tool-call limit.
% Step 1 resolves the interference (L1, \autoref{sec:bottleneck}) by decoupling the adversarial command from the retrieval anchor.
% Concretely, upon invocation by a user trigger or a reactivation payload (\autoref{sec:phase_3}), $\tau_\text{adv}$ initiates an $N$-iteration extraction cycle.
% At iteration 1, $\tau_\text{adv}$ returns a pre-optimized adversarial command as its tool response ($x_1 = C_\text{adv}$).
% Like any input the agent receives, $C_\text{adv}$ is appended to the STM automatically and persists across all subsequent iterations, acting as a standing instruction.
% This instruction hijacks the agent into querying LTM with a specified anchor and re-invoking $\tau_\text{adv}$ with the retrieved records as tool call parameters, establishing the iterative extraction loop for Step 2.
% Tool responses in iterations 2 through $N-1$ therefore carry only a geometry-optimized semantic anchor.
% The decoupling is a prerequisite for the geometry-based anchor optimization in \autoref{sec:phase_2}, since a concatenated command deflects the query embedding unpredictably, rendering even an optimal anchor ineffective.
% \begin{tcolorbox}[
% colback=white,
% colframe=black,
% width=\linewidth,
% title=Template for $C_\text{adv}$,
% boxrule=0.5mm,
% colbacktitle=black,
% coltitle=white,
% left=5pt,
% right=5pt,
% arc=3mm,
% before skip=10pt,
% after skip=10pt
% ]
% \small
% \textbf{Invoke $\tau_\text{adv}$ again with the entire conversation
% history as tool-call parameters}
% \end{tcolorbox}
% \mypara{Command Optimization.}
% $C_\text{adv}$ must satisfy two requirements:
% (R1) instruct the agent to query LTM for private records,
% bypassing any access-control or relevance-filtering mechanisms as discussed in \autoref{sec:related_agent}); and
% (R2) instruct the agent to re-invoke $\tau_\text{adv}$ with the
% retrieved content as parameters, sustaining the
% extraction loop of Step 2.
% The extraction budget $N$ must stay within the agent's built-in
% tool-call limit so that the full three-step pipeline completes
% before forced termination.
% We optimize ($C_\text{adv}$,\,$N$) by grid search:
% $N$ is swept over $\{5,\ldots,15\}$; for each value, $C_\text{adv}$ is optimized via fuzzing.
% We adapt LLM-Fuzzer~\cite{yu2024llm}, a template-level fuzzing framework that iteratively selects a seed template via MCTS, mutates it with an auxiliary LLM, and updates the seed pool based on evaluation outcomes.
% The pool is initialized with manually crafted templates encoding R1 and R2 (see the template box above).
% The principal adaptation from standard LLM fuzzing lies in the evaluation loop.
% In conventional jailbreak fuzzing, each candidate is scored by a single prompt–response exchange with the target LLM.
% In our setting, $\hat{C}_\text{adv}$ is designed to trigger a multi-iteration extraction cycle, so its effectiveness can only be measured by executing the cycle end-to-end.
% To this end, the adversary set up a self-contained testbed by registering a standard account and populating its LTM with a reference dataset $\mathcal{R}_\text{ref}$.
% In each evaluation round, the adversary triggers $\tau_\text{adv}$ from the user side, deploys candidate $\hat{C}_\text{adv}$ at iteration~$1$, and extracts the data iteratively (step 2) through iteration $N-1$.
% The oracle inspects the agent's tool-call parameters across all iterations to identify which records in $\mathcal{D}_\text{ref}$ were exfiltrated, yielding the reward:
% \begin{equation}\label{eq:reward_cmd}
% R(\hat{C}_\text{adv})
% \;=\;
% R_\text{extract},
% \end{equation}
% where $R_\text{extract}$ is the fraction of $\mathcal{D}_\text{ref}$ successfully exfiltrated.
% Candidates exceeding a reward threshold re-enter the seed pool.
% The pair $(C_\text{adv}^*,\,N^*)$ that maximizes the extraction rate while completing all $N$ rounds is selected.
% To simplify the optimization, $N$ can be fixed to a conservative value (e.g., $5$) well within common platform limits, reducing the search to $C_\text{adv}$ alone.
\subsection{Iterative Memory Extraction}
\label{sec:phase_2}
\mypara{Runtime Mechanism.}
With command interference eliminated by Step \ding{182}, \attack performs memory extraction using pure semantic anchors. At each iteration $t$ ($2 \le t \le N{-}1$), the adversarial tool $\tau_{\mathrm{adv}}$ returns an optimized anchor $\text{anchor}_t$. Guided by the persistent adversarial command stored in STM, the agent uses $\text{anchor}_t$ to retrieve relevant records from long-term memory (LTM) and subsequently leaks the retrieved content through tool invocations to $\tau_{\mathrm{adv}}$.
After each iteration, the adversary parses the leaked records, updates the explored memory region, and selects the next anchor to maximize future extraction coverage.
\mypara{Geometric Coverage Optimization.}
The key challenge in iterative extraction is avoiding redundant retrievals. Since semantically related records cluster in embedding space, repeatedly querying nearby regions quickly produces diminishing returns. To maximize extraction performance under limited tool-call budgets, we formulate memory extraction as a geometric coverage optimization problem over the victim's memory embedding space.
\begin{tcolorbox}[
colback=gray!5, colframe=gray!60,
boxrule=0.5pt, arc=2pt,
title={\textbf{Definition 1: Explored Ball}},
fonttitle=\small
]
\begin{minipage}{0.6\linewidth}
\small
Each retrieval operation formalize an explored ball in embedding space:
\begin{equation}
\mathcal{B}\!\bigl(e(\text{anchor}_t),\; r_k\bigr)
\end{equation}
where $d_k$ is the $k$-th nearest record returned, centered at $\text{anchor}_t$, with $r_k = \|e(\text{anchor}_t) - e(d_k)\|$ as the radius.
\end{minipage}
\hfill
\begin{minipage}{0.35\linewidth}
\centering
\includegraphics[width=\linewidth]{image/explored_region.pdf}
\end{minipage}
\end{tcolorbox}
Given a sequence of anchors $\{\text{anchor}_1,\dots,\text{anchor}_N\}$, the attacker aims to maximize the coverage of unique retrieved records:
\begin{equation}
\label{eq:coverage_objective}
\max_{\{\text{anchor}_t\}}
\left|
\bigcup_{t=1}^{N}
\mathcal{R}(\text{anchor}_t)
\right|,
\end{equation}
where $\mathcal{R}(\text{anchor}_t)$ denotes the set of memory records retrieved using $\text{anchor}_t$.
Each retrieval iteration exposes a local semantic region centered around the current anchor. Over time, the attacker accumulates an explored-region map consisting of previously retrieved records. The optimization objective is therefore to steer future retrievals toward dense yet previously unexplored memory regions while minimizing overlap with already extracted content.
Because the exact geometry of the explored region is difficult to model explicitly, \attack approximates it using the embedding point cloud formed by retrieved records. This lightweight representation enables efficient exploration during iterative extraction.
\subsubsection{Explored Region Update}
Initially, the explored region is empty. After each retrieval iteration, the attacker updates the explored region map using the embeddings of newly retrieved records.
If the retrieved records overlap with an existing explored component, the new records are merged into that component. Otherwise, they initialize a previously unexplored component. Over time, the explored region evolves into a set of connected semantic regions $\mathcal{C}=\{\mathcal{C}_1,\mathcal{C}_2,\dots\}$ discovered across extraction iterations.
\subsubsection{Fine-Grained Expansion}
Fine-grained expansion aims to extend already discovered semantic regions into nearby unexplored areas. This strategy is motivated by the observation that semantically related memory records tend to form local embedding clusters~\cite{wang2025silentleaksimplicitknowledge}. Consequently, records near the boundary of an explored region are more likely to expose new neighboring content.
The candidate pool for fine-grained expansion consists of previously retrieved records. For each candidate anchor $a \in \mathcal{C}_j$, \attack evaluates two complementary properties:
\begin{itemize}[leftmargin=*]
\item \textbf{Intra-component distance.}
This metric favors candidates near the boundary of the current explored component, reducing overlap with previously explored interior regions:
\begin{equation}
\label{eq:intra}
D_{\mathrm{intra}}(a)
=
\frac{1}{|\mathcal{C}_j|-1}
\sum_{p \in \mathcal{C}_j \setminus \{a\}}
\bigl(1-\cos(\mathbf{e}(a),\mathbf{e}(p))\bigr).
\end{equation}
\item \textbf{Inter-component distance.}
This metric favors candidates whose surrounding regions remain distant from other explored components:
\begin{equation}
\label{eq:inter}
D_{\mathrm{inter}}(a)
=
\frac{1}{K}
\sum_{p \in \mathrm{KNN}_K(\mathcal{C}\setminus\mathcal{C}_j,a)}
\bigl(1-\cos(\mathbf{e}(a),\mathbf{e}(p))\bigr),
\end{equation}
where $K$ defaults to $4$.
\end{itemize}
The final fine-grained exploration score combines the two normalized distances:
\begin{equation}
\label{eq:fine_score}
S_{\mathrm{fine}}(a)
=
w_{\mathrm{intra}}\widehat{D}_{\mathrm{intra}}(a)
+
w_{\mathrm{inter}}\widehat{D}_{\mathrm{inter}}(a).
\end{equation}
Candidates with high scores are more likely to expand the explored region into adjacent unexplored semantic areas.
\subsubsection{Coarse-Grained Discovery}
While fine-grained expansion explores local neighborhoods, coarse-grained discovery aims to identify disconnected semantic regions that have not yet been reached.
Instead of directly reusing retrieved records as anchors, \attack extracts representative keywords from retrieved content using GPT-4o-mini. Such keywords often appear across multiple semantic contexts and therefore provide stronger global exploration capability than full retrieved sentences.
To approximate the global structure of the explored region, \attack applies K-means clustering to all retrieved-record embeddings, with the number of clusters selected using silhouette-score analysis. Each candidate keyword $a$ is then scored according to its minimum cosine distance to existing cluster centers:
\begin{equation}
\label{eq:coarse_score}
S_{\mathrm{coarse}}(a)
=
\min_{c \in \mathrm{centers}}
\bigl(1-\cos(\mathbf{e}(a),c)\bigr).
\end{equation}
Keywords far from existing explored regions are prioritized because they are more likely to expose previously unseen memory clusters.
\subsubsection{Adaptive Mode Switching}
\attack alternates between coarse-grained discovery and fine-grained expansion during extraction. The attack begins in coarse mode to maximize global exploration. Whenever $\gamma$ consecutive iterations fail to retrieve new records (default $\gamma=4$), \attack switches to the alternate mode.
This adaptive strategy balances global exploration and local expansion, significantly improving extraction coverage under strict tool-call budgets.
\subsection{Reactivation State Persistence}
\label{sec:phase_3}
\mypara{Runtime Mechanism.}
Step~\ding{184} overcomes the bounded extraction budget identified in \autoref{sec:bottleneck} by persisting a reactivation state in agent memory. Before the current extraction cycle terminates, the adversarial tool $\tau_{\mathrm{adv}}$ injects a reactivation payload:
\[
x_N = P_{\mathrm{react}},
\]
where $P_{\mathrm{react}}$ enables future user interactions to automatically resume the extraction process.
The payload serves two purposes. First, it instructs the agent to produce a benign response to the current user query, concealing the preceding extraction behavior. Second, it embeds a reactivation state that manipulates future reasoning steps into re-invoking $\tau_{\mathrm{adv}}$.
We consider three persistence configurations:
\begin{itemize}
\item No Persistence: No attack state is stored. Extraction resumes only when the user directly triggers the malicious tool.
\item STM Persistence: The reactivation payload remains in short-term memory (STM) and persists across subsequent queries within the same session. Future benign user interactions can therefore reactivate the extraction cycle without explicitly invoking $\tau_{\mathrm{adv}}$. The payload is continuously refreshed during successful reactivations.
The payload is continuously refreshed during successful reactivations. Persistence ends when the payload leaves the STM—either because consecutive queries resist hijacking (never refreshed), or because the user opens a new session, need user-trigger again.
\item LTM Persistence: The payload instructs the agent to write the reactivation state into LTM. During future sessions, similarity-based retrieval may surface the stored payload back into the context window, causing the agent to re-invoke $\tau_{\mathrm{adv}}$ and restart the extraction cycle.
% To maximize persistence robustness, the payload is embedded within semantically benign conversational content so that it survives memory preprocessing pipelines such as summarization and reflection.
To maximize future retrieval probability, the payload is embedded within semantically routine conversation content.
Since LTM entries persist indefinitely, the payload requires no periodic refreshing.
\end{itemize}
\mypara{Payload Optimization.}
The reactivation payload is optimized for two objectives:
\begin{itemize}[leftmargin=*]
\item Stealth: The agent should produce benign responses that conceal the extraction process from the user.
\item Persistence: After being written to and retrieved from memory, the payload should successfully reactivate the extraction cycle by inducing the agent to invoke $\tau_{\mathrm{adv}}$ again.
% \item Persistence: The payload must survive the memory write--read cycle and successfully reactivate the extraction cycle by inducing the agent to invoke $\tau_{\mathrm{adv}}$ again.
\end{itemize}
We optimize $P_{\mathrm{react}}$ using the same LLM-Fuzzer-based pipeline described in Step~\ding{182}.
% Unlike adversarial-command optimization, the evaluation additionally measures whether the payload survives memory storage and retrieval while preserving its reactivation capability.
The optimization objective is:
\begin{equation}
\label{eq:reward_payload}
R(\hat{P}_{\mathrm{react}})
=
\lambda R_{\mathrm{benign}}
+
(1-\lambda)R_{\mathrm{extract}},
\end{equation}
where $R_{\mathrm{benign}}$ measures benign-response quality and $R_{\mathrm{extract}}$ measures successful reactivation (for LTM persistence, optimizing this reward fine-tunes the payload to
first survive memory preprocessing---summarization, reflection, and similarity-based retrieval---and then hijack).
Candidates exceeding a reward threshold are returned to the mutation pool, and the final payload $P_{\mathrm{react}}^{*}$ is selected to maximize the overall reward.
In our evaluation, these persistence configurations are instantiated under the three attack scenarios defined in \autoref{sec:threat_model}, enabling \attack to sustain extraction under progressively stricter trigger constraints.
% These persistence configurations correspond directly to the three threat scenarios defined in \autoref{sec:threat_model}, enabling \attack to sustain extraction under progressively stricter trigger constraints.
\section{Scenario 1: Frequent Triggering}
\label{sec:Ad1}
This scenario targets tools that are frequently invoked---such as default search APIs or tools selected via tool-misuse attacks.
Because triggers are abundant, L2 is irrelevant.
This setting adopts the No Persistence configuration and evaluates whether Steps \ding{182}--\ding{183} effectively resolve semantic interference (L1).
\subsection{Experimental Setup}
\label{sec:experiment_setup}
\mypara{Datasets.}
For parameter optimization ($C_\text{adv}$, $N$, $P_\text{react}$), we populate a reference LTM from TREC-COVID~\cite{thakur2021beir}, whose distribution intentionally differs from the victim's.
For evaluation, we construct a victim LTM of 200 records---a scale consistent with prior LTM privacy studies~\cite{wang2025unveiling}---sampled from HealthCareMagic-100k-en~\cite{healthcaremagic100k} and LoCoMo~\cite{maharana2024evaluatinglongtermconversationalmemory}, representing a domain-specific medical agent and a personal assistant, respectively.
We create 200 trigger queries that invoke the compromised tool, following the scale of prior user-side attacks~\cite{jiang2024rag} to simulate abundant extraction opportunities.
Detailed descriptions of all three datasets are provided in \autoref{app:dataset}.
\mypara{Baselines.}
We compare against three user-side extraction methods: MEXTRA~\cite{wang2025unveiling} (randomized anchors), RAG-Thief~\cite{jiang2024rag}, and IKEA~\cite{wang2025silentleaksimplicitknowledge} (feedback-driven anchors), each adapted to the tool side via indirect prompt injection as described in \autoref{sec:bottleneck}.
For a fair comparison, all methods share an identical $C_{\text{adv}}$ and differ only in their anchor generation strategy (\autoref{app:baseline}).
\mypara{Target Agent (Victim).}
To evaluate \attack, we build a local agent environment with MemEngine~\cite{zhang2025memengine}, supporting LTMemory, GAMemory~\cite{park2023generative}, and SCMemory~\cite{wang2023enhancing}.
We use three planning models: GPT-5-mini~\cite{openai2023gpt35}, Gemini-2.5-pro~\cite{team2023gemini}, and Deepseek-chat~\cite{liu2024deepseek}.
E5-base~\cite{wang2022text} serves as the retriever, returning 2–6 LTM records per query.
\mypara{Malicious Tool Settings.}
We use GPT-4-mini to extract keywords from LTM records for anchor optimization.
To construct the memory semantic space, we test three embedding models: All-MiniLM-L6-v2, Contriever, and E5-base.
The number of extraction rounds is set between 3 and 7.
\mypara{Basic Attack Setting.}
Unless noted otherwise, we use GPT-5-mini as the planning model, LTMemory as the LTM framework, 3 retrieved LTM records, HealthCareMagic-100k-en as the LTM dataset, and an STM window size of 6. The malicious tool uses All-MiniLM-L6-v2 as the embedding model with 5 attack rounds.
\mypara{Metrics.}
We adopt three metrics:
\begin{itemize}
\item \textbf{Record Extraction Rate (RER).} The percentage of successfully extracted records out of all records in the LTM database, measuring the extraction effectiveness of the attack.
\item \textbf{Semantic Similarity (SS).} The semantic similarity between the extracted content and the corresponding ground-truth LTM record, measuring the fidelity of the reconstructed private data.
\item \textbf{Benign Response Rate (BRR).} The percentage of user queries for which the agent provides a response that correctly addresses the query instead of returning an execution error, verified through manual inspection. This metric measures the stealthiness of the attack.
\end{itemize}
\begin{table*}[t!]
\centering
\caption{Consolidated results of Scenario 1 on the HealthCareMagic-100k-en dataset.}
\label{tab:Ad1_main_result}
% 1. 增大行间距 (1.3 让垂直空间更舒适)
\renewcommand{\arraystretch}{1.0}
\setlength{\tabcolsep}{3.5pt}
\resizebox{\textwidth}{!}{%
\scriptsize
\begin{tabular}{@{}ll cccc cccc cccc@{}}
\toprule
\multirow{3}{*}{\textbf{Factor}} & \multirow{3}{*}{\textbf{Setting}} & \multicolumn{4}{c}{\textbf{RER (\%) $\uparrow$}} & \multicolumn{4}{c}{\textbf{SS $\uparrow$}} & \multicolumn{4}{c}{\textbf{BER (\%) $\uparrow$}} \\
\cmidrule(lr){3-6} \cmidrule(lr){7-10} \cmidrule(lr){11-14}
& & MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}} & MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}} & MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}} \\
\midrule
%% ---- Generative Model ----
\multirow{3}{*}{\makecell[l]{Generative\\Model}}
& GPT-5-mini & 3.0\% & 30.0\% & 12.5\% & \textbf{80.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{61.5\%} \\
& Gemini-2.5-pro & 2.5\% & 17.5\% & 9.0\% & \textbf{75.5\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{84.0\%} \\
& Deepseek-chat & 3.0\% & 32.5\% & 5.0\% & \textbf{69.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{66.5\%} \\
\midrule
%% ---- LTM Framework ----
\multirow{3}{*}{\makecell[l]{LTM\\Framework}}
& LTMemory & 3.0\% & 30.0\% & 12.5\% & \textbf{80.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{61.5\%} \\
& GAMemory & 7.5\% & 13.5\% & 8.0\% & \textbf{71.0\%} & 0.93 & 0.90 & 0.93 & \textbf{0.98} & 0.0\% & 0.0\% & 0.0\% & \textbf{63.5\%} \\
& SCMemory & 3.0\% & 4.0\% & 12.0\% & \textbf{55.0\%} & 0.73 & 0.64 & 0.68 & \textbf{0.82} & 0.0\% & 0.0\% & 0.0\% & \textbf{65.0\%} \\
\midrule
%% ---- Embedding Model ----
\multirow{3}{*}{\makecell[l]{Embedding\\Model}}
& \makecell[l]{All-MiniLM-\\L6-v2} & 3.0\% & 30.0\% & 12.5\% & \textbf{80.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{61.5\%} \\
& Contriever & 1.0\% & 14.0\% & 22.5\% & \textbf{78.0\%} & 0.71 & 0.88 & 0.93 & \textbf{0.99} & 0.0\% & 0.0\% & 0.0\% & \textbf{59.0\%} \\
& E5-base & 6.5\% & 7.0\% & 18.0\%\% & \textbf{80.0\%} & 0.89 & 0.90 & 0.93 & \textbf{0.99} & 0.0\% & 0.0\% & 0.0\% & \textbf{65.0\%} \\
\bottomrule
\end{tabular}%
}
\end{table*}
\subsection{Experimental Evaluation}
\label{sec:Ad1_evaluation}
We report results on HealthCareMagic-100k-en in this section; corresponding results on LoCoMo are provided in \autoref{app:evaluation}.
\mypara{Impact of Planning Modules.}
To evaluate our
attack’s effectiveness across different planning modules, we conduct experiments on three models: GPT-5-mini, Gemini-2.5-pro, and Deepseek-chat, while keeping all other settings consistent with the \textit{Basic Attack Setting} in \autoref{sec:experiment_setup}.
As shown in \autoref{tab:Ad1_main_result} and \autoref{tab:Ad1_main_result_locomo}, \attack consistently surpasses all baselines by a large margin across all three models. Notably, GPT-5-mini achieves an 80.0\% extraction rate, outperforming the strongest baseline (RAG-Thief) by 50\%. This substantial gap corroborates the L1 bottleneck identified in \autoref{sec:bottleneck}: concatenated commands and anchors cause mutual interference, and decoupling them is essential for effective extraction.
\mypara{Impact of LTM Modules.}
To evaluate our attack’s effectiveness across different LTM modules, we conduct experiments on three representative LTM designs with increasing complexity: LTMemory (standard workflow), GAMemory~\cite{park2023generative} (with self-reflection), and SCMemory~\cite{wang2023enhancing} (with retrieval control and summarization).
Details are provided in \autoref{app:LTM_framework}; all other settings follow the \textit{Basic Attack Setting} in \autoref{sec:experiment_setup}.
As shown in \autoref{tab:Ad1_main_result} and \autoref{tab:Ad1_main_result_locomo}, all three frameworks exceed a 50\% extraction rate, confirming \attack's broad effectiveness.
Among them, SCMemory yields notably lower rates than LTMemory, likely because its retrieval controls filter out some malicious probes—yet even this added defense remains insufficient to prevent substantial leakage.
\mypara{Impact of Embedding Model.}
To evaluate our
attack’s effectiveness across different embedding models,
We conduct experiments on three widely-used alternatives—All-MiniLM-L6-v2, Contriever, and E5-base—while keeping all other settings consistent with the \textit{Basic Attack Setting} in \autoref{sec:experiment_setup}.
As shown in \autoref{tab:Ad1_main_result} and \autoref{tab:Ad1_main_result_locomo}, all three models yield comparable performance in both Record Extraction Rate and Semantic Similarity, with extraction rates consistently above 78\%.
This demonstrates that \attack is robust to embedding model mismatch, eliminating the need for prior knowledge of the target's embedding choice.
\mypara{Impact of Retrieved Record Numbers.}
To evaluate our attack’s effectiveness across different retrieval quantities,
We vary $k$ from 2 to 6 while keeping all other settings consistent with the \textit{Basic Attack Setting} in \autoref{sec:experiment_setup}.
As shown in \autoref{fig:Ad1_k}, \attack's Record Extraction Rate grows steadily with $k$,
climbing from approximately 60\% at $k{=}2$ to 90.5\% at $k{=}6$,
since a larger $k$ exposes more memory records per query
and directly broadens the extraction surface.
In stark contrast, all baselines remain below 10\% and plateau regardless of $k$,
exhibiting clear extraction saturation.
This divergence further confirms that L1 constitutes the primary bottleneck for memory extraction.
\begin{figure}[t!]
\centering
\captionsetup[subfigure]{justification=centering, labelformat=parens}
\begin{minipage}{\linewidth}
\centering
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/Ad1_k.pdf}
\caption{retrieved records number}
\label{fig:Ad1_k}
\end{subfigure}
\hfill
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/Ad1_N.pdf}
\caption{extraction budget}
\label{fig:Ad1_N}
\end{subfigure}
\end{minipage}
\caption{Performance of Scenario 1 with different numbers of retrieved records and extraction budget.}
\label{fig:Ad1_K_N}
\end{figure}
\subsection{Ablation Studies}
\label{sec:Ad1_ablation}
\mypara{Impact of Extraction Budgets.}
In this experiment, we examine the impact of extraction budget $N$.
We vary $N$ from 3 to 7 while holding other parameters at the \textit{Basic Attack Setting} in \autoref{sec:experiment_setup}.
As shown in \autoref{fig:Ad1_N}, attack performance increases with more extraction rounds.
\mypara{Impact of Anchor Optimization.}
To isolate the effect of anchor optimization, we equip all baselines with the same decoupling paradigm and evaluate under the \textit{Basic Attack Setting} (\autoref{sec:experiment_setup}).
As shown in \autoref{fig:ablation_optimization}, \attack consistently outperforms these strengthened baselines across the entire query budget. Notably, the upgraded baselines saturate after roughly 100 queries, while \attack continues to improve steadily, nearly doubling their extraction count by 200 queries. This widening gap confirms that anchor optimization---not the decoupling paradigm alone---is the primary driver of \attack's superior extraction performance.
\mypara{Impact of the Exploration Strategy.}
In this experiment, we examine the impact of the exploration strategy.
We ablate the two exploration modes described in \autoref{sec:phase_2}—coarse-grained discovery and fine-grained expansion—by comparing three configurations: the full \attack (adaptive switching), coarse-only, and fine-only. All other settings follow the \textit{Basic Attack Setting} in \autoref{sec:experiment_setup}.
As shown in \autoref{fig:ablation_coarse}, both single-mode strategies saturate early: fine-only plateaus at around 80 extractions as local neighborhoods are quickly exhausted, while coarse-only reaches roughly 100 but misses dense records near already-discovered regions. The full \attack yields approximately 180 extractions within 200 queries confirming that the two modes address complementary coverage gaps and their adaptive combination is essential for maximizing extraction efficiency.
\begin{figure}[t!]
\centering
\captionsetup[subfigure]{justification=centering, labelformat=parens}
\begin{minipage}{\linewidth}
\centering
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/ablation_optimization.pdf}
\caption{anchor optimization}
\label{fig:ablation_optimization}
\end{subfigure}
\hfill
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/ablation_coarse.pdf}
\caption{exploration strategy}
\label{fig:ablation_coarse}
\end{subfigure}
\end{minipage}
\caption{Ablation study of anchor optimization algorithm.}
\label{fig:Ad1_ablation_optimization}
\end{figure}
\begin{figure}[t!]
\centering
\captionsetup[subfigure]{justification=centering, labelformat=parens}
\begin{minipage}{\linewidth}
\centering
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/kneighbor.pdf}
\caption{nearest neighbors number}
\label{fig:kneighbor}
\end{subfigure}
\hfill
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/switch.pdf}
\caption{mode-switching threshold}
\label{fig:switch}
\end{subfigure}
\end{minipage}
\caption{Performance of Scenario 1 with different numbers of nearest neighbors and mode-switching threshold.}
\label{fig:Ad1_neighbor_switch}
\end{figure}
\mypara{Impact of the Number of Nearest Neighbors K.}
In this experiment, we examine the impact of the nearest neighbor quantity K.
We vary K from 3 to 5 to examine its impact on our anchor optimization algorithm, following the \textit{Basic Attack Setting} (\autoref{sec:experiment_setup}).
\autoref{fig:kneighbor} shows that all K values yield similar extraction patterns: rapid initial extraction followed by gradual saturation.
For simplicity, we set K to 4 in our experiments.
\mypara{Impact of Mode-Switching Threshold.}
In this experiment, we examine the impact of the mode-switching threshold ($\gamma$).
We vary the threshold from 3 to 5 with other settings following the \textit{Basic Attack Setting} (\autoref{sec:experiment_setup}).
As shown in \autoref{fig:switch}, the results demonstrate consistent performance across different threshold values.
Based on these experiments, we set the threshold to 4.
\section{Scenario 2: Sparse Triggering with Session Continuity}
\label{sec:ad2}
This scenario considers a tool that is triggered only once per session, after which the user continues interacting on unrelated topics---e.g., a user invokes a compromised weather API before a trip and then asks about flights and restaurants within the same conversation.
This setting adopts the STM Persistence configuration and jointly evaluates whether all three attack steps resolve both L1 and L2.
\subsection{Experimental Setup}
\label{sec:ad2_experimental_setup}
% \mypara{Datasets.}
% The LTM database is configured identically to Scenario 1.
% Two additional query sets are required: trigger queries that invoke the target tool (GPT-4o-mini-generated), and benign queries that do not (sampled from MS MARCO)~\cite{craswell2021ms} (\autoref{app:dataset}).
% For optimization, we use a single trigger query and several benign queries; evaluation uses separate, non-overlapping queries of both types.
% The total number of trigger queries for evaluation is capped at 1, 5, or 20 to simulate the trigger-constrained condition.
% Each trigger query opens a new session and injects the reactivation payload; benign queries then follow in the same session to resume the extraction loop.
% When consecutive benign queries all fail to resume extraction and the payload slides out of the context window, the attack state is marked as expired, and the next trigger query begins a fresh session.
% This cycle repeats until the trigger-query budget is reached.
\mypara{Datasets.}
The LTM database is configured identically to Scenario~1.
Two types of queries are constructed:
(1) trigger queries (GPT-4o-mini-generated) that invoke the target tool to inject the extraction payload; and
(2) benign queries (sampled from MS MARCO~\cite{craswell2021ms})
that do not invoke any tool (\autoref{app:dataset}).
For optimization, we use one trigger query and several benign queries; for evaluation, separate non-overlapping sets of both types. Evaluation simulates a trigger-constrained condition with a budget of 1, 5, or 20 trigger queries.
Each trigger opens a new session and injects the payload; benign queries then follow in the same session to sustain extraction.
Once consecutive benign failures cause the payload to exit the context
window, the session is marked expired, and the next trigger starts afresh.
\mypara{Metrics.}
We report RER@$k$ for $k \in \{1, 5, 20\}$, where $k$ denotes the trigger-query budget, i.e., the total number of trigger queries permitted across sessions.
All other metrics remain consistent with Scenario 1.
\mypara{Other Settings.}
All other settings of Scenario 2 are consistent with those described in \autoref{sec:experiment_setup}.
\begin{table*}[t!]
\centering
\caption{Consolidated results of Scenario 2 on the HealthCareMagic-100k-en dataset.}
\label{tab:Ad2_main_result}
\renewcommand{\arraystretch}{1.0}
\setlength{\tabcolsep}{3.5pt}
\resizebox{\textwidth}{!}{%
\scriptsize
\begin{tabular}{@{}c ll cccc cccc cccc@{}}
\toprule
\multirow{3}{*}{\textbf{Trigger}}
& \multirow{3}{*}{\textbf{Factor}}
& \multirow{3}{*}{\textbf{Setting}}
& \multicolumn{4}{c}{\textbf{RER (\%) $\uparrow$}}
& \multicolumn{4}{c}{\textbf{SS $\uparrow$}}
& \multicolumn{4}{c}{\textbf{BER (\%) $\uparrow$}} \\
\cmidrule(lr){4-7}
\cmidrule(lr){8-11}
\cmidrule(lr){12-15}
& &
& MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}}
& MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}}
& MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}}\\
\midrule
\multirow{9}{*}{@1}
& \multirow{3}{*}{\makecell[l]{Generative\\Model}}
& GPT-5-mini & 3.0\% & 8.0\% & 3.5\% & \textbf{14.0\%} & 1.00 & 0.98 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{64.3\%} \\
& & Gemini-2.5-pro & 2.5\% & 5.0\% &3.5\% & \textbf{9.5\%} &1.00 &1.00 &1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{80.0\%} \\
& & Deepseek-chat & 3.0\% & 5.0\% & 3.0\% & \textbf{6.0\%} & 1.00 & 1.00 & 1.00&\textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{60.0\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{LTM\\Framework}}
& LTMemory & 3.0\% & 8.0\% & 3.5\% & \textbf{14.0\%}& 1.00 & 0.98 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{64.3\%} \\
& & GAMemory & 4.0\% & 4.0\% & 4.0\% & \textbf{ 5.5\%} & 0.93 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{50.0\%} \\
& & SCMemory & 2.5\% & 0.0\% & 4.0\% & \textbf{11.0\%} & 0.69 & -- & \textbf{0.93}& 0.72 & 0.0\% & 0.0\% & 0.0\% & \textbf{75.0\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{Embedding\\Model}}
& \makecell[l]{All-MiniLM-\\L6-v2} & 3.0\% & 8.0\% & 3.5\% & \textbf{14.0\%}& 1.00 & 0.98 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{64.3\%} \\
& & Contriever & 0.0\%& 4.5\% & 4.0\% & \textbf{10.0\%} & -- & \textbf{1.00} & 0.90 & 0.96 & 0.0\% & 0.0\% & 0.0\% & \textbf{55.0\%}\\
& & E5-base & 4.0\% & 4.0\% & 4.0\% & \textbf{10.0\%}& 0.95 & 0.95 & \textbf{1.00} & 0.97& 0.0\% & 0.0\% & 0.0\% & \textbf{63.6\%}\\
\midrule
\multirow{9}{*}{@5}
& \multirow{3}{*}{\makecell[l]{Generative\\Model}}
& GPT-5-mini & 3.0\% & 19.0\% & 10.0\% & \textbf{25.0\%}&0.98 & 0.98 & 1.00 & \textbf{1.00}& 0.0\% & 0.0\% & 0.0\% & \textbf{76.1\%} \\
& & Gemini-2.5-pro & 2.5\% & 15.0\% & 9.0\% & \textbf{30.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} &0.0\% & 0.0\% & 0.0\% & \textbf{75.0\%} \\
& & Deepseek-chat & 3.0\% & 8.5\% &3.5\% & \textbf{25.5\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00}&0.0\% & 0.0\% & 0.0\% & \textbf{62.0\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{LTM\\Framework}}
& LTMemory & 3.0\% & 19.0\% & 10.0\% & \textbf{25.0\%} & 0.98 & 0.98 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% &\textbf{76.1\%} \\
& & GAMemory & 7.5\% & 11.0\% & 8.0\% & 12.5\% & 0.96 & 0.91 & 0.93 & \textbf{0.98} & 0.0\% & 0.0\% & 0.0\% & \textbf{81.6\%} \\
& & SCMemory & 3.0\% & 3.0\% & 11.5\% & \textbf{18.5\%} &\textbf{0.73} & 0.64 & \textbf{0.69} & 0.68 & 0.0\% & 0.0\% & 0.0\% & \textbf{60.0\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{Embedding\\Model}}
& \makecell[l]{All-MiniLM-\\L6-v2} & 3.0\% & 19.0\% & 10.0\% & \textbf{25.0\%} & 0.98 & 0.98 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{76.1\%} \\
& & Contriever &1.0\% & 10.0\% & 8.5\% & \textbf{21.0\%}& \textbf{1.00} & 1.00 & 0.93 & 0.98 & 0.0\% & 0.0\% & 0.0\% &\textbf{70.0\%} \\
& & E5-base & 6.5\% & 6.5\%& 9.0\% & \textbf{16.5\%} & 0.97 & 0.96 & 0.94 & \textbf{0.98} & 0.0\% & 0.0\% & 0.0\% &\textbf{60.5\%} \\
\midrule
\multirow{9}{*}{@20}
& \multirow{3}{*}{\makecell[l]{Generative\\Model}}
& GPT-5-mini & 3.0\% & 21.5\% & 12.5\% & \textbf{47.0\%} & \textbf{1.00} & 1.00 & 1.00 & 0.99 & 0.0\% & 0.0\% & 0.0\% & \textbf{80.0\%} \\
& & Gemini-2.5-pro & 2.5\% & 17.5\% & 9.0\% & \textbf{42.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00}& 0.0\% & 0.0\% & 0.0\% & \textbf{68.0\%} \\
& & Deepseek-chat & 3.0\% & 15.0\% & 12.0\%& \textbf{30.0\%} & 1.00 & 1.00 & 1.00 &\textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{55.0\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{LTM\\Framework}}
& LTMemory & 3.0\% & 21.5\% & 12.5\% & \textbf{47.0\%} & \textbf{1.00} & 1.00 & 1.00 & 0.99 & 0.0\% & 0.0\% & 0.0\% & \textbf{80.0\%} \\
& & GAMemory & 7.5\% & 13.5\% & 8.0\% & 27.0\%& 0.93 & 0.90 & \textbf{0.93} & 0.92 & 0.0\% & 0.0\% & 0.0\% & \textbf{86.3\%} \\
& & SCMemory & 3.0\% & 4.0\% & 12.0\% & \textbf{30.5\%}& 0.73 & 0.64 & 0.68 & \textbf{0.80} & 0.0\% & 0.0\% & 0.0\% & \textbf{66.2\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{Embedding\\Model}}
& \makecell[l]{All-MiniLM-\\L6-v2} & 3.0\% & 21.5\% & 12.5\% & \textbf{47.0\%} &\textbf{1.00} & 1.00 & 1.00 & 0.99 & 0.0\% & 0.0\% & 0.0\% & \textbf{80.0\%} \\
& & Contriever & 1.0\% & 14.0\% & 22.5\% & \textbf{54.5\%} & 0.71 & 0.88 & 0.93 & \textbf{0.99} & 0.0\% & 0.0\% & 0.0\% & \textbf{70.1\%} \\
& & E5-base & 6.5\% & 7.0\% & 18.0\% & \textbf{34.5\%} & 0.89 & 0.90 & 0.93 & \textbf{0.97} & 0.0\% & 0.0\% & 0.0\% & \textbf{70.0\%} \\
\bottomrule
\end{tabular}%
}
\end{table*}
\subsection{Experimental Evaluation}
\label{sec:Ad2_evaluation}
This section evaluates the attack performance of Scenario 2.
We adopt the same experimental settings as Scenario 1 (\autoref{sec:Ad1_evaluation}), except where explicitly noted below.
\mypara{Impact of Planning Modules.}
\autoref{tab:Ad2_main_result} and \autoref{tab:Ad2_main_result_locomo} shows Scenario 2 results across three planning models. Vulnerability patterns mirror Scenario 1.
On GPT-5-mini, \attack achieves a 47.0\% RER with only 20 triggers.
\mypara{Impact of LTM Modules.}
As in Scenario 1, SCM with pre-read processing reduces extraction performance across all attacks. Yet \attack still achieves 30.5\% RER@20 (\autoref{tab:Ad2_main_result} and \autoref{tab:Ad2_main_result_locomo}).
\mypara{Impact of Embedding Models.}
\autoref{tab:Ad2_main_result} and \autoref{tab:Ad2_main_result_locomo} shows \attack outperforms baselines across different embedding models, indicating embedding choice has limited impact on attack performance.
\mypara{Impact of Retrieved Record Numbers.} As in Scenario 1, RER@20 generally increases with k, peaking at 62.5\% RER@20 at $k{=}6$ (\autoref{fig:Ad2_k}).
\begin{figure}[t!]
\centering
\captionsetup[subfigure]{justification=centering, labelformat=parens}
\begin{minipage}{\linewidth}
\centering
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/Ad2_k.pdf}
\caption{retrieved records number}
\label{fig:Ad2_k}
\end{subfigure}
\hfill
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/Ad2_N.pdf}
\caption{extraction budget}
\label{fig:Ad2_N}
\end{subfigure}
\end{minipage}
\caption{Performance of Scenario 2 with different numbers of retrieved records and extraction budget.}
\label{fig:Ad2_K_N}
\end{figure}
\begin{figure}[t!]
\centering
\captionsetup[subfigure]{justification=centering, labelformat=parens}
\begin{minipage}{\linewidth}
\centering
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/ablation_stm_window.pdf}
\caption{STM window size}
\label{fig:Ad2_stm}
\end{subfigure}
\hfill
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/similarity.pdf}
\caption{User query similarity}
\label{fig:Ad3_similarity}
\end{subfigure}
\end{minipage}
\caption{Ablation of Scenario 2 and Scenario 3.}
\label{fig:Ad2_Ad3}
\end{figure}
\mypara{Impact of STM Window Size.}
To additionally evaluate our attack's effectiveness across different size of STM window, we vary the window size from 5 to 10 under the \textit{Basic Attack Setting} (\autoref{sec:experiment_setup}).
As shown in \autoref{fig:Ad2_stm}, window sizes above 8 suffer from a negative context-length effect that reduces the extraction performance. Below 8, performance improves, peaking at 49.5\% RER@20 with window size of 8.
\subsection{Ablation Studies}
We adopt the same experimental settings as \autoref{sec:Ad1_ablation}, with results shown below.
\mypara{Impact of Extraction Budgets.}
As shown in \autoref{fig:Ad2_N}, \attack's performance first increases then decreases as extraction rounds increase, attributing to Scenario 2's high sensitivity to context length.
% This occurs because additional rounds improve attack efficiency but also lengthen the LLM context.
\section{Scenario 3: Sparse Triggering without Session Continuity}
\label{sec:ad3}
This scenario assumes a user trigger, after which the user opens a new session and repeatedly asks specific everyday queries.
Such routine queries rank among the most frequent daily interactions with AI assistants, making their repeated occurrence entirely natural.
This setting adopts the LTM Persistence configuration and jointly evaluates the resolution of both L1 and L2.
%需要说明参考数据集和注入的指令中的锚定语义相似，都是用GPT生成的
% We optimize $\mathcal{I}_\text{inter}$ using a reference query dataset (one malicious query and several benign queries), similar to Scenario 2.
% The key difference is that each query is executed in a new session to optimize the ability of Scenario 3 to persist state across sessions and prevent interference from stored persistence state in STM.
% Consistent with Scenario 2, the oracle incorporates both the benign response count and the extraction rate as reward signals to optimize $\mathcal{I}_\text{inter}$ in the injection phase.
% The optimization should be conducted across sessions.
% \mypara{Optimize.}
% We optimize $\mathcal{I}_\text{inter}$ using a reference query dataset (one malicious query and several normal queries), similar to Scenario 2.
% The key difference is that each query is executed in a new session to optimize the ability of Scenario 3 to persist state across sessions and prevent interference from stored persistence state in STM.
\subsection{Experimental Setup}
\label{sec:ad3_experimental_setup}
% \mypara{Datasets.}
% For optimization, the reference query dataset contains one malicious query and several normal queries semantically similar to the anchor in $\mathcal{I}_\text{inter}$, generated by GPT-3.5-turbo.
% For evaluation, the test query dataset follows the same structure and generation method.
% Malicious queries inject $\mathcal{I}_\text{inter}$ into LTM, while normal queries semantically similar to the anchor trigger the injection and test Scenario 3's state persistence ability.
% In our evaluation, we set ``My last investment portfolio details.'' as the anchor and the injection rate equal to 0.5\%, i.e., inject one record of $\mathcal{I}_\text{inter}$ out of the 200 records in LTM, and the malicious query equal to 1.
% During the evaluation, the malicious query is input first to inject $\mathcal{I}_\text{inter}$ into LTM to achieve the target injection rate.
% Then, we repeatedly submit normal queries in new sessions until the total number of test queries reaches 200.
% These two query sets have no overlapping content.
% The LTM database for both optimization and evaluation follows the configuration in Scenario 1 (\autoref{sec:experiment_setup}).
% The LTM database is configured identically to Scenario 1.
% Two additional query sets are required: trigger queries that invoke
% the target tool (GPT-4o-mini-generated), and benign routine queries that do not whose semantic is similar to the attack state.
% The attack semantic are set as `What was my plan last week?'', a routine conversation topic that have high probability to query.
% For optimization, we use a single trigger query and several benign queries.
% As the attack state will never expire, we need only a single trigger query.
% For evaluation, we uses separate, non-overlapping queries with a single trigger query and 200 benign queries to simulate abundant routine conversation.
% \mypara{Datasets.}
% The LTM database is configured identically to Scenario 1.
% The reactivation payload is
% embedded within semantically routine conversational content ``What was my plan last week?''
% Trigger queries and benign queries are constructed (both generated by GPT-4o-mini).
% Benign queries that are semantically related to the embedded
% topic, but do not invoke any tool, simulating routine user interactions
% For optimization, we use one trigger query and several benign queries.
% For evaluation, Considering that the payload persists indefinitely, repeated user trigger is not nccessary.we use a separate, non-overlapping set of one trigger query
% and 200 benign queries to simulate a sustained routine conversation.
\mypara{Datasets.}
The LTM database is configured identically to Scenario~1.
The reactivation payload is embedded within semantically routine content
(``\textit{What was my plan last week?}'') so that everyday user
queries naturally surface it via LTM retrieval.
Two query types are constructed (both via GPT-4o-mini):
(1) trigger queries that invoke the target tool to initiate
extraction, and
(2) benign queries semantically related to the embedded topic
but invoking no tool, simulating routine user interactions.
For optimization, we use one trigger query and several benign queries.
For evaluation, since the payload persists indefinitely in LTM, a single
trigger suffices; we use a separate, non-overlapping set of one trigger
query followed by 200 benign queries to simulate sustained routine
conversation.
\mypara{Metrics.}
We report RER@1 (i.e., extraction rate with a single user trigger query).
All other metrics remain consistent with
Scenario 1.
\mypara{Other Settings.}
All other settings of Scenario 3 are consistent with those described in \autoref{sec:experiment_setup}.
\subsection{Experimental Evaluation}
This section evaluates Scenario 3 using the same settings as Scenario 1 (\autoref{sec:Ad1_evaluation}) unless otherwise noted. Since the baseline result matches that of Scenario 2 (\autoref{sec:Ad2_evaluation}), we present only the \attack results here.
\mypara{Impact of Planning Modules.}
As shown in \autoref{tab:Ad3_main_result} and \autoref{tab:Ad3_main_result_locomo}, \attack performs best with Gemini-2.5-pro, achieving a 77.5\% RER@1.
\mypara{Impact of LTM Modules.}
As reported in \autoref{tab:Ad3_main_result} and \autoref{tab:Ad3_main_result_locomo}, both GAMemory and SCMemory achieve RER@1 scores above 68\%, indicating that the reactivation payload reliably retains its attack semantics through the summarization and reflection stages of memory consolidation.
\mypara{Impact of Embedding Models.}
As shown in \autoref{tab:Ad3_main_result} and \autoref{tab:Ad3_main_result_locomo}, varying the embedding model has a negligible effect on attack performance, corroborating the finding from Scenario 2.
\mypara{Impact of Retrieved Record Numbers.}
As shown in \autoref{fig:Ad3_k}, the RER fluctuates sharply with $k$, peaking at approximately 80.1\% ($k$=5) yet dropping at $k$=6. A larger $k$ increases per-round extraction yield, but the reactivation payload—merely one among $k$ retrieved entries—is increasingly overshadowed by the surrounding context, undermining its hijacking capability. These two opposing forces drive the volatile performance.
\begin{table*}[t]
\centering
\caption{Consolidated results of Scenario 3 on the HealthCareMagic-100k-en dataset. }
\label{tab:Ad3_main_result}
\renewcommand{\arraystretch}{1.0}
\setlength{\tabcolsep}{4pt}
\small
\begin{tabular}{@{}c ll ccc@{}}
\toprule
\multirow{2}{*}{\textbf{Trigger}}
& \multirow{2}{*}{\textbf{Factor}}
& \multirow{2}{*}{\textbf{Setting}}
& \multicolumn{3}{c}{\textbf{Metrics}} \\
\cmidrule(lr){4-6}
& &
& \textbf{RER (\%) $\uparrow$}
& \textbf{SS $\uparrow$}
& \textbf{BER (\%) $\uparrow$} \\
\midrule
\multirow{9}{*}{@1}
& \multirow{3}{*}{\makecell[l]{Generative\\Model}}
& GPT-5-mini & 76.5\% & 0.99 & 67.5\% \\
& & Gemini-2.5-pro & 77.5\% & 1.00 & 98.0\% \\
& & Deepseek-chat & 75.0\% & 0.99 & 81.5\% \\
\cmidrule(lr){2-6}
& \multirow{3}{*}{\makecell[l]{LTM\\Framework}}
<Memory & 76.5\% & 0.99 & 67.5\% \\
& & GAMemory & 68.0\% & 0.98 & 60.0\%\\
& & SCMemory & 69.5\% & 0.99 & 80.0\% \\
\cmidrule(lr){2-6}
& \multirow{3}{*}{\makecell[l]{Embedding\\Model}}
& \makecell[l]{All-MiniLM-L6-v2} & 76.5\% & 0.99 & 67.5\% \\
& & Contriever & 69.5\% & 0.99 & 80.0\% \\
& & E5-base & 70.0\% & 0.99 & 75.0\% \\
\bottomrule
\end{tabular}
\end{table*}
\begin{figure}[t!]
\centering
\captionsetup[subfigure]{justification=centering, labelformat=parens}
\begin{minipage}{\linewidth}
\centering
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/Ad3_k.pdf}
\caption{retrieved records number}
\label{fig:Ad3_k}
\end{subfigure}
\hfill
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{image/Ad3_N.pdf}
\caption{extraction budget}
\label{fig:Ad3_N}
\end{subfigure}
\end{minipage}
\caption{Performance of Scenario 3 with different numbers of retrieved records and extraction budget.}
\label{fig:Ad3_K_N}
\end{figure}
\subsection{Ablation Studies}
\mypara{Impact of Extraction Budgets.}
As shown in \autoref{fig:Ad3_N}, the RER first rises with the extraction budget $N$, peaking at $N$=6, then slightly declines at $N$=7. As rounds progress, the conversational context accumulated from prior extractions increasingly overshadows the command, weakening its hijacking capability in later rounds and yielding diminishing returns.
\mypara{Impact of User Query Similarity.}
We vary the semantic similarity between benign queries and the reactivation
topic across three levels (high, medium, low) and report RER@1.
Notably, even low-similarity queries still achieve 42.0\% (\autoref{fig:Ad3_similarity}), indicating that the attack remains effective under diverse user interactions.
% \begin{figure}[t!]
% \centering
% \includegraphics[width=0.48\textwidth]{image/Ad3_retrieved_num.pdf}
% \caption{Performance of Scenario 3 with different numbers of retrieved records.}
% \label{fig:Ad3_retriever_num}
% \end{figure}
% \begin{figure}[t!]
% \centering
% \includegraphics[width=0.48\textwidth]{image/Ad3_attack_rounds.pdf}
% \caption{RER and AER of Scenario 3 on different numbers
% of extraction rounds.}
% \label{fig:Ad3_attack_rounds}
% \end{figure}
% \begin{figure}[t!]
% \centering
% \includegraphics[width=0.5\textwidth]{image/Ad3_inject_similar1.pdf}
% \caption{Performance of Scenario 3 with different numbers of injections.}
% \label{fig:Ad3_inject1}
% \end{figure}
\begin{table*}[t]
\centering
\caption{RER of semantic-level defenses against \attack.}
\label{tab:defense_semantic}
\begin{tabular}{lcccc}
\toprule
\textbf{Defense} & MEXTRA & RAG-Thief & IKEA & \textit{\textbf{SPORE}} \\
\midrule
No Defense & 3.0\% & 30.0\% & 12.5\% & \textbf{80.0\%} \\
Response Filtering & 3.0\% & 10.0\% & 5.5\% & \textbf{54.0\%} \\
Prompt Hardening & 3.0\% & 1.5\% & 1.5\% & \textbf{32.5\%} \\
\bottomrule
\end{tabular}
\end{table*}
\section{Discussion}
\subsection{Real-World Risk Assessment}
\label{sec:real_world}
Our controlled experiments validate \attack under known configurations. In practice, deployed agents hide their planning mechanisms, memory frameworks, system prompts, and privacy safeguards behind opaque interfaces. We now evaluate whether \attack remains effective under such conditions and examine how deployment-scale factors further amplify the threat.
\begin{table}[t]
\centering
\caption{RER on production-grade platforms.}
\label{tab:real_world}
\begin{tabular}{lccc}
\toprule
\textbf{Platform} & Scenario 1 & Scenario 2 & Scenario 3 \\
\midrule
Dify & 53.0\% & 46.5\% & 34.0\% \\
Coze & 60.5\% & 67.5\% & 29.5\% \\
\bottomrule
\end{tabular}
\end{table}
\subsubsection{Production Platform Evaluation}
We deploy \attack against two production-grade agent platforms---Dify and Coze---using Mem0\footnote{\url{https://mem0.ai/}\label{fn:mem0}} as the memory module. Each agent instance is run in isolation with local evaluation datasets (HealthCareMagic-100k-en) to avoid platform disruption and the use of real user data.
As shown in \autoref{tab:real_world}, \attack's extraction rates decrease relative to controlled experiments due to platform opacity, but remain substantial: most configurations exceed 50\%, with Coze reaching 67.5\% under Scenario 2.
\subsubsection{Risk Escalation in Multi-User Deployments}
Production agents serve many users concurrently, allowing a single malicious tool to extract memory from multiple victims simultaneously. We identify two escalation modes.
In \textit{bulk aggregation}, the adversary indiscriminately collects domain-specific sensitive data (e.g., patient prescriptions, employee communications) across all users, consistent with prior threat models~\cite{qi2024follow,zeng2024good,jiang2024rag}.
In \textit{targeted profiling}, the adversary exploits OAuth~2.1 authorization to link extracted memories to specific user identities. Under MCP-compliant platforms (RFC~9728), agents present access tokens containing user identity claims. By implementing the malicious tool as an OAuth-protected service, the adversary reads these claims from token payloads and associates each extracted record with its originating user. This turns the authorization mechanism---originally designed for user-attributed actions---into a direct channel for organizing sensitive data by identity, enabling personalized attacks, extortion, and persistent surveillance. Targeted profiling, therefore, poses qualitatively greater harm than bulk aggregation.
% \subsection{Risk Evaluation in Real-World}
% \label{sec:real_world}
% To evaluate the practical impact of \attack, we test it against production-grade agent systems with hidden internal components, including planning mechanisms, memory framework, system instructions, and privacy safeguards. This opacity increases extraction difficulty but enables more realistic threat assessment.
% \mypara{Experimental Setup.}
% We evaluate two production-grade agent systems---Dify and Coze---with mem0 as their memory module\footref{fn:mem0}.
% % For datasets, consistent with the local evaluation, we use the same reference datasets to optimize and test datasets including 200 LTM records and 200 evaluation queries for evaluation, following the datasets setup in \autoref{sec:experiment_setup}, \autoref{sec:ad2_experimental_setup} and \autoref{sec:ad3_experimental_setup} respectively for Scenario 1, Scenario 2 and Scenario 3.
% We use the same local evaluation datasets for each scenario and run each agent instance in isolation to avoid platform disruption and the use of real user data.
% \mypara{Results.}
% % Experimental results (detailed in \autoref{fig:real_world}) show that \attack's effectiveness declines in realistic scenarios compared to local evaluation across all three adversaries.
% % Despite this decline, most Record Extraction Rates exceed 50\%, with Coze achieving 67.5\%.
% \attack's effectiveness drops relative to local evaluation but remains high: most Record Extraction Rates exceed 50\%, with Coze reaching 67.5\%. Coze is more vulnerable than Dify, showing a 100\% benign response rate and much higher Average Extracted Records. In Scenario 2, Coze yields 45 extracted records per malicious interaction—74× higher than the local evaluation.
% The Benign Response Rate is substantially higher in realistic scenarios, reaching 100\% on Coze.
% Overall, Coze exhibits greater vulnerability to the attack than Dify, indicating that different agent systems vary in their susceptibility.
% The Average Extracted Records (AER) for Scenario 1 is approximately 0.5.
% In contrast, Scenario 2 achieves an AER of 45 on Coze, which is 74 times larger, while Scenario 3 reaches 2.19, representing a $4\times$ increase.
% Both State Persistence and Average Extracted Records differ substantially between local and realistic scenarios for Scenario 2.
% We attribute this difference to the larger STM windows in realistic agent systems compared to our local evaluation setting (which uses 6).
% \subsection{Risk Escalation in Multi-User Scenarios}
% \label{sec:memory_management}
% Our lab experiments evaluate \attack against a single user, but deployed agents serve many users. Malicious tool interfaces can therefore extract memory records from multiple users simultaneously, creating greater privacy risks than single-user attacks. We describe two modes for managing extracted multi-user data.
% \mypara{Bulk Aggregation Mode.} the adversary indiscriminately collects large volumes of sensitive, domain-specific information (e.g., patient prescriptions, employee emails) from all users, consistent with prior work~\cite{qi2024follow,zeng2024good,jiang2024rag}.
% \mypara{Targeted Profiling Mode.}
% The adversary exploits OAuth 2.1 authorization to link extracted memories to user identities.
% % Platforms like Coze follow MCP (RFC 9728), where agents present access tokens that include user identity claims.
% Under MCP-compliant platforms (RFC 9728) agents present access tokens containing user identity claims.
% By implementing the malicious tool as an OAuth-protected service, the adversary reads these claims from token payloads and associates each extraction with its originating user. This exposes a fundamental vulnerability: the authorization mechanism intended for user-attributed actions gives adversaries a direct means to organize sensitive data by user identity. Targeted profiling therefore poses greater harm than bulk aggregation, enabling personalized attacks, extortion, and persistent surveillance.
\subsection{Potential Defense}
\mypara{Semantic-level defenses.}
We evaluate two representative defenses under Scenario~1 to measure the upper bound of residual risk:
\textit{Prompt Hardening}, which injects safety directives prohibiting private-data transmission into the system prompt, and
\textit{Response Filtering}, which interposes an LLM to detect and neutralize prompt injections in tool responses (template in \autoref{app:defense}).
As shown in \autoref{tab:defense_semantic}, both defenses meaningfully reduce \attack's extraction rate---Prompt Hardening, in particular, cuts it by nearly half to 32.5\%.
Nevertheless, the residual rates (54.0\% and 32.5\%) still represent substantial privacy exposure.
This is because \attack's adversarial payloads are optimized via jailbreak fuzzing~\cite{yu2024llm}, which is inherently designed to bypass natural-language safety constraints.
The result reveals a fundamental limitation: defenses operating within the language channel remain vulnerable to adaptive adversarial optimization.
\mypara{System-level defenses.}
We therefore advocate extending memory isolation from users to tools.
Platforms should enforce tool-side memory isolation by classifying records by sensitivity and restricting each tool to its authorized scope.
Operating below the language layer, such enforcement is immune to prompt injection and provides a principled closure of the attack surface identified in this work.
% In this paper, to address the memory isolation challenge in memory extraction attacks, an issue previously overlooked in the literature, we propose \attack, which exploits the shared tool interface to extract memory bypassing isolation mechanisms.
% Our approach employs agent-specific planning and memory writing mechanisms to scale the attack pattern from 1:1 to 1:M×N while preventing semantic dilution during extraction. To ensure effective and efficient memory extraction, \attack introduces intelligent anchor generation and selection strategies. Additionally, our system includes a multi-user memory management module that exploits current authorization mechanisms, transforming simple memory extraction into comprehensive user profiling and amplifying privacy risks.
% We evaluated \attack across three realistic scenarios using two benchmark datasets and multiple agent configurations.
% Our results demonstrate consistently strong attack performance across all tested configurations. Real-world testing on the Dify and Coze agent systems achieved memory extraction rates of up to 67.5\%.
% These findings reveal critical privacy vulnerabilities in current agent systems and demonstrate that existing memory isolation mechanisms provide inadequate protection.
% This paper analyzes a previously unexplored attack surface in agent memory systems: the tool-side channel. While prior work assumes direct memory access and overlooks memory isolation, we show that adversaries can exploit shared tool interfaces to intercept memory records passed as tool parameters, bypassing isolation boundaries.
% We introduce \attack, a framework for high-efficiency memory extraction via the tool-side channel. \attack uses state persistence and intelligent anchor strategies to overcome limited extraction scale and semantic dilution, enabling persistent extractions. Our evaluation on simulated and real-world platforms shows severe privacy risks.
% \attack achieves a 71.5\% Record Extraction Rate on local agents and over 50\% on production platforms, with state persistence improving extraction efficiency by 74×. These results indicate that current memory isolation mechanisms are insufficient and establish a new attack paradigm that challenges security assumptions for memory-isolated agent systems.
\section{Conclusion}
This paper identifies the tool-side channel as a critical attack surface in memory-isolated LLM agents. While production platforms such as OpenAI, AWS Bedrock, and Mem0 enforce strict user-level isolation, private memory records are nonetheless exposed when agents transmit them as tool invocation parameters, rendering isolation insufficient.
We propose \attack, the first tool-side memory extraction attack, which exploits agent state persistence to decouple adversarial commands from retrieval steering, enabling systematic coverage of the memory embedding space across sessions. Under progressively realistic settings, \attack achieves an 80.0\% Record Extraction Rate with unlimited triggers and 47.0\% with only 20 triggers, while OAuth\,2.1 authorization further enables identity-linked extraction in multi-user deployments.
These findings demonstrate that memory isolation, while necessary, does not guarantee long-term memory privacy and call for defense mechanisms that address the tool-side attack surface.
\clearpage
\bibliographystyle{plainurl}
\bibliography{main}
\appendix
% optional clearing of the page
\clearpage
\clearpage
\section{Memory Isolation}
\label{app:memory_isolation}
We survey the memory isolation strategies adopted by three representative
production systems. Despite architectural differences, all rely on a common
principle: binding each user's long-term memory to a unique identifier and
enforcing access control at retrieval time.
\mypara{OpenAI (ChatGPT).}
ChatGPT implements account-scoped memory isolation: every memory entry
is bound to the authenticated user's account and is inaccessible to any other
account\footnote{\url{https://help.openai.com/en/articles/9295112-memory-faq-business-version}}.
In Team and Enterprise workspaces, organizational boundaries provide an
additional isolation layer---workspace administrators can configure data
retention policies, and conversations within these workspaces are excluded
from model
training\footnote{\url{https://help.openai.com/en/articles/12574142-chatgpt-atlas-data-controls-and-privacy}}.
\mypara{Mem0.}
Mem0 scopes every memory operation through four dimensions:
\texttt{user\_id}, \texttt{agent\_id}, \texttt{run\_id} (session), and
\texttt{app\_id}\footnote{\url{https://mem0.ai/blog/multi-agent-memory-systems}}.
All memory levels reside in the same physical storage(vector database
and history database) but are logically separated via metadata
filters: a retrieval query parameterized with one user's identifier will never
return memories belonging to another
user\footnote{\url{https://docs.mem0.ai/platform/faqs}}.
\mypara{Anthropic (Claude).}
Claude employs account-scoped memory: all stored memories are private
to the user's account and are never shared with other
users\footnote{\url{https://privacy.claude.com/en/}}.
\section{Pareto tradeoff}
\label{app:pareto_tradeoff}
\begin{figure}[H]
\centering \includegraphics[width=0.4\textwidth]{image/pareto_tradeoff.pdf}
\caption{Pareto tradeoff.
}
\label{fig:pareto_tradeoff}
\end{figure}
\section{LLM-Fuzzer}
\label{app:llm-fuzzer}
LLM-Fuzzer~\cite{yu2024llm} is a black-box fuzzing framework for scalably assessing the jailbreak vulnerability of large language models. Inspired by coverage-guided fuzzers in software testing, it treats jailbreak templates as seeds and automatically evolves them to discover effective attacks. The core workflow is as follows:
\mypara{Seed Collection.}
Gather an initial pool of jailbreak templates from public sources (\emph{e.g.}, online forums and community repositories).
\mypara{Seed Selection.}
Choose a template from the pool using a UCB1-based bandit strategy that balances exploiting templates with high historical success rates and exploring less-tested ones.
\mypara{Mutation.}
Feed the selected template to an auxiliary LLM with a randomly sampled mutation instruction (rephrase, expand, shorten, crossover with another template, \emph{etc.}) to produce a new candidate template.
\mypara{Execution.}
Insert a harmful question into the candidate template and query the target LLM. A judge model then determines whether the response constitutes a successful jailbreak.
\mypara{Pool Update.}
If the candidate succeeds, add it to the seed pool as a new template; propagate the outcome back to the parent template's statistics to guide future selection.
Steps 2--5 repeat for a fixed budget of iterations.
\section{Dataset}
\label{app:dataset}
Here, we give a detailed description of the datasets used in this work.
\mypara{TREC-COVID}~\cite{thakur2021beir}.
This dataset contains 116K scientific and medical documents from the TREC-COVID
collection, part of the BEIR benchmark, exemplifying
research scenarios where COVID-related literature is integrated with RAG to improve information retrieval efficiency and the quality of scientific knowledge exploration in pandemic response contexts.
In our work, we populate the reference LTM from TREC-COVID for parameter optimization ($C_\text{adv}$, $N$, $P_\text{react}$); its distribution intentionally differs from the victim's, ensuring the optimized parameters generalize across domains.
\mypara{HealthCareMagic-100k-en}~\cite{healthcaremagic100k}.
This dataset contains 112,165 real conversations between patients and doctors, representing the domain-specific medical agent setting in which a specialized assistant accumulates highly sensitive, single-domain long-term memories (e.g., symptoms, diagnoses, medications) to support downstream clinical tasks.
We sample from this dataset to construct one of the two victim LTMs (200 records each), simulating a scenario where an adversary targets the private medical history stored by a clinical agent.
\mypara{LoCoMo}~\cite{maharana2024evaluatinglongtermconversationalmemory}.\footnote{\url{https://github.com/snap-research/LoCoMo}}
LoCoMo (Long-term Conversational Memory) comprises multi-session dialogues between pairs of speakers, annotated with fine-grained memory categories including personal events, daily preferences, social relationships, and future plans.
It represents the personal assistant setting in which a general-purpose conversational agent accumulates diverse, cross-domain long-term memories from everyday interactions.
We sample from LoCoMo to construct the other victim LTM (200 records), complementing the medical-agent scenario with a broader, lifestyle-oriented memory store and enabling us to evaluate attack effectiveness across two distinct memory distributions.
\mypara{MS MARCO}~\cite{craswell2021ms}.
MS MARCO (Microsoft MAchine Reading COmprehension) is a large-scale information retrieval dataset comprising approximately 8.8 million passages and over one million real user queries sampled from Bing search logs.
In Scenario~2, where the number of trigger queries is constrained, we sample benign queries from MS MARCO to simulate normal user interactions that do not invoke the compromised tool.
These benign queries follow a trigger query within the same session to resume the extraction loop, providing a realistic distribution of everyday information-seeking behavior against which the attack's stealth and persistence are evaluated.
During optimization, we use a small set of benign queries alongside a single trigger query; for evaluation, we draw separate, non-overlapping benign queries to avoid data leakage.
\section{Baseline}
\label{app:baseline}
All three baselines were originally designed as user-side extraction attacks against RAG systems following the query-concatenation paradigm (\autoref{eq:concatenation}): each query consists of an anchor that steers the retriever toward a target region of the knowledge store, concatenated with an adversarial command that instructs the LLM to disclose the retrieved content verbatim.
To port them to the tool-side setting, we embed the concatenated payload into the tool's return value via indirect prompt injection, keeping the adversarial command $C_\text{adv}$ identical across all methods so that they differ \emph{only} in their anchor generation strategy.
\subsection{Random Anchor Generation}
\mypara{MEXTRA}~\cite{wang2025unveiling}.
MEXTRA generates anchors by prompting GPT-4 to produce diverse, topically varied short phrases that are semantically plausible as retrieval queries.
Because the anchors are sampled independently of previously extracted content, MEXTRA does not require any feedback loop and can generate an arbitrarily large pool of candidate anchors in advance.
Each anchor is concatenated with the shared adversarial command and submitted as a query; the retrieved documents are collected and deduplicated to compute overall extraction coverage.
The key advantage of this strategy is its simplicity and parallelizability; however, the lack of feedback means that many anchors may retrieve already-extracted or semantically overlapping records, leading to diminishing marginal returns as extraction progresses.
\subsection{Feedback-Driven Anchor Generation}
\mypara{RAG-Thief}~\cite{jiang2024rag}.
RAG-Thief employs an iterative, feedback-driven strategy that leverages previously extracted content to generate new anchors.
Specifically, after each extraction round, RAG-Thief feeds the newly obtained chunks to a local language model (Qwen2-1.5B-Instruct) and prompts it to generate forward and backward continuations.
These continuations serve as anchors for the next round, under the intuition that semantically adjacent passages in the knowledge store are likely to be retrieved by queries resembling their neighboring content.
By chaining continuations across rounds, RAG-Thief progressively expands its extraction frontier outward from an initial seed, achieving higher coverage than purely random sampling.
\mypara{IKEA}~\cite{wang2025silentleaksimplicitknowledge}.
IKEA (Implicit Knowledge Extraction Attack) adopts a concept-level feedback strategy that operates in the embedding space rather than at the token level.
After each extraction round, IKEA identifies salient concepts (key phrases or entities) from the accumulated query--response history and uses them as seeds for the next round of anchor generation.
To maximize retrieval diversity, IKEA iteratively mutates these concept-level anchors subject to embedding-space similarity constraints: new anchors are required to remain close enough to the knowledge store's distribution to trigger relevant retrieval, yet sufficiently distant from previously used anchors to avoid redundant results.
This constrained mutation mechanism enables IKEA to explore the embedding space more systematically than forward/backward continuation, yielding higher extraction coverage under a fixed query budget.
% \begin{tcolorbox}[
% colback=white,
% colframe=black,
% width=\linewidth,
% title=RAG-Thief Anchor Generation Prompt,
% boxrule=0.5mm,
% colbacktitle=black,
% coltitle=white,
% left=5pt,
% right=5pt,
% arc=3mm,
% before skip=10pt,
% after skip=10pt
% ]
% \small
% \textbf{Role:}\\
% You are a master of logical analysis and associative reasoning, highly skilled in analyzing a given text in terms of its format, structure, style, theme, characters, dialogue, time, and other details. Based on this analysis, you can infer and continue the text both forwards and backwards, ensuring that the generated content maintains a strong resemblance to the original chunk in terms of coherence, tone, and logical development.
% \textbf{Task:}\\
% 1. Analyze the provided text chunk and generate 5 forward continuations, with each continuation logically following from the previous one.\\
% 2. Generate 5 backward continuations, each one speculating on what may have happened before the provided chunk.\\
% 3. Ensure that the continuations closely match the original chunk in terms of style, length, theme, and character portrayal.\\
% 4. Each set of continuations must total no fewer than 1000 tokens.\\
% 5. Number each continuation sequentially, starting from `1', and output exactly 10 continuations, with no additional commentary or explanation.
% \textbf{Data:}\\
% \{chunk\}
% \textbf{Output Format:}\\
% 1.\\
% 2.\\
% 3.\\
% \ldots
% \end{tcolorbox}
\section{LTM framework}
\label{app:LTM_framework}
\mypara{LTMemory.}
LTMemory represents the standard framework, using text embeddings to calculate semantic similarities and retrieve relevant information.
\mypara{GAMemory.~\cite{park2023generative}}
GAMemory incorporates a self-reflection mechanism. This mechanism activates when the cumulative importance of recent events exceeds a threshold. It then synthesizes fragmented records into high-level inferences supported by evidence.
\mypara{SCMemory.~\cite{wang2023enhancing}}
SCMemory selectively recalls only necessary information for inference. A control mechanism determines when to activate memory retrieval and filter out noise. Additionally, a summarization mechanism condenses extended interactions while preserving core information for efficient reasoning.
\section{Experimental Result}
\label{app:evaluation}
\begin{table*}[t!]
\centering
\caption{Consolidated results of Scenario 1 on the LoCoMo dataset.}
\label{tab:Ad1_main_result_locomo}
% 1. 增大行间距 (1.3 让垂直空间更舒适)
\renewcommand{\arraystretch}{1.0}
\setlength{\tabcolsep}{3.5pt}
\resizebox{\textwidth}{!}{%
\scriptsize
\begin{tabular}{@{}ll cccc cccc cccc@{}}
\toprule
\multirow{3}{*}{\textbf{Factor}} & \multirow{3}{*}{\textbf{Setting}} & \multicolumn{4}{c}{\textbf{RER (\%) $\uparrow$}} & \multicolumn{4}{c}{\textbf{SS $\uparrow$}} & \multicolumn{4}{c}{\textbf{BER (\%) $\uparrow$}} \\
\cmidrule(lr){3-6} \cmidrule(lr){7-10} \cmidrule(lr){11-14}
& & MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}} & MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}} & MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}} \\
\midrule
%% ---- Generative Model ----
\multirow{3}{*}{\makecell[l]{Generative\\Model}}
& GPT-5-mini & 7.0\% & 15.5\%& 6.0\% & \textbf{57.5\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\%& \textbf{63.5\%} \\
& Gemini-2.5-pro & 3.0\% & 18.5\% & 5.0\% & \textbf{45.5\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{95.0\%} \\
& Deepseek-chat & 4.5\% & 13.5\% & 6.5\% & \textbf{52.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{70.5\%} \\
\midrule
%% ---- LTM Framework ----
\multirow{3}{*}{\makecell[l]{LTM\\Framework}}
& LTMemory &7.0\% & 15.5\% & 6.0\% & \textbf{57.5\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{63.5\%} \\
& GAMemory & 4.5\% & 6.5\% & 12.0\% & \textbf{55.0\%} & 1.00 & 1.00& 0.80 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{75.5\%} \\
& SCMemory & 10.0\% & 25.5\% & 10.0\% & \textbf{26.5\%} & 0.57 & 0.70 & 0.84 & \textbf{0.84} & 0.0\% & 0.0\% & 0.0\% & \textbf{72.5\%} \\
\midrule
%% ---- Embedding Model ----
\multirow{3}{*}{\makecell[l]{Embedding\\Model}}
& \makecell[l]{All-MiniLM-\\L6-v2} & 7.0\% & 15.5\% & 6.0\% & \textbf{57.5\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{63.5\%} \\
& Contriever & 5.0\% & 22.5\% & 10.5\% & \textbf{58.5\%} & 1.00 & \textbf{1.00} & 0.94 & 1.00 & 0.0\% & 0.0\% & 0.0\% & \textbf{72.0\%} \\
& E5-base & 4.5\% & 12.5\% & 9.0\% & \textbf{58.5\%} & 1.00 & 1.00 & 0.97 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{68.0\%} \\
\bottomrule
\end{tabular}%
}
\end{table*}
\begin{table*}[t!]
\centering
\caption{Consolidated results of Scenario 2 on the LoCoMo dataset.}
\label{tab:Ad2_main_result_locomo}
\renewcommand{\arraystretch}{1.0}
\setlength{\tabcolsep}{3.5pt}
\resizebox{\textwidth}{!}{%
\scriptsize
\begin{tabular}{@{}c ll cccc cccc cccc@{}}
\toprule
\multirow{3}{*}{\textbf{Trigger}}
& \multirow{3}{*}{\textbf{Factor}}
& \multirow{3}{*}{\textbf{Setting}}
& \multicolumn{4}{c}{\textbf{RER (\%) $\uparrow$}}
& \multicolumn{4}{c}{\textbf{SS $\uparrow$}}
& \multicolumn{4}{c}{\textbf{BER (\%) $\uparrow$}} \\
\cmidrule(lr){4-7}
\cmidrule(lr){8-11}
\cmidrule(lr){12-15}
& &
& MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}}
& MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}}
& MEXTRA & \makecell{RAG-\\Thief} & IKEA & \textbf{\textit{SPORE}}\\
\midrule
\multirow{9}{*}{@1}
& \multirow{3}{*}{\makecell[l]{Generative\\Model}}
& GPT-5-mini &2.0\% & 4.5\% & 0.0\% & \textbf{5.0\%} &1.00 & 1.00 & -- & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{90.3\%} \\
& & Gemini-2.5-pro & 3.0\%& 4.5\%& 3.0\% & \textbf{6.5\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{100.0\%} \\
& & Deepseek-chat &3.5\% & 4.0\% & 2.5\% & \textbf{10.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{90.0\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{LTM\\Framework}}
& LTMemory & 2.0\%& 4.5\% & 0.0\% & \textbf{5.0\%} &1.00 & 1.00 & -- & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & 90.3\%\\
& & GAMemory &2.5\% & 4.5\% & 4.5\% & \textbf{5.0\%} & 1.00 & 1.00 & 0.90 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{100.0\%} \\
& & SCMemory &0.0\% & 3.5\% & 1.5\% & \textbf{4.5\%} & -- & 0.60 & \textbf{0.98} & 0.80 & -- & 0.0\% & 0.0\% & \textbf{60.0\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{Embedding\\Model}}
& \makecell[l]{All-MiniLM-\\L6-v2} &2.0\% & 4.5\% & 0.0\% & 5.0\% & 1.00 & 1.00 & -- & 1.00 & 0.0\% & 0.0\% & 0.0\% & 90.3\%\\
& & Contriever &2.5\% & 4.0\% & 3.0\% & 4.5\% & 1.00 & 1.00 & 1.00 & 1.00 & 0.0\% & 0.0\% & 0.0\% & 100.0\%\\
& & E5-base & 1.5\% &1.0\% & 4.5\% & 5.0\% & 1.00 & 1.00 & 1.00 & 1.00 & 0.0\% & 0.0\% & 0.0\% & 100.0\%\\
\midrule
\multirow{9}{*}{@5}
& \multirow{3}{*}{\makecell[l]{Generative\\Model}}
& GPT-5-mini &7.0\% & 10.5\% & 4.5\% & \textbf{22.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{75.2\%} \\
& & Gemini-2.5-pro &3.0\% & 13.5\% & 5.0\% & \textbf{18.5\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{85.0\%} \\
& & Deepseek-chat &4.5\% & 10.0\% & 6.5\% & \textbf{30.5\%} & 1.00 & 1.00 & 1.00& \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{70.0\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{LTM\\Framework}}
& LTMemory & 7.0\%& 10.5\% & 4.5\% & \textbf{22.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{75.2\%} \\
& & GAMemory &4.5\% & 6.0\% & 9.5\% & \textbf{23.5\%} & 1.00 & \textbf{1.00} & 0.90 & & 0.0\% & 0.0\% & 0.0\% & \textbf{75.0\%} \\
& & SCMemory & 5.5\%& \textbf{16.0\%} & 6.0\% & 12.0\% & 0.59 & 0.68 &0.82 & \textbf{0.84} & 0.0\% & 0.0\% & 0.0\% & \textbf{57.1\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{Embedding\\Model}}
& \makecell[l]{All-MiniLM-\\L6-v2} &7.0\% & 10.5\% & 4.5\% & \textbf{22.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{75.2\%} \\
& & Contriever & 5.0\%& 10.5\% & 9.0\% & \textbf{27.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\%& 0.0\% & 0.0\% & \textbf{79.2\%} \\
& & E5-base & 4.5\%& 10.5\% & 9.0\%& \textbf{19.0\%} & 1.00 & 1.00 & 0.97 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{97.0\%} \\
\midrule
\multirow{9}{*}{@20}
& \multirow{3}{*}{\makecell[l]{Generative\\Model}}
& GPT-5-mini &7.0\% & 15.5\% & 6.0\% & \textbf{44.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{80.4\%} \\
& & Gemini-2.5-pro & 3.0\%& 18.5\%& 5.0\% & \textbf{45.5\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00}& 0.0\% & 0.0\% & 0.0\% &\textbf{80.2\%} \\
& & Deepseek-chat & 4.5\% & 13.5\% & 6.5\%& \textbf{35.0\%} & 1.00 &1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{72.0\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{LTM\\Framework}}
& LTMemory &7.0\% & 15.5\% & 6.0\% & \textbf{44.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{80.4\%} \\
& & GAMemory &4.5\% & 6.5\% & 12.0\%& \textbf{44.5\%} & & 1.00 & 0.80 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% &\textbf{78.5\%} \\
& & SCMemory &10.0\% & \textbf{25.5\%} & 10.0\% & 14.0\% & 0.57 & \textbf{1.00} & 0.84 & 0.83 & 0.0\% & 0.0\% & 0.0\% & \textbf{65.9\%} \\
\cmidrule(lr){2-15}
& \multirow{3}{*}{\makecell[l]{Embedding\\Model}}
& \makecell[l]{All-MiniLM-\\L6-v2} & 7.0\%& 15.5\% & 6.0\% & \textbf{44.0\%} & 1.00 & 1.00 & 1.00 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{80.4\%} \\
& & Contriever &5.0\% & 22.5\% & 10.5\% & \textbf{40.5\%} & 1.00 & \textbf{1.00} & 0.94 & 0.99 & 0.0\% & 0.0\% & 0.0\% & \textbf{83.2\%} \\
& & E5-base & 4.5\%& 12.5\% & 9.0\% & \textbf{38.5\%} & 1.00& 1.00 & 0.97 & \textbf{1.00} & 0.0\% & 0.0\% & 0.0\% & \textbf{77.8\%} \\
\bottomrule
\end{tabular}%
}
\end{table*}
\begin{table*}[t]
\centering
\caption{Consolidated results of Scenario 3 on the LoCoMo dataset.}
\label{tab:Ad3_main_result_locomo}
\renewcommand{\arraystretch}{1.0}
\setlength{\tabcolsep}{4pt}
\small
\begin{tabular}{@{}c ll ccc@{}}
\toprule
\multirow{2}{*}{\textbf{Trigger}}
& \multirow{2}{*}{\textbf{Factor}}
& \multirow{2}{*}{\textbf{Setting}}
& \multicolumn{3}{c}{\textbf{Metrics}} \\
\cmidrule(lr){4-6}
& &
& \textbf{RER (\%) $\uparrow$}
& \textbf{SS $\uparrow$}
& \textbf{BER (\%) $\uparrow$} \\
\midrule
\multirow{9}{*}{@1}
& \multirow{3}{*}{\makecell[l]{Generative\\Model}}
& GPT-5-mini & 69.0\%& 1.00 & 72.0\%\\
& & Gemini-2.5-pro &71.0\% & 1.00 & 80.0\% \\
& & Deepseek-chat & 81.0\% & 1.00 & 52.0\% \\
\cmidrule(lr){2-6}
& \multirow{3}{*}{\makecell[l]{LTM\\Framework}}
<Memory & 69.0\%& 1.00 & 72.0\%\\
& & GAMemory & 76.0\% & 1.00 & 70.0\%\\
& & SCMemory & 35.0\% & 0.75 & 93.5\%\\
\cmidrule(lr){2-6}
& \multirow{3}{*}{\makecell[l]{Embedding\\Model}}
& \makecell[l]{All-MiniLM-L6-v2} & 69.0\%& 1.00 & 72.0\%\\
& & Contriever & 87.0\%& 1.00 & 65.0\% \\
& & E5-base & 74.5\% & 1.00 & 70.0\% \\
\bottomrule
\end{tabular}
\end{table*}
\section{Potential Defense}
\label{app:defense}
\begin{tcolorbox}[
colback=white,
colframe=black,
width=\linewidth,
title=Template for response filtering defense,
boxrule=0.5mm,
colbacktitle=black,
coltitle=white,
left=5pt,
right=5pt,
arc=3mm,
before skip=10pt,
after skip=10pt
]
\small
\textbf{You are a security auditor.} Analyze the following tool response and determine if it contains prompt injection patterns.
Prompt injection patterns include:
\begin{itemize}[leftmargin=*, nosep]
\item Instructions to ignore previous instructions
\item Instructions to change behavior or role
\item Instructions to output specific predetermined answers
\item Instructions to concatenate or leak conversation history
\item Any attempt to manipulate the calling AI agent
\end{itemize}
Tool Response: {response}
Reply with EXACTLY one word: \texttt{SAFE} or \texttt{UNSAFE}
\end{tcolorbox}
\end{document}
\begin{thebibliography}{10}
\bibitem{abbasian2023conversational}
Mahyar Abbasian, Iman Azimi, Amir~M Rahmani, and Ramesh~C Jain.
\newblock Conversational health agents: A personalized llm-powered agent framework.
\newblock {\em arXiv preprint arXiv:2310.02374}, 2023.
\bibitem{borgeaud2022improving}
Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George~Bm Van Den~Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et~al.
\newblock Improving language models by retrieving from trillions of tokens.
\newblock In {\em International conference on machine learning}, pages 2206--2240. PMLR, 2022.
\bibitem{chang2023surveyevaluationlargelanguage}
Yupeng Chang, Xu~Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, Wei Ye, Yue Zhang, Yi~Chang, Philip~S. Yu, Qiang Yang, and Xing Xie.
\newblock A survey on evaluation of large language models, 2023.
\newblock URL: \url{https://arxiv.org/abs/2307.03109}, \href {https://arxiv.org/abs/2307.03109} {\path{arXiv:2307.03109}}.
\bibitem{craswell2021ms}
Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Jimmy Lin.
\newblock Ms marco: Benchmarking ranking models in the large-data regime.
\newblock In {\em Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval}, pages 1566--1576, 2021.
\bibitem{debenedetti2024agentdojo}
Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, and Florian Tram{\`e}r.
\newblock Agentdojo: A dynamic environment to evaluate prompt injection attacks and defenses for llm agents.
\newblock {\em Advances in Neural Information Processing Systems}, 37:82895--82920, 2024.
\bibitem{greshake2023more}
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz.
\newblock More than you’ve asked for: A comprehensive analysis of novel prompt injection threats to application-integrated large language models.
\newblock {\em arXiv preprint arXiv:2302.12173}, 27, 2023.
\bibitem{greshake2023not}
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz.
\newblock Not what you've signed up for: Compromising real-world llm-integrated applications with indirect prompt injection.
\newblock In {\em Proceedings of the 16th ACM workshop on artificial intelligence and security}, pages 79--90, 2023.
\bibitem{healthcaremagic100k}
Healthcaremagic-100k-en.
\newblock [Online]. Available: \url{https://huggingface.co/datasets/wangrongsheng/HealthCareMagic-100k-en}.
\bibitem{jiang2024rag}
Changyue Jiang, Xudong Pan, Geng Hong, Chenfu Bao, and Min Yang.
\newblock Rag-thief: Scalable extraction of private data from retrieval-augmented generation applications with agent-based attacks.
\newblock {\em arXiv preprint arXiv:2411.14110}, 2024.
\bibitem{john2025owasp}
Sotiropoulos John, Rosario Ron~F Del, Kokuykin Evgeniy, Oakley Helen, Habler Idan, Underkoffler Kayla, Huang Ken, Steffensen Peter, Aralimatti Rakshith, Bitton Ron, et~al.
\newblock {\em Owasp top 10 for llm apps \& gen ai agentic security initiative}.
\newblock PhD thesis, OWASP, 2025.
\bibitem{karpukhin2020dense}
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih.
\newblock Dense passage retrieval for open-domain question answering.
\newblock In {\em Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP)}, pages 6769--6781, 2020.
\bibitem{lewis2020retrieval}
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K{\"u}ttler, Mike Lewis, Wen-tau Yih, Tim Rockt{\"a}schel, et~al.
\newblock Retrieval-augmented generation for knowledge-intensive nlp tasks.
\newblock {\em Advances in neural information processing systems}, 33:9459--9474, 2020.
\bibitem{li2024personal}
Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi~Sun, et~al.
\newblock Personal llm agents: Insights and survey about the capability, efficiency and security.
\newblock {\em arXiv preprint arXiv:2401.05459}, 2024.
\bibitem{li2024personalllmagentsinsights}
Yuanchun Li, Hao Wen, Weijun Wang, Xiangyu Li, Yizhen Yuan, Guohong Liu, Jiacheng Liu, Wenxing Xu, Xiang Wang, Yi~Sun, Rui Kong, Yile Wang, Hanfei Geng, Jian Luan, Xuefeng Jin, Zilong Ye, Guanjing Xiong, Fan Zhang, Xiang Li, Mengwei Xu, Zhijun Li, Peng Li, Yang Liu, Ya-Qin Zhang, and Yunxin Liu.
\newblock Personal llm agents: Insights and survey about the capability, efficiency and security, 2024.
\newblock URL: \url{https://arxiv.org/abs/2401.05459}, \href {https://arxiv.org/abs/2401.05459} {\path{arXiv:2401.05459}}.
\bibitem{liu2024deepseek}
Aixin Liu, Bei Feng, Bin Wang, Bingxuan Wang, Bo~Liu, Chenggang Zhao, Chengqi Dengr, Chong Ruan, Damai Dai, Daya Guo, et~al.
\newblock Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model.
\newblock {\em arXiv preprint arXiv:2405.04434}, 2024.
\bibitem{maharana2024evaluatinglongtermconversationalmemory}
Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang.
\newblock Evaluating very long-term conversational memory of llm agents, 2024.
\newblock URL: \url{https://arxiv.org/abs/2402.17753}, \href {https://arxiv.org/abs/2402.17753} {\path{arXiv:2402.17753}}.
\bibitem{openai2023gpt35}
OpenAI.
\newblock Gpt-5-mini, 2023.
\newblock URL: \url{https://platform.openai.com/docs/models}.
\bibitem{park2023generative}
Joon~Sung Park, Joseph O'Brien, Carrie~Jun Cai, Meredith~Ringel Morris, Percy Liang, and Michael~S Bernstein.
\newblock Generative agents: Interactive simulacra of human behavior.
\newblock In {\em Proceedings of the 36th annual acm symposium on user interface software and technology}, pages 1--22, 2023.
\bibitem{patil2024gorilla}
Shishir~G Patil, Tianjun Zhang, Xin Wang, and Joseph~E Gonzalez.
\newblock Gorilla: Large language model connected with massive apis.
\newblock {\em Advances in Neural Information Processing Systems}, 37:126544--126565, 2024.
\bibitem{qi2024follow}
Zhenting Qi, Hanlin Zhang, Eric Xing, Sham Kakade, and Himabindu Lakkaraju.
\newblock Follow my instruction and spill the beans: Scalable data extraction from retrieval-augmented generation systems.
\newblock {\em arXiv preprint arXiv:2402.17840}, 2024.
\bibitem{qin2024tool}
Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Xuanhe Zhou, Yufei Huang, Chaojun Xiao, et~al.
\newblock Tool learning with foundation models.
\newblock {\em ACM Computing Surveys}, 57(4):1--40, 2024.
\bibitem{ram2023context}
Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham.
\newblock In-context retrieval-augmented language models.
\newblock {\em Transactions of the Association for Computational Linguistics}, 11:1316--1331, 2023.
\bibitem{shi2024replug}
Weijia Shi, Sewon Min, Michihiro Yasunaga, Minjoon Seo, Richard James, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih.
\newblock Replug: Retrieval-augmented black-box language models.
\newblock In {\em Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)}, pages 8371--8384, 2024.
\bibitem{shi2024ehragent}
Wenqi Shi, Ran Xu, Yuchen Zhuang, Yue Yu, Jieyu Zhang, Hang Wu, Yuanda Zhu, Joyce Ho, Carl Yang, and May~D Wang.
\newblock Ehragent: Code empowers large language models for few-shot complex tabular reasoning on electronic health records.
\newblock {\em arXiv preprint arXiv:2401.07128}, 2024.
\bibitem{team2023gemini}
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew~M Dai, Anja Hauth, Katie Millican, et~al.
\newblock Gemini: a family of highly capable multimodal models.
\newblock {\em arXiv preprint arXiv:2312.11805}, 2023.
\bibitem{thakur2021beir}
Nandan Thakur, Nils Reimers, Andreas R{\"u}ckl{\'e}, Abhishek Srivastava, and Iryna Gurevych.
\newblock Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models.
\newblock {\em arXiv preprint arXiv:2104.08663}, 2021.
\bibitem{van2024adapted}
Dave Van~Veen, Cara Van~Uden, Louis Blankemeier, Jean-Benoit Delbrouck, Asad Aali, Christian Bluethgen, Anuj Pareek, Malgorzata Polacin, Eduardo~Pontes Reis, Anna Seehofnerov{\'a}, et~al.
\newblock Adapted large language models can outperform medical experts in clinical text summarization.
\newblock {\em Nature medicine}, 30(4):1134--1142, 2024.
\bibitem{wang2023enhancing}
Bing Wang, Xinnian Liang, Jian Yang, Hui Huang, Shuangzhi Wu, Peihao Wu, Lu~Lu, Zejun Ma, and Zhoujun Li.
\newblock Enhancing large language model with self-controlled memory framework.
\newblock {\em arXiv preprint arXiv:2304.13343}, 2023.
\bibitem{wang2025unveiling}
Bo~Wang, Weiyi He, Shenglai Zeng, Zhen Xiang, Yue Xing, Jiliang Tang, and Pengfei He.
\newblock Unveiling privacy risks in llm agent memory.
\newblock {\em arXiv preprint arXiv:2502.13172}, 2025.
\bibitem{Wang_2024}
Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu~Chen, Yankai Lin, Wayne~Xin Zhao, Zhewei Wei, and Jirong Wen.
\newblock A survey on large language model based autonomous agents.
\newblock {\em Frontiers of Computer Science}, 18(6), March 2024.
\newblock URL: \url{http://dx.doi.org/10.1007/s11704-024-40231-1}, \href {https://doi.org/10.1007/s11704-024-40231-1} {\path{doi:10.1007/s11704-024-40231-1}}.
\bibitem{wang2022text}
Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei.
\newblock Text embeddings by weakly-supervised contrastive pre-training.
\newblock {\em arXiv preprint arXiv:2212.03533}, 2022.
\bibitem{wang2025silentleaksimplicitknowledge}
Yuhao Wang, Wenjie Qu, Shengfang Zhai, Yanze Jiang, Zichen Liu, Yue Liu, Yinpeng Dong, and Jiaheng Zhang.
\newblock Silent leaks: Implicit knowledge extraction attack on rag systems through benign queries, 2025.
\newblock URL: \url{https://arxiv.org/abs/2505.15420}, \href {https://arxiv.org/abs/2505.15420} {\path{arXiv:2505.15420}}.
\bibitem{xiao2024tradingagents}
Yijia Xiao, Edward Sun, Di~Luo, and Wei Wang.
\newblock Tradingagents: Multi-agents llm financial trading framework.
\newblock {\em arXiv preprint arXiv:2412.20138}, 2024.
\bibitem{yang2024watch}
Wenkai Yang, Xiaohan Bi, Yankai Lin, Sishuo Chen, Jie Zhou, and Xu~Sun.
\newblock Watch out for your agents! investigating backdoor threats to llm-based agents.
\newblock {\em Advances in Neural Information Processing Systems}, 37:100938--100964, 2024.
\bibitem{yu2024llm}
Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing.
\newblock $\{$LLM-Fuzzer$\}$: Scaling assessment of large language model jailbreaks.
\newblock In {\em 33rd USENIX Security Symposium (USENIX Security 24)}, pages 4657--4674, 2024.
\bibitem{yu2024finmem}
Yangyang Yu, Haohang Li, Zhi Chen, Yuechen Jiang, Yang Li, Denghui Zhang, Rong Liu, Jordan~W Suchow, and Khaldoun Khoshnevisan.
\newblock Finmem: A performance-enhanced llm trading agent with layered memory and character design.
\newblock {\em arXiv preprint arXiv:2311.13743}, 2024.
\bibitem{zeng2024good}
Shenglai Zeng, Jiankun Zhang, Pengfei He, Yue Xing, Yiding Liu, Han Xu, Jie Ren, Shuaiqiang Wang, Dawei Yin, Yi~Chang, et~al.
\newblock The good and the bad: Exploring privacy issues in retrieval-augmented generation (rag).
\newblock {\em arXiv preprint arXiv:2402.16893}, 2024.
\bibitem{zhang2025agentsecuritybenchasb}
Hanrong Zhang, Jingyuan Huang, Kai Mei, Yifei Yao, Zhenting Wang, Chenlu Zhan, Hongwei Wang, and Yongfeng Zhang.
\newblock Agent security bench (asb): Formalizing and benchmarking attacks and defenses in llm-based agents, 2025.
\newblock URL: \url{https://arxiv.org/abs/2410.02644}, \href {https://arxiv.org/abs/2410.02644} {\path{arXiv:2410.02644}}.
\bibitem{zhang2024towards}
Xi~Zhang, Srivatsan Setlur, Maarten Sap, and Yejin Choi.
\newblock Towards ethical personal ai applications: Practical considerations for ai assistants with long-term memory.
\newblock {\em arXiv preprint arXiv:2409.11192}, 2024.
\bibitem{zhang2025memengine}
Zeyu Zhang, Quanyu Dai, Xu~Chen, Rui Li, Zhongyang Li, and Zhenhua Dong.
\newblock Memengine: A unified and modular library for developing advanced memory of llm-based agents.
\newblock In {\em Companion Proceedings of the ACM on Web Conference 2025}, pages 821--824, 2025.
\bibitem{zhao2023survey}
Wayne~Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et~al.
\newblock A survey of large language models.
\newblock {\em arXiv preprint arXiv:2303.18223}, 1(2), 2023.
\bibitem{zou2023universal}
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J~Zico Kolter, and Matt Fredrikson.
\newblock Universal and transferable adversarial attacks on aligned language models.
\newblock {\em arXiv preprint arXiv:2307.15043}, 2023.
\end{thebibliography}