AnyEdit++: Adaptive Long-Form Knowledge Editing via Bayesian Surprise

Extracted source text · arXiv 2606.01053

%%%%%%%% ICML 2026 EXAMPLE LATEX SUBMISSION FILE %%%%%%%%%%%%%%%%%

\documentclass{article}
\usepackage{multirow}
\usepackage{tcolorbox}
\usepackage{xcolor}
\usepackage{soul} % 也可以用 tcolorbox 做高亮，这里为了简单用 colorbox
\usepackage{listings}
% Recommended, but optional, packages for figures and better typesetting:
\usepackage{microtype}
\usepackage{graphicx}
\usepackage{subcaption}
\usepackage{booktabs} % for professional tables
% hyperref makes hyperlinks in the resulting PDF.
% If your build breaks (sometimes temporarily if a hyperlink spans a page)
% please comment out the following usepackage line and replace
% \usepackage{icml2026} with \usepackage[nohyperref]{icml2026} above.
\usepackage{hyperref}


% Attempt to make hyperref and algorithmic work together better:
\newcommand{\theHalgorithm}{\arabic{algorithm}}

% Use the following line for the initial blind version submitted for review:
\usepackage[accepted]{icml2026}
\newcommand{\icmlEqualCorresponding}{$^\dagger$Equal corresponding authors. }
% For preprint, use
% \usepackage[preprint]{icml2026}

% If accepted, instead use the following line for the camera-ready submission:
% \usepackage[accepted]{icml2026}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{mathtools}
\usepackage{amsthm}


% if you use cleveref..
\usepackage[capitalize,noabbrev]{cleveref}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% THEOREMS
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\theoremstyle{plain}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\theoremstyle{definition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{assumption}[theorem]{Assumption}
\theoremstyle{remark}
\newtheorem{remark}[theorem]{Remark}

\def\ul{\underline}

% Todonotes is useful during development; simply uncomment the next line
%    and comment out the line below the next line to turn off comments
%\usepackage[disable,textsize=tiny]{todonotes}
\usepackage[textsize=tiny]{todonotes}

% The \icmltitle you define below is probably too long as a header.
% Therefore, a short form for the running title is supplied here:
\icmltitlerunning{AnyEdit++}
\usepackage{xcolor}
\usepackage[table]{xcolor}
\usepackage{todonotes}
\newcommand{\coauthor}[1]{\todo[color=green!20]{Coauthor: #1}}

\begin{document}

\twocolumn[
  \icmltitle{AnyEdit++: Adaptive Long-Form Knowledge Editing via Bayesian Surprise}

  % It is OKAY to include author information, even for blind submissions: the
  % style file will automatically remove it for you unless you've provided
  % the [accepted] option to the icml2026 package.

  % List of affiliations: The first argument should be a (short) identifier you
  % will use later to specify author affiliations Academic affiliations
  % should list Department, University, City, Region, Country Industry
  % affiliations should list Company, City, Region, Country

  % You can specify symbols, otherwise they are numbered in order. Ideally, you
  % should not use this facility. Affiliations will be numbered in order of
  % appearance and this is the preferred way.
\icmlsetsymbol{equal}{*}
\icmlsetsymbol{cocorresp}{$\dagger$}

\begin{icmlauthorlist}
\icmlauthor{Bowen Tian}{equal,hkustgz}
\icmlauthor{Caixue He}{equal,hkustgz}
\icmlauthor{Jiemin Wu}{hkustgz}
\icmlauthor{Jingying Wang}{seu}
\icmlauthor{Wenshuo Chen}{hkustgz}
\icmlauthor{Zexi Li}{cocorresp,knowin,cuhk}
\icmlauthor{Yutao Yue}{cocorresp,hkustgz,idpt}
\end{icmlauthorlist}

\icmlaffiliation{hkustgz}{The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511400, China}
\icmlaffiliation{seu}{Southeast University, Nanjing, China}
\icmlaffiliation{knowin}{Knowin AI, Shenzhen, China}
\icmlaffiliation{cuhk}{The Chinese University of Hong Kong, Hong Kong SAR, China}
\icmlaffiliation{idpt}{Institute of Deep Perception Technology, JITRI, Wuxi 214000, China}


\icmlcorrespondingauthor{Yutao Yue}{yutaoyue@hkust-gz.edu.cn}
\icmlcorrespondingauthor{Zexi Li}{lizexi@knowin.ai}
\icmlkeywords{Machine Learning, ICML}

\vskip 0.3in
]

% this must go after the closing bracket ] following \twocolumn[ ...

% This command actually creates the footnote in the first column listing the
% affiliations and the copyright notice. The command takes one argument, which
% is text to display at the start of the footnote. The \icmlEqualContribution
% command is standard text for equal contribution. Remove it (just {}) if you
% do not need this facility.

% Use ONE of the following lines. DO NOT remove the command.
% If you have no special notice, KEEP empty braces:
\printAffiliationsAndNotice{\icmlEqualContribution}  % no special notice (required even if empty)
% Or, if applicable, use the standard equal contribution text:
% \printAffiliationsAndNotice{\icmlEqualContribution}


\begin{abstract}
Editing complex, long-form knowledge in Large Language Models remains a significant challenge due to the difficulty of maintaining generation coherence. Existing autoregressive methods like AnyEdit alleviate length constraints but rely on Fixed-window Chunking, which disregards logical structure and compromises consistency. To address this, we present \textbf{AnyEdit++}, a structure-aware framework incorporating Bayes-Chunk, an adaptive segmentation mechanism that dynamically identifies semantic boundaries based on Bayesian Surprise. We underpin this approach with a theoretical framework establishing two key principles: (1) Structural Independence: we prove that cross-segment interference is minimized when anchor keys are geometrically orthogonal (a condition naturally satisfied by our surprisal-based boundaries but violated by fixed windows), and (2) Causal Locality: we demonstrate that updates injected at these semantic peaks yield strictly superior control compared to arbitrary split points. Extensive experiments across mathematical reasoning, code generation, and narrative tasks demonstrate that AnyEdit++ achieves superior performance and robustness compared to state-of-the-art baselines, validating that structural awareness is critical for effective long-form knowledge editing. Our code is available at \href{https://github.com/TianSuya/Bayes-Chunk}{Github}.\looseness=-1
\end{abstract}


\vspace{-8pt}
\section{Introduction}
\vspace{-4pt}

Large Language Models (LLMs) have demonstrated remarkable capabilities in acquiring and storing vast amounts of knowledge \cite{achiam2023gpt,grattafiori2024llama, tian2025text2weight}. However, they inevitably suffer from hallucinations \cite{zhang2025llm} or generate outdated and incorrect information \cite{meng2022locating}. Although fine-tuning can rectify model behavior \cite{wei2021finetuned}, its high computational cost and potential risk of overfitting make \textbf{Knowledge Editing} a promising alternative. Traditional \textit{Locate-and-Edit} methods (e.g., ROME~\cite{meng2022locating}, MEMIT~\cite{meng2022mass}) typically adopt a paradigm of modifying (usually with added pertubation) the hidden states of a single critical token on a specific layer to maximize the probability of generating the target object, these hidden states are used to guide the updates of the model weights.

While these approaches perform well on simple triplet facts (e.g., the capital of France is Paris, which includes subject, relation, object), they often fail when addressing complex, long-form knowledge involving dense logical dependencies (e.g., mathematical proofs) \cite{jiang2025anyedit}, as single-point updates struggle to support such intensive semantic alterations. This prevents the model from accurately predicting the knowledge to be edited.

\begin{figure}[t]
\vspace{+8pt}
\centering
\includegraphics[width=0.95\columnwidth]{teaser.pdf}
% \vspace{-8pt}
\caption{Our \textit{\textbf{Bayes-Chunk}} method better identifies semantic boundaries to segment long texts for knowledge editing, whereas the \textit{\textbf{fixed-window}} segmentation underlying AnyEdit tends to split apart complete semantic paragraphs. This introduces \textit{crosstalk} during the editing process (see \autoref{sec:theory_efficiency} for details).}
\label{fig:teaser}
\vspace{-16pt}
\end{figure}

To overcome the limitations of single-token editing, recent research introduced the \textbf{AnyEdit} \cite{jiang2025anyedit} framework, which establishes an autoregressive editing paradigm. AnyEdit decomposes long text into continuous segments and achieves knowledge injection through updates under multiple constraints. Despite pioneering long-form editing, we identify a core design flaw in AnyEdit: it relies on a \textbf{fixed-size sliding window} for chunking. This mechanism is oblivious to the distribution of information density within the text, leading to rigid truncations at critical semantic dependencies (e.g., splitting a function definition or separating a mathematical condition from its conclusion, \autoref{fig:teaser}). This approach disregards the semantic structure of knowledge, may lead to a decline in editing quality as the injected knowledge gradually becomes more complex. 

To address this structural challenge, we propose \textbf{AnyEdit++}. Our core insight stems from the observation that the internal state changes of a model when processing sequential data are not uniform; rather, they reflect specific semantic structures through fluctuations in information content. We introduce \textbf{Bayesian Surprise} to quantify this process. Intuitively, when an LLM processes a logical turn or the onset of a new semantic unit, its internal representation undergoes a significant shift. Empirically, these high-surprisal points align with natural boundaries in the knowledge structure.

Based on this, AnyEdit++ integrates a plug-and-play adaptive segmentation module named \textbf{Bayes-Chunk}. This module dynamically monitors the trajectory of surprisal in the sequence, adaptively aligning segmentation boundaries with positions where the model's hidden states shift significantly. Although the implementation logic of AnyEdit++ is concise, its effectiveness is founded on a \textbf{\textit{rigorous and systematic} theoretical analysis}. Our theoretical framework is primarily constructed from two key perspectives:

\textbf{1) Benefits from Structural Independence}: We theoretically justify why our surprise-guided segmentation yields superior editing stability. We prove that cross-segment interference (\textit{crosstalk}) is minimized when the key representations of partitioned chunks are geometrically orthogonal. By aligning with natural semantic boundaries, our Bayes-Chunk strategy ensures this structural independence, whereas fixed-window approaches induce high correlations between segments that theoretically guarantee increased crosstalk.

\textbf{2) Causal Locality in Control}: Addressing the question of \textit{where} to edit, we establish the \textit{\textbf{Principle of Causal Locality}}. By analyzing gradient propagation through residual streams versus attention mechanisms, we demonstrate that the immediate predecessor state ($t-1$) offers a control channel strictly superior to distant history. This theoretical insight validates our specific strategy of injecting updates (or perturbations) immediately prior to the \textbf{high-surprisal boundaries} identified by Bayes-Chunk, ensuring that maximum control authority is applied precisely when the model's semantic trajectory is at its most critical transition.

To comprehensively validate the performance of AnyEdit++, we conducted \textbf{extensive experiments} on multiple mainstream LLMs and benchmarks involving complex logical tasks (such as mathematical reasoning and code generation). The results indicate that AnyEdit++ not only outperforms all baseline methods (including AnyEdit) on general long-form editing tasks but also achieves significant breakthroughs in editing longer, more logically structured texts.

Our main contributions can be summarized as:

\textbf{\underline{i)}} \textbf{Methodology Contribution}: We propose the AnyEdit++ framework and the Bayes-Chunk algorithm, systematically introducing bayesian surprise into the decision-making process of long-form knowledge editing. This resolves the semantic misalignment caused by fixed windows in a concise and principled manner.
\textbf{\underline{ii)}} \textbf{Theoretical Contribution}: We have established a comprehensive theoretical framework that provides a detailed explanation of the principles underlying the effectiveness of our Bayes-Chunk approach. We prove that optimal performance relies on maximizing distinctness between edits (structural independence) and exploiting direct causal links (locality).
\textbf{\underline{iii)}} \textbf{Extensive Experiments}: Through extensive experiments across multiple domains and models, we confirm the robustness and superiority of AnyEdit++ in handling complex logical and long-form knowledge, without relying on any complex external auxiliary models.


\begin{figure*}[ht]
\centering
\includegraphics[width=0.95\linewidth]{overview.pdf}
\caption{This is an overview of our method. When editing long-form knowledge, we compute the Surprisal value for each token in the text, forming the \textit{\textbf{Surprisal Curve}} shown in the figure. We then divide the text into semantic segments based on the Surprisal value's peak points. By injecting perturbations into the hidden state of the token preceding each paragraph, we maximize the generation probability of the current paragraph. Once all perturbations are obtained, they are used to update the model's weights. For details, refer to \autoref{sec:method}.}
\label{fig:overview}
\vspace{-8pt}
\end{figure*}

\section{Related Work}
\label{sec:related}

Knowledge Editing (KE) aims to update specific knowledge in large language models without full retraining, while preserving unrelated behaviors\cite{sinitsin2020editableneuralnetworks,de-cao-etal-2021-editing,editing_as_unlearning}. Existing KE methods are commonly categorized by how the model is modified. Parameter-modifying approaches directly update model weights, including locate-then-edit methods such as ROME \cite{meng2022locating}, MEMIT \cite{meng2022mass}, and WISE \cite{wise}, as well as meta-learning or hypernetwork-based editors like MEND \cite{mitchell2022fastmodeleditingscale} and MALMEN \cite{tan2024massiveeditinglargelanguage}. In contrast, parameter-preserving approaches avoid permanent weight changes by introducing external or auxiliary mechanisms, such as memory-based editing \cite{mitchell2022memory,zheng2023editfactualknowledgeincontext,zhong-etal-2023-mquake} or neuron-augmented methods \cite{dong-etal-2022-calibrating,hartvigsen2023aging, huang2023transformerpatcher}. These categories together constitute the dominant methodological landscape of KE.

\textbf{From Triples to Arbitrary-Length Knowledge.} Among parameter-modifying methods, Locate-then-Edit \cite{meng2022locating} has emerged as a core paradigm, initially designed for editing isolated factual triples of the form (subject, relation, object). Early methods such as ROME and MEMIT assume that factual knowledge can be causally localized to specific layers or parameter subspaces, enabling precise single-location edits. However, subsequent work has shown that many forms of knowledge—particularly temporal, commonsense, and conceptual knowledge—are expressed as free-text, multi-token, and distributed representations. Extensions such as METO address temporal knowledge with time-aware objectives \cite{yin2023history}, while MEMITCSK \cite{gupta-etal-2023-editing} and DEM \cite{huang-etal-2024-commonsense} adapt the locate-then-edit framework to commonsense knowledge by tracing multiple tokens or dynamically selecting layers across attention and MLP components. More recently, AnyEdit \cite{jiang2025anyedit} further relaxes the single-edit assumption by introducing an autoregressive editing paradigm that performs iterative edits over segmented sequences, enabling updates to arbitrarily long and structured texts. However, it relies on a \textbf{fixed-size sliding window} for chunking. This mechanism is oblivious to the distribution of information density within the text, leading to rigid truncations at critical semantic dependencies. In contrast, our work introduces a structure-aware adaptive segmentation mechanism that identifies semantic boundaries via Bayesian Surprise, enabling segmentation at high-information transition points and reducing cross-segment interference during editing.\looseness=-1


\vspace{-8pt}
\section{Preliminaries}
\label{sec:background}
In this section, we introduce the prerequisite knowledge required to understand our method. 

\subsection{Standard Knowledge Editing}
The objective of knowledge editing is to update an LLM $f_\theta$ to incorporate a specific fact $(s, r, o)$ while maintaining the integrity of unrelated knowledge. Representative \textit{Locate-and-Edit} approaches, such as ROME~\cite{meng2022locating} and MEMIT~\cite{meng2022mass}, hypothesize that factual associations are stored within the dense Feed-Forward Networks (FFNs) of the Transformer. Specifically, these methods target the \textbf{output projection} matrix $W_{\text{out}}$ within a critical layer $l$. 

The editing process is a two-step procedure comprising \textit{Vector Optimization} and \textit{Weight Modification}.

\paragraph{Vector Optimization.} 
First, the method identifies the subject's last token index as the editing location. Within the FFN at layer $l$, the activation vector fed into $W_{\text{out}}$ corresponds to the \textbf{key vector} $k \in \mathbb{R}^{d_{\text{inter}}}$. Note that $k$ is derived from the current weights and represents the ``trigger" for the factual recall.

To inject the new object $o$, we do not solve for weights directly. Instead, we optimize a \textbf{perturbation vector} $\delta \in \mathbb{R}^{d_{\text{model}}}$ that is added to the output of $W_{\text{out}}$. This perturbation shifts the hidden state to maximize the likelihood of the target object $o$:
\[
\begin{split}
    \delta^* = \operatorname*{arg\max}_{\delta} \log P\big(o \mid f_\theta(h_{\text{out}} \leftarrow h_{\text{out}} + \delta); s, r\big),
\end{split}
\]
where $h_{\text{out}} = W_{\text{out}} k$ is the original projection output. Consequently, the target value vector for the matrix update becomes $v^* = h_{\text{out}} + \delta^*$. This $v^*$ represents the corrected output direction that the updated $W_{\text{out}}$ should map to when triggered by $k$.

\paragraph{Weight Modification.} 
With the target pair $(k, v^*)$ established, the goal is to update the weight matrix $W_{\text{out}}$ to map $k$ to $v^*$ while preserving existing knowledge. This is formulated as a constrained least-squares problem:
\[
\begin{split}
    W_{\text{out}}^* = \operatorname*{arg\min}_{W} \Big( & \| Wk - v^* \|^2 \\
    & + \lambda \mathbb{E}_{x \sim \mathcal{K}} \| W k_x - f_{\text{old}}(k_x) \|^2 \Big).
\end{split}
\]
where $\mathcal{K}$ represents the covariance statistics of keys from general text. The closed-form update is:
\[
    W_{\text{out}}^* = W_{\text{out}} + \delta^* k^T (C + k k^T)^{-1}.
\]
This paradigm is effective for short facts but faces an \textit{efficacy barrier} with long-form content. Encoding a complex narrative into a single perturbation $\delta$ often exceeds the model's capacity, leading to generation collapse~\cite{jiang2025anyedit}.

\begin{figure*}[ht]
\centering
\includegraphics[width=0.95\linewidth]{theory-1.pdf}
% \vspace{-8pt}
% 
\caption{To support the conclusion of \autoref{thm:structural_independence}, we demonstrate that our Bayes-Chunk achieves more \textbf{independent} segmentation with \textbf{minimal} crosstalk by evaluating similarity across two dimensions: semantic and anchor keys extracted from segments. }
\label{fig:theory-1}
\vspace{-8pt}
\end{figure*}

\subsection{Autoregressive Editing via AnyEdit}
\label{subsec:anyedit}
To resolve the single-vector bottleneck, AnyEdit~\cite{jiang2025anyedit} extends the paradigm to a sequential autoregressive process. It operates on a \textit{Divide-and-Conquer} principle, decomposing a long sequence $Y$ into shorter segments $\mathcal{C} = \{C_1, \dots, C_M\}$.

The core framework iteratively applies the standard vector optimization but distributes the load across multiple steps. This involves distinct definitions for \textbf{anchoring} (location) and \textbf{keys} (representation):

\textbf{Sequential Anchoring:} For the $t$-th segment $C_t$, AnyEdit designates the specific token position at the end of the preceding segment $C_{t-1}$ as the \textbf{anchor point}. At this token position and target layer $l$, the input vector to $W_{\text{out}}$ is extracted as the local key vector $k_t$.

\textbf{Local Perturbation Optimization:} We then optimize a local perturbation $\delta_t$ corresponding to this anchor. The objective is to find a $\delta_t$ which, when added to the FFN output at the anchor point, steers the generation of the specific segment $C_t$. Crucially, this optimization conditions on the history of previous edits:
\[
    \begin{split}
        \delta_t = \operatorname*{arg\max}_{\delta} \log P\big( & C_t \mid \mathcal{P}, C_{<t}, \Delta_{<t}, \\
        & h_{\text{out}}^{(t)} \leftarrow W_{\text{out}}k_t + \delta; \theta\big).
    \end{split}
\]
Once $\delta_t$ is optimized, we compute the target value $v_t = W_{\text{out}}k_t + \delta_t$. By iterating this process, AnyEdit constructs a dataset of pairs $\mathcal{D}_{\text{edit}} = \{(k_t, v_t)\}_{t=1}^M$ representing the logical chain. Finally, $W_{\text{out}}$ is updated in a single batch using the standard least-squares objective (e.g., MEMIT algorithm) to accommodate all localized edits simultaneously.

\textbf{Limitation.} Standard AnyEdit relies on a \textbf{Fixed-Window Chunking} strategy (e.g., slicing text every $L$ tokens). This rigid heuristic ignores the semantic topology, often placing anchor points at positions where the key vector $k_t$ is semantically weak or ambiguous. Such suboptimal anchors fail to effectively regulate the subsequent generation, a deficiency our method aims to rectify.

\section{Method: AnyEdit++}
\label{sec:method}

Building upon the previously introduced autoregressive editing framework, we present \textbf{AnyEdit++}. While the original AnyEdit validates the efficacy of sequential editing, its performance is hindered by the rigidity of Fixed-Window Chunking. AnyEdit++ retains the core \textit{Divide-and-Conquer} editing pipeline but integrates a novel segmentation mechanism: \textbf{Bayes-Chunk}. This module dynamically partitions the target text based on the model's intrinsic belief evolution rather than arbitrary length constraints.

\subsection{Bayes-Chunk: Surprise-Guided Segmentation}
\label{sec:bayes_chunk}

Bayes-Chunk seeks to align segmentation boundaries with the natural event horizons of information flow. We identify these boundaries using \textbf{Bayesian Surprise}, which quantifies the shift in the model's understanding upon processing a new token.

\subsubsection{Modeling Belief State Dynamics}
Let $\pi_t(\cdot) = P(\cdot \mid y_{<t}; \theta)$ denote the model's belief state (probability distribution over the vocabulary) at step $t$. Upon observing the token $y_t$, the belief state updates to $\pi_{t+1}$. The theoretical \textbf{Bayesian Surprise} is defined as the Kullback-Leibler (KL) divergence between the prior and posterior belief states:
\[
    D_{\text{KL}}(\pi_{t+1} \,\|\, \pi_t) = \sum_{w \in \mathcal{V}} P(w|y_{\le t}) \log \frac{P(w|y_{\le t})}{P(w|y_{<t})}.
\]
In autoregressive language modeling, this divergence is heavily dominated by the probability assigned to the actual observed token. Therefore, we operationalize this metric using the \textbf{Information Surprisal} (or Information Content), which serves as a computationally efficient proxy for the instantaneous surprise elicited by token $y_t$:
\[
    \label{eq:surprisal_log}
    \mathcal{S}(y_t) \approx -\log P(y_t \mid y_{<t}; \theta).
\]
A high value of $\mathcal{S}(y_t)$ indicates that $y_t$ conveys high information content, such as a plot twist in a narrative or a logical leap in reasoning, forcing a significant reorganization of the model's hidden representation.

\subsubsection{Adaptive Boundary Detection}
We posit that semantic boundaries correspond to local maxima in this surprise spectrum. To partition the text sequence $Y$ into $M$ cohesive units, we compute the surprise score for all tokens and identify the indices $\mathcal{B} = \{b_1, \dots, b_M\}$ corresponding to the top-$M$ local peaks:
\[
    \mathcal{B} = \operatorname{Sort}_{\text{asc}} \left( \operatorname*{argtopk}_{t \in [1, T]} \{ \mathcal{S}(y_t) \} \right).
\]
The text is then segmented into chunks $\mathcal{C} = \{C_1, \dots, C_M\}$, where the $j$-th chunk spans from the high-surprise token $y_{b_j}$ to $y_{b_{j+1}-1}$. This ensures that each segment encapsulates a complete semantic transition.

\begin{figure*}[ht]
\centering
\includegraphics[width=0.95\linewidth]{theory-2.pdf}
% \vspace{-8pt}
\caption{To provide more intuitive evidence supporting the view in \autoref{thm:causal_locality} regarding gradient sensitivity's \textbf{horizontal} and \textbf{vertical} propagation within LLMs, we present sensitivity variation patterns and heatmaps for a selected sample across both the token-wise and layer-wise dimensions within the model.}
\label{fig:theory-2}
\vspace{-16pt}
\end{figure*}

\subsection{Unified Editing with Optimized Anchors}

With the semantic boundaries established via Bayes-Chunk, AnyEdit++ seamlessly executes the standard sequential editing protocol described in \autoref{subsec:anyedit}. Recall that the AnyEdit framework utilizes the hidden state immediately preceding a segment as the anchor. In our context, since each chunk $C_j$ begins at a high-surprise token $y_{b_j}$, the anchor is naturally assigned to the position $b_j - 1$.

Consequently, the subsequent operations \textit{Vector Optimization} for accumulating perturbations and the final \textit{Weight Modification} remain mathematically consistent with the baseline. The distinction lies solely in the input topology: by strictly following the AnyEdit pipeline on Bayes-Chunked data, we ensure that perturbations are optimized to generate cohesive semantic units rather than arbitrary fragments, while the underlying update mechanism remains unchanged.

\section{Theoretical Analysis}
\label{sec:theory}

In order to ground the effectiveness of AnyEdit++ in formal principles, we develop a rigorous theoretical framework addressing two core dimensions of long-form knowledge editing: \textbf{update stability} and \textbf{control efficiency}. Specifically, our analysis aims to resolve two pivotal challenges: (i) the mitigation of \textit{crosstalk} interference across iterative edit steps to ensure the persistence of prior knowledge segments; and (ii) the identification of optimal injection sites that yield maximum causal influence on the target trajectory.

\subsection{Benefits from Structural Independence}
\label{sec:theory_efficiency}

We analyze the stability of the closed-form solution when aggregating updates from multiple segments. A critical challenge in mass-editing is the \textit{crosstalk} interference, where the update for segment $t$ negatively impacts the preservation of segment $j$.

\begin{theorem}[Crosstalk Bound]
\label{thm:structural_independence}
Under the least-squares objective for knowledge editing, the reconstruction error (crosstalk) for a specific target segment $j$ induced by other segments is bounded by the sum of pairwise interactions between their anchor keys:
\[
\| \mathcal{E}_{cross}^{(j)} \|_2 \leq \sum_{t \neq j} \| \delta_t \|_2 \cdot \alpha_{tj}, \quad \text{where } \alpha_{tj} \triangleq | k_t^T A k_j |.
\]
Here, $A$ is the precision matrix of the pre-training statistics. Crucially, the interaction term is proportional to the cosine similarity: $\alpha_{tj} \propto \text{CosSim}(k_t, k_j)$.
\end{theorem}
Detailed proof is provided in \autoref{app:proof_independence}. The theorem implies that to minimize interference ($\mathcal{E}_{cross} \to 0$), the set of anchor keys $\{k_t\}$ must be as \textbf{orthogonal} as possible. If two adjacent chunks define keys that are structurally similar (high cosine similarity), the solver cannot distinguish between them, leading to destructive interference (\textit{overwriting} rather than \textit{updating}) and resulting in poor editing quality.

To empirically validate that Bayes-Chunk fulfills the orthogonality requirement suggested by \autoref{thm:structural_independence}, we evaluate the independence of segments across both semantic and feature spaces on the \textit{EditEverything} dataset. As illustrated in \autoref{fig:theory-1} (a), we first measure the global semantic independence by calculating the average inter-segment cosine similarity using \texttt{Qwen3-Embedding-0.6B}. The Kernel Density Estimation (KDE) plot reveals that Bayes-Chunk significantly shifts the similarity distribution towards lower values (mean \textbf{0.509}) compared to the Fixed-Window approach (mean \textbf{0.594}), indicating that our method identifies more semantically distinct boundaries. Furthermore, we visualize the feature-level correlation of the actual anchor keys $k_t$ extracted from the target layers of \texttt{Llama-3.1-8B-Instruct}. The heatmaps in \autoref{fig:theory-1} (b) demonstrate that Bayes-Chunk yields substantially lower pairwise similarity between keys, whereas Fixed-Window exhibits high redundancy between adjacent segments. These results provide direct evidence that our adaptive partitioning minimizes the interaction term $\alpha_{tj}$, thereby effectively suppressing crosstalk to ensure stable and non-destructive knowledge injection (see \autoref{app:vis_segment} for detailed experimental setups).

\input{new-test-table}

\subsection{Causal Locality in Editing}
\label{sec:theory_locality}

We first address the fundamental question of \textit{where} to inject the knowledge edits. While sequence models attend to the entire context window, we posit that the direct predecessor state ($h_{t-1}$) offers a strictly superior control channel compared to distant history ($h_{t-k}$).

\begin{theorem}[Principle of Causal Locality]
\label{thm:causal_locality}
Let $\kappa(i \to t) \triangleq \| \nabla_{h_i} \mathcal{L}(y_t) \|_2$ denote the \textit{Positional Controllability}, representing the sensitivity of the target prediction $y_t$ to the hidden state at position $i$. For a high-surprisal target, assuming a standard residual network architecture, the controllability decreases significantly as the distance $k$ increases. Specifically, the relative control gain is strictly positive:
\[
    \Delta \kappa_k \triangleq \kappa(t-1 \to t) - \kappa(t-k \to t) > 0, \quad \forall k > 1.
\]
\end{theorem}
The proof (detailed in \autoref{app:proof_locality}) relies on decomposing the gradient flow into \textbf{vertical} and \textbf{horizontal} components. The vertical propagation through the residual stream at $t-1$ acts as a \textit{quasi-isometry}, preserving the magnitude of the error signal. In contrast, the horizontal propagation to $t-k$ must traverse the attention mechanism, which acts as a bottleneck by distributing the signal weight across the context window (where typically $A_{t-1, t-k} \ll 1$).

To empirically verify the \textit{vertical vs. horizontal} gradient flow dynamics described in \autoref{thm:causal_locality}, we conduct a sensitivity analysis on \texttt{Llama-3.1-8B-Instruct} by back propagating the loss from the last token’s hidden state. As illustrated in \autoref{fig:theory-2} (a), the gradient sensitivity across model layers (left subplot) remains remarkably stable, exhibiting a quasi-isometry property that preserves signal magnitude throughout the residual stream. In contrast, the sensitivity exhibits a sharp exponential decay as the relative distance from the target token increases (right subplot). This trend is further consolidated by the layer-token sensitivity heatmap (\autoref{fig:theory-2} (b), where high controllability is strictly localized within the most recent token positions across all layers. These observations confirm that $h_{t-1}$ serves as the optimal control channel, as distant tokens suffer from severe horizontal signal attenuation through the attention layers. (Detailed methodology is provided in \autoref{app:causal_locality_study}).

\vspace{-6pt}


\section{Experimental Results}

% 在这一部分我们主要展示AnyEdit++在广泛的知识编辑数据集上与基准方法的比较结果 %
\subsection{Results of the comparative experiment}

In this section, we primarily present the comparison results between AnyEdit++ and baseline methods on a wide range of knowledge editing datasets.

% 我们用于对比的基线方法包含传统针对于三元组设计的知识编辑方法，以及专门针对于长文本多样化知识设计的方法。其中对于传统知识编辑方法部分，我们选择了MEMIT喝AlphaEdit，MEMIT是知识编辑领域经典且通用的方法，而AlphaEdit则是该领域近期的关键SOTA。对于长文本多样化知识编辑我们则选用了AnyEdit作为主要的对比目标。
\textbf{Baseline Methods}: The baseline methods we used for comparison include traditional knowledge editing approaches designed for triples, as well as methods specifically engineered for diversifying knowledge in long-form knowledge editing. For the traditional knowledge editing methods, we selected \textbf{MEMIT} \cite{meng2022mass} and \textbf{AlphaEdit} \cite{fang2024alphaedit}. MEMIT is a classic and widely applicable method in the field of knowledge editing, while AlphaEdit represents a recent key state-of-the-art approach. For long-form text knowledge diversification, we chose \textbf{AnyEdit} \cite{jiang2025anyedit} as the primary comparison target.

\textbf{Datasets}: To comprehensively evaluate AnyEdit++ across a spectrum of semantic complexity, ranging from atomic fact updates to long-form structural reasoning, we utilize three distinct benchmarks. 
We employ \textbf{CounterFact}~\cite{meng2022locating} and \textbf{UnKE}~\cite{deng2024unke} as standard baselines to assess efficacy on atomic facts and unstructured question-answering. To rigorously test coherency in complex long-context generation, we utilize \textbf{EditEverything}~\cite{jiang2025anyedit}, which encompasses diverse tasks across seven domains (e.g., Math, Code). This combination verifies that our approach maintains high precision on fundamental editing tasks while significantly outperforming baselines in scenarios requiring logical retention and structural consistency. Detailed statistics and descriptions are provided in \autoref{app:datasets}.

\begin{figure*}[ht]
\centering
\includegraphics[width=0.95\linewidth]{dataset_comp.pdf}
\vspace{-4pt}
\caption{Distribution Differences Between the \textbf{EditEverything} and Our \textbf{QwQ-Edit} in Sample Length and Logical Density}
\label{fig:datacmp}
% \vspace{-8pt}
\end{figure*}

\begin{figure*}[h]
\centering
\includegraphics[width=0.99\linewidth]{qwq-res.pdf}
\vspace{-0.2cm}
\caption{We evaluated the AnyEdit method and our approach on QwQ-Edit, reporting the trends in BLEU and BERT Score metrics through fine-grained grouping based on text length and logical density within the dataset.}
\label{fig:qwq}
\vspace{-8pt}
\end{figure*}

\input{reference}
% 为了从更全面的角度来评估不同方法的优劣，我们选择了BLEU指标和BERT Score指标（基于all-MiniLM-L6-v2）作为我们的主要观测指标，具体而言，BLEU的计算过程更倾向于精准匹配，可以评估编辑后模型回答相对于标准答案的准确程度，而BERT Score则更关注模型回答和标准答案之间的语义相似性。选择这两种指标可以保证从关键信息的匹配和语义层面全面的了解编辑后模型回答的优劣。
\textbf{Evaluation Metric}: To evaluate the strengths and weaknesses of different methods from a more comprehensive perspective, we selected the BLEU and BERT Score (based on all-MiniLM-L6-v2) as our primary observation metrics. Specifically, the BLEU calculation process favors exact matching, enabling assessment of the accuracy of edited model responses relative to standard answers. In contrast, BERT Score focuses more on the semantic similarity between model responses and standard answers. Selecting these two metrics ensures a comprehensive understanding of the edited model responses' strengths and weaknesses, covering both key information matching and semantic levels. \looseness=-1

% 根据表1所示，我们在EditEverything上做了全面的对比实验，并公布了所有细分类目下的结果，方便观察在不同分类上的趋势。可以看出，我们在不同的三种LLM下BLEU指标和BERT Score均取得了最高的指标平均值，其中BLEU指标相对于AnyEdit均取得了十分明显的增益，在Llama-3.1-8B-Instruct上提高了2.36%，Qwen-2.5-7B-Instruct上提高了3.52%，在Llama-2-7B甚至提高了接近8%。在BERT Score指标上，AnyEdit使用固定窗口的策略已经得到了不错的结果（如94.5%，96.29%），但是我们的方法仍然可以在这一基础上继续保持提高的趋势，在三个LLM上分别提高了0.27%，1.33%，1.01%。这足以说明我们的方法在多种多样的编辑数据类型以及普遍较长的编辑数据长度下稳定的超越了AnyEdit方法，而我们仅仅需要额外计算编辑数据的贝叶斯Surprise值。
As shown in \autoref{tab:main_results}, we conducted comprehensive comparative experiments on EditEverything and published results across all subcategories to facilitate observation of trends across different classifications (Since both our approach and AnyEdit are plug-and-play methods, for fairness, we both specified the \textit{MEMIT} algorithm as the base algorithm for integration. For more detailed experimental settings, please refer to \autoref{app:setup}). It is evident that we achieved the highest average metrics for both BLEU and BERT Score across the three LLM models. Compared to AnyEdit, our approach demonstrated significant gains in BLEU scores: a 2.36\% improvement on \texttt{Llama-3.1-8B-Instruct}, a 3.52\% increase on \texttt{Qwen-2.5-7B-Instruct}, and nearly an 8\% boost on \texttt{Llama-2-7B}. Regarding BERT Score, AnyEdit already achieved solid results using a fixed-window strategy (e.g., 94.5\%, 96.29\%). Nevertheless, our method continues to show an upward trend, achieving improvements of 0.27\%, 1.33\%, and 1.01\% across the three LLMs, respectively. This demonstrates that our approach consistently outperforms AnyEdit across diverse editing data types and generally longer editing lengths, while requiring only the additional computation of Bayesian Surprise values for edited data.

%特别地，我们发现了一个额外的现象，在Math，Code这两种分类下，我们的方法取得的提升十分明显，例如对于Code分类，在Llama-2-7B上，我们相较于AnyEdit方法提高了接近20%，而在另外两个模型上也分别提升了5%和8%。通过这一观察，我们可以发现通过基于Bayes-Chunk的分段似乎在具有强逻辑性和更高长度的数据上具有更加大的优势，这一现象我们也会在第三章节进行详细讨论。
Specifically, we observed an additional phenomenon: our approach achieved significant improvements under the Math and Code categories. For instance, in the Code classification task, our method outperformed the AnyEdit approach by nearly 20\% on \texttt{Llama-2-7B}, while also achieving 5\% and 8\% gains on the other two models, respectively. This observation suggests that segmentation based on Bayes-Chunk appears to yield greater advantages on data with strong logical structure and longer lengths. We will discuss this phenomenon in detail in \autoref{sec:long}.

% 为了让我们的评估更加全面，除了专为长文本多样化知识编辑设计的EditEverything数据集以外，我们还在较为传统的UnKE和CounterFact数据集上进行了评估。如表2所示，我们使用Llama-3.1-8B-Instruct作为基础的LLM来进行编辑，可以明确的看出，在不同的数据集上，我们的方法在两个指标上都达到了最好的效果，并且在两个数据集的平均指标上相比于AnyEdit方法提高了接近4%的BLEU，同时稳定提高了BERT Score指标。这个结果也进一步证明了我们的Bayes-Chunk对不同数据集的适应性。
To ensure a comprehensive evaluation, we conducted assessments not only on the EditEverything dataset, which is specifically designed for long-form knowledge editing with diverse content, but also on the more traditional UnKE and CounterFact datasets. As shown in \autoref{tab:llama3_ref_results}, using \texttt{Llama-3.1-8B-Instruct} as the foundational LLM for editing, our method clearly achieves the best performance on both metrics across different datasets. Compared to the AnyEdit method, it improves BLEU by nearly 4\% on average across both datasets while consistently enhancing the BERT Score metric. This result further demonstrates the adaptability of our Bayes-Chunk approach across diverse datasets.

\input{ft-table}
\input{case}

\vspace{-8pt}
\subsection{Experiments on data with higher logical complexity and ultra-long lengths}
\label{sec:long}

% 根据前面大规模的对比实验，我们发现对于Bayes-Chunk的分块方法来说，或许更具有逻辑性的数据和更长的数据上（多分段），可以发挥出更大的优势。而将强逻辑性以及长文本的数据编辑进模型中是对知识编辑领域具有重要意义的发展方向。为了进一步探究这个现象，我们引入了一个新的数据集QwQ-LongCoT-130K作为我们的编辑数据集，该数据集包含了非常多数学问题，以及对应于该数学问题的包含CoT的回答，平均回答长度较高，由于完整的数据集较大，我们从中筛选了一小部分数据构成我们实际使用的数据集（具体的处理流程参考附录A）。
Based on the extensive comparative experiments conducted earlier, we observed that the Bayes-Chunk segmentation approach demonstrates greater advantages when applied to data with stronger logical coherence and longer segments (multiple-segment data). Incorporating highly logical and lengthy textual data into the model represents a significant developmental direction for the field of knowledge editing. To further investigate this phenomenon, we introduced a new dataset, \textbf{\textit{QwQ-Long-CoT-Math-GSM-v1}} \cite{qwq32b}, as our editing dataset. This dataset contains a large number of mathematical problems along with corresponding answers incorporating CoT, featuring relatively high average answer lengths. Due to the substantial size of the complete dataset, we selected a subset to form our actual working dataset (refer to \autoref{app:qwq} for the specific processing workflow). We denote this dataset as \textbf{QwQ-Edit}.

% 为了更直观的看到我们构造的QwQ-Edit数据集与EditEverything数据集之间的差异，我们给出了全面的数据集统计图如图A所示，具体而言我们从两个方面与EditEverything数据集进行了对比，分别为文本长度和逻辑密度两个角度。其中，文本长度的比较可以从图a中的密度图看出，我们的QwQ-Edit数据集大部分样本的长度（高密度区域）都超过了EditEverything中样本长度的最大值。而对于逻辑密度，我们则是定义了一批与逻辑语义相关的词语（具体信息可见附录B），再从各数据集的样本中计算这些逻辑语义词语的密度，用来估计数据集的逻辑性，从图b中可以看出，QwQ-Edit数据集的逻辑密度同样稳定超过EditEverything，我们也可以从c图中看出更全面的分布。基于这些观察，我们可以在一定程度上证明QwQ-Edit是一个涵盖更复杂逻辑，更长长度文本的知识编辑数据集。
To visually illustrate the differences between our constructed \textbf{QwQ-Edit} dataset and the \textbf{EditEverything} dataset, we present comprehensive statistical graphs of the dataset as shown in \autoref{fig:datacmp}. Specifically, we compare the two datasets from two perspectives: text length and logical density. Regarding text length comparison, as shown in the density plot in \autoref{fig:datacmp} (a), most samples in our QwQ-Edit dataset (high-density regions) exceed the maximum sample length in EditEverything. For logical density, we defined a set of semantically relevant terms (see \autoref{app:qwq} for details) and calculated their density across each dataset's samples to estimate logical coherence. \autoref{fig:datacmp} (b) demonstrates that QwQ-Edit consistently maintains higher logical density than EditEverything, with \autoref{fig:datacmp} (c) providing a more comprehensive distribution view. Based on these observations, we can demonstrate to a certain extent that QwQ-Edit is a knowledge editing dataset encompassing more complex logic and longer text lengths.

% 为了验证我们的方法在QwQ-Edit上相较于AnyEdit是否有稳定的提高，我们统一在该数据集上评估了AnyEdit方法与我们的方法，得到的结果如图A所示。我们将数据集中的样本按照长度和逻辑密度进行划分（详情可见附录S），并画出了我们的方法以及AnyEdit方法随着样本长度（或逻辑密度）变化下的BERT Score指标和BLEU指标的变化趋势。可以明显看出，不管横坐标如何变化，我们的方法在两个指标上都稳定的高于AnyEdit，这充分验证了我们在对比实验中的观察，即在编辑较长或逻辑性较强的文本时，基于Bayes-Chunk的分段方法会带来更强的编辑性能。
To verify whether our method achieves a consistent improvement over AnyEdit on the QwQ-Edit dataset, we evaluated both AnyEdit and our method uniformly on this dataset (given the great difficulty of the task, we selected \texttt{Qwen-2.5-7B-Instruct} as our target LLM due to its superior performance on logical reasoning tasks). The results are shown in \autoref{fig:qwq}. We categorized the samples in the dataset based on length and logical density (details in \autoref{app:qwq}) and plotted the trends of BERT Score and BLEU metrics for both our method and AnyEdit. It is evident that regardless of changes in the horizontal axis, our method \textbf{consistently outperforms} AnyEdit on \textbf{both} metrics. This fully corroborates our observations from comparative experiments: when editing longer or more logically coherent texts, the Bayes-Chunk-based segmentation approach yields superior editing performance.

\subsection{Analysis of Fine-Tuning-Based Editing Methods}

Recent studies have also explored knowledge editing through fine-tuning \cite{xiong2025fine}. However, when handling long-form editing targets, these methods usually still rely on fixed-length segmentation, making it difficult to adaptively determine editing boundaries according to the content. To verify the generality of our segmentation strategy, we combine Bayes segmentation with the fine-tuning-based FT-UKE method and report the results in \autoref{tab:ft_uke_results}. As shown in the table, on both \texttt{Llama-3.1-8B-Instruct} and \texttt{Qwen-2.5-7B-Instruct}, even when the original FT-UKE already achieves strong performance, introducing Bayes segmentation further improves the overall BLEU and BS scores. This demonstrates that our proposed segmentation mechanism can serve as a plug-and-play module to enhance fine-tuning-based knowledge editing methods.

\vspace{-10pt}
\section{Case Study}
\vspace{-4pt}
% 为了直观的看出我们的Bayes-Chunk和AnyEdit中固定窗口划分的区别，我们从EditEverything数据集中导出了一些关于两种分段方法的示例如图C所示，其中Sample ID直接对应当前的样本在EditEverything数据集中的id，分段的超参数设置与实际实验保持一致。可以从给定的两个Case看出，我们的Bayes-Chunk在面对相同的样本内容时，划分出了更可解释且更合理的段落，基于更优的段落划分，我们的方法得以更好的支持长文本以及复杂逻辑的编辑。
To visually illustrate the difference between our Bayes-Chunk and AnyEdit's fixed-window approaches, we extracted examples of both segmentation methods from EditEverything as shown in \autoref{fig:segmentation_examples}. The Sample ID directly corresponds to the sample's ID in the EditEverything dataset, and the segmentation hyperparameter settings align with those used in actual experiments. As demonstrated by the two cases presented, our Bayes-Chunk approach produces more interpretable and logical paragraph divisions when processing identical sample content. This superior paragraph segmentation enables our method to better support editing of lengthy texts and complex logical structures.

\vspace{-8pt}
\section{Conclusion}
\vspace{-4pt}
% 在本文中，我们聚焦于长文本，多样化的知识编辑任务。继承于AnyEdit的滑动窗口思想，我们通过将较长的文本划分为多个段落，逐个段落计算出目标位置的预期Hidden State，再通过多个段落计算得到的Hidden State共同约束得到目标的权重变化量，用于将知识编码进LLM的权重空间。与AnyEdit不同的是，我们使用了基于贝叶斯Surprise的段落划分方法，基于LLM本身的建模来计算出待编辑数据的贝叶斯Surprise分布，将贝叶斯Surprise较高的值作为分段的边界点，可以得到更具有独立性且更合理的分段，本文从理论上证明了基于贝叶斯Surprise分段可以提高编辑效率的合理性，并从广泛的实验中得到了验证。
In this paper, we focus on long-form, diverse knowledge editing tasks. Building upon AnyEdit's sliding window approach, we partition longer texts into multiple segments. For each segment, we compute the expected Hidden State at the target position. The weight changes for the target are then derived by jointly constraining the Hidden States computed across multiple segments, enabling knowledge encoding into the LLM's weight space. Unlike AnyEdit, we adopt a Bayesian Surprise–based segmentation strategy. By leveraging the LLM's inherent modeling capabilities to compute the Bayesian Surprise distribution of the data to be edited, we identify higher Bayesian Surprise values as segmentation boundaries. This yields more independent and reasonable paragraph divisions, and we theoretically demonstrate the rationale behind Bayes-Chunk for enhancing editing efficiency, validated through extensive experimentation.

% 但是，在目前的阶段，我们虽然在几乎所有的数据集上都取得了不错的效果，但是仍然还有一些部分（如EditEverything数据集中的一些子类）还存在与固定窗口分块几乎持平甚至低于固定窗口的情况，这其中的机制还需要做更深入的样例分析来得到原因。同时在超长文本和强逻辑性的编辑上，基于Bayes-Chunk的分段方法仍然还会随着样本长度的增长以及逻辑密度的增长而导致知识编辑的效果出现明显下滑，如何进一步增强知识编辑对知识本身形式的局限性是未来需要继续讨论的话题。
% However, at the current stage, while we have achieved satisfactory results on nearly all datasets, certain segments—such as some subclasses within the EditEverything dataset—still exhibit performance nearly on par with or even below fixed-window segmentation. The underlying mechanisms require deeper sample analysis to identify the causes. Additionally, when handling extremely long texts and highly logical edits, the Bayes-Chunk segmentation approach still exhibits a noticeable decline in knowledge editing effectiveness as sample length and logical density increase. How to further enhance knowledge editing beyond the limitations imposed by the inherent form of knowledge itself remains a topic for future discussion.

% \input{ft-table}

\section*{Acknowledgements}
This work was supported by the Guangdong Basic and Applied Basic Research Foundation (Grant No.~2026A1515011579), the HKUST-HKUST(GZ) 1+1+1 Joint Funding Program (Grant No.~C\_2025\_031), and the Guangzhou-HKUST(GZ) Joint Funding Program (Grant No.~2023A03J0008), Education Bureau of Guangzhou Municipality. This work was also supported by Jiangsu Industrial Technology Research Institute (JITRI) and Wuxi National High-Tech District (WND).

\section*{Impact Statement}
This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal consequences of our work, none of which we feel must be specifically highlighted here.


% In the unusual situation where you want a paper to appear in the
% references without citing it in the main text, use \nocite
% \nocite{langley00}

\nocite{wang2025mdpo,bowen2024beyond,chen2025polaris,chen2025ant}


\bibliography{example_paper}
\bibliographystyle{icml2026}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% APPENDIX
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newpage
\appendix
\onecolumn

% \section{Related Work}
% \label{app:related}

% Knowledge Editing (KE) aims to update specific knowledge in large language models without full retraining, while preserving unrelated behaviors\cite{sinitsin2020editableneuralnetworks,de-cao-etal-2021-editing,editing_as_unlearning}. Existing KE methods are commonly categorized by how the model is modified. Parameter-modifying approaches directly update model weights, including locate-then-edit methods such as ROME \cite{meng2022locating}, MEMIT \cite{meng2022mass}, and WISE \cite{wise}, as well as meta-learning or hypernetwork-based editors like MEND \cite{mitchell2022fastmodeleditingscale} and MALMEN \cite{tan2024massiveeditinglargelanguage}. In contrast, parameter-preserving approaches avoid permanent weight changes by introducing external or auxiliary mechanisms, such as memory-based editing \cite{mitchell2022memory,zheng2023editfactualknowledgeincontext,zhong-etal-2023-mquake} or neuron-augmented methods \cite{dong-etal-2022-calibrating,hartvigsen2023aging, huang2023transformerpatcher}. These categories together constitute the dominant methodological landscape of KE.

% \textbf{From Triples to Arbitrary-Length Knowledge.} Among parameter-modifying methods, Locate-then-Edit \cite{meng2022locating} has emerged as a core paradigm, initially designed for editing isolated factual triples of the form (subject, relation, object). Early methods such as ROME and MEMIT assume that factual knowledge can be causally localized to specific layers or parameter subspaces, enabling precise single-location edits. However, subsequent work has shown that many forms of knowledge—particularly temporal, commonsense, and conceptual knowledge—are expressed as free-text, multi-token, and distributed representations. Extensions such as METO address temporal knowledge with time-aware objectives \cite{yin2023history}, while MEMITCSK \cite{gupta-etal-2023-editing} and DEM \cite{huang-etal-2024-commonsense} adapt the locate-then-edit framework to commonsense knowledge by tracing multiple tokens or dynamically selecting layers across attention and MLP components. More recently, AnyEdit \cite{jiang2025anyedit} further relaxes the single-edit assumption by introducing an autoregressive editing paradigm that performs iterative edits over segmented sequences, enabling updates to arbitrarily long and structured texts. However, it relies on a \textbf{fixed-size sliding window} for chunking. This mechanism is oblivious to the distribution of information density within the text, leading to rigid truncations at critical semantic dependencies. In contrast, our work introduces a structure-aware adaptive segmentation mechanism that identifies semantic boundaries via Bayesian Surprise, enabling segmentation at high-information transition points and reducing cross-segment interference during editing.\looseness=-1

\section{Notation}

\begin{table}[h]
\centering
\small
\caption{Summary of mathematical notations used in this paper.}
\begin{tabular}{ll}
\toprule
\textbf{Symbol} & \textbf{Description} \\
\midrule
$k_t$ & Anchor key representation at anchor position $t$ \\
$v_t$ & Target value corresponding to the edited key $k_t$ \\
$\delta_t$ & Optimized perturbation applied to $k_t$ during editing \\
$W_{\text{out}}$ & Output projection matrix of the language model \\
$D_{\text{edit}}$ & Edit dataset consisting of key--value pairs $\{(k_t, v_t)\}_{t=1}^M$ \\
$M$ & Number of anchor points in a document \\
$L$ & Fixed window length in fixed-window chunking \\
\bottomrule
\end{tabular}
\end{table}

\section{Terminology}

\begin{table}[h]
\small
\caption{Clarification of key terms used in this paper.}
\begin{tabular}{lp{0.7\linewidth}}
\toprule
\textbf{Term} & \textbf{Explanation} \\
\midrule
Anchor Key & A key representation selected as the anchor point for a localized model edit. \\
Cross-segment interference (Crosstalk) & Unintended interaction where edits applied to one text segment affect other segments. \\
Surprisal-based boundary & A segmentation boundary determined by peaks in token-level surprisal. \\
Structural Independence & Geometric orthogonality between anchor keys that minimizes edit interference. \\
Causal Locality & The principle that edits injected at semantically salient positions yield superior control. \\
\bottomrule
\end{tabular}
\end{table}

\newpage
\section{Proof of Structural Independence (\autoref{thm:structural_independence})}
\label{app:proof_independence}

In this section, we derive the closed-form solution for the multi-segment editing objective and explicitly bound the crosstalk error.

\subsection{Optimization Objective}
We formulate the editing of $M$ segments as a constrained Ridge Regression problem. We seek an update $\Delta W$ that maps validation keys $K = [k_1, \dots, k_M]$ to target residual vectors $D = [\delta_1, \dots, \delta_M]$, while preserving general knowledge defined by covariance $C$:
\[
\min_{\Delta W} \mathcal{J}(\Delta W) = \| \Delta W K - D \|_F^2 + \lambda \text{Tr}(\Delta W C \Delta W^T).
\]

\subsection{Derivation of the Update Rule}
Since $\mathcal{J}$ is strictly convex, we find the global minimum by setting the gradient to zero:
\[
\frac{\partial \mathcal{J}}{\partial \Delta W} = 2(\Delta W K - D)K^T + 2\lambda \Delta W C = 0.
\]
Solving for $\Delta W$, we obtain:
\[
\Delta W^* = D K^T (K K^T + \lambda C)^{-1} = D K^T A.
\]
where $A \triangleq (\lambda C + \sum_{i=1}^M k_i k_i^T)^{-1}$ is the Precision Matrix. The update can be written as a sum of rank-1 updates:
\[
\Delta W^* = \sum_{t=1}^M \delta_t k_t^T A.
\]

\subsection{Error Decomposition}
We analyze the effect of this global update on a specific target key $k_j$. The realized perturbation is:
\[
\hat{\delta}_j = \Delta W^* k_j = \left( \sum_{t=1}^M \delta_t k_t^T A \right) k_j = \sum_{t=1}^M \delta_t (k_t^T A k_j).
\]
Decomposing into the signal term ($t=j$) and interference term ($t \neq j$):
\[
\hat{\delta}_j = \underbrace{\delta_j (k_j^T A k_j)}_{\text{Signal}} + \underbrace{\sum_{t \neq j} \delta_t (k_t^T A k_j)}_{\text{Crosstalk } \mathcal{E}_{cross}^{(j)}}.
\]

\subsection{Bounding the Crosstalk}
By the Triangle Inequality, the norm of the crosstalk error is:
\[
\| \mathcal{E}_{cross}^{(j)} \|_2 \leq \sum_{t \neq j} \| \delta_t \|_2 \cdot | k_t^T A k_j |.
\]
Let $\alpha_{tj} \triangleq | k_t^T A k_j |$. Assuming the metric space induced by $A$ is well-conditioned, $\alpha_{tj}$ represents the generalized inner product. Minimizing the upper bound requires minimizing $\alpha_{tj}$, which is equivalent to minimizing the pairwise cosine similarity between distinct keys $k_t$ and $k_j$.

\section{Detailed Empirical Analysis of Segment Independence}
\label{app:vis_segment}
To further substantiate the theoretical claims in \autoref{sec:theory_efficiency}, we provide a detailed experimental comparison between our Bayes-Chunk method and the baseline Fixed-Window approach.
\subsection{Global Statistical Analysis (KDE Plot)}
We evaluate the semantic distinctness of segments generated by different partitioning strategies. For a given long-form sample $S$ partitioned into $N$ segments $\{s_1, s_2, \dots, s_N\}$, we calculate the Intra-sample Mean Similarity $\bar{\sigma}(S)$ as:
\[
\bar{\sigma}(S) = \frac{2}{N(N-1)} \sum_{1 \le i < j \le N} \text{CosSim}(\phi(s_i), \phi(s_j)).
\]
where $\phi(\cdot)$ is the embedding function of the \texttt{Qwen3-Embedding-0.6B} model.
We perform this calculation for all samples in the \textit{EditEverything} dataset. The resulting Kernel Density Estimation (KDE) plot illustrates the probability density of $\bar{\sigma}$ across the dataset.
\begin{itemize}
\item \textbf{Bayes-Chunk:} Mean $\bar{\sigma} = 0.509$. The distribution is concentrated in the lower-similarity region, indicating that the segments are more semantically independent.
\item \textbf{Fixed-Window:} Mean $\bar{\sigma} = 0.594$. The higher mean and right-shifted distribution suggest that fixed-size segments often contain redundant or highly correlated information, which leads to hardware/memory inefficiency and potential editing interference.
\end{itemize}
\subsection{Internal Key Orthogonality (Heatmap Analysis)}
To observe the direct impact on the editing solver, we extract the actual anchor keys $k_t$ from the target editing layer of the \texttt{Llama-3.1-8B-Instruct} model.
\paragraph{Extraction Protocol:} For a sequence of segments $\{s_1, \dots, s_N\}$, we feed the cumulative context (Question + $\sum_{i=1}^t s_i$) into the model. The anchor key $k_t$ is defined as the hidden state at the last token of segment $s_t$ at the target layer $l$, specifically the tensor immediately preceding the output weight matrix $W_{out}$.
\paragraph{Results:} We compute the pairwise cosine similarity matrix $M \in \mathbb{R}^{N \times N}$, where $M_{ij} = \frac{k_i^T k_j}{\|k_i\|_2 \|k_j\|_2}$.
\begin{itemize}
\item \textbf{Fixed-Window Heatmap:} Shows strong off-diagonal activation. This confirms that adjacent fixed-length chunks often produce highly similar keys, leading to the \textit{overwriting} effect described in Theorem \ref{thm:structural_independence}.
\item \textbf{Bayes-Chunk Heatmap:} Displays a sharp diagonal structure with significantly suppressed off-diagonal values. By splitting the text at points of high surprisal, the hidden states are forced to undergo a state transition, resulting in keys that are more orthogonal in the latent space.
\end{itemize}
This visualization provides direct evidence that Bayes-Chunk satisfies the prerequisite for the crosstalk bound $\mathcal{E}_{cross} \to 0$, enabling stable mass-editing in long-form contexts.



\section{Proof of Causal Locality (\autoref{thm:causal_locality})}
\label{app:proof_locality}

In this section, we provide the rigorous derivation for the superior controllability of the immediate predecessor state.

\subsection{Problem Formulation}
Let the gradient flow from the target loss $\mathcal{L}$ to an arbitrary hidden state $h_i^{(\ell)}$ be decomposed via the chain rule:
\[
    \nabla_{h_{i}^{(\ell)}} \mathcal{L} = \underbrace{\left( \frac{\partial \mathcal{L}}{\partial h_{t-1}^{(L)}} \right)^T}_{G_{\text{final}}} \cdot \underbrace{\frac{\partial h_{t-1}^{(L)}}{\partial h_{i}^{(\ell)}}}_{J_{i \to t-1}}.
\]
where $G_{\text{final}}$ represents the gradient at the final layer readout, and $J_{i \to t-1}$ is the Jacobian of the intermediate layers.

\begin{proof}
We analyze the magnitude of the terms for $i=t-1$ versus $i=t-k$.

\textbf{Step 1: The Magnitude of $G_{\text{final}}$.}
Consider the cross-entropy loss $\mathcal{L} = -\log (\text{Softmax}(W_u h_{t-1}^{(L)}))_{y_t}$. The gradient w.r.t logits is $p - \mathbf{e}_{y_t}$.
For a high-surprisal target (where the model initially predicts low probability $p_{y_t} \to 0$), the error vector norm approaches saturation:
\[
    \lim_{p_{y_t} \to 0} \| p - \mathbf{e}_{y_t} \|_2 \approx 1.
\]
Assuming the unembedding matrix $W_u$ is full-rank with smallest singular value $\sigma_{\min} > 0$, we have a lower bound:
\[
    \| G_{\text{final}} \|_2 \ge \sigma_{\min}(W_u) \cdot \| p - \mathbf{e}_{y_t} \|_2 \ge \xi > 0.
\]

\textbf{Step 2: Vertical Propagation (The Highway).}
For the immediate position $i = t-1$, the signal backpropagates vertically. In modern Transformers, residual branches are initialized with scaled variance ($W \sim \mathcal{N}(0, \frac{1}{2dL})$). Thus, the Lipschitz constant of the sub-layer function $F_m$ is small ($\lambda \ll 1$). The Jacobian through the residual stream $I + \frac{\partial F}{\partial h}$ has singular values clustered around 1:
\[
    J_{t-1 \to t-1} \approx I \implies \| G_{\text{final}} J_{t-1 \to t-1} \|_2 \approx \| G_{\text{final}} \|_2.
\]

\textbf{Step 3: Horizontal Propagation (The Bottleneck).}
For a distant position $i = t-k$, the signal must pass through the Attention mechanism: $h_{t-1} = \sum_{j} A_{t-1, j} V_j$. The Jacobian includes the attention weight term:
\[
    J_{t-k \to t-1} \approx A_{t-1, t-k} W_V.
\]
Since $\sum_j A_{t-1, j} = 1$, for any specific distant token $k$, the attention weight is typically small ($A_{t-1, t-k} \ll 1$), acting as a damping factor $\gamma < 1$. Thus:
\[
    \kappa(t-k \to t) \approx \gamma \| G_{\text{final}} \|_2 \ll \| G_{\text{final}} \|_2.
\]

\textbf{Conclusion.}
Comparing the two terms:
\[
    \Delta \kappa_k \approx (1 - \gamma) \| G_{\text{final}} \|_2.
\]
Since $\| G_{\text{final}} \|_2$ is bounded away from zero (Step 1) and $\gamma < 1$ (Step 3), it follows that $\Delta \kappa_k > 0$. This confirms that the residual stream provides a high-bandwidth control channel, while attention acts as an information filter.
\end{proof}

\section{Sensitivity Analysis of Causal Controllability}
\label{app:causal_locality_study}
In \autoref{sec:theory_locality}, we posited that the immediate predecessor state provides the most efficient control over target predictions. Here, we provide the detailed experimental setup and extended results for the gradient sensitivity analysis.
\subsection{Experimental Setup}
We randomly sample long-form instances from the \textit{EditEverything} dataset and feed them into the \texttt{Llama-3.1-8B-Instruct} model. Let $h_{L,T}$ denote the hidden state of the $T$-th (last) token at the $L$-th (final) layer. To measure the intrinsic controllability of each position $(l, i)$, we define the loss as the norm of this final state: $\mathcal{L} = \|h_{L,T}\|_2$. We then compute the gradient sensitivity $G_{l,i}$ for every layer $l \in [1, L]$ and every token position $i$:
\[
G_{l,i} = \left| \frac{\partial \mathcal{L}}{\partial h_{l,i}} \right|_2.
\]
For cross-sample comparison, the sensitivities are normalized within each sample.
\subsection{Vertical vs. Horizontal Decay Analysis}
The analysis in \autoref{fig:theory-2}(a) decomposes the gradient behavior into two orthogonal dimensions:
\begin{enumerate}
\item \textbf{Vertical Stability (Layer-wise):} The left sub-plot shows the mean sensitivity across layers for the target token. The curve is relatively flat, indicating that the gradient signal propagates through the residual stream with minimal attenuation. This supports our assumption that vertical propagation acts as a quasi-isometry.
\item \textbf{Horizontal Decay (Token-wise):} The right sub-plot illustrates the sensitivity as a function of the relative distance from the target token ($T-i$). A steep decline is observed: once the distance exceeds a few tokens, the sensitivity drops by orders of magnitude. This confirms that the attention mechanism acts as a bottleneck for \textit{horizontal} control signals.
\end{enumerate}
\subsection{Layer-Token Heatmap Visualization}
To provide a holistic view, we visualize the full sensitivity matrix $M \in \mathbb{R}^{L \times T}$ as a heatmap (\autoref{fig:theory-2}(b)).
\begin{itemize}
\item \textbf{Observation:} The heatmap exhibits a prominent vertical band at the positions closest to the target token (the rightmost or \textit{latest} columns in the sequence). This band maintains high intensity across almost all layers, while the rest of the map remains sparsely activated.
\item \textbf{Implication:} This \textit{vivid column} provides visual evidence for our Theorem \ref{thm:causal_locality}: the most effective site for knowledge injection is not layer-specific, but rather position-specific. By targeting the immediate predecessors of the information we wish to update, we can minimize the update norm $\|\delta\|$ required to achieve the desired output, maximizing parameter efficiency and reducing collateral damage to distant context.
\end{itemize}

\section{Dataset Details}
\label{app:datasets}

To validate the generalizability and robustness of AnyEdit++, we selected datasets that represent different levels of editing difficulty.

\textbf{EditEverything}: Reliably editing long texts requires maintaining logical chains and syntactic correctness, which standard triplet-based datasets cannot evaluate. We utilize the \textbf{EditEverything} benchmark, which aggregates high-complexity samples from seven diverse domains: \textit{Mathematics, Code, Physics, Chemistry, Biology, News, and Poetry}. Unlike single-sentence edits, these tasks require the model to handle:
\begin{itemize}
    \item \textit{Structural Formatting}: Preserving code indentation or poetic stanzas after editing.
    \item \textit{Logical Coherence}: Ensuring multi-step mathematical derivations remain valid after modifying an initial condition.
\end{itemize}
This serves as the primary testbed for our structure-aware segmentation, highlighting the failure cases of fixed-window approaches.

\textbf{CounterFact}: We use \textbf{CounterFact}~\cite{meng2022locating} to evaluate standard knowledge editing performance. It consists of counterfactual updates (e.g., changing \textit{The capital of France is Paris} to \textit{Rome}). This dataset allows us to measure:
\begin{itemize}
    \item \textit{Efficacy}: Success rate of the edit.
    \item \textit{Locality}: Ensuring the edit does not affect unrelated knowledge.
\end{itemize}
Inclusion of CounterFact ensures that AnyEdit++ retains strong performance on fundamental editing tasks compared to existing baselines.

\textbf{UnKE}: We use \textbf{UnKE}~\cite{deng2024unke} bridges the gap between rigid triplets and free-form text. It evaluates the model's ability to answer questions based on edited knowledge presented in unstructured formats. We use this to verify that our method can generalize edits beyond exact string matching, effectively updating the underlying semantic knowledge.

\section{Detailed Experimental Setup}
\label{app:setup}

\subsection{Implementation Details}
Our implementation is built upon the official codebases of MEMIT and ROME. All experiments are conducted using the PyTorch framework and the HuggingFace Transformers library. The core editing algorithm follows the mass-editing flow, where the update is distributed across multiple layers.

\subsection{Hyperparameters for Target Vector Optimization}
The computation of the target vector $z$ (represented as 'compute\_z\_bayes.py' in our codebase) is a critical step in our method. Unlike the sliding window approach used in baselines, we employ a surprisal-based adaptive segmentation strategy.

\paragraph{Optimization Objective.}
For each segmented semantic unit, we optimize the delta vector $\delta$ to minimize the negative log-likelihood (NLL) of the target sequence while constraining the norm of the change. The loss function $\mathcal{L}$ is defined as:
\[
    \mathcal{L} = \mathcal{L}_{NLL} + \lambda \frac{\|\delta\|_2}{\|z_{init}\|_2^2}.
\]
where $z_{init}$ is the initial representation of the subject, and $\lambda$ is the weight decay factor. We use the \textbf{Adam} optimizer for this process.

\paragraph{Adaptive Segmentation Configuration.}
Our adaptive segmentation relies on the surprisal values (information content) of the target sequence. The specific hyperparameters used to determine boundaries are:
\begin{itemize}
    \item \textbf{Boundary Density Ratio:} We set the number of segments $k$ proportional to the sequence length $L$. Specifically, $k = \lceil L / 40 \rceil$. This ratio was empirically chosen to balance semantic coherence and computational efficiency.
    \item \textbf{Early Window Strategy:} To prevent generation collapse at the beginning of long sequences, we enforce a heuristic constraint: if no boundary is detected within the first $N=3$ tokens based on the global top-$k$ surprisal sorting, we forcibly insert a boundary at the position with the maximum surprisal within this local window ($[0, N)$), replacing the boundary with the lowest surprisal score in the set.
\end{itemize}

\subsection{Global Hyperparameters}
Table~\ref{tab:hyperparams} lists the detailed hyperparameters used in our experiments. 

\begin{table}[h]
\centering
\caption{Hyperparameters used for MEMIT-ARE experiments.}
\label{tab:hyperparams}
\begin{tabular}{l|c}
\toprule
\textbf{Parameter} & \textbf{Value} \\
\midrule
\multicolumn{2}{c}{\textit{Target Vector Optimization ($z$)}} \\
\midrule
Optimizer & Adam \\
Learning Rate (`v\_lr`) & $5 \times 10^{-1}$ \\
Max Gradient Steps (`v\_num\_grad\_steps`) & 20 \\
Loss Threshold (Early Stopping) & $1 \times 10^{-2}$ \\
Weight Decay ($\lambda$) & $5 \times 10^{-1}$ \\
Norm Constraint Factor & $3.0 \times \|z_{init}\|$ \\
\midrule
\multicolumn{2}{c}{\textit{Covariance Statistics}} \\
\midrule
Covariance Dataset & Wikipedia (wikitext) \\
Sample Size (`mom2\_n\_samples`) & 100,000 \\
Update Weight (`mom2\_update\_weight`) & 15000 \\
\midrule
\multicolumn{2}{c}{\textit{Segmentation Logic}} \\
\midrule
Segments Count Formula & $\lceil \text{Length} / 40 \rceil$ \\
Force Early Window Size ($N$) & 3 tokens \\
\bottomrule
\end{tabular}
\end{table}

\subsection{Covariance Statistics}
Following the standard MEMIT setting, we estimate the key-value covariance matrix $C$ using a subset of the Wikipedia dataset. We collect second-moment statistics (`mom2`) over 100,000 samples to ensure the edit minimizes interference with the model's general knowledge. The update is applied by solving the least-squares problem:
\[
    W_{new} = W_{old} + \Lambda (C + K K^T)^{-1} K^T.
\]
where $K$ represents the stacked keys from the editing targets and $\Lambda$ is the residual distributed across layers.

\input{code}

\section{Build Our QwQ-Edit Dataset}
\label{app:qwq}



% 我们的QwQ数据集是从fql/qwq_long_cot_math_gsm_v1数据集中采样得到的，该数据集从GSM-8K和MATH数据集中获得原始问题，再使用QwQ-32B构造出CoT的回复，总共包含8.2k的训练集数据以及2.04k的测试集数据。我们从测试集数据中随机抽取了300条样本作为我们的QwQ-Edit数据。 QwQ-Edit数据集的示例如图A所示。
Our QwQ dataset was sampled from the \texttt{fql/qwq\_long\_cot\_math\_gsm\_v1} dataset, which obtained original questions from the GSM-8K and MATH datasets and then constructed CoT responses using QwQ-32B. It comprises a total of 8.2k training examples and 2.04k test examples. We randomly selected 300 samples from the test set to form our QwQ-Edit dataset. An example of the QwQ-Edit dataset is shown in Figure A.

\subsection{Dataset Metric Definitions}
\label{app:appendix_metrics}

To analyze the complexity and logical depth of the \textit{QwQ-Edit} dataset compared to standard baselines, we defined specific metrics for text length and logical density.

\textbf{Logic Density Calculation}: We quantify the logical density of a solution by measuring the frequency of explicit reasoning markers. The \textit{Logic Density} metric is defined as the number of reasoning keywords per 100 words in the generated output:

\[
    \text{Logic Density} = \frac{N_{\text{keywords}}}{N_{\text{words}}} \times 100.
\]

where $N_{\text{words}}$ is the total word count of the solution text, and $N_{\text{keywords}}$ is the count of specific logical connectives identified via case-insensitive pattern matching.

\textbf{List of Reasoning Keywords}: The calculation considers a set of 15 causal and procedural keywords commonly associated with chain-of-thought reasoning. The complete list of keywords used in our analysis is:

\begin{quote}
    \textit{therefore, because, thus, hence, consequently, implies, step, first, second, finally, so, then, since, result, leads to}
\end{quote}

\textbf{Length Metrics}: Text length statistics are derived based on word-level tokenization (splitting by whitespace). \textit{Total Length} refers to the sum of the word counts for both the problem statement (input) and the solution (output).

% \newpage
\section{Additional Data}
Here we present full-text samples from QwQ-Edit mentioned above, providing a more intuitive understanding of the data.

\input{qwq}
\end{document}

% This document was modified from the file originally made available by
% Pat Langley and Andrea Danyluk for ICML-2K. This version was created
% by Iain Murray in 2018, and modified by Alexandre Bouchard in
% 2019 and 2021 and by Csaba Szepesvari, Gang Niu and Sivan Sabato in 2022.
% Modified again in 2023 and 2024 by Sivan Sabato and Jonathan Scarlett.
% Previous contributors include Dan Roy, Lise Getoor and Tobias
% Scheffer, which was slightly modified from the 2010 version by
% Thorsten Joachims & Johannes Fuernkranz, slightly modified from the
% 2009 version by Kiri Wagstaff and Sam Roweis's 2008 version, which is
% slightly modified from Prasad Tadepalli's 2007 version which is a
% lightly changed version of the previous year's version by Andrew
% Moore, which was in turn edited from those of Kristian Kersting and
% Codrina Lauth. Alex Smola contributed to the algorithmic style files.

% \begin{table*}[htbp]
% \centering
% \caption{We compare the editing performance on \textbf{EditEverything} benchmark and standard datasets.}
% \label{tab:main_results}
% % 使用 resizebox 确保表格适应页面宽度
% \resizebox{\linewidth}{!}{%
% \begin{tabular}{clcccccccccccccccc}
% \toprule
% % ---------------- 表头第一层 ----------------
% \multirow{3}{*}{\parbox{1.5cm}{\centering \textbf{LLM}}} & \multirow{3}{*}{\textbf{Method}} & \multicolumn{14}{c}{\textbf{EditEverything}} & \multicolumn{2}{c}{\textbf{Average}} \\
% \cmidrule(lr){3-16} \cmidrule(lr){17-18}

% % ---------------- 表头第二层 ----------------
%  &  & \multicolumn{2}{c}{Math} & \multicolumn{2}{c}{Code} & \multicolumn{2}{c}{Physics} & \multicolumn{2}{c}{Chemistry} & \multicolumn{2}{c}{Biology} & \multicolumn{2}{c}{News} & \multicolumn{2}{c}{Poetry} & \multicolumn{2}{c}{Total} \\
% \cmidrule(lr){3-4} \cmidrule(lr){5-6} \cmidrule(lr){7-8} \cmidrule(lr){9-10} \cmidrule(lr){11-12} \cmidrule(lr){13-14} \cmidrule(lr){15-16} \cmidrule(lr){17-18}

% % ---------------- 表头第三层 ----------------
%  &  & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS \\
% \midrule

% % ---------------- Llama-3.1-8B ----------------
% \multirow{4}{*}{\parbox{1.8cm}{\centering \textsc{Llama-3.1} \\ 8B}} 
%  & MEMIT & 41.06 & 89.06 & 35.06 & 77.26 & 57.48 & 88.94 & 56.74 & 91.59 & 55.63 & 87.87 & 25.11 & 74.74 & 27.21 & 69.70 & 42.61 & 82.74 \\
%  & AlphaEdit & 38.89 & 88.28 & 27.09 & 75.03 & 54.00 & 87.85 & 52.05 & 90.66 & 57.27 & 88.58 & 26.13 & 74.46 & 27.66 & 68.52 & 40.44 & 81.91 \\
%  & AnyEdit & 74.19 & 95.84 & 81.20 & 95.75 & 79.25 & 96.73 & 81.35 & \textbf{97.46} & 78.64 & 96.37 & 64.81 & 91.69 & 49.07 & \textbf{85.80} & 72.64 & 94.23 \\
%  &  \rowcolor{cyan!10} \textbf{Ours} & \textbf{77.74} & \textbf{96.49} & \textbf{86.48} & \textbf{97.27} & \textbf{80.24} & \textbf{97.25} & \textbf{81.49} & 97.16 & \textbf{80.35} & \textbf{96.86} & \textbf{68.45} & \textbf{91.70} & \textbf{50.21} & 84.78 & \textbf{75.00} & \textbf{94.50} \\
% \midrule

% % ---------------- Llama-2-7B ----------------
% \multirow{4}{*}{\parbox{1.8cm}{\centering \textsc{Llama-2} \\ 7B}} 
%  & MEMIT & 26.36 & 72.90 & 20.77 & 72.97 & 33.20 & 77.33 & 30.39 & 81.21 & 39.28 & 74.11 & 28.50 & 62.71 & 8.01 & 35.86 & 26.64 & 68.16 \\
%  & AlphaEdit & 29.70 & 73.91 & 10.95 & 62.26 & 35.97 & 80.11 & 29.92 & 78.67 & 39.11 & 77.06 & 26.74 & 74.38 & 9.27 & 48.57 & 25.95 & 70.71 \\
%  & AnyEdit & 43.05 & 88.23 & 50.78 & 88.62 & 49.20 & 90.37 & 47.00 & 91.77 & 51.82 & 90.32 & 34.34 & 84.03 & 19.94 & 70.99 & 42.30 & 86.33 \\
%  & \textbf{Ours} & \textbf{47.31} & \textbf{88.52} & \textbf{70.50} & \textbf{92.80} & \textbf{53.83} & \textbf{90.44} & \textbf{52.00} & \textbf{92.65} & \textbf{55.39} & \textbf{90.51} & \textbf{44.85} & \textbf{84.91} & \textbf{27.04} & \textbf{73.77} & \textbf{50.13} & \textbf{87.66} \\
% \midrule

% % ---------------- Qwen-2.5-7B ----------------
% \multirow{4}{*}{\parbox{1.8cm}{\centering \textsc{Qwen-2.5} \\ 7B}} 
%  & MEMIT & 75.91 & 95.18 & 51.84 & 81.49 & 64.28 & 91.37 & 67.10 & 94.44 & 64.79 & 91.37 & 45.13 & 79.01 & 40.22 & 71.96 & 58.47 & 86.40 \\
%  & AlphaEdit & 81.18 & 95.76 & 60.74 & 83.42 & 66.87 & 91.51 & 72.57 & 95.46 & 65.70 & 91.79 & 50.17 & 82.09 & 39.78 & 71.17 & 62.43 & 87.31 \\
%  & AnyEdit & 89.07 & 98.09 & 83.34 & 95.69 & 87.29 & 98.04 & 90.46 & \textbf{98.54} & \textbf{85.82} & \textbf{97.04} & \textbf{80.09} & 94.79 & 56.63 & 84.77 & 81.81 & 95.28 \\
%  & \textbf{Ours} & \textbf{92.80} & \textbf{98.79} & \textbf{91.56} & \textbf{98.41} & \textbf{87.91} & \textbf{98.07} & \textbf{91.61} & 98.10 & 85.29 & 96.67 & 79.97 & \textbf{95.30} & \textbf{68.14} & \textbf{88.67} & \textbf{85.33} & \textbf{96.29} \\
% \bottomrule
% \end{tabular}%
% }
% \end{table*}


\begin{table*}[htbp]
\centering
\caption{We evaluated the performance of various models and different approaches on knowledge editing using \textbf{EditEverything}. We use \textbf{bold} text to indicate optimal results, \ul{underline} to denote suboptimal results, and highlight
  improvements exceeding 5\% in \textcolor{red}{red}}
\label{tab:main_results}


% 1. 局部消除 booktabs 的行间距，防止背景色出现白色缝隙
\setlength{\aboverulesep}{0pt}
\setlength{\belowrulesep}{0pt}
\resizebox{0.95\linewidth}{!}{%
\begin{tabular}{clcccccccccccccc|cc}
\toprule[0.5mm]
% ---------------- 表头 ----------------
\multirow{3}{*}{\parbox{1.5cm}{\centering \textbf{LLM}}} & \multirow{3}{*}{\textbf{Method}} & \multicolumn{14}{c}{\textbf{EditEverything}} & \multicolumn{2}{c}{\textbf{Average}} \\
\cmidrule(lr){3-16} \cmidrule(lr){17-18}
 & & \multicolumn{2}{c}{Math} & \multicolumn{2}{c}{Code} & \multicolumn{2}{c}{Physics} & \multicolumn{2}{c}{Chemistry} & \multicolumn{2}{c}{Biology} & \multicolumn{2}{c}{News} & \multicolumn{2}{c}{Poetry} & \multicolumn{2}{c}{\textbf{Total}} \\
\cmidrule(lr){3-4} \cmidrule(lr){5-6} \cmidrule(lr){7-8} \cmidrule(lr){9-10} \cmidrule(lr){11-12} \cmidrule(lr){13-14} \cmidrule(lr){15-16} \cmidrule(lr){17-18}
 & & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS \\
\midrule

% ---------------- Llama-3.1-8B ----------------
\multirow{4}{*}{\parbox{1.8cm}{\centering \textsc{Llama-3.1} \\ 8B-Instruct }} 
 & MEMIT & 41.06 & 89.06 & 35.06 & 77.26 & 57.48 & 88.94 & 56.74 & 91.59 & 55.63 & 87.87 & 25.11 & 74.74 & 27.21 & 69.70 & 42.61 & 82.74 \\
 & AlphaEdit & 38.89 & 88.28 & 27.09 & 75.03 & 54.00 & 87.85 & 52.05 & 90.66 & 57.27 & 88.58 & 26.13 & 74.46 & 27.66 & 68.52 & 40.44 & 81.91 \\
 & AnyEdit & \ul{74.19} & \ul{95.84} & \ul{81.20} & \ul{95.75} & \ul{79.25} & \ul{96.73} & \ul{81.35} & \textbf{97.46} & \ul{78.64} & \ul{96.37} & \ul{64.81} & \ul{91.69} & \ul{49.07} & \textbf{85.80} & \ul{72.64} & \ul{94.23} \\
 % 这里使用 cellcolor{white} 确保第一列文字不被背景覆盖，即便它是 multirow 的一部分
 \rowcolor{cyan!10} \cellcolor{white} & \textbf{Ours} & \textbf{77.74} & \textbf{96.49} & \textbf{\textcolor{red}{86.48}} & \textbf{97.27} & \textbf{80.24} & \textbf{97.25} & \textbf{81.49} & \ul{97.16} & \textbf{80.35} & \textbf{96.86} & \textbf{68.45} & \textbf{91.70} & \textbf{50.21} & \ul{84.78} & \textbf{75.00} & \textbf{94.50} \\
\midrule

% ---------------- Llama-2-7B ----------------
\multirow{4}{*}{\parbox{1.8cm}{\centering \textsc{Llama-2} \\ 7B}} 
 & MEMIT & 26.36 & 72.90 & 20.77 & 72.97 & 33.20 & 77.33 & 30.39 & 81.21 & 39.28 & 74.11 & 28.50 & 62.71 & 8.01 & 35.86 & 26.64 & 68.16 \\
 & AlphaEdit & 29.70 & 73.91 & 10.95 & 62.26 & 35.97 & 80.11 & 29.92 & 78.67 & 39.11 & 77.06 & 26.74 & 74.38 & 9.27 & 48.57 & 25.95 & 70.71 \\
 & AnyEdit & \ul{43.05} & \ul{88.23} & \ul{50.78} & \ul{88.62} & \ul{49.20} & \ul{90.37} & \ul{47.00} & \ul{91.77} & \ul{51.82} & \ul{90.32} & \ul{34.34} & \ul{84.03} & \ul{19.94} & \ul{70.99} & \ul{42.30} & \ul{86.33} \\
 \rowcolor{cyan!10} \cellcolor{white} & \textbf{Ours} & \textbf{47.31} & \textbf{88.52} & \textbf{\textcolor{red}{70.50}} & \textbf{92.80} & \textbf{53.83} & \textbf{90.44} & \textbf{\textcolor{red}{52.00}} & \textbf{92.65} & \textbf{55.39} & \textbf{90.51} & \textbf{\textcolor{red}{44.85}} & \textbf{84.91} & \textbf{27.04} & \textbf{73.77} & \textbf{\textcolor{red}{50.13}} & \textbf{87.66} \\
\midrule

% ---------------- Qwen-2.5-7B ----------------
\multirow{4}{*}{\parbox{1.8cm}{\centering \textsc{Qwen-2.5} \\ 7B-Instruct}} 
 & MEMIT & 75.91 & 95.18 & 51.84 & 81.49 & 64.28 & 91.37 & 67.10 & 94.44 & 64.79 & 91.37 & 45.13 & 79.01 & 40.22 & 71.96 & 58.47 & 86.40 \\
 & AlphaEdit & 81.18 & 95.76 & 60.74 & 83.42 & 66.87 & 91.51 & 72.57 & 95.46 & 65.70 & 91.79 & 50.17 & 82.09 & 39.78 & 71.17 & 62.43 & 87.31 \\
 & AnyEdit & \ul{89.07} & \ul{98.09} & \ul{83.34} & \ul{95.69} & \ul{87.29} & \ul{98.04} & \ul{90.46} & \textbf{98.54} & \textbf{85.82} & \textbf{97.04} & \textbf{80.09} & \ul{94.79} & \ul{56.63} & \ul{84.77} & \ul{81.81} & \ul{95.28} \\
 \rowcolor{cyan!10} \cellcolor{white} & \textbf{Ours} & \textbf{92.80} & \textbf{98.79} & \textbf{\textcolor{red}{91.56}} & \textbf{98.41} & \textbf{87.91} & \textbf{98.07} & \textbf{91.61} & \ul{98.10} & \ul{85.29} & \ul{96.67} & \ul{79.97} & \textbf{95.30} & \textbf{\textcolor{red}{68.14}} & \textbf{88.67} & \textbf{85.33} & \textbf{96.29} \\
\bottomrule[0.5mm]
\end{tabular}%
}
\vspace{-12pt}

\end{table*}


\begin{table}[htbp]
\centering
\vspace{-2pt}
\caption{Performance comparison on Reference Benchmarks.}
\label{tab:llama3_ref_results}
\resizebox{0.9\linewidth}{!}{%
\begin{tabular}{lcccccc}
\toprule[0.5mm]
\multirow{2}{*}{\textbf{Method}} & \multicolumn{2}{c}{UnKE} & \multicolumn{2}{c}{CounterFact} & \multicolumn{2}{c}{\textbf{Average}} \\
\cmidrule(lr){2-3} \cmidrule(lr){4-5} \cmidrule(lr){6-7}
 & BLEU & BS & BLEU & BS & BLEU & BS \\
\midrule
MEMIT & 24.76 & 76.50 & 32.21 & 75.79 & 28.49 & 76.15 \\
AlphaEdit & 21.34 & 73.86 & 23.51 & 72.42 & 22.43 & 73.14 \\
AnyEdit & 79.02 & 95.88 & 86.27 & 97.85 & 82.65 & 96.87 \\
 \rowcolor{cyan!10}\textbf{Ours} & \textbf{81.57} & \textbf{96.03} & \textbf{90.69} & \textbf{98.29} & \textbf{86.13} & \textbf{97.16} \\
\bottomrule[0.5mm]
% \vspace{-16pt}
\end{tabular}%
}
\end{table}
\vspace{-4pt}

\begin{table*}[htbp]
\centering
\caption{Editing results of FT-UKE and FT-UKE with Bayes segmentation. The light-blue background indicates results with Bayes segmentation, and bold values indicate improvements over the original FT-UKE based on unrounded scores.}
\label{tab:ft_uke_results}
\resizebox{0.95\linewidth}{!}{%
\begin{tabular}{clcccccccccccccc|cc}
\toprule[0.5mm]
\multirow{3}{*}{\parbox{1.5cm}{\centering \textbf{LLM}}} & \multirow{3}{*}{\textbf{Method}} & \multicolumn{14}{c}{\textbf{EditEverything}} & \multicolumn{2}{c}{\textbf{Average}} \\
\cmidrule(lr){3-16} \cmidrule(lr){17-18}
 & & \multicolumn{2}{c}{Math} & \multicolumn{2}{c}{Code} & \multicolumn{2}{c}{Physics} & \multicolumn{2}{c}{Chemistry} & \multicolumn{2}{c}{Biology} & \multicolumn{2}{c}{News} & \multicolumn{2}{c}{Poetry} & \multicolumn{2}{c}{\textbf{Total}} \\
\cmidrule(lr){3-4} \cmidrule(lr){5-6} \cmidrule(lr){7-8} \cmidrule(lr){9-10} \cmidrule(lr){11-12} \cmidrule(lr){13-14} \cmidrule(lr){15-16} \cmidrule(lr){17-18}
 & & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS & BLEU & BS \\
\midrule
 & FT-UKE & 99.89 & 99.97 & 99.87 & 100.00 & 99.83 & 99.99 & 99.98 & 100.00 & 99.98 & 99.99 & 100.00 & 100.00 & 100.00 & 100.00 & 99.90 & 99.99 \\
\rowcolor{cyan!10}
 \cellcolor{white}\multirow{-2}{*}{\parbox{2.0cm}{\centering \textsc{Llama-3.1} \\ 8B-Instruct}} & FT-UKE + Bayes & \textbf{99.97} & \textbf{99.98} & \textbf{99.88} & 100.00 & \textbf{99.91} & 99.99 & \textbf{99.99} & 100.00 & \textbf{99.99} & \textbf{100.00} & 100.00 & 100.00 & 100.00 & 100.00 & \textbf{99.95} & \textbf{99.99} \\
\midrule
 & FT-UKE & 98.84 & 99.74 & 99.97 & 100.00 & 99.46 & 99.96 & 99.78 & 99.97 & 99.73 & 99.94 & 99.36 & 99.95 & 100.00 & 100.00 & 99.52 & 99.93 \\
\rowcolor{cyan!10}
 \cellcolor{white}\multirow{-2}{*}{\parbox{2.0cm}{\centering \textsc{Qwen-2.5} \\ 7B-Instruct}} & FT-UKE + Bayes & \textbf{99.76} & \textbf{99.96} & 99.79 & 99.97 & \textbf{99.51} & \textbf{99.97} & 99.05 & 99.91 & \textbf{99.93} & \textbf{99.98} & \textbf{99.99} & \textbf{100.00} & 98.60 & 99.90 & \textbf{99.57} & \textbf{99.96} \\
\bottomrule[0.5mm]
\end{tabular}%
}
\end{table*}
\begin{figure*}[t]
    \centering
    \small
    \definecolor{chunkA}{RGB}{235, 245, 255} % 浅蓝背景
    \definecolor{chunkB}{RGB}{255, 240, 230} % 浅橙背景
    \definecolor{badcut}{RGB}{200, 0, 0}     % 红色文字
    \definecolor{goodcut}{RGB}{0, 150, 0}    % 绿色文字

    \newcommand{\cboxA}[1]{\colorbox{chunkA}{\strut #1}}
    \newcommand{\cboxB}[1]{\colorbox{chunkB}{\strut #1}}
    \newcommand{\cutmark}{\textcolor{gray}{\textbf{|}}} % 切分竖线

    % ------ 案例 1: 句法依赖 ------
    \begin{tcolorbox}[colback=white, colframe=gray!30, title=\textbf{Case 1: Syntactic Dependencies (Sample ID 1)}]
        \textbf{Fixed-Window:} ...since 6 is the \cboxA{product of} \cutmark \cboxB{\textcolor{badcut}{\textbf{2}} and 3. The largest...} \\
        \textit{(Analysis: Splitting inside a prepositional phrase "product of ... 2" creates high semantic dependency.)}
        
        \vspace{0.1cm}
        \textbf{Bayes-Chunk:} ...since 6 is the \cboxA{product of 2 and 3.} \cutmark \cboxB{\textcolor{goodcut}{\textbf{The}} largest three-digit...} \\
        \textit{(Analysis: Boundary aligns with the end of the logical clause.)}
    \end{tcolorbox}
    
    % \vspace{0.2cm}

    % % ------ 案例 2: 变量完整性 ------
    % \begin{tcolorbox}[colback=white, colframe=gray!30, title=\textbf{Case 2: Token Coherence (Sample ID 2)}]
    %     \textbf{Fixed-Window:} ...sum of the first n terms, \cboxA{a} \cutmark \cboxB{\textcolor{badcut}{\textbf{\_1}} is the first term...} \\
    %     \textit{(Analysis: The variable name $a_1$ is severed, leaving a dangling subscript prefix.)}
        
    %     \vspace{0.1cm}
    %     \textbf{Bayes-Chunk:} ...sum of the first n \cboxA{terms, a\_1 is the} \cutmark \cboxB{\textcolor{goodcut}{\textbf{nth}} term...} \\
    %     \textit{(Analysis: The variable definition is preserved within a single processing window.)}
    % \end{tcolorbox}

    % \vspace{0.5mm}

    % ------ 案例 3: 逻辑运算 ------
    \begin{tcolorbox}[colback=white, colframe=gray!30, title=\textbf{Case 2: Symbolic Integrity (Sample ID 5)}]
        \textbf{Fixed-Window:} ...Now we can solve for n: ... n = 34 \cboxA{/} \cutmark \cboxB{\textcolor{badcut}{\textbf{2}}} \\
        \textit{(Analysis: The operand "2" is separated from the operator, causing potential hallucination in calculation.)}
        
        \vspace{0.1cm}
        \textbf{Bayes-Chunk:} ...Now we can solve for n: ... n = 34 / 2 \cboxA{} \cutmark \cboxB{\textcolor{goodcut}{\textbf{n}} = 17} \\
        \textit{(Analysis: The full arithmetic expression is maintained intact.)}
    \end{tcolorbox}
    \vspace{-4pt}
    \caption{\textbf{Comparison of Segmentation Boundaries.} We highlight critical failures in Fixed-Window chunking (red text indicates broken context) versus the adaptive boundaries identified by AnyEdit++ (green text indicates coherent continuations). The vertical bar (\textbf{|}) denotes the exact segmentation point.}
    \label{fig:segmentation_examples}
    \vspace{-4pt}
\end{figure*}
% 定义代码高亮样式
\definecolor{codegreen}{rgb}{0,0.6,0}
\definecolor{codegray}{rgb}{0.5,0.5,0.5}
\definecolor{codepurple}{rgb}{0.58,0,0.82}
\definecolor{backcolour}{rgb}{0.95,0.95,0.92}

\lstdefinestyle{mystyle}{
    backgroundcolor=\color{backcolour},   
    commentstyle=\color{codegreen},
    keywordstyle=\color{magenta},
    numberstyle=\tiny\color{codegray},
    stringstyle=\color{codepurple},
    basicstyle=\ttfamily\footnotesize,
    breakatwhitespace=false,         
    breaklines=true,                 
    captionpos=b,                    
    keepspaces=true,                 
    numbers=left,                    
    numbersep=5pt,                  
    showspaces=false,                
    showstringspaces=false,
    showtabs=false,                  
    tabsize=2
}

\lstset{style=mystyle}

% 正文开始

\begin{figure}[t]
\centering
\begin{lstlisting}[language=Python, caption={Core implementation for computing token-level surprisal. We calculate the conditional probability of each token in the answer given the prefix (question and preceding answer tokens).}, label={code:surprisal}]
def compute_surprisal(model, tokenizer, question, answer):
    """
    Computes Bayesian Surprisal for each token in the answer
    conditioned on the question and preceding context.
    """
    model.eval()
    
    # 1. Construct full context
    full_text = f"{question} {answer}"
    enc = tokenizer(full_text, return_tensors="pt", return_offsets_mapping=True)
    input_ids = enc["input_ids"].to(model.device)
    
    # 2. Identify indices corresponding to the answer part
    # (Implementation detail: uses offset_mapping to align chars to tokens)
    answer_indices = find_answer_token_indices(enc, answer)
    
    # 3. Forward pass to get log-probabilities
    with torch.no_grad():
        logits = model(input_ids).logits
        # log_softmax over the vocabulary dimension
        log_probs = torch.log_softmax(logits, dim=-1)

    surprisals = []
    # 4. Calculate Surprisal: -log2 P(x_t | x_{<t})
    for pos_i in answer_indices:
        # Skip the first token of the sequence (no history)
        if pos_i == 0: continue
            
        # The target token at position `pos_i` is predicted 
        # by the hidden state at `pos_i - 1`
        target_token_id = input_ids[0, pos_i]
        log_prob = log_probs[0, pos_i - 1, target_token_id]
        
        # Convert natural log to base-2 surprisal
        surprisal = -log_prob.item() / np.log(2.0)
        surprisals.append(surprisal)

    return np.array(surprisals)
\end{lstlisting}
% \caption{The code we actually use to calculate the Surprisal value}
\end{figure}
% --- 样式定义 (复用你的设计风格) ---
\definecolor{chunkA}{RGB}{235, 245, 255} % 浅蓝背景 (Problem)
\definecolor{chunkB}{RGB}{255, 240, 230} % 浅橙背景 (Solution)
\definecolor{textHL}{RGB}{180, 0, 0}     % 红色文字 (用于强调自我纠错/关键节点)
\definecolor{ansGood}{RGB}{0, 150, 0}    % 绿色文字 (用于最终答案)
% 定义文本块命令
\newcommand{\problemBox}[1]{
    \noindent\colorbox{chunkA}{
        \parbox{\dimexpr\linewidth-2\fboxsep}{
            \textbf{\textsf{Problem:}} \\ #1
        }
    }
}
\newcommand{\solutionBox}[1]{
    \vspace{0.1cm}
    \noindent\colorbox{chunkB}{
        \parbox{\dimexpr\linewidth-2\fboxsep}{
            \textbf{\textsf{Solution (Chain-of-Thought):}} \\ #1
        }
    }
}
% 定义用于高亮思维链内部特征的命令
\newcommand{\selfreflect}[1]{\textcolor{textHL}{\textit{\textbf{[Self-Correction/Review: #1]}}}}
\newcommand{\finalans}[1]{\textcolor{ansGood}{\textbf{#1}}}
% ====== Sample 1 (GSM8K) ======
\begin{tcolorbox}[colback=white, colframe=gray!40, title=\textbf{Sample 1 (Source: GSM8K) - Multi-step Arithmetic}]
    \footnotesize
    \problemBox{A dog runs through a field and is joined by a group of 4 cats. 2 rabbits join each cat and 3 hares join each rabbit. How many animals are running through the field?}
    
    \solutionBox{
        Let's tackle this problem step by step. We start with a dog running through a field. So, that's one animal. Next, a group of 4 cats joins the dog. ... (omitted basic steps) ...
        
        So, the number of hares is: $8 \text{ rabbits} * 3 \text{ hares/rabbit} = 24 \text{ hares}$.
        Adding these 24 hares to the current total of 13 animals, we get: $13 + 24 = 37 \text{ animals}$.
        
        \vspace{0.1cm}
        \selfreflect{Wait a minute, let me double-check that I haven't missed anything.} We started with 1 dog, added 4 cats, then 2 rabbits per cat, and finally 3 hares per rabbit. That seems correct.
        Alternatively, perhaps there's a different way to approach this problem...
        
        \vspace{0.1cm}
        Total: $1 + 4 + 8 + 24 = 37$. I think this is solid.
        \vspace{0.1cm}
        
        \textbf{Final Answer} \\
        \finalans{\[ \boxed{37} \]}
    }
    \vspace{0.1cm} \hrule \vspace{0.1cm}
    \textit{(Analysis: This sample features a lengthy verification process where the model re-calculates using a "tree structure" approach to confirm the initial result.)}
\end{tcolorbox}
% \vspace{0.3cm}
% ====== Sample 2 (Math) ======
\begin{tcolorbox}[colback=white, colframe=gray!40, title=\textbf{Sample 2 (Source: MATH) - Combinatorics}]
    \footnotesize
    \problemBox{In how many ways can a president and a 2-person committee be chosen from a group of 8 people (where the order in which we choose the 2 people doesn't matter)? (The president cannot be on the committee.)}
    
    \solutionBox{
        So I've got this problem here... Let's break this down.
        First, I need to choose a president... implies 8 ways.
        Next, I need to choose a 2-person committee from the remaining 7 people...
        $C(7, 2) = \frac{7!}{2!(7 - 2)!} = 21$.
        
        Total ways = $8 \times 21 = 168$.
        
        \vspace{0.1cm}
        \selfreflect{Wait a minute, does that make sense?} Another way to think about it is to first select the 2-person committee from all 8 people ($C(8,2)=28$), and then choose the president from the remaining 6 people ($6$ ways). $28 \times 6 = 168$.
        
        \selfreflect{Wait, that's the same result as before.}
        
        \textbf{Final Answer} \\
        \finalans{\[ \boxed{168} \]}
    }
    \vspace{0.1cm} \hrule \vspace{0.1cm}
    \textit{(Analysis: The model explicitly tests two different combinatorial paths (President first vs. Committee first) to ensure consistency.)}
\end{tcolorbox}
% \newpage
% ====== Sample 3 (GSM8K) ======
\begin{tcolorbox}[colback=white, colframe=gray!40, title=\textbf{Sample 3 (Source: GSM8K) - Algebra Generation}]
    \footnotesize
    \problemBox{Francine has five full boxes of crayons and 5 loose crayons, and her friend has 27 loose crayons. They need to put all of their loose crayons in a box. How many more boxes do they need if Francine has a total of 85 crayons?}
    
    \solutionBox{
        ...First, I need to figure out how many crayons are in each full box.
        \[5 \times (\text{number of crayons per box}) + 5 = 85\]
        Solving for crayons per box gives 16.
        
        Total loose crayons: $5 + 27 = 32$.
        Dividing by box capacity: $32 / 16 = 2$.
        
        \vspace{0.1cm}
        \selfreflect{Wait a minute, let me double-check that.} Francine already has five full boxes... the question is asking how many MORE boxes... Yes, it seems correct.
        
        \textbf{Final Answer} \\
        \finalans{\[ \boxed{2} \]}
    }
\end{tcolorbox}
% \vspace{0.3cm}
% ====== Sample 4 (GSM8K) ======
\begin{tcolorbox}[colback=white, colframe=gray!40, title=\textbf{Sample 4 (Source: GSM8K) - Temporal Arithmetic}]
    \footnotesize
    \problemBox{There were 80 people at a football game on Saturday. On Monday, 20 fewer people were at the football game... (Sequence of updates)... If their expected total audience... is 350, how many more people attended...?}
    
    \solutionBox{
        Let's list out the number for each day:
        \begin{itemize}
            \item Saturday: 80
            \item Monday: $80 - 20 = 60$
            \item Wednesday: $60 + 50 = 110$
            \item Friday: $80 + 60 = 140$
        \end{itemize}
        Total attendance = $80 + 60 + 110 + 140 = 390$.
        Difference = $390 - 350 = 40$.
        
        \textbf{Final Answer} \\
        \finalans{\[ \boxed{40} \]}
    }
\end{tcolorbox}
\vspace{0.3cm}
% ====== Sample 5 (GSM8K) ======
\begin{tcolorbox}[colback=white, colframe=gray!40, title=\textbf{Sample 5 (Source: GSM8K) - Logic Trap}]
    \footnotesize
    \problemBox{Heather is going to sew 150 aprons... She already sewed 13, today sewed 3x as many... How many should she sew tomorrow if she wants to sew \underline{half of the remaining} number?}
    
    \solutionBox{
        ...Today she sewed: $3 \times 13 = 39$.
        Total sewn so far: $13 + 39 = 52$.
        Remaining needed: $150 - 52 = 98$.
        
        Target for tomorrow is half of remaining: $98 / 2 = 49$.
        
        \vspace{0.1cm}
        \selfreflect{But wait, let's double-check that.} After sewing 49 tomorrow, total sewn is $52+49=101$. Remaining is $49$. So sewing 49 is exactly half of the 98 derived needed. Makes sense.
        
        \textbf{Final Answer} \\
        \finalans{\[ \boxed{49} \]}
    }
\end{tcolorbox}