EMAG: Self-Rectifying Diffusion Sampling with Exponential Moving Average Guidance
Extracted source text · arXiv 2512.17303
% CVPR 2026 Paper Template; see https://github.com/cvpr-org/author-kit
\documentclass[10pt,twocolumn,letterpaper]{article}
%%%%%%%%% PAPER TYPE - PLEASE UPDATE FOR FINAL VERSION
% \usepackage{cvpr} % To produce the CAMERA-READY version
% \usepackage[review]{cvpr} % To produce the REVIEW version
\usepackage[pagenumbers]{cvpr} % To force page numbers, e.g. for an arXiv version
% Import additional packages in the preamble file, before hyperref
\input{preamble}
% It is strongly recommended to use hyperref, especially for the review version.
% hyperref with option pagebackref eases the reviewers' job.
% Please disable hyperref *only* if you encounter grave issues,
% e.g. with the file validation for the camera-ready version.
%
% If you comment hyperref and then uncomment it, you should delete *.aux before re-running LaTeX.
% (Or just hit 'q' on the first LaTeX run, let it finish, and you should be clear).
\definecolor{cvprblue}{rgb}{0.21,0.49,0.74}
\usepackage[pagebackref,breaklinks,colorlinks,allcolors=cvprblue]{hyperref}
% Custom Packages for Algorithm formating
\usepackage{algorithm}
\usepackage{algorithmic}
% \usepackage{amsmath}
% \usepackage{amssymb}
% \newcommand{\lingqiao}[0]{\textcolor{red}}
%%%%%%%%% PAPER ID - PLEASE UPDATE
% \def\paperID{2705} % *** Enter the Paper ID here
\def\confName{CVPR}
\def\confYear{2026}
%%%%%%%%% TITLE - PLEASE UPDATE
\title{EMAG: Self-Rectifying Diffusion Sampling with Exponential Moving Average Guidance}
\author{
Ankit Yadav \quad Ta Duc Huy \quad Lingqiao Liu\\
\small Australian Institute for Machine Learning, The University of Adelaide, Australia\\
\tt\small \{ankit.yadav, huy.ta, lingqiao.liu\}@adelaide.edu.au
}
\begin{document}
\maketitle
\begin{abstract}
In diffusion and flow-matching generative models, guidance techniques are widely used to improve sample quality and consistency. Classifier-free guidance (CFG) is the de facto choice in modern systems and achieves this by contrasting conditional and unconditional samples. Recent work explores contrasting negative samples at inference using a weaker model, via strong/weak model pairs, attention-based masking, stochastic block dropping, or perturbations to the self-attention energy landscape. While these strategies refine the generation quality, they still lack reliable control over the granularity or difficulty of the negative samples, and target-layer selection is often fixed. We propose \emph{Exponential Moving Average Guidance (EMAG)}, a training-free mechanism that modifies attention at inference time in diffusion transformers, with a statistics-based, adaptive layer-selection rule. Unlike prior methods, EMAG produces harder, semantically faithful negatives (fine-grained degradations), surfacing difficult failure modes, enabling the denoiser to refine subtle artifacts, boosting the quality and human preference score (HPS) by \textbf{+0.46} over CFG.
We further demonstrate that EMAG naturally composes with advanced guidance techniques, such as APG and CADS, further improving HPS.
\end{abstract}
\section{Introduction}
\label{sec:intro}
\begin{figure}[t]
\centering
\includegraphics[width=1\linewidth]{resources/Slide2.jpg}
\caption{\textbf{CFG (left) vs. EMAG (right)} Compared to CFG \cite{ho2022classifier}, EMAG produces more \textit{semantically plausible} images, preserving global structure while sharpening fine details and suppressing minor artifacts. When applied to SD3-Medium, EMAG’s outputs align more closely with human-preference proxies (HPS \cite{wu2023human}).
}
\label{fig:front_image}
\end{figure}
In recent times, Denoising Diffusion Models (DDMs) \cite{sohl2015deep,ho2020denoising,dhariwal2021diffusion,song2020score} and flow\mbox{-}matching models \cite{lipman2022flow,esser2024scaling} have surpassed GANs \cite{goodfellow2020generative} at modeling data across modalities (images \cite{rombach2022high}, audio \cite{wang2023audit}, video \cite{blattmann2023stable}).Their success stems from an iterative denoising process that starts from Gaussian noise and progressively removes it to sample the target distribution \cite{kingma2023understanding}.The advent of Conditional diffusion models, where a condition such as a class \cite{dhariwal2021diffusion} or text prompt \cite{nichol2021glide} can be used to guide the generation, further propelled their popularity. However, the results with just the naive guidance signals have insufficient quality and weaker prompt adherence. \cite{mukhopadhyay2023diffusion}, leading to the introduction of strategies like classifier guidance \cite{dhariwal2021diffusion} and classifier-free guidance \cite{ho2022classifier}, where the prior method requires an explicit guidance model while the latter relies on the implicit Bayesian classifier of the same diffusion model by contrasting the conditional with the unconditional signal. This led to a significant improvement in conditional generation quality and prompt adherence. While these improve conditional quality and alignment, they can reduce diversity and induce over\mbox{-}saturation \cite{sadat2024eliminating}. Recent work addresses these effects by providing more semantic aware guidance (S-CFG) \cite{shen2024rethinking}, decomposing the CFG update into parallel and orthogonal components and down-weights the parallel term (with rescaling/momentum) to mitigate oversaturation (APG) \cite{sadat2024eliminating}, and by perturbing the conditioning signal over a annealed schedule improving diversity (CADS) \cite{sadat2023cads}.
Another line of work exposes failure modes by contrasting strong and weak predictions. Such negative (weak) signals are obtained by perturbing attention maps \cite{ahn2024self,hong2023improving} or weights \cite{hong2024smoothed}, by guiding with deliberately weaker models (reduced capacity or degraded variants) \cite{karras2024guiding}, by sampling sub-networks \cite{chen2025s}, or via energy-based formulations such as ERG \cite{ifriqi2025entropy} and SEG \cite{hong2024smoothed}. These strategies are attractive because, unlike CFG, they apply to both conditional and unconditional settings and often yield \emph{Pareto-favorable} trade-offs. However, most approaches provide a mechanism to obtain an inferior model but do not explicitly construct \emph{hard counterexamples}. We hypothesize that if degradation is too strong (e.g., heavily distorting an image), modern diffusion models can often repair it, yielding \emph{easy} negatives. Motivated by this, we propose Exponential Moving Average Guidance (EMAG). which, unlike other approaches, by design can regulate how much \emph{high\mbox{-}frequency} information the weaker model is allowed to destroy by controlling the EMA updates, deliberately injecting subtle, semantically faithful artifacts. These yield \emph{hard negatives} that reveal failure modes that base models are more likely to miss, enabling more effective rectification during sampling, see Fig~\ref{fig:front_image}. In practice, EMAG improves HPS by +0.46 over CFG and further boosts advanced guidance when composed with APG and CADS, underscoring the value of controllable, fine-grained negatives in high-quality regimes.
\paragraph{Contributions}
\begin{itemize}
\item We propose \textbf{EMAG}, a training\mbox{-}free attention-map EMA guidance that produces \emph{controllable} weak signals without extra training.
\item We propose a statistics\mbox{-}based \emph{adaptive layer selection} that chooses target layers per timestep to create controllable degradations in negative samples.
\item We conduct extensive experiments on DiT~\cite{peebles2023scalable} and MMDiT (SD3~\cite{esser2024scaling}) backbones, covering both class\mbox{-}conditional and text\mbox{-}to\mbox{-}image generation, and report competitive performance against strong guidance baselines, including notable improvement in Human Preference Score (HPS)~\cite{wu2023human}.
\item We demonstrate the complementary nature of our approach with advanced guidance methods such as APG\cite{sadat2024eliminating} and CADS\cite{sadat2023cads}, consistently improving HPS.
\end{itemize}
% ###################################### TABLE #####################################################################################
\section{Related work}
\label{sec:Related work}
Classifier-free guidance (CFG) \cite{ho2022classifier} combines conditional and unconditional predictions to improve prompt alignment and perceptual quality. However, CFG suffers from inherent drawbacks such as reduced generation diversity \cite{astolfi2024consistency, ifriqi2025entropy, sadat2023cads} and visual over-saturation \cite{ho2022classifier}. To alleviate these issues, recent studies have revisited CFG and developed strategies that better balance semantic alignment and visual fidelity by decomposing the CFG update into orthogonal/parallel components and down-weighting saturation-prone components, enabling higher guidance scales with fewer artifacts, Adaptive Projected Guidance (APG) \cite{sadat2024eliminating}, or by boosting diversity via annealing the conditioning strength during inference, especially at high guidance scales, Condition-Annealed Diffusion Sampler (CADS) \cite{sadat2023cads}.
A useful unifying view is \emph{auto-guidance} \cite{karras2024guiding}, where a strong model is contrasted with a deliberately weakened (reduced parameters or training steps) variant to steer sampling. This separation helps stabilize updates and expose failure modes, highlighting the artifacts typical of high guidance scales. \emph{Karras et al.}\ report that autoguidance disentangles control over image quality while preserving variation, yielding more favorable quality–diversity trade-offs than CFG \cite{karras2024guiding}.
However, obtaining a pair of strong and weak models may not be optimal in all scenarios; hence, several strategies have been introduced to achieve a similar effect with a single model. Self-Attention Guidance (SAG) degrades the \emph{input} by blurring regions indicated by self-attention, yielding a weaker score that guides the original model \cite{hong2023improving}. Smoothed Energy Guidance (SEG) weakens the prediction by smoothing the attention energy (Gaussian blurring of attention weights), with an efficient \emph{query blurring} equivalent; in practice, one can approximate this by \emph{replacing image queries with their channel-wise means} \cite{hong2024smoothed}. Entropy Rectifying Guidance (ERG) perturbs the attention mechanism (including conditioning pathways) at inference to improve quality, diversity, and prompt consistency \cite{ifriqi2025entropy}. Perturbed-Attention Guidance (PAG) forms weak intermediates by replacing selected self-attention maps with the identity and then guiding away from them \cite{ahn2024self}. S$^2$-Guidance (Stochastic Self-Guidance) constructs weak sub-networks by stochastically dropping layers \cite{chen2025s}.
While guiding with weaker/negative signals can outperform CFG, most methods offer limited control or diagnostics over the \emph{quality} (hardness and granularity) of these negatives. We introduce \emph{Exponential Moving Average Guidance (EMAG)}, a training-free approach that generates \emph{controllable, semantically faithful hard negatives} by regulating fine-grained (high-frequency) degradations via attention-space EMA and adaptive layer selection. This design exposes hard failure modes that coarse degradations often miss and complements advanced guidances such as APG and CADS.
\section{Method}
\label{sec:Method}
\subsection{Diffusion preliminary}
\begin{table}[t]
\centering
\small
\setlength{\tabcolsep}{6pt}
\caption{We compare SOTA guidance methods that complement CFG against our EMAG. Across SD3 experiments using the identical COCO 2014 validation split, sampling steps, and evaluation code, we observe consistent gains in HPS when EMAG is combined with other guidance strategies. \emph{*EMAG naturally extends CFG, so CFG+EMAG $\equiv$ EMAG}.}
\label{tab:combination_experiment}
\begin{tabular}{lcc}
\toprule
\textbf{Guidance} & \textbf{HPS} $\uparrow$ & \textbf{HPS} $\uparrow$ \\
\midrule
& & \textbf{+EMAG} \\
\midrule
\textbf{CFG} & 29.22 & \textbf{29.68}* \\
\textbf{ERG} & 29.35 & \textbf{29.56} \\
\textbf{S2} & 29.15 & \textbf{29.51} \\
\textbf{SAG + CFG} & 28.40 & - \\
\textbf{SEG + CFG} & 29.25 & \textbf{29.29} \\
\midrule
\textbf{EMAG} & \textbf{29.68} & - \\
\bottomrule
\end{tabular}
\end{table}
% ###################################### TABLE #####################################################################################
\paragraph{Diffusion preliminaries.}
In diffusion-based generation~\cite{dhariwal2021diffusion,lipman2022flow,esser2024scaling}, a clean image \(x_0\) is progressively corrupted with Gaussian noise \(\epsilon\!\sim\!\mathcal{N}(0,1)\) over timesteps \(t\in[0,T]\), yielding noisy samples \(x_t\) and this is known as the forward process. This can be represented as a stochastic differential equation (SDE)
\begin{equation}
\label{eq:preelim-1}
dx \;=\; f(x,t)\,dt \;+\; g(t)\,dw,
\end{equation}
where \(f(x,t)\) and \(g(t)\) control the corruption process and \(w\) is a standard Wiener process. To recover \(x_0\) from \(x_t\), one integrates the reverse-time SDE~\cite{song2020denoising}
\begin{equation}
\label{eq:preelim-2}
dx \;=\; \big[f(x,t) \;-\; g(t)^2 \nabla_x \log p_t(x)\big]\,dt \;+\; g(t)\,d\bar{w},
\end{equation}
where \(\nabla_x \log p_t(x)\) is the score of the perturbed distribution and \(\bar w\) is a standard Wiener process in reverse time. A neural network (e.g., a Transformer) is trained to approximate the score, \(e_\theta(x,t)\!\approx\!\nabla_x\log p_t(x)\), enabling generation from random Gaussian noise. With external conditioning \(c\) (e.g., class labels or text), the conditional score is learned, \(e_\theta(x,t,c)\!\approx\!\nabla_x\log p_t(x\mid c)\)~\cite{dhariwal2021diffusion}.
\paragraph{Flow matching}
Flow matching trains a \emph{deterministic} neural ODE to transport a simple prior to the data by regressing a time-dependent velocity field:
\begin{equation}
\label{eq:fm-ode}
\dot{x} \;=\; v_\theta(x,t),
\end{equation}
with the standard regression objective
\begin{equation}
\label{eq:fm-loss}
\min_{\theta}\;
\mathbb{E}_{t\sim\mathcal{U}(0,1),\,(x_0,x_1)}\;
\big\|\,v_\theta\!\big((1{-}t)x_0{+}t x_1,\,t\big) \;-\; (x_1{-}x_0)\big\|_2^{2},
\end{equation}
where using the linear path yields the popular \emph{rectified flow} instance.
In contrast, diffusion employs a \emph{stochastic} reverse SDE or its deterministic probability-flow ODE; the latter evolves the same marginals as the reverse SDE:
\begin{equation}
\label{eq:pf-ode}
\frac{dx}{dt} \;=\; f(x,t)\;-\;\tfrac{1}{2}\,g(t)^{2}\,\nabla_x \log p_t(x).
\end{equation}
\subsection{Existing guidance approaches}
% * image for negatvie sample
% * categorize the methods into category and discuss them briefly after diffusion method. connecting with diffusion try to have common equations for them.
Conditioned Diffusion models have recently achieved the capability to generate high-quality multimedia. Classifier Free Guidance (CFG) \cite{ho2022classifier} is one of the major breakthroughs behind this dramatic improvement in the generation quality, and prompt alignment, where the unconditional and conditional score estimates are combined to create the final estimate with the help of guidance scale as depicted in Eq. \ref{eq:cfg_eq} where $\epsilon_\theta$ is the denoiser neural network, $x_t$ is the input latent, $c$ is the conditioning input and $\varnothing$ is null input. Hence $\epsilon_\theta(x_t, \varnothing)$ and $\epsilon_\theta(x_t, c)$ are the denoiser network with null-conditioning (unconditional) and conditioning and guidance scale $w > 1$.
\begin{equation}
\label{eq:cfg_eq}
\tilde{\epsilon}_\theta(x_t, c) = \epsilon_\theta(x_t, \varnothing) + w \cdot (\epsilon_\theta(x_t, c) - \epsilon_\theta(x_t, \varnothing))
\end{equation}
However, CFG suffers from inherent drawbacks such as reduced generation diversity and visual over-saturation . To alleviate these issues, recent studies have revisited CFG and developed strategies that better balance semantic alignment and visual fidelity. One such direction is Auto-guidance , which employs two versions of the denoiser that share the same network structure but differ in capacity. Specifically as shown in Eq. \ref{eq:autoguidance},
\begin{equation}
\label{eq:autoguidance}
\tilde{\epsilon}(x_t, c) = \epsilon_0(x_t, c) + w \cdot (\epsilon_1(x_t, c) - \epsilon_0(x_t, c))
\end{equation}where the weaker model is denoted as $\epsilon_0$ and the stronger model as $\epsilon_1$. Here the weaker model’s score prediction guides the stronger one in Eq. \ref{eq:autoguidance}.
This Auto-guidance framework in Eq. \ref{eq:autoguidance} can be achieved by using different ways to
%perspective unifies a series of works that
construct the weaker model through controlled degradation. For example, SEG forms the weaker model by directly perturbing the denoiser’s attention weights, such as by adding Gaussian noise or replacing image queries with their channel-wise means. Let the perturbed denoiser be $\epsilon_1'$. With this weaker version defined, replacing $\epsilon_0(x_t,c)$ with $\epsilon_1'(x_t,\varnothing)$ in Eq. \ref{eq:autoguidance} produces the SEG guidance formulation (only for unconditional).
More recent approaches, such as ERG, extend this perturbation to both the network and the conditioning encoder, resulting in a perturbed denoiser $\epsilon_1'$ and perturbed conditioning $c'$. Substituting these into Eq. \ref{eq:autoguidance} leads to the ERG objective, i.e., replacing $\epsilon_0(x_t,c)$ with $\epsilon_1'(x_t,c')$.
\begin{equation}
\label{eq:PAG}
\tilde{\epsilon}(x_t) = \epsilon_1(x_t) + w \cdot (\epsilon_1(x_t) - \epsilon_1'(x_t))
\end{equation}
PAG perturbs self-attention by replacing selected attention maps with the identity, and then guides sampling away from the resulting degraded intermediates; its guidance rule departs from Eq.~\ref{eq:autoguidance} and is given in Eq.~\ref{eq:PAG}, where $\epsilon_1'$ denotes the network with perturbed attention. Similarly, SAG uses self-attention maps from the denoiser $\epsilon_\theta$ (the score prediction network) to detect high-frequency regions in the latent input. These regions are then selectively blurred to obtain a degraded latent $x_t'$, effectively producing a weaker model $\epsilon_1(x_t')$. Substituting this degraded score into Eq. \ref{eq:PAG} in place of perturbed noise $\epsilon_1'(x_t)$ yields the SAG objective.
Alternatively,
\emph{S$^2$-Guidance} constructs the weaker surrogate via stochastic layer dropping and, instead of Eq.~\ref{eq:autoguidance}, improves the CFG update by subtracting the perturbed model’s noise prediction scaled by $s$; equivalently, Eq.~\ref{eq:cfg_eq} is augmented with the term $-\,s \cdot \epsilon_1'(x_t, c, m)$, where $m$ controls which layers are dropped at each timestep, $\epsilon_1'$ is the perturbed sub-network, and $s$ is the S$^2$ scale.
In the auto-guidance framework, a weaker model helps steer sampling toward higher-quality regions\cite{karras2024guiding}. When derived from the same backbone, the weaker model tends to replicate the strong model’s errors but with greater magnitude. This discrepancy highlights failure modes that the strong model can correct. \cite{karras2024guiding}. Following this intuition, we target subtle, fine-grained errors (\emph{hard negatives}) by constructing lightly degraded surrogates that act as guidance, improving fidelity and HPS \cite{wu2023human}. Our approach follows the attention-perturbation family, e.g., ERG, PAG, and SEG, which instantiate such surrogates by modifying self-attention during sampling.
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{resources/negative_sample_grid.png}
\caption{\textbf{Negative-sample comparison (top: negative samples; bottom: positive samples).} Prior guidance (SAG \cite{hong2023improving}, auto guidance \cite{karras2024guiding}, ERG \cite{ifriqi2025entropy}, S$^2$-Guidance \cite{chen2025s}) often yield obvious “easy” degradations. In contrast, \textbf{EMAG} produces subtle, semantically near-miss negatives that reveal difficult failure modes yet retain global structure, enabling finer refinement. (Samples from DiT)}
\label{fig: hard_negatives}
\end{figure}
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{resources/Negative_example_degradation_control.png}
\caption{\textbf{CFG(left) vs\ EMAG hard negatives(right)} EMAG samples at varying $\beta$; larger $\beta$ produces stronger degradations.}
\label{fig:Negative_sample_controlled}
\end{figure}
\subsection{Why quality of negative samples matters?}
In prior approaches, including Autoguidance \cite{karras2024guiding}, Entropy Rectifying Guidance (ERG) \cite{ifriqi2025entropy}, and S$^2$-Guidance \cite{chen2025s} instantiate a weaker surrogate to surface failure modes complementary to a strong model, thereby guiding sampling toward higher quality. However, these methods primarily specify how to obtain a weaker surrogate (e.g., capacity reduction, attention perturbation, or stochastic layer dropping) rather than offering direct, fine-grained control over which failure modes to elicit and to what extent. In this work, we introduce a strategy that explicitly controls the degradation level while preserving core semantics, yielding “hard negatives”, subtle, fine-grained degradations that expose nuanced errors without altering global structure and enable the model to rectify them see Fig~\ref{fig: hard_negatives} and \ref{fig:Negative_sample_controlled}. As evidenced in Fig~\ref{fig:front_image} and Table~\ref{tab:combination_experiment}, this controllable hard-negative guidance improves the Human Preference Score (HPS) \cite{wu2023human} over both the baseline and several strong methods.
\subsection{Exponential Moving Average Guidance}
We adopt a strategy akin to prior works \cite{karras2024guiding, hong2023improving, hong2024smoothed} where we contrast denoising outputs from a strong model versus a deliberately degraded weaker prediction via Eq.~\ref{eq:autoguidance}. Following \cite{hong2024smoothed, ifriqi2025entropy, ahn2024self}, we perturb self‐attention by replacing the attention map at a selected layer and timestep \(t\), \(A_t\), with its exponential moving average \(\mathrm{EMA}(A_{<t})\), using our adaptive layer‐selection strategy (Sec~\ref{sec:layer selection}). The detailed steps are provided in Alg.~\ref{alg:EMAG-uncond} and Alg.~\ref{alg:EMAG-cond}. Intuitively, since self‐attention refines semantics from coarse to fine, swapping in the EMA suppresses high‐frequency refinements while preserving global structure, yielding a controllable, semantically faithful “hard negative.” We then contrast this negative with the original prediction during sampling. Figure~\ref{fig: hard_negatives} shows that our EMA‐guided approach degrades selectively without losing core semantic details, and Fig.~\ref{fig:Negative_sample_controlled} demonstrates how adjusting the decay factor \(\beta\) controls the degradation strength. Now we will discuss our approach and its components in detail.
% ---------------- Algorithm 1: Unconditional ----------------
\begin{algorithm}[t]
% \caption{Exponential Moving Average Guidance (Unconditional)}
\caption{EMAG: (Unconditional)}
\label{alg:EMAG-uncond}
\begin{algorithmic}[1]
\REQUIRE $\tau_s$: starting timestep for EMAG
\REQUIRE $\tau_e$: end timestep for EMAG
\REQUIRE $\epsilon_\theta$: Denoiser network
\REQUIRE $x_t$: input at timestep $t$
\REQUIRE $w_{e}$: EMAG Scale
\STATE $x_T \sim \mathcal{N}(0,I)$
\FOR{$t = T, T-1, \ldots, 1$}
\STATE $z_t \leftarrow \epsilon_\theta(x_t)$
\IF{$\tau_e < t < \tau_s$}
\STATE $\hat{z}_t \leftarrow \epsilon'_\theta(x_t)$ \COMMENT{Eq.~\eqref{eq: layer_selection}, \eqref{eq: L1_norm_calcuation}, \eqref{eq: EMA replacement}}
\STATE $\bar{z}_t \leftarrow \hat{z}_t + w_{e}\cdot(z_t - \hat{z}_t)$ \COMMENT{Eq.~\eqref{eq:EMAG_uncond_update}}
\ELSE
\STATE $\bar{z}_t \leftarrow z_t$
\ENDIF
\ENDFOR
\RETURN $\bar{z}_t$
\end{algorithmic}
\end{algorithm}
% ---------------- Algorithm 2: Conditional (with CFG) ----------------
\begin{algorithm}[t]
% \caption{Exponential Moving Average Guidance (Conditional)}
\caption{EMAG (Conditional)}
\label{alg:EMAG-cond}
\begin{algorithmic}[1]
\REQUIRE \textbf{Same declarations as Alg.~\ref{alg:EMAG-uncond}; additionally:}
\REQUIRE $c$ (optional): Conditional Signal,\;
\REQUIRE $w_{cfg}$ (optional): CFG Scale
\STATE $x_T \sim \mathcal{N}(0,I)$
\FOR{$t = T, T-1, \ldots, 1$}
\STATE $z_t^{c} \leftarrow \epsilon_\theta(x_t,c)$
\STATE $z_t \leftarrow \epsilon_\theta(x_t)$
\IF{$\tau_e < t < \tau_s$}
\STATE $\hat{z}_t^{c} \leftarrow \epsilon'_\theta(x_t,c)$ \COMMENT{Eq.~\eqref{eq: layer_selection}, \eqref{eq: L1_norm_calcuation}, \eqref{eq: EMA replacement}}
\STATE $\bar{z}_t^{c} \leftarrow \hat{z}_t^{c} + w_{e}\cdot(z_t^{c} - \hat{z}_t^{c})$ \COMMENT{Eq.~\eqref{eq:EMAG_gudiance_update}}
\STATE $\bar{z}_t \leftarrow z_t + w_{cfg}\cdot(\bar{z}_t^{c} - z_t)$ \COMMENT{Eq.~\eqref{eq:EMAG_gudiance_update_2}}
\ELSE
\STATE $\bar{z}_t \leftarrow z_t + w_{cfg}\cdot(z_t^{c} - z_t)$ \COMMENT{Eq.~\eqref{eq:cfg_eq}}
\ENDIF
\ENDFOR
\RETURN $\bar{z}_t$
\end{algorithmic}
\end{algorithm}
\subsubsection{Attention EMA calculation}
Let $A_t \in [0,1]^{B\times H\times Q\times K}$ denote the softmax-normalized attention at timestep $t$ (rows sum to $1$ over $K$).
We maintain an exponential moving average (EMA) $E_t$ of the same shape as $X_t$, initialized on the first call as $E_1 \!:=\! X_1$, and updated as mentioned in Eq.\ref{eq:EMA update}.
% In implementation, $X_t$ is cast to \texttt{float32} for numerical stability.
\begin{equation}
\label{eq:EMA update}
E_t \;=\; \beta\, E_{t-1} \;+\; (1-\beta)\, X_t, \qquad \beta=0.988.
\end{equation}
We set the beta with a halflife $H = 50 $ using the equation $\beta = e^{-ln(2)/H}$. It's interesting to note that the higher the value of H, the stronger the perturbation to the energy landscape. We also introduce parameters $\delta_t, \tau_s, \tau_e$ where $\delta_t$ indicates the warmup period for the EMA to stabilize before we can start swapping the attention maps with the EMA version, $\tau_s$ indicates the timesteps when we start accumulating EMA and apply the perturbation, and $\tau_e$ indicates the timestep when we don't perturb the attention. Since the denoising happens in reverse, we have $t_0 < \tau_e < \tau_s<t_{max}$.
Replacing the attention map with its EMA effectively suppresses the incremental (high-frequency) refinements the model applies over timesteps while preserving the global structure. This induces inferior but semantically faithful predictions that guide sampling toward higher quality by introducing precise failure modes. In practice, the exponential smoothing prevents drift: even with $\delta_t = 1$, the guidance remains faithful to the original structure, unlike strong perturbation approaches \cite{chen2025s,ifriqi2025entropy} (see Fig.~\ref{fig: hard_negatives}).
\subsubsection{Layer Selection}
\label{sec:layer selection}
As prior work shows \cite{ifriqi2025entropy, hong2023improving, ahn2024self}, middle layers are especially effective for attention perturbations. For instance, \cite{ifriqi2025entropy} demonstrates that middle layers exhibit lower maximum attention probabilities than first or last layers. Methods like \cite{hong2023improving, ahn2024self} also extract masks or apply perturbations in these middle layers of the U-Net. Inspired by this, we apply EMAG specifically to middle layers: for DiT-XL/2 \cite{peebles2023scalable} we set $(l_{\min}, l_{\max}) = (12,15)$, and for SD3-Medium \cite{esser2024scaling} we set $(l_{\min}, l_{\max}) = (6,8)$.
At each timestep $t$ and layers $(l_{min},l_{max})$ we select a layer $l_n \text{ where } n \in (l_{min},l_{max})$ using the Eq.\ref{eq: layer_selection} and Eq. \ref{eq: L1_norm_calcuation} where $l^t_n$ is the selected layer at time step $t$ and $|A_t|$ is the total number of elements in the attention map. For per-layer selection, we compute $\Delta_{t}^{(\ell)}$ on the attention tensor of layer $\ell$ and choose layers with the largest $\Delta_{t}^{(\ell)}$.
% \begin{equation}
\begin{align}
\label{eq: layer_selection}
l^t_n = \arg\max_{n \in \{l_{min}, ..., l_{max}\}} \Delta_{t}^{(n)}
% \end{equation}
\\
% \begin{equation}
\label{eq: L1_norm_calcuation}
\Delta_{t} \;=\; \operatorname{MAE}(E_t, A_t) = \frac{1}{|A_t|}\sum_{i} \bigl| E_t^{(i)} - A_t^{(i)} \bigr|
% \end{equation}
\end{align}
Finally, we replace the EMA for the attention layer using a convex blend, see Eq.\ref{eq: EMA replacement} with $\lambda = 1$ (hard replace) as our default setting. $\lambda \in [0, 1]$ allows for a soft update, and $\widetilde{A}_t$ is the perturbed attention map.
\begin{equation}
\label{eq: EMA replacement}
\widetilde{A}_t \;=\; (1-\lambda)\,A_t \;+\; \lambda\,E_t,
\end{equation}
This enables us to select the layer with the largest ~$\Delta_t$ at each timestep~$t$, ensuring maximal perturbation. Since different layers contribute unequally across timesteps~\cite{ma2024learning}, our statistic-guided selection picks the layer actively inserting high-frequency content, yielding stronger negative samples. This approach also opens the possibility for providing more controlled negative samples by carefully designing optimization for the $\Delta_{t}^{(\ell)}$ as against the greedy approach Eq. \ref{eq: layer_selection}. We defer this to future work.
\subsubsection{Hopfield view and our motivation}
\label{sec:hopfiled_view}
Hopfield Networks \cite{hopfield1982neural} are memory models to connect an input with similar patterns by building an energy landscape with the basins of attraction representing the desired patterns. Recently \cite{ramsauer2020hopfield} introduced Modern Hopfield networks with a new energy function extending Hopfield networks to continuous states Eq. \ref{eq:hopfield_1}, and for $ 0 \leq E \leq 2M^2 $ and using $p=softmax(\beta X^T \zeta)$ define a update rule Eq. \ref{eq:energy_update_rule} where $N$ is the count of stored patterns (keys) $x_i \in \mathcal{R^d}$ and $X = {x_1,...,x_N}$, $M=max_i||x_i||$ , state pattern (query) $\zeta \in \mathcal(R^d)$ and lse is \textit{log-sum-exp function} for $0 < \beta$ for more detailed view refer to \cite{ramsauer2020hopfield}.
\begin{equation}
\label{eq:hopfield_1}
E = -lse(\beta,X^T\zeta) + \frac{1}{2}\zeta^T\zeta + \beta^{-1}logN + \frac{1}{2}M^{2} \quad \text{\cite{ramsauer2020hopfield}}
\end{equation}
\begin{equation}
\label{eq:energy_update_rule}
\zeta^{new} = Xsoftmax(\beta X^T \zeta) \quad \text{\cite{ramsauer2020hopfield}}
\end{equation}
From Eq.\ref{eq:energy_update_rule} we observe that the attention mechanism is equivalent to the update rule in modern hopfield network.
% \cite{ramsauer2020hopfield}
Crucially, Ramsauer et\,al.\ prove that (i) the dynamics in~\eqref{eq:energy_update_rule} monotonically decrease the energy and converge to a stationary point (\textbf{Thm.\,1 and Thm.\,2: global convergence}); (ii) the model admits \emph{exponential} storage capacity in dimension $d$ (\textbf{Thm.\,3}); and (iii) retrieval can succeed after a \emph{single} update (\textbf{Thm.\,4 and Thm.\,5: one-step retrieval and small retrieval error })~\cite{ramsauer2020hopfield}. In the context of diffusion transformers, repeated attention applications across timesteps therefore act as successive energy-descent refinements of a state, empirically evolving attention from coarse to fine structure.
% (cf.\ Fig.~\ref{fig:negative_sampling}).
Building on the intuitions from the Continuous Hopfield Networks \cite{ramsauer2020hopfield} model of Self-Attention, see Eq.\ref{eq:energy_update_rule} obtained from \cite{ramsauer2020hopfield}., which suggests that for $t \rightarrow\infty$ the energy $E(\zeta^t) \rightarrow E(\zeta^*)$ and any sequence $\{\zeta^t\}^\infty_{t=0}$ generated from Eq.\ref{eq:energy_update_rule} will converge to a fixed point $\zeta^*$ where $\zeta \in \mathbb{R}^d $ is the state query in our case the query vector in self attention. Hence, from this observation, we can say that the self attention converges from coarse to finer detail with each time step or application of the update rule Eq.\ref{eq:energy_update_rule}, detailed discussion in Supplementary Sec.~\ref{sec:background}. These insights provide a strong motivation to employ EMA in attention-space: as a low-pass operator, EMA selectively attenuates high-frequency content, enabling construction of \emph{hard negatives} via subtle, semantics-preserving degradations.
\subsubsection{EMAG-I}
In MMDiT's joint attention over text and image tokens, our default EMAG perturbs both Image$\rightarrow$Image and Image$\rightarrow$Text sub-blocks. While \textbf{EMAG-I} perturbs only the Image$\rightarrow$Image block. See Supplementary Sec.~\ref{sec:additional_implementation_details}.
\subsection{Guidance update}
Once we obtain the perturbed noise estimates, we perform the guidance update in two steps first, we leverage Eq. \ref {eq:autoguidance} for the guidance update and rewrite it as Eq.\ref{eq:EMAG_gudiance_update} presenting the first step of Guidance update for EMAG, where $\epsilon'_\theta(x_t, c)$ is the degraded score estimate and $w_{e}$ is the EMAG scale. Now using this perturbed signal, we perform a CFG-style update but with our perturbed conditional noise, see eq.\ref{eq:EMAG_gudiance_update_2} where $w$ is CFG scale.
\begin{align}
\label{eq:EMAG_gudiance_update}
&\tilde{\epsilon}(x_t, c)' = \epsilon'_\theta(x_t, c) + w_{e} \cdot (\epsilon_\theta(x_t, c) - \epsilon'_\theta(x_t, c))
\\
\label{eq:EMAG_gudiance_update_2}
&\tilde{\epsilon}(x_t, c) = \epsilon_\theta(x_t, \varnothing) + w \cdot (\epsilon_\theta(x_t, c)' - \epsilon_\theta(x_t, \varnothing))
\end{align}
We can also apply EMAG for unconditional generation as presented in Eq. \ref{eq:EMAG_uncond_update}.
\begin{equation}
\label{eq:EMAG_uncond_update}
\tilde{\epsilon}(x_t) = \epsilon'_\theta(x_t , \varnothing) + w_{e} \cdot (\epsilon_\theta(x_t) - \epsilon'_\theta(x_t , \varnothing))
\end{equation}
% ########################################### Uncond and cond generation example ##################################################################
\begin{figure*}[t]
\centering
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{resources/uncondtional_sample_grid_sd3_small.png}
% \caption{}
\caption{Unconditional generation. \textbf{Top:} EMAG. \textbf{Bottom:} unguided (baseline).}
\label{fig:uncond-samples}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.48\linewidth}
\centering
\includegraphics[width=\linewidth]{resources/condtional_sample_grid_sd3_small.png}
% \caption{}
\caption{Class-conditional generation. \textbf{Top:} EMAG. \textbf{Bottom:} CFG (baseline).}
\label{fig:cond-samples}
\end{subfigure}
\caption{Qualitative comparison for (a) Unconditional and (b) Text-conditional settings.}
\label{fig:generation-samples}
\end{figure*}
% ########################################### Uncond and cond generation example ##################################################################
\subsubsection{Combining EMAG with APG and CADS}
We can also combine EMAG with more advanced guidance techniques like APG \cite{sadat2024eliminating} and CADS \cite{sadat2023cads}, where for the APG we follow the algorithm provided by the author, specifically the eq:\ref{eq:EMAG_gudiance_update} remains the same while APG is applied to the eq: \ref{eq:EMAG_gudiance_update_2}. In essence, it's similar to applying APG to CFG, but in this case, we use our perturbed conditioning.
% and simply add the EMAG component $+ w_{e} \cdot (\epsilon_\theta(x_t, c) - \epsilon'_\theta(x_t, c))$ to the APG guidance update.
For CADS, we simply apply the interpolation between the positive tokens and the Gaussian noise to the encoder hidden states of the network. We apply this interpolation to the conditioning of the perturbed network as well.
\section{Experimentation and results}
\label{sec:Experiment}
\subsection{Experimental setup}
\label{sec:experimental_setup}
% We experiment with Transformer-based diffusion architectures, specifically, we focus on DiT and MMDiT architectures, where for DiT, we experiment with class-conditioned DiT-XL-2 for both 256x256 and 512x512 resolutions, and for the MMDIT architecture, we choose the Text to Image generation, SD3 Medium model for 1024x1024 resolution. For each experiment, we generate a sample size of 50000 Images.
% For evaluation, we leverage FID, precision, HPS, and density for measuring the quality of the sampling, with recall and coverage for measuring the diversity. We also use Clip-Score and HPS to measure input consistency. We leverage EvalGIM, official HPS repository, and the evaluation library proposed in [] for calculating the metrics. To calculate the HPS score for class conditioned model, we use the following template prompt \textit{A photo of \{Class Label\}}.
\paragraph{Models}
We study Transformer-based diffusion architectures: (i) \textbf{DiT-XL/2} in the \emph{class-conditional} setting at \textbf{256$\times$256} and \textbf{512$\times$512} resolutions \cite{peebles2023scalable}, and (ii) \textbf{MMDiT} via the \textbf{Stable Diffusion~3 (SD3) Medium} checkpoint for \textbf{1024$\times$1024} \emph{text-to-image} generation \cite{esser2024scaling}. Unless stated otherwise, we generate \textbf{50{,}000} samples for class-conditional ImageNet experiments and \textbf{40{,}000} samples for text-to-image experiments (one per caption from the COCO-2014 validation split) with the seed set to 8.
\paragraph{Datasets \& tasks}
We evaluate both \emph{conditional} and \emph{unconditional} generation.
For \emph{class-conditional} experiments with DiT-XL/2 we use ImageNet-1K labels \cite{deng2009imagenet} and evaluate against the \emph{official ADM reference batches/statistics } from the Guided Diffusion repository \cite{dhariwal2021diffusion,deng2009imagenet}.
For \emph{text-conditioned} experiments with SD3 we use captions from the \textbf{COCO-2014} \textbf{40k} validation split \cite{lin2014microsoft}.
\paragraph{Samplers \& guidance windows}
For \textbf{DiT-XL/2}, we use \textbf{DDIM} with \textbf{250} steps (deterministic, $\eta{=}0$), indexing timesteps as $t\in\{0,\ldots,t_{\max}\}$ with $t_{\max}{=}250$ and $t_0{=}0$ \cite{song2020denoising}.
For \textbf{SD3 Medium (MMDiT)}, we use the \textbf{FlowMatchEulerDiscrete} sampler with \textbf{26} steps \cite{lipman2022flow}.
Unless noted otherwise, we use a late-step (``tail'') schedule where EMAG is applied while timesteps traverse \(\tau_s \rightarrow \tau_e\) (high-to-low noise), with \(\tau_s = t_{\max}\) and \(\tau_e = \lfloor 0.2\, t_{\max} \rfloor\).
Concretely, this gives \(\tau_e = 5\) for SD3 with 26 steps and \(\tau_e = 50\) for DiT with DDIM-250; EMAG is disabled for \(t < \tau_e\).
\paragraph{Baselines and compared guidance methods}
We compare against \emph{standard} conditional generation (no guidance) and \emph{classifier-free guidance} (CFG), as well as recent \emph{attention-based} guidance methods:
\textbf{SAG}~\cite{hong2023improving},
\textbf{SEG}~\cite{hong2024smoothed},
% \textbf{PAG}~\cite{ahn2024self},
\textbf{ERG}~\cite{ifriqi2025entropy},
and stochastic block dropping method \textbf{S$^2$} (Stochastic Self-Guidance)~\cite{chen2025s}.
We also evaluate two advanced, orthogonal methods,
\textbf{APG} (Adaptive Projected Guidance)~\cite{sadat2024eliminating} and
\textbf{CADS} (Condition-Annealed Diffusion Sampling)~\cite{sadat2023cads},
and further demonstrate that \emph{combining} APG/CADS with our approach yields additional gains (see Table~\ref{tab:main_results_SD3_cond}).
For methods originally proposed for U-Net backbones, we adapt them to Transformer backbones (DiT/MMDiT) by operating within the self-attention stack (e.g., perturbing queries or attention maps, or inserting attention-space degradations) while preserving the residual/MLP pathways; implementations are based on the strategies adopted in \cite{ifriqi2025entropy}, specifics are provided in the Supplementary Sec.~\ref{sec:additional_implementation_details}.
\paragraph{Metrics:}
We assess \emph{quality} with \textbf{FID}~\cite{heusel2017gans}, \textbf{Precision}/\textbf{Density}~\cite{naeem2020reliable}, and \textbf{HPS\,v2}~\cite{wu2023human}; and \emph{diversity} with \textbf{Recall}/\textbf{Coverage}~\cite{naeem2020reliable}. For text–image alignment, we report \textbf{CLIPScore}~\cite{hessel2021clipscore}. We scale HPS by 100 when reporting. In text-to-image (conditional) results, we primarily report \textbf{HPS} and \textbf{FID}.
\paragraph{Evaluation tooling:}
We rely on the \emph{Guided Diffusion/ADM} evaluation toolkit for ImageNet reference statistics \cite{dhariwal2021diffusion}, the official \emph{PRDC} implementation for density/coverage \cite{naeem2020reliable}, the official \emph{HPSv2} repository \cite{wu2023human}, and \emph{EvalGIM} for unified, reproducible evaluation pipelines where applicable \cite{hall2024evalgimlibraryevaluatinggenerative}.
\paragraph{Hyperparameter sweeps:}
For each baseline (CFG, SAG, SEG, ERG, $S^2$, APG, CADS), we tune the guidance scale $g$ and method-specific controls over the ranges recommended in the original papers and official repositories; where canonical ranges are not specified, we perform a grid search, refer to Supplementary Sec.~\ref{sec:additional_experimetns_and_ablation} for more detail.
% ###################################### MERGED TABLE #####################################################################################
\begin{table}[t]
\centering
\small
\setlength{\tabcolsep}{4pt}
\caption{Quantitative results for unconditional generation on the COCO 2014 validation set (SD3) and ImageNet-256 (DiT). Lower is better for FID; all other metrics are higher-is-better. All methods share the same data, sampling steps and evaluation code. Unless noted, settings follow author recommendations or a brief hyperparameter sweep (Sec.~\ref{sec:additional_experimetns_and_ablation}).}
\label{tab:merged_uncond_results}
\resizebox{\columnwidth}{!}{%
\begin{tabular}{lccccc}
\toprule
\textbf{Guidance} & \textbf{FID} $\downarrow$ & \textbf{Precision} $\uparrow$ & \textbf{Recall} $\uparrow$ & \textbf{Density} $\uparrow$ & \textbf{Coverage} $\uparrow$ \\
\midrule
\multicolumn{6}{l}{\textit{SD3 (MMDiT)\cite{esser2024scaling}}} \\
\midrule
(No-Guidance) & 101.94 & 0.283 & \textbf{0.298} & 0.199 & 0.095 \\
SAG & 95.38 & 0.339 & 0.098 & 0.248 & 0.046 \\
% SEG & 102.469 & 0.280 & 0.3097 & 0.195 & 0.085 \\
SEG & 90.21 & \textbf{0.609} & 0.181 & \textbf{0.674} & 0.090 \\
% SD3 + SEG & 110.260 & -- & \textbf{0.6198} & 0.0 & \textbf{0.6490} & 0.074 \\
I-ERG & \underline{82.11} & \underline{0.557} & 0.173 & 0.608 & \underline{0.090} \\
\midrule
\textbf{EMAG (ours)} & \textbf{74.98} & 0.562 & \underline{0.186} & \underline{0.642} & \textbf{0.112} \\
\midrule
\multicolumn{6}{l}{\textit{DiT-XL/2\cite{peebles2023scalable}} 256x256} \\
\midrule
(No-Guidance) & 52.76 & 0.431 & \underline{0.664} & 0.392 & 0.554 \\
PAG & 39.51 & \underline{0.626} & 0.357 & \underline{0.811} & \underline{0.703} \\
SAG & 40.87 & 0.513 &\textbf{ 0.655} & 0.521 & 0.672 \\
SEG & \underline{39.17} & \textbf{0.638 }& 0.340 & \textbf{0.844} & 0.701 \\
\midrule
\textbf{EMAG (ours)} & \textbf{29.85} & 0.621 & 0.561 & 0.805 & \textbf{0.768} \\
\bottomrule
\end{tabular}
}
\end{table}
% ###################################### MERGED TABLE #####################################################################################
\subsection{Results}
\label{Results}
\paragraph{Conditional generation}
We compare EMAG with conditioned baselines (with and without CFG) and with SOTA training\mbox{-}free guidance methods applicable to text\mbox{-}to\mbox{-}image generation, including $S^{2}$\mbox{-}Guidance and ERG. Quantitative results in Table~\ref{tab:main_results_SD3_cond} show that EMAG achieves the highest HPS while remaining competitive on FID.
We further evaluate compatibility with advanced guidance strategies such as APG and CADS, and discover that it consistently yields additional gains in HPS without sacrificing their benefits on FID. Additional PRDC results in Supplementary Table~\ref{tab:main_results_SD3_cond_supp}.
We also compare EMAG with guidance methods that are complementary to CFG and designed to boost its performance (Table~\ref{tab:combination_experiment}). EMAG delivers the largest HPS gain over CFG alone, attributable to EMA\mbox{-}induced hard negatives that encourage subtle refinements, and it also acts as a plug\mbox{-}in complement to state\mbox{-}of\mbox{-}the\mbox{-}art methods such as $S^{2}$ and ERG, further increasing their HPS. Qualitative examples are presented in Fig~\ref{fig:generation-samples} and Supplementary Sec.~\ref{sec:additional_qualitative_results}
\paragraph{Unconditional generation}
We evaluate EMAG against a no\mbox{-}guidance baseline and representative guidance methods applicable in this setting, like SAG, SEG (adapted to SD3), and I\mbox{-}ERG, and additionally report results for DiT\mbox{-}XL/2. Quantitative results in Table~\ref{tab:merged_uncond_results} show that EMAG attains the lowest FID and the highest Coverage across both settings, while remaining competitive on the remaining metrics. Qualitative comparisons in Fig~\ref{fig:generation-samples}(a) indicate that, relative to the no\mbox{-}guidance baseline, EMAG can retain the core structure from the no guidance baseline but refine it by adding fine details owing to EMA\mbox{-}induced weak models that provide hard negatives.
% ###################################### TABLE V2 #####################################################################################
\begin{table}[t]
\centering
\small
\setlength{\tabcolsep}{6pt}
\caption{Quantitative SD3 Text-Image results on COCO 2014 validation set. All methods share identical data, sampling steps, and evaluation code. Settings follow authors’ recommendations or, if unspecified, a brief hyperparameter sweep (Sec.~\ref{sec:additional_experimetns_and_ablation}). The bottom block highlights cases where the combination surpasses the corresponding SOTA baseline.}
\label{tab:main_results_SD3_cond}
\begin{tabular}{lcc}
\toprule
\textbf{Guidance} & \textbf{FID} $\downarrow$ & \textbf{HPS} $\uparrow$ \\
\midrule
% \multicolumn{3}{l}{\textit{SD3 / SD3.5 (MMDiT)}} \\
% \midrule
(No CFG) & 23.859 & 21.61 \\
CFG & 22.877 & 29.22 \\
SAG & 18.743 & 28.40 \\
SEG + CFG & 23.453 & 29.25 \\
APG & 21.816 & 29.45 \\
CADS & \textbf{18.320} & 28.36 \\
S2 & 20.821 & 29.15 \\
ERG & 23.989 & 29.35 \\
\midrule
\textbf{EMAG-I (ours)} & 22.154 & 29.56 \\
\textbf{EMAG (ours)} & 22.980 & \textbf{29.68} \\
\midrule
\textbf{EMAG + APG (ours)} & 21.819 & \textbf{29.79} \\
\textbf{EMAG + CADS (ours)} & 19.950 & \textbf{28.86} \\
\bottomrule
\end{tabular}
\end{table}
% ###################################### TABLE ABLATION V2 #####################################################################################
\begin{table}[t]
\centering
\small
\setlength{\tabcolsep}{6pt}
\caption{We compare the impact of the adaptive layer selection strategy. The experiments are conducted for 5000 samples with SD3. All runs use the identical COCO 2014 validation set, sampling steps, and evaluation code. }
\label{tab:Adaptive_layer_selection_ablation}
% \resizebox{\columnwidth}{!}{
\begin{tabular}{lcc}
\toprule
\textbf{Guidance} & \textbf{FID} $\downarrow$ & \textbf{HPS} $\uparrow$ \\
\midrule
\textbf{EMAG (L 6)} & \textbf{27.47} & 29.54 \\
\textbf{EMAG (L 7)} & 28.94 & 29.51 \\
\textbf{EMAG (L 8)} & 28.23 & \textbf{29.66} \\
\textbf{EMAG (L all)} & 35.60 & 28.80 \\
\midrule
\textbf{EMAG (Adaptive)} & 28.52 & 29.60 \\
\bottomrule
\end{tabular}
% }
\end{table}
% ############################################################################################################
\subsection{Ablation results}
\label{sec:Result_ablation}
\paragraph{Adaptive layer selection}
In Table~\ref{tab:Adaptive_layer_selection_ablation}, we ablate the adaptive layer-selection mechanism by applying the perturbation to (i) each layer individually and (ii) all layers simultaneously.
We observe a trade-off between FID and HPS: single-layer variants improve FID at the expense of HPS, while the naive “all-layers” strategy increases memory usage and degrades both. Our adaptive selection yields balanced performance across FID and HPS.
\paragraph{Guidance scale selection}
Fig.~\ref{fig:guidance} shows how FID and HPS vary with CFG and EMAG scales. Unless noted otherwise (e.g., in ablations), we use a CFG scale of \textbf{7} and an EMAG scale of \textbf{1.75} for conditional and \textbf{5.125} for unconditional.
\begin{figure}[t]
\centering
\begin{subfigure}[t]{0.5\linewidth}
\centering
\includegraphics[width=\linewidth]{resources/individual_plots/guidance_fid.png}
\caption{}
% \label{fig:emag_scale}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.5\linewidth}
\centering
\includegraphics[width=\linewidth]{resources/individual_plots/guidance_hps.png}
\caption{}
% \label{fig: guidance_scale}
\end{subfigure}
\caption{HPS and FID vs guidance scale (1000 samples). Colors denote EMAG scales {1.25, 1.5, 1.75, 2.0}. (a) HPS ↑. (b) FID ↓. Same split, steps, and evaluation as main experiments.}
\label{fig:guidance}
\end{figure}
\section{Conclusion}
\label{sec:Conclusion}
We introduced \emph{Exponential Moving Average Guidance (EMAG)}, a training\mbox{-}free attention\mbox{-}level perturbation that exposes hard failure modes and induces subtle, semantically faithful refinements (Fig~\ref{fig:front_image}). EMAG consistently improves HPS across conditional settings while remaining competitive on fidelity and diversity metrics. Beyond its standalone gains, EMAG complements advanced guidance strategies such as APG and CADS, yielding additional HPS improvements without sacrificing their strengths on other metrics. Moreover, the hard\mbox{-}negative signal induced by EMAG is complementary to SOTA guidance methods like $S^{2}$, producing further HPS gains even for already high\mbox{-}quality generations. Hence we believe our work opens new avenues for fine-grained refinement in guidance sampling techniques.
\section{Limitations and ethical considerations}
Our study focuses on image generation with Transformer backbones; extending EMAG to video/audio/3D is left for future work. As a test-time guidance method, EMAG inherits biases and misuse risks from its foundation models; we keep built-in safety filters enabled and avoid sensitive content.
{
\small
\bibliographystyle{ieeenat_fullname}
\bibliography{main}
}
% ################################################################################################################################################################################################################################################
\clearpage
\appendix
\setcounter{page}{1}
\setcounter{table}{0}
\setcounter{figure}{0}
\renewcommand{\thefigure}{S\arabic{figure}}
\renewcommand{\thetable}{S\arabic{table}}
\section{Background}
\label{sec:background}
As discussed in Section~\ref{sec:hopfiled_view}, Eq.~\ref{eq:energy_update_rule} gives the update rule for modern Hopfield networks~\cite{ramsauer2020hopfield}. From Theorems~1 and~2 in~\cite{ramsauer2020hopfield}, for fixed patterns $X$, the iterations $\{\zeta^t\}_{t=0}^\infty$ generated by Eq.~\ref{eq:energy_update_rule} monotonically decrease the energy $E(\zeta^t)$ and converge to a fixed point $\zeta^*$, i.e., $E(\zeta^t) \rightarrow E(\zeta^*)$ as $t \rightarrow \infty$, where $\zeta \in \mathbb{R}^d$ plays the role of a query vector in self-attention. In our diffusion transformer, keys/values in a given self-attention layer $L_n$ at each reverse step $T_t \rightarrow T_0$ play the role of stored patterns, while token embeddings act as queries. Empirically, for a fixed layer we observe that attention patterns become more concentrated and temporally stable over the reverse process, and often settle into a small number of recurring association structures (see Figure~\ref{fig: section_a_entropy_layer_wise_pattern}). We also observe that, as the timesteps progress from $T_t \rightarrow T_0$ in the reverse process, the average token-wise entropy of the attention maps tends to decrease (Figure~\ref{fig: section_a_entropy_delta_all_layers}). This behaviour is qualitatively reminiscent of trajectories moving into attractor basins in a Hopfield energy landscape, although the effective “memory” and energy function here are time-dependent and influenced by stochastic noise, unlike the autonomous dynamics assumed in~\cite{ramsauer2020hopfield}. We therefore use the Hopfield perspective as an interpretive analogy for attention evolution across timesteps.
Theorem~4 of~\cite{ramsauer2020hopfield} further states that, under sufficient pattern separation $\Delta_i$, a single Hopfield update can reduce the distance $d_{\text{hopfield}}(f(\zeta), x^*)$ between the updated state $f(\zeta)$ and a stored pattern $x^*$ to an exponentially small value in the separation parameter. In our diffusion setting, however, the “patterns’’ (final images or latents) lie on a highly entangled, continuous data manifold rather than being well-separated discrete memories, so the effective separation $\Delta_i$ is small and one-step retrieval is neither realistic nor desirable. Instead, diffusion and flow-matching models are explicitly formulated as multi-step denoising / integration procedures that gradually transform noise into data by following a time-varying vector field or score function~\cite{song2020score,ho2020denoising,lipman2022flow}. This iterative refinement can be viewed as repeatedly applying Hopfield-like associative updates while slowly lowering an implicit energy, moving through intermediate latent states that become progressively closer to the target basin.
Within this viewpoint, EMAG can be interpreted as a controlled perturbation of the attention-induced energy landscape. Rather than perturbing the attention landscape at timestep $t$, by manipulating the temperature or replacing with channel-wise mean or an identity, as in prior approaches~\cite{ifriqi2025entropy,ahn2024self,hong2024smoothed}, EMAG replaces them with an exponential moving average over past steps, which we denote schematically as $\widetilde{A}_t \;=\; (1-\lambda)\,A_t \;+\; \lambda\,E_t$ (eq:\ref{eq: EMA replacement}) where we keep $\lambda=0$. Because $\widetilde{A}_t$ is a temporally smoothed version of $A_t$, it largely preserves the core semantic structure of the current attractor (which regions attend to which), but lags behind the most recent, highly sharpened configuration. Replacing $A_t$ by $\bar{A}_t$ therefore nudges the current state slightly away from the bottom of its local basin by a small energy increment $\Delta E$: the perturbation is strong enough to move the trajectory off its current path, yet too weak to collapse the global layout or object identities. In Hopfield terms, EMAG creates hard negatives that remain close to the attractor, subtly degraded versions of the same pattern rather than arbitrary corruptions. The reverse process is then forced to re-relax toward low-energy configurations from these slightly displaced starting points, encouraging the model to explore nearby trajectories in the energy landscape and correcting fine-grained artefacts (e.g., local texture and geometry) while keeping the overall structure intact.
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{resources/section_a/attn_map_l_all_entropy_curve_multi.png}
\caption{\textbf{Self-attention entropy over time.} Average token-wise entropy (in nats) of the self-attention maps in layers L12–L15 along the reverse diffusion trajectory of DiT-XL/2 with 250-step DDIM sampling.}
\label{fig: section_a_entropy_delta_all_layers}
\end{figure}
%
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{resources/section_a/grid/attn_map_l_12.png}
\caption{\textbf{Self-attention maps across timesteps.} Visualization of self-attention in layer \textbf{L12} along the reverse diffusion trajectory of DiT-XL/2 with 250-step DDIM sampling.}
\label{fig: section_a_entropy_layer_wise_pattern}
\end{figure}
%
\section{Additional experiments and ablation results}
\label{sec:additional_experimetns_and_ablation}
\subsection{Additional results for SD3 experiments}
In Table~\ref{tab:main_results_SD3_cond_supp}, we show that EMAG improves generation quality and human preference, as measured by HPS, while inducing only minimal changes in FID and other distributional metrics. When combined with CFG, EMAG consistently boosts HPS with minimal degradation in FID, indicating that its fine-grained perturbations largely preserve the behavior of the base guidance strategy. A similar pattern holds when EMAG is composed with stronger guidance baselines such as APG and CADS, as well as other SOTA guidance strategies like $S^2$.
Among these, CADS exhibits a slightly larger drop on the other metrics; this is expected, since its early-timestep weakening of the conditional signal delays the formation of a stable core structure in the self-attention landscape, which in turn reduces the relative benefit of EMAG’s fine-grained perturbations at early timesteps. For fairness, we keep EMAG’s hyperparameters fixed across methods; in practice, increasing the temporal-window parameters $\delta_t$ and $\tau_s$ can partially mitigate this effect, and we leave a more systematic study of this setting to future work.
Table~\ref{tab:sd3_erg_emag_ablation} compares EMAG to ERG across different guidance scales. As the guidance scale is increased, ERG shows the expected trade-off, with noticeable FID degradation in exchange for HPS gains. However, in our SD3 conditional setting, even at its best operating point, ERG still yields lower HPS than EMAG, suggesting that EMAG’s perturbations are more closely aligned with human preference under comparable conditions.
% ###################################### TABLE #####################################################################################
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{6pt}
\caption{Quantitative SD3 text-to-image results on COCO 2014 validation set. All methods share identical prompts, sampling steps, and evaluation code. Hyperparameters follow authors’ recommendations; when unspecified, we do a brief hyperparameter sweep (Sec.~\ref{sec:additional_experimetns_and_ablation}). The bottom block highlights cases where the combination surpasses the corresponding SOTA baseline.}
\label{tab:main_results_SD3_cond_supp}
\begin{tabular}{lccccccc}
\toprule
\textbf{Model / Guidance} & \textbf{FID} $\downarrow$ & \textbf{CLIP Score} $\uparrow$ & \textbf{Precision} $\uparrow$ & \textbf{Recall} $\uparrow$ & \textbf{Density} $\uparrow$ & \textbf{Coverage} $\uparrow$ & \textbf{HPS} $\uparrow$\\
\midrule
\multicolumn{6}{l}{\textit{SD3 / SD3.5 (MMDiT)}} \\
\midrule
(No CFG) & 23.86 & 23.79 & 0.494 & 0.371 & 0.591 & 0.505 & 21.61 \\
CFG & 22.88 & 26.61 & 0.694 & 0.277 & 1.176 & 0.615 & 29.22 \\
SAG & 18.74 & 24.16 & 0.496 &\textbf{ 0.400} & 0.588 & 0.517 & 28.40 \\
SEG + CFG & 23.45 & 25.45 & 0.656 & 0.206 & 1.012 & 0.578 & 29.25 \\
APG & 21.82 & 26.59 & \textbf{0.703} & 0.295 & \textbf{1.235} & 0.624 & 29.45 \\
CADS & \textbf{18.32} & 26.39 & 0.699 & 0.316 & 1.219 & \textbf{0.644} & 28.36 \\
S2 & 20.82 & \textbf{27.52} & 0.695 & 0.273 & 1.210 & 0.617 & 29.15 \\
ERG & 23.99 & 25.71 & 0.660 & 0.209 & 1.025 & 0.580 & 29.35 \\
\midrule
\textbf{EMAG-I (ours)} & 22.15 & 26.63 & 0.663 & 0.263 & 1.082 & 0.595 & 29.56 \\
\textbf{EMAG (ours)} & 22.98 & 26.72 & 0.644 & 0.239 & 1.021 & 0.568 & \textbf{29.68} \\
\midrule
\textbf{EMAG + APG (ours)} & 21.82 & 26.55 & 0.685 & 0.258 & 1.174 & 0.613 & \textbf{29.79} \\
\textbf{EMAG + CADS (ours)} & 19.95 & 26.27 & 0.619 & 0.270 & 0.950 & 0.577 & \textbf{28.86} \\
\textbf{SD3 + S2 + EMAG} & \textbf{20.12} & 27.16 & 0.636 & 0.248 & 1.006 & 0.583 & \textbf{29.51} \\
\bottomrule
\end{tabular}
\end{table*}
% ###################################### TABLE #####################################################################################
\subsection{Additional experimental results on DiT-XL/2}
Since HPS requires a prompt–image pair, for class-conditioned experiments on ImageNet we follow the template prompt \textit{``A photo of \{class label\}.''}.
For DiT-XL/2 experiments we fix the CFG scale to \textbf{1.5}, as recommended by Peebles and Xie~\cite{peebles2023scalable}, and tune only the additional guidance scales introduced by each baseline method. For SAG, SEG, $S^2$, APG, EMAG, and related approaches, we sweep their additional scales over the ranges recommended by the original papers and evaluate FID, PRDC on a 1k-sample subset (Table~\ref{tab:dit_cond_sag_seg_s2_erg_apg_cads}). For ERG, which does not define a separate additional scale, we instead sweep the CFG scale itself over a small range following the authors’ settings, to maintain fairness: other methods with additional scales can effectively strengthen or weaken the overall guidance signal, whereas ERG can only do so via CFG. This protocol allows us to assess the impact of each method under a fixed base CFG strength while fairly optimising its own additional scale for the DiT backbone. Methods such as SAG and SEG, originally designed for unconditional generation, are combined with CFG in our setup to enable class-conditional sampling based on the settings provided by the authors.
Although reducing the scales below the author-recommended ranges can recover lower FID for these guidance methods, we deliberately focus on the published ranges to evaluate each method in its intended operating regime and to make their impact on a strong DiT baseline directly comparable. Consequently, we observe that methods with additional scales in their original ranges, like SEG Scale in \cite{hong2024smoothed} often exhibit higher FID than pure CFG for DiT, but they also improve perceptual quality and HPS. We report class-conditioned ImageNet results for DiT at $256^2$ and $512^2$ in Tables~\ref{tab:imagenet_classcond_256_sup} and~\ref{tab:imagenet_classcond_512_supp}, and the corresponding unconditional results in Tables~\ref{tab:merged_uncond_results} and~\ref{tab:imagenet_uncond_512_supp}. Qualitative samples and baseline comparisons are shown in Figures~\ref{fig: section_D_uncond_sd3_dit_512} and~\ref{fig: section_D_baseline_vs_emag_512_dit}.
On DiT-XL/2 at 256$\times$256 resolution, EMAG’s fine-grained refinements remain competitive but do not surpass the strongest baselines; its FID-HPS trade-off is generally tighter than most methods, with the notable exception of $S^{2}$. At 512$\times$512 resolution, where the base model already operates in a higher-quality regime, EMAG yields a more favorable balance between FID and HPS, and in our setting attains the best HPS among all variants while keeping FID comparable to other strong baselines. This trend is consistent with our SD3 results and further suggests that EMAG is particularly effective in high-resolution, high-quality regimes, where subtle attention-level refinements can translate into noticeable gains in human preference without substantially distorting the underlying sample distribution.
For unconditional DiT experiments, we ablate the guidance scales solely with respect to FID on a 1k-sample subset for each baseline (see Table~\ref{tab:dit_uncond_pag_sag_seg}) and then report final metrics on 50k generated samples using the best FID configuration, see Tables~\ref{tab:imagenet_uncond_512_supp} and \ref{tab:merged_uncond_results}.
\subsection{Further ablation on EMAG hyperparameters}
In Table~\ref{tab:Adaptive_layer_selection_ablation_supp}, the adaptive layer selection strategy yields more balanced performance across PRDC metrics, FID, and HPS. This suggests that adaptive selection navigates the trade-off more effectively than either using a single fixed layer or applying EMAG to all layers simultaneously.
We further analyze the behaviour of EMAG with respect to PRDC metrics on SD3-Medium in Figures~\ref{fig:emag_guidance_scale_ablation_supp} and~\ref{fig:emag_scale_ablation_supp}, which we use to select a balanced combination of guidance and EMAG scales. While Figure~\ref{fig:emag_guidance_scale_ablation_supp} examines the effect of varying the guidance scale, whereas Figure~\ref{fig:emag_scale_ablation_supp} varies the EMAG scale; both studies are evaluated on 1{,}000 generated samples.
We also provide qualitative example for SD3 in Figure~\ref{fig: section_D_ablation_scale_sd3}, where we analyse the joint impact of guidance scale and EMAG scale on the generated samples. At very high EMAG scales, we observe overexposure and oversharpening artifacts, whereas smaller EMAG scales remain stable even under large guidance scales. In the medium EMAG regime, EMAG effectively refines background structure and fine details while preserving the overall composition.
Similarly, Figure~\ref{fig: section_D_ablation_scale_dit} illustrates the effect of varying the EMAG scale for a fixed guidance scale of 1.5 on DiT-XL/2 at 512$\times$512 resolution. We again find that excessively large EMAG scales lead to overexposed, overly sharpened generations, while medium EMAG scales provide the most visually appealing trade-off, yielding subtle refinements (e.g., in background texture and small object details) without destabilising the base sample.
For completeness, we summarise the EMAG hyperparameters and layer ranges used across different tasks and architectures in Table~\ref{tab:emag_hparams}.
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{resources/guidance_scale_ablation.png}
\caption{Variation of HPS, FID, Clip-Score, and PRDC metrics as a function of guidance scale for SD3 (1{,}000 samples).}
\label{fig:emag_guidance_scale_ablation_supp}
\end{figure*}
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{resources/emag_scale_ablation.png}
\caption{Variation of HPS, FID, Clip-Score, and PRDC metrics as a function of EMAG scale for SD3 (1{,}000 samples).}
\label{fig:emag_scale_ablation_supp}
\end{figure*}
% #################################### ablation of gudiance scale SD3 #####################################################################
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{resources/section_d/D_EMAG_scale_ablation_figure.png}
\caption{\textbf{Qualitative analysis of the effect of guidance scale and EMAG scale on Text to Image conditional generation with SD3. Prompt: \textit{A loft bed with a dresser underneath it.}}}
\label{fig: section_D_ablation_scale_sd3}
\end{figure*}
% #############################################################################################################################################
% #################################### ablation of gudiance scale DIT #####################################################################
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{resources/section_d/D_EMAG_scale_ablation_figure_DIT_512.png}
\caption{\textbf{Qualitative analysis of the effect of EMAG scale on class-conditional generation with DiT at $512^2$ resolution for fixed CFG scale. \textit{Class label: electric ray}}}
\label{fig: section_D_ablation_scale_dit}
\end{figure*}
% #############################################################################################################################################
\subsection{Hyperparameter selection for baselines}
In this section, we report the ablation experiments used to select guidance scales for all baseline guidance methods for our main experiments. For SD3, ablations on conditional text-to-image generation are shown in Table~\ref{tab:sd3_cond_sag_seg_s2_erg_apg_cads}, while unconditional generation with null prompts is reported in Table~\ref{tab:sd3_uncond_sag_seg_ierg}. For DiT, the unconditional ablations are provided in Table~\ref{tab:dit_uncond_pag_sag_seg}, and the class conditional ablations are summarized in Table~\ref{tab:dit_cond_sag_seg_s2_erg_apg_cads}.
% ###################################### TABLE SD3 Unconditional #####################################################################################
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{5pt}
\caption{Ablation of guidance scales for unconditional generation using SD3 (null prompts) across multiple baselines. Lower is better for FID; higher is better for all other metrics. All results are reported on 1{,}000 generated samples.}
\label{tab:sd3_uncond_sag_seg_ierg}
\begin{tabular}{lcccccc}
\toprule
\textbf{Method} & \textbf{Hyperparameters} & \textbf{FID} $\downarrow$ & \textbf{Precision} $\uparrow$ & \textbf{Recall} $\uparrow$ & \textbf{Density} $\uparrow$ & \textbf{Coverage} $\uparrow$ \\
\midrule
\multicolumn{7}{l}{\textit{SD3 (Unconditional)}} \\
\midrule
SAG & SAG Scale $s=0.1$ & 126.85 & 0.513 & 0.516 & 0.239 & 0.024 \\
SAG & SAG Scale $s=0.2$ & 151.01 & 0.389 & 0.408 & 0.147 & 0.015 \\
SAG & SAG Scale $s=0.3$ & 214.21 & 0.249 & 0.166 & 0.074 & 0.006 \\
\midrule
SEG & SEG Scale $s=2$ & 119.36 & 0.770 & 0.524 & 0.609 & 0.051 \\
SEG & SEG Scale $s=3$ & 138.06 & 0.781 & 0.423 & 0.600 & 0.047 \\
SEG & SEG Scale $s=4$ & 149.59 & 0.773 & 0.354 & 0.510 & 0.032 \\
\midrule
I-ERG & ERG Scale $s=2$ & 111.73 & 0.730 & 0.581 & 0.594 & 0.056 \\
I-ERG & ERG Scale $s=3$ & 123.80 & 0.743 & 0.466 & 0.545 & 0.053 \\
I-ERG & ERG Scale $s=4$ & 134.66 & 0.681 & 0.340 & 0.474 & 0.043 \\
\bottomrule
\end{tabular}
\end{table*}
% ##################################################################################################################################
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{5pt}
\caption{Ablation of guidance scales for unconditional generation using DiT-XL/2 at 256$\times$256 and 512$\times$512 resolutions. Lower is better for FID; higher is better for all other metrics. All results are reported on 1{,}000 generated samples per setting.}
\label{tab:dit_uncond_pag_sag_seg}
\begin{tabular}{lcccccc}
\toprule
\textbf{Method} & \textbf{Hyperparameters} & \textbf{FID} $\downarrow$ & \textbf{Precision} $\uparrow$ & \textbf{Recall} $\uparrow$ & \textbf{Density} $\uparrow$ & \textbf{Coverage} $\uparrow$ \\
\midrule
\multicolumn{7}{l}{\textit{DiT-XL/2 (Unconditional), $256^2$}} \\
\midrule
PAG & PAG Scale $s=2$ & 69.05 & 0.657 & 0.364 & 0.9052 & 0.222 \\
PAG & PAG Scale $s=3$ & 75.71 & 0.670 & 0.314 & 0.869 & 0.203 \\
PAG & PAG Scale $s=4$ & 80.12 & 0.635 & 0.287 & 0.848 & 0.196 \\
\midrule
SAG & SAG Scale $s=0.1$ & 77.87 & 0.535 & 0.536 & 0.567 & 0.156 \\
SAG & SAG Scale $s=0.2$ & 74.58 & 0.541 & 0.524 & 0.582 & 0.156 \\
SAG & SAG Scale $s=0.3$ & 73.16 & 0.563 & 0.519 & 0.596 & 0.159 \\
\midrule
SEG & SEG Scale $s=2$ & 66.36 & 0.682 & 0.388 & 0.961 & 0.238 \\
SEG & SEG Scale $s=3$ & 75.58 & 0.658 & 0.297 & 0.903 & 0.200 \\
SEG & SEG Scale $s=4$ & 81.39 & 0.579 & 0.271 & 0.762 & 0.174 \\
\midrule
\multicolumn{7}{l}{\textit{DiT-XL/2 (Unconditional), $512^2$}} \\
\midrule
PAG & PAG Scale $s=2$ & 84.25 & 0.656 & 0.330 & 1.206 & 0.776 \\
PAG & PAG Scale $s=3$ & 92.05 & 0.611 & 0.252 & 1.122 & 0.678 \\
PAG & PAG Scale $s=4$ & 96.64 & 0.587 & 0.250 & 0.996 & 0.682 \\
\midrule
SAG & SAG Scale $s=0.1$ & 81.55 & 0.601 & 0.451 & 0.997 & 0.781 \\
SAG & SAG Scale $s=0.2$ & 79.32 & 0.633 & 0.449 & 1.052 & 0.777 \\
SAG & SAG Scale $s=0.3$ & 79.20 & 0.653 & 0.450 & 1.114 & 0.748 \\
\midrule
SEG & SEG Scale $s=2$ & 81.15 & 0.703 & 0.336 & 1.410 & 0.767 \\
SEG & SEG Scale $s=3$ & 93.86 & 0.598 & 0.256 & 1.084 & 0.621 \\
SEG & SEG Scale $s=4$ & 104.42 & 0.525 & 0.197 & 0.833 & 0.556 \\
\bottomrule
\end{tabular}
\end{table*}
% ##################################################################################################################################
% ###################################### TABLE SD3 Conditional #####################################################################################
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{5pt}
\caption{Ablation of guidance configurations for SD3 text-to-image generation on the COCO 2014 validation set across multiple baselines. Lower is better for FID; higher is better for all other metrics. For SAG, SEG, and $S^{2}$, the \textit{Scale} column reports the fixed CFG scale, while \textit{Hyperparameters} lists the method-specific scale swept in this ablation (e.g., SAG/SEG scale $s$, $S^{2}$ weight $w$). For ERG and APG, the \textit{Scale} column itself is the swept guidance scale, and \textit{Hyperparameters} reports the remaining settings used in this ablation. All results are reported on 1{,}000 generated images.}
\label{tab:sd3_cond_sag_seg_s2_erg_apg_cads}
\begin{tabular}{lccccccc}
\toprule
\textbf{Method} & \textbf{Scale} & \textbf{Hyperparameters} & \textbf{FID} $\downarrow$ & \textbf{Precision} $\uparrow$ & \textbf{Recall} $\uparrow$ & \textbf{Density} $\uparrow$ & \textbf{Coverage} $\uparrow$ \\
\midrule
\multicolumn{8}{l}{\textit{SD3 (Conditional)}} \\
\midrule
SAG + CFG & 7.0 & SAG scale $s=0.1$ & 59.26 & 0.802 & 0.650 & 0.924 & 0.133 \\
SAG + CFG & 7.0 & SAG scale $s=0.2$ & 62.48 & 0.733 & 0.606 & 0.728 & 0.108 \\
SAG + CFG & 7.0 & SAG scale $s=0.3$ & 67.73 & 0.646 & 0.576 & 0.535 & 0.081 \\
\midrule
SEG + CFG & 7.0 & SEG scale $s=2.0$ & 60.10 & 0.786 & 0.610 & 0.971 & 0.139 \\
SEG + CFG & 7.0 & SEG scale $s=3.0$ & 60.96 & 0.769 & 0.592 & 0.865 & 0.127 \\
SEG + CFG & 7.0 & SEG scale $s=4.0$ & 62.45 & 0.717 & 0.573 & 0.722 & 0.111 \\
\midrule
$S^{2}$ & 7.5 & $w=0.25$ & 54.66 & 0.821 & 0.702 & 1.209 & 0.173 \\
$S^{2}$ & 7.5 & $w=0.15$ & 60.44 & 0.753 & 0.650 & 0.856 & 0.128 \\
\midrule
ERG & 2.0 & Setting from Original paper & 51.49 & 0.847 & 0.718 & 1.178 & 0.178 \\
ERG & 3.0 & Setting from Original paper & 57.52 & 0.814 & 0.644 & 0.999 & 0.148 \\
ERG & 4.0 & Setting from Original paper & 61.02 & 0.777 & 0.602 & 0.880 & 0.135 \\
\midrule
APG & 10 & $\beta=-0.5,r=7.5$ & 55.62 & 0.850 & 0.694 & 1.329 & 0.184 \\
APG & 12 & $\beta=-0.5,r=7.5$ & 55.50 & 0.839 & 0.685 & 1.271 & 0.178 \\
APG & 15 & $\beta=-0.5,r=7.5$ & 55.20 & 0.835 & 0.690 & 1.174 & 0.167 \\
APG & 10 & $\beta=-0.75,r=7.5$ & 55.33 & 0.860 & 0.692 & 1.358 & 0.189 \\
APG & 12 & $\beta=-0.75,r=7.5$ & 55.29 & 0.861 & 0.706 & 1.262 & 0.179 \\
APG & 15 & $\beta=-0.75,r=7.5$ & 54.84 & 0.845 & 0.693 & 1.276 & 0.180 \\
APG & 10 & $\beta=-0.5,r=15.0$ & 55.23 & 0.833 & 0.675 & 1.327 & 0.186 \\
APG & 12 & $\beta=-0.5,r=15.0$ & 55.34 & 0.847 & 0.676 & 1.259 & 0.175 \\
APG & 15 & $\beta=-0.5,r=15.0$ & 54.95 & 0.830 & 0.679 & 1.086 & 0.154 \\
APG & 10 & $\beta=-0.75,r=15.0$ & 55.63 & 0.844 & 0.695 & 1.311 & 0.184 \\
APG & 12 & $\beta=-0.75,r=15.0$ & 55.09 & 0.847 & 0.683 & 1.306 & 0.184 \\
APG & 15 & $\beta=-0.75,r=15.0$ & 55.42 & 0.815 & 0.704 & 1.192 & 0.169 \\
\midrule
\multicolumn{8}{l}{\textit{CADS (We use the settings from the original paper for Stable Diffusion) $w_{cfg}=9.0,\tau_1 = 0.6, \tau_2 = 0.9, s=0.25, \psi=1$}} \\
\bottomrule
\end{tabular}
\end{table*}
% ##################################################################################################################################
%#################################################################################################################################
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{5pt}
\caption{Ablation of guidance configurations for class-conditional DiT-XL/2 on ImageNet at 256$\times$256 and 512$\times$512 resolutions. Lower is better for FID; higher is better for all other metrics. Each \textit{Resolution} block corresponds to a fixed sampling resolution. For SAG, SEG, and $S^{2}$, the \textit{Scale} column reports the fixed CFG scale, while \textit{Hyperparameters} lists the method-specific scale swept in this ablation (e.g., SAG/SEG scale $s$, $S^{2}$ scale $w$). For ERG, the \textit{Scale} column itself is the swept guidance scale. APG and CADS use the authors' recommended configurations for the corresponding resolution. All results are reported on 1{,}000 generated images.}
\label{tab:dit_cond_sag_seg_s2_erg_apg_cads}
\begin{tabular}{lccccccc}
\toprule
\textbf{Method} & \textbf{Scale} & \textbf{Hyperparameters} & \textbf{FID} $\downarrow$ & \textbf{Precision} $\uparrow$ & \textbf{Recall} $\uparrow$ & \textbf{Density} $\uparrow$ & \textbf{Coverage} $\uparrow$ \\
\midrule
\multicolumn{8}{l}{\textit{DiT-XL/2 (Conditional), $256^2$}} \\
\midrule
SAG + CFG & 1.5 & SAG scale $s=0.1$ & 54.56 & 0.936 & 0.702 & 1.351 & 0.455 \\
SAG + CFG & 1.5 & SAG scale $s=0.2$ & 54.86 & 0.927 & 0.697 & 1.353 & 0.451 \\
SAG + CFG & 1.5 & SAG scale $s=0.3$ & 54.44 & 0.920 & 0.694 & 1.359 & 0.456 \\
\midrule
SEG + CFG & 1.5 & SEG scale $s=2$ & 52.25 & 0.859 & 0.644 & 1.198 & 0.412 \\
SEG + CFG & 1.5 & SEG scale $s=3$ & 54.53 & 0.760 & 0.610 & 1.039 & 0.359 \\
SEG + CFG & 1.5 & SEG scale $s=4$ & 59.51 & 0.656 & 0.573 & 0.873 & 0.301 \\
\midrule
$S^{2}$ & 1.5 & $w=0.01$ & 44.39 & 0.808 & 0.735 & 1.154 & 0.420 \\
$S^{2}$ & 1.5 & $w=0.05$ & 48.84 & 0.719 & 0.727 & 0.940 & 0.353 \\
$S^{2}$ & 1.5 & $w=0.1$ & 83.72 & 0.403 & 0.625 & 0.379 & 0.147 \\
$S^{2}$ & 1.5 & $w=0.15$ & 140.74 & 0.231 & 0.447 & 0.135 & 0.041 \\
$S^{2}$ & 1.5 & $w=0.20$ & 187.93 & 0.209 & 0.241 & 0.100 & 0.010 \\
$S^{2}$ & 1.5 & $w=0.25$ & 233.99 & 0.189 & 0.035 & 0.117 & 0.003 \\
$S^{2}$ & 1.5 & $w=0.30$ & 270.78 & 0.206 & 0.003 & 0.193 & 0.002 \\
\midrule
ERG & 1.5 & Setting from Original paper & 44.27 & 0.857 & 0.702 & 1.278 & 0.458 \\
ERG & 2.0 & Setting from Original paper & 48.09 & 0.896 & 0.673 & 1.376 & 0.467 \\
ERG & 3.0 & Setting from Original paper & 51.90 & 0.857 & 0.650 & 1.255 & 0.436 \\
ERG & 4.0 & Setting from Original paper & 53.59 & 0.815 & 0.625 & 1.142 & 0.383 \\
\midrule
\multicolumn{8}{l}{\textit{APG (We use the settings from the original paper for DiT ImageNet-256)}} \\
\midrule
\multicolumn{8}{l}{\textit{CADS (We use the settings from the original paper for DiT ImageNet-256)}}\\
\midrule
\multicolumn{8}{l}{\textit{DiT-XL/2 (Conditional), $512^2$}} \\
\midrule
SAG + CFG & 1.5 & SAG scale $s=0.1$ & 53.87 & 0.845 & 0.713 & 1.024 & 0.956 \\
SAG + CFG & 1.5 & SAG scale $s=0.2$ & 53.39 & 0.844 & 0.709 & 1.068 & 0.966 \\
SAG + CFG & 1.5 & SAG scale $s=0.3$ & 53.17 & 0.850 & 0.694 & 1.117 & 0.967 \\
\midrule
SEG + CFG & 1.5 & SEG scale $s=2$ & 57.91 & 0.777 & 0.567 & 1.252 & 0.951 \\
SEG + CFG & 1.5 & SEG scale $s=3$ & 68.38 & 0.695 & 0.519 & 1.072 & 0.918 \\
SEG + CFG & 1.5 & SEG scale $s=4$ & 79.57 & 0.610 & 0.458 & 0.867 & 0.828 \\
\midrule
$S^{2}$ & 1.5 & $w=0.01$ & 44.94 & 0.850 & 0.695 & 1.216 & 0.977 \\
$S^{2}$ & 1.5 & $w=0.05$ & 58.00 & 0.701 & 0.662 & 0.883 & 0.949 \\
$S^{2}$ & 1.5 & $w=0.1$ & 124.33 & 0.315 & 0.497 & 0.258 & 0.593 \\
$S^{2}$ & 1.5 & $w=0.15$ & 200.08 & 0.131 & 0.263 & 0.065 & 0.135 \\
$S^{2}$ & 1.5 & $w=0.20$ & 253.80 & 0.154 & 0.049 & 0.075 & 0.065 \\
$S^{2}$ & 1.5 & $w=0.25$ & 305.95 & 0.170 & 0.000 & 0.047 & 0.024 \\
$S^{2}$ & 1.5 & $w=0.30$ & 387.04 & 0.051 & 0.000 & 0.010 & 0.002 \\
\midrule
ERG & 1.5 & Setting from Original paper & 45.26 & 0.864 & 0.669 & 1.508 & 0.984 \\
ERG & 2.0 & Setting from Original paper & 48.67 & 0.860 & 0.635 & 1.365 & 0.978 \\
ERG & 3.0 & Setting from Original paper & 54.51 & 0.831 & 0.573 & 1.311 & 0.960 \\
ERG & 4.0 & Setting from Original paper & 60.76 & 0.767 & 0.533 & 1.171 & 0.943 \\
\midrule
\multicolumn{8}{l}{\textit{APG (We use the settings from the original paper for DiT ImageNet-512)}}\\
\midrule
\multicolumn{8}{l}{\textit{CADS (We use the settings from the original paper for DiT ImageNet-512)}}\\
\bottomrule
\end{tabular}
\end{table*}
% ##################################################################################################################################
% ###################################### TABLE SD3 ERG vs EMAG #####################################################################################
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{5pt}
\caption{Comparison of ERG variants at different guidance scales with EMAG on SD3 text-to-image generation. Lower is better for FID; all other metrics are higher-is-better. The \textit{Scale} column denotes the guidance / CFG scale. All configurations are evaluated with identical sampling settings on 40k generated images conditioned on captions from the COCO 2014 validation set.}
\label{tab:sd3_erg_emag_ablation}
\begin{tabular}{lcccccccc}
\toprule
\textbf{Method} & \textbf{Scale} & \textbf{Hyperparameters} & \textbf{FID} $\downarrow$& \textbf{Precision} $\uparrow$ & \textbf{Recall} $\uparrow$ & \textbf{Density} $\uparrow$ & \textbf{Coverage} $\uparrow$ & \textbf{HPS} $\uparrow$ \\
\midrule
\multicolumn{9}{l}{\textit{SD3}} \\
\midrule
ERG & 2.0 & Setting from Original paper & \textbf{17.88} & \textbf{0.709} & \textbf{0.307} & \textbf{1.199} & \textbf{0.679} & 29.35 \\
ERG & 3.0 & Setting from Original paper & 23.99 & 0.660 & 0.209 & 1.025 & 0.580 & 29.35 \\
ERG & 4.0 & Setting from Original paper & 27.78 & 0.605 & 0.164 & 0.872 & 0.510 & 29.15 \\
\midrule
\textbf{EMAG-I (ours)} & \textbf{7.0} & See Table~\ref{tab:emag_hparams} & 22.154 & 0.663 & 0.263 & 1.082 & 0.595 & \textbf{29.56 } \\
\bottomrule
\end{tabular}
\end{table*}
% ##################################################################################################################################
% ###################################### TABLE #####################################################################################
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{6pt}
\caption{We compare the impact of the adaptive layer selection strategy. The experiments are conducted for 5000 samples. All runs use the identical COCO 2014 validation set, sampling steps, and evaluation code. }
\label{tab:Adaptive_layer_selection_ablation_supp}
% \resizebox{\columnwidth}{!}{
\begin{tabular}{lccccccc}
\toprule
\textbf{Model / Guidance} & \textbf{FID} $\downarrow$ & \textbf{CLIP Score} $\uparrow$ & \textbf{Precision} $\uparrow$ & \textbf{Recall} $\uparrow$ & \textbf{Density} $\uparrow$ & \textbf{Coverage} $\uparrow$ & \textbf{HPS} $\uparrow$\\
\midrule
\multicolumn{6}{l}{\textit{SD3(MMDiT)}} \\
\midrule
\textbf{EMAG (L 6)} & \textbf{27.47 } & 26.604 & \textbf{0.662} &\textbf{ 0.342} & \textbf{1.061} & \textbf{0.223 }& 29.54 \\
\textbf{EMAG (L 7)} & 28.94 & 26.732 & 0.628 & 0.298 & 0.966 & 0.209 & 29.51 \\
\textbf{EMAG (L 8)} & 28.23 & 26.768 & 0.649 & 0.338 & 0.995 & 0.211 & \textbf{29.66} \\
\textbf{EMAG (L all)} & 35.60 & \textbf{26.848} & 0.495 & 0.257 & 0.618 & 0.142 & 28.80 \\
\midrule
\textbf{EMAG (Adaptive)} & 28.52 & 26.733 & 0.643 & 0.320 & 1.003 & 0.213 & 29.60 \\
\bottomrule
\end{tabular}
% }
\end{table*}
% ###################################### TABLE #####################################################################################
% ###################################### TABLE #####################################################################################
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{6pt}
\caption{Quantitative class-conditional results on ImageNet at 256$\times$256 resolution. Lower is better for FID; all other metrics are higher-is-better. All methods share identical samples, sampling steps, and evaluation code. We fix the CFG scale to 1.5 for all methods; for additional guidance scales, we adopt the authors' recommended settings or, when unspecified, select them via a brief hyperparameter sweep (Sec.~\ref{sec:additional_experimetns_and_ablation}).}
\label{tab:imagenet_classcond_256_sup}
\begin{tabular}{lcccccc}
\toprule
\textbf{Model / Guidance} & \textbf{FID} $\downarrow$ & \textbf{Precision} $\uparrow$ & \textbf{Recall} $\uparrow$ & \textbf{Density} $\uparrow$ & \textbf{Coverage} $\uparrow$ & \textbf{HPS} $\uparrow$ \\
\midrule
\multicolumn{7}{l}{\textit{DiT-XL/2 \cite{peebles2023scalable}} 256x256} \\
\midrule
(No CFG) & 9.53 & 0.668 & 0.678 & 0.855 & 0.997 & 21.64\\
CFG & \textbf{2.30} & 0.826 & 0.575 & 1.188 & \textbf{0.999} & 23.41 \\
SAG & 13.53 & 0.932 & 0.214 & 1.335 & 0.990 & 25.60 \\
SEG & 13.14 & 0.838 & 0.167 & 1.208 & 0.984 & \textbf{26.41} \\
APG & 11.53 & 0.930 & 0.353 & 1.338 & 0.997 & 25.33 \\
CADS & 2.70 & 0.711 & 0.674 & 0.918 & 0.998 & 22.97 \\
$S^{2}$ & 2.40 & 0.891 & \textbf{0.689} & 1.161 & \textbf{0.999} & 24.38 \\
I-ERG & 8.37 & \textbf{0.945} & 0.355 & \textbf{1.350} & 0.997 & 25.69\\
\midrule
\textbf{EMAG (ours)}& 4.16 & 0.872 & 0.517 & 1.292 & \textbf{0.999} & 24.48\\
\bottomrule
\end{tabular}
\end{table*}
% ###################################### TABLE #####################################################################################
% ####################################### DIT -X/L 512x512 TABLE ##############################################
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{6pt}
\caption{Quantitative class-conditional results on ImageNet at 512$\times$512 resolution. Lower is better for FID; all other metrics are higher-is-better. All methods share identical samples, sampling steps, and evaluation code. We fix the CFG scale to 1.5 for all methods; for additional guidance scales, we adopt the authors' recommended settings or, when unspecified, select them via a brief hyperparameter sweep (Sec.~\ref{sec:additional_experimetns_and_ablation}).}
\label{tab:imagenet_classcond_512_supp}
\begin{tabular}{lcccccc}
\toprule
\textbf{Model / Guidance} & \textbf{FID} $\downarrow$ & \textbf{Precision} $\uparrow$ & \textbf{Recall} $\uparrow$ & \textbf{Density} $\uparrow$ & \textbf{Coverage} $\uparrow$ & \textbf{HPS} $\uparrow$ \\
\midrule
\multicolumn{7}{l}{\textit{DiT-XL/2 \cite{peebles2023scalable}} 512x512} \\
\midrule
(No CFG) & 11.78 & 0.750 & 0.646 & 1.071 & 1.000 & 21.40 \\
CFG & \textbf{3.08} & 0.838 & 0.532 & 1.219 & 1.000 & 23.68 \\
% + PAG & 16.55 & 0.788 & 0.292 & 1.151 & 1.000 & -- \\
% + SAG & 11.73 & 0.754 & 0.646 & 1.086 & 1.000 & -- \\
SAG & 18.58 & 0.843 & 0.134 & 1.075 & 1.000 & 25.55 \\
SEG & 22.51 & 0.789 & 0.069 & 1.298 & 1.000 & 24.55 \\
APG & 10.24 & 0.849 & 0.347 & 1.102 & 1.000 & 25.25 \\
CADS & 11.19 & 0.688 & \textbf{0.719} & 0.785 & 1.000 & 21.54 \\
$S^{2}$ & 6.20 & 0.831 & 0.005 & 1.178 & 1.000 & 24.17 \\
I-ERG & 5.00 & 0.854 & 0.435 & \textbf{1.477} & 1.000 & 24.64 \\
\midrule
\textbf{EMAG (ours)}& 7.59 & \textbf{0.864} & 0.094 & 1.343 & 1.000 & \textbf{25.66}\\
\bottomrule
\end{tabular}
\end{table*}
% ###################################### TABLE #####################################################################################
% ###################################### TABLE Unconditonal 512x512 DIT #####################################################################################
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{6pt}
% \caption{Unconditional ImageNet-512 results. Same metric conventions as Table~\ref{tab:imagenet_classcond}.}
\caption{Quantitative results for unconditional ImageNet at 512$\times$512 resolution. Lower is better for FID; all other metrics are higher-is-better. All methods share identical data, sampling steps, and evaluation code. Settings follow authors' recommendations or, if unspecified, a brief hyperparameter sweep (Sec.~\ref{sec:additional_experimetns_and_ablation}).}
\label{tab:imagenet_uncond_512_supp}
\begin{tabular}{lccccc}
\toprule
\textbf{Model / Guidance} & \textbf{FID} $\downarrow$ & \textbf{Precision} $\uparrow$ & \textbf{Recall} $\uparrow$ & \textbf{Density} $\uparrow$ & \textbf{Coverage} $\uparrow$ \\
\multicolumn{6}{l}{\textit{DiT-XL/2 \cite{peebles2023scalable}} 512x512} \\
\midrule
% \textit{Imagenet 256x256}
\midrule
(No Guidance) & 55.58 & 0.519 & \textbf{0.652} & 0.579 & 0.995 \\
PAG & 54.62 & 0.622 & 0.327 & 1.112 & 0.991 \\
SAG & 46.55 & 0.586 & 0.633 & 0.764 & \textbf{0.998} \\
SEG & 45.55 & 0.711 & 0.001 & 1.422 & 0.987 \\
\midrule
\textbf{EMAG (ours)} & \textbf{45.01} & \textbf{0.762} & 0.001 & \textbf{1.664} & 0.985 \\
\bottomrule
\end{tabular}
\end{table*}
% ###################################### TABLE #####################################################################################
\section{Additional implementation details}
\label{sec:additional_implementation_details}
\subsection{EMAG-I vs.\ EMAG}
For MMDiT-based diffusion transformers, at a given timestep $t$, we partition the per-head attention over image queries into
image-to-image and image-to-text blocks:
\begin{equation}
A_t^{\mathrm{img}}
\;=\;
\big[\,A_t^{\mathrm{ii}} \;\big|\; A_t^{\mathrm{it}}\,\big],
\end{equation}
where $A_t^{\mathrm{ii}} \in \mathbb{R}^{Q_{\mathrm{img}} \times K_{\mathrm{img}}}$ denotes image$\to$image attention
and $A_t^{\mathrm{it}} \in \mathbb{R}^{Q_{\mathrm{img}} \times K_{\mathrm{txt}}}$ denotes image$\to$text attention.
\paragraph{EMAG-I (image-to-image only).}
EMAG-I maintains an EMA only over the image-to-image block and perturbs that block:
\begin{align}
E_t^{\mathrm{ii}}
&= \beta\,E_{t-1}^{\mathrm{ii}} + (1-\beta)\,A_t^{\mathrm{ii}}, \\
\widetilde{A}_t^{\mathrm{ii}}
&= (1-\lambda)\,A_t^{\mathrm{ii}} + \lambda\,E_t^{\mathrm{ii}}.
\end{align}
\paragraph{EMAG (image-to-image + image-to-text).}
For EMAG, we maintain EMA over the full image-query row (image-to-image + image-to-text):
\begin{align}
E_t^{\mathrm{img}}
&= \beta\,E_{t-1}^{\mathrm{img}} + (1-\beta)\,A_t^{\mathrm{img}}, \\
\widetilde{A}_t^{\mathrm{img}}
&= (1-\lambda)\,A_t^{\mathrm{img}} + \lambda\,E_t^{\mathrm{img}}.
\end{align}
In the case of DiT-based architectures, EMAG-I is our default mode; throughout this work, any reference to ``EMAG'' in the context of DiT-based diffusion transformers corresponds to EMAG-I.
\subsection{Implementation details for baselines}
For APG and CADS, we follow the authors' reference pseudocode and adapt it to both DiT and SD3 codebases. For ERG, we use the official reference implementation and plug it into DiT and SD3 in the same way. S2 is reimplemented directly from the algorithm provided in the original paper. SEG is adapted from the official Stable Diffusion XL (SDXL) implementation, replacing the UNet attention processor with the joint-attention processor used by SD3-medium. For UNet-based approaches such as SAG and PAG, we adapt them to DiT and SD3 following the recommendations in \cite{ifriqi2025entropy}. PAG is used only for the DiT unconditional setting.
We now summarize the main implementation choices for SAG, SEG, and PAG.
\paragraph{SAG}
For SAG, we follow the authors’ SDXL implementation and adapt it to both DiT and SD3. We compute a global average pooling over the selected attention maps to obtain a spatial mask, threshold this mask at $1$ as in the original paper, and then apply a Gaussian blur with kernel size $9$. The SAG guidance scales for conditional and unconditional generation are chosen according to the ablations reported in Tables~\ref{tab:sd3_cond_sag_seg_s2_erg_apg_cads},\ref{tab:sd3_uncond_sag_seg_ierg},
\ref{tab:dit_cond_sag_seg_s2_erg_apg_cads},and~\ref{tab:dit_uncond_pag_sag_seg}. For DiT, we apply SAG at block~\textbf{13}, mirroring the original setup. For SD3 (MMDiT), following the ERG paper’s recommendation that blocks 6--8 are comparable to the UNet middle block, we apply SAG at layer~\textbf{7} to construct the SAG mask.
\paragraph{SEG}
For SEG, we start from the official SDXL implementation and adapt it to SD3-medium and DiT. In SD3, we adapt the UNet \texttt{AttentionProcessor2\_0} to the joint-attention processor used by the MMDiT backbone. In the original formulation, SEG operates on the unconditional branch; we preserve this behavior in the CFG setting by manipulating only the unconditional path when combining SEG with CFG for conditional generation. Throughout the paper, SEG+CFG is treated as the default SEG configuration for conditional setups. Following the recommendations in \cite{ifriqi2025entropy}, where SEG is applied at the UNet middle block, we target the corresponding transformer layers: blocks 6--8 in SD3 and blocks 12--15 in DiT. In all cases, we perturb only the image-to-image portion of the attention map, leaving image-to-text entries unchanged when applicable.
\paragraph{PAG}
For PAG, we implement the method for DIT using the authors' recommendations for unconditional configuration only.
\textbf{Note.} For simplicity, all DiT experiments use the DDIM sampler, while all SD3 experiments use the FlowMatchEulerDiscrete sampler. For HPS, we randomly subsample 1{,}000 images (without replacement, using a fixed random seed \textbf{8} ) from the pool of generated samples for each method and report the mean HPS over these images.
% ################################################################# Hyper Params Table ##############################################################################################3
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{6pt}
\caption{EMAG hyperparameter configurations for each task and backbone considered in our experiments.}
\label{tab:emag_hparams}
% \resizebox{\columnwidth}{!}{
\begin{tabular}{lccccccc}
\toprule
\textbf{Task} & $w_e$ & $\tau_{s}$ & $\tau_{e}$ & $\delta_{t}$ & $\beta$ & $l_{min}$ & $l_{max}$ \\
\midrule
T2I (SD3-Medium, MMDiT) & 1.75 & 26 & 5 & 5 & 0.988 & 6 & 8 \\
\midrule
Unconditional T2I (SD3-Medium) & 5.125 & 26 & 5 & 5 & 0.988 & 6 & 8 \\
\midrule
Class-conditional (DiT-XL/2) & 1.5 & 250 & 50 & 50 & 0.988 & 12 & 15 \\
\midrule
Unconditional (DiT-XL/2) & 7 & 250 & 50 & 50 & 0.988 & 12 & 15 \\
\bottomrule
\end{tabular}
% }
\end{table*}
% #####################################################################################################################################################################################
\section{Additional qualitative results}
\label{sec:additional_qualitative_results}
In this section, we provide additional qualitative examples.
Figures~\ref{fig: section_D_uncond_sd3_EMAG} and~\ref{fig: section_D_cond_sd3_EMAG} show unconditional and conditional samples from EMAG on SD3, respectively, while Figure~\ref{fig: section_D_uncond_sd3_dit_512} presents unconditional generations from DiT-XL/2 at 512$\times$512 resolution.
Figure~\ref{fig: section_D_baseline_vs_emag} provides a qualitative comparison between EMAG and other guidance baselines on SD3, and Figure~\ref{fig: section_D_cond_APG_CADS_with_EMAG} compares EMAG to alternative guidance strategies such as CFG, APG, and CADS, as well as their combinations with EMAG.
Finally, Figure~\ref{fig: section_D_baseline_vs_emag_512_dit} reports qualitative comparisons for DiT-XL/2 at 512$\times$512 resolution, contrasting EMAG-based guidance with competing baselines for class-conditional generation. All prompts used for the qualitative figures are listed in Table~\ref{tab:qual_prompts}.
% #################################### Unconditional Samples for SD3 EMAG #####################################################################
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{resources/section_d/D_uncond_SD3_EMAG.png}
\caption{\textbf{Additional qualitative samples for unconditional EMAG with $ w_{emag} = 5.125 $} on SD3-Medium}
\label{fig: section_D_uncond_sd3_EMAG}
\end{figure*}
% #############################################################################################################################################
% #################################### Conditional Samples for SD3 EMAG #####################################################################
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{resources/section_d/D_cond_SD3_EMAG.png}
\caption{\textbf{Additional qualitative samples for conditional Text to Image generation using SD3 with EMAG with $ w_{emag} = 1.75 $}}
\label{fig: section_D_cond_sd3_EMAG}
\end{figure*}
% #############################################################################################################################################
% #################################### Conditional Samples for SD3 EMAG #####################################################################
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{resources/section_d/D_cond_SD3_EMAG_comparasion_CFG_EMAG_APG_CADS.png}
\caption{\textbf{Additional qualitative comparison of conditional text-to-image generation with SD3 using EMAG, APG, CADS, and their combinations.}}
\label{fig: section_D_cond_APG_CADS_with_EMAG}
\end{figure*}
% #############################################################################################################################################
% #################################### Conditional Samples for SD3 EMAG comparasong with others #####################################################################
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{resources/section_d/D_cond_SD3_EMAG_comparasion_with_baselines.png}
\caption{\textbf{Additional qualitative comparison of EMAG and other baseline guidance methods for conditional text-to-image generation with SD3.}}
\label{fig: section_D_baseline_vs_emag}
\end{figure*}
% #############################################################################################################################################
% #################################### Conditional Samples for DIT EMAG comparasong with other baselines #####################################################################
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{resources/section_d/D_cond_DIT_512x512_EMAG_comparasion_with_baselines.png}
\caption{\textbf{Additional qualitative comparison of EMAG and other baseline guidance methods for class conditional generation with DiT-XL/2 512x512.}}
\label{fig: section_D_baseline_vs_emag_512_dit}
\end{figure*}
% ############################## Prompt Tables ###########################################################################################################
\begin{table*}[t]
\centering
\small
\setlength{\tabcolsep}{6pt}
\begin{tabular}{l p{0.8\linewidth}}
\toprule
\textbf{Figure} & \textbf{Prompt} \\
\midrule
Fig.~\ref{fig:front_image} & Ferry deck in windy daylight: white railings with a safety placard ``MUSTER STATION B''; a life ring labeled ``MV CORAL''; a paper cup with tea pressed by a metal spoon; distant headland under broken clouds; a timetable brochure on a bench marked ``11:40 $\rightarrow$ 12:15''. Bright, high-key light; ripples on water; clean maritime palette of whites, blues, and orange accents. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & A bus parked in a large parking lot. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & A man holding a camera up over his left shoulder. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & A person doing karate in a field at night. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & A close up of a plate of food containing broccoli. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & A beautiful woman taking a picture with her smart phone. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & Two people standing in a kitchen looking around. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & A young man wearing black attire and a flowered tie is standing and smiling. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & A sink is shown with a mirror and lights. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & A bus that is sitting in the street. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & A plane flies over water with two islands nearby. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & A guy jumping with a tennis racket in his hand. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & Two people standing in a kitchen looking around. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & Two beautiful young ladies giving an elephant a bath in a river. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & a fogy picture with a red light in the back ground. \\
Fig.~\ref{fig: section_D_cond_sd3_EMAG} & Three teddy bears, each a different color, snuggling together. \\
Fig.~\ref{fig:cond-samples} & This wire metal rack holds several pairs of shoes and sandals. \\
Fig.~\ref{fig:cond-samples} & A small kitchen with low a ceiling. \\
Fig.~\ref{fig:cond-samples} & A woman stands in the dining area at the table. \\
\bottomrule
\end{tabular}
\caption{Prompts used for qualitative examples.}
\label{tab:qual_prompts}
\end{table*}
% ############################## Prompt Tables ###########################################################################################################
% #################################### Unconditional Samples for DIT 512 EMAG #####################################################################
\begin{figure*}[t]
\centering
\includegraphics[width=\linewidth]{resources/section_d/D_EMAG_uncond_DIT_512x512.png}
\caption{\textbf{Qualitative samples (uncurated) for unconditional EMAG with $ w_{emag} = 7 $} for DIT-XL/2 512x512 }
\label{fig: section_D_uncond_sd3_dit_512}
\end{figure*}
\end{document}
%% This file contains a number of tweaks that are typically applied to the main document.
%% They are not enabled by default, but can be enabled by uncommenting the relevant lines.
%%
%% Inline annotations; for predefined colors, refer to "dvipsnames" in the xcolor package:
%% https://tinyurl.com/overleaf-colors
%%
\newcommand{\red}[1]{{\color{red}#1}}
\newcommand{\todo}[1]{{\color{red}#1}}
\definecolor{huycolor}{RGB}{10, 210, 30}
\newcommand{\huy}[1]{\textcolor{huycolor}{#1}}
%%
%% disable for camera ready / submission by uncommenting these lines
%%
% \renewcommand{\TODO}[1]{}
% \renewcommand{\todo}[1]{#1}
%%
%% work harder in optimizing text layout. Typically shrinks text by 1/6 of page, enable
%% it at the very end of the writing process, when you are just above the page limit
%%
% \usepackage{microtype}
%%
%% fine-tune paragraph spacing
%%
% \renewcommand{\paragraph}[1]{\vspace{.5em}\noindent\textbf{#1.}}
%%
%% globally adjusts space between figure and caption
%%
% \setlength{\abovecaptionskip}{.5em}
%%
%% Allows "the use of \paper to refer to the project name"
%% with automatic management of space at the end of the word
%%
% \usepackage{xspace}
% \newcommand{\paper}{ProjectName\xspace}
%%
%% Commonly used math definitions
%%
% \DeclareMathOperator*{\argmin}{arg\,min}
% \DeclareMathOperator*{\argmax}{arg\,max}
%%
%% Tigthen underline
%%
% \usepackage{soul}
% \setuldepth{foobar}