BeCARE: Budgeted Cache Refresh for Diffusion Transformer Acceleration

Extracted source text · arXiv 2605.27075

\documentclass[11pt]{article}

\usepackage[final]{acl}

\usepackage{times}
\usepackage{latexsym}
\usepackage[T1]{fontenc}
\usepackage[utf8]{inputenc}
\usepackage{microtype}
\usepackage{inconsolata}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{booktabs}
\usepackage[table]{xcolor}
\usepackage{pifont}

\title{SoftCap: Soft-Budget Control for Diffusion Transformer Acceleration}
\author{
  \textbf{Yuhang Zhang\textsuperscript{1,*}},
  \textbf{Junxiang Qiu\textsuperscript{2,*}},
  \textbf{Huixia Ben\textsuperscript{3}},
  \textbf{Zhenhua Tang\textsuperscript{4}},
  \textbf{Shuo Wang\textsuperscript{2}},
  \textbf{Yanbin Hao\textsuperscript{1,\ensuremath{\dagger}}}
  \\[0.5em]
  {\small
    \textsuperscript{1}Hefei University of Technology,
    \textsuperscript{2}University of Science and Technology of China,
    \textsuperscript{3}Anhui University of Science and Technology,
    \textsuperscript{4}University of Macau.
  }
  \\[0.5em]
  {\small
  \texttt{
    2023210023@mail.hfut.edu.cn,
    qiujx@mail.ustc.edu.cn,
    \{huixiaben, shuowang.edu\}@gmail.com,
    haoyanbin@hfut.edu.cn
  }
}
}

\begin{document}
\maketitle

\begin{abstract}
Diffusion Transformers (DiTs) achieve strong visual quality, but their iterative denoising process requires many costly Transformer evaluations. Training-free acceleration methods reduce this cost by caching, forecasting, or verifying intermediate features, yet the runtime decision of when to execute a Full step is often driven by fixed schedules or hand-tuned thresholds. We propose \textbf{SoftCap}, a training-free control layer for cache-based DiT inference. SoftCap couples a Trajectory Drift Observer, which estimates local cache risk from lightweight hidden-state statistics, with a Soft-Budget PI Controller, which adjusts the Full-triggering threshold from realized compute relative to a fixed reference profile. The budget is a soft ceiling: it shapes the threshold but does not require a run to spend a prescribed number of Full evaluations. On FLUX.1-dev, SoftCap improves over SpeCa at a comparable middle-compute operating point, raising ImageReward from 0.967 to 0.981 and reducing LPIPS-Full from 0.518 to 0.498 at nearly identical FLOPs, while target-sweep diagnostics show the intended soft-ceiling behavior as the budget is relaxed.
\end{abstract}

\input{sec/1_introduction}

\input{sec/0_pipeline_figure}

\input{sec/2_related_work}

\input{sec/3_method}

\input{sec/4_experiments}

\input{sec/5_conclusion}

\input{sec/6_limitations}

\bibliography{custom}

\newpage
\appendix
% \onecolumn
\section*{Appendix}
\section{Additional Ablation Studies}
\label{app:ablation}

This appendix provides additional ablations for the three design choices in SoftCap: the soft-budget controller, the Trajectory Drift Observer (TDO), and the monitored layer used to compute drift. All image-quality metrics in this section are computed against the Full FLUX output generated from the same prompt and seed. Therefore, FID-Full, PSNR-Full, SSIM-Full, and LPIPS-Full should be read as internal generated-vs-Full fidelity metrics rather than dataset-level FID.

\paragraph{Soft-budget controller.}
Figure~\ref{fig:app-controller-ablation} compares fixed-threshold decision rules with PI-based threshold control on the 200-prompt FLUX.1-dev ablation run. The fixed threshold variant uses a static risk threshold, while the PI controller adapts the threshold according to the gap between realized and reference Full usage. The PI controller reduces FID-Full from 84.19 to 83.15 and LPIPS-Full from 0.313 to 0.310, with a moderate increase in average FLOPs from 557.44T to 574.09T. Adding the temporal cue further improves FID-Full to 82.98 and LPIPS-Full to 0.309. These results support the controller role of SoftCap: it does not simply spend a fixed number of Full evaluations, but adjusts cache acceptance according to online budget state.

\begin{figure}[h]
\centering
\includegraphics[width=\linewidth]{fig/fig_controller_ablation.pdf}
\caption{Controller ablation on FLUX.1-dev. PI-based policies trade a small amount of computation for improved Full-referenced fidelity relative to fixed-threshold rules. FID-Full and LPIPS-Full are internal generated-vs-Full metrics.}
\label{fig:app-controller-ablation}
\end{figure}

\paragraph{TDO cue ablation.}
Figure~\ref{fig:app-tdo-cue-ablation} ablates the four low-cost cues in the Trajectory Drift Observer. Removing magnitude drift causes the largest degradation among leave-one-out variants, increasing FID-Full by 7.78 and LPIPS-Full by 0.021 relative to the full TDO. Using temporal volatility alone is also insufficient: it reaches a higher apparent speedup but substantially degrades ImageReward, FID-Full, and LPIPS-Full. These results indicate that TDO benefits from combining magnitude, directional, anchor-distance, and temporal-volatility signals instead of relying on a single proxy.

\begin{figure}[h]
\centering
\includegraphics[width=\linewidth]{fig/fig_tdo_cue_ablation.pdf}
\caption{TDO cue ablation. The complete observer combines complementary drift signals; removing or isolating cues can reduce Full-referenced fidelity, especially when magnitude drift is omitted or temporal volatility is used alone.}
\label{fig:app-tdo-cue-ablation}
\end{figure}

\paragraph{TDO weight sensitivity.}
Figure~\ref{fig:app-weight-sensitivity} evaluates several TDO score weight settings, where the tuple denotes the weights for magnitude, direction, anchor deviation, and temporal volatility. The tested settings remain in a narrow fidelity range: FID-Full varies around 103.7--104.7 and LPIPS-Full around 0.401--0.405 while FLOPs range from 577.07T to 630.02T. The selected weighting $(0.45,0.25,0.15,0.15)$ lies in this stable region and is not an isolated optimum.

\begin{figure}[h]
\centering
\includegraphics[width=\linewidth]{fig/fig_weight_sensitivity.pdf}
\caption{Sensitivity to TDO score weights. The selected weighting lies in a stable region of the fidelity-compute trade-off rather than relying on a single isolated configuration.}
\label{fig:app-weight-sensitivity}
\end{figure}

\paragraph{Monitored layer.}
Figure~\ref{fig:app-layer-sweep} studies the DiT layer used by TDO to monitor hidden-state drift. We sweep layers 1, 7, 13, 19, 25, 31, and 37 on 50 prompts. Later layers produce stronger drift separation: layer 37 has the largest early-late L1 gap (0.202) and cosine gap (0.024), and also gives the best Full-referenced fidelity in this sweep, with FID-Full 143.71, PSNR-Full 16.05, SSIM-Full 0.628, and LPIPS-Full 0.315. This supports using an explicitly validated monitored layer rather than treating the layer index as arbitrary.

\begin{figure}[h]
\centering
\includegraphics[width=\linewidth]{fig/fig_layer_sweep.pdf}
\caption{Monitored-layer sweep for the Trajectory Drift Observer. The layer choice affects both Full-referenced fidelity and drift separation; in this sweep, the late monitored layer provides the strongest separation and best LPIPS-Full.}
\label{fig:app-layer-sweep}
\end{figure}

\paragraph{Soft ceiling behavior.}
We also include the soft-ceiling diagnostic in Figure~\ref{fig:app-soft-ceiling}. Increasing $N_{cap}$ from 8 to 40 expands the allowed compute envelope, but the realized number of Full evaluations only grows from about 10 to 13. At the same time, observed risk crossings are converted into Full evaluations. Thus the compute plateau is caused by limited high-risk crossings rather than a downstream eligibility bottleneck. This supports the interpretation of $N_{cap}$ as a soft computational ceiling instead of a hard Full-evaluation target.

\begin{figure}[h]
\centering
\includegraphics[width=0.72\linewidth]{fig/fig_soft_ceiling_diagnostic.pdf}
\caption{Soft-ceiling diagnostic on a five-prompt target sweep. Increasing $N_{cap}$ enlarges the allowed compute envelope, but realized Full evaluations saturate when the observer does not produce enough high-risk crossings.}
\label{fig:app-soft-ceiling}
\end{figure}






\end{document}

\section{Introduction}

Diffusion Transformers (DiTs) have become a central backbone for high-fidelity visual generation, but their inference cost remains high. A single sample requires many denoising evaluations, and each evaluation runs a large Transformer network over high-dimensional latent tokens. This cost is especially visible in recent text-to-image and video systems, where generation latency and serving throughput are constrained not only by the number of sampling steps but also by the per-step cost of attention and feed-forward blocks. Reducing this cost without retraining the base model is therefore a practical requirement for deploying large DiTs.

Training-free feature caching has emerged as a practical way to reduce the per-step cost of diffusion inference. Rather than executing all Transformer blocks at every denoising step, cache-based methods reuse or approximate intermediate features. Representative examples include DeepCache-style feature reuse and TaylorSeer-style trajectory forecasting \citep{deepcache,taylorseer}; speculative feature caching further studies when a predicted feature should be accepted \citep{speca}. Together, these methods show that diffusion trajectories contain substantial temporal redundancy and have advanced the design of cache engines and feature predictors.

The remaining question is when a runtime system should trust a cached approximation. Many Full/Cache policies still rely on static schedules, fixed thresholds, or manually selected acceptance rules. Such open-loop policies miss an important asymmetry: the value of a Full evaluation depends on both local approximation risk and the amount of computation already spent along the current sample trajectory. When the run is close to its compute ceiling, a moderate-risk cached approximation may be acceptable; when the run has spent less compute than expected, a high-risk step may justify a Full evaluation. Conversely, if the trajectory remains stable, executing additional Full steps only to meet a nominal target may bring little marginal benefit. DiT acceleration therefore needs a policy layer that regulates Full evaluations under a compute ceiling, in addition to a cache engine that approximates features.

We propose SoftCap, a training-free soft-budget control framework for accelerating DiTs. SoftCap treats Full/Cache selection as a feedback-controlled decision rather than a fixed caching schedule. It combines a Trajectory Drift Observer (TDO), which estimates the risk of continuing cached approximation from lightweight drift cues, with a Soft-Budget PI Controller, which adapts the Full-triggering threshold according to realized compute. A final Risk-Gated Full/Cache Decision compares the observer risk with the controller threshold and selects either a Full evaluation or the underlying cache engine. The design deliberately separates local trajectory risk from global compute state: the observer asks whether the current approximation looks risky, while the controller asks how strict the risk gate should be under the current soft budget.

A key property of SoftCap is that its budget parameter is a \emph{soft computational ceiling}, not a required spending target. The controller can make Full evaluations easier or harder to trigger, but it does not force the system to spend computation when the observed trajectory risk remains low. This behavior is useful for deployment: a user-specified compute limit should shape latency without requiring unnecessary model evaluations.

We evaluate SoftCap on FLUX.1-dev with a 200-prompt text-to-image benchmark. Under a consistent generated-vs-full evaluation protocol, SoftCap provides a competitive quality-efficiency tradeoff against representative caching, forecasting, and speculative baselines. Target-sweep diagnostics further support the soft-ceiling interpretation: relaxing the budget raises the allowed compute, but the realized number of Full evaluations saturates when the observer does not identify enough high-risk steps. Detailed metrics and matched-compute comparisons are provided in the experimental section.

Our contributions are:
\begin{itemize}
    \item We formulate DiT Full/Cache scheduling as a soft-budget feedback control problem and introduce SoftCap, a training-free control layer for feature caching.
    \item We design a Trajectory Drift Observer that estimates caching risk from low-cost magnitude, direction, anchor-deviation, and temporal-volatility cues.
    \item We introduce a Soft-Budget PI Controller that adapts the Full-triggering threshold from realized compute relative to a fixed reference budget profile.
    \item We provide FLUX.1-dev evidence showing that SoftCap yields a competitive quality-efficiency tradeoff and that its budget parameter behaves as a soft computational ceiling rather than a required number of Full evaluations.
\end{itemize}

\begin{figure*}[t]
\centering
\includegraphics[width=\textwidth]{fig/softcap_pipeline.pdf}
\caption{
Overview of SoftCap for cache-based DiT acceleration. A Taylor cache engine proposes inexpensive cached features, the Trajectory Drift Observer estimates trajectory-dependent cache risk $s_t$, the Soft-Budget PI Controller produces a dynamic threshold $\tau_t$ from realized Full usage, and the Risk-Gated Decision selects Full or Cache at each denoising step.
}
\label{fig:softcap_pipeline}
\end{figure*}

\section{Related Work}

\subsection{Efficient Diffusion and DiT Inference}

Diffusion models generate samples through iterative denoising, which makes inference more expensive than single-pass generative models. DiTs further increase the per-step cost because their denoising networks rely on Transformer blocks with attention and feed-forward computation over many latent tokens \citep{dit}. Existing acceleration methods reduce either the number of denoising evaluations or the cost of each evaluation. Step-reduction methods include improved numerical solvers, distillation, consistency-style modeling, and consistency transfer for multimodal diffusion systems \citep{mgct}, but they may require a different sampler or additional training. SoftCap focuses on the complementary setting: the sampling schedule and base model are kept intact, and the per-step network cost is reduced by training-free feature caching.

\subsection{Feature Caching and Forecasting}

Feature caching exploits temporal redundancy across nearby denoising steps. Early cache-based methods reuse intermediate activations from previous steps to skip parts of the denoising network \citep{deepcache,fora,teacache}. This strategy is attractive because it can be applied as an inference-time wrapper, but direct reuse becomes less reliable when the cache horizon grows. Forecasting-based methods address this issue by predicting the current feature from historical states. TaylorSeer models feature trajectories with finite differences and Taylor expansion \citep{taylorseer}; PrediT uses linear multistep forecasting with a corrector for high-dynamics regions \citep{predit}; FoCa interprets hidden-feature evolution through an ODE lens and combines forecasting with calibration \citep{foca}. Other work decomposes the feature space itself. FreqCa separates low- and high-frequency feature dynamics \citep{freqca}, while SVD-Cache uses a subspace-aware decomposition to stabilize principal and residual components \citep{svdcache}. A related branch explicitly compensates cache-induced approximation error: Error-Optimized Cache introduces prior-knowledge-based correction for reused modules, and Gradient-Optimized Cache propagates cached feature differences to reduce positional drift in skipped computations \citep{eoc,goc}.

These methods primarily improve the approximation mechanism: how cached features are reused, forecasted, decomposed, or corrected. SoftCap addresses a different layer of the acceleration pipeline. Given a cache engine that can produce an approximate hidden state, SoftCap controls when the approximation should be accepted under the current risk and compute state.

\subsection{Adaptive Caching and Verification Policies}

Several recent methods improve caching by making the decision more adaptive. ToCa observes that tokens have different sensitivity to cached approximation and applies token-wise feature caching \citep{toca}. TAP uses a low-cost probe to select token-level predictors \citep{tap}. ClusCa clusters spatial tokens and computes representative tokens within each cluster \citep{clusca}. RFC estimates output-feature changes from input-output relations and uses relational cache scheduling to trigger recomputation when expected changes are large \citep{rfc}. Hybrid-grained caching further shows that cache granularity can differ across model components, such as block-level and prompt-level reuse in controllable generation \citep{hgc}. These works show that uniform caching policies are often insufficient because approximation risk varies across tokens, layers, samples, denoising stages, and model components.

Speculative verification offers another perspective on the Full/Cache decision. SpeCa, for example, predicts intermediate features with a lightweight draft mechanism and verifies whether the drafted features should be accepted \citep{speca}. This line of work studies the reliability of a proposed approximation. SoftCap uses a different abstraction: it does not introduce a draft-and-verify pipeline, but places a feedback policy around a cache engine. Its observer summarizes runtime trajectory state through drift cues, and its controller adjusts the Full-triggering threshold according to realized compute. This shifts the policy question from only ``is this approximation reliable?'' to ``how strict should the risk gate be under a soft compute ceiling?''

\subsection{Budget-Aware Compute Allocation}

The most relevant distinction of SoftCap is its treatment of compute allocation. Existing caching methods often report results at different acceleration ratios, but the runtime policy is rarely formulated as a closed-loop budget controller. SoftCap maintains a soft budget ceiling, a fixed reference progress profile, and the realized number of Full evaluations. The discrepancy between the reference and realized compute controls the risk threshold, while the Trajectory Drift Observer supplies the local approximation-risk signal. This decoupling makes the role of each component explicit: the observer estimates trajectory-dependent risk, the controller regulates how strict the decision should be under the current compute state, and the final gate converts risk crossings into Full evaluations. SoftCap therefore acts as a control layer for cache-based DiT acceleration rather than a replacement for existing cache engines or feature predictors.

\section{Method}

\subsection{Preliminaries}

\paragraph{Diffusion Transformer inference.}
A diffusion model generates a sample by iteratively denoising a noisy latent. At each denoising step, a neural denoiser predicts an update direction and the sampler advances the latent to the next noise level. In DiTs, the denoiser is implemented as a stack of Transformer blocks over latent tokens. Let $\mathbf{h}^{l}_{t}$ denote the hidden feature at layer or module $l$ and denoising step $t$. A standard DiT inference pass evaluates all relevant blocks at every step, which makes the total cost scale with both the number of denoising steps and the per-step Transformer computation.

\paragraph{Feature caching.}
Training-free caching reduces this cost by reusing or approximating hidden features at selected steps. A cache engine stores an anchor feature from a recent Full evaluation and uses it to approximate later features. If the approximation is accepted, expensive downstream block computation can be skipped; if it is rejected, the model performs a Full evaluation and refreshes the cache state. SoftCap operates in this cached-inference setting. It is cache-engine agnostic in principle, and we instantiate it with a Taylor-based cache engine in our experiments.

\subsection{SoftCap Overview}

SoftCap introduces a feedback-controlled Full/Cache policy for cached DiT inference. Figure~\ref{fig:softcap_pipeline} summarizes the overall pipeline. At each step, the system chooses
\begin{equation}
a_t \in \{\mathrm{Full},\mathrm{Cache}\}.
\end{equation}
A Full action executes the original model computation for the selected modules and refreshes the cache anchor. A Cache action uses the underlying cache engine to approximate the corresponding hidden features and skip expensive block computation.

The central design is to separate local trajectory risk from global compute state. The \textbf{Trajectory Drift Observer} estimates a risk score $s_t$ from lightweight drift cues. The \textbf{Soft-Budget PI Controller} converts the realized compute state into a dynamic threshold $\tau_t$. The \textbf{Risk-Gated Full/Cache Decision} then applies
\begin{equation}
a_t =
\begin{cases}
\mathrm{Full}, & s_t \geq \tau_t, \\
\mathrm{Cache}, & s_t < \tau_t .
\end{cases}
\label{eq:softcap_decision_en}
\end{equation}
Here $s_t$ measures the risk of continuing along the cached trajectory, while $\tau_t$ controls how much risk is required to justify a Full evaluation under the current soft budget.

\subsection{Taylor Cache Engine}

We use a Taylor-based cache engine as the underlying approximation mechanism. Let $a$ denote the most recent Full anchor step. At a Full step, the engine stores the hidden state and finite-difference terms at a selected layer or module,
\begin{equation}
\left\{\mathbf{h}^{l}_{a}, \Delta^{(1)}\mathbf{h}^{l}_{a}, \ldots, \Delta^{(m)}\mathbf{h}^{l}_{a}\right\}.
\end{equation}
When a later step is assigned to Cache, the engine estimates the hidden feature with a finite-order Taylor approximation,
\begin{equation}
\widehat{\mathbf{h}}^{l}_{t}
=
\mathbf{h}^{l}_{a}
+
\sum_{r=1}^{m}
\alpha_r(t,a)\Delta^{(r)}\mathbf{h}^{l}_{a},
\label{eq:taylor_cache_en}
\end{equation}
where $\alpha_r(t,a)$ denotes the coefficient determined by the cache engine from the step distance and finite-difference convention. If the current step is assigned to Full, the true forward computation is executed and the anchor state is refreshed. SoftCap only requires the cache engine to provide an approximate feature and maintain its anchor state; the controller described below is independent of the specific Taylor coefficient parameterization.

\subsection{Trajectory Drift Observer}

The Trajectory Drift Observer (TDO) estimates whether the current cached trajectory has drifted away from the latest Full anchor. Let $\mathbf{x}^{\mathrm{in}}_{t,l}$ be the input hidden state at the monitored DiT layer $l$ and step $t$, and let $\mathbf{x}^{\mathrm{in}}_{a,l}$ be the corresponding input stored at the latest Full anchor. TDO uses four low-cost drift cues.

\paragraph{Magnitude Drift.}
Magnitude Drift measures relative amplitude change from the anchor:
\begin{equation}
f_{\mathrm{mag}}(t)=
\frac{\left\|\mathbf{x}^{\mathrm{in}}_{t,l}-\mathbf{x}^{\mathrm{in}}_{a,l}\right\|_1}
{\left\|\mathbf{x}^{\mathrm{in}}_{t,l}\right\|_1+\epsilon}.
\label{eq:magnitude_drift_en}
\end{equation}

\paragraph{Directional Drift.}
Directional Drift measures angular deviation after vectorizing the layer input:
\begin{equation}
f_{\mathrm{dir}}(t)
=
1-
\frac{
\left\langle \mathrm{vec}(\mathbf{x}^{\mathrm{in}}_{t,l}),
\mathrm{vec}(\mathbf{x}^{\mathrm{in}}_{a,l})\right\rangle}
{
\left\|\mathrm{vec}(\mathbf{x}^{\mathrm{in}}_{t,l})\right\|_2
\left\|\mathrm{vec}(\mathbf{x}^{\mathrm{in}}_{a,l})\right\|_2
+\epsilon}.
\label{eq:directional_drift_en}
\end{equation}

\paragraph{Anchor Deviation.}
Anchor Deviation tracks the normalized cache distance from the latest Full anchor,
\begin{equation}
f_{\mathrm{anc}}(t)=
\min\left(\frac{d(t,a)}{D_{\max}},1\right),
\label{eq:anchor_deviation_en}
\end{equation}
where $d(t,a)$ is the cache distance from the current step to the anchor and $D_{\max}$ is the configured maximum skip distance.

\paragraph{Temporal Volatility.}
Temporal Volatility reuses the first-order difference term maintained by the cache engine:
\begin{equation}
f_{\mathrm{vol}}(t)=
\sqrt{\frac{1}{n}\left\|\Delta^{(1)}\mathbf{h}^{l}_{a}\right\|_2^2},
\label{eq:temporal_volatility_en}
\end{equation}
where $n$ is the number of feature elements. This term captures local feature dynamics around the anchor.

The four cues are normalized and clipped,
\begin{equation}
\phi_i(t)=\operatorname{clip}\left(\frac{f_i(t)}{c_i},0,1\right),
\end{equation}
and then combined into a base risk score,
\begin{equation}
s^{\mathrm{base}}_t=\sum_i w_i \phi_i(t).
\label{eq:tdo_base_en}
\end{equation}
In our main experiments, the normalization constants for Magnitude Drift, Directional Drift, Anchor Deviation, and Temporal Volatility are fixed to $(0.5,0.08,0.5,4.0)$, and the weights are fixed to $(0.45,0.25,0.15,0.15)$. These values are calibrated once and are not tuned per prompt, random seed, or budget setting.

TDO also supports a lightweight positive increment option:
\begin{equation}
\Delta s_t=\max(0,s^{\mathrm{base}}_t-s^{\mathrm{base}}_{t-1}),
\end{equation}
\begin{equation}
s_t=\operatorname{clip}\left(s^{\mathrm{base}}_t+\gamma_t\Delta s_t,0,1\right).
\label{eq:tdo_score_en}
\end{equation}
This term amplifies abrupt increases in risk while leaving risk decreases unchanged. It is an optional observer switch rather than a separate trained component. Setting $\gamma_t=0$ disables the increment and reduces TDO to the base weighted score; enabling it provides a lightweight calibration for settings where upward risk changes should be emphasized.

\begin{table*}[t]
\centering
\caption{\textbf{Quantitative comparison in text-to-image generation on FLUX.} SoftCap rows are highlighted. FLOPs are averaged per image in tera-FLOPs, and FLOPs speedup is computed against the Full FLUX 50-step baseline. Full-referenced metrics compare accelerated outputs with Full FLUX outputs generated from the same prompts.}
\vspace{-2mm}
\setlength{\tabcolsep}{4pt}
\small
\resizebox{0.98\textwidth}{!}{
\begin{tabular}{l | c | c c | c c | c c c c}
\toprule
{\bf Method} & {\bf Efficient} & \multicolumn{2}{c|}{\bf Acceleration} & \multicolumn{2}{c|}{\bf Semantic Quality} & \multicolumn{4}{c}{\bf Full-Referenced Fidelity} \\
\cline{3-10}
{\bf FLUX.1-dev} & {\bf Attention} & {\bf FLOPs(T) $\downarrow$} & {\bf FLOPs Speed $\uparrow$} & {\bf ImageReward $\uparrow$} & {\bf CLIPScore $\uparrow$} & {\bf FID $\downarrow$} & {\bf PSNR $\uparrow$} & {\bf SSIM $\uparrow$} & {\bf LPIPS $\downarrow$} \\
\midrule
{\bf Full FLUX, 50 steps} & \ding{52} & 3719.50 & 1.00$\times$ & 0.996 & 27.765 & 0.00 & $\infty$ & 1.000 & 0.000 \\
FLUX, 25 steps & \ding{52} & 1859.75 & 2.00$\times$ & 0.975 & 27.696 & 86.14 & 13.957 & 0.550 & 0.485 \\
FLUX, 10 steps & \ding{52} & 743.90 & 5.00$\times$ & 0.821 & 27.592 & 107.67 & 12.775 & 0.501 & 0.568 \\
\midrule
SpeCa & \ding{52} & 967.75 & 3.84$\times$ & 0.986 & 27.735 & \textbf{86.95} & 13.705 & \textbf{0.539} & \textbf{0.492} \\
TaylorSeer   & \ding{52} & 967.07 & 3.85$\times$ & 0.941 & \textbf{27.808} & 101.16 & 12.884 & 0.509 & 0.541 \\
FORA ($N=4$)   & \ding{52} & 967.07 & 3.85$\times$ & 0.870 & 27.613 & 111.21 & 11.118 & 0.449 & 0.654 \\
\rowcolor{gray!18}
\textbf{SoftCap} & \ding{52} & \textbf{917.39} & \textbf{4.05$\times$} & \textbf{0.995} & 27.710 & 87.74 & \textbf{13.707} & 0.536 & 0.493 \\
\midrule
SpeCa & \ding{52} & 725.58 & \textbf{5.13$\times$} & 0.967 & 27.652 & 90.52 & 13.311 & 0.513 & 0.518 \\
TaylorSeer ($O=2$)   & \ding{52} & 743.90 & 5.00$\times$ & 0.980 & 27.709 & 114.54 & 10.204 & 0.390 & 0.688 \\
FORA ($N=5$)   & \ding{52} & 743.90 & 5.00$\times$ & 0.809 & 27.624 & 114.08 & 11.252 & 0.446 & 0.657 \\
\rowcolor{gray!18}
\textbf{SoftCap} & \ding{52} & \textbf{724.62} & \textbf{5.13$\times$} & \textbf{0.981} & \textbf{27.815} & \textbf{89.29} & \textbf{13.677} & \textbf{0.530} & \textbf{0.498} \\
\midrule
SpeCa & \ding{52} & 603.44 & 6.16$\times$ & 0.964 & \textbf{28.124} & 96.82 & 12.919 & 0.483 & 0.550 \\
TaylorSeer ($O=2$)   & \ding{52} & \textbf{595.12} & \textbf{6.25$\times$} & 0.876 & 27.489 & 120.66 & 10.019 & 0.366 & 0.715 \\
FORA ($N=7$)   & \ding{52} & \textbf{595.12} & \textbf{6.25$\times$} & 0.663 & 27.225 & 120.53 & 11.468 & 0.451 & 0.671 \\
\rowcolor{gray!18}
\textbf{SoftCap} & \ding{52} & 614.55 & 6.05$\times$ & \textbf{0.987} & 27.879 & \textbf{89.82} & \textbf{13.598} & \textbf{0.517} & \textbf{0.508} \\
\bottomrule
\end{tabular}}
\label{table:FLUX-Metrics}
\footnotesize
% \begin{itemize}
% \item \textcolor{gray!70}{Gray rows}: SoftCap.  : configurations with substantial semantic or Full-referenced fidelity degradation under this protocol.
% \item Bold numbers mark the best accelerated result within each compute block, following the metric direction. FID, PSNR, SSIM, and LPIPS are Full-referenced internal metrics, not dataset-FID or real-data distribution scores.
% \end{itemize}
\vspace{-4mm}
\end{table*}

\subsection{Soft-Budget PI Controller}

The controller converts a soft compute ceiling into a dynamic threshold. Let $N_{\mathrm{cap}}$ be the user-specified ceiling on Full evaluations. SoftCap also uses a fixed, non-decreasing reference profile $C(t)\in[0,1]$ over denoising progress. In our experiments, $C(t)$ is a lookup table that tabulates the cumulative Full-step fraction of a fixed-threshold reference policy as a function of normalized denoising progress. The table is constructed once before evaluation and then held fixed for all prompts, seeds, and budget ceilings; feedback changes $N_{\mathrm{actual}}(t)$ and $\tau_t$, not the reference profile itself.

At step $t$, the controller computes the reference cumulative Full count
\begin{equation}
N_{\mathrm{ref}}(t)=C(t)N_{\mathrm{cap}}.
\label{eq:nref_en}
\end{equation}
Let $N_{\mathrm{actual}}(t)$ be the number of Full evaluations that have actually occurred before the current decision. The budget-tracking error is
\begin{equation}
e_t=N_{\mathrm{actual}}(t)-N_{\mathrm{ref}}(t).
\label{eq:budget_error_en}
\end{equation}
Positive tracking error means that realized compute is ahead of the reference profile. Negative tracking error means that realized compute is below it.

SoftCap maintains a clamped integral state,
\begin{equation}
I_t=\operatorname{clip}(I_{t-1}+e_t,I_{\min},I_{\max}),
\label{eq:integral_en}
\end{equation}
and updates the threshold with a PI rule,
\begin{equation}
\tau_t=
\operatorname{clip}
\left(
\tau_0+K_p e_t+K_i I_t,
\tau_{\min},\tau_{\max}
\right).
\label{eq:pi_threshold_en}
\end{equation}
Since Full is triggered by $s_t\geq\tau_t$, overspending raises the threshold and makes Full less likely, while underspending lowers the threshold and makes Full easier to trigger. The controller therefore regulates the acceptance threshold rather than directly forcing a fixed number of Full evaluations.

\subsection{Risk-Gated Decision and State Update}

After computing $s_t$ and $\tau_t$, SoftCap applies Eq.~\ref{eq:softcap_decision_en}. The realized Full counter is then updated as
\begin{equation}
N_{\mathrm{actual}}(t+1)=
\begin{cases}
N_{\mathrm{actual}}(t)+1, & a_t=\mathrm{Full},\\
N_{\mathrm{actual}}(t), & a_t=\mathrm{Cache}.
\end{cases}
\label{eq:nactual_update_en}
\end{equation}
This updated counter is fed back into the next step. In the implementation, standard warmup and cache-distance guard paths are evaluated before the risk gate to initialize and protect the cache state. These structural Full evaluations are counted in $N_{\mathrm{actual}}(t)$ because they consume full model computation.

The soft-ceiling interpretation follows from this update structure. $N_{\mathrm{cap}}$ affects the threshold through $N_{\mathrm{ref}}(t)$, but it does not force Full evaluations when the observed risk remains below threshold. Thus, SoftCap can naturally realize fewer Full evaluations than the ceiling when the trajectory is stable.




\subsection{Complexity}

Let $C_{\mathrm{full}}$ be the cost of a Full step and $C_{\mathrm{cache}}$ the cost of a Cache step, with $C_{\mathrm{cache}}\ll C_{\mathrm{full}}$. Let $C_{\mathrm{obs}}$ and $C_{\mathrm{ctrl}}$ denote the small overheads of TDO and the PI controller. If a run uses $N_{\mathrm{full}}$ Full evaluations, its cost is approximately
\begin{equation}
\begin{aligned}
\mathcal{C}_{\mathrm{SoftCap}}
= {}&
N_{\mathrm{full}} C_{\mathrm{full}}
+(T-N_{\mathrm{full}})C_{\mathrm{cache}}\\
&+T(C_{\mathrm{obs}}+C_{\mathrm{ctrl}}).
\end{aligned}
\end{equation}
The observer and controller use only tensor statistics and scalar updates, so the dominant factor is the realized number of Full evaluations. SoftCap improves efficiency by reducing Full evaluations and concentrating them at risk-gated points of the trajectory.

\section{Experiments}
\begin{figure*}[t]
\centering
\includegraphics[width=\textwidth]{fig/sample.pdf}
\caption{
Comparison of Samples generated by different methods.
}
\label{fig:sample}
\end{figure*}

\subsection{Experimental Setup}

We evaluate SoftCap on FLUX.1-dev text-to-image generation. The main benchmark contains 200 prompts, and all accelerated outputs are compared with images generated by the Full FLUX baseline under the same prompts and $1024\times1024$ resolution. SoftCap is instantiated with the Taylor cache engine described in Section~3. Unless otherwise noted, the main comparison uses the base weighted TDO score, while the optional positive-increment switch is evaluated separately.

We compare against representative training-free acceleration baselines, including SpeCa~\citep{speca}, TaylorSeer~\citep{taylorseer}, and FORA~\citep{fora}. The primary efficiency evidence in our comparison is the measured average FLOPs in tera-FLOPs (TF), with speedup computed relative to the Full FLUX baseline. For quality, we report ImageReward and CLIPScore as semantic preference and text-image alignment metrics. We also report internal Full-referenced fidelity metrics, including FID-Full, PSNR-Full, SSIM-Full, and LPIPS-Full. These Full-referenced metrics compare each method's generated images against the Full FLUX outputs on the same prompts. They are therefore useful for same-protocol relative comparison, but should not be interpreted as dataset-FID or as universal perceptual scores.

\input{tab/FLUX-Metrics}

\subsection{Main Results}

Table~\ref{table:FLUX-Metrics} summarizes the 200-prompt results. SoftCap provides the clearest advantage in the middle compute regime: it uses nearly the same FLOPs as SpeCa, while improving ImageReward from 0.967 to 0.981, CLIPScore from 27.652 to 27.815, and LPIPS-Full from 0.518 to 0.498. In the higher-FLOPs regime, SoftCap reduces FLOPs relative to SpeCa and obtains the best ImageReward among accelerated methods, although SpeCa remains slightly stronger on several Full-referenced fidelity metrics and TaylorSeer gives the highest CLIPScore. In the lower-FLOPs regime, SoftCap spends slightly more FLOPs than the most aggressive TaylorSeer and FORA settings, but those settings show substantial fidelity degradation. Under this quality-efficiency tradeoff, SoftCap preserves markedly stronger ImageReward and Full-referenced fidelity.



\subsection{Positive-Increment Switch}

Table~\ref{table:delta-switch} isolates the optional positive-increment term in the Trajectory Drift Observer. The switch is not a separate method; it only changes how the observer responds to upward changes in the base risk score. At the middle and lower compute settings, enabling the switch improves several Full-referenced fidelity metrics and ImageReward, with small or moderate FLOPs changes. At the higher-FLOPs setting, the base score is preferable on most semantic and efficiency metrics. We therefore report the base TDO score in the main comparison and treat the positive increment as an optional calibration knob for settings where stronger response to abrupt risk increases is desired.

\input{tab/DeltaSwitch}

\subsection{Soft-Ceiling Diagnostic}

The main table evaluates quality and efficiency, but it does not by itself show whether $N_{\mathrm{cap}}$ behaves as a soft ceiling. We therefore run a five-prompt target-sweep diagnostic in which the SoftCap budget ceiling is varied while the rest of the policy is kept fixed. This diagnostic is smaller than the main benchmark and should be read as mechanism evidence rather than as a standalone quality benchmark.

\input{tab/SoftCeiling-Sweep}

Table~\ref{table:softceiling-sweep} shows the result. Increasing $N_{\mathrm{cap}}$ from 8 to 40 raises the allowed compute envelope by a factor of five, but the realized Full count increases only from 10 to 13. This is the intended behavior. The controller changes the threshold, but Full evaluations still require trajectory-risk crossings. When the observer does not identify enough high-risk steps, the run can remain below the ceiling instead of spending extra computation simply because the ceiling is available. In this diagnostic, the risk crossings are converted into Full evaluations, so the plateau is not caused by a downstream eligibility failure.

Additional platform diagnostics support this interpretation. Risk crossings are converted into Full evaluations in the checked runs, and neither forced Full paths nor max-skip guards explain the saturation pattern. The plateau therefore appears to come from limited natural risk crossings, not from a downstream eligibility bottleneck. This distinction matters: SoftCap is designed to regulate the strictness of the risk gate, not to force the realized compute to hit a prescribed target.








\subsection{Visualization}
As shown in Figure~\ref{fig:sample}, our method achieves a 6.05× inference speedup, outperforming Fora and Taylorseer's 5× and approaching Speca's 6.16×, while excelling in detail restoration and generation quality. It accurately renders the orange cat Otto von Garfield holding a fork to eat lasagna with his Prussian Pickelhaube helmet, correctly generates four dogs on the street, and perfectly reproduces the New York skyline with "Diffusion" fireworks and the correctly spelled "Google Brain Toronto" sign, eliminating the prevalent spelling errors in baselines. Our method produces clear, coherent images with natural subject-background integration and superior semantic consistency, achieving the optimal balance between inference speed and generation accuracy.
% \begin{table*}[t]
% \centering
% \caption{\textbf{Quantitative comparison in text-to-image generation on FLUX.} SoftCap rows are highlighted. FLOPs are averaged per image in tera-FLOPs. Full-referenced metrics compare accelerated outputs with Full FLUX outputs generated from the same prompts.}
% \vspace{-2mm}
% \setlength{\tabcolsep}{4pt}
% \small
% \resizebox{0.98\textwidth}{!}{
% \begin{tabular}{l | c | c c | c c | c c c c}
% \toprule
% {\bf Method} & {\bf Efficient} & \multicolumn{2}{c|}{\bf Acceleration} & \multicolumn{2}{c|}{\bf Semantic Quality} & \multicolumn{4}{c}{\bf Full-Referenced Fidelity} \\
% \cline{3-10}
% {\bf FLUX.1-dev} & {\bf Attention} & {\bf FLOPs(T) $\downarrow$} & {\bf Speed $\uparrow$} & {\bf ImageReward $\uparrow$} & {\bf CLIPScore $\uparrow$} & {\bf FID $\downarrow$} & {\bf PSNR $\uparrow$} & {\bf SSIM $\uparrow$} & {\bf LPIPS $\downarrow$} \\
% \midrule
% {\bf Full FLUX, 50 steps} & \ding{52} & 3719.50 & 1.00$\times$ & 0.996 & 27.765 & 0.00 & $\infty$ & 1.000 & 0.000 \\
% \midrule
% SpeCa & \ding{52} & 966.71 & 3.85$\times$ & 1.000 & 27.753 & 87.11 & 13.715 & 0.536 & 0.493 \\
% TaylorSeer & \ding{52} & 967.07 & 3.85$\times$ & 0.941 & 27.808 & 101.16 & 12.884 & 0.509 & 0.541 \\
% ClusCa & \ding{52} & 956.21 & 3.89$\times$ & 0.986 & 27.640 & 74.89 & 15.928 & 0.594 & 0.411 \\
% \rowcolor{gray!18}
% \textbf{SoftCap} & \ding{52} & \textbf{917.39} & \textbf{4.05$\times$} & \textbf{0.995} & \textbf{27.710} & \textbf{87.74} & \textbf{13.707} & \textbf{0.536} & \textbf{0.493} \\
% \midrule
% SpeCa & \ding{52} & 725.58 & 5.13$\times$ & 0.967 & 27.652 & 90.52 & 13.311 & 0.513 & 0.518 \\
% TaylorSeer & \ding{52} & 743.90 & 5.00$\times$ & 0.897 & 27.673 & 107.86 & 12.694 & 0.488 & 0.561 \\
% ClusCa & \ding{52} & 778.00 & 4.78$\times$ & 0.961 & 27.526 & 82.44 & 14.281 & 0.551 & 0.464 \\
% \rowcolor{gray!18}
% \textbf{SoftCap} & \ding{52} & \textbf{724.62} & \textbf{5.13$\times$} & \textbf{0.981} & \textbf{27.815} & \textbf{89.29} & \textbf{13.677} & \textbf{0.530} & \textbf{0.498} \\
% \midrule
% SpeCa & \ding{52} & 603.44 & 6.16$\times$ & 0.964 & 28.124 & 96.82 & 12.919 & 0.483 & 0.550 \\
% TaylorSeer & \ding{52} & 669.51 & 5.56$\times$ & 0.805 & 27.355 & 110.25 & 12.608 & 0.491 & 0.567 \\
% ClusCa & \ding{52} & 659.19 & 5.64$\times$ & 0.954 & 27.635 & 87.68 & 13.658 & 0.521 & 0.497 \\
% \rowcolor{gray!18}
% \textbf{SoftCap} & \ding{52} & \textbf{656.80} & \textbf{5.66$\times$} & \textbf{0.988} & \textbf{27.825} & \textbf{89.72} & \textbf{13.647} & \textbf{0.523} & \textbf{0.503} \\
% \bottomrule
% \end{tabular}}
% \label{table:FLUX-Metrics}
% \footnotesize
% %\begin{itemize}
% %\item \textcolor{gray!70}{Gray rows}: SoftCap. Speedups are computed against the Full FLUX baseline with 3719.50 TFLOPs.
% %\item FID, PSNR, SSIM, and LPIPS are Full-referenced internal metrics, not dataset-FID or real-data distribution scores.
% %\end{itemize}
% \vspace{-4mm}
% \end{table*}

\begin{table*}[t]
\centering
\caption{\textbf{Effect of the optional positive-increment switch in TDO.} ``Off'' uses the base weighted drift score. ``On'' adds the positive-increment term, which amplifies upward changes in the observer risk.}
\vspace{-2mm}
\setlength{\tabcolsep}{4pt}
\small
\resizebox{0.88\textwidth}{!}{
\begin{tabular}{l c | c c | c c | c c c c}
\toprule
{\bf Setting} & {\bf Increment} & \multicolumn{2}{c|}{\bf Acceleration} & \multicolumn{2}{c|}{\bf Semantic Quality} & \multicolumn{4}{c}{\bf Full-Referenced Fidelity} \\
\cline{3-10}
 & {\bf Switch} & {\bf FLOPs(T) $\downarrow$} & {\bf FLOPs Speed $\uparrow$} & {\bf ImageReward $\uparrow$} & {\bf CLIPScore $\uparrow$} & {\bf FID $\downarrow$} & {\bf PSNR $\uparrow$} & {\bf SSIM $\uparrow$} & {\bf LPIPS $\downarrow$} \\
\midrule
SoftCap & Off & \textbf{917.39} & \textbf{4.05$\times$} & \textbf{0.995} & \textbf{27.710} & \textbf{87.74} & 13.707 & 0.536 & 0.493 \\
SoftCap & On & 920.37 & 4.04$\times$ & 0.979 & 27.686 & 87.85 & \textbf{13.717} & \textbf{0.536} & \textbf{0.493} \\
\midrule
SoftCap & Off & \textbf{724.62} & \textbf{5.13$\times$} & 0.981 & \textbf{27.815} & 89.29 & 13.677 & 0.530 & 0.498 \\
SoftCap & On & \textbf{724.62} & \textbf{5.13$\times$} & \textbf{0.982} & 27.731 & \textbf{88.64} & \textbf{13.687} & \textbf{0.530} & \textbf{0.497} \\
\midrule
SoftCap & Off & \textbf{614.55} & \textbf{6.05$\times$} & 0.987 & \textbf{27.879} & 89.82 & 13.598 & 0.517 & 0.508 \\
SoftCap & On & 627.05 & 5.93$\times$ & \textbf{0.989} & 27.877 & \textbf{89.53} & \textbf{13.611} & \textbf{0.520} & \textbf{0.506} \\
\bottomrule
\end{tabular}}
\label{table:delta-switch}
\vspace{-3mm}
\end{table*}

\begin{table}[t]
\centering
\small
\caption{SoftCap target-sweep diagnostic. The realized number of Full evaluations saturates even as the soft ceiling increases, supporting the interpretation of $N_{\mathrm{cap}}$ as a ceiling rather than a required spending target.}
\vspace{-2mm}
\begin{tabular}{r r r r}
\toprule
$N_{\mathrm{cap}}$ & Actual Full & Crossing Full & Warmup \\
\midrule
8 & 10 & 43 & 10 \\
12 & 11 & 47 & 10 \\
16 & 11 & 49 & 10 \\
20 & 12 & 49 & 10 \\
24 & 12 & 50 & 10 \\
28 & 12 & 52 & 10 \\
32 & 12 & 55 & 10 \\
40 & 13 & 57 & 10 \\
\bottomrule
\end{tabular}
\label{table:softceiling-sweep}
\end{table}

\section{Conclusion}

In this paper, We presented SoftCap, a training-free soft-budget control framework for cache-based DiT acceleration. Instead of treating Full/Cache scheduling as a fixed schedule or a manually selected threshold, SoftCap separates two decisions that are often conflated: whether the current cached trajectory appears risky, and how strict the Full-triggering gate should be under the realized compute state. The Trajectory Drift Observer estimates local cache risk from lightweight drift statistics, while the Soft-Budget PI Controller adjusts the dynamic threshold using a fixed reference budget profile and the realized number of Full evaluations.

Experiments on FLUX.1-dev show that SoftCap provides competitive quality-efficiency tradeoffs under the same evaluation protocol, with its clearest matched-compute gain in the middle FLOPs regime. Mechanism diagnostics further support the soft-ceiling interpretation: increasing the budget ceiling changes the decision threshold, but it does not require the run to execute additional Full evaluations when the observer does not find enough high-risk steps. This makes SoftCap a policy layer for regulating cache-based acceleration, rather than a replacement for cache engines themselves. Future work can extend this control view to additional cache mechanisms, denoising horizons, and multimodal generation backbones.

\section*{Limitations}
Although SoftCap demonstrates promising performance, there can sometimes be a substantial gap between the actual number of Full steps executed and the predefined target budget. In future work, we will explore adaptive reference accumulation curves to achieve more precise alignment with the computational budget, thereby maximizing generation quality under a given computational quota.