Solving Inverse Problems with FLAIR
Extracted source text · arXiv 2506.02680
\documentclass{article}
% if you need to pass options to natbib, use, e.g.:
% \PassOptionsToPackage{numbers, compress}{natbib}
% before loading neurips_2024
% ready for submission
\usepackage[preprint]{neurips_2025}
% to compile a preprint version, e.g., for submission to arXiv, add add the
% [preprint] option:
% \usepackage[preprint]{neurips_2024}
% to compile a camera-ready version, add the [final] option, e.g.:
% \usepackage[final]{neurips_2024}
% to avoid loading the natbib package, add option nonatbib:
% \usepackage[nonatbib]{neurips_2024}
\usepackage{soul}
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc} % use 8-bit T1 fonts
\usepackage{hyperref} % hyperlinks
\usepackage{url} % simple URL typesetting
\usepackage{booktabs} % professional-quality tables
\usepackage{amsfonts} % blackboard math symbols
\usepackage{nicefrac} % compact symbols for 1/2, etc.
\usepackage{microtype} % microtypography
\usepackage[table, dvipsnames]{xcolor}
%\usepackage{algorithm}
%\usepackage{algorithmic}
\usepackage{amsmath}
\usepackage[ruled,vlined]{algorithm2e}
\usepackage{booktabs} % for \toprule, \midrule, \bottomrule
\usepackage{siunitx} % for alignment of numbers
\usepackage{graphicx}
\usepackage{pifont} % for \cmark/\xmark
\usepackage{todonotes}
\usepackage{amsthm}
\newtheorem{proposition}{Proposition}
% define commands for convenience
\newcommand{\cmark}{\ding{51}}%
\newcommand{\xmark}{\ding{55}}%
\newcommand{\I}{\mathbbm{1}}
\newcommand{\C}{\mathbb{C}}
\newcommand{\Ic}{\mathcal{I}}
\newcommand{\Xc}{\mathcal{X}}
\newcommand{\Yc}{\mathcal{Y}}
\newcommand{\Zc}{\mathcal{Z}}
\newcommand{\SSIM}{\textrm{SSIM}}
\newcommand{\prob}{\mathrm{p}}
\newcommand{\Prob}{\mathrm{P}}
\DeclareMathOperator*{\argmin}{argmin}
\DeclareMathOperator*{\argmax}{argmax}
\newcommand{\dx}{\,\mathrm{d}}
\newcommand{\E}{\mathbb{E}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\Id}{\mathrm{Id}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\method}{FLAIR}
\SetKw{KwBy}{by}
\definecolor{myrowcolour}{rgb}{0.859,0.859,0.859}
\newcommand{\jan}[1]{\textcolor{blue}{\textbf{JEL:}{ #1}}}
\newcommand{\tr}{\operatorname{tr}}
\newcommand{\expnumber}[2]{{#1}\mathrm{e}{#2}}
\newcommand{\const}{C}
\DeclareMathOperator*{\Var}{Var}
\DeclareMathOperator*{\TV}{TV}
\DeclareMathOperator*{\TVH}{TV_{H}}
\DeclareMathOperator{\prox}{prox}
\renewcommand{\d}[1]{\ensuremath{\operatorname{d}\!{#1}}}
\newcommand{\bernt}[1]{\textcolor[rgb]{0.08, 0.38, 0.74}{\textbf{Bernt:} #1}}
\title{Solving Inverse Problems with FLAIR}
% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to LaTeX to determine where to break the
% lines. Using \AND forces a line break at that point. So, if LaTeX puts 3 of 4
% authors names on the first line, and the last on the second line, try using
% \AND instead of \And before the third author name.
\author{%
Julius Erbach$^{1,2}$ \quad
Dominik Narnhofer$^{1}$ \quad
Andreas Dombos$^{1}$ \\ % ← forces a new line
\textbf{Bernt Schiele}$^{2}$ \quad
\textbf{Jan Eric Lenssen}$^{2}$ \quad
\textbf{Konrad Schindler}$^{1}$ \\[0.5em]
$^{1}$ETH Zürich \qquad
$^{2}$Max Planck Institute for Informatics, Saarland Informatics Campus
}
\begin{document}
\maketitle
\begin{abstract}
%\bernt{detail: you seem to use the neurips24 style - there is a neurips25 style - not sure if there are any relevant differences but I would use the style from 25 - at least the footer line on each page currently states it is submitted to neurips24...}
Flow-based latent generative models such as Stable Diffusion 3 are able to generate images with remarkable quality, even enabling photorealistic text-to-image generation. Their impressive performance suggests that these models should also constitute powerful priors for inverse imaging problems, but that approach has not yet led to comparable fidelity.
%
There are several key obstacles: (i) the encoding into a lower-dimensional latent space makes the underlying (forward) mapping non-linear; (ii) the data likelihood term is usually intractable; and (iii) learned generative models struggle to recover rare, atypical data modes during inference.
%
We present \method{}, a novel training free variational framework that leverages flow-based generative models as a prior for inverse problems. To that end, we introduce a variational objective for flow matching that is agnostic to the type of degradation, and combine it with deterministic trajectory adjustments to recover atypical modes.
%
To enforce exact consistency with the observed data, we decouple the optimization of the data fidelity and regularization terms. Moreover, we introduce a time-dependent calibration scheme in which the strength of the regularization is modulated according to off-line accuracy estimates.
%
Results on standard imaging benchmarks demonstrate that \method{} consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.
\end{abstract}
\section{Introduction}
% \jan{I assume this is work in progress but I will mention it anyway: Intro should only have a smaller paragraph about the overall field and then focus on the delta between previous works and ours (what are we solving, how are we solving it, more like what is written in related work currently and less background). Reviewers want to directly understand what our contribution is.}
% General Intro to the field
Flow-based generative models are at the core of modern image generators like Stable Diffusion or FLUX~\cite{esser24icml}. Beyond image generation based on text prompts, these models have emerged as powerful data-driven priors for a whole range of visual computing tasks.
%
Their comprehensive representation of the visual world, learned from internet-scale training datasets, makes them an attractive alternative to traditional handcrafted image priors. Often, they can be used without any task-specific retraining.
While it is evident that a model capable of generating photorealistic images should be suitable as prior (a.k.a.\ regularizer) for inverse imaging problems, a practical implementation faces several challenges.
%
On the one hand, flow-based models normally operate in the lower-dimensional latent space of a variational autoencoder (VAE), which means that the forward operator (the relationship between the observed, degraded image and the desired, clean target image) is no longer linear.
%
On the other hand, the iterative nature of the generative process means that intermediate stages are corrupted with (time-dependent) random noise. Hence, one cannot explicitly evaluate their data likelihood, which renders the data term intractable.
%
Moreover, learned generative models tend to overly favor regions of the training distribution that have a high sample density. For test samples that fall in low-density regions, the prior will have a too strong tendency to pull towards outputs with higher a-priori likelihood, compromising fidelity to the input observations.
%\textcolor{red}{What about the other of "several" challenges? In the abstract there were three...}
Here, we propose \underline{f}low-based \underline{l}atent \underline{a}daptive \underline{i}nference with deterministic \underline{r}e-noising (\method{}), a novel training free variational framework explicitly tailored to integrate flow-based latent diffusion models into inverse problem-solving.
%
To the best of our knowledge, \method{} is the first scheme that combines latent generative modeling, flow matching and variational inference into a unified formulation for inverse problems.
%
Our main contributions are
%\textcolor{red}{I feel for variational+flow+latent, we might be in a position to claim "we are the first...". Right or not?}:
\begin{itemize}
\setlength\itemsep{.1em}
\item A novel variational objective for inverse problems with flow-matching priors.
\item Deterministic trajectory adjustments to robustly handle rare or atypical data scenarios.
\item Decoupled optimization of data and regularization terms, enabling hard data consistency.
\item A novel, time-dependent weighting scheme, calibrated via offline accuracy estimates, that adapts the regularization along the flow trajectory to match the changing reliability of the model's predictions, ensuring robust inference.
\end{itemize}
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{arxiv_figures/compressed_teaser3.pdf}
\vspace{-.5cm}
\caption{Starting from the adjoint based initialization, we alternate between (i) regularizer updates via a flow-matching loss that aligns the posterior velocity $u_t$ with the learned velocity field $v_\theta$, and (ii) hard data consistency steps that project the current estimate onto the measurement manifold.}
\label{fig:teaser}
\end{figure}
% \noindent\textbf{Inverse problems.}
% Many imaging tasks such as image restoration~\cite{}, medical imaging~\cite{} and remote sensing~\cite{} aim to recover a target signal $x\in\R^n$ from a distorted observation $y\in\R^m$.
% The observation is generated by applying a forward operator $A:\R^n \mapsto \R^m$ to the target signal, typically corrupted by additive Gaussian noise $\nu \in \R^m$.
% \begin{equation}
% y = \mathcal{A}x+\nu.
% \label{eq:inverse_problem}
% \end{equation}
% In most practical applications, the forward operator $\mathcal{A}$ is either non-invertible or severely ill-conditioned, making \eqref{eq:inverse_problem} generally ill-posed.
% %Variational methods are a well-established approach for addressing ill-posed inverse problems, where the restored output image is obtained by minimizing an energy functional
% Variational methods solve ill-posed inverse problems by minimizing an energy functional
% \begin{equation}
% \mathcal{E}(x,z)=\mathcal{D}(x,y)+\mathcal{R}(x).
% \label{eq:MAP}
% \end{equation}
% to recover the solution.
% % These methods can be interpreted probabilistically through Bayes’ theorem, which expresses the posterior distribution~$\Prob(x\vert z)$ of the reconstruction~$x$ given the observed data~$z$ as being proportional to the product $\Prob(z\vert x)\Prob(x)$.
% % In the negative log-domain, the data term $\mathcal{D}(x,z)$ corresponds to $-\log \Prob(z \vert x)$ and the regularizer $\mathcal{R}(x)$ to $-\log \Prob(x)$.
% Interpreted probabilistically through Bayes’ theorem, the posterior distribution~$\Prob(x\vert z)$ is being proportional to the product $\Prob(z\vert x)\Prob(x)$.
% In the negative log-domain, this yields the data term~$\mathcal{D}(x,z) = -\log \Prob(z \vert x)$ and the regularizer~$\mathcal{R}(x) = -\log \Prob(x)$.
% While handcrafted priors based on regularity assumptions like sparsity \cite{rof92, sa01, chpo11, da04, ho20_ip_review} have long been the standard, they have largely been replaced by deep learning-based methods in modern data-driven applications.
% %The following chapter provides a comprehensive overview of these approaches.
% \noindent\textbf{Flow based Priors.}
% Flow-matching-based models~\cite{lipman2023flow} learn a time-dependent vector field $v_{\theta}(x_t, t)$ that continuously transforms samples from a simple initial distribution $\prob_0(x)$ to a complex target data distribution $\prob_1(x)$.
% Formally, this transformation is described by solving the ordinary differential equation (ODE):
% \begin{equation}\label{eq:ode}
% \frac{\d{}}{\d t} \psi_t(x) = v_{\theta,t}(\psi_t(x)), \quad t \in [0,1],
% \end{equation}
% where $\psi_t(x)$ represents the trajectory of a sample evolving smoothly from an initial sample drawn at $t=0$ toward a target sample at $t=1$.
% %Since the integrated ODE path described by~\ref{eq:ode} naturally transforms samples from the simple initial distribution $\prob_1(x)$ to the complex target distribution $\prob_0(x)$, the learned flow-based model inherently captures the structure and statistics of the target data. Consequently, we can leverage this property and use the trained flow-matching model as a powerful, expressive prior in solving inverse problems.
% Since the integrated ODE path maps the simple distribution~$\prob\_0(x)$ to the complex target~$\prob\_1(x)$, the learned flow-based model captures the structure of the data and can therefore serve as a powerful prior for solving inverse problems.
% \\\\
% \noindent\textbf{Latent Diffusion Models.}
% To reduce the high computational cost of diffusion models at large image resolutions, latent diffusion models (LDMs)~\cite{rombach2022high} shift the generative process to a lower-dimensional latent space using a pretrained autoencoder with encoder $A:\R^n \mapsto \R^d$ and decoder $D:\R^d \mapsto \R^n$ with $d<<n$. However, leveraging these priors for inverse problems is challenging, as the non-linearity of the VAE disrupts the linear relationship between measurements and targt signal, leading to a non-linear forward operator.\\\\
% Our proposed approach tackles the challenging task of solving inverse problems by leveraging flow-based latent diffusion models within a variational framework.
\section{Related Work}
\noindent\textbf{Deep learning based priors.}
Deep learning–based methods typically follow one of two main approaches: they either directly learn an inverse mapping~\cite{zb18, ja08, le17, becker2023neural}, or aim to learn a suitable prior, either through non-generative approaches like unrolled optimization networks~\cite{ham18, koef21, aga18, narnhofer2021bayesian} or through generative models such as generative adversarial networks~\cite{na19,bor17,shah2018solving}, or diffusion~\cite{ho2020denoising_ddpm, song2020denoising_ddim, song2021scorebased} respectively flow based models~\cite{lipman2023flow,liu2022flow}.
The latter have demonstrated impressive performance in image generation tasks, sparking growing interest in leveraging them as priors for solving inverse problems, particularly through posterior sampling techniques.\\\\
\noindent\textbf{Posterior sampling.} Although incorporating the prior learned from a diffusion or flow-based model seems straightforward, problems arise due to the inherent time-dependent structure of diffusion models, which makes the likelihood term intractable~\cite{chung2023diffusion}.
A variety of approaches have been proposed for diffusion-based posterior sampling, including, enforcing the trajectory to stay on the respective noise manifold~\cite{chung2023diffusion, chung2022improving}, applying an SVD to run diffusion in spectral domain~\cite{kawar2021snips}, utilizing range-null space decomposition during the reverse diffusion process~\cite{kawar2021snips}, guidance by the pseudo-inverse of the forward operator~\cite{song2023pseudoinverse}.
%While many of the aforementioned methods perform well in pixel space, they are challenging to utilize in latent-space diffusion models, either due to non-linearity of the VAE or memory issues.
Many prior methods perform well in pixel space but are difficult to apply in latent diffusion models due to VAE non-linearity or memory constraints.
In order to circumvent this issue, the authors of ReSample~\cite{song2024solving} rely on enforcing hard data consistency through optimization and resampling during the reverse diffusion process.
PSLD~\cite{rout2023solving}, introduces additional objectives terms in order to ensure that all gradient updates point to the same optima in the latent space.
FlowChef~\cite{patel2024flowchef} incorporates guidance into the flow trajectory during inference, whereas FlowDPS~\cite{kim2025flowdps} separates the update step into two components: one for estimating the clean image and another for estimating the noise.
% Prolific Dreamer\cite{wang2023prolificdreamer} proposed Variational Score Distillation (VSD) to tackle this issue. A key factor in their success is that they learn the score of their noisy variational distribution to improve updates in low density regions. In order to do that their learned score network is conditioned on the text prompt. Equivalently, in image inverse problems the score of the noisy variational distribution gives bad update directions, because it is not directly conditioned on the measurement $y$. \jan{Here needs to be one sentence saying how your approach differs.}
In contrast \method{} follows another class of posterior sampling-based methods, which integrate diffusion priors with inverse problems by directly optimizing a variational objective that approximates the data posterior~\cite{mardani2024a}. This framework was recently extended by RSD~\cite{zilberstein2025repulsive}, which incorporates a repulsion mechanism to promote sample diversity and applied to latent diffusion models. A known issue with this type of optimization is mode collapse \cite{pooledreamfusion}, which leads to blurry results for these methods. Our method targets this problem by introducing a deterministic trajectory adjustment.
% Inspired by this line of work, we propose Flow-based Latent Adaptive Inference with deterministic Re-noising \method{}, a variational framework that integrates flow-based priors into inverse problem solving. Our key contributions include:
% \begin{itemize}
% \setlength\itemsep{.1em}
% \item A novel variational objective designed specifically for flow matching in inverse problems.
% \item Deterministic trajectory adjustment enabling the recovery of rare or atypical data.
% \item Decoupled optimization of data and regularization terms, to enforce hard data consistency.
% \item A time-dependent calibration scheme for the regularization weight, informed by offline estimated accuracies during the inference process.
% %\item A versatile framework that is compatible with state-of-the-art latent diffusion models
% \end{itemize}
% 1. Inverse Problems with deep learning priors
% 2. Inverse Problems with generative priors (GAN + Diffusion)
% 3. Test time guidance
\section{Background}
\subsection{Inverse problems}
In many imaging tasks, such as inpainting~\cite{bertalmio2000image}, super-resolution~\cite{park2003super} or tomographic reconstruction~\cite{sidky2008image}, one aims to recover a target signal $x\in\R^n$ from a distorted observation $y\in\R^m$.
%
The observation is regarded as the result of applying a forward operator $A:\R^n \mapsto \R^m$ to the target signal, corrupted by additive Gaussian noise $\nu \in \R^m$ with standard deviation $\sigma_\nu$.
\begin{equation}
y = \mathcal{A}x+\nu.
\label{eq:inverse_problem}
\end{equation}
In most practical applications, the forward operator $\mathcal{A}$ is either non-invertible or severely ill-conditioned, making \eqref{eq:inverse_problem} generally ill-posed.
%Variational methods are a well-established approach for addressing ill-posed inverse problems, where the restored output image is obtained by minimizing an energy functional
Variational methods solve ill-posed inverse problems by minimizing an energy functional
\begin{equation}
\mathcal{E}(x,z)=\mathcal{D}(x,y)+\mathcal{R}(x).
\label{eq:MAP}
\end{equation}
to recover the solution.
% These methods can be interpreted probabilistically through Bayes’ theorem, which expresses the posterior distribution~$\Prob(x\vert z)$ of the reconstruction~$x$ given the observed data~$z$ as being proportional to the product $\Prob(z\vert x)\Prob(x)$.
% In the negative log-domain, the data term $\mathcal{D}(x,z)$ corresponds to $-\log \Prob(z \vert x)$ and the regularizer $\mathcal{R}(x)$ to $-\log \Prob(x)$.
Interpreted probabilistically via Bayes’ theorem, the posterior distribution~$p(x\vert z)$ is proportional to the product $p(z\vert x)p(x)$.
In the negative log-domain, this yields the data term~$\mathcal{D}(x,z) = -\log p(z \vert x)$ and the regularizer~$\mathcal{R}(x) = -\log p(x)$.
Handcrafted priors based on regularity assumptions like sparsity \cite{rof92, sa01, chpo11, da04, ho20_ip_review} have long been the standard, but have largely been replaced by deep learning-based methods in modern data-driven schemes.
%The following chapter provides a comprehensive overview of these approaches.
\subsection{Flow based priors}
Models based on flow matching~\cite{lipman2023flow} learn a time-dependent vector field $v_{\theta}(x_t, t)$ that continuously transforms samples from a simple initial distribution $p_1(x)$ to a complex target data distribution $p_0(x)$.
Formally, this transformation is described by solving the ordinary differential equation (ODE):
\begin{equation}\label{eq:ode}
\frac{\d{}}{\d t} \psi_t(x) = v_{\theta,t}(\psi_t(x)), \quad t \in [0,1],
\end{equation}
where $\psi_t(x)$ represents the trajectory of a sample evolving smoothly from an initial sample drawn at $t=1$ toward a target sample at $t=0$.
%Since the integrated ODE path described by~\ref{eq:ode} naturally transforms samples from the simple initial distribution $\prob_1(x)$ to the complex target distribution $\prob_0(x)$, the learned flow-based model inherently captures the structure and statistics of the target data. Consequently, we can leverage this property and use the trained flow-matching model as a powerful, expressive prior in solving inverse problems.
Since the integrated ODE path maps the simple distribution~$p_1(x)$ to the complex target~$p_0(x)$, the learned flow-based model captures the structure of the data and can therefore serve as a powerful prior for solving inverse problems.
To make this approach tractable for high-resolution data, we adopt the latent diffusion model (LDM) framework~\cite{rombach2022high}, which shifts the generative process to a lower-dimensional latent space using a pretrained autoencoder with encoder $E:\R^n \mapsto \R^d$ and decoder $D:\R^d \mapsto \R^n$ with $d\ll n$.
However, applying such priors to inverse problems introduces challenges, as the non-linearity of the VAE disrupts the linear relationship between measurements and the target signal, resulting in a nonlinear forward operator.
\subsection{Variational flow sampling}
To solve inverse problems from a Bayesian perspective, we aim to sample from the posterior
\begin{equation}\label{eq:bayes_thm}
p(x_0|y) \propto p(y|x_0)p(x_0),
\end{equation}
where the likelihood is given by $p(y|x_0) = \mathcal{N}(\mathcal{A}x_0, \sigma^2\Id)$, and $p(x_0)$ represents the prior modeled by the flow-based generative model.
% The inverse problem formulation can be defined as follows:
% \begin{equation}
% y = A(x_0) + \epsilon, \quad \epsilon \sim \mathcal{N}(0, \sigma_\epsilon^2 I)
% \label{eq:inv_problems}
% \end{equation}
% We are interested in finding $x_0$ given the degraded measurement $y$. We assume that the forward operator $A$ is known. In our work, we are utilizing latent flow models. \\\\
%Recently, \cite{mardani2024variational} proposed to solve inverse problems in the pixel space using a variational approach, and concurrent work \cite{zilberstein2024repulsive} proposed to adapt it to latent diffusion models.
Inspired by previous work~\cite{mardani2024a, zilberstein2025repulsive} we introduce a variational distribution $q(x_0|y) = \mathcal{N}(\mu_x, \sigma_x^2)$ to approximate the true posterior $p(x_0|y)$, by minimizing their Kullback–Leibler divergence:
% Since a single Gaussian cannot capture a multi modal posterior, we simplify to a deterministic approximation, setting $\sigma_x^2=0$.
% Equivalently, this corresponds to a single-particle approximation in the sense of Stein variational methods~\cite{liu2016stein}.
%To fit a unimodal Gaussian distribution to a complex multi modal posterior distribution is suboptimal and we decide to set the variance $\sigma_x^2=0$ or equivalently do a particle approximation \cite{liu2016stein} and set the number of particles to 1.:
\begin{equation}
q(x_0|y) \in \arg\min_{q(x_0|y)} \text{KL}(q(x_0|y) \| p(x_0|y)).
\label{eq:kl_minimization}
\end{equation}
Rewriting the KL divergence by means of the variational lower bound leads to:
\begin{equation}
KL(q(x_0|y) \| p(x_0|y)) = -\underbrace{\mathbb{E}_{q(x_0|y)}[\log p(y|x_0)]}_{\text{$\mathcal{D}(x,y)$}} + \underbrace{KL(q(x_0|y) \| p(x_0))}_{\text{$\mathcal{R}(x)$}} + \underbrace{\log p(y)}_{\text{const}}.
\label{eq:example_label}
\end{equation}
Since a single Gaussian cannot capture a multi-modal posterior, we simplify to a deterministic approximation, setting $\sigma_x^2=0$.
Equivalently, this corresponds to a single-particle approximation in the sense of Stein variational methods~\cite{liu2016stein}. As shown in~\cite{song2021maximum}, rewriting~\autoref{eq:example_label} under this approximation and extending it to the time-dependent noisy posterior yields:
\begin{equation}
\arg\min_{q(x_0|y)}
\underbrace{
\mathbb{E}_{q(x_0|y)} \left[ \frac{\|y - f(\mu_x)\|^2}{2\sigma_\nu^2} \right]
}_{\text{$\mathcal{D}(x,y)$}}
+
\underbrace{
\int_0^T \omega(t)~ \mathbb{E}_{q(x_t|y)} \left[ \left\| \nabla_x \log q(x_t|y) - \nabla_x \log p(x_t) \right\|^2 \right] dt
}_{\text{$\mathcal{R}(x)$}}
\label{eq:combined_}
\end{equation}
The first term in \autoref{eq:combined_} describes the data term $\mathcal{D}(x,y)$ and the second the regularizer $\mathcal{R}(x)$, where the integral ensures optimization over the entire diffusion trajectory.
Notably, the latter constitutes a weighted score-matching objective, where $\nabla_x \log p(x_t)$ represents the score function \cite{song2021scorebased}, which may be extracted from a pretrained diffusion or flow model.
The score of the noisy variational distribution depends on the forward diffusion process and can be computed analytically.
% TODO define beta and say that the objective requires backprop through denoiser
Note that for $ \omega(t) = \beta(t)/2$ the weighted score-matching loss recovers the gradient of the diffusion model’s evidence lower bound, so that optimizing it yields the maximum likelihood estimate of the data distribution~\cite{song2021maximum}.
However, optimizing \autoref{eq:combined_} is costly, as it requires computing the gradient through the flow model.
As shown in~\cite{wang2023prolificdreamer} this can be circumvented by reformulating the regularizer in terms of the Wasserstein gradient flow:
% For $ \omega(t) = \frac{\beta(t)}{2} $ as choice of weighting term in \autoref{eq:combined_loss} the regularizer corresponds to a maximum likelihood estimation \cite{song2021maximum}.
%The regularization part of optimization \autoref{eq:combined_loss} can be solved with Wasserstein gradient flow as shown in \cite{wang2023prolificdreamer}:
\begin{equation}
\nabla_{\mu_x} \mathcal{R}(x) =
\mathbb{E}_{q(x_t|y)} \left[ \omega(t) (\underbrace{\nabla_x \log q(x_t|y)}_{\text{score of noisy variational distribution}} - \underbrace{\nabla_x \log p(x_t)}_{\text{score of noisy prior distribution}}) \right]
\label{eq:gradient_step_reg}
\end{equation}
Note that optimizing only the regularization term, without the data term, at test time is equivalent to the objective of Score Distillation Sampling (SDS) \cite{pooledreamfusion}.
\section{Method}
\label{sec:method}
% \jan{In terms of structure, I would turn this section around: (1) Start with an overview, Algorithm 1, and an intuitive explanation. After the overview it does not hurt to again explain what is novel wrt previous works. Then zoom in closer by first introducing the main objective (Eq. 17), e.g. in a proposition, and have the derivation after (or in appendix, depending on space). Then, discuss individual aspects (calibration, trajectory, consistency) last (but mention them already in the overview).}
% \noindent\textbf{Flow Formulation.}
% The variational formulation in \autoref{eq:combined_loss} is formulated for the score, which can be reformulated into a denoising or $\epsilon_\theta$ parametrization \cite{song2021scorebased, mardani2024a}.
% However, we are interested in a variational objective that depends on the velocity field $v_\theta(x_t, t)$, which characterizes the probabilistic trajectory connecting noise and data distributions.
% %Different methods proposed ways to compute the score given the velocity field \cite{liao2022maptr, sauer2024fasthighresolutionimagesynthesis, singh2024stochastic}.
% We take inspiration from prior methods that derive the score function based on the velocity field~\cite{liao2022maptr, sauer2024fasthighresolutionimagesynthesis, singh2024stochastic}, and build our framework upon this foundation. \jan{I would try to avoid these types of statements. They sound incremental. Instead, mention relations to previous works after you introduced your method.}
% The linear conditional flow and it's corresponding velocity are defined as:
% \begin{equation}
% \label{eq:flow1}
% \psi_t(x_0 \mid \epsilon)
% = (1 - t)\,x_0 + t\,\epsilon,
% \quad
% \epsilon \sim \mathcal{N}(0, I).
% \end{equation}
% \begin{equation}
% \label{eq:flow2}
% u_t(x_t|\epsilon) = \frac{\d{\psi_t}}{\d t}(\psi_t^{-1}(x_t|\epsilon)|\epsilon)
% \end{equation}
% The score of the noisy variational distribution can be analytically computed with:
% \begin{equation}
% \label{eq:q(x_t)}
% q(x_t|y) = \mathcal{N}((1-t)\mu_x, t^2\,I) \quad
% \end{equation}
% \begin{equation}
% \label{eq:score_q(x_t)}
% \nabla_x \log q(x_t|y) = -\frac{\epsilon}{t}
% \end{equation}
% We compute $\frac{\d{\psi_t}}{\d t}(x_0|\epsilon)=-x_0 +\epsilon$ and $\psi_t^{-1}(x_t|\epsilon)$ and insert it into \autoref{eq:flow2}:
% \begin{equation}
% \label{eq:eps_to_u}
% u_t(x_t|\epsilon) = \frac{\epsilon - x_t}{1-t}.
% \end{equation}
% Solving \autoref{eq:eps_to_u} for $\epsilon$ and inserting in \autoref{eq:score_q(x_t)} gives:
% \begin{equation}
% \label{eq:score_to_u}
% \nabla_x \log q(x_t|y) = -\frac{(1-t)u_t(x_t|\epsilon) + x_t}{t}
% \end{equation}
% For the learned velocity $v_\theta(x_t,t)$ a similar approximation holds, which we prove in the Appendix.
% \begin{equation}
% \label{eq:v_theta_to_eps}
% v_\theta(x_t, t) \approx \frac{-t\nabla_x \log p(x_t) - x_t}{1-t}
% \end{equation}
% We can therefore approximate the score of the noisy prior with our learned velocity field $v_\theta$
% \begin{equation}
% \nabla_x \log p(x_t) \approx -\frac{(1-t) v_\theta(x_t, t) + x_t}{t}
% \label{eq:v_to_score}
% \end{equation}
% and we see that for $\omega(t) = \frac{t}{1-t}$ we obtain the conditional flow matching objective for $\mathcal{R}(x)$. We therefore set $\omega(t) = \frac{t}{1-t}$
% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5
% and end up at our final objective:
% \begin{equation}
% \arg\min_{q(x_0|y)}
% \underbrace{
% \mathbb{E}_{q(x_0|y)} \left[ \frac{\|y - f(\mu_x)\|^2}{2\nu^2} \right]
% }_{\text{$\mathcal{D}(x,y)$}}
% +
% \underbrace{
% \int_0^T ~ \mathbb{E}_{q(x_t|y)} \left[ \left\|v_\theta(x_t,t) - u_t(x_t|\epsilon) \right\|^2 \right] dt
% }_{\text{$\mathcal{R}(x)$}}
% \label{eq:combined_loss_app_flow}
% \end{equation}
% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% Again, the gradient step then becomes:
% \begin{equation}
% \nabla_{\mu_x} \mathcal{R}(x) =
% \mathbb{E}_{q(x_t|y)} \left[ v_\theta(x_t,t) - u_t(x_t|\epsilon) \right]
% \label{eq:gradient_step_reg_weighted}
% \end{equation}
\noindent\textbf{Flow Formulation.}
The variational formulation in \autoref{eq:combined_} is formulated for the score, but can be reformulated into a denoising or $\epsilon_\theta$ parametrization \cite{song2021scorebased, mardani2024a}.
However, we are interested in a variational objective that depends on the velocity field $v_\theta(x_t, t)$, which characterizes the probabilistic trajectory that connects the noise and data distributions. %\jan{Maybe add 1-2 sentences here, what the specific goal is of what comes, e.g. defining the scores from Eq. 8.}
\begin{proposition}
We propose to replace the score-based regularizer in the standard variational objective with a flow matching formulation, resulting in the following objective function:
\label{prop:propostion1}
\begin{equation}
\arg\min_{q(x_0|y)}
\underbrace{
\mathbb{E}_{q(x_0|y)} \left[ \frac{\|y - f(\mu_x)\|^2}{2\sigma_\nu^2} \right]
}_{\text{$\mathcal{D}(x,y)$}}
+
\underbrace{
\int_0^T \lambda_{\mathcal{R}}(t)~ \mathbb{E}_{q(x_t|y)} \left[ \left\|v_\theta(x_t,t) - u_t(x_t|\epsilon) \right\|^2 \right] dt
}_{\text{$\mathcal{R}(x)$}}
\label{eq:combined_loss_app_flow}
\end{equation}
\end{proposition}
The flow-matching term that defines the regularizer arises by re-parameterizing the variational distribution to $q(x_t|y) = \mathcal{N}((1-t)\mu_x, t^2\,I)$.
This corresponds to sampling via the deterministic map $\psi_t(x_0 \mid \epsilon)= (1 - t)\,x_0 + t\,\epsilon$, with $\epsilon \sim \mathcal{N}(0, I)$.
By re-formulating the score in terms of the target velocity field $u_t$, we get:
\begin{equation}
\label{eq:score_to_u}
\nabla_x \log q(x_t|y) = -\frac{(1-t)u_t(x_t|\epsilon) + x_t}{t}
\end{equation}
For the learned velocity $v_\theta(x_t,t)$ a similar approximation holds -- for a full derivation please see the supplementary material~\autoref{sec:derivation_flow_adjusted_ap}.
\begin{equation}
\label{eq:v_theta_to_eps}
v_\theta(x_t, t) \approx \frac{-t\nabla_x \log p(x_t) - x_t}{1-t}
\end{equation}
We can therefore approximate the score of the noisy prior with our learned velocity field $v_\theta$
\begin{equation}
\nabla_x \log p(x_t) \approx -\frac{(1-t) v_\theta(x_t, t) + x_t}{t}.
\label{eq:v_to_score}
\end{equation}
\noindent\textbf{Hard Data Consistency.}
Existing variational posterior sampling approaches~\cite{mardani2024a,zilberstein2025repulsive} impose soft constraints on the data fidelity term $\mathcal{D}(x, y)$.
In contrast, recent work~\cite{song2024solving} has demonstrated that, when sampling from latent diffusion models, enforcing hard data consistency generally leads to better reconstructions with improved visual fidelity.
Our method shares this motivation, but differs in that we optimize over a variational distribution, i.e., we compute $\min ~\mathbb{E}_{q(x_0|y)}[-\log p(y|x_0)]$.
An additional advantage of this variational setup is that it allows us to initialize the optimization variable with an adjoint based initialization $\mu_x=E(A^\top y)$, with $E$ being the encoder of the VAE.
\noindent\textbf{Accuracy Calibration.}
As our framework evaluates the trajectory at each timestep, we aim to weight the regularizer’s contribution according to its reliability.
It has been shown that the difficulty of the prediction task depends on the network parameterization, as well as on the specific timestep $t$~\cite{Karras2022edm}.
%
Since the regularization term $\mathcal{R}(x)$ in our approach is equivalent to the training objective of the pre-trained flow model, we can easily weight it by the expected model error, which we calibrate on a small set of images.
%
Specifically, we sample $N$ calibration images and compute the conditional flow matching objective for 100 linearly spaced time steps between 0 and 1, then average the error over all images to obtain the expected model error at each time step.
%We observe large errors for $t<0.2$ and therefore exclude those from the optimization:
%
Different functions of the model error can be chosen as weight for the regularizer. We choose:
\begin{equation}
\label{eq:weight}
\lambda_{\mathcal{R}}(t) = \frac{1}{N} \left( \sum_{i=1}^N \left\| v_\theta(x^{(i)}_{t}, t) - u_t(x^{(i)}_{t} \mid \epsilon) \right\|^2 \right)^{-1}
\end{equation}
and set $\lambda_{\mathcal{R}}(t)=0$ for all $t<0.2 $, since the accuracy of SD3 is heavily degraded for low noise levels.
\noindent\textbf{Deterministic Trajectory Adjustment.}
Score distillation sampling relies on the assumption that $x_t=(1-t)\mu_x + t\epsilon$ lies in a region of the learned prior that has reasonably high support/density. In practice, this is not always the case. When not tightly conditioned (usually with extensive text prompts), even the best available diffusion models assign low density to many plausible regions of the latent space, leading to bad gradient steps. Therefore, we increase the probability of $p(x_t)$ by additionally conditioning $x_t$ on the estimated "end-point" $\mu$.
%Formally, we construct the distribution $q(x_t \mid y)$ by applying the reverse-time flow trajectory starting from the estimated $x_0$.
\begin{proposition}
We introduce a reparameterized variational distribution with a mean that linearly interpolates between the posterior mean~$\mu_x$ and a model-guided ~$\hat{x}_1$:
\begin{equation}
q(x_t \mid y) = \mathcal{N}\left((1 - t)\mu_x + t\alpha \hat{x}_1,\; t^2(1 - \alpha^2) I\right),
\label{eq:biased_variational}
\end{equation}
where $\hat{x}_1 = x_{t + \delta t} + (1 - t - \delta t)v_\theta(x_{t + \delta t}, t + \delta t)$ is a single-step velocity-based predictor, and $\alpha \in [0,1]$ controls the trade-off between deterministic guidance and random noise.
%
This reparameterization induces the following reference velocity field:
\begin{equation}
u_t(x_t \mid \epsilon) = \frac{\alpha \hat{x}_1 + \sqrt{1 - \alpha^2} \epsilon - x_t}{1 - t}.
\label{eq:biased_velocity}
\end{equation}
\end{proposition}
Intuitively, changing the formulation in this manner ensures that the model relocates the sample to its expected position on the learned manifold, rather than injecting arbitrary noise, which could drive it in a direction that has high prior likelihood but is not consistent with the observation.
%
To further encourage exploration and avoid collapsing onto the trajectory of the adjoint measurement, we inject an additional, stochastic component $\epsilon$ during this process.
A full derivation can be found in the supplementary material,~\autoref{sec:derivation_flow_adjusted_ap}.
\subsection{Algorithm}
The following pseudo-code summarizes our method, integrating all the components discussed above.
%We adapt the standard scheme~\cite{mardani2024a, zilberstein2025repulsive} of linearly traversing time from $t = 1$ to $t = 0$ by stopping at $t=0.2$ as explained in \autoref{sec:method}.
We adapt the standard scheme~\cite{mardani2024a, zilberstein2025repulsive} of linearly traversing time in a descending manner and stop at $t=0.2$ as explained in \autoref{sec:method}.
We choose $\alpha=1-t$. Gradient updates to enforce hard data consistency are performed using stocastic gradient descent. For further implementation details and ablations, see~\autoref{sec:ap_imlpementation_details}
\begin{algorithm}[H]
\small % \footnotesize, \scriptsize … if you prefer
\caption{The \method{} solver for inverse imaging problems}\label{alg:inverse_problem}
\DontPrintSemicolon % hide the printed ; but keep the line breaks
\SetKwComment{C}{\hfill$\triangleright$~}{} % right-aligned comments
\KwIn{$\mu_x = A^\top(y)$, $\lambda_R$, $\alpha$, $y$, $A$, $v_\theta$}
\KwOut{$\mu_x$}
$\hat\epsilon \sim \mathcal N(0,I)$; \C*[r]{initial noise sample}
\For{$t\gets 1$ \KwTo $0$ \KwBy $-\Delta t$}{%
$x_t \gets (1-t)\,\mu_x + t\,\hat\epsilon$; \C*[r]{sample noisy latent}
$u_t(x_t\mid\hat\epsilon) \gets \dfrac{\hat\epsilon - x_t}{1-t}$;
$\nabla_{\mu_x} R \gets v_\theta(x_t,t) - u_t(x_t\mid\hat\epsilon)$;
$\mu_x \gets \mu_x - \lambda_R\,\nabla_{\mu_x} R$; \C*[r]{update w.r.t.\ regularizer}
$\mu_x \gets \arg\min_{\mu_x} \| y - A(\mu_x) \|^2$; \C*[r]{hard data consistency}
$\epsilon \sim \mathcal N(0,I)$;
$\hat x_1 \gets x_t + (1-t)\,v_\theta(x_t,t)$; \C*[r]{predict deterministic noise}
$\hat\epsilon \gets \alpha\,\hat x_1 + \sqrt{1-\alpha^2}\,\epsilon$; \C*[r]{update noise estimate}
}
\end{algorithm}
\section{Experiments}
We evaluate the performance of \method{} on a variety of inverse imaging tasks and compare against several baselines, using the SD3 backbone without any fine-tuning.
%
We used several metrics including SSIM~\cite{wang2004image}, LPIPS~\cite{zhang2018unreasonable} and patchwise FID~\cite{heusel2017gans} (pFID) to comprehensively assess the perceptual and quantitative quality of the reconstructions.
FID is computed using InceptionV3 features on patches of 256x256 resolution.
All experiments were performed on a single NVidia RTX 4090 GPU with 24GB of VRAM.
%
For completeness we also show PSNR values, but point out that the metric is not well suited for our purposes: PSNR favors the posterior mean, while the goal of the variational approach is to sample from the posterior distribution. Accordingly, PSNR is known to prefer over-smoothed, blurry outputs over sharp ones with even the slightest misalignment of high frequency content~\cite{blau2018perception}.
\subsection{Setup}
\noindent\textbf{Datasets.} We utilize two high-resolution image datasets: FFHQ~\cite{karras2019style} and DIV2K~\cite{Agustsson_2017_CVPR_Workshops}. FFHQ consists of 70k diverse face images at 1024×1024 resolution of which we take the first 1000 samples. It is covering variations in age, pose, lighting, and ethnicity.
DIV2K contains 800 high-quality images in 2K resolution that span a range of natural scenes with varied textures and structures.
%To simulate each inverse task, we generate degraded observations using standard corruptions—downsampling for 8× and 12× super-resolution, Gaussian blur for motion deblurring, random masks for inpainting.
\noindent\textbf{Baselines.}
Our method is benchmarked against several recent inverse imaging solvers based on posterior sampling.
%
Specifically, we compare to ReSample~\cite{song2024solving}, FlowDPS~\cite{kim2025flowdps}, FlowChef~\cite{patel2024flowchef}, and RSD~\cite{zilberstein2025repulsive}. The latter is used without repulsive term as it delivers better results.
%
To ensure a fair and meaningful comparison, all methods are evaluated with the same number of function evaluations.
\noindent\textbf{Problem Setting.} We run and evaluate all methods at a fixed output resolution of 768$\times$768 pixels. For single image super-resolution, we consider scaling factors of $8\times$ and $12\times$.
The corresponding low-resolution inputs are generated by bicubic downsampling. % with anti-aliasing.
Motion blur is simulated with a blur kernel of size 61.
For box inpainting, we mask large, continuous rectangles that cover approximately one third of the observation.
%
All synthesized observations are corrupted with additive Gaussian noise, with standard deviation $\sigma_\nu$ of 0.5\%.
% %
% \noindent\textbf{Text Prompts.}
For inference on the FFHQ dataset, we use a predefined text prompt of the form \textit{"A high quality photo of a face"}, and for DIV2k \textit{"A high quality photo of"} concatenated with an image-specific description retrieved by applying DAPE~\cite{Wu_2024_CVPR} to the observation.
\subsection{Experimental Results}
\label{sec:results}
\noindent\textbf{Inverse Problems.}
Our experiments clearly demonstrate that FLAIR outperforms existing flow-based approaches in terms of all perceptual metrics, see~\autoref{tab:restoration_results}. %across multiple inverse imaging tasks, as clearly shown in~\autoref{fig:results}
In the case of image inpainting, our method produces high-quality reconstructions that fully leverage the power of the generative model and blend naturally into the surrounding context, avoiding degradations and artifacts that we observe in the baselines.
%
In particular, FlowDPS tends to produce implausible textures in the inpainted regions, while FlowChef regularly fails to generate semantically consistent content at all.
For single-image super-resolution, \method{} consistently delivers the most perceptually convincing and realistic outputs.
%
Notably, the FID scores remain low for both $\times 8$ and $\times 12$ magnification, indicating an effective usage of the generative prior to overcome the increasing ill-posedness. Again, FlowDPS suffers from blur and low texture quality, whereas FlowChef tends to lose semantic coherence.
In motion deblurring, \method{} also restores sharper and semantically more credible content than competing approaches, which often suffer from residual blur or inconsistent details.
%
The boost in reconstruction quality is quantitatively reflected by all metrics, confirming that FLAIR reconstructs images with high fidelity.
%
For further qualitative examples, please see~\autoref{sec:additional_results}.
\begin{figure}
\centering
\includegraphics[width=\linewidth]{arxiv_figures/experim_c.pdf}
\vspace{-.5cm}
\caption{\textbf{Qualitative comparison}.
\method{} produces posterior samples of high perceptual quality while maintaining high data likelihood. Best viewed zoomed in.}
\label{fig:results}
\end{figure}
%\input{figures/figure2}
\begingroup
\setlength{\tabcolsep}{3pt}
\begin{table*}[htbp]
\centering
% \small
\caption{\textbf{Quantitative results} with 50~NFE and $\sigma_{\nu} = 0.01$.}
\vspace{0.5em}
\label{tab:restoration_results}
%––––– global siunitx settings (nothing here forces extra rounding) –––––
\sisetup{
detect-all,
round-mode = places,
table-align-text-post = false
}
%––––– table –––––
\resizebox{\linewidth}{!}{%
\begin{tabular}{
@{}l
% 16 numeric columns: (LPIPS) (FID) (SSIM) (PSNR) × 4 task blocks
S[table-format=1.3, round-precision=3] % 2 int + 2 dec – LPIPS
S[table-format=2.1, round-precision=1] % 1 int + 3 dec – FID
S[table-format=1.3, round-precision=3] % 1 int + 3 dec – SSIM
S[table-format=2.2, round-precision=2]| % 2 int + 2 dec – PSNR
S[table-format=1.3, round-precision=3] % 2 int + 2 dec – LPIPS
S[table-format=2.1, round-precision=1] % 1 int + 3 dec – FID
S[table-format=1.3, round-precision=3] % 1 int + 3 dec – SSIM
S[table-format=2.2, round-precision=2]| % 2 int + 2 dec – PSNR
S[table-format=1.3, round-precision=3] % 2 int + 2 dec – LPIPS
S[table-format=2.1, round-precision=1] % 1 int + 3 dec – FID
S[table-format=1.3, round-precision=3] % 1 int + 3 dec – SSIM
S[table-format=2.2, round-precision=2]| % 2 int + 2 dec – PSNR
S[table-format=1.3, round-precision=3] % 2 int + 2 dec – LPIPS
S[table-format=2.1, round-precision=1] % 1 int + 3 dec – FID
S[table-format=1.3, round-precision=3] % 1 int + 3 dec – SSIM
S[table-format=2.2, round-precision=2]| % 2 int + 2 dec – PSNR
@{}
}
\toprule
& \multicolumn{4}{c|}{SR $\times$8}
& \multicolumn{4}{c|}{SR $\times$12}
& \multicolumn{4}{c|}{Motion Deblurring}
& \multicolumn{4}{c}{Inpainting}\\
%\cmidrule(lr){2-5}\cmidrule(lr){6-9}
%\cmidrule(lr){10-13}\cmidrule(lr){14-17}
%\specialrule{1.pt}{0pt}{0pt}
\midrule
Method
& {\small LPIPS$\downarrow$} & {\small FID$\downarrow$} & {\small SSIM$\uparrow$} & {PSNR$\uparrow$}
& {\small LPIPS$\downarrow$} & {\small FID$\downarrow$} & {\small SSIM$\uparrow$} & {PSNR$\uparrow$}
& {\small LPIPS$\downarrow$} & {\small FID$\downarrow$} & {\small SSIM$\uparrow$} & {PSNR$\uparrow$}
& {\small LPIPS$\downarrow$} & {\small FID$\downarrow$} & {\small SSIM$\uparrow$} & {PSNR$\uparrow$}\\
\toprule
\multicolumn{17}{c}{\textbf{FFHQ 768$\times$768}}\\
\midrule
ReSample
& 0.4003311097621918 & 55.58622741699219 & \bfseries0.8146893978118896 & 26.36746597290039
& 0.4735182523727417 & 80.26885986328125 & \bfseries 0.7855629920959473 & 25.469425201416016
& 0.4574111998081207 & 82.89214324951172 & \bfseries 0.7877886891365051 & 25.446083068847656
& 0.36591094732284546 & 70.83206939697266 & \underline{0.827} & 21.82782745361328 \\
FlowDPS
& 0.37437254190444946 & 40.29948425292969 & 0.7563266158103943 & 29.238021850585938
& 0.41304320096969604 & 47.244354248046875 & 0.7412288188934326 & \underline{28.05}
& 0.4312819540500641 & 49.48529815673828 & 0.7316177487373352 & 27.63728141784668
& \underline{0.344} & \underline{38.3} & 0.7707209587097168 & 19.191648483276367 \\
RSD
& 0.39147356152534485 & 51.66742706298828 & 0.7760330438613892 & \bfseries29.690019607543945
& 0.46238717436790466 & 71.73455047607422 & \underline{0.743} & \bfseries 28.108963012695312
& 0.45782342553138733 & 77.31913757324219 & 0.7430248856544495 & \underline{27.67}
& 0.47828179597854614 & 73.29340362548828 & 0.7355696558952332 & \underline{21.97} \\
FlowChef
& \underline{0.341} & \underline{30.5} & 0.7600728869438171 & 28.424997329711914
& \underline{0.373} & \underline{41.0} & 0.7304214239120483 & 26.99635887145996
& \underline{0.406} & \underline{35.0} & 0.7161194682121277 & 25.80912971496582
& 0.3941635489463806 & 71.04671478271484 & 0.7801031470298767 & 18.184221267700195 \\
\rowcolor{myrowcolour}
\textbf{Ours}
& \bfseries 0.2132353037595749 & \bfseries13.26621150970459 & \underline{0.777} & \underline{29.54}
& \bfseries0.27056485414505005 & \bfseries16.201229095458984 & 0.7402094602584839 & 27.714468002319336
& \bfseries0.23615501821041107 & \bfseries10.732836723327637 & \underline{0.772} & \bfseries 29.61139488220215
& \bfseries0.18383462727069855 & \bfseries8.74171257019043 & \bfseries 0.8277943730354309 & \bfseries 23.69305419921875 \\
\toprule
\multicolumn{17}{c}{\textbf{DIV2K 768$\times$768}}\\
\toprule
%\specialrule{1.pt}{0pt}{0pt}
ReSample
& 0.533 & 54.99737167358398 & \bfseries 0.625 & 22.34
& 0.6428295969963074 & 88.0516586303711 & \bfseries0.5615006685256958 & 20.85
& 0.5558531880378723 & 79.6540298461914 & \underline{0.617} & 21.79242706298828
& \underline{0.285} & 51.85907745361328 & \underline{0.796} & 22.676401138305664 \\
FlowDPS
& \underline{0.476} & 54.160160064697266 & 0.5673845410346985 & 23.00572967529297
& 0.5472385287284851 & 67.42438507080078 & \underline{0.528} & \underline{21.79}
& 0.5584609508514404 & 60.432220458984375 & 0.5359930396080017 & 21.877470016479492
& 0.3276353180408478 & \underline{34.2} & 0.6919097900390625 & 21.714540481567383 \\
RSD
& 0.538948655128479 & 60.93032455444336 & 0.5913122296333313 & \bfseries 23.44598960876465
& 0.683576822280883 & 95.72624969482422 & 0.5232758522033691 & \bfseries21.964975357055664
& 0.6376681327819824 & 97.64688873291016 & 0.5511103272438049 & \underline{22.10}
& 0.46377840638160706 & 63.870880126953125 & 0.6775234341621399 & \underline{23.23} \\
FlowChef
& 0.49032270908355713 & \underline{48.6} & 0.5394924283027649 & 21.8426456451416
& \underline{0.525} & \underline{55.3} & 0.4921541213989258 & 20.52199935913086
& \underline{0.561} & \underline{51.3} & 0.48604729771614075 & 19.899375915527344
& 0.4888574182987213 & 86.28651428222656 & 0.6592622995376587 & 20.874286651611328 \\
\rowcolor{myrowcolour}
\textbf{Ours}
& \bfseries 0.3529984951019287 & \bfseries 26.456972122192383 & \underline{0.607} & \underline{23.30}
& \bfseries 0.42149055004119873 & \bfseries 32.1186637878418 & 0.525365948677063 & 21.385475158691406
& \bfseries 0.3145533800125122 & \bfseries 21.111848831176758 & \bfseries 0.6527808904647827 & \bfseries24.444496154785156
& \bfseries 0.16288119554519653 & \bfseries 11.026673316955566 & \bfseries 0.8146522045135498 & \bfseries23.75210189819336 \\
\bottomrule
\end{tabular} }
\end{table*}
\endgroup
\noindent\textbf{Posterior Variance.}
To demonstrate that FLAIR does not suffer from mode collapse, we assess the posterior variance~$\Var[x \vert z]$ for the task of $\times12$ Super Resolution, by drawing 32 samples for a fixed observation~$z$ and computing their pixel-wise variance.
We conduct that experiment for our FLAIR approach, for RSD with repulsive term, and for FlowDPS~\cite{kim2025flowdps}.
The example in \autoref{fig:variance_zoom} illustrates that FLAIR has the highest sample diversity, which is also reflected in the corresponding variance maps.
%
Notably, the sample variance is concentrated in regions with high-frequency textures. This indicates that our method reliably reconstructs the posterior, whose low-frequency part is, in the super-resolution setting, tightly constrained by the likelihood term.
%\input{figures/variance_figure_more columns}
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{arxiv_figures/compressed_variance_new.pdf}
\vspace{-.5cm}
\caption{\textbf{Zoomed-in reconstructions for x12 Super Resolution.} We show posterior samples (col. 1–4) of \method{}, FlowDPS, and RSD, posterior mean and standard deviation (over 32 samples, col. 5,6). $0$ \protect\includegraphics[width=1.5cm,height=.2cm]{arxiv_figures/bar.png} $0.16$}
\label{fig:variance_zoom}
\end{figure}
\noindent\textbf{Editing.}
Beyond image restoration, we observe that our method also performs remarkably well for text-based image editing, by simply presenting suitable target prompts during inpainting.
\autoref{fig:editing} illustrates a variety of edited images generated from the same photograph with the help of the depicted masks and prompts.
\begin{figure}
\centering
\includegraphics[width=\linewidth]{arxiv_figures/julius_c.pdf}
\vspace{-.5cm}
\caption{\textbf{Edited images} shown alongside original, with prompts: "A high resolution portrait of a..."}
\label{fig:editing}
\end{figure}
%\input{figures/editing_figure}
\subsection{Ablation Studies}
We systematically analyze the impact of key design choices in our method. Specifically, we ablate the deterministic trajectory adjustment, the use of hard data consistency, and the calibration of the regularizer weight for $\times 12$ super-resolution, using a subset of 100 samples from the FFHQ and DIV2K datasets. Quantitative and qualitative results are shown in~\autoref{tab:ablation} and~\autoref{fig:ablation}, respectively.
\noindent\textbf{Hard Data Consistency (HDC).}
Dropping the hard data consistency degrades both metrics, with PSNR being particularly affected due to poorer alignment with the input observation, which is also evident in the visual example: the reconstruction is plausible, but deviates from the observation.
\noindent\textbf{Deterministic Trajectory Adjustment (DTA).}
The biggest performance drop compared to the full setup occurs when removing the deterministic trajectory adjustment, as standard noise injection harms the gradient updates in low-density regions of the prior. The reconstruction appears overly smooth and lacks texture details.
\noindent\textbf{Calibrated Regularizer Weight (CRW).}
Replacing our calibrated regularizer weight with $\lambda_{\mathcal{R}}(t)=t$ also has a strong impact on perceptual quality: the result is visibly blurred if one ignores the changing accuracy of the regularizer along the flow trajectory.
\begin{figure}
\centering
\includegraphics[width=\linewidth]{arxiv_figures/ablation_c.pdf}
\vspace{-.5cm}
\caption{\textbf{Qualitative samples from the ablation study} on $\times$12 Super Resolution.}
\label{fig:ablation}
\end{figure}
%\input{figures/ablation_figure}
%\input{tables/ablations}
\begin{table}[ht]
\centering
\caption{\textbf{Ablation study} for $\times$12 super-resolution on DIV2K and FFHQ. Model components are individually switched on or off.} % to assess their influence on reconstruction quality, measured in terms of LPIPS and PSNR.
\label{tab:ablation}
\begin{tabular}{ccc|cc|cc}
\toprule
HDC & DTA & CRW & \multicolumn{2}{c|}{FFHQ} & \multicolumn{2}{c}{DIV2K} \\
& & & LPIPS ↓ & PSNR ↑ & LPIPS ↓ & PSNR ↑\\
\midrule
\cmark & \cmark & \cmark & 0.259 & 27.45 & 0.427 & 21.05 \\
\xmark & \cmark & \cmark & 0.297 & 27.17 & 0.467 & 20.82 \\ %28.00 & 0.276 (1 step in table 3step in comment)
\cmark & \xmark & \cmark & 0.432 & 27.20 & 0.622 & 21.69 \\
\cmark & \cmark & \xmark & 0.363 & 28.58 & 0.583 & 21.98 \\
\xmark & \xmark & \xmark & 0.392 & 28.33 & 0.605 & 21.99 \\
\bottomrule
\end{tabular}
\vspace{1ex}
\parbox{\linewidth}{\small
\textbf{Legend.}
HDC: Hard Data Consistency;
DTA: Deterministic Trajectory Adjustment;\\
CRW: Calibrated Regularizer Weight.
\cmark{} = included, \xmark{} = ablated.
}
\end{table}
% 1. Inverse Problems (x8, x12 superres, motion debluring, inpainting, denoising) \\
% 2. SD3 + SD3.5Large + Flux + Pixel Space \\
% 3. Ablations: 1. separate optimizers (hard datat consistency) 2. deterministic renoising 3. optimal regularizer weighting
% Baselines: 1. FlowDPS (arxiv), FlowChef (arxiv), PSLD , Resample , Repulsive Score Distillation (Var approach latent) all for the same model (SD3)
%\input{figures/var_figure}
\section{Conclusion and Limitations}
\label{sec:conclusion}
We have presented \method{}, a training-free variational framework for inverse problems that uses a flow-based generative model as its image prior. By combining the power of (latent) flow-based models with a principled reconstruction of the posterior distribution, \method{} addresses key limitations of existing methods.
%
First, it is able to recover atypical modes, by combining the degradation-agnostic flow matching loss with deterministic trajectory adjustments.
%
Second, it enables hard data consistency without sacrificing sample diversity, by decoupling the data consistency constraint from the regularization, while adaptively reweighting the latter according to its expected accuracy, calibrated offline.
%
Experiments with different image datasets and tasks confirm that \method{} consistently achieves higher reconstruction quality than existing baselines based on either flow matching or denoising diffusion.
%
Notably, our proposed method achieves, at the same time, excellent perceptual quality, close adherence to the input observations, and high sample diversity.
Evidently, \method{} inherits the limitations of the underlying generative model, in our implementation Stable Diffusion 3.
%
These include biases caused by the selection of training data, constraints w.r.t.\ the output resolution, and a limited ability to recover out-of-distribution modes.
%
Furthermore, our approach introduces additional hyper-parameters needed to control the deterministic trajectory adjustment.
%
We note that high fidelity image restoration methods can potentially be misused for unethical image manipulations.
\clearpage
\bibliographystyle{plain}
\bibliography{bib} % Entries are in the bib.bib file
\clearpage
\appendix
\section*{\Large Supplementary Material}
%\setcounter{table}{0}
%\setcounter{figure}{0}
In the following, we provide detailed line-by-line derivations of the mathematical formulations used in the paper, as well as additional implementation details and experimental results.
\vspace{-.2cm}
\section{Derivations}
\label{sec:derivations}
\subsection{Derivation of flow-based variational formulation}
\label{sec:derivation_flow}
The linear conditional flow and it's corresponding velocity are defined as:
\begin{equation}
\label{eq:flow1}
\psi_t(x_0 \mid \epsilon)
= (1 - t)\,x_0 + t\,\epsilon,
\quad
\epsilon \sim \mathcal{N}(0, I)\;,
\end{equation}
\begin{equation}
\label{eq:vel_ap}
u_t(x_t|\epsilon) = \frac{\d{\psi_t}}{\d t}(\psi_t^{-1}(x_t|\epsilon)|\epsilon)\;.
\end{equation}
The score of the noisy variational distribution can be analytically computed with:
\begin{equation}
\label{eq:q(x_t)}
q(x_t|y) = \mathcal{N}((1-t)\mu_x, t^2\,I)\;,
\end{equation}
\begin{equation}
\label{eq:score_q(x_t)}
\nabla_{x_t} \log q(x_t|y) = -\frac{\epsilon}{t}\;.
\end{equation}
We compute $\frac{\d{\psi_t}}{\d t}(x_0|\epsilon)=-x_0 +\epsilon$ and $\psi_t^{-1}(x_t|\epsilon)$ and insert it into \autoref{eq:vel_ap}:
\begin{equation}
\label{eq:eps_to_u}
u_t(x_t|\epsilon) = \frac{\epsilon - x_t}{1-t}\;.
\end{equation}
Solving \autoref{eq:eps_to_u} for $\epsilon$ and inserting in \autoref{eq:score_q(x_t)} gives:
\begin{equation}
\label{eq:score_to_u_ap}
\nabla_{x_t} \log q(x_t|y) = -\frac{(1-t)u_t(x_t|\epsilon) + x_t}{t}\;.
\end{equation}
For the learned velocity $v_\theta(x_t,t)$ a similar approximation holds:
\begin{equation}
\label{eq:v_theta_to_eps_ap}
v_\theta(x_t, t) \approx \frac{-t\nabla_x \log p(x_t) - x_t}{1-t}\;.
\end{equation}
Hence, we can approximate the score of the noisy prior with our learned velocity field $v_\theta$
\begin{equation}
\nabla_{x_t} \log p(x_t) \approx -\frac{(1-t) v_\theta(x_t, t) + x_t}{t}\;,
\label{eq:v_to_score_ap2}
\end{equation}
and we see that for $\omega(t) = \frac{t}{1-t}$ we obtain the conditional flow matching objective for $\mathcal{R}(x)$. We therefore set $\omega(t) = \frac{t}{1-t}$
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%5
and end up at our final objective:
\begin{equation}
\arg\min_{q(x_0|y)}
\underbrace{
\mathbb{E}_{q(x_0|y)} \left[ \frac{\|y - f(\mu_x)\|^2}{2\nu^2} \right]
}_{\text{$\mathcal{D}(x,y)$}}
+
\underbrace{
\int_0^T ~ \mathbb{E}_{q(x_t|y)} \left[ \left\|v_\theta(x_t,t) - u_t(x_t|\epsilon) \right\|^2 \right] dt
}_{\text{$\mathcal{R}(x)$}}\;.
\label{eq:combined_loss}
\end{equation}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Again, the gradient step for the regularizer becomes:
\begin{equation}
\nabla_{\mu_x} \mathcal{R}(x) =
\mathbb{E}_{q(x_t|y)} \left[ v_\theta(x_t,t) - u_t(x_t|\epsilon) \right]\;.
\label{eq:gradient_step_reg_weighted}
\end{equation}
\subsection{Derivation of trajectory adjusted flow-based variational formulation}
\label{sec:derivation_flow_adjusted_ap}
%As we aim to approximate the posterior distribution with the variational distribution $q(x_t\vert y)$, we construct $x_t = (1-t)\mu+t\epsilon$, with $\epsilon \sim \mathcal{N}(0,\Id)$, leading to the score $\nabla_x \log q(x_t|y) = -\frac{\epsilon}{t}$.
To achieve the proposed trajectory adjustment, we modify the forward process to:
\begin{equation}
\hat{x}_1 = x_{t+dt} + (1-t-dt)v_\theta(x_{t+dt}, t+dt)\;,
\end{equation}
\begin{equation}
\label{eq:new_forward_x_t}
x_t = (1-t) \mu_x + t\underbrace{(\alpha\hat{x}_1 + \sqrt{1-\alpha^2}\epsilon)}_{\text{$\hat{\epsilon}$}}\;,
\end{equation}
where $\hat{x}_1$ is the noise vector prediction from the last optimization iteration.
This induces a variational distribution:
\begin{equation}
q(x_t \mid y) = \mathcal{N}\left((1 - t)\mu_x + t\alpha \hat{x}_1,\, t^2(1 - \alpha^2)I\right)\;,
\end{equation}
leading to a score of
\begin{equation}
\nabla_{x_t} \log q(x_t \mid y) = -\frac{1}{t^2(1 - \alpha^2)} \cdot t\sqrt{1 - \alpha^2} \epsilon
= -\frac{\epsilon}{t\sqrt{1 - \alpha^2}}\;.
\end{equation}
The velocity field is again computed by \autoref{eq:vel_ap}.
We start by defining the flow:
\begin{equation}
\psi_t(x_0 \mid \epsilon) = (1 - t)x_0 + t\left(\alpha \hat{x}_1 + \sqrt{1 - \alpha^2} \epsilon\right)\;.
\end{equation}
The resulting derivative reads
\begin{equation}
\frac{d}{dt} \psi_t(x_0 \mid \epsilon) = \alpha \hat{x}_1 - x_0 + \sqrt{1 - \alpha^2} \epsilon\;,
\end{equation}
and the inverse becomes
\begin{equation}
x_0 = \psi_t^{-1}(x_t \mid \epsilon) = \frac{x_t - t \alpha \hat{x}_1 - t \sqrt{1 - \alpha^2} \epsilon}{1 - t}\;.
\end{equation}
Plugging these results into \autoref{eq:vel_ap}:
\begin{equation}
u_t(x_t \mid \epsilon) = \frac{\alpha \hat{x}_1 + \sqrt{1 - \alpha^2} \epsilon - x_t}{1 - t}\;.
\end{equation}
\subsection{Derivation of Score from Flow}
The score matching objective reads as:
\begin{equation}
\label{eq:scorefunc}
\nabla_{x_t} \ln p_t(x_t) = \underset{\theta}{\arg \min} ~\mathbb{E}_{t \sim \mathcal{U}[0,1], x_0 \sim p_0, \epsilon \sim \mathcal{N}(0,I)} \left[ w(t) \cdot \left\| s_\theta(x_t, t) + \frac{1}{\sigma(t)^2} \left( x_t - \mu(x_0, t) \right) \right\|^2 \right],
\end{equation}
where,
\begin{equation}
-\frac{1}{\sigma(t)^2} \left( x_t - \mu(x_0, t) \right) = \nabla_{x_t} \log p_t(x_t \mid x_0),
\end{equation}
with $p_t(x_t \mid x_0) = \mathcal{N}(\mu_t(x_0), \sigma_t^2 I)$. Note that as usual we assume $\mu_t(x_0)$ being linear in $x_0$.\\
\autoref{eq:scorefunc} is solved by:
\begin{equation}
\nabla_{x_t} \log p_t(x_t) = \mathbb{E}_{p_t(x_0 | x_t)} \left[\nabla_{x_t} \log p_t(x_t | x_0)\right],
\end{equation}
and can be written as:
\begin{equation}
\nabla_{x_t} \log p_t(x_t) = \frac{-(x_t - \mu(\mathbb{E}[x_0 \mid x_t], t))}{\sigma(t)^2}.
\label{eq:nab}
\end{equation}
In the case of OT flow-matching, we obtain
\begin{equation}
x_t = (1-t)x_0+tx_1,
\end{equation}
$x_1\sim\mathcal{N}(0,\Id)$ and $p(x_t|x_0) =\mathcal{N}((1 - t)x_0 , t^2)$.
The optimal velocity under the flow matching loss is given by:
\begin{equation}
v^*(x_t, t) = \mathbb{E}[x_1 - x_0 \mid x_t].
\label{eq:opt_v}
\end{equation}
Expressing $x_1 = \frac{x_t-(1-t)x_0}{t}$, we can insert into \autoref{eq:opt_v} and obtain:
\begin{equation}
\mathbb{E}[x_0 \mid x_t] = x_t-t\mathbb{E}[x_1 - x_0 \mid x_t].
\end{equation}
Inserting in \autoref{eq:nab} leads to:
\begin{equation}
\nabla_{x_t} \log p_t(x_t) = -\frac{x_t-(1-t)(x_t-t\mathbb{E}[x_1 - x_0 \mid x_t]) }{t^2},
\end{equation}
which for $v^*(x_t, t) = \mathbb{E}[x_1 - x_0 \mid x_t]$ reads as:
\begin{equation}
\nabla_{x_t} \log p(x_t) \approx -\frac{(1-t) v_\theta(x_t, t) + x_t}{t}.
\label{eq:v_to_score_ap}
\end{equation}
\subsection{Implementation details}
\label{sec:ap_imlpementation_details}
\noindent\textbf{Flow Model and Regularizer Settings.} As flow matching model, we us Stable Diffusion 3.5-Medium, which has been released under the Stability Community License. The classifier-free guidance scale is set to 2 for all experiments. To minimize the regularization term, we use stochastic gradient descent with a learning rate of $1$.
\noindent\textbf{Data Likelihood Term.} We use stochastic gradient descent for the minimization of the data term towards hard data consistency. For numerical stability, the squared error is summed over all measurements instead of computing the mean. The learning rate has to be adjusted accordingly, to compensate for the varying number of measurements $y$. Moreover, the minimization is terminated with early stopping once the likelihood term reaches $1\times 10^{-4} \cdot \text{len}(y)$, to not overfit the noise in the image observation.
\noindent\textbf{Super-resolution.} We employ bicubic downsampling as the forward operator, as implemented in \cite{wang2022zero}. The learning rate is set to 12 for $\times$12 super-resolution and to 6 for $\times$8 super-resolution.
\noindent\textbf{Motion Deblurring.} A different motion blur kernel is created for each sample using the \emph{MotionBlur} package \cite{borodenko2025motionblur}, available via github, with kernel size 61 and intensity 0.5.
The learning rate for our data term optimizer is set to $10^{-1}$.
\noindent\textbf{Inpainting.} For inpainting on FFHQ we always use the same rectangular mask at a fixed position, chosen such that it roughly masks out the right side of the face (Figure~\ref{fig:inpaint_ffhq}). For DIV2k we also use a fixed mask for all samples, consisting of six randomly generated rectangles (Figure~\ref{fig:inpaint_2dk}).
\noindent\textbf{Data.} We use the publicly available Flickr Faces High Quality dataset \cite{Karras2019stylegan}, which is realeased under the Creative Commons BY 2.0 License and the DIV2K dataset \cite{Agustsson2017DIV2K}, which is released under a research only license. For FFHQ we use the first 1000 samples of the evaluation dataset and for DIV2K we use the 800 training samples.
We downscale both datasets to $768\times768$~px by applying bicubic sampling so that the shorter edge of the frame has 768~px and apply central cropping afterwards.
\subsection{Baselines}
\label{sec:supp_baselines}
For comparability, all baselines use Stable Diffusion 3.5-Medium and the same task definitions as in \ref{sec:ap_imlpementation_details}.
\noindent\textbf{FlowDPS} \cite{kim2025flowdps} The standard FlowDPS implementation \cite{kim2025flowdpsgithub} is applied with 50 NFE, a classifier-free guidance scale of 2, and step sizes of 15 for inpainting and 10 for all other tasks.
\noindent\textbf{FlowChef} \cite{patel2024flowchef} Additionally, \cite{kim2025flowdpsgithub} is employed for FlowChef as well, using 50 NFE, a classifier-free guidance scale of 2, and a step size of 1 for all tasks.
\noindent\textbf{Repulsive Score Distillation (RSD)} \cite{zilberstein2025repulsive}. We implement RSD for flow-matching models by applying Proposition~\autoref{prop:propostion1} with $\omega(t)=t$, resulting in a weighting term consistent with the original RSD approach. However, we omit the pixel-space augmentation as it negatively affected performance when combined with the SD3 VAE. Consistent with the original findings from RSD, we observed that incorporating the repulsive term improves sample diversity but reduces fidelity. Therefore, we set the repulsive term to 0 for all results presented in the table, employing it exclusively for comparing posterior variances.
\noindent\textbf{ReSample} \cite{song2024solving} We re implement ReSample for flow-matching by setting $\bar{\alpha}_{t}=\frac{\left(1-t\right)^{2}}{t^{2}+\left(1-t\right)^{2}}$. Furthermore, we compute the hard-data consistency at every iteration as larger skip steps seem to harm performance. We set the learning rate of the data term optimizer to 15 for all inverse problems.
\noindent\textbf{PSLD} \cite{rout2023solving} Our attempt to adapt PSLD following \cite{kim2025flowdpsgithub}—using 500 NFE, a classifier-free guidance scale of 2, and step sizes of 1 ($\times$12 super-resolution), 0.5 ($\times$8 super-resolution and motion deblurring) and 0.1 (inpainting)—did not yield meaningful results.
\subsection{Regularizer weighting}
\autoref{fig:sd3_loss} displays the mean and standard deviation of the conditional flow matching loss $\mathcal{L}_{CFM}$ as a function of $t$, estimated over 100 samples.
The loss function starts with high values at $t=1$, decreases over time, but then starts to rise again, and when reaching $t\approx0.2$ even exceeds its initial value .
The rising loss when approaching $t=0$ is due, in part, to the increasing difficulty of distinguishing high-frequency image content from residual noise. Another factor is that near $t=0$ the model operates in a highly sensitive regime where small prediction errors can cause disproportionately large deviations from the target, making accurate flow estimation particularly challenging in the final stages of the trajectory. We therefore modulate the regularization term according to the model error. Different weighting functions for $f(\mathcal{L}_{CFM})$ could be chosen that fulfill the condition $\lambda_{\mathcal{R}(t=0)}=0$. We simply take the reciprocal of the model error $\lambda_{\mathcal{R}(t)}=\mathcal{L}_{CFM,t}^{-1}$ as the regularization weight while $t\geq0.2$, then set it to 0 for $t<0.2$.
%
An alternative would be to shift the reciprocal of $\mathcal{L}_{CFM,t}^{-1}$ by $\mathcal{L}_{CFM,t=0}^{-1}$, such that $\lambda_{\mathcal{R}(t)} = \mathcal{L}_{CFM,t}^{-1} - \mathcal{L}_{CFM,t=0}^{-1}$. In \autoref{tab:reg_shift} we compare our default weighting with this variant, denoted as $\lambda_{shift}$.
\begin{figure}[h]
\centering
\includegraphics[width=.8\linewidth]{arxiv_figures/reg_weight_v_div2k_reverse.pdf}
\caption{The Flow-Matching loss over time $t$.}
\label{fig:sd3_loss}
\end{figure}
%\input{tables/supplementary_results.tex/FFHQ_reg_weight_shift}
\begingroup
\setlength{\tabcolsep}{3pt}
\begin{table*}[htbp]
\centering
% \small
\caption{\textbf{Quantitative results} with 50 NFE and $\sigma_{\nu} = 0.01$. We compare different weighting functions $\lambda_{\mathcal{R}(t)}$ based on the model error}
\vspace{0.5em}
\label{tab:reg_shift}
%––––– global siunitx settings –––––
\sisetup{
detect-all,
round-mode = places,
table-align-text-post = false
}
\resizebox{\linewidth}{!}{%
\begin{tabular}{
@{}l
% 16 numeric columns
S[table-format=1.3, round-precision=3] % LPIPS
S[table-format=2.1, round-precision=1] % FID
S[table-format=1.3, round-precision=3] % SSIM
S[table-format=2.2, round-precision=2]| % PSNR
S[table-format=1.3, round-precision=3]
S[table-format=2.1, round-precision=1]
S[table-format=1.3, round-precision=3]
S[table-format=2.2, round-precision=2]|
S[table-format=1.3, round-precision=3]
S[table-format=2.1, round-precision=1]
S[table-format=1.3, round-precision=3]
S[table-format=2.2, round-precision=2]|
S[table-format=1.3, round-precision=3]
S[table-format=2.1, round-precision=1]
S[table-format=1.3, round-precision=3]
S[table-format=2.2, round-precision=2]@{}
}
\toprule
& \multicolumn{4}{c|}{SR $\times$8}
& \multicolumn{4}{c|}{SR $\times$12}
& \multicolumn{4}{c|}{Motion Deblurring}
& \multicolumn{4}{c}{Inpainting}\\
\midrule
Method
& {\small LPIPS$\downarrow$} & {\small FID$\downarrow$} & {\small SSIM$\uparrow$} & {PSNR$\uparrow$}
& {\small LPIPS$\downarrow$} & {\small FID$\downarrow$} & {\small SSIM$\uparrow$} & {PSNR$\uparrow$}
& {\small LPIPS$\downarrow$} & {\small FID$\downarrow$} & {\small SSIM$\uparrow$} & {PSNR$\uparrow$}
& {\small LPIPS$\downarrow$} & {\small FID$\downarrow$} & {\small SSIM$\uparrow$} & {PSNR$\uparrow$}\\
\toprule
\multicolumn{17}{c}{\textbf{FFHQ 768$\times$768}}\\
\midrule
$\lambda_{shift}$
& 0.246 & 27.4 & \bfseries 0.793 & \bfseries 29.91
& 0.286 & 24.4 & \bfseries 0.766 & \bfseries 28.19
& 0.237 & 14.5 & \bfseries 0.790 & \bfseries 29.84
& \bfseries 0.180 & \bfseries 8.2 & \bfseries 0.828 & 23.58 \\
\rowcolor{myrowcolour}
\textbf{Ours}
& \bfseries 0.2132353037595749 & \bfseries 13.26621150970459 & 0.777 & 29.54
& \bfseries 0.27056485414505005 & \bfseries 16.201229095458984 & 0.7402094602584839 & 27.714468002319336
& \bfseries 0.23615501821041107 & \bfseries 10.732836723327637 & 0.772 & 29.61139488220215
& 0.18383462727069855 & 8.74171257019043 & 0.8277943730354309 & \bfseries 23.69305419921875 \\
\toprule
\multicolumn{17}{c}{\textbf{DIV2K 768$\times$768}}\\
\toprule
$\lambda_{shift}$
& 0.379 & 30.4 & \bfseries 0.625 & \bfseries 23.58
& 0.434 & 37.5 & 0.522 & \bfseries 21.40
& 0.337 & 25.8 & \bfseries 0.664 & \bfseries 24.54
& \bfseries 0.151 & \bfseries 9.0 & \bfseries 0.819 & \bfseries 23.79 \\
\rowcolor{myrowcolour}
\textbf{Ours}
& \bfseries 0.3529984951019287 & \bfseries 26.456972122192383 & 0.607 & 23.30
& \bfseries 0.42149055004119873 & \bfseries 32.1186637878418 & \bfseries 0.525365948677063 & 21.385475158691406
& \bfseries 0.3145533800125122 & \bfseries 21.111848831176758 & 0.6527808904647827 & 24.444496154785156
& 0.16288119554519653 & 11.026673316955566 & 0.8146522045135498 & 23.75210189819336 \\
\bottomrule
\end{tabular}%
}
\end{table*}
\endgroup
\subsection{Effect of captioning}
Given the diversity of DIV2k, we use DAPE \cite{wu2024seesr} to generate captions for it and include them in the prompt \textit{A high quality photo of [DAPE caption].} For FFHQ we always prompt with \textit{A high quality photo of a face.}. The effect of the text prompt is to increase the likelihood of our sample under the prior of the (pre-trained, frozen) image generator. For comparison, we also ran experiments without data specific captions, where we always used the generic prompt \textit{A high quality photo.} Results are shown in \autoref{tab:wo_captions}
\begin{table*}[htbp]
\centering
\caption{Quantitative results with 50 NFE and $\sigma_{\nu}=0.01$. We compare our version with data-specific captions and a version without captions.}
\vspace{0.5em}
\label{tab:wo_captions}
\resizebox{\linewidth}{!}{%
\begin{tabular}{@{}l
r r r r |
r r r r @{}}
\toprule
& \multicolumn{4}{c|}{FFHQ}
& \multicolumn{4}{c}{DIV2K} \\
Method
& \small LPIPS$\downarrow$ & \small FID$\downarrow$ & \small SSIM$\uparrow$ & PSNR$\uparrow$
& \small LPIPS$\downarrow$ & \small FID$\downarrow$ & \small SSIM$\uparrow$ & PSNR$\uparrow$ \\
\midrule
wo captions
& 0.278 & 17.0 & 0.734 & 27.66
& 0.488 & 51.5 & \textbf{0.546} & \textbf{21.82} \\
\rowcolor{myrowcolour}
\textbf{Ours}
& \textbf{0.271} & \textbf{16.2} & \textbf{0.740} & \textbf{27.71}
& \textbf{0.421} & \textbf{32.1} & 0.525 & 21.39 \\
\bottomrule
\end{tabular}%
}
\end{table*}
\subsection{Additional Experimental Results}
\label{sec:additional_tables}
% -------------------------- Table 1 --------------------------
\begin{table*}[htbp]
\centering
\caption{\textbf{Quantitative results} with 50 NFE and $\sigma_{\nu}=0.01$ – Super-resolution ($\times8$ and $\times12$).}
\label{tab:suppl_std_results}
\resizebox{\linewidth}{!}{
\begin{tabular}{@{}lcccccccc@{}}
\toprule
& \multicolumn{4}{c}{SR $\times8$} & \multicolumn{4}{c}{SR $\times12$}\\
\cmidrule(lr){2-5}\cmidrule(lr){6-9}
Method & LPIPS$\downarrow$ & FID$\downarrow$ & SSIM$\uparrow$ & PSNR$\uparrow$
& LPIPS$\downarrow$ & FID$\downarrow$ & SSIM$\uparrow$ & PSNR$\uparrow$\\
\midrule
\multicolumn{9}{c}{\textbf{FFHQ 768$\times$768}}\\
\midrule
ReSample
& 0.400 $\pm$ 0.069 & 55.6 & \textbf{0.815 $\pm$ 0.051} & 26.37 $\pm$ 1.00
& 0.474 $\pm$ 0.078 & 80.3 & \textbf{0.786 $\pm$ 0.056} & 25.47 $\pm$ 1.16\\
FlowDPS
& 0.374 $\pm$ 0.107 & 40.3 & 0.756 $\pm$ 0.075 & 29.24 $\pm$ 2.04
& 0.413 $\pm$ 0.107 & 47.2 & 0.741 $\pm$ 0.074 & \underline{28.05 $\pm$ 2.06}\\
RSD
& 0.391 $\pm$ 0.079 & 51.7 & 0.776 $\pm$ 0.052 & \textbf{29.69 $\pm$ 2.04}
& 0.462 $\pm$ 0.093 & 71.7 & \underline{0.743 $\pm$ 0.059} & \textbf{28.11 $\pm$ 2.00}\\
FlowChef
& \underline{0.341 $\pm$ 0.083} & \underline{30.5} & 0.760 $\pm$ 0.064 & 28.42 $\pm$ 2.22
& \underline{0.373 $\pm$ 0.084} & \underline{41.0} & 0.730 $\pm$ 0.068 & 27.00 $\pm$ 2.07\\
\rowcolor{myrowcolour}\textbf{Ours}
& \textbf{0.213 $\pm$ 0.056} & \textbf{13.3} & \underline{0.777 $\pm$ 0.051} & \underline{29.54 $\pm$ 2.02}
& \textbf{0.271 $\pm$ 0.071} & \textbf{16.2} & 0.740 $\pm$ 0.058 & 27.71 $\pm$ 2.00\\
\midrule
\multicolumn{9}{c}{\textbf{DIV2K 768$\times$768}}\\
\midrule
ReSample
& 0.533 $\pm$ 0.130 & 55.0 & \textbf{0.625 $\pm$ 0.132} & 22.34 $\pm$ 2.27
& 0.643 $\pm$ 0.152 & 88.1 & \textbf{0.562 $\pm$ 0.151} & 20.85 $\pm$ 3.02\\
FlowDPS
& \underline{0.476 $\pm$ 0.129} & 54.2 & 0.567 $\pm$ 0.139 & 23.01 $\pm$ 3.01
& 0.547 $\pm$ 0.139 & 67.4 & \underline{0.528 $\pm$ 0.146} & \underline{21.79 $\pm$ 2.94}\\
RSD
& 0.539 $\pm$ 0.121 & 60.9 & 0.591 $\pm$ 0.124 & \textbf{23.45 $\pm$ 2.96}
& \textbf{0.684 $\pm$ 0.137} & 95.7 & 0.523 $\pm$ 0.132 & \textbf{21.96 $\pm$ 2.86}\\
FlowChef
& 0.490 $\pm$ 0.116 & \underline{48.6} & 0.539 $\pm$ 0.137 & 21.84 $\pm$ 2.96
& \underline{0.525 $\pm$ 0.118} & \underline{55.3} & 0.492 $\pm$ 0.145 & 20.52 $\pm$ 2.85\\
\rowcolor{myrowcolour}\textbf{Ours}
& \textbf{0.353 $\pm$ 0.112} & \textbf{26.5} & \underline{0.607 $\pm$ 0.127} & \underline{23.30 $\pm$ 2.90}
& \textbf{0.421 $\pm$ 0.131} & \textbf{32.1} & 0.525 $\pm$ 0.136 & 21.39 $\pm$ 2.67\\
\bottomrule
\end{tabular}
}
\end{table*}
% -------------------------- Table 2 --------------------------
\begin{table*}[htbp]
\centering
\caption{\textbf{Quantitative results} with 50 NFE and $\sigma_{\nu}=0.01$ – Motion deblurring and in-painting.}
\label{tab:motion_inpaint_results}
\resizebox{\linewidth}{!}{
\begin{tabular}{@{}lcccccccc@{}}
\toprule
& \multicolumn{4}{c}{Motion Deblurring} & \multicolumn{4}{c}{In-painting}\\
\cmidrule(lr){2-5}\cmidrule(lr){6-9}
Method & LPIPS$\downarrow$ & FID$\downarrow$ & SSIM$\uparrow$ & PSNR$\uparrow$
& LPIPS$\downarrow$ & FID$\downarrow$ & SSIM$\uparrow$ & PSNR$\uparrow$\\
\midrule
\multicolumn{9}{c}{\textbf{FFHQ 768$\times$768}}\\
\midrule
ReSample
& 0.457 $\pm$ 0.087 & 82.9 & \textbf{0.788 $\pm$ 0.058} & 25.45 $\pm$ 1.46
& 0.366 $\pm$ 0.053 & 70.8 & \underline{0.827 $\pm$ 0.033} & 21.83 $\pm$ 1.68\\
FlowDPS
& 0.431 $\pm$ 0.117 & 49.5 & 0.732 $\pm$ 0.078 & 27.64 $\pm$ 2.20
& \underline{0.344 $\pm$ 0.060} & \underline{38.3} & 0.771 $\pm$ 0.048 & 19.19 $\pm$ 3.19\\
RSD
& 0.458 $\pm$ 0.098 & 77.3 & 0.743 $\pm$ 0.059 & \underline{27.67 $\pm$ 2.47}
& 0.478 $\pm$ 0.082 & 73.3 & 0.736 $\pm$ 0.048 & \underline{21.97 $\pm$ 2.58}\\
FlowChef
& \underline{0.406 $\pm$ 0.093} & \underline{35.0} & 0.716 $\pm$ 0.072 & 25.81 $\pm$ 2.61
& 0.394 $\pm$ 0.069 & 71.0 & 0.780 $\pm$ 0.051 & 18.18 $\pm$ 2.84\\
\rowcolor{myrowcolour}\textbf{Ours}
& \textbf{0.236 $\pm$ 0.070} & \textbf{10.7} & \underline{0.772 $\pm$ 0.055} & \textbf{29.61 $\pm$ 2.24}
& \textbf{0.184 $\pm$ 0.038} & \textbf{8.7} & \textbf{0.828 $\pm$ 0.029} & \textbf{23.69 $\pm$ 2.77}\\[2pt]
\midrule
\multicolumn{9}{c}{\textbf{DIV2K 768$\times$768}}\\
\midrule
ReSample
& \underline{0.556 $\pm$ 0.146} & 79.7 & \underline{0.617 $\pm$ 0.134} & 21.79 $\pm$ 2.52
& \underline{0.285 $\pm$ 0.073} & 51.9 & \underline{0.796 $\pm$ 0.067} & 22.68 $\pm$ 1.84\\
FlowDPS
& 0.558 $\pm$ 0.153 & 60.4 & 0.536 $\pm$ 0.148 & 21.88 $\pm$ 3.02
& 0.328 $\pm$ 0.103 & \underline{34.2} & 0.692 $\pm$ 0.112 & 21.71 $\pm$ 2.67\\
RSD
& 0.638 $\pm$ 0.156 & 97.6 & 0.551 $\pm$ 0.136 & \underline{22.10 $\pm$ 3.07}
& 0.464 $\pm$ 0.112 & 63.9 & 0.678 $\pm$ 0.077 & \underline{23.23 $\pm$ 2.21}\\
FlowChef
& 0.561 $\pm$ 0.123 & \underline{51.3} & 0.486 $\pm$ 0.148 & 19.90 $\pm$ 3.06
& 0.489 $\pm$ 0.148 & 86.3 & 0.659 $\pm$ 0.131 & 20.87 $\pm$ 2.65\\
\rowcolor{myrowcolour}\textbf{Ours}
& \textbf{0.315 $\pm$ 0.107} & \textbf{21.1} & \textbf{0.653 $\pm$ 0.121} & \textbf{24.44 $\pm$ 3.05}
& \textbf{0.163 $\pm$ 0.053} & \textbf{11.0} & \textbf{0.815 $\pm$ 0.054} & \textbf{23.75 $\pm$ 2.74}\\
\bottomrule
\end{tabular}
}
\end{table*}
We present the experimental results from the main paper in \autoref{tab:suppl_std_results}, now augmented with sample-wise standard deviations for all metrics except FID.
\subsection{Additional Qualitative Examples}
\label{sec:additional_results}
To illustrate the visual differences behind the error metrics, we present additional qualitative results for both FFHQ and DIV2k, comparing \method{} with existing approaches. These examples complement the images in the main paper and highlight the visual fidelity, consistency, and robustness of our method across diverse scenes and different degradations.
\autoref{fig:var_full_} features a full sized version of the variance figure in section \autoref{sec:results}.
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{arxiv_figures/compressed_inpaint_0_10_15_691_451.pdf}
\vspace{-.5cm}
\caption{Inpainting results on FFHQ. Shown are observation, reference methods, \method{} and ground truth. \method{} produces realistic, high-frequency details while previous works either fail to inpaint the region correctly or collapse to overly smooth solutions.
}
\label{fig:inpaint_ffhq}
\end{figure}
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{arxiv_figures/compressed_x12_1_34_64_511_755.pdf}
\vspace{-.5cm}
\caption{$\times12$ super-resolution results on FFHQ. Shown are observation, reference methods, \method{} and ground truth. \method{} produces sharp and results which still fulfill the data term, whereas the baselines tend to predict blurry images.}
\label{fig:x12_ffhq}
\end{figure}
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{arxiv_figures/compressed_deblur_111_349_555_688_955.pdf}
\vspace{-.5cm}
\caption{Motion de-blur results on FFHQ. Shown are observation, reference methods, \method{} and ground truth. \method{} produces sharp and results which still fulfill the data term, whereas the baselines tend to predict blurry images.}
\label{fig:deblur_ffhq}
\end{figure}
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{arxiv_figures/compressed_inpaint_2dk_353_626_590_782_639.pdf}
\vspace{-.5cm}
\caption{Inpainting results on DIV2k. Shown are observation, reference methods, \method{} and ground truth. \method{} produces realistic, high-frequency details while previous works either fail to inpaint the region correctly or collapse to overly smooth solutions. Moreover they do not fit the data term (not inpainted region) very well.}
\label{fig:inpaint_2dk}
\end{figure}
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{arxiv_figures/compressed_sr12_d2k_38_37_473_108_35.pdf}
\vspace{-.5cm}
\caption{$\times12$ super-resolution results on DIV2k. Shown are observation, reference methods, \method{} and ground truth. \method{} produces sharp and results which still fulfill the data term, whereas the baselines tend to predict blurry images.}
\label{fig:sr12_d2k}
\end{figure}
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{arxiv_figures/compressed_deblur_d2k_594_8_621_33_634.pdf}
\vspace{-.5cm}
\caption{Motion de-blur results results on DIV2k. Shown are observation, reference methods, \method{} and ground truth. \method{} produces sharp and results which still fulfill the data term, whereas the baselines tend to predict blurry images.}
\label{fig:deblur_d2k}
\end{figure}
%\input{figures/var_figure}
\begin{figure}
\centering
\includegraphics[width=\linewidth]{arxiv_figures/fat_c.pdf}
\vspace{-.5cm}
\caption{Individual samples for x12 Super Resolution with zoom and std. \method{} produces varied samples from the posterior. For superresoltion The variance is expected to be mostly in the high frequencies, because the data term limits low frequency variations. The baselines tend to predict very similar looking images with less detail.}
\label{fig:var_full_}
\end{figure}
\subsection{Failure cases}
We observe two main failure modes for \method{}, see~\autoref{fig:fail}.
First, we find that super-resolution on DIV2k occasionally results in grainy textures, usually in regions with abundant high-frequency detail and complicated light transport. Potentially, this happens for images which do not have high probability under the prior. We do not observe those artifacts for the FFHQ dataset.
Second, we observe a few instances where the strong generative prior hallucinates semantically inconsistent or misaligned structures -- especially facial features.
\begin{figure}[t]
\centering
\includegraphics[width=\linewidth]{arxiv_figures/compressed_fails2.pdf}
\vspace{-.5cm}
\caption{Qualitative failure cases of \method{} on DIV2k and FFHQ. Top row: grainy results from systematic error. Those errors potentially stem from a weak prior for those images. For example we do not observe them for the FFHQ dataset Bottom row: Semantically inconsistent failures. Sometimes the model lacks the ability to incorporate globally consistent semantics into its restorations.}
\label{fig:fail}
\end{figure}
\end{document}
\begin{thebibliography}{10}
\bibitem{aga18}
Hemant~K Aggarwal, Merry~P Mani, and Mathews Jacob.
\newblock {MoDL}: Model-based deep learning architecture for inverse problems.
\newblock {\em IEEE Transactions on Medical Imaging}, 38(2):394--405, 2018.
\bibitem{Agustsson_2017_CVPR_Workshops}
Eirikur Agustsson and Radu Timofte.
\newblock {NTIRE} 2017 challenge on single image super-resolution: Dataset and study.
\newblock In {\em CVPR Workshops}, 2017.
\bibitem{Agustsson2017DIV2K}
Eirikur Agustsson and Radu Timofte.
\newblock {NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study}.
\newblock In {\em Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops}, July 2017.
\bibitem{becker2023neural}
Alexander Becker, Rodrigo~Caye Daudt, Nando Metzger, Jan~Dirk Wegner, and Konrad Schindler.
\newblock Neural fields with thermal activations for arbitrary-scale super-resolution.
\newblock {\em arXiv:2311.17643}, 2023.
\bibitem{bertalmio2000image}
Marcelo Bertalmio, Guillermo Sapiro, Vincent Caselles, and Coloma Ballester.
\newblock Image inpainting.
\newblock In {\em Proceedings of the 27th annual conference on Computer graphics and interactive techniques}, pages 417--424, 2000.
\bibitem{blau2018perception}
Yochai Blau and Tomer Michaeli.
\newblock The perception-distortion tradeoff.
\newblock In {\em CVPR}, 2018.
\bibitem{bor17}
Ashish Bora, Ajil Jalal, Eric Price, and Alexandros~G Dimakis.
\newblock Compressed sensing using generative models.
\newblock In {\em ICML}, 2017.
\bibitem{borodenko2025motionblur}
Levi Borodenko.
\newblock motionblur: Generate authentic motion blur kernels and apply them to images.
\newblock \url{https://github.com/LeviBorodenko/motionblur}, 2025.
\newblock Accessed: 2025-05-19.
\bibitem{ho20_ip_review}
Kristian Bredies and Martin Holler.
\newblock Higher-order total variation approaches and generalisations.
\newblock {\em Inverse Problems. Topical Review}, 36(12):123001, 2020.
\bibitem{chpo11}
Antonin Chambolle and Thomas Pock.
\newblock A first-order primal-dual algorithm for convex problems with applications to imaging.
\newblock {\em Journal of Mathematical Imaging and Vision}, 40(1):120--145, 2011.
\bibitem{chung2023diffusion}
Hyungjin Chung, Jeongsol Kim, Michael~T McCann, Marc~L Klasky, and Jong~Chul Ye.
\newblock Diffusion posterior sampling for general noisy inverse problems.
\newblock In {\em ICLR}, 2023.
\bibitem{chung2022improving}
Hyungjin Chung, Byeongsu Sim, Dohoon Ryu, and Jong~Chul Ye.
\newblock Improving diffusion models for inverse problems using manifold constraints.
\newblock {\em NeurIPS}, 35, 2022.
\bibitem{da04}
Ingrid Daubechies, Michel Defrise, and Christine De~Mol.
\newblock An iterative thresholding algorithm for linear inverse problems with a sparsity constraint.
\newblock {\em Communications on Pure and Applied}, 57(11):1413--1457, 2004.
\bibitem{esser24icml}
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach.
\newblock Scaling rectified flow transformers for high-resolution image synthesis.
\newblock In {\em ICML}, 2024.
\bibitem{ham18}
Kerstin Hammernik, Teresa Klatzer, Erich Kobler, Michael~P Recht, Daniel~K Sodickson, Thomas Pock, and Florian Knoll.
\newblock Learning a variational network for reconstruction of accelerated {MRI} data.
\newblock {\em Magnetic Resonance in Medicine}, 79(6):3055--3071, 2018.
\bibitem{heusel2017gans}
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.
\newblock {GANs} trained by a two time-scale update rule converge to a local {Nash} equilibrium.
\newblock In {\em NeurIPS}, 2017.
\bibitem{ho2020denoising_ddpm}
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
\newblock Denoising diffusion probabilistic models.
\newblock {\em NeurIPS}, 2020.
\bibitem{ja08}
Viren Jain and Sebastian Seung.
\newblock Natural image denoising with convolutional networks.
\newblock {\em NeurIPS}, 2008.
\bibitem{Karras2022edm}
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine.
\newblock Elucidating the design space of diffusion-based generative models.
\newblock In {\em NeurIPS}, 2022.
\bibitem{karras2019style}
Tero Karras, Samuli Laine, and Timo Aila.
\newblock A style-based generator architecture for generative adversarial networks.
\newblock In {\em CVPR}, 2019.
\bibitem{Karras2019stylegan}
Tero Karras, Samuli Laine, and Timo Aila.
\newblock A style-based generator architecture for generative adversarial networks.
\newblock {\em Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, pages 4401--4410, 2019.
\newblock Introduces the Flickr-Faces-HQ (FFHQ) dataset.
\bibitem{kawar2021snips}
Bahjat Kawar, Gregory Vaksman, and Michael Elad.
\newblock {SNIPS}: Solving noisy inverse problems stochastically.
\newblock {\em NeurIPS}, 34, 2021.
\bibitem{kim2025flowdps}
Jeongsol Kim, Bryan~Sangwoo Kim, and Jong~Chul Ye.
\newblock {FlowDPS}: Flow-driven posterior sampling for inverse problems.
\newblock {\em arXiv:2503.08136}, 2025.
\bibitem{kim2025flowdpsgithub}
Jeongsol Kim, Bryan~Sangwoo Kim, and Jong~Chul Ye.
\newblock {FlowDPS}: Flow-driven posterior sampling for inverse problems.
\newblock \url{https://https://github.com/FlowDPS-Inverse/FlowDPS}, 2025.
\newblock Accessed: 2025-05-19.
\bibitem{zb18}
Florian Knoll, Jure Zbontar, Anuroop Sriram, Matthew~J. Muckley, Mary Bruno, Aaron Defazio, Marc Parente, Krzysztof~J. Geras, Joe Katsnelson, Hersh Chandarana, Zizhao Zhang, Michal Drozdzalv, Adriana Romero, Michael Rabbat, Pascal Vincent, James Pinkerton, Duo Wang, Nafissa Yakubova, Erich Owens, C.Lawrence Zitnick, Michael~P. Recht, Daniel~K. Sodickson, and Yvonne~W. Lui.
\newblock {fastMRI}: A publicly available raw k-space and {DICOM} dataset of knee images for accelerated {MR} image reconstruction using machine learning.
\newblock {\em Radiology: Artificial Intelligence}, 2(1):e190007, 2020.
\bibitem{koef21}
Erich Kobler, Alexander Effland, Karl Kunisch, and Thomas Pock.
\newblock Total deep variation: A stable regularization method for inverse problems.
\newblock {\em IEEE Transactions on Pattern Analysis and Machine Intelligence}, 2021.
\bibitem{le17}
Christian Ledig, Lucas Theis, Ferenc Husz{\'a}r, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et~al.
\newblock Photo-realistic single image super-resolution using a generative adversarial network.
\newblock In {\em CVPR}, 2017.
\bibitem{lipman2023flow}
Yaron Lipman, Ricky T.~Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le.
\newblock Flow matching for generative modeling.
\newblock In {\em ICLR}, 2023.
\bibitem{liu2016stein}
Qiang Liu and Dilin Wang.
\newblock Stein variational gradient descent: A general purpose bayesian inference algorithm.
\newblock In {\em NeurIPS}, volume~29, pages 2378--2386, 2016.
\bibitem{liu2022flow}
Xingchao Liu, Chengyue Gong, and Qiang Liu.
\newblock Flow straight and fast: Learning to generate and transfer data with rectified flow.
\newblock {\em arXiv:2209.03003}, 2022.
\bibitem{mardani2024a}
Morteza Mardani, Jiaming Song, Jan Kautz, and Arash Vahdat.
\newblock A variational perspective on solving inverse problems with diffusion models.
\newblock In {\em ICLR}, 2024.
\bibitem{narnhofer2021bayesian}
Dominik Narnhofer, Alexander Effland, Erich Kobler, Kerstin Hammernik, Florian Knoll, and Thomas Pock.
\newblock Bayesian uncertainty estimation of learned variational {MRI} reconstruction.
\newblock {\em IEEE Transactions on Medical Imaging}, 41(2):279--291, 2021.
\bibitem{na19}
Dominik Narnhofer, Kerstin Hammernik, Florian Knoll, and Thomas Pock.
\newblock Inverse {GANs} for accelerated {MRI} reconstruction.
\newblock In {\em Wavelets and Sparsity XVIII}, volume 11138. SPIE, 2019.
\bibitem{park2003super}
Sung~Cheol Park, Min~Kyu Park, and Moon~Gi Kang.
\newblock Super-resolution image reconstruction: a technical overview.
\newblock {\em IEEE signal processing magazine}, 20(3):21--36, 2003.
\bibitem{patel2024flowchef}
Maitreya Patel, Song Wen, Dimitris~N. Metaxas, and Yezhou Yang.
\newblock Steering rectified flow models in the vector field for controlled image generation.
\newblock {\em arXiv:2412.00100}, 2024.
\bibitem{pooledreamfusion}
Ben Poole, Ajay Jain, Jonathan~T Barron, and Ben Mildenhall.
\newblock Dreamfusion: Text-to-3d using 2d diffusion.
\newblock In {\em ICLR}, 2023.
\bibitem{rombach2022high}
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj{\"o}rn Ommer.
\newblock High-resolution image synthesis with latent diffusion models.
\newblock In {\em CVPR}, 2022.
\bibitem{rout2023solving}
Litu Rout, Negin Raoof, Giannis Daras, Constantine Caramanis, Alex Dimakis, and Sanjay Shakkottai.
\newblock Solving linear inverse problems provably via posterior sampling with latent diffusion models.
\newblock In {\em NeurIPS}, 2023.
\bibitem{rof92}
Leonid~I Rudin, Stanley Osher, and Emad Fatemi.
\newblock Nonlinear total variation based noise removal algorithms.
\newblock {\em Physica D: Nonlinear Phenomena}, 60(1-4):259--268, 1992.
\bibitem{sa01}
Sylvain Sardy, Paul Tseng, and Andrew Bruce.
\newblock Robust wavelet denoising.
\newblock {\em IEEE Transactions on Signal Processing}, 49(6):1146--1152, 2001.
\bibitem{shah2018solving}
Viraj Shah and Chinmay Hegde.
\newblock Solving linear inverse problems using gan priors: An algorithm with provable guarantees.
\newblock In {\em ICASSP}, 2018.
\bibitem{sidky2008image}
Emil~Y Sidky and Xiaochuan Pan.
\newblock Image reconstruction in circular cone-beam computed tomography by constrained, total-variation minimization.
\newblock {\em Physics in Medicine \& Biology}, 53(17):4777, 2008.
\bibitem{song2024solving}
Bowen Song, Soo~Min Kwon, Zecheng Zhang, Xinyu Hu, Qing Qu, and Liyue Shen.
\newblock Solving inverse problems with latent diffusion models via hard data consistency.
\newblock In {\em ICLR}, 2024.
\bibitem{song2020denoising_ddim}
Jiaming Song, Chenlin Meng, and Stefano Ermon.
\newblock Denoising diffusion implicit models.
\newblock In {\em ICLR}, 2021.
\bibitem{song2023pseudoinverse}
Jiaming Song, Arash Vahdat, Morteza Mardani, and Jan Kautz.
\newblock Pseudoinverse-guided diffusion models for inverse problems.
\newblock In {\em ICLR}, 2023.
\bibitem{song2021maximum}
Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon.
\newblock Maximum likelihood training of score-based diffusion models.
\newblock {\em NeurIPS}, 34, 2021.
\bibitem{song2021scorebased}
Yang Song, Jascha Sohl-Dickstein, Diederik~P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.
\newblock Score-based generative modeling through stochastic differential equations.
\newblock In {\em ICLR}, 2021.
\bibitem{wang2022zero}
Yinhuai Wang, Jiwen Yu, and Jian Zhang.
\newblock Zero-shot image restoration using denoising diffusion null-space model.
\newblock {\em ICLR}, 2023.
\bibitem{wang2023prolificdreamer}
Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu.
\newblock Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation.
\newblock In {\em NeurIPS}, 2023.
\bibitem{wang2004image}
Zhou Wang, Alan~C Bovik, Hamid~R Sheikh, and Eero~P Simoncelli.
\newblock Image quality assessment: from error visibility to structural similarity.
\newblock {\em IEEE Transactions on Image Processing}, 13(4):600--612, 2004.
\bibitem{Wu_2024_CVPR}
Rongyuan Wu, Tao Yang, Lingchen Sun, Zhengqiang Zhang, Shuai Li, and Lei Zhang.
\newblock Seesr: Towards semantics-aware real-world image super-resolution.
\newblock In {\em CVPR}, pages 25456--25467, June 2024.
\bibitem{wu2024seesr}
Rongyuan Wu, Tao Yang, Lingchen Sun, Zhengqiang Zhang, Shuai Li, and Lei Zhang.
\newblock Seesr: Towards semantics-aware real-world image super-resolution.
\newblock In {\em CVPR}, pages 25456--25467, 2024.
\bibitem{zhang2018unreasonable}
Richard Zhang, Phillip Isola, Alexei~A Efros, Eli Shechtman, and Oliver Wang.
\newblock The unreasonable effectiveness of deep features as a perceptual metric.
\newblock In {\em CVPR}, 2018.
\bibitem{zilberstein2025repulsive}
Nicolas Zilberstein, Morteza Mardani, and Santiago Segarra.
\newblock Repulsive latent score distillation for solving inverse problems.
\newblock In {\em ICLR}, 2025.
\end{thebibliography}