Beyond the Ground Truth: Enhanced Supervision for Image Restoration
Extracted source text · arXiv 2512.03932
% CVPR 2026 Paper Template; see https://github.com/cvpr-org/author-kit
\documentclass[10pt,twocolumn,letterpaper]{article}
%%%%%%%%% PAPER TYPE - PLEASE UPDATE FOR FINAL VERSION
% \usepackage{cvpr} % To produce the CAMERA-READY version
% \usepackage[review]{cvpr} % To produce the REVIEW version
\usepackage[pagenumbers]{cvpr} % To force page numbers, e.g. for an arXiv version
% Import additional packages in the preamble file, before hyperref
\input{preamble}
% It is strongly recommended to use hyperref, especially for the review version.
% hyperref with option pagebackref eases the reviewers' job.
% Please disable hyperref *only* if you encounter grave issues,
% e.g. with the file validation for the camera-ready version.
%∑
% If you comment hyperref and then uncomment it, you should delete *.aux before re-running LaTeX.
% (Or just hit 'q' on the first LaTeX run, let it finish, and you should be clear).
\definecolor{cvprblue}{rgb}{0.21,0.49,0.74}
\usepackage[pagebackref,breaklinks,colorlinks,allcolors=cvprblue]{hyperref}
\usepackage[table]{xcolor}
\usepackage{amsmath}
\usepackage{amsfonts}
\usepackage{bm}
\usepackage{graphicx}
\usepackage{subcaption}
\usepackage{booktabs}
\usepackage{multirow}
\usepackage{wrapfig}
\usepackage{float}
\usepackage{placeins}
\usepackage{nicefrac}
\usepackage{microtype}
\usepackage{tikz}
\usepackage{url}
\usepackage{hyperref}
\newcommand{\bhb}[1]{{\textcolor{blue}{#1}}}
\newcommand{\bhr}[1]{{\textcolor{red}{#1}}}
\newcommand{\dhr}{\textcolor{orange}}
%%%%%%%%% PAPER ID - PLEASE UPDATE
\def\paperID{8356} % *** Enter the Paper ID here
\def\confName{CVPR}
\def\confYear{2026}
%%%%%%%%% TITLE - PLEASE UPDATE
\title{Beyond the Ground Truth: Enhanced Supervision for Image Restoration}
%%%%%%%%% AUTHORS - PLEASE UPDATE
\author{Donghun Ryou$^{*2}$ \qquad Inju Ha$^{*1}$ \qquad Sanghyeok Chu$^{1}$ \qquad Bohyung Han$^{1,2}$ \\
Computer Vision Laboratory, $^1$ECE \& $^2$IPAI, Seoul National University\\
{\tt\small \{dhryou, hij1112, sanghyeok.chu, bhhan\}@snu.ac.kr}\\
}
\begin{document}
\maketitle
\begingroup
\renewcommand\thefootnote{} % Remove the footnote number
\footnotetext{\protect\hspace{-0.5em}${}^*$indicates equal contribution.}
\endgroup
% \input{sec/0_abstract}
% \input{sec/1_intro}
% \input{sec/2_formatting}
% \input{sec/3_finalcopy}
% \input{Tables/tmp.tex}
\input{Sections/Abstract.tex}
\input{Sections/Introduction.tex}
\input{Sections/RelatedWorks.tex}
\input{Sections/Method.tex}
\input{Sections/Experiment.tex}
\input{Sections/Conclusion.tex}
\par
{
\small
\bibliographystyle{ieeenat_fullname}
\bibliography{main}
}
\clearpage
\appendix
\setcounter{section}{0}
\setcounter{figure}{0}
\setcounter{table}{0}
\renewcommand{\thetable}{S.\arabic{table}}
\renewcommand{\thefigure}{S.\arabic{figure}}
\maketitlesupplementary
\input{Sections/Appendix.tex}
% WARNING: do not forget to delete the supplementary pages from your submission
% \input{sec/X_suppl}
\end{document}
%% This file contains a number of tweaks that are typically applied to the main document.
%% They are not enabled by default, but can be enabled by uncommenting the relevant lines.
%%
%% Inline annotations; for predefined colors, refer to "dvipsnames" in the xcolor package:
%% https://tinyurl.com/overleaf-colors
%%
\newcommand{\red}[1]{{\color{red}#1}}
\newcommand{\todo}[1]{{\color{red}#1}}
\newcommand{\TODO}[1]{\textbf{\color{red}[TODO: #1]}}
%%
%% disable for camera ready / submission by uncommenting these lines
%%
% \renewcommand{\TODO}[1]{}
% \renewcommand{\todo}[1]{#1}
%%
%% work harder in optimizing text layout. Typically shrinks text by 1/6 of page, enable
%% it at the very end of the writing process, when you are just above the page limit
%%
% \usepackage{microtype}
%%
%% fine-tune paragraph spacing
%%
% \renewcommand{\paragraph}[1]{\vspace{.5em}\noindent\textbf{#1.}}
%%
%% globally adjusts space between figure and caption
%%
% \setlength{\abovecaptionskip}{.5em}
%%
%% Allows "the use of \paper to refer to the project name"
%% with automatic management of space at the end of the word
%%
% \usepackage{xspace}
% \newcommand{\paper}{ProjectName\xspace}
%%
%% Commonly used math definitions
%%
% \DeclareMathOperator*{\argmin}{arg\,min}
% \DeclareMathOperator*{\argmax}{arg\,max}
%%
%% Tigthen underline
%%
% \usepackage{soul}
% \setuldepth{foobar}
% !TEX root = ./../main.tex
\begin{abstract}
Deep learning-based image restoration has achieved significant success.
However, when addressing real-world degradations, model performance is limited by the quality of ground-truth images in datasets due to practical constraints in data acquisition.
To address this limitation, we propose a novel framework that enhances existing ground truth images to provide higher-quality supervision for real-world restoration.
Our framework generates perceptually enhanced ground truth images using super-resolution by incorporating adaptive frequency masks, which are learned by a conditional frequency mask generator.
These masks guide the optimal fusion of frequency components from the original ground truth and its super-resolved variants, yielding enhanced ground truth images.
This frequency-domain mixup preserves the semantic consistency of the original content while selectively enriching perceptual details, preventing hallucinated artifacts that could compromise fidelity.
The enhanced ground truth images are used to train a lightweight output refinement network that can be seamlessly integrated with existing restoration models.
Extensive experiments demonstrate that our approach consistently improves the quality of restored images.
We further validate the effectiveness of both supervision enhancement and output refinement through user studies.
Code is available at \url{https://github.com/dhryougit/Beyond-the-Ground-Truth}.
\end{abstract}
% !TEX root = ./../main.tex
\vspace{-2mm}
\section{Introduction}
% \input{Figures/mixup_analysis.tex}
\input{Figures/enhanced_gt_intro.tex}
\label{Introduction}
%\shm{Background: Recent progress in image restoration.}
Image restoration has achieved remarkable progress through supervised training on paired low-quality and ground truth images using deep neural networks.
Across various degradation types, a range of architectures~\citep{zhang2017beyond, kupyn2018deblurgan, liang2021swinir, chen2022simple, zamir2022restormer, guo2024mambair} and learning strategies~\citep{lehtinen2018noise2noise, ulyanov2018deep, yoo2020rethinking, zhang2022self, wu2024id} have been proposed to align restored outputs closely with ground truth images.
Recently, the focus has shifted toward improving perceptual quality of the restored outputs, leveraging advances in generative models to produce visually compelling results~\citep{wang2024exploiting, lin2024diffbir, yu2024scaling, wu2024seesr}.
% \input{Figures/gt_perceptual_hist.tex}
%\shm{Problem: Suboptimal ground truth quality of real-world restoration datasets.}
Despite these advances, improving perceptual quality remains a major challenge in real-world image restoration, where obtaining ideal reference images is inherently difficult due to practical data acquisition constraints.
Many existing datasets rely on indirect ways to construct ground truth images.
For instance, in deblurring datasets~\citep{nah2017deep, shen2019human, nah2019ntire}, ground truth images are selected from video sequences, which often contain slight camera shake or object movements, limiting the image sharpness.
Likewise, in denoising datasets~\citep{nam2016holistic, abdelhamed2018high, xu2018real}, ground truth images are constructed by averaging multiple noisy captures, often resulting in blurred references.
As a result, models trained on such suboptimal ground truth images inevitably tend to inherit those imperfections, limiting their ability to achieve high-quality restoration.
%\shm{Proposed method: GT enhancement framework using SR and frequency-domain mixup.}
To address this limitation, we propose a novel supervision enhancement framework designed to improve the perceptual quality of suboptimal ground truth images.
The proposed framework consists of two main components: (1) super-resolution using a one-step diffusion model to generate perceptually enhanced ground truth variants, and (2) frequency-domain mixup to produce the final enhanced ground truth images.
For the frequency-domain mixup, we introduce a conditional frequency mask generator that adaptively produces masks to guide the optimal fusion of frequency components from the original ground truth image and its super-resolved variants.
%As illustrated in Figure~\ref{fig:edge_score}, the resulting enhanced ground truth images exhibit significantly improved visual clarity and achieve higher perceptual quality scores compared to the original references.
As illustrated in Figure~\ref{fig:enhnaced_gt}, the resulting enhanced ground truth image provides clearer details and higher perceptual quality than the original.
%\shm{Proposed refinement network.}
Building upon the enhanced ground truth images, we design a lightweight output refinement network that can be seamlessly integrated into a wide range of pretrained restoration models without requiring architectural changes or retraining.
%\shm{Experimental results.}
%Experiments show that the proposed refinement network consistently improves perceptual quality \shm{while preserving fidelity}, benefiting from the enhanced supervision provided by our framework.
Experiments show that the refinement network consistently improves the quality of restored images, benefiting from the enhanced supervision provided by our framework.
%Experiments show that the proposed refinement network consistently improves \shb{the quality of restored images}, benefiting from the enhanced supervision provided by our framework.
Moreover, the network exhibits strong robustness in out-of-distribution scenarios, effectively removing residual degradations that remain after initial restoration.
%User studies further validate the perceptual superiority of both the enhanced ground truth images and the refined outputs.
User studies further confirm the superior quality of both the enhanced ground truth images and the refinement outputs.
In summary, our key contributions are as follows:
\begin{itemize}
\item We identify the limitations of conventional ground truth images as a critical bottleneck in real-world image restoration, and propose a supervision enhancement framework based on frequency-domain mixup of an original ground truth image and its super-resolved variants. This design preserves semantic fidelity while enriching perceptual details, resulting in more reliable supervisory signals.
\item We introduce a lightweight refinement network that is trained solely on the original and enhanced ground truth images, requiring no additional annotations. The module is model-agnostic, seamlessly integrating with arbitrary restoration backbones without architectural modifications or retraining, and is empirically shown to be robust even under out-of-distribution degradations.
\item We validate our approach through extensive experiments and user studies, demonstrating consistent improvements in both enhanced ground truth quality and restored image fidelity.
%The results confirm the effectiveness of our framework across diverse degradations and highlight its superiority in human perceptual assessments.
\end{itemize}
\iffalse
%\shm{Summary}
In summary, our key contributions are organized as follows:
\begin{itemize}
\item We identify suboptimal ground truth images as a key bottleneck of poor perceptual quality outputs in real-world image restoration.
\item We propose a supervision enhancement framework that enriches perceptual details while preserving the semantic consistency of original ground truth images, leveraging diffusion-based super-resolution and frequency-domain mixup.
\item We design a lightweight, model-agnostic and task-agnostic output refinement network that can be seamlessly integrated into existing restoration models without retraining.
% \item \ijb{We demonstrate that our output refinement network exhibits strong robustness in out-of-distribution scenarios, effectively removing residual degradations that remain after initial restoration.}
% \item We validate the effectiveness of our approach through extensive experiments and user studies, demonstrating significant perceptual improvements across multiple benchmarks \shm{while maintaining fidelity}.
%\item \shb{We validate the effectiveness of our approach through extensive experiments and user studies, demonstrating consistent improvements in restoration quality across diverse settings.}
\item We validate our approach through extensive experiments and user studies, demonstrating consistent improvements across diverse degradations.
\end{itemize}
\fi
% !TEX root = ./../main.tex
\begin{figure}[t]
\centering
\scalebox{0.9}{
\begin{minipage}{\linewidth}
\begin{subfigure}[t]{\linewidth}
\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\scalebox{1}{
\includegraphics[width=\linewidth]{Figures/enhanced_gt/original.png}
}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=3pt, yshift=-0.5pt] at (image.north west) {\small Original GT};
\node[anchor=north east, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=-3pt, yshift=-0.5pt, align=right] at (image.north east) {\small MUSIQ: 47.82 \\ \small VisualQuality-R1: 3.50};
\end{tikzpicture}
\vspace{-3mm}
\end{subfigure}
\begin{subfigure}[t]{\linewidth}
\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\scalebox{1}{
\includegraphics[width=\linewidth]{Figures/enhanced_gt/enhanced.png}
}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=3pt, yshift=-0.5pt] at (image.north west) {\small Enhanced GT};
\node[anchor=north east, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=-3pt, yshift=-0.5pt, align=right] at (image.north east) {\small MUSIQ: 71.56 \\ \small VisualQuality-R1: 4.80};
\end{tikzpicture}
\end{subfigure}
% \hfill
% \begin{subfigure}[t]{0.32\linewidth}
% % \includegraphics[width=\linewidth]{Figures/detection/comparison_GOPR0384_11_05-004001_ours0.3.png}
% \includegraphics[width=\linewidth]{Figures/detection/comparison_GOPR0384_11_03-002198_osediff.png}
% \centering OSEDiff
% \vspace{-3mm}
% \end{subfigure}
% \vspace{2mm}
\vspace{-6mm}
\end{minipage}
}
\caption{
Visualization of our enhanced ground truth (GT).
Our enhanced GT not only demonstrates sharper text and superior perceptual quality but also maintains semantic consistency with respect to the original GT.
Zoom in for better visualizaiton.
}
\label{fig:enhnaced_gt}
\vspace{-5mm}
\end{figure}
% !TEX root = ./../main.tex
\section{Related Works}
\label{sec:relatedworks}
%\subsection{Image Restoration}
With the rise of deep learning, traditional image restoration methods have largely been replaced by data-driven approaches trained on paired low-quality and ground truth images.
A wide range of architectures has been proposed, including Convolutional Neural Networks~\citep{dong2014learning, zhang2017beyond, chen2022simple}, Transformer-based models~\citep{zamir2020learning, liang2021swinir, zamir2022restormer}, Generative Adversarial Networks~\citep{ledig2017photo, kupyn2018deblurgan}, and more recently, state-space models such as Mamba~\citep{guo2024mambair, guo2024mambairv2}.
These models have typically been trained to maximize metrics such as PSNR and SSIM, which quantify the pixel-wise similarity to the original ground truth images.
While these approaches achieve high performance on standard benchmarks, their outputs often lack perceptual realism of high-quality images.
%\subsection{Improving Perceptual Quality in Image Restoration}
Recently, there is a growing interest in enhancing the perceptual quality of restored images.
This has been particularly prominent in image super-resolution, demonstrating significant advancements in generating visually plausible high-frequency details~\citep{xia2023diffir, delbracio2023inversion, wang2024exploiting, lin2024diffbir, yu2024scaling, wu2024seesr}.
This trend extends to broader restoration tasks, such as deblurring and denoising, where diffusion models have been leveraged to enhance perceptual quality~\citep{ohayon2021high, kawar2022denoising, luo2023image, zhu2023denoising, yue2024efficient, liu2025one}.
While these methods effectively enhance perceptual quality, they often incur significant inference overhead and, more critically, risk hallucinating details or textures absent in the ground truth.
% \ijb{
In contrast, our approach aims to enhance perceptual quality while preserving the semantics of the original content by our novel frequency mixup strategy.
To assess the perceptual quality of restored images, we employ a combination of deep-learning based image quality assessments~\citep{ke2021musiq, yang2022maniqa, chen2024topiq, zhang2023blind, chen2025toward}, and emerging Vision-Language Model-based methods~\citep{wu2025visualquality,li2025q}.
% }
% \shm{
% Instead of relying on traditional pixel-wise measures, we verify semantic integrity using downstream tasks such as object detection and segmentation to ensure fidelity.
% Our assessment also includes perceptual similarity metrics (LPIPS~\citep{zhang2018unreasonable}, DISTS~\citep{ding2020image}), no-reference image quality assessments~\citep{ke2021musiq, yang2022maniqa, chen2024topiq, zhang2023blind}, and emerging Vision-Language Model-based methods~\citep{wu2025visualquality,li2025q}.
% This approach offers a more holistic perspective on restoration performance.
% }
% !TEX root = ./../main.tex
\section{Supervision Enhancement Framework}
In this section, we introduce our supervision enhancement framework, which improves the perceptual quality of ground truth images in existing datasets to provide better supervision for image restoration tasks.
The framework consists of two main components: (1) super-resolution using a one-step diffusion model to generate perceptually enhanced ground truth variants, and (2) combining these variants with the original ground truth image through frequency-domain mixup using masks generated by a conditional frequency mask generator.
Figure~\ref{fig:overview} (a) illustrates an overview of our framework.
\input{Figures/overview.tex}
\subsection{Image Enhancement with Super-Resolution}
%\ijb{ISR enhances perceptual quality:}
Recent Image super-resolution (ISR) models have shown remarkable capability in improving perceptual quality.
%These models are trained using a combination of reconstruction and regularization losses, where the regularization term is crucial in learning natural image distributions and improving perceptual richness.
These models are trained using a combination of reconstruction and regularization losses, where the regularization term is crucial in learning natural image distributions and improving output quality.
Typically, ISR models are trained to align the distribution of generated samples $q(\hat{x})$ with the distribution of high-quality real images $p(x_H)$, by minimizing the Kullback-Leibler divergence:
%
\begin{equation}
\mathcal{D}_\text{KL}(q(\hat{x})||p(x_H)).
\end{equation}
%
% Typically, the distribution $p(x_H)$ is acquired from datasets with genuinely high-quality images, such as DIV2K~\citep{agustsson2017ntire} or LSDIR~\citep{li2023lsdir}, or by leveraging the high-quality image manifold of large-scale pre-trained diffusion models~\citep{rombach2022high}.
Typically, the distribution $p(x_H)$ is acquired from datasets with genuinely high-quality images, such as DIV2K~\citep{agustsson2017ntire} or LSDIR~\citep{li2023lsdir}, or by leveraging the high-quality image manifold of large-scale pre-trained diffusion models.
%As a result, ISR models trained on this regularization effectively generate super-resolved outputs \shm{$\hat{x}_H$} of high perceptual quality.
As a result, ISR models trained on this regularization effectively generate super-resolved outputs {$\hat{x}$} of high perceptual quality.
%Leveraging this capability, we adopt an one-step diffusion ISR model~\citep{wu2024one} to improve the perceptual quality of the suboptimal ground truth images.
Leveraging this capability, we adopt an one-step diffusion ISR model~\citep{wu2024one} to enhance the suboptimal ground truth images.
%
Specifically, each original ground truth image $I_0^{\text{GT}}$ is first upsampled using bicubic interpolation with $N$ multiple scale factors.w
%Then, the diffusion-based ISR model is applied to these upsampled images, generating perceptually enhanced variants.
%Then, the diffusion-based ISR model is applied to these images, and the outputs are downsampled back to the original resolution.
%This result a set of perceptually improved ground truth images $\{I_i^{\text{GT}}\}_{i=1}^{N}$.
%Finally, these enhanced variants are downsampled back to the original resolution, resulting in a set of perceptually improved ground truth images $\{I_i^{\text{GT}}\}_{i=1}^{N}$.
The diffusion-based ISR model is then applied to these upsampled images, and the outputs are downsampled back to the original resolution, yielding a set of perceptually improved ground truth variants $\{I_i^{\text{GT}}\}_{i=1}^{N}$.
\subsection{Frequency-Domain Mixup}
% Although enhancing original ground truth images with ISR can improve perceptual quality, it often inadvertently alters the content due to the generative nature of super-resolution models.
% To address this issue, we combine the original ground truth image with its super-resolved variants.
% Since the super-resolved images deviate from the originals in both color tone and semantic details, pixel-wise mixing becomes difficult to optimize and often introduces unrealistic artifacts.
% Therefore, we propose an adaptive frequency mixup that provides fine-grained control, preserving essential low-frequency structures from the original image while selectively integrating perceptually richer high-frequency details from the enhanced variants.
\iffalse
Although enhancing ground truth images with ISR improves perceptual quality, it often inadvertently alters the content due to the generative nature of super-resolution models.
To mitigate these distortions, one may consider combining the original ground truth image with its super-resolved variants.
However, since the super-resolved images deviate from the originals in both color tone and semantic details, pixel-wise mixing is difficult to optimize and often introduces unrealistic artifacts.
To overcome this limitation, we propose an adaptive frequency mixup that enables fine-grained control, preserving essential low-frequency structures from the original image while selectively incorporating perceptually richer high-frequency details from the enhanced variants.
\fi
Although image super-resolution (ISR) can enhance perceptual quality, the generative nature of ISR models often introduces undesirable distortions in both semantic structures and photometric attributes.
To address this, we construct enhanced ground-truth images by integrating the original ground truth with multiple super-resolved variants.
A pixel-wise fusion in the spatial domain is problematic, since it amounts to selecting or averaging pixel intensities across images, making it difficult to preserve high-level semantic structures and frequently introducing unrealistic artifacts.
In contrast, we propose an adaptive frequency mixup, which provides fine-grained control by preserving essential low-frequency components in the original image while selectively incorporating perceptually richer high-frequency details from the super-resolved variants.
This frequency-domain formulation is particularly suitable for image restoration tasks as it naturally harmonizes images with differing photometric characteristics, yielding more stable and visually coherent results than spatial-domain alternatives.
% Although enhancing original ground truth images with ISR can improve perceptual quality, it often inadvertently alters the content due to the generative nature of super-resolution models.
% To mitigate this issue, it is necessary to combine the original ground truth image with its super-resolved variants.
% A straightforward pixel-wise mixing, however, is difficult to optimize and frequently introduces unrealistic artifacts.
% To overcome this limitation, we perform the combination in the frequency domain, which enables fine-grained control by preserving essential low-frequency structures from the original image while selectively incorporating perceptually richer high-frequency details from the enhanced variants.
To facilitate optimal frequency fusion, we introduce a Conditional Frequency Mask Generator.
As illustrated in the Figure~\ref{fig:overview}~(b), given a set of input images $\{I_i^{\text{GT}}\}_{i=0}^{N}$, where $i=0$ denotes the original ground truth and $i=1,\dots,N$ denote its super-resolved variants, the mask generator outputs frequency masks $M_i$ by combining a set of predefined ring-shaped Gaussian basis masks $\{R_b\}_{b=1}^{B}$, and predicted coefficients for each basis.
The design of our ring-shaped Gaussian basis masks is crucial for two reasons.
First, the ring-shape enables precise control from low to high frequencies in a band-wise manner.
Second, the Gaussian shape ensures smooth transitions between frequencies, unlike discrete masks that introduce sharp boundaries, causing training instability and visual artifacts.
Specifically, each basis mask $R_b\in\mathbb{R}^{H\times W}$ is defined as:
\begin{equation}
(R_b)_{h,w} = \exp\left(-(d(h,w)-\mu_b)^2 / 2\sigma_b^2 \right),
% \vspace{-1mm}
\end{equation}
for $1 \leq h \leq H$ and $1 \leq w \leq W$, where $d(h,w)$ denotes the $\ell_2$-distance from the center (DC component), and $\mu_b, \sigma_b$ represent the Gaussian parameters of the $b$-th mask.
The visualization of these basis masks is in the Appendix.
%Given the original ground truth image $I_0^{\text{GT}}$ and the conditional parameter $\lambda$ that balances fidelity and perceptual quality, a mask coefficient prediction network $g$ predicts coefficients $c_{i,b}\in\mathbb{R}$ as follows:
Given the original ground truth image $I_0^{\text{GT}}$ and the conditional parameter $\lambda$ that adjusts the weight between the original and its variants, a mask coefficient prediction network $g$ predicts coefficients $c_{i,b}\in\mathbb{R}$ as follows:
\begin{equation}
% \vspace{-1mm}
c_{i,b}=g(I_0^{\text{GT}}\hspace{-1mm},~\lambda).
\end{equation}
Internally, $g$ augments the RGB input with its FFT representation, enabling joint use of spatial and frequency-domain information for mask coefficient prediction.
Then, the adaptive frequency masks $M_i$ are computed by combining these bases using predicted coefficients:
\begin{equation}
M_i=\text{softmax}_i\Big(\sum_{b=1}^{B}c_{i,b}R_b\Big),
\end{equation}
where the softmax operation ensures masks sum to one, $\sum_{i=0}^{N} M_i(h,w)=1, \forall (h,w)$.
Finally, the enhanced ground truth image $\hat{I}^{\text{GT}}$ is constructed by fusing frequency components of the original ground truth $I_{0}^{\text{GT}}$ and its super-resolved variants $\{I_i^{\text{GT}}\}_{i=1}^{N}$ through frequency-domain mixup:
\begin{equation}
% \widehat{I}_{\text{GT}} = \mathcal{F}^{-1}\left(\sum_{i=0}^{N} M_i \odot \mathcal{F}(I_i^{\text{GT}})\right),
\hat{I}^{\text{GT}} = \mathcal{F}^{-1}\Big(\sum_{i=0}^{N} M_i \odot \mathcal{F}(I_i^{\text{GT}})\Big),
\end{equation}
where $\mathcal{F}$ and $\mathcal{F}^{-1}$ denote Fourier and inverse Fourier transforms, and $\odot$ represents element-wise multiplication.
\subsection{Optimization}
%To effectively predict the mask coefficients for optimal frequency fusion described above, we train the network $g$, which predicts coefficients for each basis, by employing a composite loss function consisting of two terms: a reconstruction term and a perceptual quality term.
To predict the mask coefficients for optimal frequency fusion, we train the network $g$, which predicts coefficients for each basis, with a composite loss that balances semantic integrity and perceptual quality.
%\dhr{The fidelity term} is defined by the pixel-wise $\ell_2$-loss with respect to the original ground truth $I_0^{\text{GT}}$:
The reconstruction loss is a $\ell_2$-loss that enforces consistency with the the original ground truth $I_0^{\text{GT}}$:
\begin{equation}
%\mathcal{L}_{\text{fidelity}} = \|\hat{I}^{\text{GT}} - I_0^{\text{GT}}\|_2^2.
\mathcal{L}_{\text{recon}} = \|\hat{I}^{\text{GT}} - I_0^{\text{GT}}\|_2^2.
\end{equation}
%Perceptual quality is measured by combining multiple no-reference IQA metrics (\eg MUSIQ~\citep{ke2021musiq}, MANIQA~\citep{yang2022maniqa}, TOPIQ~\citep{chen2024topiq}), denoted as $\text{IQA}_k(\cdot)$, to encourage perceptually richer results:
% \shb{
The perceptual loss is defined by a combination of multiple no-reference IQA metrics that evaluates perceptual quality of images (\eg MUSIQ~\citep{ke2021musiq},
MANIQA~\citep{yang2022maniqa}, TOPIQ~\citep{chen2024topiq}), denoted as $\text{IQA}_k(\cdot)$:
% }
\begin{equation}
\mathcal{L}_{\text{percep}} = -\sum_{k} \text{IQA}_k(\hat{I}^{\text{GT}}).
\end{equation}
%The overall loss for training the network $g$ is controlled by the parameter $\lambda$, which adjusts the balance between fidelity and perceptual enhancement:
The final training loss combines these two terms, with their relative weights controlled by $\lambda \in [0,1]$:
\begin{equation}
% \mathcal{L}=(1-\lambda) \mathcal{L}_{\text{fidelity}} +\lambda \mathcal{L}_{\text{percep}},
\mathcal{L}=(1-\lambda) \mathcal{L}_{{\text{recon}}} +\lambda \mathcal{L}_{\text{percep}}.
\end{equation}
%where smaller $\lambda$ emphasizes \shm{fidelity} and larger $\lambda$ emphasizes perceptual enhancement.
% \ijb{uniform sampling?}
\section{Output Refinement Network}
%\ijb{Propose a lightweight output refinement network:}
We demonstrate the effectiveness of our enhanced ground truth images by training a lightweight Output Refinement Network (ORNet) to improve the outputs of existing restoration models.
While training a full restoration network from scratch using low-quality inputs and enhanced ground truth images is possible, we observe that many state-of-the-art models are already well-optimized for original ground truth images.
Therefore, we propose an efficient strategy that builds on top of a fixed, pre-trained restoration model $R_\phi$.
Specifically, we introduce a modular output refinement network $R_\theta$, which is trained to refine the output of $R_\phi$.
The overall process is formulated as:
\begin{equation}
\hat{I} = R_\theta(R_\phi(I^\text{LQ}), \lambda),
\end{equation}
where $I^\text{LQ}$ is the low-quality input image, $\lambda$ is a parameter to control the level of perceptual enhancement, and $\hat{I}$ is the final restoration output.
%\ijb{Optimization:}
Since the pre-trained restoration model $R_\phi$ produces outputs close to the original ground truth (i.e., $R_\phi(I^{\text{LQ}}) \approx {I_0^{\text{GT}}}$), we train $R_\theta$ to map the $I_0^{\text{GT}}$, which resembles the output of $R_\phi$, toward the enhanced ground truth $\hat{I}^\text{GT}$ generated by our framework.
The training objective for $R_\theta$ is given by:
\begin{equation}
\mathcal{L}_\text{ref} = \| \hat{I} - \hat{I}^\text{GT} \|_2^2 \approx \| R_\theta(I_0^{\text{GT}}, \lambda) - \hat{I}^\text{GT} \|_2^2.
\end{equation}
This refinement strategy is model-agnostic, allowing it to be flexibly applied on top of various restoration models without architectural modifications.
% We show the effectiveness and versatility of our approach through extensive experiments.
\begin{figure*}[t]
\centering
\scalebox{0.93}{
\includegraphics[width=\linewidth]{Figures/overview/overview.pdf}
}
\vspace{-1mm}
\caption{
Overview of our framework.
(a) The supervision enhancement framework produces enhanced ground truth images by fusing frequency components from the original ground truth $I_0^{\text{GT}}$ and its super-resolved variants $I^{\text{GT}}_i$ using adaptive frequency masks $M_i$.
(b) The conditional frequency mask generator constructs $M_i$ by combining predefined masks $R_b$ weighted with predicted coefficients $c_{i,b}$, followed by a softmax function.
% (c) The output refinement network (ORNet) is trained to refine the restoration output to improve perceptual quality.
(c) The output refinement network (ORNet) is trained with these enhanced supervision to improve perceptual quality.
}
\label{fig:overview}
\vspace{-5mm}
\end{figure*}
% !TEX root = ./../main.tex
% \input{Tables/main_gopro.tex}
% \input{Tables/main_sidd.tex}
\input{Tables/main_exp.tex}
\input{Tables/deblurring_gopro_blur.tex}
\input{Tables/deblurring_gopro_noise.tex}
\section{Experiments}
\label{sec:exp}
In this section, we evaluate the perceptual quality of both the enhanced ground truth images, and the outputs of our refinement network, through comprehensive quantitative and qualitative experiments.
In addition, we conduct user studies to assess the perceptual validity of both the enhanced ground truth and the refined outputs.
\subsection{Experimental Settings}
\paragraph{Implementation Details}
For the supervision enhancement framework, we adopt OSEDiff~\citep{wu2024one} as the super-resolution network.
We generate three super-resolved ground truth variants using scaling factors of 2, 3, and 4.
The number of predefined masks, B, for constructing the final mask is set to 25.
% Detailed visualizations of these basis masks are provided in the Appendix.
Both the mask coefficient prediction network $g$ and the output refinement network (ORNet) are built using NAFBlocks, following the architectural design of NAFNet~\citep{chen2022simple}.
Specifically, $g$ consists of 4 NAFBlocks and 2 FC layers, while ORNet is built as a U-Net architecture of 4 encoder blocks, 1 middle block, and 4 decoder blocks.
Finally, the weighting parameter $\lambda$ used in our refinement formulation is set to 0.3 during evaluation.
A detailed description of how $\lambda$ is applied, along with additional evaluation results for different $\lambda$ values, is provided in the Appendix.
% \vspace{-2mm}
\paragraph{Training Details}
We train two core networks: the mask coefficient prediction network $g$ and ORNet, using a combined dataset of GoPro~\citep{nah2017deep} and SIDD~\citep{abdelhamed2018high}.
Both $g$ and ORNet are trained for 100$K$ iterations with a batch size of 8, using random $512 \times 512$ crops.
AdamW~\citep{loshchilov2018decoupled} optimizer with cosine annealing learning rate scheduler is used.
The initial learning rate is $1 \times 10^{-4}$ for $g$, and $3 \times 10^{-4}$ for ORNet.
% The parameter $\lambda$ is uniformly sampled from the range $[0, 1]$ during training to support learning of diverse enhancement levels, and is set to 0.3 during evaluation.
The parameter $\lambda$ is uniformly sampled from $[0, 1]$ during training to support learning of diverse enhancement levels.
% Inspired by recent NTIRE challenge~\citep{chen2025ntire} which uses various metrics to assess the restoration, we employ a comprehensive set of metrics, categorized into three groups~\citep{zhang2025augmenting}: low-level reference metrics, mid-level reference metrics, and no-reference metrics.
% The low-level reference metrics include PSNR and SSIM.
% For mid-level reference, we utilize LPIPS~\citep{zhang2018unreasonable} and DISTS~\citep{ding2020image}.
% No-reference perceptual quality is assessed using MUSIQ~\citep{ke2021musiq}, MANIQA~\citep{yang2022maniqa}, TOPIQ~\citep{chen2024topiq} and LIQE~\citep{zhang2023blind}.
% For each category, we compute the average performance across the respective metrics.
% The final evaluation score is derived by summing the averaged scores from all three categories:
% This evaluation strategy enables a holistic assessment of both the fidelity and perceptual quality of the restored images, effectively balancing from low-level distortions to high-level perceptual attributes.
% \input{Tables/deblurring_realblur.tex}
% \input{Tables/segmentation_full.tex}
% \dhr{
% To evaluate the perceptual quality of our supervision enhancement framework, we utilize KonIQ++~\citep{su2021koniq++} to measure the blur and noise levels of the images.
% To verify the fidelity of our enhanced ground truth, we assess its semantic integrity.
% This is measured using CLIP similarity, object detection, and segmentation performance between the original ground truth and our enhanced ground truth.
% }
% \dhr{
% % Additionally, to comprehensively assess the output refinement quality, we employ diverse metrics categorized into three groups: feature-level reference metrics, no-reference perceptual metrics, and Vision-Language Model (VLM) based metrics.
% % First, we assess the output refinement quality, we employ diverse metrics categorized into three groups: feature-level reference metrics, no-reference perceptual metrics, and Vision-Language Model (VLM) based metrics.
% % In the feature-level reference category, we utilize LPIPS~\citep{zhang2018unreasonable} and DISTS~\citep{ding2020image}.
% % For no-reference perceptual quality assessment, our evaluation includes MUSIQ~\citep{ke2021musiq}, MANIQA~\citep{yang2022maniqa}, TOPIQ~\citep{chen2024topiq}, and LIQE~\citep{zhang2023blind}.
% % Furthermore, we integrate cutting-edge VLM-based approaches, VisualQuality-R1~\citep{wu2025visualquality} and Q-Insight~\citep{li2025q}, which leverage reinforcement learning to achieve more human-aligned quality assessments.
% %We deliberately exclude conventional pixel-level metrics such as PSNR and SSIM.
% % This decision is in line with recent works, such as A-FINE~\citep{chen2025toward}, which argue that pixel-wise similarity to a potentially suboptimal ground truth may not accurately reflect true restoration quality.
% % This decision stems from the recognition that pixel-wise similarity to a potentially suboptimal ground truth may not accurately reflect true restoration quality.
% % Instead, to verify the fidelity, we assess the semantic integrity of our enhanced ground truth.
% To comprehensively evaluate the image quality, we employ diverse metrics.
% First, for assessing perceptual quality without a reference, we utilize a diverse set of no-reference metrics: MUSIQ~\citep{ke2021musiq}, MANIQA~\citep{yang2022maniqa}, TOPIQ~\citep{chen2024topiq}, and LIQE~\citep{zhang2023blind}.
% We also incorporate cutting-edge Vision-Language Model (VLM) based methods, including VisualQuality-R1~\citep{wu2025visualquality} and Q-Insight~\citep{li2025q}, which leverage reinforcement learning to achieve more human-aligned quality assessments.
% Furthermore, to provide a comprehensive analysis, we conduct a reference-based evaluation by measuring PSNR, SSIM, and LPIPS~\citep{zhang2018unreasonable} against both the original Ground Truth (GT) and our enhanced GT.
% }
% \vspace{-3mm}
% \paragraph{Evaluation Setup}
% We evaluate under two regimes, in-distribution (ID) and out-of-distribution (OOD).
% The ID regime corresponds to standard restoration setups with their provided GT images, while the OOD regime is constructed by applying additional synthetic degradations (\eg blur, noise) to test robustness.
% %
% In ID, the GT images are themselves suboptimal.
% %Reference-based metrics such as PSNR and SSIM only measure similarity to these imperfect GTs, and thus cannot capture the benefit of our approach, which produces outputs that may surpass the given GT quality.
% Reference-based metrics such as PSNR, SSIM, and LPIPS~\citep{zhang2018unreasonable} measure pixel- or feature-level similarity to the given GTs; however, when the GTs themselves are imperfect, these scores no longer provide a valid assessment of true restoration quality.
% For example, a higher PSNR against those suboptimal GTs does not necessarily indicate better restoration, and vice versa.
% %For this reason, in Tables~\ref{tab:gopro_refinement} and \ref{tab:sidd_refinement}, we report only no-reference perceptual metrics, MUSIQ~\citep{ke2021musiq}, MANIQA~\citep{yang2022maniqa}, TOPIQ~\citep{chen2024topiq}, LIQE~\citep{zhang2023blind}, together with two recent VLM-based IQA measures (VisualQuality-R1~\citep{wu2025visualquality}, Q-Insight~\citep{li2025q}), which better reflect human perception.
% % Therefore, for experiments under ID regime, we report only no-reference perceptual metrics, MUSIQ~\citep{ke2021musiq}, MANIQA~\citep{yang2022maniqa}, TOPIQ~\citep{chen2024topiq}, LIQE~\citep{zhang2023blind}, together with two recent VLM-based IQA measures (VisualQuality-R1~\citep{wu2025visualquality}, Q-Insight~\citep{li2025q}), which better reflect human perception.
% Therefore, for experiments under the ID regime, we primarily report no-reference perceptual metrics, including MUSIQ~\citep{ke2021musiq}, MANIQA~\citep{yang2022maniqa}, TOPIQ~\citep{chen2024topiq}, and LIQE~\citep{zhang2023blind}, together with two recent VLM-based IQA measures (VisualQuality-R1~\citep{wu2025visualquality}, Q-Insight~\citep{li2025q}), which better reflect human perception.
% \dhr{
% In addition, we include A-FINE~\citep{chen2025toward}, an adaptive fidelity-naturalness evaluator.
% Although A-FINE is reference-based, it offers a comprehensive evaluation by considering the naturalness of both images.
% Thus, we also measure the A-FINE scores between the Original GT and the model output.
% }
% %
\paragraph{Evaluation Setup}
We evaluate models under two regimes: in-distribution (ID) and out-of-distribution (OOD).
The ID regime corresponds to standard setups using the provided GT images, whereas the OOD regime is constructed by applying synthetic degradations (\eg blur, noise) to test robustness.
In the ID setting, the provided GT images are suboptimal.
This limits the reliability of standard reference-based metrics such as PSNR, SSIM, and LPIPS~\citep{zhang2018unreasonable}, which measure pixel- or feature-level similarity to these imperfect GTs.
A higher PSNR against such GTs does not necessarily indicate better restoration, and outputs that surpass the GT quality may even receive lower scores.
To mitigate this limitation, ~\citet{chen2025toward} introduced A-FINE, an adaptive fidelity–naturalness evaluator.
Although reference-based, it provides a more comprehensive assessment than traditional similarity metrics; therefore, we measure A-FINE between the original GT and the model output.
Nevertheless, we mainly report no-reference perceptual metrics—MUSIQ~\citep{ke2021musiq}, MANIQA~\citep{yang2022maniqa}, TOPIQ~\citep{chen2024topiq}, LIQE~\citep{zhang2023blind}—as well as two recent VLM-based IQA measures (VisualQuality-R1~\citep{wu2025visualquality}, Q-Insight~\citep{li2025q}), which better align with human perception.
In the OOD setting, restored outputs are often far worse than the original GT due to the additional degradations.
In this case, the GT—though imperfect—still provides a meaningful reference for evaluating fidelity.
Thus, we complement the perceptual metrics with reference-based measures (PSNR, SSIM, LPIPS) computed against both the original and enhanced GTs, providing a comprehensive evaluation of both fidelity and perceptual quality.
% \paragraph{Evaluation Metrics}
% \dhr{
% We divide our evaluation into two distinct scenarios: in-distribution and out-of-distribution.
% For the in-distribution evaluation on the standard test set, we argue that comparisons to a original suboptimal GT are redundant.
% Consequently, we exclusively assess performance using no-reference perceptual metrics (MUSIQ~\citep{ke2021musiq}, MANIQA~\citep{yang2022maniqa}, TOPIQ~\citep{chen2024topiq}, LIQE~\citep{zhang2023blind}) and cutting-edge Vision-Language Model (VLM) based methods (VisualQuality-R1~\citep{wu2025visualquality}, Q-Insight~\citep{li2025q}), which better align with human perception.
% Conversely, the OOD scenario is designed to analyze model robustness against severe, unseen degradations.
% In this heavily degraded context, the quality of a model's output is often substantially lower than that of the original GT.
% This significant quality gap re-establishes the GT—even if perceptually suboptimal—as a valid directional target, meaning reference-based metrics regain their significance as a meaningful surrogate measurement for fidelity.
% Therefore, for the OOD evaluation, we conduct a comprehensive analysis, measuring reference-based metrics (PSNR, SSIM, LPIPS~\citep{zhang2018unreasonable}) against both the original GT and our enhanced GT, alongside the full set of no-reference metrics to concurrently assess perceptual gains.
% }
% \dhr{
% \shm{In our method, the degree of perceptual quality enhancement can be controlled by a parameter, $\lambda$.
% For the evaluation, we carefully set $\lambda$ as 0.3, significantly increasing perceptual quality while preserving fidelity.
% Additional evaluation results for different $\lambda$ values are presented in the Appendix.}
% }
%
%
% \vspace{-4mm}
% \begin{equation}
% \label{eq:score}
% \small \text{Score} =
% \underbrace{\text{avg} \left( \frac{\text{PSNR}}{100} , \text{SSIM} \right)}_{\text{Low-level Ref.}} +
% \underbrace{\text{avg} \left( 1 - \text{LPIPS} , 1 - \text{DISTS} \right)}_{\text{Mid-level Ref.}} +
% \underbrace{\text{avg} \left( \frac{\text{MUSIQ}}{100} , \text{MANIQA} , \text{TOPIQ} , \frac{\text{LIQE}}{10} \right)}_{\text{No Ref.}}
% \end{equation}
%\input{Figures/koniq_analysis.tex}
%\input{Tables/fidelity.tex}
% \input{Figures/detection.tex}
%\input{Figures/enhanced_gt.tex}
\vspace{-1mm}
\subsection{Results}
%\subsubsection{Supervision Enhancement}
%\paragraph{Perceptual Quality}
%Figure~\ref{fig:score_histogram} shows the results of our supervision enhancement framework.
%We assess the quality of images using KonIQ++~\citep{su2021koniq++} blur level ($\downarrow$) and noise level ($\downarrow$).
%We observe that the ground truth (GT) images in the GoPro~\citep{nah2017deep} dataset, captured using single high-shutter-speed frames from action cameras, tend to be relatively noisy.
%In addition, the GT images in the SIDD~\citep{abdelhamed2018high} dataset exhibit high blur scores, indicating that the averaging process used to obtain GT images introduces blurriness.
%Our enhancement framework effectively improves the quality of such suboptimal GT images, reducing both blur and noise.
%\input{Figures/segmentation.tex}
%\paragraph{Fidelity}
%\dhr{
%To verify that our method preserves semantic fidelity, we conduct several experiments.
%First, on the original restoration datasets, GoPro testset, we measure the CLIP similarity between the original ground truth images and their enhanced counterparts.
%Furthermore, we assess semantic integrity using an object detection task.
%We employ a detection model YOLOv8 on the original GT images to generate pseudo-ground truth labels.
%These labels are then compared against the detection results obtained from our enhanced GT images.
%As shown in Table~\ref{tab:clip_detection_main}, our supervision enhancement achieves a CLIP similarity score nearing 1.0 and demonstrates a very high degree of similarity in object detection performance.
%This indicates that the fidelity loss from our method is negligible.
%The qualitative results in Figure~\ref{fig:detection} further support this, visualizing detection outcomes that are nearly identical to those from the original GT.
%In some cases, our enhancement appears to aid the model in detecting objects more clearly.
%}
%\dhr{
%Additionally, we extend our experiments to an external segmentation dataset, PASCAL VOC 2012 validation set~\citep{everingham2015pascal}.
%We apply our supervision enhancement to this dataset and compared the performance of a segmentation model, DeepLabv3~\citep{chen2017rethinking} on both the original and enhanced versions, measuring perceptual quality of the dataset using MUSIQ.
%The results, presented in Table~\ref{tab:segmentaion}, reveal that the segmentation model's performance remains nearly identical between the original and enhanced datasets, while the overall perceptual quality of the dataset shows a marked improvement.
%These experimental findings demonstrate that our method successfully elevates perceptual quality without compromising the semantic fidelity of the original data.
%}
%\subsubsection{Output Refinement Network}
% \subsubsection{In-distribution Quantitative results}
% % \paragraph{Quantitative results}
% %As our refinement strategy is model-agnostic, we integrate it into two base restoration models for each task: AdaRevD~\citep{mao2024adarevd} and FFTformer~\citep{kong2023efficient} for image deblurring, and NAFNet~\citep{chen2022simple} and Xformer~\citep{zhang2023xformer} for image denoising.
% As ORNet is model-agnostic \shb{and task-agnostic}, we integrate it into two base restoration models for each task: AdaRevD~\citep{mao2024adarevd} and FFTformer~\citep{kong2023efficient} for image deblurring, and NAFNet~\citep{chen2022simple} and Xformer~\citep{zhang2023xformer} for image denoising.
% Restormer~\citep{zamir2022restormer}, ResShift~\citep{yue2024efficient}, IR-SDE~\citep{luo2023image}, DiffIR~\citep{xia2023diffir}, and HIDiff~\citep{chen2024hierarchical} for image deblurring, with AP-BSN~\citep{lee2022ap}, MIRNet-v2~\citep{zamir2020learning}, and Restormer for image denoising are presented for comparison.
% Table~\ref{tab:gopro_refinement} and Table~\ref{tab:sidd_refinement} report the quantitative results on the GoPro deblurring and SIDD denoising datasets, respectively.
% % Models trained with $\ell_1$ or $\ell_2$-loss perform well on low-level reference metrics, which focus on pixel-wise fidelity.
% % Diffusion-based methods, ResShift and IR-SDE, excel in mid-level reference scores, which assess perceptual similarity in feature space, but they often struggle to faithfully reconstruct the original image.
% % Our output refinement network, ORNet, when applied to the outputs of high-fidelity models, significantly improves no-reference scores, further enhancing the perceptual quality of the restored images.
% \dhr{
% Models trained with $\ell_1$ or $\ell_2$-loss and diffusion-based models exhibit comparable performance in terms of both no-reference and VLM-based scores.
% This suggests that the restoration quality of existing models is upper-bounded by the quality of the original ground truth.
% In contrast, our method leverages an enhanced ground truth, thereby achieving a significant improvement in perceptual quality.
% We note that reference-based metrics are invalid for evaluation, as there are two different ground truths: the original and the enhanced.
% }
% \input{Figures/enhanced_gt.tex}
\input{Figures/qual_GOPRO.tex}
\input{Figures/qual_SIDD.tex}
\subsubsection{In-distribution Quantitative results}
Table~\ref{tab:main_exp} reports the quantitative results on the GoPro deblurring and SIDD denoising datasets, respectively.
As ORNet is model agnostic, we apply it on top of representative base models from two restoration tasks.
For image deblurring, we integrate ORNet into AdaRevD~\citep{mao2024adarevd} and FFTformer~\citep{kong2023efficient};
for image denoising, we use NAFNet~\citep{chen2022simple} and Xformer~\citep{zhang2023xformer}.
We compare against diverse state-of-the-art methods, including Restormer~\citep{zamir2022restormer}, ResShift~\citep{yue2024efficient}, IR-SDE~\citep{luo2023image}, DiffIR~\citep{xia2023diffir}, HIDiff~\citep{chen2024hierarchical} for deblurring, and AP-BSN~\citep{lee2022ap}, MIRNet-v2~\citep{zamir2020learning}, Restormer for denoising.
Models trained with $\ell_1$ or $\ell_2$-loss and diffusion-based models tend to exhibit low performance.
This suggests that the restoration quality of existing models is upper-bounded by the quality of the original ground truth.
In contrast, our method leverages an enhanced ground truth, thereby achieving a significant improvement in perceptual quality.
% We note that reference-based metrics are invalid for evaluation, as there are two different ground truths: the original and the enhanced.
% \dhr{
% We note that the decrease in LPIPS and DISTS scores is an expected outcome, as they are measured against the original GT.
% This drop is an unavoidable consequence of producing an enhanced GT that is perceptually superior to the original.
% However, this does not signify a loss of fidelity, as demonstrated in our experiments.
% Furthermore, the degree of enhancement can be controlled via the conditioning parameter, $\lambda$, allowing for results closer to the original GT if desired.
% An ablation study on $\lambda$ is included in the appendix due to space constraints.
% }
\subsubsection{Out-of-distribution Quantitative results}
\input{Figures/mask_analysis.tex}
% \paragraph{Generalization Performance}
% To evaluate the generalization performance of our refinement network, we conduct experiments on out-of-distribution (OOD) mixed degradations, including the SIDD~\citep{abdelhamed2018high} dataset with additional Gaussian Blur, and GoPro~\citep{nah2017deep} dataset with additional white noise.
% Our results demonstrate that the enhanced ground truth (GT) images produced by our method consistently improve the robustness of existing restoration models when combined with our refinement network.
% This indicates that our refinement network not only enhances the perceptual quality of the training data but also improves the robustness of the models when faced with unseen degradations.
% \dhr{
To evaluate the generalization performance of our refinement network, we conduct experiments in out-of-distribution (OOD) settings.
These are constructed by augmenting the inputs of the GoPro test set~\citep{nah2017deep} with additional, unseen degradations: one set with Gaussian blur and another with white noise.
We posit that existing state-of-the-art deblurring model FFTformer~\citep{kong2023efficient} overfit to the specific degradation characteristics of their GoPro training data, causing their performance to degrade sharply in such OOD conditions.
In contrast, our ORNet is not trained for a specific degradation; it robustly enhances the output of any given restoration model. As demonstrated in Tables~\ref{tab:gopro_blur} and~\ref{tab:gopro_noise}, applying ORNet leads to a substantial increase in perceptual quality. Simultaneously, reference-based metrics improve against both the original GT and our enhanced GT, which validates that ORNet also effectively preserves semantic details.
Additional analysis of the generalization performance of our ORNet is provided in the Appendix.
% \vspace{-2mm}
\subsubsection{Qualitative results}
% We present qualitative results highlighting two key aspects of our approach: (1) the enhanced ground truth images generated by our supervision framework, and (2) the restoration outputs refined by ORNet.
% As shown in Figure~\ref{fig:enhnaced_gt}, our enhanced GT preserves the semantic content of the original GT while providing sharper and more perceptually pleasing details.
% In Figures~\ref{fig:qual_GOPRO} and \ref{fig:sidd_qual}, we compare restored outputs on the GoPro and SIDD datasets.
% When combined with existing restoration models, ORNet consistently improves perceptual quality in fine details such as the cracks between stones in Figure~\ref{fig:qual_GOPRO} and the sharpness of text and edges in Figure~\ref{fig:sidd_qual}, yielding outputs with sharper and cleaner details than those in the original GT images.
We present qualitative results demonstrating the effectiveness of ORNet in refining restoration outputs.
Figures~\ref{fig:qual_GOPRO} and \ref{fig:sidd_qual} demonstrate comparisons on the GoPro and SIDD datasets, respectively.
When applied on top of existing restoration models, ORNet consistently enhances perceptual quality by sharpening fine structures—such as the cracks between stones in Figure~\ref{fig:qual_GOPRO} and the clarity of text and edges in Figure~\ref{fig:sidd_qual}.
Across diverse scenes, ORNet produces outputs with cleaner details and noticeably improved visual sharpness compared to those produced by the original restoration models.
\subsection{Ablation study}
\paragraph{Mask design}
% Figure~\ref{fig:mask_analysis} presents an analysis of frequency masks generated by two distinct methods: our conditional frequency mask generator with ring-shaped Gaussian basis, and an element-wise baseline in frequency domain.
% Our method generates frequency masks as a weighted combination of smooth Gaussian basis, promoting spatial coherence.
% The base mask $M_0$ is applied to the original ground truth and preserves low-frequency content, while the additional masks $M_1$, $M_2$, and $M_3$ selectively incorporate high-frequency components from super-resolved variants.
% The element-wise baseline in frequency domain uses a simple U-Net to directly predict mask values for each spatial cordinate in frequency domain.
% Although it exhibits a similar frequency selection tendency, the resulting masks are spatially inconsistent, often leading to artifacts in the final enhanced ground truth.
% Figure~\ref{fig:mask_analysis} presents a comparative analysis of frequency masks generated by three distinct methods: our conditional frequency mask generator with ring-shaped Gaussian basis, and an element-wise baseline in frequency domain and spatial domain.
% Our method generates frequency masks as a weighted combination of smooth Gaussian basis, promoting spatial coherence.
% The base mask $M_0$ is applied to the original ground truth and preserves low-frequency content, while the additional masks $M_1$, $M_2$, and $M_3$ selectively incorporate high-frequency components from super-resolved variants.
% This structured decomposition enables fine-grained and interpretable frequency control across spatial regions.
% In contrast, the element-wise baseline in frequency domain uses a simple U-Net to directly predict mask values for each spatial cordinate in frequency domain.
% Although it exhibits a similar frequency selection tendency, the resulting masks are spatially inconsistent, often leading to artifacts in the final enhanced ground truth.
% Additionally, the element-wise baseline in spatial domain utilizes the same network designed above with the only modification being that the mixup is performed in the spatial domain instead of the frequency domain.
% As shown in Figure~\ref{fig:mask_analysis}, employing element--wise mixup directly in the frequency and spatial domain leads to unstable mask generation.
% These artifacts arise in part from the use of perceptual loss guided by no-reference image quality assessment (IQA) models, which struggle to detect subtle unnatural distortions.
% As a result, relying solely on IQA-based perceptual loss makes it difficult to avoid such inconsistency-induced artifacts.
% Our Gaussian basis constraint allows stable control over frequency content, enabling effective use of perceptual loss while suppressing artifacts caused by spatial irregularities.
Figure~\ref{fig:mask_analysis} presents a comparative analysis of frequency masks generated by three approaches: our conditional frequency mask generator with ring-shaped Gaussian bases, an element-wise baseline in the frequency domain, and an element-wise baseline in the spatial domain.
Our method constructs frequency masks as a weighted combination of smooth Gaussian bases, promoting spatially coherent mask patterns.
The base mask $M_0$ preserves low-frequency content from the original ground truth, while the additional masks $M_1$, $M_2$, and $M_3$ selectively incorporate high-frequency components from the super-resolved variants.
This structured decomposition enables fine-grained and interpretable frequency control across spatial regions.
In contrast, the element-wise baseline in the frequency domain employs a simple U-Net to directly predict mask values for each spatial coordinate in the frequency domain.
Although it exhibits a similar frequency-selection tendency, the resulting masks lack spatial consistency, which often leads to artifacts in the enhanced ground truth.
Likewise, the element-wise baseline in the spatial domain uses the same network architecture but applies the mixup operation directly in the spatial domain, resulting in similarly unstable and irregular mask patterns.
These inconsistencies arise because element-wise mask prediction provides no structural constraints, making it difficult for perceptual-loss–based optimization to avoid subtle but unnatural distortions.
By contrast, our formulation imposes smoothness and structural coherence, enabling stable frequency control while suppressing artifacts caused by irregular spatial variations.
\input{Tables/full_finetuning.tex}
\input{Figures/koniq_analysis.tex}
\vspace{-3mm}
\paragraph{Finetuning with Enhanced Supervision}
Our enhanced supervision can also be applied by directly finetuning existing restoration models.
Table~\ref{tab:full_finetuning} summarizes the results. The first row (FFTformer) reports the performance of the model pretrained on the original GoPro dataset, while the second row shows the effect of applying our refinement network to its outputs.
FFTFormer* corresponds to the pretrained FFTformer finetuned with our enhanced supervision.
We observe that finetuning with our enhanced supervision enables existing restoration models to achieve substantially higher perceptual quality in no-reference metrics.
However, this strategy requires retraining each model with its own enhanced supervision.
In contrast, our modular refinement network serves as a plug-and-play component that can be attached to various restoration models, achieving superior performance without additional per-model finetuning.
\vspace{-1mm}
\subsection{Analysis}
\label{sec:analysis}
\vspace{-1mm}
\paragraph{KonIQ++ analysis}
Figure~\ref{fig:score_histogram} shows the results of our supervision enhancement framework.
We assess the quality of images using KonIQ++~\citep{su2021koniq++} blur level ($\downarrow$) and noise level ($\downarrow$).
We observe that the ground truth (GT) images in the GoPro~\citep{nah2017deep} dataset, captured using single high-shutter-speed frames from action cameras, tend to be relatively noisy.
In addition, the GT images in the SIDD~\citep{abdelhamed2018high} dataset exhibit high blur scores, indicating that the averaging process used to obtain GT images introduces blurriness.
Our enhancement framework effectively improves the quality of such suboptimal GT images, reducing both blur and noise.
\vspace{-1mm}
\paragraph{Hallucination issue}
Although naive diffusion-based super-resolution methods often introduce undesirable distortions and artifacts, our supervision-enhancement approach is robust to hallucination and effectively preserves semantic consistency with the original GT.
Figure~\ref{fig:mixed_sample} shows that, even though one of the super-resolved GTs contains substantial artifacts on the man's face, hands, and shirt pattern, our enhance GT through the proposed frequency-domain mixup strategy is almost free of such issues.
\input{Figures/mixed_sample.tex}
\input{Tables/clip_similarity.tex}
\input{Tables/macs.tex}
To validate this observation, we compute the cosine similarity of CLIP embeddings between the enhanced/super-resolved GTs and the original GT of the GoPro training set, and assess object detection performance using a pretrained YOLOv8 model.
For detection evaluation, we use the predictions obtained from the original GT as the reference.
As reported in Table~\ref{tab:clip_similarity}, our method achieves substantially higher semantic alignment (0.9976 vs.\ 0.8782) and more effectively preserves object-level cues (precision 0.9448 vs.\ 0.7599).
These results demonstrate that our approach enhances perceptual quality without sacrificing semantic fidelity, making it a more reliable choice for downstream tasks; additional results are provided in the Appendix.
% \paragraph{Ablation study on mask design}
% Figure~\ref{fig:mask_analysis} presents a comparative analysis of frequency masks generated by three distinct methods: our conditional frequency mask generator with ring-shaped Gaussian basis, and an element-wise baseline in frequency domain.
% Our method generates frequency masks as a weighted combination of smooth Gaussian basis, promoting spatial coherence.
% The base mask $M_0$ is applied to the original ground truth and preserves low-frequency content, while the additional masks $M_1$, $M_2$, and $M_3$ selectively incorporate high-frequency components from super-resolved variants.
% The element-wise baseline in frequency domain uses a simple U-Net to directly predict mask values for each spatial cordinate in frequency domain.
% Although it exhibits a similar frequency selection tendency, the resulting masks are spatially inconsistent, often leading to artifacts in the final enhanced ground truth.
\vspace{-3mm}
\paragraph{Computational Efficiency}
Table~\ref{tab:params_flops} summarizes the number of parameters and multiply–accumulate operations (MACs) of our refinement network and existing restoration models, computed with an input resolution of 512$\times$512 pixels.
Our refinement network (ORNet) is significantly more lightweight than the baseline restoration models, enabling easy integration into existing architectures with negligible computational overhead.
\vspace{-3mm}
\input{Figures/user.tex}
\paragraph{User study}
We conduct two user studies to evaluate the perceptual quality of our supervision enhancement and output refinement network.
Both studies involve 70 participants, each presented with 25 randomly sampled images from the GoPro dataset.
For supervision enhancement, participants are asked to compare the original ground truth (baseline) and enhanced ground truth (ours), based on how well each image appears to restore the low-quality input.
For the output refinement network, participants evaluate which output, FFTformer (baseline) or FFTformer + ORNet (ours), provides a better restoration of the low-quality input.
As shown in Figure~\ref{fig:user}, both our enhanced ground truth images and refinement outputs received significantly higher preference scores.
% \vspace{-1mm}
% \dhr{need to compare with directly using SR?}
\begin{table*}[!t]
\centering
\caption{
Evaluation of restoration performance of our ORNet on GoPro deblurring and SIDD denoising test sets.
}
\renewcommand{\arraystretch}{0.9}
\setlength{\tabcolsep}{12.5pt}
\vspace{-2mm}
\scalebox{0.8}{
\begin{tabular}{l l | cccc | cc | c}
\toprule
\rowcolor{gray!15}
& & \multicolumn{4}{c|}{\textit{Perceptual Quality Metrics}} & \multicolumn{2}{c|}{\textit{VLM-based Metrics}} & \\
\rowcolor{gray!15}
{} & {Method} & {MUSIQ}$\uparrow$ & {MANIQA}$\uparrow$ & {TOPIQ}$\uparrow$ & {LIQE}$\uparrow$ & {VisualQuality-R1}$\uparrow$ & {Q-Insight}$\uparrow$ & {A-FINE}$\downarrow$ \\
\midrule
\multirow{10.5}{*}{\rotatebox[origin=c]{90}{\large\textit{GoPro Deblurring}}} & \text{Restormer~\citep{zamir2022restormer}} & 45.05 & 0.5265 & 0.3346 & 1.5264 & 4.1246 & 3.4000 & 37.82 \\
& \text{NAFNet~\citep{chen2022simple}} & 45.33 & 0.5346 & 0.3368 & 1.5542 & 4.1539 & 3.4262 & 36.98 \\
& \text{ResShift~\citep{yue2024efficient}} & 44.30 & 0.4934 & 0.3127 & 1.4090 & 3.9105 & 3.3480 & 45.27 \\
& \text{IR-SDE~\citep{luo2023image}} & 46.13 & 0.5336 & 0.3410 & 1.6140 & 3.9735 & 3.3619 & 44.98 \\
& \text{DiffIR~\citep{xia2023diffir}} & 46.00 & 0.5366 & 0.3412 & 1.5820 & 4.1544 & 3.4269 & 37.14 \\
& \text{HI-Diff~\citep{chen2024hierarchical}} & 45.86 & 0.5337 & 0.3398 & 1.5576 & 4.1554 & 3.4207 & 37.21 \\
\cmidrule{2-9}
& \text{AdaRevD~\citep{mao2024adarevd}} & 45.49 & 0.5363 & 0.3393 & 1.5660 & 4.1737 & 3.4386 & 36.19 \\
& \text{+ ORNet \textbf{(Ours)}} & \textbf{64.25} & \textbf{0.5916} & \textbf{0.4880} & \textbf{2.4291} & \textbf{4.1952} & \textbf{3.5206} & \textbf{14.38} \\
\cmidrule{2-9}
& \text{FFTformer~\citep{kong2023efficient}} & 46.47 & 0.5420 & 0.3456 & 1.6130 & 4.0942 & 3.4569 & 36.91 \\
& \text{+ ORNet \textbf{(Ours)}} & \textbf{64.57} & \textbf{0.5949} & \textbf{0.4924} & \textbf{2.4664} & \textbf{4.1995} & \textbf{3.5278} & \textbf{15.69} \\
\midrule
\multirow{7.5}{*}{\rotatebox[origin=c]{90}{\large\textit{SIDD Denoising}}} & \text{AP-BSN~\citep{lee2022ap}} & 20.17 & 0.3613 & 0.1977 & 1.0556 & 1.0170 & 1.5496 & 58.18 \\
& \text{MIRNet-v2~\citep{zamir2020learning}} & 22.18 & 0.3770 & 0.2402 & 1.1855 & 1.0484 & 1.6197 & 52.81 \\
& \text{Restormer~\citep{zamir2022restormer}} & 22.55 & 0.3839 & 0.2439 & 1.2190 & 1.0653 & 1.6620 & 52.36 \\
\cmidrule{2-9}
& \text{Xformer~\citep{chen2024hierarchical}} & 22.57 & 0.3828 & 0.2472 & 1.2040 & 1.0759 & 1.6710 & 52.33 \\
& \text{+ ORNet \textbf{(Ours)}} & \textbf{35.68} & \textbf{0.4310} & \textbf{0.3710} & \textbf{1.9510} & \textbf{1.3228} & \textbf{2.1227} & \textbf{40.25} \\
\cmidrule{2-9}
& \text{NAFNet~\citep{chen2022simple}} & 22.73 & 0.3937 & 0.2458 & 1.2189 & 1.0826 & 1.7060 & 51.86 \\
& \text{+ ORNet \textbf{(Ours)}} & \textbf{35.87} & \textbf{0.4380} & \textbf{0.3776} & \textbf{1.9591} & \textbf{1.3513} & \textbf{2.1584} & \textbf{40.98} \\
\bottomrule
\end{tabular}
}
\label{tab:main_exp}
\vspace{-2mm}
\end{table*}
\begin{table*}[!t]
\centering
\caption{
Evaluation on an OOD environment. Additional Gaussian blur ($\sigma$ = 2.5) is applied to the blurry input images of the GoPro test set.
}
\renewcommand{\arraystretch}{0.9}
\setlength{\tabcolsep}{10pt}
\vspace{-2mm}
\scalebox{0.8}{
\begin{tabular}{l | ccc | ccc | cccc}
\toprule
\rowcolor{gray!15}
& \multicolumn{3}{c|}{\textit{Original GT}} & \multicolumn{3}{c|}{\textit{Enhanced GT}} & \multicolumn{4}{c}{\textit{Perceptual Quality Metrics}} \\
\rowcolor{gray!15}
{Method} & {PSNR}$\uparrow$ & {SSIM}$\uparrow$ & {LPIPS}$\downarrow$ & {PSNR}$\uparrow$ & {SSIM}$\uparrow$ & {LPIPS}$\downarrow$ & {MUSIQ}$\uparrow$ & {MANIQA}$\uparrow$ & {TOPIQ}$\uparrow$ & {LIQE}$\uparrow$ \\
\midrule
\text{FFTFormer~\citep{kong2023efficient}} & 24.56 & 0.7532 & 0.4714 & 23.68 & 0.7224 & 0.5441 & 22.3812 & 0.2284 & 0.1832 & 1.0108 \\
\text{+ORNet \textbf{(Ours)}} & \textbf{24.58} & \textbf{0.7670} & \textbf{0.3429} & \textbf{23.81} & \textbf{0.7405} & \textbf{0.3777} & \textbf{42.9131} & \textbf{0.2638} & \textbf{0.2646} & \textbf{1.0656} \\
\bottomrule
\end{tabular}
}
\label{tab:gopro_blur}
\vspace{-2mm}
\end{table*}
\begin{table*}[!t]
\centering
\caption{
Evaluation on an OOD environment. Additional white noise ($\sigma$ = 9) is applied to the blurry input images of the GoPro test set.
}
\renewcommand{\arraystretch}{0.9}
\setlength{\tabcolsep}{10pt}
\vspace{-2mm}
\scalebox{0.8}{
\begin{tabular}{l | ccc | ccc | cccc}
\toprule
\rowcolor{gray!15}
& \multicolumn{3}{c|}{\textit{Original GT}} & \multicolumn{3}{c|}{\textit{Enhanced GT}} & \multicolumn{4}{c}{\textit{Perceptual Quality Metrics}} \\
\rowcolor{gray!15}
{Method} & {PSNR}$\uparrow$ & {SSIM}$\uparrow$ & {LPIPS}$\downarrow$ & {PSNR}$\uparrow$ & {SSIM}$\uparrow$ & {LPIPS}$\downarrow$ & {MUSIQ}$\uparrow$ & {MANIQA}$\uparrow$ & {TOPIQ}$\uparrow$ & {LIQE}$\uparrow$ \\
\midrule
\text{FFTFormer~\citep{kong2023efficient}} & 24.35 & 0.5867 & 0.4463 & {23.83} & {0.5713} & {0.4751} & 30.1430 & 0.4517 & 0.2655 & 1.1819 \\
\text{+ORNet \textbf{(Ours)}} & \textbf{24.41} & \textbf{0.6179} & \textbf{0.4070} & \textbf{23.97} & \textbf{0.6057} & \textbf{0.4233} & \textbf{41.8760} & \textbf{0.4699} & \textbf{0.3188} & \textbf{1.4321} \\
\bottomrule
\end{tabular}
}
\label{tab:gopro_noise}
\vspace{-4mm}
\end{table*}
% !TEX root = ./../main.tex
\begin{figure*}[!t]
\centering
\scalebox{0.95}{
\begin{minipage}{\textwidth}
\centering
% Row 1 (image1)
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image1/input_crop.png}
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image1/IRSDE_crop.png}
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image1/DiffIR_crop.png}
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image1/AdaRevID_crop.png}
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image1/fftformer_crop.png}
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image1/fftformer_enhance_topiq_crop.png}
\end{subfigure}
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image1/target_crop.png}
\end{subfigure}
\vfill
% Row 2 (image2)
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image2/input_crop.png}
\small Blurry Input
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image2/IRSDE_crop.png}
\small IR-SDE
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image2/DiffIR_crop.png}
\small DiffIR
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image2/AdaRevID_crop.png}
\small AdaRevID
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image2/fftformer_crop.png}
\small FFTformer
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image2/fftformer_enhance_topiq_crop.png}
\small \bf{Ours}
\end{subfigure}
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/GOPRO_mini/image2/target_crop.png}
\small Original GT
\end{subfigure}
\end{minipage}
}
\vspace{-3mm}
\caption{
Qualitative comparison of state-of-the-art deblurring methods, including ours (ORNet applied to FFTformer), on the GoPro dataset.
Our method significantly improves the visual quality of the deblurred image. Zoom in for better visualization.
}
\vspace{-3mm}
\label{fig:qual_GOPRO}
\end{figure*}
% !TEX root = ./../main.tex
\begin{figure*}[!t]
\centering
\scalebox{0.95}{
\begin{minipage}{\textwidth}
\centering
% Row 1
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/27/0011_11_2.png}
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/27/0011_11_3.png}
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/27/0011_11_4.png}
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/27/0011_11_7.png}
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/27/0011_11_5.png}
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/27/0011_11_1.png}
\end{subfigure}
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/27/0011_11.png}
\end{subfigure}
\vfill
% Row 2
% \begin{subfigure}[b]{0.137\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/qual/SIDD/38/0017_29_2.png}
% \end{subfigure}\hfill
% \begin{subfigure}[b]{0.137\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/qual/SIDD/38/0017_29_3.png}
% \end{subfigure}\hfill
% \begin{subfigure}[b]{0.137\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/qual/SIDD/38/0017_29_4.png}
% \end{subfigure}\hfill
% \begin{subfigure}[b]{0.137\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/qual/SIDD/38/0017_29_7.png}
% \end{subfigure}\hfill
% \begin{subfigure}[b]{0.137\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/qual/SIDD/38/0017_29_5.png}
% \end{subfigure}\hfill
% \begin{subfigure}[b]{0.137\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/qual/SIDD/38/0017_29_1.png}
% \end{subfigure}
% \begin{subfigure}[b]{0.137\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/qual/SIDD/38/0017_29.png}
% \end{subfigure}
% \vfill
% Row 3
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/63/0011_31_2.png}
\small Noisy Input
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/63/0011_31_3.png}
\small AP-BSN
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/63/0011_31_4.png}
\small MIRNet
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/63/0011_31_7.png}
\small Xformer
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/63/0011_31_5.png}
\small NAFNet
\end{subfigure}\hfill
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/63/0011_31_1.png}
%\small \text{\textbf{Ours}}
\small \bf{Ours}
\end{subfigure}
\begin{subfigure}[b]{0.137\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/qual/SIDD/63/0011_31.png}
\small Original GT
\end{subfigure}
\end{minipage}
}
\vspace{-3mm}
\caption{
% Comparison between the qualitative results of state-of-the-art denoising methods including ours (output refinement network applied to NAFNet) on SIDD dataset. Our method significantly improves the visual quality of the denoised image. Zoom in for better visualization.
Qualitative comparison of state-of-the-art denoising methods, including ours (ORNet applied to NAFNet), on the SIDD dataset.
Our method significantly improves the visual quality of the denoised image. Zoom in for better visualization.
}
\vspace{-4mm}
\label{fig:sidd_qual}
\end{figure*}
% !TEX root = ./../main.tex
\begin{figure*}[t]
\centering
% \vspace{-1mm}
% Row 1
\begin{subfigure}[c]{0.04\textwidth}\centering
\rotatebox{90}{\parbox{3cm}{\centering Freq. domain\\{Ring-shaped}}}
\end{subfigure}\hspace{2mm}%
\scalebox{.88}{%
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/ours/cond_0.3/weight_map_channel_0.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_0$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/ours/cond_0.3/weight_map_channel_1.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_1$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/ours/cond_0.3/weight_map_channel_2.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_2$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/ours/cond_0.3/weight_map_channel_3.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_3$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/ours/mixed_output_with_box_full.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$\hat{I}^{\text{GT}}$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/ours/mixed_output_zoomed_crop_only.png}
};
\draw[red, line width=1.5pt] ([shift={(1pt,1pt)}]image.south west) rectangle ([shift={(-1pt,-1pt)}]image.north east);
\end{tikzpicture}
\end{subfigure}
}\\[0pt]
% Row 2
\vspace{-5mm}
\begin{subfigure}[c]{0.04\textwidth}\centering
\rotatebox{90}{\parbox{3cm}{\centering Freq. domain\\Elementwise}}
\end{subfigure}\hspace{2mm}%
\scalebox{.88}{%
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/element/cond_0.3/weight_map_channel_0.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_0$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/element/cond_0.3/weight_map_channel_1.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_1$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/element/cond_0.3/weight_map_channel_2.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_2$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/element/cond_0.3/weight_map_channel_3.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_3$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/element/mixed_output_with_box_full.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$\hat{I}^{\text{GT}}$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/element/mixed_output_zoomed_crop_only.png}
};
\draw[red, line width=1.5pt] ([shift={(1pt,1pt)}]image.south west) rectangle ([shift={(-1pt,-1pt)}]image.north east);
\end{tikzpicture}
\end{subfigure}
}\\[0pt]
\vspace{-5mm}
% Row 3
\begin{subfigure}[c]{0.04\textwidth}\centering
\rotatebox{90}{\parbox{3cm}{\centering Spatial domain\\Elementwise}}
\end{subfigure}\hspace{2mm}%
\scalebox{.88}{%
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/weight_map_channel_0.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_0$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/weight_map_channel_1.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_1$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/weight_map_channel_2.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_2$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/weight_map_channel_3.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_3$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/mixed_output_with_box_full.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$\hat{I}^{\text{GT}}$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.16\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/mixed_output_zoomed_crop_only.png}
};
\draw[red, line width=1.5pt] ([shift={(1pt,1pt)}]image.south west) rectangle ([shift={(-1pt,-1pt)}]image.north east);
\end{tikzpicture}
\end{subfigure}
}
\vspace{-4.5mm}
\caption{
The top row shows the results when the conditional frequency mask generator is trained using our method.
The second row shows the results when it is trained in an element-wise manner without ring-shaped Gaussian basis in frequency domain.
The bottom row shows the results when it is trained in an element-wise manner in spatial domain.
$M_i$ denotes the generated masks, and $\hat{I}^{\text{GT}}$ represents the enhanced ground truth generated using these masks.
Zoom in for better visualization.
}
\vspace{-4mm}
\label{fig:mask_analysis}
\end{figure*}
\begin{table}[t]
\center
\caption{Comparison between our output refinement network (ORNet) and directly finetuning the restoration model using our enhanced supervision. FFTFormer* denotes the model finetuned on the GoPro training set where the original ground truth is replaced with our enhanced ground truth.}
\renewcommand{\arraystretch}{0.9}
\vspace{-1.5mm}
\scalebox{0.85}{
\setlength{\tabcolsep}{5.5pt}
\begin{tabular}{l | cc cccc}
\toprule
\rowcolor{gray!20}
Method & MUSIQ$\uparrow$ & MANIQA$\uparrow$ & TOPIQ$\uparrow$ & LIQE$\uparrow$ \\
\midrule
\text{FFTformer}~\citep{kong2023efficient} & 46.47 & 0.5420 & 0.3456 & 1.6130 \\
\text{+ ORNet } & \bf{64.57} & \bf{0.5949} & \bf{0.4924} & \bf{2.4664} \\
\midrule
FFTformer*~\citep{kong2023efficient} & 60.54 & 0.5854 & 0.4509 & 2.2359 \\
\bottomrule
\end{tabular}
}
\label{tab:full_finetuning}
\vspace{-5mm}
\end{table}
\begin{figure}[t]
\centering
\begin{subfigure}[t]{0.48\linewidth}
\hspace{-2mm}
\includegraphics[width=\linewidth]{Figures/GT_perceptual_hist/blur_score_histogram.png}
\vspace{-4mm}
\end{subfigure}
\begin{subfigure}[t]{0.48\linewidth}
\includegraphics[width=\linewidth]{Figures/GT_perceptual_hist/artifact_score_histogram.png}
\vspace{-4mm}
\end{subfigure}
% \vspace{1mm}
\begin{subfigure}[t]{0.48\linewidth}
\hspace{-2mm}
\includegraphics[width=\linewidth]{Figures/GT_perceptual_hist_SIDD/blur_score_histogram.png}
\vspace{-4mm}
\end{subfigure}
\begin{subfigure}[t]{0.48\linewidth}
\includegraphics[width=\linewidth]{Figures/GT_perceptual_hist_SIDD/artifact_score_histogram.png}
\vspace{-4mm}
\end{subfigure}
\vspace{-3mm}
\caption{
Our enhanced GT images demonstrate improved blur and noise levels, assessed with KonIQ++~\citep{su2021koniq++}. Histograms compare low quality (LQ), ground truth (GT), and enhanced GT images from GoPro~\citep{nah2017deep} (Top) and SIDD~\citep{abdelhamed2018high} (Bottom) datasets.
}
\label{fig:score_histogram}
\vspace{-5mm}
\end{figure}
% !TEX root = ./../main.tex
\begin{figure}[t]
\centering
\begin{subfigure}[b]{0.325\linewidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/mixed_sample/GOPR0384_11_01-001011_x408_y428_cropped/target.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt] at (image.north west) {$I^\text{GT}_0$};
\node[anchor=south west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=0pt, yshift=0pt, align=left] at (image.south west) {\scriptsize CLIP Similarity: -};
\end{tikzpicture}
Original GT
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.325\linewidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/mixed_sample/GOPR0384_11_01-001011_x408_y428_cropped/cond_0.3.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt] at (image.north west) {$\hat{I}^\text{GT}$};
\node[anchor=south west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=0pt, yshift=0pt, align=left] at (image.south west) {\scriptsize CLIP Similarity: 0.9964};
\end{tikzpicture}
Enhanced GT
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.325\linewidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/mixed_sample/GOPR0384_11_01-001011_x408_y428_cropped/target_osediff_2x.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt] at (image.north west) {$I^\text{GT}_i$};
\node[anchor=south west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=0pt, yshift=0pt, align=left] at (image.south west) {\scriptsize CLIP Similarity: 0.8995};
\end{tikzpicture}
Super-resolved GT
\end{subfigure}
% \scalebox{0.95}{
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/1/crop_1_GOPR0384_11_00-000001.png}
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/1/crop_2_GOPR0384_11_00-000001.png}
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/1/crop_3_GOPR0384_11_00-000001.png}
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/1/crop_4_GOPR0384_11_00-000001.png}
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/1/crop_5_GOPR0384_11_00-000001.png}
% \end{subfigure}
% }
% \scalebox{0.95}{
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/GOPR0871_11_01-000259_x836_y47/cond_0.3.png}
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/GOPR0871_11_01-000259_x836_y47/target.png}
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/GOPR0871_11_01-000259_x836_y47/target_osediff_2x.png}
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/GOPR0871_11_01-000259_x836_y47/target_osediff_3x.png}
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/GOPR0871_11_01-000259_x836_y47/target_osediff_4x.png}
% \end{subfigure}
% }
% \vspace{2mm}
% % Row 2
% \scalebox{0.95}{
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/2/crop_1_GOPR0881_11_00-000018.png}
% Ours
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/2/crop_2_GOPR0881_11_00-000018.png}
% $\hat{I}^{\text{GT}}_0$ (Original GT)
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/2/crop_3_GOPR0881_11_00-000018.png}
% $\hat{I}^{\text{GT}}_1$
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/2/crop_4_GOPR0881_11_00-000018.png}
% $\hat{I}^{\text{GT}}_2$
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/mixed_sample/2/crop_5_GOPR0881_11_00-000018.png}
% $\hat{I}^{\text{GT}}_3$
% \end{subfigure}
% }
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/weight_map_channel_0.png}
% \small $M_0$
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/weight_map_channel_1.png}
% \small $M_1$
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/weight_map_channel_2.png}
% \small $M_2$
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/weight_map_channel_3.png}
% \small $M_3$
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/mixed_output_with_box_full.png}
% \small $\hat{I}^{\text{GT}}$
% \end{subfigure}
% \begin{subfigure}[b]{0.18\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/masks/spatial/cond_0.3/mixed_output_zoomed_crop_only.png}
% \small Zoomed
% \end{subfigure}
% \vfill
% \end{minipage}%
% \hfill
% \begin{minipage}[t]{0.4\textwidth}
% \vspace{-15mm}
% \begin{subfigure}[b]{0.4\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/masks/ours/mixed_output_crop.png}
% Ours
% \end{subfigure}\hfill
% \begin{subfigure}[b]{0.4\textwidth}\centering
% \includegraphics[width=\linewidth]{Figures/masks/element/mixed_output_crop.png}
% Element-wise mixed
% \end{subfigure}
% \end{minipage}
\vspace{-2mm}
\caption{
Comparison of original, our enhanced, and super-resolved GTs.
Our enhanced GT obtains high perceptual quality while maintaining semantic consistency.
Zoom in for better visualization.
}
\vspace{-2mm}
\label{fig:mixed_sample}
\end{figure}
% !TEX root = ./../main.tex
\begin{table}[t]
\centering
\caption{Comparison of Super-resolved GT and Enhanced GT in terms of semantic consistency and object detection performance.}
\label{tab:clip_similarity}
\vspace{-2mm}
\scalebox{0.85}{
\setlength{\tabcolsep}{3.5pt}
\begin{tabular}{l|ccc}
\toprule
\rowcolor{gray!20}
& CLIP embedding & \multicolumn{2}{c}{Object detection} \\
\rowcolor{gray!20}
\multirow{-2}{*}{Method} & (Cosine Similarity$\uparrow$) & (Precision$\uparrow$) & (Recall$\uparrow$) \\
\midrule
Super-resolved GT & 0.8782 & 0.7599 & 0.8345 \\
Enhanced GT & \textbf{0.9976} & \textbf{0.9448} & \textbf{0.9678} \\
\bottomrule
\end{tabular}
}
\vspace{-2mm}
\end{table}
% !TEX root = ./../main.tex
% \begin{wraptable}{r}{0.45\textwidth}
\begin{table}[t]
\centering
\renewcommand{\arraystretch}{0.9}
\setlength{\tabcolsep}{20pt}
\caption{
Parameters and MACs of existing restoration networks and our lightweightoutput refinement network.
%The numbers are calculated with an input of 512$\times$512.
}
\vspace{-2mm}
% \setlength{\tabcolsep}{8pt}
\scalebox{0.85}{
\begin{tabular}{l c c}
\toprule
\rowcolor{gray!20}
\multicolumn{1}{l}{Architecture} & Params. (M) & MACs (G) \\
\midrule
% OSEDiff~\citep{wu2024one} & 1775 & 2265 \\
% Uformer~\citep{wang2022uformer} & 50.39 & 86.07 \\
% Restormer~\citep{zamir2022restormer} & 26 & 564 \\
AdaRev~\citep{mao2024adarevd} & 68.0 & 1386 \\
FFTformer~\citep{kong2023efficient} & 14.9 & 525 \\
NAFNet~\citep{chen2022simple} & 115.9 & 254 \\
Xformer~\citep{chen2024hierarchical} & 25.1 & 571 \\
\midrule
\text{ORNet \textbf{(Ours)}} & 4.5 & 20 \\
\bottomrule
\end{tabular}
}
\label{tab:params_flops}
\vspace{-4mm}
% \end{wraptable}
\end{table}
% \begin{wrapfigure}{r}{0.45\linewidth}
\begin{figure}[t]
\centering
\scalebox{1}{
\includegraphics[width=\linewidth]{Figures/user_pref_piechart.png}
}
\vspace{-7mm}
\caption{
User preference study results. Participants consistently preferred our enhanced GT and ORNet outputs over the baselines.
}
% \vspace{-4mm}
\label{fig:user}
\vspace{-5mm}
\end{figure}
% \end{wrapfigure}
% !TEX root = ./../main.tex
\section{Conclusion}
\label{sec:conclusion}
We introduce a novel supervision enhancement framework that addresses the fundamental limitation of current image restoration models: suboptimal ground truth images in real-world image restoration.
By generating perceptually superior GT variants via super-resolution and optimally fusing them with original GT images in the frequency domain using adaptive masks, we achieve enhanced supervision targets.
Comprehensive evaluations, including user studies and diverse metrics, confirm that our method significantly improves the perceptual quality of the restored images.
This enhanced supervision then enables the training of a lightweight, model-agnostic refinement network, which can seamlessly integrate with existing restoration models to further boost their output.
We emphasize that our framework offers a practical path toward higher-fidelity and more visually compelling results in real-world restoration scenarios.
% !TEX root = ./../supplementary.tex
\section{Additional implementation details}
\subsection{Basis masks}
\input{Figures/mask_basis.tex}
\label{sec:basis_mask}
In our main paper, Equation (2) defines the $b$-th ring-shaped Gaussian basis mask $R_b$ using parameters $\mu_b$ and $\sigma_b$.
Here, $\mu_b$ represents the radial distance from the frequency-domain center where the mask has its peak, and $\sigma_b$ indicates the spread of the mask.
% \paragraph{Setting $\mu_b$ and $\sigma_b$ Values:}
We construct a total of $B = 25$ Gaussian basis masks.
The first mask is centered at $\mu_1 = 0$ with a standard deviation of $\sigma_1 = 0.05$.
For $b = 1, \dots, B$, the peak positions $\mu_b$ are arranged by quadratically spacing values between \(0\) and $\sqrt{H^2 + W^2}/2$, where $H$ and $W$ are height and width of the image, yielding denser coverage near the DC component and sparser placement at higher frequencies.
Simultaneously, the spreads \(\sigma_b\) increase quadratically from \(0.05\) up to \(0.55\), providing narrower rings at low frequencies and broader ones at high frequencies.
This design ensures fine control around the low-frequency region and efficient coverage of the full frequency range.
The all predefined masks are visualized in Figure~\ref{fig:mask_basis}.
\subsection{Architecture details}
\input{Figures/nafblock.tex}
Figure~\ref{fig:nafblock} shows the details of the NAFBlock, utilized within the frequency mask generator illustrated in Figure~2 (b).
The foundational block structures, including the Simple Gate and Simplified Channel Attention (SCA), are adopted from the NAFNet architecture~\citep{chen2022simple}.
An additional Downsample operation, composed of a convolution with a kernel size of $2 \times 2$ and a stride of 2, is incorporated into this NAFBlock variant.
And the proposed output refinement network (ORNet) consists of the 4 encoder, 1 middle, and 4 decoder blocks, employing the NAFBlock as their fundamental building unit.
The encoder block is same with figure~\ref{fig:nafblock}.
The middle block does not incorporate last downsampling operation.
The decoder block implements an upsampling instead of downsampling: it first doubles the channel dimensionality using a $1 \times 1$ convolution, followed by a pixel shuffle module that doubles both the height and width of the feature maps, following the NAFNet architecture~\citep{chen2022simple}.
% \dhr{To control the perceptual enhancement intensity, $\lambda$ is utilized as a model input for both the frequency mask generator and the output refinement network.}
% We expand $\lambda$ into a tensor of size $H \times W$ to match the input resolution and concatenate it with the input image in the channel dimension.
To control the perceptual enhancement intensity, we incorporate $\lambda$ as a conditional input for both the frequency mask generator and the output refinement network.
Specifically, we spatially expand the scalar $\lambda$ into a tensor of size $H \times W$ to match the input resolution and concatenate it with the input image along the channel dimension.
This conditioning allows the network to recognize the target enhancement level directly.
\section{Additional experiments}
\subsection{Visualization of Enahnced GT}
\input{Figures/enhanced_gt_sup.tex}
In Figure~\ref{fig:enhnaced_gt_sup}, we provide additional qualitative examples of our enhanced ground truth (GT) images.
Our method consistently shows superior perceptual quality by effectively removing residual degradations—such as noise and blur—from the original GTs, while preserving semantic fidelity.
In Figure~\ref{fig:gt_comparison}, we compare our supervision enhancement with a simple super-resolved variant.
While the super-resolved version mainly sharpens edges and increases overall brightness, our approach yields richer perceptual improvements without distorting semantic structure or altering the original color tone.
Lastly, Figure~\ref{fig:mixup_spatial_visual} illustrates how the enhanced GT is composed.
Our enhanced GT selectively integrates semantic details from the original GT and fine high-frequency components from the super-resolved variants, resulting in a sharper yet semantically consistent target image.
\input{Figures/sr_vs_enhanced.tex}
\input{Figures/mixup_spatial_visual.tex}
\subsection{Generalization on out-of-distribution dataset and unseen task}
\label{sec:generalization}
\input{Tables/hide_full_control.tex}
\input{Tables/lowlight_full_control.tex}
\paragraph{Quantitative results}
Furthermore, to evaluate generalization performance, we test our method on HIDE~\citep{shen2019human} as an out-of-distribution (OOD) deblurring benchmark.
We additionally evaluate it on LOL~\citep{wei2018deep}, a low-light enhancement benchmark, as an unseen restoration task.
For evaluating low light-enhancement, we adapt our refinement network to Retinexformer~\citep{cai2023retinexformer} and CIDNet~\citep{yan2025hvi}.
As shown in Table~\ref{tab:hide_full} and Table~\ref{tab:lol_full}, our approach consistently enhances perceptual quality even for the unknown dataset and task, demonstrating its strong generalization capabilities.
\paragraph{Qualitative results}
\input{Figures/lol.tex}
Figure~\ref{fig:lol} presents the qualitative results of our ORNet ($\lambda$=0.3) when applied to the output of CIDNet~\citep{yan2025hvi} on the LOL low light enhancement dataset~\citep{wei2018deep}.
Our ORNet effectively enhances the overall quality of the output, resulting in a more visually appealing image.
\subsection{Comparison with using single GT variant}
\input{Tables/gt_variant.tex}
To generate an enhanced ground truth, we first employ a super-resolution model to create ground truth variants.
For this purpose, we utilize upscale factors of 2, 3, and 4.
By applying a frequency mixup strategy to these diverse ground truth variants, we successfully construct an enhanced ground truth.
As an ablation study, we conduct an experiment with different grount truth variant setting, solely using upscale factor of 4.
Using these 4x variants, we follow our supervision enhancement framework and output refinement network.
The results are presented in Table~\ref{tab:gt_variant}.
As shown, ORNet\_4x, representing the model trained exclusively with the 4 upscale factor, achieved perceptual scores that are marginally lower than those obtained by the model trained with a mixture of variants.
This indicates that the incorporation of diverse GT variants is beneficial for achieving optimal final enhancement.
\subsection{Comparison with using ISR network instead of ORNet}
Our output refinement network (ORNet) is trained to refine the output of any existing image restoration model, which is trained with enhanced supervision.
To enhance the output of the restoration model, we could also directly apply the diffusion based image super-resolution (ISR) model used in our framework.
However, ISR model, which is designed to enhance the perceptual quality, often hallucinates details that are not present in the input image when applied directly.
Figure~\ref{fig:direct_isr} shows the results of applying the ISR model directly to the output of the restoration model, where the refined output destroys the textual detail.
% Table~\ref{tab:direct_isr} shows the quantitative results of directly applying the ISR model and using our ORNet to refine the output of the restoration model, where ISR model severely compromises the fidelity metrics.
% \subsection{VLM quality assessment study}
\input{Figures/directISR.tex}
\subsection{Ablation study on number of Gaussian ring-shaped basis}
\input{Figures/mask_band_num_ablation.tex}
We conduct an ablation study on the number of predefined Gaussian ring-shaped bases, denoted as $B$, to analyze its impact on frequency control.
As shown in Fig.~\ref{fig:mask_analysis}, we compare the generated masks and results with $B \in \{5, 25, 100\}$.
When $B$ is too small (e.g., $B=5$), we observe that the model fails to achieve fine-grained control over the frequency components due to the limited resolution of the basis functions.
However, once $B$ exceeds a certain threshold, the model exhibits a consistent tendency in frequency selectivity.
Since we observed that $B=25$ already provides sufficient capacity for precise frequency control, we adopt $B=25$ as the default setting in our experiments.
% \input{Tables/directISR.tex}
\subsection{Ablation study on enhancement level $\lambda$}
% \input{Tables/main_hide_lol.tex}
% \input{Tables/deblurring_full_control.tex}
% \input{Tables/denoising_full_control.tex}
\input{Tables/supp_exp.tex}
\input{Tables/deblurring_gopro_blur_full.tex}
\input{Tables/deblurring_gopro_noise_full.tex}
\paragraph{Quantative result}
% \input{Tables/fidelity_full.tex}
% Tables~\ref{tab:supp_exp}, \ref{tab:gopro_blur_full} and \ref{tab:gopro_noise_full} show the full expanded versions of Tables 1–4 in the main paper, with different $\lambda$ values (0.1, 0.3, 0.5, 0.7, and 0.9).
% A consistent trend observed is the steady improvement in perceptual quality metric scores as the value of $\lambda$ increases.
% However, as shown in Table~\ref{tab:supp_exp}, the VLM-based score exhibits a different behavior depending the dataset.
% This suggests that for datasets with very low initial quality, such as SIDD, a larger $\lambda$ leads to continuous improvement.
% Conversely, for datasets with relatively higher quality, like GoPro, excessive enhancement may introduce perceptually adverse artifacts, such as over-saturation and over-sharpening.
Tables~\ref{tab:supp_exp}, \ref{tab:gopro_blur_full}, and \ref{tab:gopro_noise_full} present the extended results across different $\lambda$ values.
We consistently observe that increasing $\lambda$ leads to better scores in perceptual quality metrics.
However, an excessively high $\lambda$ exacerbates the risks discussed in the main paper, such as hallucination issues.
% It also increases the potential for significant deviations in color tone and semantic content from the actual scene.
Furthermore, in the out-of-distribution (OOD) settings shown in Tables~\ref{tab:gopro_blur_full} and~\ref{tab:gopro_noise_full}, we observe that an excessively high $\lambda$ value can degrade reference-based performance against both the original and enhanced GTs.
This appears to be because when $\lambda$ is too large, the ORNet applies changes in color tone and further enhancements that go beyond removing the remaining degradations (blur, noise), resulting in a reference based performance drop.
This highlights the importance of selecting an appropriate $\lambda$ to achieve a balance between enhancement and fidelity.
% When $\lambda$ is small, the perceptual quality is not sufficiently enhanced, leading to lower scores.
% When $\lambda$ is large, there is a significant loss of fidelity, resulting in a decrease in the overall score.
% The results show that the overall performance is best achieved when $\lambda$ is set to 0.3, except for the Retinexformer~\cite{cai2023retinexformer}, where the best performance is achieved with $\lambda$ set to 0.5.
\paragraph{Qualitative results}
\input{Figures/enhanced_gt_cond.tex}
Figure~\ref{fig:enhanced_gt_cond} visualizes the results of our Ground Truth (GT) enhancement with varying values of the hyperparameter $\lambda$.
As $\lambda$ increases, the perceptual quality may be enhanced, but this can introduce undesirable artifacts such as altered color tones and semantic changes that deviate from the original GT.
In contrast, an optimally chosen $\lambda$ effectively removes residual noise and blur, leading to a perceptually improved image while preserving the color and semantic integrity of the original.
\paragraph{Mask visualization on diverse enhancement level $\lambda$}
\input{Figures/mask_ablation_full.tex}
Figure~\ref{fig:mask_analysis_full} shows the visualization of the generated masks with different $\lambda$ values.
When $\lambda$ is small, the generated masks are mostly focused on $M_0$, dedicated for original ground truth image.
As $\lambda$ increases, generated masks cover a diverse other ground truth variants.
\paragraph{Additional user study}
\input{Tables/user_study.tex}
We conducted a user study to validate the perceptual quality of the output images of ORNet with different $\lambda$ values.
Following the same protocol as described in the main paper, we evaluated three levels of the refinement weight $\lambda \in \{0.1, 0.3, 0.5\}$.
As shown in Figure~\ref{fig:user_study_full}, the results indicate that while both $\lambda=0.3$ and $\lambda=0.5$ achieved similarly high preference rates, $\lambda=0.3$ yielded the lowest loss rate.
\subsection{Training Stability}
To assess the training stability of our modular output refinement network (ORNet), we trained the ORNet with five independent times using distinct random seeds.
Following training, each refinement network was applied to the outputs of a pretrained AdaRevD~\citep{mao2024adarevd} on the GoPro test dataset.
Then, the standard deviation is calculated for each metric with $\lambda=0.3$.
The resulting standard deviations were as follows: PSNR (0.023), SSIM (0.0002), LPIPS (0.0005), DISTS (0.0005), MUSIQ (0.097), MANIQA (0.0007), TOPIQ (0.001), and LIQE (0.01).
The observed standard deviations for each metric are notably low.
This outcome indicates a high degree of stability in our training procedure for the output refinement network.
% \shm{
% \section{Limitation}
% Our method enhances the perceptual quality of the output of existing restoration models through training with enhanced supervision.
% However, perceptual quality metrics are often configured with subjective human preferences, which can vary across different datasets and tasks.
% In addition, the perceptual quality of the output does not always align with the fidelity measured with the original ground truth image.
% We alleviate this limitation by adopting a overall score that combines diverse metrics to balance the perceptual quality and fidelity.
% }
% \input{Figures/qual_GoPro.tex}
\section{Limitations}
Although our enhanced supervision framework achieves notable perceptual gains, selecting the enhancement strength $\lambda$ remains inherently challenging. Existing no-reference IQA metrics capture perceptual naturalness but provide limited insight into semantic fidelity, making it difficult to determine $\lambda$ in a fully principled manner.
To partially mitigate this issue, we evaluate fidelity at $\lambda$ = 0.3 using the available surrogate metrics—CLIP embedding similarity together with object-level consistency assessments based on detection alignment with the original ground truth (see Figure 7 and Table 5 of the main paper).
These analyses indicate that $\lambda$ = 0.3 preserves semantic structures while providing noticeable perceptual enhancement. Nevertheless, this evaluation remains constrained by the lack of a unified metric that jointly reflects fidelity and naturalness.
Developing such a metric, or an adaptive mechanism that balances the two aspects automatically, is an important direction for future work.
% !TEX root = ./../supplementary.tex
\vspace{-2mm}
\begin{figure*}[h]
\centering
% Row 1
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_0.png}
$R_1$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_1.png}
$R_2$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_2.png}
$R_3$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_3.png}
$R_4$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_4.png}
$R_5$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_5.png}
$R_6$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_6.png}
$R_7$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_7.png}
$R_8$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_8.png}
$R_9$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_9.png}
$R_{10}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_10.png}
$R_{11}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_11.png}
$R_{12}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_12.png}
$R_{13}$
\end{subfigure}
\vfill
\vspace{1mm}
% Row 2
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_13.png}
$R_{14}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_14.png}
$R_{15}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_15.png}
$R_{16}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_16.png}
$R_{17}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_17.png}
$R_{18}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_18.png}
$R_{19}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_19.png}
$R_{20}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_20.png}
$R_{21}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_21.png}
$R_{22}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_22.png}
$R_{23}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_23.png}
$R_{24}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/predefined_masks/mask_bin_24.png}
$R_{25}$
\end{subfigure}\hfill
\begin{subfigure}[b]{0.074\textwidth}\centering
\hspace{0pt}
\end{subfigure}
\vfill
\caption{Visualization of the predefined masks $R_1$-$R_{25}$.
It demonstrates denser partitioning in the low-frequency domain and broader partitioning in the high-frequency domain.}
\label{fig:mask_basis}
\end{figure*}
\vspace{-5mm}
\begin{figure*}[h]
\centering
\vspace{2mm}
\scalebox{0.7}{
\includegraphics[width=\linewidth]{Figures/nafblock.png}
}
\caption{The details of the NAFBlock.}
\label{fig:nafblock}
\vspace{-2mm}
\end{figure*}
% !TEX root = ./../supplementary.tex
\begin{figure*}[t]
\centering
\begin{subfigure}[t]{0.49\textwidth}
% \includegraphics[width=\linewidth]{Figures/detection/comparison_GOPR0384_11_05-004001_ori.png}
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0374_11_01-000246.png/target_GOPR0374_11_01-000246_magnified.png}
\centering GoPro GT
\vspace{-3mm}
\end{subfigure}
% \hfill
\begin{subfigure}[t]{0.49\textwidth}
% \includegraphics[width=\linewidth]{Figures/detection/comparison_GOPR0384_11_05-004001_ours0.3.png}
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0374_11_01-000246.png/cond_0.3_GOPR0374_11_01-000246_magnified.png}
\centering Enhanced GT
\vspace{-3mm}
\end{subfigure}
\\
\begin{subfigure}[t]{0.49\textwidth}
% \includegraphics[width=\linewidth]{Figures/detection/comparison_GOPR0384_11_05-004001_ori.png}
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0857_11_00-000093.png/target_GOPR0857_11_00-000093_magnified.png}
\centering GoPro GT
\vspace{-3mm}
\end{subfigure}
% \hfill
\begin{subfigure}[t]{0.49\textwidth}
% \includegraphics[width=\linewidth]{Figures/detection/comparison_GOPR0384_11_05-004001_ours0.3.png}
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0857_11_00-000093.png/cond_0.3_GOPR0857_11_00-000093_magnified.png}
\centering Enhanced GT
\vspace{-3mm}
\end{subfigure}
\\
\hspace{0.03em}
\begin{subfigure}[t]{0.49\textwidth}
% \includegraphics[width=\linewidth]{Figures/detection/comparison_GOPR0384_11_05-004001_ori.png}
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0884_11_00-000285.png/target_GOPR0884_11_00-000285_magnified.png}
\centering GoPro GT
\vspace{-3mm}
\end{subfigure}
% \hfill
\begin{subfigure}[t]{0.49\textwidth}
% \includegraphics[width=\linewidth]{Figures/detection/comparison_GOPR0384_11_05-004001_ours0.3.png}
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0884_11_00-000285.png/cond_0.3_GOPR0884_11_00-000285_magnified.png}
\centering Enhanced GT
\vspace{-3mm}
\end{subfigure}
% \hfill
% \begin{subfigure}[t]{0.32\textwidth}
% % \includegraphics[width=\linewidth]{Figures/detection/comparison_GOPR0384_11_05-004001_ours0.3.png}
% \includegraphics[width=\linewidth]{Figures/detection/comparison_GOPR0384_11_03-002198_osediff.png}
% \centering OSEDiff
% \vspace{-3mm}
% \end{subfigure}
\vspace{1mm}
\caption{
Visualization of enhanced ground truth.
Our enhanced GT not only exhibits sharper text and superior perceptual quality but also maintains semantic consistency.
Zoom in for better visualizaiton.
}
\label{fig:enhnaced_gt_sup}
\vspace{-4mm}
\end{figure*}
\begin{figure*}[t]
\captionsetup[subfigure]{labelformat=empty}
\centering
\begin{subfigure}[t]{0.33\linewidth}
\centering
\includegraphics[width=\linewidth]{Figures/target/GOPR0384_11_00-000001_inset.png}
\caption{Original GT}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.33\linewidth}
\centering
\includegraphics[width=\linewidth]{Figures/target_enhanced/ours/GOPR0384_11_00-000001_inset.png}
\caption{Enhanced GT (Ours)}
\end{subfigure}\hfill
\begin{subfigure}[t]{0.33\linewidth}
\centering
\includegraphics[width=\linewidth]{Figures/target_enhanced/2x/GOPR0384_11_00-000001_inset.png}
\caption{SR Variant ($I^\text{GT}_i$)}
\end{subfigure}
\vspace{-2mm}
\caption{Comparison of supervision enhancement with SR variant. Zoom in for better visualization of the semantic details and color tone.}
\label{fig:gt_comparison}
\end{figure*}
% !TEX root = ./../supplementary.tex
\begin{figure*}[t]
\centering
% \vspace{-1mm}
% Row 1
\scalebox{1.0}{%
\begin{subfigure}[c]{0.195\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/mixup_spatial/cond_0.5/spatial_component_ch0.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {\small $\mathcal{F}^{-1}( M_0 \odot \mathcal{F}(I_0^{\text{GT}}))$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.195\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/mixup_spatial/cond_0.5/spatial_component_ch1.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {\small $\mathcal{F}^{-1}( M_1 \odot \mathcal{F}(I_1^{\text{GT}}))$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.195\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/mixup_spatial/cond_0.5/spatial_component_ch2.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {\small $\mathcal{F}^{-1}( M_2 \odot \mathcal{F}(I_2^{\text{GT}}))$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.195\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/mixup_spatial/cond_0.5/spatial_component_ch3.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {\small $\mathcal{F}^{-1}( M_3 \odot \mathcal{F}(I_3^{\text{GT}}))$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.195\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/mixup_spatial/cond_0.5/mixed_output.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {\large $\hat{I}^{\text{GT}}$};
\end{tikzpicture}
\end{subfigure}
}
% Row 2
\vspace{-1mm}
\caption{
Visualization of composition of our enhanced ground truth image.
Super-resolved variants and the original ground truth image are blended in the frequency domain using the generated masks.
Each image visualized is the inverse Fourier transform of the masked frequency components of the corresponding original or super-resolved variant ground truth image.
Zoom in for better visualization.
}
% \vspace{-4mm}
\label{fig:mixup_spatial_visual}
\end{figure*}
\begin{table*}[t]
\centering
\caption{
The results are evaluated on the HIDE dataset~\citep{shen2019human} with the same settings as in the main paper.
}
\renewcommand{\arraystretch}{1.1}
\scalebox{0.8}{
\begin{tabular}{l | cccc | cc | c}
\toprule
\rowcolor{gray!15}
& \multicolumn{4}{c|}{\textit{No Ref.}} & \multicolumn{2}{c}{\textit{VLM-based.}} & \\
\rowcolor{gray!15}
Method & MUSIQ$\uparrow$ & MANIQA$\uparrow$ & TOPIQ$\uparrow$ & LIQE$\uparrow$ & {VisualQuality-R1}$\uparrow$ & {Q-Insight}$\uparrow$ & {A-FINE}$\downarrow$ \\
\midrule
AdaRevD~\citep{mao2024adarevd} & 55.91 & 0.5882 & 0.4064 & 2.0823 & 4.3013 & 3.4613 & 33.18 \\
% + ORNet ($\lambda$=0.1) & 58.34 & 0.5967 & 0.4211 & 2.1717 & 4.3630 & 3.5226 \\
% + ORNet ($\lambda$=0.2) & 64.64 & 0.6174 & 0.4772 & 2.4490 & 4.3590 & 3.5538 \\
{+ ORNet ($\lambda$=0.3)} & 68.48 & 0.6282 & 0.5375 & 2.6209 & 4.3341 & 3.5697 & 10.86 \\
% + ORNet ($\lambda$=0.4) & 30.07 & 0.9140 & 0.1354 & 0.1229 & 70.74 & 0.6362 & 0.6001 & 2.7866 & 2.0338 \\
% + ORNet ($\lambda$=0.5) & 71.70 & 0.6443 & 0.6314 & 2.9715 & 4.2557 & 3.5661 \\
% + ORNet ($\lambda$=0.6) & 28.89 & 0.9007 & 0.1597 & 0.1449 & 72.02 & 0.6509 & 0.6401 & 3.1022 & 2.0228 \\
% + ORNet ($\lambda$=0.7) & 72.03 & 0.6549 & 0.6409 & 3.1910 & 4.1948 & 3.5494 \\
% + ORNet ($\lambda$=0.8) & 28.14 & 0.8910 & 0.1745 & 0.1555 & 71.84 & 0.6547 & 0.6390 & 3.2325 & 2.0050 \\
% + ORNet ($\lambda$=0.9) & 71.57 & 0.6493 & 0.6361 & 3.2012 & 4.1217 & 3.5372 \\
% + ORNet ($\lambda$=1.0) & 27.17 & 0.8796 & 0.1862 & 0.1604 & 71.46 & 0.6495 & 0.6340 & 3.1750 & 1.9813 \\
\midrule
FFTFormer~\citep{kong2023efficient} & 54.42 & 0.5768 & 0.3978 & 2.0440 & 4.3176 & 3.4733 & 36.35 \\
% + ORNet ($\lambda$=0.1) & 57.72 & 0.5943 & 0.4125 & 2.1316 & 4.3599 & 3.5229 \\
% + ORNet ($\lambda$=0.2) & 64.26 & 0.6148 & 0.4687 & 2.4098 & 4.3613 & 3.5545 \\
{+ ORNet ($\lambda$=0.3)} & 68.19 & 0.6256 & 0.5295 & 2.5832 & {4.3351} & 3.5691 & 13.57 \\
% + ORNet ($\lambda$=0.4) & 29.74 & 0.9077 & 0.1357 & 0.1186 & 70.48 & 0.6335 & 0.5925 & 2.7507 & 2.0268 \\
% + ORNet ($\lambda$=0.5) & 71.42 & 0.6416 & 0.6237 & 2.9388 & 4.2458 & 3.5645 \\
% + ORNet ($\lambda$=0.6) & 28.65 & 0.8950 & 0.1588 & 0.1402 & 71.70 & 0.6483 & 0.6323 & 3.0737 & 2.0175 \\
% + ORNet ($\lambda$=0.7) & 71.67 & 0.6524 & 0.6329 & 3.1664 & 4.1871 & 3.5476 \\
% + ORNet ($\lambda$=0.8) & 27.93 & 0.8858 & 0.1742 & 0.1511 & 71.40 & 0.6517 & 0.6308 & 3.2091 & 1.9993 \\
% + ORNet ($\lambda$=0.9) & 71.08 & 0.6461 & 0.6278 & 3.1784 & 4.1332 & 3.5341 \\
% + ORNet ($\lambda$=1.0) & 27.02 & 0.8750 & 0.1859 & 0.1562 & 70.98 & 0.6465 & 0.6257 & 3.1559 & 1.9759 \\
\bottomrule
\end{tabular}
% \vspace{-10mm}
}
\label{tab:hide_full}
\end{table*}
\begin{table*}[t]
\centering
\caption{
The results are evaluated on the LOL dataset~\citep{wei2018deep} with the same settings as in the main paper.}
\renewcommand{\arraystretch}{1.1}
\scalebox{0.8}{
\begin{tabular}{l | cccc | cc | c}
\toprule
\rowcolor{gray!15}
& \multicolumn{4}{c|}{\textit{No Ref.}} & \multicolumn{2}{c|}{\textit{VLM-based.}} & \\
\rowcolor{gray!15}
Method & MUSIQ$\uparrow$ & MANIQA$\uparrow$ & TOPIQ$\uparrow$ & LIQE$\uparrow$ & {VisualQuality-R1}$\uparrow$ & {Q-Insight}$\uparrow$ & {A-FINE}$\downarrow$ \\
\midrule
\text{Retinexformer~\citep{cai2023retinexformer}} & 63.15 & 0.5870 & 0.5419 & 2.8354 & 3.4400 & 3.3153 & 49.50 \\
% + ORNet ($\lambda$=0) & 25.15 & 0.8456 & 0.1308 & 0.1376 & 63.38 & 0.5872 & 0.5429 & 2.8527 & 1.9267 \\
% + ORNet ($\lambda$=0.1) & 64.28 & 0.5944 & 0.5488 & 2.9363 & 3.5200 & 3.4080 \\
% + ORNet ($\lambda$=0.2) & 25.13 & 0.8484 & 0.1238 & 0.1233 & 69.65 & 0.6296 & 0.5988 & 3.4683 & 1.9942 \\
+ ORNet ($\lambda$=0.3) & 72.80 & 0.6652 & 0.6435 & 3.9872 & 3.5200 & 3.5047 & 26.96 \\
% + ORNet ($\lambda$=0.4) & 24.73 & 0.8414 & 0.1513 & 0.1365 & 74.16 & 0.6953 & 0.6616 & 4.3185 & 2.0330 \\
% {+ ORNet ($\lambda$=0.5)} & 74.76 & 0.7043 & 0.6669 & 4.4898 & 3.7667 & 3.6507 \\
% + ORNet ($\lambda$=0.6) & 24.44 & 0.8331 & 0.1689 & 0.1500 & 75.04 & 0.7155 & 0.6698 & 4.5606 & 2.0272 \\
% + ORNet ($\lambda$=0.7) & 75.18 & 0.7153 & 0.6715 & 4.5909 & 3.8467 & 3.7107 \\
% + ORNet ($\lambda$=0.8) & 24.24 & 0.8283 & 0.1782 & 0.1565 & 75.25 & 0.7147 & 0.6726 & 4.6027 & 2.0180 \\
% + ORNet ($\lambda$=0.9) & 75.31 & 0.7138 & 0.6735 & 4.6085 & 3.7400 & 3.7067 \\
% + ORNet ($\lambda$=1.0) & 24.03 & 0.8246 & 0.1848 & 0.1606 & 75.36 & 0.7122 & 0.6746 & 4.6143 & 2.0102 \\
\midrule
\text{CIDNet~\citep{yan2025hvi}} & 69.51 & 0.6256 & 0.6288 & 3.8336 & 3.9133 & 3.6967 & 46.52 \\
% + ORNet ($\lambda$=0) & 25.40 & 0.8653 & 0.0898 & 0.0800 & 69.69 & 0.6252 & 0.6298 & 3.8577 & 2.0592 \\
% + ORNet ($\lambda$=0.1) & 70.45 & 0.6329 & 0.6366 & 3.9853 & 4.0400 & 3.7447 \\
% + ORNet ($\lambda$=0.2) & 25.35 & 0.8657 & 0.1020 & 0.0852 & 73.38 & 0.6696 & 0.6774 & 4.4776 & 2.0981 \\
{+ ORNet ($\lambda$=0.3)} & 74.88 & 0.7035 & 0.7108 & 4.7373 & 3.9000 & 3.7613 & 20.05 \\
% + ORNet ($\lambda$=0.4) & 24.89 & 0.8539 & 0.1474 & 0.1366 & 75.49 & 0.7233 & 0.7185 & 4.8272 & 2.0792 \\
% + ORNet ($\lambda$=0.5) & 75.77 & 0.7295 & 0.7191 & 4.8546 & 4.0000 & 3.7667 \\
% + ORNet ($\lambda$=0.6) & 24.56 & 0.8438 & 0.1677 & 0.1569 & 75.91 & 0.7355 & 0.7189 & 4.8554 & 2.0571 \\
% + ORNet ($\lambda$=0.7) & 75.97 & 0.7350 & 0.7183 & 4.8512 & 4.0467 & 3.7760 \\
% + ORNet ($\lambda$=0.8) & 24.33 & 0.8381 & 0.1774 & 0.1651 & 75.998 & 0.7339 & 0.7178 & 4.8464 & 2.0436 \\
% + ORNet ($\lambda$=0.9) & 76.03 & 0.7324 & 0.7176 & 4.8431 & 4.0267 & 3.7447 \\
% + ORNet ($\lambda$=1.0) & 24.09 & 0.8334 & 0.1839 & 0.1694 & 76.06 & 0.7311 & 0.7177 & 4.8423 & 2.0339 \\
\bottomrule
\end{tabular}
}
% \vspace{-5mm}
\label{tab:lol_full}
\end{table*}
% \begin{table}[t]
% \centering
% \caption{Evaluation on LOL test dataset with controllable refinement network}
% \resizebox{1.0\textwidth}{!}{
% \begin{tabular}{l|ccccccccccc}
% \hline
% \ & \textbf{PSNR}($\uparrow$) & \textbf{SSIM}($\uparrow$) & \textbf{LPIPS}($\downarrow$) & \textbf{DISTS}($\downarrow$) & \textbf{CLIPQA}($\uparrow$) & \textbf{NIQE}($\downarrow$) & \textbf{MUSIQ}($\uparrow$) & \textbf{MANIQA}($\uparrow$) & \textbf{TOPIQ}($\uparrow$) & \textbf{LIQE}($\uparrow$) \\
% \hline
% \textbf{CIDNet} & 25.4036 & 0.8653 & 0.0897 & 0.0801 & 0.4727 & 4.0432 & 69.5141 & 0.6256 & 0.6288 & 3.8336 \\
% \textbf{+Ours ($\lambda$=0)} & 25.4040 & 0.8653 & 0.0898 & 0.0800 & 0.4718 & 4.0435 & 69.6949 & 0.6252 & 0.6298 & 3.8577 \\
% \textbf{+Ours ($\lambda$=0.1)} & 25.4090 & 0.8655 & 0.0902 & 0.0795 & 0.4725 & 4.0741 & 70.4464 & 0.6329 & 0.6366 & 3.9853 \\
% \textbf{+Ours ($\lambda$=0.2)} & 25.3486 & 0.8657 & 0.1020 & 0.0852 & 0.5002 & 4.3201 & 73.3797 & 0.6696 & 0.6774 & 4.4776 \\
% \textbf{+Ours ($\lambda$=0.3)} & 25.1346 & 0.8615 & 0.1262 & 0.1142 & 0.5228 & 5.0083 & 74.8802 & 0.7035 & 0.7108 & 4.7373 \\
% \textbf{+Ours ($\lambda$=0.4)} & 24.8868 & 0.8539 & 0.1474 & 0.1366 & 0.5377 & 5.5311 & 75.4856 & 0.7233 & 0.7185 & 4.8272 \\
% \textbf{+Ours ($\lambda$=0.5)} & 24.7000 & 0.8482 & 0.1597 & 0.1494 & 0.5275 & 5.9259 & 75.7701 & 0.7295 & 0.7191 & 4.8546 \\
% \textbf{+Ours ($\lambda$=0.6)} & 24.5555 & 0.8438 & 0.1677 & 0.1569 & 0.5164 & 6.1255 & 75.9104 & 0.7355 & 0.7189 & 4.8554 \\
% \textbf{+Ours ($\lambda$=0.7)} & 24.4403 & 0.8406 & 0.1732 & 0.1617 & 0.5072 & 6.2457 & 75.9720 & 0.7350 & 0.7183 & 4.8512 \\
% \textbf{+Ours ($\lambda$=0.8)} & 24.3346 & 0.8381 & 0.1774 & 0.1651 & 0.4987 & 6.2970 & 75.9983 & 0.7339 & 0.7178 & 4.8464 \\
% \textbf{+Ours ($\lambda$=0.9)} & 24.2187 & 0.8357 & 0.1809 & 0.1676 & 0.4917 & 6.3394 & 76.0264 & 0.7324 & 0.7176 & 4.8431 \\
% \textbf{+Ours ($\lambda$=1.0)} & 24.0937 & 0.8334 & 0.1839 & 0.1694 & 0.4878 & 6.3486 & 76.0637 & 0.7311 & 0.7177 & 4.8423 \\
% \hline
% \textbf{Retinexformer} & 25.1528 & 0.8456 & 0.1314 & 0.1381 & 0.4265 & 3.4705 & 63.1454 & 0.5870 & 0.5419 & 2.8354 \\
% \textbf{+Ours ($\lambda$=0)} & 25.1535 & 0.8456 & 0.1308 & 0.1376 & 0.4259 & 3.4734 & 63.3811 & 0.5872 & 0.5429 & 2.8527 \\
% \textbf{+Ours ($\lambda$=0.1)} & 25.1591 & 0.8459 & 0.1282 & 0.1352 & 0.4255 & 3.4784 & 64.2772 & 0.5944 & 0.5488 & 2.9363 \\
% \textbf{+Ours ($\lambda$=0.2)} & 25.1263 & 0.8484 & 0.1238 & 0.1233 & 0.4506 & 3.6663 & 69.6525 & 0.6296 & 0.5988 & 3.4683 \\
% \textbf{+Ours ($\lambda$=0.3)} & 24.9559 & 0.8472 & 0.1350 & 0.1253 & 0.4771 & 4.4603 & 72.7992 & 0.6652 & 0.6435 & 3.9872 \\
% \textbf{+Ours ($\lambda$=0.4)} & 24.7251 & 0.8414 & 0.1513 & 0.1365 & 0.4978 & 4.9247 & 74.1625 & 0.6953 & 0.6616 & 4.3185 \\
% \textbf{+Ours ($\lambda$=0.5)} & 24.5602 & 0.8366 & 0.1619 & 0.1448 & 0.4887 & 5.2407 & 74.7558 & 0.7043 & 0.6669 & 4.4898 \\
% \textbf{+Ours ($\lambda$=0.6)} & 24.4373 & 0.8331 & 0.1689 & 0.1500 & 0.4836 & 5.3960 & 75.0441 & 0.7155 & 0.6698 & 4.5606 \\
% \textbf{+Ours ($\lambda$=0.7)} & 24.3383 & 0.8304 & 0.1740 & 0.1537 & 0.4780 & 5.5151 & 75.1847 & 0.7153 & 0.6715 & 4.5909 \\
% \textbf{+Ours ($\lambda$=0.8)} & 24.2429 & 0.8283 & 0.1782 & 0.1565 & 0.4719 & 5.5777 & 75.2519 & 0.7147 & 0.6726 & 4.6027 \\
% \textbf{+Ours ($\lambda$=0.9)} & 24.1354 & 0.8263 & 0.1818 & 0.1588 & 0.4667 & 5.6006 & 75.3122 & 0.7138 & 0.6735 & 4.6085 \\
% \textbf{+Ours ($\lambda$=1.0)} & 24.0250 & 0.8246 & 0.1848 & 0.1606 & 0.4633 & 5.5960 & 75.3646 & 0.7122 & 0.6746 & 4.6143 \\
% \hline
% \end{tabular}
% }
% \label{tab:sidd_restormer_control}
% \end{table}
\begin{figure*}[t]
\centering
% turn off (a), (b), ... locally
\captionsetup[subfigure]{labelformat=empty,justification=centering}
\vspace{1mm}
% 첫 번째 행
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/lol_totoro/22_LQ.png}
\vspace{-5mm}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/lol_totoro/22_CIDNet.png}
\vspace{-5mm}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/lol_totoro/22_CIDNet_Enhance.png}
\vspace{-5mm}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/lol_totoro/22_GT.png}
\vspace{-5mm}
\end{subfigure}
% 행 사이 간격 추가
\vspace{2mm} % 원하는 간격 크기로 조정 (예: 10mm)
% 두 번째 행
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/lol_kitchen/111_LQ.png}
\vspace{-5mm}
\caption{Low Light Input Image}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/lol_kitchen/111_CIDNet.png}
\vspace{-5mm}
\caption{CIDNet~\citep{yan2025hvi}}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/lol_kitchen/111_CIDNet_Enhance.png}
\vspace{-5mm}
\caption{+ ORNet}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/lol_kitchen/111_GT.png}
\vspace{-5mm}
\caption{Ground Truth}
\end{subfigure}
\vspace{-1mm}
\caption{Qualitative results of our ORNet when applied to the output of CIDNet~\citep{yan2025hvi} on the LOL low light enhancement dataset~\citep{wei2018deep}.
Zoom in for better visualization.
}
\vspace{-2mm}
\label{fig:lol}
\end{figure*}
\begin{table}[t]
\center
\caption{Comparison of our ORNet with only using supervision enhancement trained with only ground truth variant with upscale factor 4.}
\renewcommand{\arraystretch}{1.1}
\vspace{-1mm}
\scalebox{0.85}{
\begin{tabular}{l | cccc}
\toprule
\rowcolor{gray!15}
Method & MUSIQ$\uparrow$ & MANIQA$\uparrow$ & TOPIQ$\uparrow$ & LIQE$\uparrow$ \\
\midrule
FFTformer~\citep{kong2023efficient} & 46.47 & 0.5420 & 0.3456 & 1.6131 \\
\midrule
+ ORNet ($\lambda$=0.1) & 48.07 & 0.5484 & 0.3537 & 1.6619 \\
{+ ORNet ($\lambda$=0.3)} & 64.57 & 0.5949 & 0.4924 & 2.4664 \\
+ ORNet ($\lambda$=0.5) & 69.18 & 0.6189 & 0.5905 & 2.9944 \\
+ ORNet ($\lambda$=0.7) & 69.76 & 0.6352 & 0.6104 & 3.2444 \\
+ ORNet ($\lambda$=0.9) & 69.76 & 0.6440 & 0.6198 & 3.3953 \\
\midrule
+ ORNet\_4x ($\lambda$=0.1) & 47.86 & 0.5469 & 0.3534 & 1.6511 \\
+ ORNet\_4x ($\lambda$=0.3) & 62.81 & 0.5509 & 0.5069 & 2.1908 \\
+ ORNet\_4x ($\lambda$=0.5) & 67.53 & 0.5792 & 0.5738 & 2.6217 \\
+ ORNet\_4x ($\lambda$=0.7) & 68.15 & 0.5970 & 0.5775 & 2.7956 \\
+ ORNet\_4x ($\lambda$=0.9) & 68.20 & 0.6051 & 0.5773 & 2.8558 \\
\bottomrule
\end{tabular}
}
\label{tab:gt_variant}
\vspace{-1mm}
\end{table}
\begin{figure*}[t]
\centering
% turn off (a), (b), ... locally
\captionsetup[subfigure]{labelformat=empty,justification=centering}
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/GOPR0881_11_01-000290/LQ.png}
\vspace{-5mm}
\caption{Blurry Input Image}
\label{fig:sub1}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/GOPR0881_11_01-000290/Refined_AdaRevID_GoPro.png}
\vspace{-5mm}
\caption{AdaRevD}
\label{fig:sub2}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/GOPR0881_11_01-000290/Refined_OSEDiff_AdaRevID_GoPro.png}
\vspace{-5mm}
\caption{+ OSEDiff}
\label{fig:sub3}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.24\textwidth}
\includegraphics[width=\linewidth]{Figures/qual/GOPR0881_11_01-000290/Refined_AdaRevID_gopro_output_enhance_GoPro_cond_0.4.png}
\vspace{-5mm}
\caption{+ ORNet}
\label{fig:sub4}
\end{subfigure}
\vspace{-1.5mm}
\caption{
Comparison of applying the ISR model (OSEDiff) directly and using our ORNet to refine the output of the restoration model (AdaRevD).
The ISR model generates details that are not present in the input image, leading to unrealistic results.
}
\vspace{-2mm}
\label{fig:direct_isr}
\end{figure*}
% !TEX root = ./../main.tex
\begin{figure*}[t]
\centering
% \vspace{-1mm}
% Row 1
\begin{subfigure}[c]{0.04\textwidth}\centering
\rotatebox{90}{\parbox{3cm}{\centering B=5}}
\end{subfigure}\hspace{2mm}%
\scalebox{.88}{%
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/5/cond_0.3/weight_map_channel_0_virid.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_0$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/5/cond_0.3/weight_map_channel_1_virid.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_1$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/5/cond_0.3/weight_map_channel_2_virid.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_2$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/5/cond_0.3/weight_map_channel_3_virid.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_3$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/5/cond_0.3/mixed_output.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$\hat{I}^{\text{GT}}$};
\end{tikzpicture}
\end{subfigure}
}\\[0pt]
% Row 2
\vspace{-1mm}
\begin{subfigure}[c]{0.04\textwidth}\centering
\rotatebox{90}{\parbox{3cm}{\centering B=25}}
\end{subfigure}\hspace{2mm}%
\scalebox{.88}{%
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/ours/cond_0.3/weight_map_channel_0.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_0$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/ours/cond_0.3/weight_map_channel_1.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_1$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/ours/cond_0.3/weight_map_channel_2.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_2$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/ours/cond_0.3/weight_map_channel_3.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_3$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/ours/cond_0.3/mixed_output.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$\hat{I}^{\text{GT}}$};
\end{tikzpicture}
\end{subfigure}
}\\[0pt]
% \vspace{-1mm}
% Row 3
\begin{subfigure}[c]{0.04\textwidth}\centering
\rotatebox{90}{\parbox{3cm}{\centering B=100}}
\end{subfigure}\hspace{2mm}%
\scalebox{.88}{%
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/100/cond_0.3/weight_map_channel_0_virid.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_0$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/100/cond_0.3/weight_map_channel_1_virid.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_1$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/100/cond_0.3/weight_map_channel_2_virid.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_2$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/100/cond_0.3/weight_map_channel_3_virid.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$M_3$};
\end{tikzpicture}
\end{subfigure}
\begin{subfigure}[c]{0.19\textwidth}\centering
\begin{tikzpicture}
\node[anchor=south west,inner sep=0] (image) at (0,0) {
\includegraphics[width=\linewidth]{Figures/masks/band_number_ablation/100/cond_0.3/mixed_output.png}
};
\node[anchor=north west, fill=black, fill opacity=0.4, text=white, text opacity=1, inner sep=2pt, xshift=1pt, yshift=-1pt] at (image.north west) {$\hat{I}^{\text{GT}}$};
\end{tikzpicture}
\end{subfigure}
}
\vspace{-1mm}
\caption{
Visualization of generated masks and enhanced results with different numbers of frequency bands ($B$).
The rows correspond to $B=5$, $B=25$ (Ours), and $B=100$, from top to bottom.
$M_i$ denotes the generated masks, and $\hat{I}^{\text{GT}}$ represents the enhanced ground truth generated using these mask.
}
\vspace{-4mm}
\label{fig:mask_analysis}
\end{figure*}
\begin{table*}[!t]
\centering
\caption{
Full table with diverse enhancement levels.
The results are evaluated on the GoPro deblurring~\citep{nah2017deep} and SIDD denoising~\citep{abdelhamed2018high} test sets.
}
\renewcommand{\arraystretch}{0.9}
\setlength{\tabcolsep}{15pt}
\vspace{-1mm}
\scalebox{0.75}{
\begin{tabular}{l l | cccc | cc| c}
\toprule
\rowcolor{gray!15}
{} & & \multicolumn{4}{c|}{\textit{Perceptual Quality Metrics}} & \multicolumn{2}{c}{\textit{VLM-based Metrics}} & \\
\rowcolor{gray!15}
{} & {Method} & {MUSIQ}$\uparrow$ & {MANIQA}$\uparrow$ & {TOPIQ}$\uparrow$ & {LIQE}$\uparrow$ & {VisualQuality-R1}$\uparrow$ & {Q-Insight}$\uparrow$ & {A-FINE}$\downarrow$ \\
\midrule
\multirow{12}{*}{\rotatebox[origin=c]{90}{\large\textit{GoPro Deblurring}}} & \text{AdaRevD~\citep{mao2024adarevd}} & 45.49 & 0.5363 & 0.3393 & 1.5656 & 4.1737 & 3.4386 & 36.19 \\
& \text{+ ORNet ($\lambda$=0.1)} & 47.42 & 0.5451 & 0.3490 & 1.6256 & 4.1939 & 3.4605 & 33.33 \\
& \text{+ ORNet ($\lambda$=0.3)} & 64.25 & 0.5916 & 0.4880 & 2.4291 & 4.1952 & 3.5206 & {14.38} \\
& \text{+ ORNet ($\lambda$=0.5)} & 69.11 & 0.6161 & 0.5893 & 2.9711 & 4.0686 & 3.5163 & 6.80 \\
& \text{+ ORNet ($\lambda$=0.7)} & 69.72 & 0.6330 & 0.6098 & 3.2308 & 4.0025 & 3.5063 & 4.77 \\
& \text{+ ORNet ($\lambda$=0.9)} & 69.73 & 0.6420 & 0.6195 & 3.3870 & 3.9146 & 3.4838 & 4.24 \\
\cmidrule{2-9}
& \text{FFTformer~\citep{kong2023efficient}} & 46.47 & 0.5420 & 0.3456 & 1.6131 & 4.0942 & 3.4569 & 36.91 \\
& \text{+ ORNet ($\lambda$=0.1)} & 48.07 & 0.5484 & 0.3537 & 1.6619 & 4.2334 & 3.4764 & 34.24 \\
& \text{+ ORNet ($\lambda$=0.3)} & 64.57 & 0.5949 & 0.4924 & 2.4664 & 4.1995 & 3.5278 & {15.69} \\
& \text{+ ORNet ($\lambda$=0.5)} & 69.18 & 0.6189 & 0.5905 & 2.9944 & 4.0918 & 3.5234 & 7.68 \\
& \text{+ ORNet ($\lambda$=0.7)} & 69.76 & 0.6352 & 0.6104 & 3.2444 & 4.0262 & 3.5162 & 5.49 \\
& \text{+ ORNet ($\lambda$=0.9)} & 69.76 & 0.6440 & 0.6198 & 3.3953 & 3.9600 & 3.4957 & 4.91 \\
\midrule
\multirow{12}{*}{\rotatebox[origin=c]{90}{\large\textit{SIDD Denoising}}} & \text{Xformer~\citep{zhang2023xformer}} & 22.57 & 0.3828 & 0.2472 & 1.2040 & 1.0759 & 1.6710 & 52.33 \\
& \text{+ ORNet ($\lambda$=0.1)} & 26.23 & 0.3819 & 0.2738 & 1.3238 & 1.1029 & 1.7972 & 51.25 \\
& \text{+ ORNet ($\lambda$=0.3)} & 35.68 & 0.4310 & 0.3710 & 1.9510 & 1.3228 & 2.1227 & {40.25} \\
& \text{+ ORNet ($\lambda$=0.5)} & 37.53 & 0.4517 & 0.3908 & 2.1195 & 1.3835 & 2.1827 & 35.46 \\
& \text{+ ORNet ($\lambda$=0.7)} & 37.99 & 0.4615 & 0.3968 & 2.1711 & 1.3975 & 2.2157 & 34.39 \\
& \text{+ ORNet ($\lambda$=0.9)} & 38.05 & 0.4661 & 0.3989 & 2.1867 & 1.4262 & 2.2176 & 33.99 \\
\cmidrule{2-9}
& \text{NAFNet~\citep{chen2022simple}} & 22.73 & 0.3937 & 0.2458 & 1.2189 & 1.0826 & 1.7060 & 51.86 \\
& \text{+ ORNet ($\lambda$=0.1)} & 26.39 & 0.3917 & 0.2776 & 1.3228 & 1.1224 & 1.8217 & 50.96 \\
& \text{+ ORNet ($\lambda$=0.3)} & 35.87 & 0.4380 & 0.3776 & 1.9591 & 1.3513 & 2.1584 & {40.98} \\
& \text{+ ORNet ($\lambda$=0.5)} & 37.89 & 0.4605 & 0.3977 & 2.1394 & 1.4030 & 2.2269 & 35.39 \\
& \text{+ ORNet ($\lambda$=0.7)} & 38.40 & 0.4709 & 0.4039 & 2.1934 & 1.4321 & 2.2498 & 34.14 \\
& \text{+ ORNet ($\lambda$=0.9)} & 38.46 & 0.4758 & 0.4057 & 2.2088 & 1.4492 & 2.2501 & 33.77 \\
\bottomrule
\end{tabular}
}
\label{tab:supp_exp}
\vspace{-0mm}
\end{table*}
\begin{table*}[!t]
\centering
\caption{
Evaluation on an OOD environment, where an additional Gaussian blur ($\sigma$ = 2.5) is applied to the blurry GoPro test set images.
}
\renewcommand{\arraystretch}{1.1}
\setlength{\tabcolsep}{7pt}
\vspace{-1mm}
\scalebox{0.8}{
\begin{tabular}{l | ccc | ccc | cccc}
\toprule
\rowcolor{gray!15}
& \multicolumn{3}{c|}{\textit{Original GT}} & \multicolumn{3}{c|}{\textit{Enhanced GT}} & \multicolumn{4}{c}{\textit{Perceptual Quality Metrics}} \\
\rowcolor{gray!15}
{Method} & {PSNR}$\uparrow$ & {SSIM}$\uparrow$ & {LPIPS}$\downarrow$ & {PSNR}$\uparrow$ & {SSIM}$\uparrow$ & {LPIPS}$\downarrow$ & {MUSIQ}$\uparrow$ & {MANIQA}$\uparrow$ & {TOPIQ}$\uparrow$ & {LIQE}$\uparrow$ \\
\midrule
\text{FFTFormer} & 24.5688 & 0.7532 & 0.4714 & 23.6891 & 0.7224 & 0.5441 & 22.3812 & 0.2284 & 0.1832 & 1.0108 \\
\text{+ORNet ($\lambda$=0.1)} & {24.5898} & {0.7550} & {0.4592} & {23.7141} & {0.7245} & {0.5310} & {23.1592} & {0.2481} & {0.1843} & {1.0111} \\
\text{+ORNet ($\lambda$=0.3)} & {24.5774} & {0.7670} & {0.3429} & {23.8124} & {0.7405} & {0.3777} & {42.9131} & {0.2638} & {0.2646} & {1.0656} \\
\text{+ORNet ($\lambda$=0.5)} & {24.5789} & {0.7742} & {0.3107} & {23.8835} & {0.7500} & {0.3334} & {49.6099} & {0.2689} & {0.3324} & {1.2576} \\
\text{+ORNet ($\lambda$=0.7)} & {24.4442} & {0.7759} & {0.3042} & {23.8007} & {0.7524} & {0.3253} & {50.9512} & {0.2950} & {0.3487} & {1.3977} \\
\text{+ORNet ($\lambda$=0.9)} & {24.2407} & {0.7754} & {0.3036} & {23.6454} & {0.7524} & {0.3243} & {51.5943} & {0.3174} & {0.3174} & {1.0656} \\
\bottomrule
\end{tabular}
}
\label{tab:gopro_blur_full}
\vspace{--2mm}
\end{table*}
\begin{table*}[!t]
\centering
\caption{
Evaluation on an OOD environment, where an additional white noise ($\sigma$ = 9) is applied to the blurry GoPro test set images.
}
\renewcommand{\arraystretch}{1.1}
\setlength{\tabcolsep}{7pt}
\vspace{-1mm}
\scalebox{0.8}{
\begin{tabular}{l | ccc | ccc | cccc}
\toprule
\rowcolor{gray!15}
& \multicolumn{3}{c|}{\textit{Original GT}} & \multicolumn{3}{c|}{\textit{Enhanced GT}} & \multicolumn{4}{c}{\textit{Perceptual Quality Metrics}} \\
\rowcolor{gray!15}
{Method} & {PSNR}$\uparrow$ & {SSIM}$\uparrow$ & {LPIPS}$\downarrow$ & {PSNR}$\uparrow$ & {SSIM}$\uparrow$ & {LPIPS}$\downarrow$ & {MUSIQ}$\uparrow$ & {MANIQA}$\uparrow$ & {TOPIQ}$\uparrow$ & {LIQE}$\uparrow$ \\
\midrule
\text{FFTFormer} & 24.3574 & 0.5867 & 0.4463 & {23.8352} & {0.5713} & {0.4751} & 30.1430 & 0.4517 & 0.2655 & 1.1819 \\
\text{+ORNet ($\lambda$=0.1)} & {24.3804} & {0.5886} & {0.4438} & {23.8616} & {0.5734} & {0.4720} & {30.5873} & {0.4528} & {0.2664} & {1.1909} \\
\text{+ORNet ($\lambda$=0.3)} & {24.4088} & {0.6179} & {0.4070} & {23.9693} & {0.6057} & {0.4233} & {41.8760} & {0.4699} & {0.3188} & {1.4321} \\
\text{+ORNet ($\lambda$=0.5)} & {24.4819} & {0.6771} & {0.3549} & {24.1483} & {0.6671} & {0.3602} & {55.2554} & {0.5204} & {0.4034} & {1.8986} \\
\text{+ORNet ($\lambda$=0.7)} & {24.3252} & {0.6943} & {0.3372} & {24.0463} & {0.6852} & {0.3397} & {58.9384} & {0.5473} & {0.4212} & {2.1779} \\
\text{+ORNet ($\lambda$=0.9)} & {24.0224} & {0.7018} & {0.3313} & {23.7722} & {0.6928} & {0.3335} & {60.1486} & {0.5618} & {0.4295} & {2.2923} \\
\bottomrule
\end{tabular}
}
\label{tab:gopro_noise_full}
\vspace{--2mm}
\end{table*}
% !TEX root = ./../main.tex
\begin{figure*}[t]
\centering
\scalebox{1.0}{
\begin{tabular}{@{}c@{\hspace{0.3em}}c@{\hspace{0.3em}}c@{\hspace{0.3em}}c@{\hspace{0.3em}}c@{}}
\parbox[t]{0.24\textwidth}{
\centering
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0884_11_00-000285.png/cropped_parts/target_GOPR0884_11_00-000285/GOPR0884_11_00-000285_crop_0.png}
\vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont Original GT}
\vspace{0mm}
} &
\parbox[t]{0.24\textwidth}{
\centering
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0884_11_00-000285.png/cropped_parts/cond_0.3_GOPR0884_11_00-000285/GOPR0884_11_00-000285_crop_0.png}
\vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont 0.3}
\vspace{0mm}
} &
\parbox[t]{0.24\textwidth}{
\centering
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0884_11_00-000285.png/cropped_parts/cond_0.5_GOPR0884_11_00-000285/GOPR0884_11_00-000285_crop_0.png}
\vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont 0.5}
} &
\parbox[t]{0.24\textwidth}{
\centering
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0884_11_00-000285.png/cropped_parts/cond_1.0_GOPR0884_11_00-000285/GOPR0884_11_00-000285_crop_0.png}
\vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont 1.0}
} \\
\vspace{2mm}
\parbox[t]{0.24\textwidth}{
\centering
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0884_11_00-000285.png/cropped_parts/target_GOPR0884_11_00-000285/GOPR0884_11_00-000285_crop_1.png}
\vspace{-7mm}
\caption*{ \fontsize{8pt}{8pt}\selectfont Original GT}
\vspace{0mm}
} &
\parbox[t]{0.24\textwidth}{
\centering
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0884_11_00-000285.png/cropped_parts/cond_0.3_GOPR0884_11_00-000285/GOPR0884_11_00-000285_crop_1.png}
\vspace{-7mm}
\caption*{ \fontsize{8pt}{8pt}\selectfont $\lambda$ = 0.3}
\vspace{0mm}
} &
\parbox[t]{0.24\textwidth}{
\centering
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0884_11_00-000285.png/cropped_parts/cond_0.5_GOPR0884_11_00-000285/GOPR0884_11_00-000285_crop_1.png}
\vspace{-7mm}
\caption*{ \fontsize{8pt}{8pt}\selectfont $\lambda$ = 0.5}
} &
\parbox[t]{0.24\textwidth}{
\centering
\includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0884_11_00-000285.png/cropped_parts/cond_1.0_GOPR0884_11_00-000285/GOPR0884_11_00-000285_crop_1.png}
\vspace{-7mm}
\caption*{ \fontsize{8pt}{8pt}\selectfont $\lambda$ = 1.0}
} \\
% \parbox[t]{0.24\textwidth}{
% \centering
% \includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0374_11_01-000246.png/cropped_parts/target_GOPR0374_11_01-000246/GOPR0374_11_01-000246_crop_0.png}
% \vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont Original GT}
% \vspace{0mm}
% } &
% \parbox[t]{0.24\textwidth}{
% \centering
% \includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0374_11_01-000246.png/cropped_parts/cond_0.3_GOPR0374_11_01-000246/GOPR0374_11_01-000246_crop_0.png}
% \vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont 0.3}
% \vspace{0mm}
% } &
% \parbox[t]{0.24\textwidth}{
% \centering
% \includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0374_11_01-000246.png/cropped_parts/cond_0.5_GOPR0374_11_01-000246/GOPR0374_11_01-000246_crop_0.png}
% \vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont 0.5}
% } &
% \parbox[t]{0.24\textwidth}{
% \centering
% \includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0374_11_01-000246.png/cropped_parts/cond_1.0_GOPR0374_11_01-000246/GOPR0374_11_01-000246_crop_0.png}
% \vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont 1.0}
% } \\
% \vspace{2mm}
% \parbox[t]{0.24\textwidth}{
% \centering
% \includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0374_11_01-000246.png/cropped_parts/target_GOPR0374_11_01-000246/GOPR0374_11_01-000246_crop_1.png}
% \vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont Original GT}
% \vspace{0mm}
% } &
% \parbox[t]{0.24\textwidth}{
% \centering
% \includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0374_11_01-000246.png/cropped_parts/cond_0.3_GOPR0374_11_01-000246/GOPR0374_11_01-000246_crop_1.png}
% \vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont 0.3}
% \vspace{0mm}
% } &
% \parbox[t]{0.24\textwidth}{
% \centering
% \includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0374_11_01-000246.png/cropped_parts/cond_0.5_GOPR0374_11_01-000246/GOPR0374_11_01-000246_crop_1.png}
% \vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont 0.5}
% } &
% \parbox[t]{0.24\textwidth}{
% \centering
% \includegraphics[width=\linewidth]{Figures/enhanced_gt/GOPR0374_11_01-000246.png/cropped_parts/cond_1.0_GOPR0374_11_01-000246/GOPR0374_11_01-000246_crop_1.png}
% \vspace{-7mm}
% \caption*{ \fontsize{8pt}{8pt}\selectfont 1.0}
% } \\
\vspace{2mm}
\vspace{-5mm}
\end{tabular}
}
\vspace{-5mm}
\centering
\caption{
% Qualitative results of diverse deblurring methods on GoPro~\cite{nah2017deep} dataset.
% Comparison between the qualitative results of state-of-the-art deblurring methods including ours (output refinement network applied to FFTformer) on GoPro dataset.
% Our method significantly improves the visual quality of the deblurred image.
% \textbf{Ours} denote the output of our proposed output refinement network (ORNet) applied to FFTformer~\cite{kong2023efficient}, which significantly improves the visual quality of the deblurred images.
Qualitative comparison of GT enhancement with varying $\lambda$ values.
Excessively large $\lambda$ values increase the risk of hallucinations, such as color shifts and semantic deviations from the original GT.
Zoom in for better visualization.
}
\vspace{-2mm}
\label{fig:enhanced_gt_cond}
\end{figure*}
% !TEX root = ./../main.tex
\begin{figure*}[h]
\centering
% Row 1
\hspace{-14mm}
\begin{subfigure}[b]{0.13\textwidth}\centering
$M_0$
\vspace{11mm}
\end{subfigure}\hspace{-9mm}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.1/weight_map_channel_0_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.3/weight_map_channel_0_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.5/weight_map_channel_0_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.7/weight_map_channel_0_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.9/weight_map_channel_0_virid.png}
\end{subfigure}\hfill
\vfill
% Row 2
\hspace{-14mm}
\begin{subfigure}[b]{0.13\textwidth}\centering
$M_1$
\vspace{11mm}
\end{subfigure}\hspace{-9mm}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.1/weight_map_channel_1_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.3/weight_map_channel_1_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.5/weight_map_channel_1_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.7/weight_map_channel_1_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.9/weight_map_channel_1_virid.png}
\end{subfigure}\hfill
\vfill
% Row 3
\hspace{-14mm}
\begin{subfigure}[b]{0.13\textwidth}\centering
$M_2$
\vspace{11mm}
\end{subfigure}\hspace{-9mm}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.1/weight_map_channel_2_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.3/weight_map_channel_2_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.5/weight_map_channel_2_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.7/weight_map_channel_2_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.9/weight_map_channel_2_virid.png}
\end{subfigure}\hfill
\vfill
% Row 4
\hspace{-14mm}
\begin{subfigure}[b]{0.13\textwidth}\centering
$M_3$
\vspace{11mm}
\end{subfigure}\hspace{-9mm}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.1/weight_map_channel_3_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.3/weight_map_channel_3_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.5/weight_map_channel_3_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.7/weight_map_channel_3_virid.png}
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
\includegraphics[width=\linewidth]{Figures/masks/ablation_full/GOPR0372_07_00-000047.png_0_details/cond_0.9/weight_map_channel_3_virid.png}
\end{subfigure}\hfill
\vfill
% Row 4
\hspace{-14mm}
\begin{subfigure}[b]{0.13\textwidth}\centering
% condition
% \vspace{1mm}
\hspace{10mm}
\end{subfigure}\hspace{-9mm}
\begin{subfigure}[b]{0.13\textwidth}\centering
$\lambda=0.1$
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
$\lambda=0.3$
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
$\lambda=0.5$
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
$\lambda=0.7$
\end{subfigure}
\begin{subfigure}[b]{0.13\textwidth}\centering
$\lambda=0.9$
\end{subfigure}\hfill
\vfill
\caption{
Frequency masks generated by the proposed controllable frequency mask generator. The enhancement level is controlled by the input parameter $\lambda$.
}
\label{fig:mask_analysis_full}
\end{figure*}
\begin{figure*}[t]
\centering
\includegraphics[width=0.5\linewidth]{Figures/user_pref_supp.png}
\vspace{-2mm}
\caption{
User study with various $\lambda$ values. Participants consistently preferred our ORNet outputs to the baselines.
}
\label{fig:user_study_full}
\end{figure*}