OMG: Opacity Matters in Material Modeling with Gaussian Splatting

Extracted source text · arXiv 2502.10988

\documentclass{article} % For LaTeX2e
\usepackage{iclr2025_conference,times}

% Optional math commands from https://github.com/goodfeli/dlbook_notation.
\input{math_commands.tex}
\definecolor{cvprblue}{rgb}{0.21,0.49,0.74}
\usepackage[colorlinks=True,
            linkcolor=cvprblue,
            anchorcolor=blue,  
            pagebackref,
            citecolor=cvprblue,
            ]{hyperref}
\usepackage{wrapfig}
\usepackage{url}
\usepackage{tabu}%
\usepackage{longtable}

\usepackage{booktabs}
\arrayrulewidth=1pt
\setlength{\arrayrulewidth}{1pt}
\usepackage{booktabs}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{multirow}
% \usepackage[dvipsnames]{xcolor}       % colors
\usepackage{graphicx,verbatimbox}
\usepackage{subcaption}
\usepackage{makecell}

\usepackage{colortbl}
\definecolor{color3}{RGB}{255, 255, 200}
\definecolor{color2}{RGB}{255, 220, 200}
\definecolor{color1}{RGB}{255, 181, 163}
\newcommand{\cc}[1]{\cellcolor{color#1}}
\newcommand{\yaqi}[1]{\textcolor{Orchid}{[Yaqi: #1 ]}}
\newcommand{\ks}[1]{\textcolor{Cyan}{[Katia: #1 ]}}
\newcommand{\RebuttalRevision}[1]{\textcolor{black}{#1}}


\title{\textbf{O{\Huge M}G}: \underline{O}pacity \underline{M}atters in \underline{M}aterial \underline{M}odeling with \underline{G}aussian splatting}

% Authors must not appear in the submitted version. They should be hidden
% as long as the \iclrfinalcopy macro remains commented out below.
% Non-anonymous submissions will be rejected without review.

\author{Silong Yong\thanks{Joint collaborators in research inception } \quad Venkata Nagarjun Pudureddiyur Manivannan\footnotemark[1] \quad Bernhard Kerbl \\
\textbf{Zifu Wan \qquad \qquad Simon Stepputtis \qquad \qquad Katia Sycara \qquad \qquad Yaqi Xie} \\ % \thanks{} \\
Carnegie Mellon University\\
\texttt{\{silongy,vpudured,bkerbl,zifuw,sstepput,sycara,yaqix\}@andrew.cmu.edu} \\
% \texttt{\{zifuw, sstepput, sycara, yaqix\}@andrew.cmu.edu}
}

% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to \LaTeX{} to determine where to break
% the lines. Using \AND forces a linebreak at that point. So, if \LaTeX{}
% puts 3 of 4 authors names on the first line, and the last on the second
% line, try using \AND instead of \And before the third author name.

\newcommand{\fix}{\marginpar{FIX}}
\newcommand{\new}{\marginpar{NEW}}

\iclrfinalcopy % Uncomment for camera-ready version, but NOT for submission.
\begin{document}
\maketitle

\begin{abstract}
 Decomposing geometry, materials and lighting from a set of images, namely inverse rendering, has been a long-standing problem in computer vision and graphics. Recent advances in neural rendering enable photo-realistic and plausible inverse rendering results. The emergence of 3D Gaussian Splatting has boosted it to the next level \RebuttalRevision{by showing real-time rendering potentials}.  An intuitive finding is that the models used for inverse rendering do not take into account the dependency of opacity w.r.t. material properties, namely cross section, as suggested by optics. Therefore, we develop a novel approach that adds this dependency to the modeling itself. Inspired by radiative transfer, we augment the opacity term by introducing a neural network that takes as input material properties to provide modeling of cross section and a physically correct activation function. The gradients for material properties are therefore not only from color but also from opacity, facilitating a constraint for their optimization. Therefore, the proposed method incorporates more accurate physical properties compared to previous works. We implement our method into 3 different baselines that use Gaussian Splatting for inverse rendering and achieve significant improvements universally in terms of novel view synthesis and material modeling. 
\end{abstract}

\section{Introduction}
\looseness=-1
Inverse rendering, a long-standing problem in computer vision and graphics, aims to recover physical properties such as geometry, materials and lighting conditions from a set of images~\citep{barron2013intrinsic, debevec2008rendering}. These properties are essential to downstream applications such as material editing, relighting and, from a broader perspective, real-to-sim transfer. However, it is well-known to be challenging and ill-posed because of the inherent ambiguity of the task: Different combinations of illumination, materials and geometry may lead to similar rendering results. Recent success in novel view synthesis~\citep{mildenhall2021nerf, barron2021mip, chen2022tensorf, muller2022instant, barron2023zip, barron2022mip} inspired the use of implicit neural fields (NeRF-like methods) for inverse rendering~\citep{zhang2021nerfactor, yao2022neilf, boss2021nerd, jin2023tensoir, srinivasan2021nerv, zhang2021physg}. On the other hand, 3D Gaussian Splatting (3DGS)~\citep{kerbl20233d} emerged as a recent, high-performance alternative to these slower architectures for novel view synthesis by introducing a much more compact scene representation and real-time rendering ability. Naturally, recent works combine 3DGS with inverse rendering~\citep{liang2024gs, gao2023relightable, jiang2024gaussianshader}, taking advantage of its high-speed nature and compact geometry. By adding additional parameters that act as material properties for each Gaussian, they are able to provide physically plausible material modeling and novel view synthesis results using physically-based rendering (PBR) thanks to the explicit nature of 3DGS. While it is intuitive to take advantage of 3DGS for fast training and rendering with high quality, directly applying 3DGS for inverse rendering may result in sub-optimal modeling. Specifically, the correlation between the material attributes of each Gaussian (e.g., opacity, albedo, roughness, etc.) remains under-explored, leading to under-constrained modeling of the scene properties. Therefore, the inverse rendering results remain to be improved.

\looseness=-1
Observing the disentangled nature of 3DGS-based methods, our work, inspired by the Bouguer-Beer-Lambert law in radiative transfer, reveals the missing correlation between materials and opacity, two sets of parameters that are essential for physically-based rendering using 3D Gaussian Splatting. By doing so, our work introduces constraints at model level to align the model better with the physical world. Specifically, we propose to model opacity in 3DGS-based inverse rendering methods strictly following the form derived from the Bouguer-Beer-Lambert law, which states that the intensity of light should decrease exponentially when passing through an absorbing body. Intuitively speaking, the decrease rate depends not only on the path length that a light travels, but also on the material and number density of the absorbing body. An illustration of the law can be found in Fig.~\ref{fig:motivation}. Consider two different translucent materials glass and gas in Fig.~\ref{fig:motivation}. They react differently when hit by light. For instance, red light would be terminated by the gas while it's able to pass through the glass nearly unaffected as illustrated by \RebuttalRevision{Fig.~\ref{fig:motivation}}. In other words, \textbf{opacity varies between different materials}. When designing the model, 3DGS-based inverse rendering methods~\citep{liang2024gs, jiang2024gaussianshader, gao2023relightable} overlook the fact that because of the disentangled representation, opacity is a standalone parameter that has no dependency on the material properties. As for NeRF-based methods that use alpha blending implicitly model the dependencies of opacity (volume density) and materials by using the same neural network to output different attributes. Therefore, for 3DGS-based methods, we propose to multiply the original opacity term with a material-dependent term and apply a physically correct activation function to give a better modeling of the physical property opacity itself. The material properties therefore serve as input for PBR as well as input for alpha-blending and receive gradients from both sides, facilitating a physically informed regularization to the model.
                                  
\begin{wrapfigure}[24]{t}{0.6\textwidth}
\begin{center}
\vspace{-10pt}
\includegraphics[width=\linewidth]{figs/motivation_l.pdf}

\end{center}
\caption{\looseness=-1 \RebuttalRevision{An illustration of the motivation that opacity depends on materials. Consider two translucent materials gas and glass. \textbf{Left}: a glass body that is hit by a light lets the light pass through with little reduction in intensity. \textbf{Right}: a gas that is hit by the same light make the light extinct inside it by absorbing it completely. This comparisons motivate our work. We perceive each Gaussian blob, paired with material properties, as an absorbing body that has its own absorption coefficient represented by opacity, which should consider material properties as influencing factors.}}
\vspace{-10pt}
\label{fig:motivation}
\end{wrapfigure}

\looseness=-1
To verify the correctness and effectiveness of the proposed formulation, we analytically conduct Taylor expansion to our approach and compare the difference w.r.t.\ the original way of computing opacity. The takeaway message is that the original way is actually an approximation of our approach. Empirically, we apply the modification to 3 state of the art baselines, namely GaussianShader~\citep{jiang2024gaussianshader}, GS-IR~\citep{liang2024gs} and R3DG~\citep{gao2023relightable} and conduct experiments on both synthetic and real-world data. The experimental results on Synthetic4Relight~\citep{zhang2022modeling}, Shiny Blender~\citep{verbin2022ref}, Glossy Synthetic~\citep{liu2023nero} and MIP-NeRF 360~\citep{barron2022mip} show that our approach enables universal performance improvement in terms of across different baselines and different data. The improvements on material modeling, i.e., albedo estimation and roughness estimation, lead to better novel view synthesis and relighting results in terms of PSNR, SSIM and LPIPS. Our experimental results indicate the significance of incorporating physically correct priors and constraints into the model. 

\section{Related Work}
\subsection{Neural Scene Representation}
\looseness=-1
Recent trends in \RebuttalRevision{neural rendering}~\citep{mildenhall2021nerf, yu2021plenoctrees, fridovich2022plenoxels, sitzmann2021light, yariv2021volume, miller2024objects, huang20242d, yong2024gl} have demonstrated impressive success in addressing visual computing problems like novel view synthesis. NeRF~\citep{mildenhall2021nerf} stands out as a representative by modeling the scene using an MLP to output volume density and color in a continuous space, which requires massive repeated queries for volume rendering during training and inference to synthesize images. Efforts have been made to improve the efficiency of NeRF by introducing additional data structures~\citep{garbin2021fastnerf, muller2022instant, hedman2021baking, yu2021plenoctrees, fridovich2022plenoxels, chen2022tensorf}. However, the discretized nature of the data structures reduces the image quality to some extent. Recently, 3D Gaussian Splatting (3DGS)~\citep{kerbl20233d} comes out and is able to achieve fast rendering speed while maintaining the rendering quality, therefore drawing the community's interest. Plenty of works focus on improving 3DGS by reducing the memory footprint~\citep{fan2023lightgaussian, navaneet2023compact3d, niedermayr2024compressed}, speeding up the training pipeline~\citep{mallick2024taming, hollein20243dgs} and removing the heuristic designs for optimization~\citep{kheradmand20243d, bulo2024revising}. Another direction is on the application side, works have been done to apply 3DGS to different domains such as semantic understanding~\citep{qin2024langsplat, guo2024semantic}, time sequence modeling~\citep{luiten2024dynamic, lin2024gaussian} and robotic manipulation~\citep{lu2024manigaussian, shorinwa2024splat}, etc. The wide range of domains for application encouraged the focus of our work on inverse rendering techniques that use 3DGS as the backbone.

\subsection{Inverse Rendering}
\looseness=-1
Inverse rendering aims to decompose image observations into geometry, material and lighting conditions (i.e., scene properties) that support a myriad of downstream tasks, such as material editing and relighting~\citep{li2023multi, zhang2022modeling, kanamori2019relighting, yang2022ps}. Normally, it is tackled by combining physically-based rendering with a differentiable renderer for optimization-based decomposition of scene properties~\citep{kajiya1986rendering, chen2019learning}. While being an inherently ambiguous problem, many works assume different constraints at input level, such as known lighting conditions~\citep{bi2020neural, srinivasan2021nerv, zhang2022modeling}, fixed lighting~\citep{dong2014appearance}, unintended shadow~\citep{verbin2024eclipse} or absence of shadow simulation~\citep{zhang2021physg}. The combination of differentiable volume rendering and physically-based rendering has enabled modeling more complex conditions towards more realistic cases thanks to the emergence of \RebuttalRevision{neural scene representation~\citep{zhang2021nerfactor, yao2022neilf, liu2023nero, verbin2022ref, jin2023tensoir, attal2024flash, boss2021nerd, munkberg2022extracting}}. Recently, 3DGS has revolutionized the field of neural scene representation. It is natural to expect inverse rendering could be tackled by 3DGS-based methods to incorporate fast and precise approximation of materials, geometry and lighting~\citep{liang2024gs, jiang2024gaussianshader, gao2023relightable}. Despite the rapid success of providing plausible material estimation, the results remain to be improved, mainly because the constraints that a model should have for inverse rendering have been overlooked.

\begin{figure}[t]
\begin{center}
\includegraphics[width=\textwidth]{figs/method.pdf}
\end{center}
\caption{\looseness=-1 Pipeline overview. We add an additional block to 3DGS-based inverse rendering methods~\citep{liang2024gs, kerbl20233d, jiang2024gaussianshader, gao2023relightable}. Specifically, instead of modeling opacity as a standalone parameter as done by previous works, we augment it against material. By introducing a neural network that takes material properties as input and output cross section and multiplying the opacity with it, we are able to incorporate the Bouguer-Beer-Lambert law into the model. During optimization, material properties not only receive the gradients from color through differentiable PBR, but also the gradients from alpha enforced by the neural network. By doing so, we add an additional constraint to material properties that makes the overall pipeline strictly follow the Bouguer-Beer-Lambert law.}
\label{fig:method}
\end{figure}

\section{Preliminaries}
\looseness=-1
In this section,  We will cover the basic concepts of 3D Gaussian Splatting, inverse rendering and the Bouguer-Beer-Lambert law as our work is built upon them. 

\subsection{3D Gaussian Splatting}
\looseness=-1
3D Gaussian Splatting (3DGS)~\citep{kerbl20233d} uses a set of points represented by 3D Gaussian primitives to model the 3D scene. Formally, each Gaussian $G_i$ is represented by a mean $\mu_i$, a covariance matrix $\Sigma_i$, an opacity $o_i$ and a view-dependent outgoing radiance $c_i$. Additionally, the covariance matrix is factorized into a quaternion $q_i$ and a scale $s_i$ and the outgoing radiance is treated as a set of spherical harmonics coefficients. For the inverse rendering problem, 3DGS-based methods~\citep{liang2024gs, jiang2024gaussianshader, gao2023relightable} assign material properties $m_i$ and normals $n_i$ to each Gaussian to support physically-based rendering. These sets of parameters are used to produce PBR color, which serves as $c_i$ in the original formulation.

\looseness=-1
After projecting the 3D Gaussians to 2D space using camera parameters $\theta$, 3DGS adopts $alpha$-blending for rendering the final color of a specific pixel location $x$ following 
\begin{equation}
\begin{aligned}
    \hat{C}(x) &= \sum_{i=1}^{N} c_i\alpha_i(x)T_i(x),\\
    T_i(x) &=\prod_{j=1}^{i-1}(1-\alpha_j(x)),
  \label{eq:alpha_blending}
\end{aligned}
\end{equation}
where $N$ represents the sorted $N$ Gaussian points, $T_i$ is the accumulated transmittance of the first $i-1$ Gaussians and $\alpha_i(x)$ is computed using
\begin{equation}
\alpha_i(x) = o_i \exp(-\frac{1}{2}(x-R(\mu_i; \theta))^TR_{\theta}(\Sigma_i)^{-1}(x-R(\mu_i; \theta))),
  \label{eq:alpha_equation}
\end{equation}
in which $R$ stands for the camera projection operation, \RebuttalRevision{$R_\theta$ is the operation of applying the camera projection to the covariance matrix} and opacity $o_i$ is weighted by the probability determined by the distance of the pixel location to the mean of the projected Gaussian. Afterwards, color loss is computed between the rendered image and the ground truth:
\begin{equation}
L = ||C - \hat{C}||^2,
  \label{eq:color_loss}
\end{equation}


\subsection{The Rendering Equation}
\looseness=-1
In inverse rendering, the outgoing radiance $C_o(x, \mathbf{v})$ at a specific surface point $x$ from viewing direction $\mathbf{v}$ is determined by the rendering equation~\citep{kajiya1986rendering}:
\begin{equation}
C_o(x, \mathbf{v}) = \int_{\Omega}L_i(x, \mathbf{l})f(\mathbf{l}, \mathbf{v}, x)(\mathbf{l}\cdot \mathbf{n})d\mathbf{l},
  \label{eq:outgoing_equation}
\end{equation}
\looseness=-1
where $L_i(x, \mathbf{l})$ is the incoming light from direction $\mathbf{l}$ at point $x$, $\mathbf{n}$ is the surface normal direction and $f$ is the Bidirectional
Reflectance Distribution Function (BRDF). The integral is computed in the upper hemisphere of the surface.

\subsection{The Bouguer-Beer-Lambert Law}
\looseness=-1
The Bouguer-Beer-Lambert law states the following~\citep{mayerhofer2020bouguer, bouguer1729essai, lambert1760photometria, beer1852bestimmung}: Consider a radiation of intensity $I$, i.e., a light of intensity $I$ passing through an absorbing body, the change of its intensity is governed by the following differential equation:
\begin{equation}
    dI_\nu = -\alpha_\nu I_\nu ds,
    \label{eq:radiative_transfer}
\end{equation}
\looseness=-1
where subscript $\nu$ means that the term depends on the frequency of light and $\alpha_\nu$ is called the extinction coefficient, an attribute that determines how energy is dampened in the absorbing body.

\looseness=-1
Solving for intensity $I_\nu$, we get
\begin{equation}
    I_\nu(s) = I_\nu(0)e^{-\alpha_\nu s},
    \label{eq:integral_radiative_transfer}
\end{equation}
here $e^{-\alpha_\nu s}$ is the transmittance of region $s$, meaning that the light intensity is exponentially decreasing when traveling through the absorbing body. If the attenuation of light in the body doesn't contain scattering effects, the extinction coefficient can be computed as
\begin{equation}
\alpha_\nu = n\sigma_\nu,
  \label{eq:extinction_coefficient}
\end{equation}

where $n$ is the number density and $\sigma_\nu$ is cross section. These two terms often appear in chemistry, $n$ describes the degree of concentration of countable objects (usually particles) and $\sigma_\nu$ models the the probability that two particles will collide, which is determined by the types of the particles. 
An illustration of the Bouguer-Beer-Lambert law can be found in Fig.~\ref{fig:cross_section}, the intensity of a light passing through a region filled with particles is reduced when ``hit'' with particles (normally the affecting area is bigger than the particle size). Intuitively speaking, the reduction in intensity is determined by the number of particles in the region that the light hits and the affecting area of the specific particle type, which corresponds to Eq.~\ref{eq:extinction_coefficient} and Eq.~\ref{eq:integral_radiative_transfer}.

\begin{wrapfigure}[30]{r}{0.45\textwidth}
  \centering
  % \fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
  \includegraphics[width=0.45\textwidth]{figs/cross_section.pdf}
  \caption{An illustration of the Bouguer-Beer-Lambert law. Consider a ray passing through an absorbing body consists of particles of some type. The cross section depends on the area that each particle would affect (usually bigger than the size of the particle and equals the size of the particle if treated as a hard sphere). When traveling, the intensity of the light would be reduced by the areas that each particle affected. The reduction in intensity is therefore affected by the area around the particles and the number of particles, corresponding to cross section $\sigma$ and number density $n$ in the main paper.}
  \label{fig:cross_section}
\end{wrapfigure}

\section{Method}
We start by introducing the intuition behind our method. It is well-known that different materials react differently when hit by a beam of light, e.g., a wooden plank would diffuse the light and let nothing pass through, while a glass pane may allow light to travel inside with no dampening. In physics, this phenomenon is described by the Bouguer-Beer-Lambert law, suggesting that the opacity of an object depends on its material. Motivated by this, we do a series of derivation and propose a plug-and-play solution suggesting that opacity should be a function of material and the computation of transmittance should follow the exponential decay rate as in the Bouguer-Beer-Lambert law.

\subsection{Gaussian Blobs as Absorbing Body}
Our approach arises from the Bouguer-Beer-Lambert law. Recall in Eq.~\ref{eq:alpha_blending}, $T_i$ stands for the accumulated transmittance of the first $(i-1)$ Gaussians, indicating the reduction of light intensity of the $i$-th Gaussian when observed from the camera perspective. For each Gaussian $G_i$, the transmittance is calculated by $1-\alpha_i(x)$. According to the Bouguer-Beer-Lambert law, the transmittance of an absorbing body should follow the form $e^{-\alpha_\nu s}$ or $e^{-n\sigma_\nu s}$ as in Eq.~\ref{eq:integral_radiative_transfer}. Therefore we have
\begin{equation}
\begin{aligned}
1-\alpha_i(x) &= e^{-n\sigma_\nu s}, \\
\alpha_i(x) &= 1-e^{-n\sigma_\nu s}.
\end{aligned}
\label{eq:transmittance}
\end{equation}

A natural question then arises: \textbf{what is $n$,  $\sigma_\nu$ and $s$ in the context of Gaussian Splatting?} 

To answer this question, recall the definition of $\alpha_i(x)$ in Eq.~\ref{eq:alpha_equation}. The term $o_i$ indicates how ``dense'' the Gaussian is, and the exponential term suggests that the Gaussian is densest at the center and becomes lighter when away from it as shown in Fig.~\ref{fig:method}. Since each Gaussian blob is not solid, it is natural to think of it as a blob of gas with its density following a Gaussian distribution. Notice that in this formulation, the density corresponds to the number density $n$ in Eq.~\ref{eq:transmittance}. Formally, we have
\begin{equation}
n_i = o_i \exp(-\frac{1}{2}(x-R(\mu_i; \theta))^TR_{\theta}(\Sigma_i)^{-1}(x-R(\mu_i; \theta))).
\label{eq:n}
\end{equation}
\looseness=-1
\RebuttalRevision{As for path length $s$, since each Gaussian is ``splatted'' to a 2D plane, i.e. density is marginalized along the viewing axis, the distance of the light passing through each Gaussian can be set as a constant $1$. In fact, the blending algorithm omits the depth of the Gaussians into the formulation of the opacity value at the pixel location. We therefore observed that we can treat this opacity value as the density and since there is no concept of depth on the 2D plane, each Gaussian is then treated equally from this perspective, therefore it is reasonable to treat the path length to constant 1.}

\subsection{Material Network for Cross Section Modeling}
\looseness=-1
After deciding on the correspondence of existing terms, we found that there is still one term missing from the Bouguer-Beer-Lambert law, namely the cross section $\sigma_\nu$. Cross section, by definition, is determined by the type of particle that fills the region as illustrated in Fig.~\ref{fig:cross_section}, namely the material of the region. On a macro level, cross section affects the color that one can perceive from the region. Since in inverse rendering, material properties, i.e., albedo, metalness and roughness, are explicitly included in the model (for 3DGS-based method they're set as additional parameters for each Gaussian), we can directly use the parameter for representing the cross section. In practice, a neural network $f(\cdot)$ that takes as input the material properties $m$ and outputs cross section $\sigma_\nu$ is introduced, namely
\begin{equation}
\sigma_\nu = f(m).
\label{eq:f}
\end{equation}

In summary, we have now determined the three terms suggested by the Bouguer-Beer-Lambert law in the context of Gaussian Splatting. Therefore, we can compute $\alpha_i(x)$ in Eq.~\ref{eq:alpha_equation} differently as  
\begin{equation}
\begin{aligned}
\alpha_i(x) &= 1-e^{-n_i(x)\sigma_{\nu i}} \\  
            &= 1-e^{-o_iG_i(x)f(m_i)}, \\
\end{aligned}
\label{eq:alpha_new}
\end{equation}
where $o_i$ is the original opacity term for the $i$-th Gaussian, $G_i(x)$ is the normal distribution in Eq.~\ref{eq:alpha_equation}, $f$ is the neural network that produces cross section and $m_i$ is the material property.

\subsection{Gradient Flow}
\looseness=-1
Consider $c_i$ in Eq.~\ref{eq:alpha_blending}, in inverse rendering: it is computed using the material properties in Eq.~\ref{eq:outgoing_equation}. Therefore, our formulation of material takes gradients not only from the color term itself, but also from the $\alpha$ term. In this case,
\begin{equation}
    \frac{\partial L}{\partial m_i} = \frac{\partial c_i}{\partial m_i}\frac{\partial L}{\partial C}\alpha_i T_i + c_i\frac{\partial L}{\partial C}T_i\frac{\partial \alpha_i}{\partial m_i} - \sum_{j=i+1}^N c_j \frac{\partial L}{\partial C}T_j\alpha_j\frac{1}{1-\alpha_i}\frac{\partial \alpha_i}{\partial m_i},
\label{eq:derivative}
\end{equation}
\looseness=-1
where $L$ stands for loss. Our formulation augments the gradient for material learning, allowing the gradient to flow from opacity to serve as an additional constraint in a physics-inspired way, as illustrated by Fig.~\ref{fig:method}.


\subsection{Analysis}
\looseness=-1
Apart from the derivation of our formulation from the Bouguer-Beer-Lambert law, we found that our formulation could also be interpreted from two other perspectives, indicating the correctness of our approach.

\paragraph{NeRF.} The alpha-blending algorithm in NeRF~\citep{mildenhall2021nerf} takes the form
\begin{equation}
\begin{aligned}
C(r) &= \sum_{i=1}^N c_i(1-\exp(-\sigma_i\delta_i))T_i, \\
T_i &= \exp(-\sum_{j=1}^{i-1}\sigma_j\delta_j),    \\
\end{aligned}
\label{eq:nerf}
\end{equation}
in which $r$ stands for the ray direction, $c_i$ is a view-dependent color of the sampled points, $\sigma_i$ is volume density and $\delta_i$ is the distance between two sampled points. In this formulation, $\alpha_i = 1-\exp(-\sigma_i\delta_i)$, which takes exactly the same form as our derivation from the Bouguer-Beer-Lambert law. Neural field-based methods~\citep{bi2020neural,boss2021nerd} that use ray marching in a particle-based manner for inverse rendering handle cross section implicitly by using the same MLP for modeling volume density and material, suggesting the necessity of introducing cross section into the \textbf{disentangled} representation of 3DGS-based methods. 

\looseness=-1
\paragraph{Taylor Expansion.} Through Taylor expansion w.r.t.\ the function $\alpha(t) = 1-\exp(-t)$, we have 
\begin{equation}
\begin{aligned}
  \alpha(t) &= 1-e^{-t} \\
  &= 1 - (1 - t + o(t)) \\
  &= t + o(t),
\end{aligned}
\label{eq:taylor}
\end{equation}
\looseness=-1
where $o(t)$ is the little-o notation. This indicates that Eq.~\ref{eq:alpha_equation} is an approximation to the function $\alpha_i(x) = 1-\exp(-o_iG_i(x))$, which is the exact same form as Eq.~\ref{eq:alpha_new}, omitting the newly introduced cross section $f(m_i)$. This derivation indicates the mathematical correctness of the newly introduced activation function.


\section{Experiments}
\label{sec:experiment}
We discuss the implementation details to show the experimental design and present experimental results on synthetic and real-world data to provide a comprehensive understanding of our method.

\subsection{Implementation Details}
\looseness=-1
Since our proposed method is independent of the underlying model for inverse rendering, we can freely plug it into three state-of-the-art baselines, namely GaussianShader~\citep{jiang2024gaussianshader}, GS-IR~\citep{liang2024gs} and R3DG~\citep{gao2023relightable} for evaluation. All the experiments are conducted on a single NVIDIA RTX 6000 Ada Generation GPU and we report the reproduced baseline results for fair comparison. We use an MLP that has 2 hidden layers with size 128 for the hidden dimension and ReLU as activation function for cross section and the output activation is a sigmoid function. The input to the MLP is designed as follows: for GaussianShader, an end-to-end trained material modeling pipeline, we directly input the albedo, roughness and specular tint terms as in their original model for MLP input. For GS-IR and R3DG, since they follow a two-stage training pipeline, in which the first stage uses normal rendering for geometry reconstruction and second stage uses physically-based rendering for material modeling, we input the material properties as well as the SH coefficients used for color in the first stage to the MLP. The logic behind this design is to make the optimization of the MLP in the first stage meaningful. We follow the experimental settings as provided by the original baselines. GaussianShader is evaluated on Shiny Blender~\citep{verbin2022ref} and Glossy Synthetic~\citep{liu2023nero} as in the orignal work. For R3DG, Synthetic4Relight dataset~\citep{zhang2022modeling} is used for getting material modeling evaluation results. GS-IR is evaluated on MIP-NeRF 360 dataset~\citep{barron2022mip} to provide insights on real-world data. For albedo and relighting evaluation, we follow the standard practice to standardize the albedo prediction against the ground truth.

\begin{figure}[h]
\begin{center}
%\framebox[4.0in]{$\;$}
% \fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
\includegraphics[width=1.0\textwidth]{figs/gamma_main.pdf}
\end{center}
\caption{Qualitative comparison of albedo, relighting and roughness on Synthetic4Relight dataset~\citep{zhang2022modeling}. Our method is able to decouple specular effects from albedo compared to R3DG, the baseline we implemented our method on. It is also capable of providing roughness estimation more precisely.\RebuttalRevision{We add a gamma correction to the relighting results to make the visual differences clearer.}}
\label{fig:qual_albedo}
\end{figure}
\begin{table}[t]
\centering
\caption{Quantitative results on Synthetic4Relight dataset~\citep{zhang2022modeling}. Our method is able to provide significant improvements universally. The improvements in novel view synthesis and relighting can be attribute to the improved estimation of albedo and roughness thanks to the correct modeling we use for constraining material properties.}
\label{tab:exp_syn4}
\resizebox{\linewidth}{!}{
\begin{tabular}{lccccccccccc}
\toprule
     & \multicolumn{3}{c}{\textbf{Novel View Synthesis}} & \multicolumn{3}{c}{\textbf{Relighting}} & \multicolumn{3}{c}{\textbf{Albedo}} & \textbf{Roughness} \\
     \cmidrule(lr){2-4}\cmidrule(lr){5-7}\cmidrule(lr){8-10}\cmidrule(lr){11-11}
                  & PSNR↑  & SSIM↑ & LPIPS↓ & PSNR↑    & SSIM↑    & LPIPS↓   & PSNR↑   & SSIM↑  & LPIPS↓  & MSE↓   \\ \midrule
R3DG    &   34.07  &  0.975   &   0.047     &  32.87       &    0.967      &    0.052 & 28.70   &    0.952    &    0.064     &    0.011   \\ 
R3DG + Ours    &   \textbf{34.47}  &  \textbf{0.977}   &   \textbf{0.046}     &  \textbf{33.21}       &    \textbf{0.969}      &    \textbf{0.050} & \textbf{29.31}   &    \textbf{0.954}    &    \textbf{0.059}     &    \textbf{0.007}   \\ 
\bottomrule
\end{tabular}
}
\end{table}

\subsection{Results on Synthetic Data}
\label{sec:mat}
\looseness=-1
\paragraph{Material Modeling.} We first evaluate our proposed module on top of R3DG~\citep{gao2023relightable} for material modeling evaluation. As can be seen from Tab.~\ref{tab:exp_syn4}, with the help of our modeling, R3DG is able to produce much better results universally. Our method outperforms the baseline on novel view synthesis by $0.4$\,db PSNR, as well as on two other metrics. The relighting results shares similar performance boosts as novel view synthesis, which could be attributed to the significant improvements in albedo ($\sim 0.6$\,db PSNR boost) and roughness estimation. Qualitatively speaking, as can be seen from the visualization in Fig.~\ref{fig:qual_albedo}, our method is capable of producing albedo estimation with less specular effects and roughness estimation that aligns better with the ground truth, resulting in pleasing relighting results.

\paragraph{Novel View Synthesis on Synthetic Data}
Other than the results on Synthetic4Relight, we also implemented our method on top of GaussianShader~\citep{jiang2024gaussianshader} to get more insights on how our model would perform on synthetic data. Tab.~\ref{tab:exp_shiny} showcases the result on Shiny Blender~\citep{verbin2022ref} and Glossy Synthetic~\citep{liu2023nero}. Our method is able to produce significant improvement in terms of novel view synthesis when built on top of GaussianShader (specifically $\sim 0.3$\,db PSNR boost on Shiny Blender and $\sim 0.4$\,db boost on Glossy Synthetic). Fig.~\ref{fig:qual_shiny} showcases some qualitative comparisons of our model with the GaussianShader baseline. While we did not add any constraint to how the model predicts normals, our method is able to produce much more satisfactory normal estimation compared to the baseline as well as better modeling of light reflection. 
\begin{figure}[h]
\begin{center}
\includegraphics[width=1.0\textwidth]{figs/viz_gs.pdf}
\end{center}
\caption{\textbf{Qualitative comparison on Shiny Blender~\citep{verbin2022ref} and Glossy Synthetic~\citep{liu2023nero} datasets.} Our method is able to provide satisfactory normal estimation compared to the baseline. Notice that our method does not include any additional supervision or constraints for normal estimation. Our method is also able to provide much more accurate view synthesis results when doing physically-based rendering. These results suggest the significance of introducing the physically correct model for inverse rendering.}
\label{fig:qual_shiny}
\end{figure}
\begin{table}[t]
\centering
\caption{Quantitative results when combined with GaussianShader~\citep{jiang2024gaussianshader}. We compare with GaussianShader on Shiny Blender~\citep{verbin2022ref} and Glossy Synthetic datasets~\citep{liu2023nero}. Our method is able to outperform GaussianShader on novel view synthesis in tems of all three standard image error metrics.}
\vspace{-2pt}
\label{tab:exp_shiny}
\resizebox{0.8\linewidth}{!}{
\begin{tabular}{lccccccc}
\toprule
     & \multicolumn{3}{c}{\textbf{Shiny Blender}} & \multicolumn{3}{c}{\textbf{Glossy Synthetic}}  \\
          \cmidrule(lr){2-4}\cmidrule(lr){5-7}        & PSNR↑  & SSIM↑ & LPIPS↓ & PSNR↑    & SSIM↑    & LPIPS↓   \\ \midrule
GaussianShader    &   30.77  &   0.953  &    0.081   &  27.25       &     0.925    &   0.083    \\
GaussianShader + Ours    &   \textbf{31.07}  &  \textbf{0.957}   &   \textbf{0.076}    &  \textbf{27.68}       &   \textbf{0.929}   &    \textbf{0.081}   \\
\bottomrule
\end{tabular}
}
\vspace{-2pt}
\end{table}

\subsection{Results on Real-World Data}
\looseness=-1
Apart from the results on synthetic data, we also analyze the effect of our method when applied to real-world data. To verify the plug-and-play attribute of our method, we apply it to GS-IR~\citep{liang2024gs} and conduct experiments on the MIP-NeRF 360 dataset~\citep{barron2022mip}. The experimental results shown in Tab.~\ref{tab:exp_real} indicate a significant universal improvement in terms of novel view synthesis thanks to the correct modeling and better estimation of materials as indicated by the results in Sec.~\ref{sec:mat}. Qualitatively speaking, our method is able to produce less blurry images compared to the GS-IR baseline as shown by the bicycle scene in the first row in Fig.~\ref{fig:qual_mip}. 

\looseness=-1
\paragraph{Analysis.} Since our method augments opacity with a material-dependent term, it turns the opacity from a purely geometry-indicator to a combined representative of geometry and material. That being said, the optimization of geometry is also affected by our method. This is obvious if we consider the gradient of $\alpha_i$ w.r.t.\ $o_i$:
\begin{equation}
\frac{\partial \alpha_i}{\partial o_i} = G_if(m_i)(1-\alpha_i),
    \label{eq:o_grad}
\end{equation}
pixel location $x$ is omitted for simplicity. We hypothesize that since MIP-NeRF 360 supports SfM initialization, it helps our method by easing the burden for geometry reconstruction, therefore leading to more improvements compared to synthetic data which uses random initialization. 

\begin{figure}[h]
\begin{center}
\includegraphics[width=1.0\textwidth]{figs/viz_gsir.pdf}
%\framebox[4.0in]{$\;$}
% \fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
\end{center}
\caption{\textbf{Qualitative Comparison of GS-IR on MIP-NeRF 360~\citep{barron2022mip}.} Our method is able to provide much more accurate novel view synthesis results compared to the baseline GS-IR~\citep{liang2024gs} in which we plugged our model. The blurriness in the Bicycle scene in the first row predicted by GS-IR is overcome by our approach. The second row showcase that our approach is able to provide much more fine-grained and accurate normal estimation.}
\label{fig:qual_mip}
\end{figure}

\begin{table}
\begin{center}
\setlength\tabcolsep{2 pt}
\begin{tabular}{cl|c|ccccccccc}
\toprule
\multicolumn{2}{c|}{\textbf{Mip-NeRF 360}} & Avg. & Bicycle & Flowers & Garden & Stump & Treehill & Room & Counter & Kitchen & Bonsai\\ \midrule\midrule
\multirow{2}{*}{PSNR$\uparrow$} & GS-IR& 25.38 & 23.31 & \textbf{20.43} & 25.76 & 25.46 & 21.75 & 28.32 & 26.31 & 28.66 & 28.44\\
 & Ours &  \textbf{25.85} & \textbf{24.09} & 20.16 & \textbf{26.24} & \textbf{25.54} & \textbf{21.97} & \textbf{29.64} & \textbf{26.85} & \textbf{28.92}  &\textbf{29.27} \\ \midrule
\multirow{2}{*}{SSIM$\uparrow$} & GS-IR & 0.760 & 0.688 & \textbf{0.542} & 0.808 & 0.717 & 0.583 & 0.874 & 0.847 &0.877 & 0.901 \\
 & Ours & \textbf{0.767} & \textbf{0.720} & 0.510 & \textbf{0.824}  & \textbf{0.720} & \textbf{0.586} & \textbf{0.897} & \textbf{0.859} & \textbf{0.884} & \textbf{0.905}\\ \midrule
\multirow{2}{*}{LPIPS$\downarrow$}  & GS-IR & 0.265 & 0.269 & \textbf{0.368} & 0.154 & 0.256 & \textbf{0.367} &  0.281 &  0.255 & 0.179 & 0.254\\
 & Ours & \textbf{0.254} & \textbf{0.230} & 0.401 & \textbf{0.140} & \textbf{0.245} & 0.368 & \textbf{0.247} & \textbf{0.244} & \textbf{0.168} & \textbf{0.243} \\
 \bottomrule
\end{tabular}
\caption{\textbf{Quantatitive Comparison on Mip-NeRF 360~\citep{barron2022mip}.} The results show that our approach, when combined with GS-IR, can significantly improve the novel view synthesis results of real-world scenes on MIP-NeRF 360~\citep{barron2022mip}. Thanks to the constraint we add to opacity, our method is able to produce $\sim 0.5$\,db PSNR gains along with SSIM and LPIPS improvements.}
\label{tab:exp_real}
\vspace{-0.6cm}
\end{center}
\end{table}

\section{Conclusion}
\vspace{-10pt}
\looseness=-1
In this work, we propose OMG, a plug-and-play module for inverse rendering based on Gaussian Splatting. Inspired by the Bouguer-Beer-Lambert law, we derive the exact form of how opacity should be modeled in a particle-based model. We analyze the proposed formulation against NeRF-based methods and from a mathematical perspective, providing more insights in the right modeling for inverse rendering. Our method also reveals again the similarity between NeRF-based methods and 3D Gaussian Splatting. To validate the effectiveness and the plug-and-play nature of the proposed modeling, we implement it on top of three different state-of-the-art baselines and conduct comprehensive experiments on synthetic and real-world data. The significant improvements showcased by the experiments indicate the correctness of the modeling and point out the significance of the right prior and modeling for visual computing. We believe our work could inspire the community to pay more attention to developing the physically correct model and introducing right priors for modeling the 3D world.
\paragraph{Limitation.} In OMG, the dependency of cross section on the frequency of light is not included in the modeling. Future work could introduce the correct dependency of cross section on frequency of light to provide more accurate modeling of the physical world. \RebuttalRevision{As shown in Sec. \ref{sec:cross_viz} and Fig. \ref{fig:cross_viz}, the learned cross section is still not ideal if applied to the specular region with strong lighting. This motivates us to explore more on how to regularize the learning of materials and cross section in a more precise way.}






% \section*{Ethics Statement}
% Our work focuses on incorporating physically correct priors to improve inverse rendering results, aiming to bring new insights to visual computing community. Our research does not involve human subjects, sensitive data, or any practices that pose privacy or security concerns.

% \section*{Reproducibility Statement}
% The baselines used in our experiments, such as GaussianShader and GS-IR, are open-source and publicly available. We have detailed our experimental setup, and other design choices, in Section~\ref{sec:experiment} of the main paper  and Section~\ref{sec:hyper} in the Appendix to ensure reproducibility. Please also be assured that the code implementation of this work will be made publicly accessible upon the acceptance of this paper.  

\section*{Acknowledgement}
The author would like to thank Julia Du for the help of the title and the beautiful figures. This work has been funded in part by the Army Research Laboratory (ARL) award W911NF-23-2-0007, DARPA award FA8750-23-2-1015, and ONR award N00014-23-1-2840.

% \newpage
\bibliography{iclr2025_conference}
\bibliographystyle{iclr2025_conference}


%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%
\clearpage
\appendix
\renewcommand{\thesection}{\Alph{section}}
\renewcommand\thefigure{\Alph{section}\arabic{figure}} 
\renewcommand\thetable{\Alph{section}\arabic{table}}  
\setcounter{section}{0}
\setcounter{figure}{0} 
\setcounter{table}{0} 

{\LARGE\sc \textbf{O{\Huge M}G}: \underline{O}pacity \underline{M}atters in \underline{M}aterial \underline{M}odeling with \underline{G}aussian splatting\par}

{\LARGE\sc Appendix\par} \vspace{10pt}
\section{Hyperparameters}
\label{sec:hyper}
The neural network introduced uses $0.001$, $0.0001$, $0.0005$ and $0.007$ as learning rate respectively for Synthetic4Relight, Shiny Blender, Glossy Synthetic and MIP-NeRF 360 datasets.\RebuttalRevision{The neural network is implemented as a fully-connected MLP with two hidden layers, 128 as hidden dimension and ReLU as activation function. We directly feed the material properties into the MLP without any encoding such as positional embedding.}
\section{Full Derivation of Equations}
Here we give the full derivation of Eq.~\ref{eq:derivative}.
\begin{equation}
\begin{aligned}
    % \frac{\partial L}{\partial m_i} &= \frac{\partial L}{\partial C}\frac{\partial C}{\partial m_i} = \frac{\partial L}{\partial C}\sum_{j=1}^N \frac{\partial (c_j\alpha_jT_j)}{\partial m_i}\\
    % &= \frac{\partial L}{\partial C}[\frac{\partial c_i}{\partial m_i}\alpha_i T_i]\\
    % &= \frac{\partial c_i}{\partial m_i}\frac{\partial L}{\partial C}\alpha_i T_i + c_i\frac{\partial L}{\partial C}T_i\frac{\partial \alpha_i}{\partial m_i} - \sum_{j=i+1}^N c_j \frac{\partial L}{\partial C}T_j\alpha_j\frac{1}{1-\alpha_i}\frac{\partial \alpha_i}{\partial m_i},
    \frac{\partial L}{\partial m_i} &= \frac{\partial L}{\partial C}\frac{\partial C}{\partial m_i} = \frac{\partial L}{\partial C}\sum_{j=1}^N \frac{\partial (c_j\alpha_jT_j)}{\partial m_i}\\
    &= \frac{\partial L}{\partial C}\sum_{j=1}^N[\frac{c_j}{m_i}\alpha_jT_j + c_j\frac{\partial \alpha_j}{\partial m_i}T_j + c_j\alpha_j\frac{\partial T_j}{\partial m_i}]\\
    &= \frac{\partial L}{\partial C}[\frac{\partial c_i}{\partial m_i}\alpha_iT_i + c_iT_i\frac{\partial \alpha_i}{\partial m_i} + \sum_{j=i+1}^{N}c_j\alpha_j\frac{\partial}{\partial m_i}\prod_{k=1}^{j-1}(1-\alpha_k)]\\
    &= \frac{\partial L}{\partial C}[\frac{\partial c_i}{\partial m_i}\alpha_iT_i + c_iT_i\frac{\partial \alpha_i}{\partial m_i} + \sum_{j=i+1}^{N}c_j\alpha_j\frac{\partial}{\partial m_i}\exp(\sum_{k=1}^{j-1}(1-\alpha_k))]\\
    &= \frac{\partial c_i}{\partial m_i}\frac{\partial L}{\partial C}\alpha_i T_i + c_i\frac{\partial L}{\partial C}T_i\frac{\partial \alpha_i}{\partial m_i} - \sum_{j=i+1}^N c_j \frac{\partial L}{\partial C}T_j\alpha_j\frac{1}{1-\alpha_i}\frac{\partial \alpha_i}{\partial m_i},
\end{aligned}
\label{eq:derivative_supp}
\end{equation}

\section{Additional Results}
We present the full results on Synthetic4Relight~\citep{zhang2022modeling}, Shiny Blender~\citep{verbin2022ref} and Glossy Synthetic~\citep{liu2023nero}. All the baseline results are reproduced by us as in the main paper. More visualization results can be found in Fig.~\ref{fig:qual_shiny_supp}, Fig.~\ref{fig:qual_mip_supp}, Fig.~\ref{fig:qual_albedo_supp}, Fig.~\ref{fig:qual_albedo_supp_2} and Fig.~\ref{fig:qual_normal_supp}.


\begin{table*}[h]
\centering
\setlength{\tabcolsep}{2pt} 
\resizebox{\textwidth}{!}{
\begin{tabular}{lc|ccc|ccc|ccc|c}
\toprule
\multirow{2}{*}[-0.7ex]{Scene} &
\multirow{2}{*}[-0.7ex]{Method} &
\multicolumn{3}{c|}{Novel View Synthesis} &
\multicolumn{3}{c|}{Albedo} &
\multicolumn{3}{c|}{Relight} &
{Roughness}\\
\cmidrule{3-5}\cmidrule{6-8}\cmidrule{9-11}\cmidrule{12-12}
& &
PSNR $\uparrow$ & SSIM $\uparrow$ & LPIPS $\downarrow$ &
PSNR $\uparrow$ & SSIM $\uparrow$ & LPIPS $\downarrow$ &
PSNR $\uparrow$ & SSIM $\uparrow$ & LPIPS $\downarrow$ & MAE $\downarrow$ \\

\midrule\midrule
\multirow{2}{*}{Balloons}
& R3DG   & 31.90  &
0.967 & \textbf{0.072} & 24.82 &
0.923 & 0.070 & 31.38 &
0.965 & 0.078 & \textbf{0.0009}  \\
& Ours      & \textbf{32.11}  &
\textbf{0.968} & \textbf{0.072} & \textbf{25.84} &
\textbf{0.926} & \textbf{0.055} & \textbf{32.50} &
\textbf{0.967} & \textbf{0.071} & 0.0011 \\
\midrule
\multirow{2}{*}{Hotdog}
& R3DG   & 32.61 &
0.972 & 0.050 & 26.53 &
0.956 & 0.089 & \textbf{29.04} &
0.954 & 0.067 & 0.0357 \\
& Ours      & \textbf{33.59} &
\textbf{0.975} & \textbf{0.047} & \textbf{26.95} &
\textbf{0.957} & \textbf{0.085} & 28.73 &
\textbf{0.955} & \textbf{0.066} & \textbf{0.0217} \\
\midrule
\multirow{2}{*}{Chair}
& R3DG   & 34.72 &
0.978 & \textbf{0.039} & 30.55 &
0.963 & 0.046 & 33.07 &
0.966 & \textbf{0.041} & \textbf{0.0036} \\
& Ours      & \textbf{35.00} &
\textbf{0.979} & 0.040 & \textbf{30.82} &
\textbf{0.964} & \textbf{0.045} & \textbf{33.46} &
\textbf{0.968} & \textbf{0.041} & 0.0039 \\
\midrule
\multirow{2}{*}{Jugs}
& R3DG   & 37.03 &
\textbf{0.984} & \textbf{0.027} & 32.88 &
0.967 & 0.064 & 37.98 &
\textbf{0.984} & \textbf{0.023} & \textbf{0.0032} \\
& Ours      & \textbf{37.17} &
\textbf{0.984} & \textbf{0.027} & \textbf{33.64} &
\textbf{0.968} & \textbf{0.049} & \textbf{38.16} &
\textbf{0.984} & \textbf{0.023} & 0.0074 \\
\bottomrule
\end{tabular}
}
\caption{Per-scene results on Synthetic4Relight dataset. For albedo reconstruction results, we follow the standard practice \citep{zhang2021nerfactor, gao2023relightable} and scale each RGB channel by a global scalar.}
\label{tab:full_syn}
\end{table*}

\begin{table*}
\begin{center}
% \setlength\tabcolsep{2 pt}
\resizebox{\textwidth}{!}{
\begin{tabular}{cl|c|cccccc}
\toprule
\multicolumn{2}{c|}{\textbf{Shiny Blender}} & Avg. & Ball & Car & Coffee & Helmet & Teapot & Toaster\\ \midrule\midrule
\multirow{2}{*}{PSNR$\uparrow$} & GaussianShader & 30.77 & 29.05 & \textbf{28.51} & \textbf{31.55} & \textbf{28.91} & \textbf{43.57} & 23.05 \\
 & Ours & \textbf{31.07} & \textbf{29.65} & 28.14 & 31.29 & 28.79 & 43.48 & \textbf{25.07} \\ \midrule
\multirow{2}{*}{SSIM$\uparrow$} & GaussianShader & 0.953 & 0.955 & \textbf{0.940} & \textbf{0.970} & \textbf{0.956} & \textbf{0.996} & 0.902 \\
 & Ours & \textbf{0.957} & \textbf{0.961} & 0.939 & 0.969 & 0.955 & \textbf{0.996} & \textbf{0.924}\\ \midrule
\multirow{2}{*}{LPIPS$\downarrow$}  & GaussianShader & 0.081 & 0.146 & \textbf{0.047} & \textbf{0.085} & 0.086 & \textbf{0.011} &  0.110\\
 & Ours & \textbf{0.076} & \textbf{0.137} & 0.050 & 0.086 & \textbf{0.085} & \textbf{0.011} & \textbf{0.089} \\
 \bottomrule
\end{tabular}
}
\caption{Per-scene results on Shiny Blender dataset.}
\label{tab:full_shiny}
\vspace{-0.6cm}
\end{center}
\end{table*}



\begin{table*}
\begin{center}
\setlength\tabcolsep{2 pt}
\begin{tabular}{cl|c|cccccccc}
\toprule
\multicolumn{2}{c|}{\textbf{Glossy Synthetic}} & Avg. & Angel & Bell & Cat & Horse & Luyu & Potion & Tbell & Teapot\\ \midrule\midrule
\multirow{2}{*}{PSNR$\uparrow$} & GaussianShader & 27.25 & 26.92 & 29.73 & \textbf{31.43} & 26.38 & {27.31} & \textbf{29.54} & 23.41 & 23.25\\
 & Ours &  \textbf{27.68} & \textbf{28.24} & \textbf{29.91} & 31.00 & \textbf{26.62} & \textbf{27.32} & 29.40 & \textbf{24.67} & \textbf{24.31} \\ \midrule
\multirow{2}{*}{SSIM$\uparrow$} & GaussianShader & 0.925 & 0.923 & 0.939 & \textbf{0.961} & \textbf{0.934} & \textbf{0.917} & \textbf{0.936} & 0.897 & 0.895 \\
 & Ours & \textbf{0.929} & \textbf{0.929} & \textbf{0.940} & 0.959  & \textbf{0.934} & 0.916 & 0.935 & \textbf{0.907} & \textbf{0.909}\\ \midrule
\multirow{2}{*}{LPIPS$\downarrow$}  & GaussianShader & 0.083 & 0.071 & 0.085 & \textbf{0.058} & \textbf{0.056} & \textbf{0.066} &  \textbf{0.096} &  0.136 & 0.098\\
 & Ours & \textbf{0.081} & \textbf{0.066} & \textbf{0.084} & 0.059 & 0.057 & 0.067 & 0.097 & \textbf{0.125} & \textbf{0.091}\\
 \bottomrule
\end{tabular}
\caption{Per-scene results on Glossy Synthetic dataset.}
\label{tab:full_glossy}
\vspace{-0.6cm}
\end{center}
\end{table*}


\begin{figure}[h]
\begin{center}
\includegraphics[width=1.0\textwidth]{figs/viz_supp_gs.pdf}
%\framebox[4.0in]{$\;$}
% \fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
\end{center}
\caption{More qualitative example in comparison to GaussianShader on Glossy Synthetic dataset.}
\label{fig:qual_shiny_supp}
\end{figure}

\begin{figure}[h]
\begin{center}
\includegraphics[width=1.0\textwidth]{figs/viz_supp_mip.pdf}
%\framebox[4.0in]{$\;$}
% \fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
\end{center}
\caption{More qualitative example in comparison to GS-IR on MIP-NeRF 360 dataset.}
\label{fig:qual_mip_supp}
\end{figure}

\begin{figure}[h]
\begin{center}
%\framebox[4.0in]{$\;$}
% \fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
\includegraphics[width=1.0\textwidth]{figs/viz_supp_alb.pdf}
\end{center}
\caption{More qualitative example in comparison to R3DG on Synthetic4Relight dataset.}
\label{fig:qual_albedo_supp}
\end{figure}

\begin{figure}[h]
\begin{center}
%\framebox[4.0in]{$\;$}
% \fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
\includegraphics[width=1.0\textwidth]{figs/gamma_supp.pdf}
\end{center}
\caption{\RebuttalRevision{More qualitative example in comparison to R3DG on Synthetic4Relight dataset. We add a gamma correction to the relighting results to make the visual differences clearer.}}
\label{fig:qual_albedo_supp_2}
\end{figure}

\begin{figure}[h]
\begin{center}
\includegraphics[width=1.0\textwidth]{figs/supp_normal.pdf}
%\framebox[4.0in]{$\;$}
% \fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
\end{center}
\caption{\RebuttalRevision{More qualitative example in comparison to GaussianShader on Glossy Synthetic and Shiny Blender datasets.}}
\label{fig:qual_normal_supp}
\end{figure}

\RebuttalRevision{
\section{Visualization of Cross Section}
\label{sec:cross_viz}
To understand what the cross section network is learning, we visualize some examples in Fig. \ref{fig:cross_viz}. In the figure, the darker region indicates a lower cross section, meaning that it lets more light pass through whereas the brighter regions stands for the regions that have a higher cross section. The example shows that the windows as well as the plastic bag on the floor actually have a lower cross section assigned to them, which corresponds to human intuition.
}

\begin{figure}[h]
\begin{center}
\includegraphics[width=1.0\textwidth]{figs/viz_supp_cross_mor.pdf}
%\framebox[4.0in]{$\;$}
% \fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
\end{center}
\caption{\RebuttalRevision{Visulization of the learned cross section on Mip-NeRF 360 dataset. Dark indicates the lower cross section (meaning it lets more light pass through) and bright stands for the higher cross section. The example shows that the windows as well as the plastic bag on the floor actually have a lower cross section assigned to them, which corresponds to human intuition.}}
\label{fig:cross_viz}
\end{figure}

\end{document}
%%%%% NEW MATH DEFINITIONS %%%%%

\usepackage{amsmath,amsfonts,bm}

% Mark sections of captions for referring to divisions of figures
\newcommand{\figleft}{{\em (Left)}}
\newcommand{\figcenter}{{\em (Center)}}
\newcommand{\figright}{{\em (Right)}}
\newcommand{\figtop}{{\em (Top)}}
\newcommand{\figbottom}{{\em (Bottom)}}
\newcommand{\captiona}{{\em (a)}}
\newcommand{\captionb}{{\em (b)}}
\newcommand{\captionc}{{\em (c)}}
\newcommand{\captiond}{{\em (d)}}

% Highlight a newly defined term
\newcommand{\newterm}[1]{{\bf #1}}


% Figure reference, lower-case.
\def\figref#1{figure~\ref{#1}}
% Figure reference, capital. For start of sentence
\def\Figref#1{Figure~\ref{#1}}
\def\twofigref#1#2{figures \ref{#1} and \ref{#2}}
\def\quadfigref#1#2#3#4{figures \ref{#1}, \ref{#2}, \ref{#3} and \ref{#4}}
% Section reference, lower-case.
\def\secref#1{section~\ref{#1}}
% Section reference, capital.
\def\Secref#1{Section~\ref{#1}}
% Reference to two sections.
\def\twosecrefs#1#2{sections \ref{#1} and \ref{#2}}
% Reference to three sections.
\def\secrefs#1#2#3{sections \ref{#1}, \ref{#2} and \ref{#3}}
% Reference to an equation, lower-case.
\def\eqref#1{equation~\ref{#1}}
% Reference to an equation, upper case
\def\Eqref#1{Equation~\ref{#1}}
% A raw reference to an equation---avoid using if possible
\def\plaineqref#1{\ref{#1}}
% Reference to a chapter, lower-case.
\def\chapref#1{chapter~\ref{#1}}
% Reference to an equation, upper case.
\def\Chapref#1{Chapter~\ref{#1}}
% Reference to a range of chapters
\def\rangechapref#1#2{chapters\ref{#1}--\ref{#2}}
% Reference to an algorithm, lower-case.
\def\algref#1{algorithm~\ref{#1}}
% Reference to an algorithm, upper case.
\def\Algref#1{Algorithm~\ref{#1}}
\def\twoalgref#1#2{algorithms \ref{#1} and \ref{#2}}
\def\Twoalgref#1#2{Algorithms \ref{#1} and \ref{#2}}
% Reference to a part, lower case
\def\partref#1{part~\ref{#1}}
% Reference to a part, upper case
\def\Partref#1{Part~\ref{#1}}
\def\twopartref#1#2{parts \ref{#1} and \ref{#2}}

\def\ceil#1{\lceil #1 \rceil}
\def\floor#1{\lfloor #1 \rfloor}
\def\1{\bm{1}}
\newcommand{\train}{\mathcal{D}}
\newcommand{\valid}{\mathcal{D_{\mathrm{valid}}}}
\newcommand{\test}{\mathcal{D_{\mathrm{test}}}}

\def\eps{{\epsilon}}


% Random variables
\def\reta{{\textnormal{$\eta$}}}
\def\ra{{\textnormal{a}}}
\def\rb{{\textnormal{b}}}
\def\rc{{\textnormal{c}}}
\def\rd{{\textnormal{d}}}
\def\re{{\textnormal{e}}}
\def\rf{{\textnormal{f}}}
\def\rg{{\textnormal{g}}}
\def\rh{{\textnormal{h}}}
\def\ri{{\textnormal{i}}}
\def\rj{{\textnormal{j}}}
\def\rk{{\textnormal{k}}}
\def\rl{{\textnormal{l}}}
% rm is already a command, just don't name any random variables m
\def\rn{{\textnormal{n}}}
\def\ro{{\textnormal{o}}}
\def\rp{{\textnormal{p}}}
\def\rq{{\textnormal{q}}}
\def\rr{{\textnormal{r}}}
\def\rs{{\textnormal{s}}}
\def\rt{{\textnormal{t}}}
\def\ru{{\textnormal{u}}}
\def\rv{{\textnormal{v}}}
\def\rw{{\textnormal{w}}}
\def\rx{{\textnormal{x}}}
\def\ry{{\textnormal{y}}}
\def\rz{{\textnormal{z}}}

% Random vectors
\def\rvepsilon{{\mathbf{\epsilon}}}
\def\rvtheta{{\mathbf{\theta}}}
\def\rva{{\mathbf{a}}}
\def\rvb{{\mathbf{b}}}
\def\rvc{{\mathbf{c}}}
\def\rvd{{\mathbf{d}}}
\def\rve{{\mathbf{e}}}
\def\rvf{{\mathbf{f}}}
\def\rvg{{\mathbf{g}}}
\def\rvh{{\mathbf{h}}}
\def\rvu{{\mathbf{i}}}
\def\rvj{{\mathbf{j}}}
\def\rvk{{\mathbf{k}}}
\def\rvl{{\mathbf{l}}}
\def\rvm{{\mathbf{m}}}
\def\rvn{{\mathbf{n}}}
\def\rvo{{\mathbf{o}}}
\def\rvp{{\mathbf{p}}}
\def\rvq{{\mathbf{q}}}
\def\rvr{{\mathbf{r}}}
\def\rvs{{\mathbf{s}}}
\def\rvt{{\mathbf{t}}}
\def\rvu{{\mathbf{u}}}
\def\rvv{{\mathbf{v}}}
\def\rvw{{\mathbf{w}}}
\def\rvx{{\mathbf{x}}}
\def\rvy{{\mathbf{y}}}
\def\rvz{{\mathbf{z}}}

% Elements of random vectors
\def\erva{{\textnormal{a}}}
\def\ervb{{\textnormal{b}}}
\def\ervc{{\textnormal{c}}}
\def\ervd{{\textnormal{d}}}
\def\erve{{\textnormal{e}}}
\def\ervf{{\textnormal{f}}}
\def\ervg{{\textnormal{g}}}
\def\ervh{{\textnormal{h}}}
\def\ervi{{\textnormal{i}}}
\def\ervj{{\textnormal{j}}}
\def\ervk{{\textnormal{k}}}
\def\ervl{{\textnormal{l}}}
\def\ervm{{\textnormal{m}}}
\def\ervn{{\textnormal{n}}}
\def\ervo{{\textnormal{o}}}
\def\ervp{{\textnormal{p}}}
\def\ervq{{\textnormal{q}}}
\def\ervr{{\textnormal{r}}}
\def\ervs{{\textnormal{s}}}
\def\ervt{{\textnormal{t}}}
\def\ervu{{\textnormal{u}}}
\def\ervv{{\textnormal{v}}}
\def\ervw{{\textnormal{w}}}
\def\ervx{{\textnormal{x}}}
\def\ervy{{\textnormal{y}}}
\def\ervz{{\textnormal{z}}}

% Random matrices
\def\rmA{{\mathbf{A}}}
\def\rmB{{\mathbf{B}}}
\def\rmC{{\mathbf{C}}}
\def\rmD{{\mathbf{D}}}
\def\rmE{{\mathbf{E}}}
\def\rmF{{\mathbf{F}}}
\def\rmG{{\mathbf{G}}}
\def\rmH{{\mathbf{H}}}
\def\rmI{{\mathbf{I}}}
\def\rmJ{{\mathbf{J}}}
\def\rmK{{\mathbf{K}}}
\def\rmL{{\mathbf{L}}}
\def\rmM{{\mathbf{M}}}
\def\rmN{{\mathbf{N}}}
\def\rmO{{\mathbf{O}}}
\def\rmP{{\mathbf{P}}}
\def\rmQ{{\mathbf{Q}}}
\def\rmR{{\mathbf{R}}}
\def\rmS{{\mathbf{S}}}
\def\rmT{{\mathbf{T}}}
\def\rmU{{\mathbf{U}}}
\def\rmV{{\mathbf{V}}}
\def\rmW{{\mathbf{W}}}
\def\rmX{{\mathbf{X}}}
\def\rmY{{\mathbf{Y}}}
\def\rmZ{{\mathbf{Z}}}

% Elements of random matrices
\def\ermA{{\textnormal{A}}}
\def\ermB{{\textnormal{B}}}
\def\ermC{{\textnormal{C}}}
\def\ermD{{\textnormal{D}}}
\def\ermE{{\textnormal{E}}}
\def\ermF{{\textnormal{F}}}
\def\ermG{{\textnormal{G}}}
\def\ermH{{\textnormal{H}}}
\def\ermI{{\textnormal{I}}}
\def\ermJ{{\textnormal{J}}}
\def\ermK{{\textnormal{K}}}
\def\ermL{{\textnormal{L}}}
\def\ermM{{\textnormal{M}}}
\def\ermN{{\textnormal{N}}}
\def\ermO{{\textnormal{O}}}
\def\ermP{{\textnormal{P}}}
\def\ermQ{{\textnormal{Q}}}
\def\ermR{{\textnormal{R}}}
\def\ermS{{\textnormal{S}}}
\def\ermT{{\textnormal{T}}}
\def\ermU{{\textnormal{U}}}
\def\ermV{{\textnormal{V}}}
\def\ermW{{\textnormal{W}}}
\def\ermX{{\textnormal{X}}}
\def\ermY{{\textnormal{Y}}}
\def\ermZ{{\textnormal{Z}}}

% Vectors
\def\vzero{{\bm{0}}}
\def\vone{{\bm{1}}}
\def\vmu{{\bm{\mu}}}
\def\vtheta{{\bm{\theta}}}
\def\va{{\bm{a}}}
\def\vb{{\bm{b}}}
\def\vc{{\bm{c}}}
\def\vd{{\bm{d}}}
\def\ve{{\bm{e}}}
\def\vf{{\bm{f}}}
\def\vg{{\bm{g}}}
\def\vh{{\bm{h}}}
\def\vi{{\bm{i}}}
\def\vj{{\bm{j}}}
\def\vk{{\bm{k}}}
\def\vl{{\bm{l}}}
\def\vm{{\bm{m}}}
\def\vn{{\bm{n}}}
\def\vo{{\bm{o}}}
\def\vp{{\bm{p}}}
\def\vq{{\bm{q}}}
\def\vr{{\bm{r}}}
\def\vs{{\bm{s}}}
\def\vt{{\bm{t}}}
\def\vu{{\bm{u}}}
\def\vv{{\bm{v}}}
\def\vw{{\bm{w}}}
\def\vx{{\bm{x}}}
\def\vy{{\bm{y}}}
\def\vz{{\bm{z}}}

% Elements of vectors
\def\evalpha{{\alpha}}
\def\evbeta{{\beta}}
\def\evepsilon{{\epsilon}}
\def\evlambda{{\lambda}}
\def\evomega{{\omega}}
\def\evmu{{\mu}}
\def\evpsi{{\psi}}
\def\evsigma{{\sigma}}
\def\evtheta{{\theta}}
\def\eva{{a}}
\def\evb{{b}}
\def\evc{{c}}
\def\evd{{d}}
\def\eve{{e}}
\def\evf{{f}}
\def\evg{{g}}
\def\evh{{h}}
\def\evi{{i}}
\def\evj{{j}}
\def\evk{{k}}
\def\evl{{l}}
\def\evm{{m}}
\def\evn{{n}}
\def\evo{{o}}
\def\evp{{p}}
\def\evq{{q}}
\def\evr{{r}}
\def\evs{{s}}
\def\evt{{t}}
\def\evu{{u}}
\def\evv{{v}}
\def\evw{{w}}
\def\evx{{x}}
\def\evy{{y}}
\def\evz{{z}}

% Matrix
\def\mA{{\bm{A}}}
\def\mB{{\bm{B}}}
\def\mC{{\bm{C}}}
\def\mD{{\bm{D}}}
\def\mE{{\bm{E}}}
\def\mF{{\bm{F}}}
\def\mG{{\bm{G}}}
\def\mH{{\bm{H}}}
\def\mI{{\bm{I}}}
\def\mJ{{\bm{J}}}
\def\mK{{\bm{K}}}
\def\mL{{\bm{L}}}
\def\mM{{\bm{M}}}
\def\mN{{\bm{N}}}
\def\mO{{\bm{O}}}
\def\mP{{\bm{P}}}
\def\mQ{{\bm{Q}}}
\def\mR{{\bm{R}}}
\def\mS{{\bm{S}}}
\def\mT{{\bm{T}}}
\def\mU{{\bm{U}}}
\def\mV{{\bm{V}}}
\def\mW{{\bm{W}}}
\def\mX{{\bm{X}}}
\def\mY{{\bm{Y}}}
\def\mZ{{\bm{Z}}}
\def\mBeta{{\bm{\beta}}}
\def\mPhi{{\bm{\Phi}}}
\def\mLambda{{\bm{\Lambda}}}
\def\mSigma{{\bm{\Sigma}}}

% Tensor
\DeclareMathAlphabet{\mathsfit}{\encodingdefault}{\sfdefault}{m}{sl}
\SetMathAlphabet{\mathsfit}{bold}{\encodingdefault}{\sfdefault}{bx}{n}
\newcommand{\tens}[1]{\bm{\mathsfit{#1}}}
\def\tA{{\tens{A}}}
\def\tB{{\tens{B}}}
\def\tC{{\tens{C}}}
\def\tD{{\tens{D}}}
\def\tE{{\tens{E}}}
\def\tF{{\tens{F}}}
\def\tG{{\tens{G}}}
\def\tH{{\tens{H}}}
\def\tI{{\tens{I}}}
\def\tJ{{\tens{J}}}
\def\tK{{\tens{K}}}
\def\tL{{\tens{L}}}
\def\tM{{\tens{M}}}
\def\tN{{\tens{N}}}
\def\tO{{\tens{O}}}
\def\tP{{\tens{P}}}
\def\tQ{{\tens{Q}}}
\def\tR{{\tens{R}}}
\def\tS{{\tens{S}}}
\def\tT{{\tens{T}}}
\def\tU{{\tens{U}}}
\def\tV{{\tens{V}}}
\def\tW{{\tens{W}}}
\def\tX{{\tens{X}}}
\def\tY{{\tens{Y}}}
\def\tZ{{\tens{Z}}}


% Graph
\def\gA{{\mathcal{A}}}
\def\gB{{\mathcal{B}}}
\def\gC{{\mathcal{C}}}
\def\gD{{\mathcal{D}}}
\def\gE{{\mathcal{E}}}
\def\gF{{\mathcal{F}}}
\def\gG{{\mathcal{G}}}
\def\gH{{\mathcal{H}}}
\def\gI{{\mathcal{I}}}
\def\gJ{{\mathcal{J}}}
\def\gK{{\mathcal{K}}}
\def\gL{{\mathcal{L}}}
\def\gM{{\mathcal{M}}}
\def\gN{{\mathcal{N}}}
\def\gO{{\mathcal{O}}}
\def\gP{{\mathcal{P}}}
\def\gQ{{\mathcal{Q}}}
\def\gR{{\mathcal{R}}}
\def\gS{{\mathcal{S}}}
\def\gT{{\mathcal{T}}}
\def\gU{{\mathcal{U}}}
\def\gV{{\mathcal{V}}}
\def\gW{{\mathcal{W}}}
\def\gX{{\mathcal{X}}}
\def\gY{{\mathcal{Y}}}
\def\gZ{{\mathcal{Z}}}

% Sets
\def\sA{{\mathbb{A}}}
\def\sB{{\mathbb{B}}}
\def\sC{{\mathbb{C}}}
\def\sD{{\mathbb{D}}}
% Don't use a set called E, because this would be the same as our symbol
% for expectation.
\def\sF{{\mathbb{F}}}
\def\sG{{\mathbb{G}}}
\def\sH{{\mathbb{H}}}
\def\sI{{\mathbb{I}}}
\def\sJ{{\mathbb{J}}}
\def\sK{{\mathbb{K}}}
\def\sL{{\mathbb{L}}}
\def\sM{{\mathbb{M}}}
\def\sN{{\mathbb{N}}}
\def\sO{{\mathbb{O}}}
\def\sP{{\mathbb{P}}}
\def\sQ{{\mathbb{Q}}}
\def\sR{{\mathbb{R}}}
\def\sS{{\mathbb{S}}}
\def\sT{{\mathbb{T}}}
\def\sU{{\mathbb{U}}}
\def\sV{{\mathbb{V}}}
\def\sW{{\mathbb{W}}}
\def\sX{{\mathbb{X}}}
\def\sY{{\mathbb{Y}}}
\def\sZ{{\mathbb{Z}}}

% Entries of a matrix
\def\emLambda{{\Lambda}}
\def\emA{{A}}
\def\emB{{B}}
\def\emC{{C}}
\def\emD{{D}}
\def\emE{{E}}
\def\emF{{F}}
\def\emG{{G}}
\def\emH{{H}}
\def\emI{{I}}
\def\emJ{{J}}
\def\emK{{K}}
\def\emL{{L}}
\def\emM{{M}}
\def\emN{{N}}
\def\emO{{O}}
\def\emP{{P}}
\def\emQ{{Q}}
\def\emR{{R}}
\def\emS{{S}}
\def\emT{{T}}
\def\emU{{U}}
\def\emV{{V}}
\def\emW{{W}}
\def\emX{{X}}
\def\emY{{Y}}
\def\emZ{{Z}}
\def\emSigma{{\Sigma}}

% entries of a tensor
% Same font as tensor, without \bm wrapper
\newcommand{\etens}[1]{\mathsfit{#1}}
\def\etLambda{{\etens{\Lambda}}}
\def\etA{{\etens{A}}}
\def\etB{{\etens{B}}}
\def\etC{{\etens{C}}}
\def\etD{{\etens{D}}}
\def\etE{{\etens{E}}}
\def\etF{{\etens{F}}}
\def\etG{{\etens{G}}}
\def\etH{{\etens{H}}}
\def\etI{{\etens{I}}}
\def\etJ{{\etens{J}}}
\def\etK{{\etens{K}}}
\def\etL{{\etens{L}}}
\def\etM{{\etens{M}}}
\def\etN{{\etens{N}}}
\def\etO{{\etens{O}}}
\def\etP{{\etens{P}}}
\def\etQ{{\etens{Q}}}
\def\etR{{\etens{R}}}
\def\etS{{\etens{S}}}
\def\etT{{\etens{T}}}
\def\etU{{\etens{U}}}
\def\etV{{\etens{V}}}
\def\etW{{\etens{W}}}
\def\etX{{\etens{X}}}
\def\etY{{\etens{Y}}}
\def\etZ{{\etens{Z}}}

% The true underlying data generating distribution
\newcommand{\pdata}{p_{\rm{data}}}
% The empirical distribution defined by the training set
\newcommand{\ptrain}{\hat{p}_{\rm{data}}}
\newcommand{\Ptrain}{\hat{P}_{\rm{data}}}
% The model distribution
\newcommand{\pmodel}{p_{\rm{model}}}
\newcommand{\Pmodel}{P_{\rm{model}}}
\newcommand{\ptildemodel}{\tilde{p}_{\rm{model}}}
% Stochastic autoencoder distributions
\newcommand{\pencode}{p_{\rm{encoder}}}
\newcommand{\pdecode}{p_{\rm{decoder}}}
\newcommand{\precons}{p_{\rm{reconstruct}}}

\newcommand{\laplace}{\mathrm{Laplace}} % Laplace distribution

\newcommand{\E}{\mathbb{E}}
\newcommand{\Ls}{\mathcal{L}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\emp}{\tilde{p}}
\newcommand{\lr}{\alpha}
\newcommand{\reg}{\lambda}
\newcommand{\rect}{\mathrm{rectifier}}
\newcommand{\softmax}{\mathrm{softmax}}
\newcommand{\sigmoid}{\sigma}
\newcommand{\softplus}{\zeta}
\newcommand{\KL}{D_{\mathrm{KL}}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\standarderror}{\mathrm{SE}}
\newcommand{\Cov}{\mathrm{Cov}}
% Wolfram Mathworld says $L^2$ is for function spaces and $\ell^2$ is for vectors
% But then they seem to use $L^2$ for vectors throughout the site, and so does
% wikipedia.
\newcommand{\normlzero}{L^0}
\newcommand{\normlone}{L^1}
\newcommand{\normltwo}{L^2}
\newcommand{\normlp}{L^p}
\newcommand{\normmax}{L^\infty}

\newcommand{\parents}{Pa} % See usage in notation.tex. Chosen to match Daphne's book.

\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}

\DeclareMathOperator{\sign}{sign}
\DeclareMathOperator{\Tr}{Tr}
\let\ab\allowbreak

\begin{thebibliography}{57}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
  \providecommand{\doi}[1]{doi: #1}\else
  \providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi

\bibitem[Attal et~al.(2024)Attal, Verbin, Mildenhall, Hedman, Barron, O'Toole, and Srinivasan]{attal2024flash}
Benjamin Attal, Dor Verbin, Ben Mildenhall, Peter Hedman, Jonathan~T Barron, Matthew O'Toole, and Pratul~P Srinivasan.
\newblock Flash cache: Reducing bias in radiance cache based inverse rendering.
\newblock \emph{arXiv preprint arXiv:2409.05867}, 2024.

\bibitem[Barron \& Malik(2013)Barron and Malik]{barron2013intrinsic}
Jonathan~T Barron and Jitendra Malik.
\newblock Intrinsic scene properties from a single rgb-d image.
\newblock In \emph{Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition}, pp.\  17--24, 2013.

\bibitem[Barron et~al.(2021)Barron, Mildenhall, Tancik, Hedman, Martin-Brualla, and Srinivasan]{barron2021mip}
Jonathan~T Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul~P Srinivasan.
\newblock Mip-nerf: A multiscale representation for anti-aliasing neural radiance fields.
\newblock In \emph{Proceedings of the IEEE/CVF International Conference on Computer Vision}, pp.\  5855--5864, 2021.

\bibitem[Barron et~al.(2022)Barron, Mildenhall, Verbin, Srinivasan, and Hedman]{barron2022mip}
Jonathan~T Barron, Ben Mildenhall, Dor Verbin, Pratul~P Srinivasan, and Peter Hedman.
\newblock Mip-nerf 360: Unbounded anti-aliased neural radiance fields.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  5470--5479, 2022.

\bibitem[Barron et~al.(2023)Barron, Mildenhall, Verbin, Srinivasan, and Hedman]{barron2023zip}
Jonathan~T Barron, Ben Mildenhall, Dor Verbin, Pratul~P Srinivasan, and Peter Hedman.
\newblock Zip-nerf: Anti-aliased grid-based neural radiance fields.
\newblock In \emph{Proceedings of the IEEE/CVF International Conference on Computer Vision}, pp.\  19697--19705, 2023.

\bibitem[Beer(1852)]{beer1852bestimmung}
Beer.
\newblock Bestimmung der absorption des rothen lichts in farbigen fl{\"u}ssigkeiten.
\newblock \emph{Annalen der Physik}, 162\penalty0 (5):\penalty0 78--88, 1852.

\bibitem[Bi et~al.(2020)Bi, Xu, Srinivasan, Mildenhall, Sunkavalli, Ha{\v{s}}an, Hold-Geoffroy, Kriegman, and Ramamoorthi]{bi2020neural}
Sai Bi, Zexiang Xu, Pratul Srinivasan, Ben Mildenhall, Kalyan Sunkavalli, Milo{\v{s}} Ha{\v{s}}an, Yannick Hold-Geoffroy, David Kriegman, and Ravi Ramamoorthi.
\newblock Neural reflectance fields for appearance acquisition.
\newblock \emph{arXiv preprint arXiv:2008.03824}, 2020.

\bibitem[Boss et~al.(2021)Boss, Braun, Jampani, Barron, Liu, and Lensch]{boss2021nerd}
Mark Boss, Raphael Braun, Varun Jampani, Jonathan~T Barron, Ce~Liu, and Hendrik Lensch.
\newblock Nerd: Neural reflectance decomposition from image collections.
\newblock In \emph{Proceedings of the IEEE/CVF International Conference on Computer Vision}, pp.\  12684--12694, 2021.

\bibitem[Bouguer(1729)]{bouguer1729essai}
Pierre Bouguer.
\newblock \emph{Essai d'optique sur la gradation de la lumi{\`e}re}.
\newblock Claude Jombert, 1729.

\bibitem[Bul{\`o} et~al.(2024)Bul{\`o}, Porzi, and Kontschieder]{bulo2024revising}
Samuel~Rota Bul{\`o}, Lorenzo Porzi, and Peter Kontschieder.
\newblock Revising densification in gaussian splatting.
\newblock \emph{arXiv preprint arXiv:2404.06109}, 2024.

\bibitem[Chen et~al.(2022)Chen, Xu, Geiger, Yu, and Su]{chen2022tensorf}
Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su.
\newblock Tensorf: Tensorial radiance fields.
\newblock In \emph{European Conference on Computer Vision}, pp.\  333--350. Springer, 2022.

\bibitem[Chen et~al.(2019)Chen, Ling, Gao, Smith, Lehtinen, Jacobson, and Fidler]{chen2019learning}
Wenzheng Chen, Huan Ling, Jun Gao, Edward Smith, Jaakko Lehtinen, Alec Jacobson, and Sanja Fidler.
\newblock Learning to predict 3d objects with an interpolation-based differentiable renderer.
\newblock \emph{Advances in neural information processing systems}, 32, 2019.

\bibitem[Debevec(2008)]{debevec2008rendering}
Paul Debevec.
\newblock Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography.
\newblock In \emph{Acm siggraph 2008 classes}, pp.\  1--10. 2008.

\bibitem[Dong et~al.(2014)Dong, Chen, Peers, Zhang, and Tong]{dong2014appearance}
Yue Dong, Guojun Chen, Pieter Peers, Jiawan Zhang, and Xin Tong.
\newblock Appearance-from-motion: Recovering spatially varying surface reflectance under unknown lighting.
\newblock \emph{ACM Transactions on Graphics (TOG)}, 33\penalty0 (6):\penalty0 1--12, 2014.

\bibitem[Fan et~al.(2023)Fan, Wang, Wen, Zhu, Xu, and Wang]{fan2023lightgaussian}
Zhiwen Fan, Kevin Wang, Kairun Wen, Zehao Zhu, Dejia Xu, and Zhangyang Wang.
\newblock Lightgaussian: Unbounded 3d gaussian compression with 15x reduction and 200+ fps.
\newblock \emph{arXiv preprint arXiv:2311.17245}, 2023.

\bibitem[Fridovich-Keil et~al.(2022)Fridovich-Keil, Yu, Tancik, Chen, Recht, and Kanazawa]{fridovich2022plenoxels}
Sara Fridovich-Keil, Alex Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa.
\newblock Plenoxels: Radiance fields without neural networks.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  5501--5510, 2022.

\bibitem[Gao et~al.(2023)Gao, Gu, Lin, Zhu, Cao, Zhang, and Yao]{gao2023relightable}
Jian Gao, Chun Gu, Youtian Lin, Hao Zhu, Xun Cao, Li~Zhang, and Yao Yao.
\newblock Relightable 3d gaussian: Real-time point cloud relighting with brdf decomposition and ray tracing.
\newblock \emph{arXiv preprint arXiv:2311.16043}, 2023.

\bibitem[Garbin et~al.(2021)Garbin, Kowalski, Johnson, Shotton, and Valentin]{garbin2021fastnerf}
Stephan~J Garbin, Marek Kowalski, Matthew Johnson, Jamie Shotton, and Julien Valentin.
\newblock Fastnerf: High-fidelity neural rendering at 200fps.
\newblock In \emph{Proceedings of the IEEE/CVF International Conference on Computer Vision}, pp.\  14346--14355, 2021.

\bibitem[Guo et~al.(2024)Guo, Ma, Fan, Liu, and Li]{guo2024semantic}
Jun Guo, Xiaojian Ma, Yue Fan, Huaping Liu, and Qing Li.
\newblock Semantic gaussians: Open-vocabulary scene understanding with 3d gaussian splatting.
\newblock \emph{arXiv preprint arXiv:2403.15624}, 2024.

\bibitem[Hedman et~al.(2021)Hedman, Srinivasan, Mildenhall, Barron, and Debevec]{hedman2021baking}
Peter Hedman, Pratul~P Srinivasan, Ben Mildenhall, Jonathan~T Barron, and Paul Debevec.
\newblock Baking neural radiance fields for real-time view synthesis.
\newblock In \emph{Proceedings of the IEEE/CVF International Conference on Computer Vision}, pp.\  5875--5884, 2021.

\bibitem[H{\"o}llein et~al.(2024)H{\"o}llein, Bo{\v{z}}i{\v{c}}, Zollh{\"o}fer, and Nie{\ss}ner]{hollein20243dgs}
Lukas H{\"o}llein, Alja{\v{z}} Bo{\v{z}}i{\v{c}}, Michael Zollh{\"o}fer, and Matthias Nie{\ss}ner.
\newblock 3dgs-lm: Faster gaussian-splatting optimization with levenberg-marquardt.
\newblock \emph{arXiv preprint arXiv:2409.12892}, 2024.

\bibitem[Huang et~al.(2024)Huang, Yu, Chen, Geiger, and Gao]{huang20242d}
Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao.
\newblock 2d gaussian splatting for geometrically accurate radiance fields.
\newblock In \emph{ACM SIGGRAPH 2024 Conference Papers}, pp.\  1--11, 2024.

\bibitem[Jiang et~al.(2024)Jiang, Tu, Liu, Gao, Long, Wang, and Ma]{jiang2024gaussianshader}
Yingwenqi Jiang, Jiadong Tu, Yuan Liu, Xifeng Gao, Xiaoxiao Long, Wenping Wang, and Yuexin Ma.
\newblock Gaussianshader: 3d gaussian splatting with shading functions for reflective surfaces.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  5322--5332, 2024.

\bibitem[Jin et~al.(2023)Jin, Liu, Xu, Zhang, Han, Bi, Zhou, Xu, and Su]{jin2023tensoir}
Haian Jin, Isabella Liu, Peijia Xu, Xiaoshuai Zhang, Songfang Han, Sai Bi, Xiaowei Zhou, Zexiang Xu, and Hao Su.
\newblock Tensoir: Tensorial inverse rendering.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  165--174, 2023.

\bibitem[Kajiya(1986)]{kajiya1986rendering}
James~T Kajiya.
\newblock The rendering equation.
\newblock In \emph{Proceedings of the 13th annual conference on Computer graphics and interactive techniques}, pp.\  143--150, 1986.

\bibitem[Kanamori \& Endo(2019)Kanamori and Endo]{kanamori2019relighting}
Yoshihiro Kanamori and Yuki Endo.
\newblock Relighting humans: occlusion-aware inverse rendering for full-body human images.
\newblock \emph{arXiv preprint arXiv:1908.02714}, 2019.

\bibitem[Kerbl et~al.(2023)Kerbl, Kopanas, Leimk{\"u}hler, and Drettakis]{kerbl20233d}
Bernhard Kerbl, Georgios Kopanas, Thomas Leimk{\"u}hler, and George Drettakis.
\newblock 3d gaussian splatting for real-time radiance field rendering.
\newblock \emph{ACM Transactions on Graphics (ToG)}, 42\penalty0 (4):\penalty0 1--14, 2023.

\bibitem[Kheradmand et~al.(2024)Kheradmand, Rebain, Sharma, Sun, Tseng, Isack, Kar, Tagliasacchi, and Yi]{kheradmand20243d}
Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Jeff Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang~Moo Yi.
\newblock 3d gaussian splatting as markov chain monte carlo.
\newblock \emph{arXiv preprint arXiv:2404.09591}, 2024.

\bibitem[Lambert(1760)]{lambert1760photometria}
Jean-Henri Lambert.
\newblock \emph{Photometria sive de mensura et gradibus luminis, colorum et umbrae}.
\newblock Sumptibus viduae Eberhardi Klett, typis Christophori Petri Detleffsen, 1760.

\bibitem[Li et~al.(2023)Li, Wang, Cheng, Pan, and Yang]{li2023multi}
Zhen Li, Lingli Wang, Mofang Cheng, Cihui Pan, and Jiaqi Yang.
\newblock Multi-view inverse rendering for large-scale real-world indoor scenes.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  12499--12509, 2023.

\bibitem[Liang et~al.(2024)Liang, Zhang, Feng, Shan, and Jia]{liang2024gs}
Zhihao Liang, Qi~Zhang, Ying Feng, Ying Shan, and Kui Jia.
\newblock Gs-ir: 3d gaussian splatting for inverse rendering.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  21644--21653, 2024.

\bibitem[Lin et~al.(2024)Lin, Dai, Zhu, and Yao]{lin2024gaussian}
Youtian Lin, Zuozhuo Dai, Siyu Zhu, and Yao Yao.
\newblock Gaussian-flow: 4d reconstruction with dynamic 3d gaussian particle.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  21136--21145, 2024.

\bibitem[Liu et~al.(2023)Liu, Wang, Lin, Long, Wang, Liu, Komura, and Wang]{liu2023nero}
Yuan Liu, Peng Wang, Cheng Lin, Xiaoxiao Long, Jiepeng Wang, Lingjie Liu, Taku Komura, and Wenping Wang.
\newblock Nero: Neural geometry and brdf reconstruction of reflective objects from multiview images.
\newblock \emph{ACM Transactions on Graphics (TOG)}, 42\penalty0 (4):\penalty0 1--22, 2023.

\bibitem[Lu et~al.(2024)Lu, Zhang, Wang, Liu, Lu, and Tang]{lu2024manigaussian}
Guanxing Lu, Shiyi Zhang, Ziwei Wang, Changliu Liu, Jiwen Lu, and Yansong Tang.
\newblock Manigaussian: Dynamic gaussian splatting for multi-task robotic manipulation.
\newblock \emph{arXiv preprint arXiv:2403.08321}, 2024.

\bibitem[Luiten et~al.(2024)Luiten, Kopanas, Leibe, and Ramanan]{luiten2024dynamic}
Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan.
\newblock Dynamic 3d gaussians: Tracking by persistent dynamic view synthesis.
\newblock In \emph{2024 International Conference on 3D Vision (3DV)}, pp.\  800--809. IEEE, 2024.

\bibitem[Mallick et~al.(2024)Mallick, Goel, Kerbl, Carrasco, Steinberger, and De~La~Torre]{mallick2024taming}
Saswat~Subhajyoti Mallick, Rahul Goel, Bernhard Kerbl, Francisco~Vicente Carrasco, Markus Steinberger, and Fernando De~La~Torre.
\newblock Taming 3dgs: High-quality radiance fields with limited resources.
\newblock \emph{arXiv preprint arXiv:2406.15643}, 2024.

\bibitem[Mayerh{\"o}fer et~al.(2020)Mayerh{\"o}fer, Pahlow, and Popp]{mayerhofer2020bouguer}
Thomas~G Mayerh{\"o}fer, Susanne Pahlow, and J{\"u}rgen Popp.
\newblock The bouguer-beer-lambert law: Shining light on the obscure.
\newblock \emph{ChemPhysChem}, 21\penalty0 (18):\penalty0 2029--2046, 2020.

\bibitem[Mildenhall et~al.(2021)Mildenhall, Srinivasan, Tancik, Barron, Ramamoorthi, and Ng]{mildenhall2021nerf}
Ben Mildenhall, Pratul~P Srinivasan, Matthew Tancik, Jonathan~T Barron, Ravi Ramamoorthi, and Ren Ng.
\newblock Nerf: Representing scenes as neural radiance fields for view synthesis.
\newblock \emph{Communications of the ACM}, 65\penalty0 (1):\penalty0 99--106, 2021.

\bibitem[Miller et~al.(2024)Miller, Chen, Lai, and Gkioulekas]{miller2024objects}
Bailey Miller, Hanyu Chen, Alice Lai, and Ioannis Gkioulekas.
\newblock Objects as volumes: A stochastic geometry view of opaque solids.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  87--97, 2024.

\bibitem[M{\"u}ller et~al.(2022)M{\"u}ller, Evans, Schied, and Keller]{muller2022instant}
Thomas M{\"u}ller, Alex Evans, Christoph Schied, and Alexander Keller.
\newblock Instant neural graphics primitives with a multiresolution hash encoding.
\newblock \emph{ACM transactions on graphics (TOG)}, 41\penalty0 (4):\penalty0 1--15, 2022.

\bibitem[Munkberg et~al.(2022)Munkberg, Hasselgren, Shen, Gao, Chen, Evans, M{\"u}ller, and Fidler]{munkberg2022extracting}
Jacob Munkberg, Jon Hasselgren, Tianchang Shen, Jun Gao, Wenzheng Chen, Alex Evans, Thomas M{\"u}ller, and Sanja Fidler.
\newblock Extracting triangular 3d models, materials, and lighting from images.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  8280--8290, 2022.

\bibitem[Navaneet et~al.(2023)Navaneet, Meibodi, Koohpayegani, and Pirsiavash]{navaneet2023compact3d}
KL~Navaneet, Kossar~Pourahmadi Meibodi, Soroush~Abbasi Koohpayegani, and Hamed Pirsiavash.
\newblock Compact3d: Compressing gaussian splat radiance field models with vector quantization.
\newblock \emph{arXiv preprint arXiv:2311.18159}, 2023.

\bibitem[Niedermayr et~al.(2024)Niedermayr, Stumpfegger, and Westermann]{niedermayr2024compressed}
Simon Niedermayr, Josef Stumpfegger, and R{\"u}diger Westermann.
\newblock Compressed 3d gaussian splatting for accelerated novel view synthesis.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  10349--10358, 2024.

\bibitem[Qin et~al.(2024)Qin, Li, Zhou, Wang, and Pfister]{qin2024langsplat}
Minghan Qin, Wanhua Li, Jiawei Zhou, Haoqian Wang, and Hanspeter Pfister.
\newblock Langsplat: 3d language gaussian splatting.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  20051--20060, 2024.

\bibitem[Shorinwa et~al.(2024)Shorinwa, Tucker, Smith, Swann, Chen, Firoozi, Kennedy, and Schwager]{shorinwa2024splat}
Olaolu Shorinwa, Johnathan Tucker, Aliyah Smith, Aiden Swann, Timothy Chen, Roya Firoozi, Monroe~David Kennedy, and Mac Schwager.
\newblock Splat-mover: multi-stage, open-vocabulary robotic manipulation via editable gaussian splatting.
\newblock In \emph{8th Annual Conference on Robot Learning}, 2024.

\bibitem[Sitzmann et~al.(2021)Sitzmann, Rezchikov, Freeman, Tenenbaum, and Durand]{sitzmann2021light}
Vincent Sitzmann, Semon Rezchikov, Bill Freeman, Josh Tenenbaum, and Fredo Durand.
\newblock Light field networks: Neural scene representations with single-evaluation rendering.
\newblock \emph{Advances in Neural Information Processing Systems}, 34:\penalty0 19313--19325, 2021.

\bibitem[Srinivasan et~al.(2021)Srinivasan, Deng, Zhang, Tancik, Mildenhall, and Barron]{srinivasan2021nerv}
Pratul~P Srinivasan, Boyang Deng, Xiuming Zhang, Matthew Tancik, Ben Mildenhall, and Jonathan~T Barron.
\newblock Nerv: Neural reflectance and visibility fields for relighting and view synthesis.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  7495--7504, 2021.

\bibitem[Verbin et~al.(2022)Verbin, Hedman, Mildenhall, Zickler, Barron, and Srinivasan]{verbin2022ref}
Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan~T Barron, and Pratul~P Srinivasan.
\newblock Ref-nerf: Structured view-dependent appearance for neural radiance fields.
\newblock In \emph{2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, pp.\  5481--5490. IEEE, 2022.

\bibitem[Verbin et~al.(2024)Verbin, Mildenhall, Hedman, Barron, Zickler, and Srinivasan]{verbin2024eclipse}
Dor Verbin, Ben Mildenhall, Peter Hedman, Jonathan~T Barron, Todd Zickler, and Pratul~P Srinivasan.
\newblock Eclipse: Disambiguating illumination and materials using unintended shadows.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  77--86, 2024.

\bibitem[Yang et~al.(2022)Yang, Chen, Chen, Chen, and Wong]{yang2022ps}
Wenqi Yang, Guanying Chen, Chaofeng Chen, Zhenfang Chen, and Kwan-Yee~K Wong.
\newblock Ps-nerf: Neural inverse rendering for multi-view photometric stereo.
\newblock In \emph{European Conference on Computer Vision}, pp.\  266--284. Springer, 2022.

\bibitem[Yao et~al.(2022)Yao, Zhang, Liu, Qu, Fang, McKinnon, Tsin, and Quan]{yao2022neilf}
Yao Yao, Jingyang Zhang, Jingbo Liu, Yihang Qu, Tian Fang, David McKinnon, Yanghai Tsin, and Long Quan.
\newblock Neilf: Neural incident light field for physically-based material estimation.
\newblock In \emph{European Conference on Computer Vision}, pp.\  700--716. Springer, 2022.

\bibitem[Yariv et~al.(2021)Yariv, Gu, Kasten, and Lipman]{yariv2021volume}
Lior Yariv, Jiatao Gu, Yoni Kasten, and Yaron Lipman.
\newblock Volume rendering of neural implicit surfaces.
\newblock \emph{Advances in Neural Information Processing Systems}, 34:\penalty0 4805--4815, 2021.

\bibitem[Yong et~al.(2024)Yong, Xie, Stepputtis, and Sycara]{yong2024gl}
Silong Yong, Yaqi Xie, Simon Stepputtis, and Katia Sycara.
\newblock Gl-nerf: Gauss-laguerre quadrature enables training-free nerf acceleration.
\newblock \emph{arXiv preprint arXiv:2410.19831}, 2024.

\bibitem[Yu et~al.(2021)Yu, Li, Tancik, Li, Ng, and Kanazawa]{yu2021plenoctrees}
Alex Yu, Ruilong Li, Matthew Tancik, Hao Li, Ren Ng, and Angjoo Kanazawa.
\newblock Plenoctrees for real-time rendering of neural radiance fields.
\newblock In \emph{Proceedings of the IEEE/CVF International Conference on Computer Vision}, pp.\  5752--5761, 2021.

\bibitem[Zhang et~al.(2021{\natexlab{a}})Zhang, Luan, Wang, Bala, and Snavely]{zhang2021physg}
Kai Zhang, Fujun Luan, Qianqian Wang, Kavita Bala, and Noah Snavely.
\newblock Physg: Inverse rendering with spherical gaussians for physics-based material editing and relighting.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  5453--5462, 2021{\natexlab{a}}.

\bibitem[Zhang et~al.(2021{\natexlab{b}})Zhang, Srinivasan, Deng, Debevec, Freeman, and Barron]{zhang2021nerfactor}
Xiuming Zhang, Pratul~P Srinivasan, Boyang Deng, Paul Debevec, William~T Freeman, and Jonathan~T Barron.
\newblock Nerfactor: Neural factorization of shape and reflectance under an unknown illumination.
\newblock \emph{ACM Transactions on Graphics (ToG)}, 40\penalty0 (6):\penalty0 1--18, 2021{\natexlab{b}}.

\bibitem[Zhang et~al.(2022)Zhang, Sun, He, Fu, Jia, and Zhou]{zhang2022modeling}
Yuanqing Zhang, Jiaming Sun, Xingyi He, Huan Fu, Rongfei Jia, and Xiaowei Zhou.
\newblock Modeling indirect illumination for inverse rendering.
\newblock In \emph{Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition}, pp.\  18643--18652, 2022.

\end{thebibliography}