Continuous Methods : Hamiltonian Domain Translation
Extracted source text · arXiv 2207.03843
%%%%%%%% ICML 2022 EXAMPLE LATEX SUBMISSION FILE %%%%%%%%%%%%%%%%%
\documentclass[nohyperref]{article}
% Recommended, but optional, packages for figures and better typesetting:
\usepackage{microtype}
\usepackage{graphicx}
\usepackage{subfigure}
\usepackage{booktabs} % for professional tables
% hyperref makes hyperlinks in the resulting PDF.
% If your build breaks (sometimes temporarily if a hyperlink spans a page)
% please comment out the following usepackage line and replace
% \usepackage{icml2022} with \usepackage[nohyperref]{icml2022} above.
\usepackage{hyperref}
% Attempt to make hyperref and algorithmic work together better:
\newcommand{\theHalgorithm}{\arabic{algorithm}}
%\newcommand{\LMcomment}[1]{{\color{orange} \textbf{#1}}}
% Use the following line for the initial blind version submitted for review:
\usepackage[accepted]{icml2022}
% If accepted, instead use the following line for the camera-ready submission:
% \usepackage[accepted]{icml2022}
% For theorems and such
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{mathtools}
\usepackage{amsthm}
\usepackage[symbol,bottom]{footmisc}
% if you use cleveref..
\usepackage[capitalize,noabbrev]{cleveref}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% THEOREMS
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\theoremstyle{plain}
\newtheorem{theorem}{Theorem}[section]
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\theoremstyle{definition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{assumption}[theorem]{Assumption}
\theoremstyle{remark}
\newtheorem{remark}[theorem]{Remark}
% Todonotes is useful during development; simply uncomment the next line
% and comment out the line below the next line to turn off comments
%\usepackage[disable,textsize=tiny]{todonotes}
\usepackage[textsize=tiny]{todonotes}
\renewcommand{\thefootnote}{\fnsymbol{footnote}}
% The \icmltitle you define below is probably too long as a header.
% Therefore, a short form for the running title is supplied here:
\icmltitlerunning{Hamiltonian Domain Translation}
\begin{document}
\onecolumn
\icmltitle{Continuous Methods : Hamiltonian Domain Translation}
% It is OKAY to include author information, even for blind
% submissions: the style file will automatically remove it for you
% unless you've provided the [accepted] option to the icml2022
% package.
% List of affiliations: The first argument should be a (short)
% identifier you will use later to specify author affiliations
% Academic affiliations should list Department, University, City, Region, Country
% Industry affiliations should list Company, City, Region, Country
% You can specify symbols, otherwise they are numbered in order.
% Ideally, you should not use this facility. Affiliations will be numbered
% in order of appearance and this is the preferred way.
\icmlsetsymbol{equal}{*}
\begin{icmlauthorlist}
\icmlauthor{Emmanuel Menier}{1,2}
\icmlauthor{Michele Alessandro Bucci}{1}
\icmlauthor{Mouadh Yagoubi}{2}
\icmlauthor{Lionel Mathelin}{3}
\icmlauthor{Marc Schoenauer}{1}
\end{icmlauthorlist}
\icmlaffiliation{1}{TAU, Inria / LISN, Université Paris-Saclay / CNRS, Orsay, France}
\icmlaffiliation{2}{IRT - SystemX, Palaiseau, France}
\icmlaffiliation{3}{LISN, Université Paris-Saclay \& CNRS, Orsay, France}
\icmlcorrespondingauthor{Emmanuel Menier}{emmanuel.menier@inria.fr}
% You may provide any keywords that you
% find helpful for describing your paper; these are used to populate
% the "keywords" metadata in the PDF but will not be shown in the document
\icmlkeywords{Deep Learning, Neural ODE, Generative models, invertibility, ICML}
\vskip 0.3in
% this must go after the closing bracket ] following \twocolumn[ ...
% This command actually creates the footnote in the first column
% listing the affiliations and the copyright notice.
% The command takes one argument, which is text to display at the start of the footnote.
% The \icmlEqualContribution command is standard text for equal contribution.
% Remove it (just {}) if you do not need this facility.
%\printAffiliationsAndNotice{} % leave blank if no need to mention equal contribution
\printAffiliationsAndNotice{} % otherwise use the standard text.
\begin{abstract}
This paper proposes a novel approach to domain translation. Leveraging established parallels between generative models and dynamical systems, we propose a reformulation of the Cycle-GAN architecture. By embedding our model with a Hamiltonian structure, we obtain a continuous, expressive and most importantly invertible generative model for domain translation.
\end{abstract}
\section{Introduction}
\label{introduction}
Domain translation is the process of transforming elements from one domain to another. One can think of applications such as neural style transfer \cite{NST} which is for example used to apply a certain painter's style to photo-realistic images. A common problem encountered in domain translation applications is that, in many cases, paired data is not available during training, which means that the problem has to be formulated in an unsupervised setting. Unsupervised learning is very common in the field of generative modeling, and several architectures have been proposed to deal with the problem of Unsupervised Domain Translation. In this work, we focus on the Cycle-GAN \cite{CycleGan} architecture, which has proved successful in various applications of Unsupervised Domain Translation \footnote{Cycle GAN project page, \url{https://junyanz.github.io/CycleGAN/}}.
Despite its success, the formulation of the Cycle-GAN method has been questioned and shown to be ill-posed. Using results from \citet{Gaussianization}, it can be shown that when considering two distinct domains, there exist an infinity of pairings between the two domains which satisfy the Cycle-GAN objective. This is an issue as the model could get stuck trying to learn wildly inefficient mappings, leading to unsatisfactory optima. This conditioning problem has been explored in depth by \citet{UDT}, as they proposed to use a regularized residual network to learn the mapping between two given domains. Borrowing ideas from optimal transport and dynamical systems, they showed that pushing the training towards simple, low-energy, transformations in latent space leads to learning a sensible and trivially invertible mapping between the two domains of interest.
The study of the links between dynamical systems theory and deep learning is still to this day a major topic of interest. One can for example cite the identification of residual networks as first order approximations of a time-continuous process which has led to the development of ground-breaking approaches such as neural ordinary differential equations (Neural ODE \cite{NeuralODE}) or invertible neural networks \cite{InvertResNet}.
Building on this existing connection, as well as the work of de~Bezenac \textit{et al.}, we propose a formulation of unsupervised domain translation as a continuous time process with conservation guarantees which ensure invertibility by construction. The proposed architecture learns the dynamics of the transformation as a Hamiltonian dynamical system. Hamiltonian systems are typically used in General Mechanics to describe the evolution of conservative systems. They preserve a quantity, called the Hamiltonian, along their trajectory. Using neural networks to learn Hamiltonian dynamics is an earlier idea that was proposed in \citet{HNN}. However this work proposes to use them to ensure invertibility of the generative process which is a desirable property to ensure the domain translation problem is well-posed. Learning conservative transformations is in fact critical to other generative modeling approaches, such as normalizing flows \cite{NF}.
\section{Method}\label{sec:method}
\subsection{Invertibility and CycleGAN}
Formally, we can look at the two domains as two separate sets $\mathcal{A},\mathcal{B} \subset \mathbb{R}^d$, where $d$ is the dimension of the space, \textit{i.e.} the pixel space for images, or any latent representation space. The goal of unsupervised domain translation is to learn the forward mapping $F:\mathcal{A}\to\mathcal{B}$ as well as the reverse map $R:\mathcal{B}\to\mathcal{A}$ so that the pair $(F,R)$ generates semantically meaningful samples of each domain. That is to say, the generated samples should be indistinguishable from samples in the target domain, while remaining coherent with their corresponding sample in the original domain.
CycleGan proposes to enforce these constraints by using a combined loss: $\mathcal{L} = \mathcal{L}_{adv} + \mathcal{L}_{cyc}$. The first term $ \mathcal{L}_{adv} = \mathcal{D}(F(\mathcal{A}),\mathcal{B})$ corresponds to an adversarial loss which measures the distance between the generated samples and the target domain. This term ensures that generated samples are indistinguishable from the target domain. In CycleGAN, $\mathcal{D}$ is implemented using Generative Adversarial Networks \cite{GAN}.
The second term in the loss is called the cyclic loss, $\mathcal{L}_{cyc} = \Vert F \circ R (x_\mathcal{A}) - x_\mathcal{A} \Vert + \Vert R \circ F (x_\mathcal{B}) - x_\mathcal{B} \Vert$. This term promotes transformations $F$ that are invertible and such that $R = F^{-1}$. Intuitively, this pushes the CycleGAN architecture towards learning minimal transformations of the samples, so as to retain a maximum of information from the initial sample and simplify the reconstruction $ R \circ F$. This second term is used to ensure coherence between the translated and initial samples. In addition, learning an invertible (thus bijective) map between the two domains is critical at the conceptual level. Indeed, one sample from a given domain should not map to multiple samples in the target domain as only one sample in the target domain should optimally satisfy the trade-off between coherence with the original sample and similarity with the target domain.
\subsection{Continuous models and Hamiltonian Neural Networks}
The previous paragraph outlined the importance of ensuring the translation map is invertible to relax the learning problem. In fact, this is not specific to the domain translation problem, as invertibility of learned maps has been linked to classical deep learning problems such as vanishing/exploding gradients in recurrent neural networks \cite{RNNConditioning}, or the training of other generative models like normalizing flows \cite{NF}. Several approaches have been proposed to push learned models towards invertibility \cite{SpectralNorm,constrainedNF}, however, they often impose significant constraints on the structure and expressivity of the models, leading to important training costs.
In this work, we propose to use a natural formulation for invertible transformations. Exploiting the parallel between the residual networks used in numerous image processing approaches, and ordinary differential equations, we propose to define domain translation as a continuous system. Starting at $t=0$ with samples from one domain $x_{t=0} \in \mathcal{A}$, we learn a transport flow $f_\theta$ so that, at $t=T$, $x_{t=T} \in \mathcal{B}$:
\begin{equation}
\frac{d x}{dt} = f_\theta (x), \quad \mathrm{s.t.} \quad x_0 \in \mathcal{A},
\: x_T \in \mathcal{B}
\end{equation}
This formulation is not enough to ensure invertibility of the transformation, as the flow $f_\theta$ could be dissipative, or even unstable. To enforce invertibility, we express the flow $f_\theta$ as a conservative operator using Hamiltonian neural networks inspired from \citet{HNN}. To do so, the samples are divided into two vectors of equal length $x = [p,q]$, (we assume $d$ to be even as a modeling choice). In general mechanics, $p$ and $q$ would respectively describe the position and momentum of the studied entities. In our setting, their significance is more abstract and is defined by another function, called the Hamiltonian $\mathcal{H}_\theta(p,q) : \mathbb{R}^{d/2} \times \mathbb{R}^{d/2} \to \mathbb{R}$, which we parameterize using a neural network, hence:
\begin{equation}
f_\theta(x) =
\left(
\begin{array}{rcr}
\frac{d p}{dt} & = & -\frac{\partial \mathcal{H}_\theta}{\partial q} \\
\frac{d q}{dt} & = & \frac{\partial \mathcal{H}_\theta}{\partial p}
\end{array}
\right)
\end{equation}
Using Neural ODEs and automatic differentiation, the function $\mathcal{H}_\theta$ can be trained to satisfy the transport objective, \textit{i.e.} $x_T \in \mathcal{B}$ given $x_0 \in \mathcal{A}$. Moreover, this formulation is invertible by design as it preserves the quantity $\mathcal{H}_\theta (x)$ along its trajectory. We show below that learning the transformation $f_\theta$ with this formulation allows for the generation of semantically correct samples, without using the cyclic loss required in CycleGAN. Thanks to the conservation properties of the flow $f_\theta$, the inverse map is trivially obtained by integrating the flow backward in time:
\begin{equation}
x_\mathcal{B} = x_\mathcal{A} + \int_0^T f_\theta(x) dt \quad \Longleftrightarrow \quad x_\mathcal{A} = x_\mathcal{B} + \int_T^0 f_\theta(x) dt\label{eq:reverse_trans}
\end{equation}
\section{Results}
\subsection{Generative results}
We apply our Hamiltonian domain translation approach to image generation tasks. As proposed in \citet{UDT}, we train an encoder $E$ and a decoder $G$ to map images from both domains to a latent space of size $d=128$. This is a common approach in many image processing approaches, as the intrinsic dimension of a given image processing problem is often much lower than the dimension of the pixel space. Thus, encoding images to a low-dimensional latent space reduces the domain translation problem complexity, as well as training costs.
Once the pair ($E,G$) is trained, it can be used to generate low-dimensional encoded vectors of images of the dataset at hand. We then use our approach to learn the transport flow $f_\theta$. The Hamiltonian $\mathcal{H}$ and discriminator $\mathcal{D}$ are implemented as multi layer perceptron with 3 hidden layers. The continuous flow is learned using the \textit{optimise-then-discretise} version of NeuralODEs. We apply the architecture to the task of translating male samples of the celebA \cite{celebA} dataset to females. Figure \ref{fig:MTF_samples} presents samples generated with this approach.
\begin{figure}[ht]
\centering
\includegraphics[width=\textwidth]{Compressed_Cropped_MTF_samples.png}
\caption{Selected samples of the male to female transport process using the proposed continuous domain translation approach. The transported encodings are decoded at regular time intervals, to illustrate the transformation applied by the model.}
\label{fig:MTF_samples}
\end{figure}
\begin{figure}[ht]
\centering
\includegraphics[width=\textwidth]{Compressed_Cropped_FTM_samples.png}
\caption{Selected samples of the reverse male to female generation process. NB : These samples are generated using equation~\eqref{eq:reverse_trans} as transport flow was solely trained to map male to females.}
\label{fig:FTM_samples}
\end{figure}
As shown on Figure \ref{fig:MTF_samples}, decoding the transported samples along the transformation trajectory shows that the flow $f_\theta$ progressively transforms the male samples to females. As expected, the conservative nature of the model promotes transformations that retain non gender-specific features, as we observe that attributes such as pose, skin tone, face shape and background are preserved during the transformation. Figure \ref{fig:FTM_samples} demonstrates an additional interest of the Hamiltonian architecture as we are able to generate males from female samples by simply integrating the flow backward (see Eq.~\ref{eq:reverse_trans}). One should note that these results were obtained without ever training the model to map females to males as we do not compute the cyclic loss used in CycleGAN. These results are similar to the results of \citet{UDT} while no penalization of the magnitude of the flow applied by the model is used but invertibility is enforced instead.
\begin{figure}[ht]
\centering
\includegraphics[width=\textwidth]{excessive_integration.png}
\caption{Results of excessive integration of the transport flow. A model trained to map males to females in 1 $t.u.$ is integrated backward for more than twice the map horizon. We observe that the generated samples retain semantic sense for up to about $1.5$ times the training horizon.}
\label{fig:excessive_integration}
\end{figure}
\subsection{Excessive Integration}
An interesting feature of using a continuous flow to carry out domain translation is that one can gain some insight in the way the model transforms samples. If a flow $f_\theta$ has been trained to map two domains in one time unit (\textit{t.u.}), $T=1$, it can be integrated for a longer period, pushing the transformation further. This is one of the major differences between learning continuous transformations and discrete residual blocks. While residual blocks approximate the flow in specific regions of the latent space, the continuous flow is defined over the whole space. Any trained model starts losing performance once it drifts too far from its training conditions but we observed interesting results when integrating our model for several \textit{t.u.}. Figure \ref{fig:excessive_integration} shows that transported samples retain semantic meaning for up to about one and a half \textit{t.u.}, as the model progressively adds more and more gender-related features such as beards, wider jaws, shorter hair, etc. This generalisation performance can be linked to the conservative architecture of the model which prevents it from diverging to unknown conditions. It also adds to the interest of the approach as it supports the idea that the model is consistent with the structure of the latent space.
\subsection{Training}
It should be noted that, once the autoencoder is trained, learning the flow $f_\theta$ is very inexpensive. The model starts generating semantically coherent samples after a single epoch, and does not require fine tuning between the training of the discriminator $\mathcal{D}$ and the flow $f_\theta$. More formal benchmarking against comparable domain translation methods are planned for the future.
\section{Conclusion}
This work proposes a novel and improved formulation for domain translation. By using a time-continuous approach, we are able to leverage results from general mechanics to obtain a model that is invertible by construction. We show that this model can quickly learn to map two domains of interest, even in a latent space learned prior to training the domain translation architecture.
With the recent success of diffusion models \cite{GLIDE,LDM}, which are based on successive transformations in a pre-defined space, the analogy between generative models and dynamical systems becomes more and more relevant. In this context, the comparison of our proposed method with existing approaches such as normalizing flows and latent diffusion models constitutes the next step to this work, for instance exploring the potential extension of our continuous generative approach to stochastic differential equations \cite{NeuralSDE}.
% In the unusual situation where you want a paper to appear in the
% references without citing it in the main text, use \nocite
\bibliography{example_paper}
\bibliographystyle{icml2022}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% APPENDIX
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\end{document}
% This document was modified from the file originally made available by
% Pat Langley and Andrea Danyluk for ICML-2K. This version was created
% by Iain Murray in 2018, and modified by Alexandre Bouchard in
% 2019 and 2021 and by Csaba Szepesvari, Gang Niu and Sivan Sabato in 2022.
% Previous contributors include Dan Roy, Lise Getoor and Tobias
% Scheffer, which was slightly modified from the 2010 version by
% Thorsten Joachims & Johannes Fuernkranz, slightly modified from the
% 2009 version by Kiri Wagstaff and Sam Roweis's 2008 version, which is
% slightly modified from Prasad Tadepalli's 2007 version which is a
% lightly changed version of the previous year's version by Andrew
% Moore, which was in turn edited from those of Kristian Kersting and
% Codrina Lauth. Alex Smola contributed to the algorithmic style files.
\begin{thebibliography}{16}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
\providecommand{\doi}[1]{doi: #1}\else
\providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi
\bibitem[Behrmann et~al.(2019)Behrmann, Grathwohl, Chen, Duvenaud, and
Jacobsen]{InvertResNet}
Behrmann, J., Grathwohl, W., Chen, R. T.~Q., Duvenaud, D., and Jacobsen, J.-H.
\newblock Invertible residual networks.
\newblock In \emph{ICML}, pp.\ 573--582, 2019.
\newblock URL \url{http://proceedings.mlr.press/v97/behrmann19a.html}.
\bibitem[Chen et~al.(2018)Chen, Rubanova, Bettencourt, and Duvenaud]{NeuralODE}
Chen, R. T.~Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D.~K.
\newblock Neural ordinary differential equations.
\newblock In Bengio, S., Wallach, H., Larochelle, H., Grauman, K.,
Cesa-Bianchi, N., and Garnett, R. (eds.), \emph{Advances in Neural
Information Processing Systems}, volume~31. Curran Associates, Inc., 2018.
\newblock URL
\url{https://proceedings.neurips.cc/paper/2018/file/69386f6bb1dfed68692a24c8686939b9-Paper.pdf}.
\bibitem[Chen \& Gopinath(2000)Chen and Gopinath]{Gaussianization}
Chen, S. and Gopinath, R.
\newblock Gaussianization.
\newblock In Leen, T., Dietterich, T., and Tresp, V. (eds.), \emph{Advances in
Neural Information Processing Systems}, volume~13. MIT Press, 2000.
\newblock URL
\url{https://proceedings.neurips.cc/paper/2000/file/3c947bc2f7ff007b86a9428b74654de5-Paper.pdf}.
\bibitem[de~B{\'e}zenac et~al.(2021)de~B{\'e}zenac, Ayed, and Gallinari]{UDT}
de~B{\'e}zenac, E., Ayed, I., and Gallinari, P.
\newblock Cyclegan through the lens of (dynamical) optimal transport.
\newblock In Oliver, N., P{\'e}rez-Cruz, F., Kramer, S., Read, J., and Lozano,
J.~A. (eds.), \emph{Machine Learning and Knowledge Discovery in Databases.
Research Track}, pp.\ 132--147, Cham, 2021. Springer International
Publishing.
\newblock ISBN 978-3-030-86520-7.
\bibitem[Gatys et~al.(2016)Gatys, Ecker, and Bethge]{NST}
Gatys, L.~A., Ecker, A.~S., and Bethge, M.
\newblock Image style transfer using convolutional neural networks.
\newblock In \emph{2016 IEEE Conference on Computer Vision and Pattern
Recognition (CVPR)}, pp.\ 2414--2423, 2016.
\newblock \doi{10.1109/CVPR.2016.265}.
\bibitem[Goodfellow et~al.(2014)Goodfellow, Pouget-Abadie, Mirza, Xu,
Warde-Farley, Ozair, Courville, and Bengio]{GAN}
Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair,
S., Courville, A., and Bengio, Y.
\newblock Generative adversarial nets.
\newblock In Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N., and
Weinberger, K. (eds.), \emph{Advances in Neural Information Processing
Systems}, volume~27. Curran Associates, Inc., 2014.
\newblock URL
\url{https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf}.
\bibitem[Greydanus et~al.(2019)Greydanus, Dzamba, and Yosinski]{HNN}
Greydanus, S., Dzamba, M., and Yosinski, J.
\newblock Hamiltonian neural networks.
\newblock In Wallach, H., Larochelle, H., Beygelzimer, A., d\textquotesingle
Alch\'{e}-Buc, F., Fox, E., and Garnett, R. (eds.), \emph{Advances in Neural
Information Processing Systems}, volume~32. Curran Associates, Inc., 2019.
\newblock URL
\url{https://proceedings.neurips.cc/paper/2019/file/26cd8ecadce0d4efd6cc8a8725cbd1f8-Paper.pdf}.
\bibitem[Li et~al.(2020)Li, Wong, Chen, and Duvenaud]{NeuralSDE}
Li, X., Wong, T.-K.~L., Chen, R. T.~Q., and Duvenaud, D.
\newblock Scalable gradients for stochastic differential equations.
\newblock \emph{International Conference on Artificial Intelligence and
Statistics}, 2020.
\bibitem[Liu et~al.(2015)Liu, Luo, Wang, and Tang]{celebA}
Liu, Z., Luo, P., Wang, X., and Tang, X.
\newblock Deep learning face attributes in the wild.
\newblock In \emph{Proceedings of International Conference on Computer Vision
(ICCV)}, December 2015.
\bibitem[Miyato et~al.(2018)Miyato, Kataoka, Koyama, and Yoshida]{SpectralNorm}
Miyato, T., Kataoka, T., Koyama, M., and Yoshida, Y.
\newblock Spectral normalization for generative adversarial networks.
\newblock In \emph{International Conference on Learning Representations}, 2018.
\newblock URL \url{https://openreview.net/forum?id=B1QRgziT-}.
\bibitem[Nichol et~al.(2021)Nichol, Dhariwal, Ramesh, Shyam, Mishkin, McGrew,
Sutskever, and Chen]{GLIDE}
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B.,
Sutskever, I., and Chen, M.
\newblock Glide: Towards photorealistic image generation and editing with
text-guided diffusion models, 2021.
\newblock URL \url{https://arxiv.org/abs/2112.10741}.
\bibitem[Pascanu et~al.(2012)Pascanu, Mikolov, and Bengio]{RNNConditioning}
Pascanu, R., Mikolov, T., and Bengio, Y.
\newblock On the difficulty of training recurrent neural networks.
\newblock \emph{30th International Conference on Machine Learning, ICML 2013},
11 2012.
\bibitem[Rezende \& Mohamed(2015)Rezende and Mohamed]{NF}
Rezende, D. and Mohamed, S.
\newblock Variational inference with normalizing flows.
\newblock In Bach, F. and Blei, D. (eds.), \emph{Proceedings of the 32nd
International Conference on Machine Learning}, volume~37 of \emph{Proceedings
of Machine Learning Research}, pp.\ 1530--1538, Lille, France, 07--09 Jul
2015. PMLR.
\newblock URL \url{https://proceedings.mlr.press/v37/rezende15.html}.
\bibitem[Rombach et~al.(2021)Rombach, Blattmann, Lorenz, Esser, and Ommer]{LDM}
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B.
\newblock High-resolution image synthesis with latent diffusion models, 2021.
\newblock URL \url{https://arxiv.org/abs/2112.10752}.
\bibitem[Rosenblatt(1952)]{constrainedNF}
Rosenblatt, M.
\newblock {Remarks on a Multivariate Transformation}.
\newblock \emph{The Annals of Mathematical Statistics}, 23\penalty0
(3):\penalty0 470 -- 472, 1952.
\newblock \doi{10.1214/aoms/1177729394}.
\newblock URL \url{https://doi.org/10.1214/aoms/1177729394}.
\bibitem[Zhu et~al.(2017)Zhu, Park, Isola, and Efros]{CycleGan}
Zhu, J.-Y., Park, T., Isola, P., and Efros, A.~A.
\newblock Unpaired image-to-image translation using cycle-consistent
adversarial networks.
\newblock In \emph{2017 IEEE International Conference on Computer Vision
(ICCV)}, pp.\ 2242--2251, 2017.
\newblock \doi{10.1109/ICCV.2017.244}.
\end{thebibliography}