Elucidated Rolling Diffusion Models for Probabilistic Forecasting of Complex Dynamics

Extracted source text · arXiv 2506.20024

\documentclass{article} %
    \PassOptionsToPackage{numbers, compress}{natbib}




    \usepackage[preprint]{neurips_2025}





\addtocontents{toc}{\protect\setcounter{tocdepth}{0}}
\input{math_commands.tex}
\input{config}


\title{Elucidated Rolling Diffusion Models\\ for Probabilistic Weather Forecasting}


\author{
    Salva Rühling Cachay%
    \\ UC San Diego
    \And
    Miika Aittala \\ NVIDIA
    \And
    Karsten Kreis \\ NVIDIA
    \And
    Noah Brenowitz \\ NVIDIA
    \AND
    Arash Vahdat \\ NVIDIA
    \And
    Morteza Mardani\textsuperscript{*} \\ NVIDIA
    \And
    Rose Yu\textsuperscript{*} \\ UC San Diego
}


\newcommand{\fix}{\marginpar{FIX}}
\newcommand{\new}{\marginpar{NEW}}

\begin{document}
\maketitle


\begin{abstract}
Diffusion models are a powerful tool for probabilistic forecasting, yet most applications in high-dimensional chaotic systems predict future snapshots one-by-one. This common approach struggles to model complex temporal dependencies and fails to explicitly account for the progressive growth of uncertainty inherent to such systems. While rolling diffusion frameworks, which apply increasing noise to forecasts at longer lead times, have been proposed to address this, their integration with state-of-the-art, high-fidelity diffusion techniques remains a significant challenge. We tackle this problem by introducing Elucidated Rolling Diffusion Models (ERDM), the first framework to successfully unify a rolling forecast structure with the principled, performant design of Elucidated Diffusion Models (EDM). 
To do this, we adapt the core EDM components--its noise schedule, network preconditioning, and Heun sampler--to the rolling forecast setting. The success of this integration is driven by three key contributions: $(i)$ a novel loss weighting scheme that focuses model capacity on the mid-range forecast horizons where determinism gives way to stochasticity; $(ii)$ an efficient initialization strategy using a pre-trained EDM for the initial window; %
and $(iii)$ a bespoke hybrid sequence architecture for robust spatiotemporal feature extraction under progressive denoising. 
On 2D Navier–Stokes simulations and ERA5 global weather forecasting at $1.5^\circ$ resolution, \ours\ consistently outperforms key diffusion-based baselines, including conditional autoregressive EDM.
\ours\ offers a flexible and powerful general framework for tackling diffusion-based sequence generation problems where modeling escalating uncertainty is paramount.\footnote{Code is available at: \href{https://github.com/salvaRC/erdm}{https://github.com/salvaRC/erdm}}
\end{abstract}
\blfootnote{\textsuperscript{*}Equal advising. Corresponding authors listed alphabetically.}
\section{Introduction}
Probabilistic forecasting of chaotic, dynamical systems is essential for realistically assessing future states (``snapshots'') and their inherent uncertainties~\cite{bauer2015thequiet, clark2001ecology}.
For example, medium-range weather forecasting ($\le 15$ days) grapples with the inherent chaotic nature of the atmosphere and high sensitivities to the initial conditions.  
Ensembles of numerical weather prediction (NWP) models are therefore standard practice for estimating forecast uncertainty and the likelihood of high-impact events~\cite{palmer2002economic, bevacqua2023smiles}. 
However, each ensemble run of a flagship system such as IFS ENS~\cite{ifs-manual-cy46r1-ens} is computationally expensive,
motivating data-driven alternatives~\cite{bouallegue2024rise}.

Diffusion models have shown notable success in generative modeling, particularly for high-dimensional data such as images and videos~\cite{sohldickstein2015deepunsupervised, dhariwal2021diffusion}. The Elucidated Diffusion Model (EDM) framework~\cite{karras2022edm}, for example, enables stable high-fidelity sample generation due to its principled design that generalizes and improves on the foundational denoising diffusion probabilistic model (DDPM)~\cite{ho2020ddpm} and related diffusion paradigms~\cite{song2021ddim, song2021scoreSDE}.
This success motivates adapting diffusion models and EDM to model time-evolving dynamical systems.


Autoregressive conditional diffusion models have been successfully applied to fluid dynamics~\cite{kohl2023benchmarking} and weather~\cite{price2023gencast}. These approaches often use limited temporal context and predict future snapshots one by one.
To explicitly model the progressively increasing uncertainty common to sequence generation tasks,
Rolling Sequence Diffusion Models (RSDM)~\cite{ruhe2024rolling, wu2023ardiffusion} employ snapshot-dependent noise schedules that escalate noise for distant future snapshots.
However, existing RSDM's are derived from DDPM, rather than leveraging improved practical design choices, such as network preconditioning and second-order sampling, that were key to EDM's success.

Our work addresses these limitations by building upon the successful EDM framework \cite{karras2022edm}. We adapt and extend EDM's core design elements—its noise schedule formulation, denoiser network parameterization, network preconditioning, loss weighting, and sampling algorithms—to the specific demands of RSDM-like sequence modeling. 
In developing ERDM, we identified and addressed key design choices for progressive noise schedules and temporal loss weighting within the EDM paradigm, aspects that received limited attention in prior work. 
Our work thus provides novel solutions for these crucial components.
All in all, our key contributions are summarized as follows:
\begin{itemize}
    \item We develop \textbf{Elucidated Rolling Diffusion Models (ERDM)}, integrating snapshot-dependent progressive noise levels into EDM, 
    by adapting EDM's \textit{noise schedule, loss weighting, preconditioning}, and \textit{sampling}, complemented by a simple, but effective first-window initialization technique and a hybrid 3D denoiser architecture.


    \item Extensive experiments on {ERA5 weather data} and {Navier-Stokes flows} show that ERDM consistently improves on the autoregressive, conditional EDM baseline in both predictive skill and calibration. On ERA5, ERDM rivals state-of-the-art weather forecasters (IFS ENS and NeuralGCM ENS) for mid- to long-range lead times, while being computationally more efficient--requiring only 4 H200 GPUs and 5 days for training--and exhibiting high physical fidelity in its power spectra.
\end{itemize}





\input{figures/latex/diagram}


\section{Related Works}
\label{sec:relatedwork}
\paragraph{Diffusion models for spatiotemporal data.}
Diffusion models, initially transformative in image and audio generation, are increasingly being adapted for spatiotemporal modeling. 
Latent-diffusion variants~\cite{vahdat2021ldm,rombach2022stablediffusion} provide the computational advantages integral to state-of-the-art video generators~\cite{ho2022imagenvideo,harvey2022flexiblevideos,blattmann2023stablevideodiffusion}.
Recent work refines the diffusion process itself to strengthen temporal structure and boost efficiency~\cite{cachay2023dyffusion,lippe2023pderefiner}.
Rolling sequence diffusion models (RSDM) advance this idea by assigning progressively larger noise levels to later timesteps, mirroring the growth of predictive uncertainty~\cite{wu2023ardiffusion,ruhe2024rolling, kim2024fifodiffusion}.
We embrace the same intuition but depart from these DDPM-based RSDM's in two crucial ways.  
(i) Our approach is derived from the EDM framework, which generalises and improves upon DDPM. %
(ii) We re-examine often-overlooked design choices—noise schedule, loss weighting, and network architecture—and show that their coordination is critical for forecasting chaotic dynamics.  
Progressive schedules like ours form a specific instance of the completely randomized schedules proposed in~\citet{chen2024diffusionforcing}, corresponding to its `pyramid sampling' scheme. However, our ablations indicate that such training-time randomization degrades performance--possibly because it misaligns with the temporally increasing uncertainty inherent to the chaotic systems that we study.


\paragraph{Data-driven medium-range weather forecasting.}
Initial machine learning (ML)-based attempts in this area 
were limited to deterministic forecasting~\cite{pathak2022fourcastnet, bi2023accurate, lam2023learning, nguyen2023climax, nguyen2023stormer, bodnar2024aurora}, often differentiated by their underlying neural network architectures~\cite{karlbauer2024comparing}. 
More recently diffusion models have shown promise for probabilistic weather forecasting~\cite{price2023gencast, andrae2025continuous}, and other tasks such as downscaling~\cite{mardani2025corrdiff, lopez2025dynamical, chen2023swinrdm, srivastava2024precip}, data assimilation~\cite{huang2024diffda, rozet2023scorebased, manshausen2025generative}, and emulation~\cite{li2024genemulation, cachay2024probemulation, pathak2024stormcast}. 
Latent-variable models offer an alternative probabilistic route~\cite{oskarsson2024probabilistic}, and physics-ML hybrids such as NeuralGCM show promise but face scalability constraints~\cite{kochkov2023neural}.
Of particular relevance to our work is GenCast~\cite{price2023gencast}, which successfully applied the EDM framework, using the graph architecture from~\cite{lam2023learning}, for next-step probabilistic weather forecasting.
Our primary baseline in this paper, EDM, corresponds to the same approach but using our experimental setup (e.g., same U-Net architecture and dataset resolution; see \cref{sec:gencastvsedm} for more details). This design ensures a controlled and direct comparison between \ours{} and an EDM-based next-step forecasting approach.




\section{Background}
In the following, we refer to clean data as $\vy$, noisy data as $\vx$, and we use $\vxbar$ to make clear when the data is corrupted according to progressively increasing noise levels. We abbreviate sequences by $\vy_{1:W}:=\vy_1,\dots,\vy_W$.
Probabilistic forecasting is concerned with predicting a future sequence $\{\vy_\ttime\}_{\ttime=1}^{\ttimemax}$ given an initial condition $\vy_{0}$, where the horizon $\ttimemax$ can be arbitrarily large. This requires learning the joint conditional distribution, 
which can be approximated with, e.g., diffusion models, by $p_\theta(\vy_{1:\ttimemax} | \vy_{0})$.
Directly modeling the high-dimensional joint distribution
is often intractable.
A common alternative is modeling a smaller window of size $W$
by learning a short-term transition, $p_\theta(\vy_{w+1:w+W} | \vy_{w})$~\citep{price2023gencast, harvey2022flexiblevideos}. Long sequences are then generated autoregressively. %

Conditional diffusion models can be applied to this task by learning a denoiser network $\denoiser(\vx_{1:W};\sigma, \vy_0)$. The denoiser is trained to predict the target sequence, $\vy_{1:W}$, given the initial condition, $\vy_0$, and data $\vx_{1:W}$ corrupted with Gaussian noise of standard deviation $\sigma\in[\smin,\smax]$. The maximum noise level needs to be chosen such that $\smax\gg\sigma_{\text{data}}$, where $\sigma_{\text{data}}$ is the standard deviation of the data $\vy\sim \pdata$ and $\vy\in\R^D$. The trained network can then be used jointly with an ODE or SDE solver to iteratively generate the clean data starting from pure noise, $\noise\sim\mathcal{N}(\boldzero, \smax^2\boldi_{W\times D})$, and $\vy_0$. See \cref{sec:background_app} for additional background on diffusion.
Often, $W=1$~\cite{price2023gencast, kohl2023benchmarking}, and the diffusion model becomes a next-step forecasting model.
However, the short window size models \textit{limited temporal interactions} and generating each step requires a full reverse diffusion process, incurring substantial \textit{computational cost}. Standard sequence diffusion models with $W>1$ typically apply noise uniformly across the prediction window, failing to capture the \textit{increasing uncertainty} inherent to future predictions.

Rolling Sequence Diffusion Models (RSDMs)~\citep{ruhe2024rolling, chen2024diffusionforcing} address prior limitations by explicitly incorporating progressive forecasting uncertainty.  The core innovation is a \textit{progressive noise schedule} with $W>1$. Instead of uniform noise, RSDMs use a monotonically increasing schedule of noise standard deviations $0 \approx \sigmaprog{1}(\dtime) < \sigmaprog{2}(\dtime) < \dots < \sigmaprog{W}(\dtime) \approx \smax$, where $\dtime$ denotes the global diffusion time.
A noisy future state is thus modeled as $\vxbar_w \sim \mathcal{N}(\vy_{w}, \varprog_{w}(\dtime)\boldi_D)$. This structure reflects higher uncertainty for predictions further into the future.
The denoiser $\denoiser(\vxbar_{1:W}; \sigmaerdmbold_{1:W}(t))$ is trained analogously to a standard diffusion model to predict the clean sequence, $\vy_{1:W}$.
In our implementation, we omit direct conditioning of the denoiser on the last known clean state, $\vy_{0}$. This is feasible because we initialize the first window using an external forecaster (itself conditioned on $\vy_{0}$). The minimally noisy start of the window provides sufficient conditioning to predict subsequent snapshots. When requiring the RSDM to self-initialize its first window, this is not possible~\cite{ruhe2024rolling}.







\section{Forecasting with Elucidated Rolling Diffusion Models}
Our work, ERDM, integrates the rolling diffusion concept with the {Elucidated Diffusion Model (EDM)} framework~\citep{karras2022edm}. While RSDMs introduce the idea of progressive noise, they usually do not follow EDM's principled design choices for preconditioning, loss weighting, and sampler design, which we hypothesize could bring benefits when incorporated into RSDMs too.
ERDM aims to systematically port and adapt these EDM principles to the rolling, progressive noise setting, thereby providing a robust and theoretically grounded approach to sequential data generation.

\paragraph{Rolling EDM noise schedule.}
In contrast to existing RSDMs, which use cosine or linear noise schedules, we propose adapting EDM's sampling-time noise schedule to capture progressively increasing noise levels, by defining %
\begin{align}
    \sigmavecprog(\offset) &= \left(\bar{\sigma}_1(\offset), \dots, \sigmabar_W(\offset)\right); \qquad
    \sigmabar_w(\offset) =\big({\smax}^\frac{1}{\rho} + t_{w, \offset} ( {\smin}^\frac{1}{\rho}\!-\!{\smax}^\frac{1}{\rho} ) \big)^\rho,%
\end{align}
\input{figures/latex/noise_schedule}%
where $t_{w,\offset} = 1 - \frac{w-\offset}{W}$ is the local diffusion time of the snapshot.
The $1-$ is important so that near-future snapshots receive minimal amounts of noise, while far-future snapshots receive high levels of noise. 
This ensures the correct inductive bias that distant snapshots are more uncertain than near-future ones.
The continuous noise schedule is effectively divided into $W$ segments such that the noise levels at snapshot $w$ satisfy $\sigmaerdm_w(t) \in 
[\sigmaerdm_{w-1}(0), \sigmaerdm_{w+1}(1)]$, where $\offset \in [0, 1], \sigmaerdm_{0}(0):=\smin,$ and $ \sigmaerdm_{W+1}(1):=\smax$.
Our formulation thus satisfies $\smin=\sigmaerdm_1(1)<\sigmaerdm_1(0)=\sigmaerdm_2(1)<\dots<\sigmaerdm_W(0)=\smax$.
Two example noise schedules are illustrated in~\cref{fig:noise_schedule}.
ERDM is trained to see any noise schedule level within the noise segments (color gradients in~\cref{fig:noise_schedule}) by randomizing $\offset\in U([0,1))$.
During sampling, each $\sigmaerdm_{w}(\dtime)$ is evolved from $\sigmaerdm_{w}(0)$ to $\sigmaerdm_{w}(1)$.
We found it important to tune the curvature parameter, $\rho$. 
Our proposed default value, $\rho=-10$, effectively causes snapshots to be under less noise compared to the default EDM choice, $\rho=7$, which provides more information for accurate denoising. Importantly, these noise schedule parameters $\smin, \smax,$ and $\rho$ need to be chosen at training time. %



\paragraph{Probability flow ODE.}
Having defined our windowed noise schedule $\sigmaerdmbold(\dtime) = (\sigmaerdm_1(\dtime), \ldots, \sigmaerdm_W(\dtime))$, we can formulate the associated probability flow ODE~\cite{song2021scoreSDE}, which describes the addition and removal of noise when moving the diffusion `time', $\dtime$, forward ($1\rightarrow0$) and backward ($0\rightarrow1$), respectively.
For a noisy sequence $\vxbar := \vxbar_{1:W}$, this ODE, as derived in~\cref{sec:proba_ode_app}, is:
\begin{align}
    \diff\vxbar &= - \text{diag}(\sigmaerdm_{1}(\dtime)\dot{\sigma}_1(\dtime)\boldi_D, \ldots, \sigmaerdm_{W}(\dtime)\dot{\sigma}_W(\dtime)\boldi_D) 
    \nabla_{\vxbar} \log p(\vxbar; \sigmaerdmbold(\dtime)) \diff \dtime,\label{eq:ode_erdm_user}
\end{align}
where $\dot{\sigma}_w(\dtime)$ denotes the diffusion time derivative of $\sigmaerdm_w(\dtime)$, and $\nabla_{\vxbar}\log p(\vxbar; \sigmaerdmbold(\dtime))$ is the score function~\cite{hyvarinen05score} that we will approximate with a denoiser network.
This ODE governs $W$ coupled diffusion processes evolving simultaneously within the window. During inference (i.e., the backward process), an ODE solver is used, typically for N discrete steps. Upon completion of one such integration to $\dtime=1$, the first snapshot, $\vxbar_1(1)$, is fully denoised (its noise level $\sigmaerdm_1(1) = \smin$) and is output as the forecast of $\vy_1$. Concurrently, the subsequent snapshots $\vxbar_{2:W}(1)$ are partially denoised. 
Key to the rolling mechanism is the design of the per-snapshot noise schedules $\sigmaerdm_w(\dtime)$, which ensure the noise level of $\vxbar_w(1)$ matches the initial noise level of the $(w-1)$-th slot (i.e., $\sigmaerdm_w(1) = \sigmaerdm_{w-1}(0)$). 
This specific noise structure at $\dtime=1$ enables the window to be advanced efficiently. For the next forecasting step, the sequence $\vxbar_{2:W}(1)$ forms the initial state for the first $W-1$ slots at the next iteration's $\dtime=0$. A new, fully-noised snapshot, $\noise_{W+1} \sim \mathcal{N}(\boldzero, \smax^2\boldi_D)$, is then appended as the $W$-th element of this new window. The ODE solving process from $\dtime=0$ to $1$ is then repeated for this updated window. This iterative sampling procedure is visualized in~\cref{fig:diagram}.

\subsection{Training Objective}
We now describe how to train a denoiser network, $\denoiser(\xnoisy; \sigmaerdmbold(\dtime))$, that can be used to solve the probability flow ODE above. Following EDM, it comprises a raw neural network $F_\theta$ and preconditioning. To account for snapshot-dependent noise levels, we vectorize the preconditioning functions: %
\begin{align}
    \denoiser(\xnoisy; \sigmaerdmbold(\dtime)) = \cskip(\sigmaerdmbold(\dtime))\xnoisy + \cout(\sigmaerdmbold(\dtime))F_\theta(\cin(\sigmaerdmbold(\dtime))\xnoisy, \cnoise(\sigmaerdmbold(\dtime))). \label{eq:denoiser_edm_style_final}
\end{align}
The preconditioning functions $\cskip, \cout, \cin, \cnoise$ are adapted from EDM and applied per-snapshot based on the corresponding $\sigmaerdm_{w}(t)$ in $\sigmaerdmbold(\dtime)$.
The denoiser network is trained to predict the clean window $\vy_{1:W}$ from its noisy version, $\xnoisy\sim\mathcal{N}(\vy_{1:W}, \sigmaerdmbold(\offset)^2\boldi)$. During training, we randomize $\offset~\sim U([0,1))$ such that the network learns to deal with the full extent of the segments of~\cref{fig:noise_schedule}. %

\paragraph{Uncertainty-aware loss reweighting.}
EDM employs a loss weighting $\lambda(\sigma) = ( \sigma^2 + \sigma_{\text{data}}^2 ) / (\sigma \sigma_{\text{data}})^2$ to ensure their core network $F_\theta$ targets a signal with unit variance, promoting training stability.
We apply $\lambda(\sigmaprog{w})$ per frame as a direct extension. However, the rolling mechanism implies that certain noise levels within the $\sigmavecprog(\dtime)$ window are more critical for learning the ``de-mixing'' of temporal information. EDM itself indirectly emphasizes intermediate noise levels by sampling $\sigma$ values from a lognormal distribution $p_{\text{train}}(\sigma)$ during training, whose PDF corresponds to %
\begin{align}
    f(\sigma; \Pmean, \Pstd) = \frac{1}{\sigma \Pstd \sqrt{2\pi}} \exp\left(-\frac{(\ln(\sigma) - \Pmean)^2}{2\Pstd^2}\right). \label{eq:lognormal_pdf}
\end{align}
Since ERDM conditions on a fixed $\sigmavecprog(\dtime)$ per training instance, always covering a wide range of noise levels (rather than sampling each $\sigmaprog{w}$ from $p_{\text{train}}(\sigma)$), we propose a distinct loss weighting for ERDM that combines the EDM unit-variance objective with an emphasis on these critical intermediate noise levels within the progressive schedule. The effective loss weighting for snapshot $w$ in ERDM is $\lambda(\sigmaprog{w}) \cdot f(\sigmaprog{w}; \Pmean, \Pstd)$,
which maintains EDM's target normalization via $\lambda(\sigmaprog{w})$ while using $f(\sigmaprog{w})$ to upweight snapshots at the most informative noise levels.
All things considered, we can write out our proposed score matching objective as
\begin{align*}
    \min_{\theta} 
    \E_{\vy_{1:W} \sim p_{\rm data}}
    \E_{\offset \in U([0,1))}
    \E_{\{\noise_w \sim \mathcal{N}(\boldzero, \sigmawindow^2_w\boldi)\}_{w=1}^W}
    \sum_{w=1}^W \lambda(\sigma_{w}) f(\sigma_w)
    \lVert \denoiser( \vy_{1:W} + \noise_{1:W}; \sigmawindow)_w - \vy_w\rVert ^2_2
    , \label{eq:score_matching_erdm}
\end{align*}
where we shorten $\sigmawindow:=\sigmaerdmbold(\offset)$.
The full training algorithm is described in~\cref{alg:training}.
\input{algos/training}

\paragraph{Remark (temporal noise correlations).}
In both training and sampling algorithms, we draw i.i.d.\ Gaussian noise $\noise_w\sim\mathcal N(\boldzero,\boldi_D)$ for each snapshot $w$ independently. Recent studies show that temporally correlated noise improves video diffusion~\citep{chang2024warpednoise,ge2024noiseprior,andrae2025continuous,liu2025equivdm}. Such priors are orthogonal to our sliding-window framework and can replace the i.i.d.\ draws in Algorithms~\ref{alg:training}–\ref{alg:sampling_simple}. In our experiments, we use the progressive noise model with $\alpha=1$ proposed by~\citet{ge2024noiseprior}, which we found to slightly improve long-range forecasts (see~\cref{sec:ablations_app}). More details are provided in~\cref{sec:noise_prior}.

\subsection{Sampling}
\label{sec:sampling}
The trained denoiser, $\denoiser$, can be used to estimate the score function, $\log p(\vxbar; \sigmaerdmbold(\dtime))$, in~\cref{eq:ode_erdm_user} with $(\denoiser(\vxbar,\sigmaerdmbold(\dtime)) - \vxbar) / \sigmaerdmbold^2(\dtime)$ for any $\dtime\in[0,1)$~\cite{karras2022edm}.
Based on this identity and the described probability flow ODE intuition,
a mathematical formulation of our proposed sampling algorithm is provided in~\cref{alg:sampling_simple}. For simplicity, we omit the second-order Heun step and sampling stochasticity, which are detailed in~\cref{sec:sampling_app}. Note, however, that in our main experiments, we always use the deterministic Heun sampler.
\paragraph{First-window initialization.}%
The backward ODE requires a noisy window $\vxbar_{1:W}$ whose clean latent $\vy_{1:W}$ is unknown. %
\citet{ruhe2024rolling} addresses this issue by co-training the rolling model to generate %
$\vxbar_{1:W}$ from clean context snapshots and a sequence of pure noise. This approach introduces extra hyperparameters and diverts the denoiser's capacity between two distinct denoising tasks.
As an alternative, we propose drawing $\hat\vy_{1:W}$ from an \emph{external} forecaster $p_\theta(\vy_{1:W}|\vy_0)$.
We then sample $\vxbar_{w}\sim\mathcal N\!\bigl(\hat\vy_{w},\,\varerdm_w(0)\boldi_D\bigr)$ for $w=1,\dots,W$, from where we can start our sliding-window sampling algorithm. This choice reuses mature short-range models and injects schedule-matched uncertainty; any existing forecaster (diffusion-based or not) can supply $\hat\vy_{1:W}$. In our experiments, we rely on the EDM baselines for initializing ERDM. We ablate this choice in~\cref{sec:init_app}.

\input{figures/tikz/arch_sketch}
\subsection{Denoiser architecture for temporal dynamics}
Our proposed diffusion model operates on temporal sequences, necessitating an architecture capable of capturing temporal dependencies. Naive adaptations of 2D architectures, such as stacking the time dimension into channels, disrupt the inherent temporal structure and led to suboptimal results in our experiments (see \cref{sec:ablations}). Alternatively, extending to 3D convolutions, while preserving temporality, often incurs significant computational overhead and potentially higher sample complexity.
Instead, we adopt a prevalent strategy from latent video diffusion models~\cite{blattmann2023stablevideodiffusion}: augmenting a well-established 2D U-Net architecture with explicit temporal processing layers. Specifically, we integrate causal temporal attention layers into the 2D ADM U-Net~\cite{dhariwal2021diffusion} used in EDM, positioning before each down- and up-sampling block. 
These temporal layers also incorporate noise level information via a mechanism analogous to the adaptive layer normalization used in the 2D U-Net blocks.
Figure~\ref{fig:arch_sketch} illustrates this interleaved modular design. %
This modular design could facilitate static pre-training of the 2D backbone, yet we observed that end-to-end training on the sequential data was more effective for our tasks (see \cref{sec:ablations_app}). While we found the proposed architecture to be key for generating good forecasts, it is slower and has higher memory needs than a 2D variant.








\input{algos/sampling_simple}  %





















\section{Experiments}
\vspace{-1mm}
\subsection{Evaluation and Metrics}
\vspace{-0.5mm}
Due to the importance of probabilistic forecasts and uncertainty quantification, we focus on two key ensemble-based probabilistic metrics: Continuous Ranked Probability Score (CRPS)~\cite{matheson1976crps}, and the spread-skill ratio (SSR) based on $M$-member ensembles, where $M=50$ for Navier-Stokes and $M=10$ for ERA5.
The CRPS is a proper scoring rule commonly used to evaluate probabilistic forecasts~\cite{gneiting2014Probabilistic}. The spread-skill ratio is defined as the ratio of the square root of the ensemble variance to the corresponding ensemble-mean RMSE. It serves as a measure of the reliability of the ensemble, where values smaller than 1 indicate 
underdispersion%
, and larger values overdispersion~\cite{fortin2014ssr, rasp2023weatherbench2}.

\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figures/raw/ns-test-rollout-metrics.png}
    \caption{\small
    Navier-Stokes test rollout over 64 time steps with 50 ensemble members. \ours{} superior performance in both CRPS and calibration compared to single- and multi-step EDM baselines, except for the initial 3 timesteps. 
    Beyond timestep 15, ERDM consistently delivers an approximate $50\%$ improvement in CRPS over the next best model (EDM $W=4$), demonstrating particular strength in long-range forecasting scenarios.
    }
    \label{fig:ns_test_rollout}
    \vspace{-3mm}
\end{figure}
\vspace{-1mm}
\subsection{Navier-Stokes fluid dynamics}
\vspace{-0.2mm}
\paragraph{Dataset.}
We use the Navier-Stokes fluid dynamics benchmark from \cite{otness21nnbenchmark}, defined on a $221 \times 42$ grid. Each simulation features four randomly placed circular obstacles influencing the flow, with fluid viscosity set to $1 \times 10^{-3}$. The dataset comprises $x$ and $y$ velocities and pressure fields. All models receive boundary conditions and obstacle masks as auxiliary inputs. For testing, models predict a 64-timestep trajectory from a single initial snapshot.


\vspace{-0.5mm}
\paragraph{Baselines.}
We benchmark ERDM against DYffusion~\cite{cachay2023dyffusion}, the current state-of-the-art method on this dataset. To ensure a fair comparison, we retrained DYffusion using our experimental setup, achieving an improvement of over $3\times$ on its originally reported CRPS scores.
Our primary focus, however, is to evaluate ERDM's performance relative to common EDM-derived approaches for dynamics forecasting.
Consequently, our key baseline is an EDM denoiser parameterized by $\denoiser(\vx_1 \ldots, \vx_W; \sigma, \vy_0)$, where all $\vx_w$ are corrupted according to the same $\sigma$. With $W=1$ we recover a next-step forecasting conditional EDM as in~\cite{price2023gencast, kohl2023benchmarking}. We tried $W\in\{1,2,4,6\}$, and found $W=4$ to work best for EDM.
All baselines are trained on three random seeds and share the same architecture as much as possible. 
For ERDM, we use $W=6, \smin=0.002, \smax=200, \rho=-10, \Pmean=0.5, \Pstd=1.2, N=1.25$.
Further details are provided in \cref{sec:impldetails}. 
\vspace{-0.8mm}
\paragraph{Results.}
In \cref{fig:ns_test_rollout}, we benchmark \ours\ against the baselines on forecasting the five Navier-Stokes test trajectories of length 64. While the EDM-based baselines start well in terms of CRPS, even beating \ours\ for the very first few time steps, they exhibit a significantly higher error growth than \ours. We attribute this to the explicit modeling of progressive uncertainty built into our model. At the end of the rollout, \ours\ achieves a $50\%$ better CRPS than the best baseline, EDM with a window size of $W=4$. Similarly, DYffusion starts worse and catches up with the EDM $W=4$ baseline at the end of the rollout, but not with \ours{}.
In terms of calibration, we observe that \ours\ consistently outperforms the EDM baselines, which are noticeably undercalibrated. We note that this is despite comprehensive tuning of these baselines, where we were only able to achieve calibration improvements at the significant expense of CRPS skill (not shown).
MSE scores, qualitative results, and example generated videos are discussed in~\cref{sec:ns_results_app}. 

\subsection{ERA5 weather forecasting}
\label{sec:era5}
\vspace{-1mm}
\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figures/raw/era5-rel.png}
    \caption{\small Relative CRPS (lower is better) over single-step EDM baseline as a function of lead time, up to 15 days, for 8 selected variables. \ours{} consistently outperforms EDM across most variables and lead times by up to $10\%$. Furthermore, it performs competitively against the state-of-the-art operational physics-based model IFS ENS and the hybrid model NeuralGCM ENS, especially for long lead times, while being more efficient. 
    \vspace{-5mm}
    }
    \label{fig:era5_rel}
\end{figure}
\paragraph{Dataset.}
We benchmark our model on medium-range weather forecasting on the ERA5 reanalysis dataset. We use the $1.5^\circ$ resolution of the data (a $240\times121$ grid) provided by~\cite{rasp2023weatherbench2}, with 69 prognostic (input and output) variables: Temperature (\texttt{t}), geopotential (\texttt{z}), specific humidity (\texttt{q}), and the u and v components of wind (\texttt{u,v}) over 13 pressure levels (numbers after the abbreviation refer to the level in hPa) as well as the four surface variables 2m temperature (\texttt{2t}), mean sea level pressure (\texttt{mslp}), and u and v components of winds at 10m (\texttt{10u, 10v}). 
We train our models on $12$-hourly data, as in~\cite{price2023gencast}, from 1979 to 2020 and evaluate on 64 2021 initial conditions at 00/12 UTC.

\paragraph{Baselines.} 
Our primary baseline is a conditional next-step forecasting EDM, trained using a methodology consistent with ERDM's development. 
To gauge the absolute performance of \ours, we also benchmark it against two prominent external models: (1) IFS ENS: The European Centre for Medium-Range Weather Forecasts' (ECMWF) operational, physics-based ensemble forecasting system~\cite{ifs-manual-cy46r1-ens}; (2) NeuralGCM ENS: A hybrid ML-physics stochastic model~\cite{kochkov2023neural}. 
Official $1.5^\circ$-resolution forecasts for both external models were sourced from Weatherbench-2~\cite{rasp2023weatherbench2} for all variables where available.
The IFS ENS evaluation uses the same 2021 initial condition dates as our EDM and ERDM models, but is verified against operational \emph{analyses} (as opposed to ERA5), as recommended by~\cite{rasp2023weatherbench2}.
Due to the unavailability of NeuralGCM forecasts for 2021, we use its 2020 forecasts and evaluate them against the corresponding 2020 ERA5 data.
Our conditional EDM baseline can be seen as a reproduction of GenCast~\cite{price2023gencast}, for which forecasts are only available at higher spatial resolutions and a limited set of variables, but using our experimental setup and neural architecture. 
Notably, training \ours{} is considerably less computationally expensive than NeuralGCM ENS $1.5^\circ$ (GenCast $1^\circ$), which required 128 (32) v5 TPUs and 10 (3.5) days to train. In contrast, \ours{} was trained on 4 H200 GPUs in 5 days only. Lastly, we also include scores for \texttt{z500} and \texttt{2t} from Graph-EFM~\cite{oskarsson2024probabilistic}, taken from their paper. These scores are provided for reference, noting that Graph-EFM is trained on 20 more years and evaluated in 2020.
For ERDM, we use $W=6,\smin=0.002,\smax=500,\rho=-10, \Pmean=2, \Pstd=1.2, N=2$.
See~\cref{sec:impldetails} for more experimental details. 

\input{figures/latex/spectra_main_text}
\paragraph{Results.}
An evaluation of ERDM's performance, illustrated in~\cref{fig:era5_rel}, reveals its consistent superiority in relative CRPS skill over our primary EDM baseline, by up to $10\%$.
Similarly, it significantly outperforms Graph-EFM on the four CRPS scores reported in their study~\cite{oskarsson2024probabilistic}.
Compared to the more computationally complex models, IFS ENS and NeuralGCM, ERDM performs competitively, but exhibits short-range weakness on some variables versus IFS ENS.
We attribute this to our EDM-based initialization strategy, which could potentially be addressed by initializing the first window of ERDM with IFS ENS. 
In terms of spread-skill ratio, ERDM, along with IFS ENS, delivers the most calibrated forecasts (see~\cref{fig:era5_ssr_appendix}). Conversely, EDM and NeuralGCM often show underdispersion in short-range forecasts.
A crucial aspect of evaluating any ensemble forecast is its physical realism. In this regard, ERDM performs exceptionally well.
As depicted in~\cref{fig:era5_spectra}, its normalized power spectra are on par with those of the physics-based IFS ENS.
This is a significant achievement, as many ML models typically struggle to reproduce such physically consistent spectra—a challenge evident with NeuralGCM in~\cref{fig:era5_spectra} and documented by~\citet{rasp2023weatherbench2}.  Results for more variables, absolute power spectra, spread-skill ratio results, as well as forecast visualizations, are included in~\cref{sec:results_app}.


\subsection{Ablations}
\label{sec:ablations}
We conducted a comprehensive series of ablation studies on the Navier-Stokes forecasting task to validate our key design choices and understand their individual contributions to ERDM's performance. We analyze it in full detail in~\cref{sec:ablations_app}, and only explain the crucial design choices here.
Our ablations reveal that certain architectural and training strategy elements are fundamental to ERDM's success, with their absence or significant alteration leading to performance substantially worse than even the single-step EDM baseline.
\begin{enumerate}
    \item \textbf{Appropriate progressive noise schedule \& fixed training:} The design of the noise schedule is paramount. Default EDM hyperparameters (e.g., $\rho=7$) prove suboptimal for ERDM, yielding $2\times$ worse CRPS than our proposed default schedule with $\rho=-10$. See~\cref{fig:noise_schedule} for intuition on why.
    Furthermore, training with a fixed, optimized noise schedule is critical; randomizing the schedule during training
    degrades performance by nearly $2\times$. 
    Despite this, such randomized training can serve as a heuristic to identify an effective fixed schedule for subsequent retraining.
    This finding contrasts with the randomized training procedure from~\citet{chen2024diffusionforcing}.
    \item \textbf{Strategic loss weighting:} Reweighing the losses based on the lognormal probability density function of noise levels, $f(\sigma)$, as proposed, is crucial. Removing this weighting 
    causes a $>2\times$ performance drop. 
    We generally found that a larger mean of the lognormal distribution, $\Pmean>0$, than EDM's default, $-1.2$, was necessary for ERDM to achieve optimal results, confirming the importance of tuning here.
    \item \textbf{Dedicated spatiotemporal architecture:} A crucial element is the use of a proper architecture that explicitly models spatiotemporal dependencies. Naively stacking the time dimension into the channel dimension of a 2D architecture 
    results in severe $4\times$ performance degradation, confirming the need for our bespoke temporal architecture.

\end{enumerate}

\section{Conclusion} %
In this work, we introduced Elucidated Rolling Diffusion Models (ERDM), a novel framework designed to address the significant challenges of long-range forecasting in complex, chaotic scientific systems. By adapting and extending the successful design of EDM, ERDM incorporates several key innovations: a progressive temporal noise schedule to explicitly model increasing forecast uncertainty, snapshot-dependent preconditioning and loss weighting tailored for sequential data, an efficient decoupled initialization strategy, and a bespoke hybrid spatiotemporal architecture. Comprehensive ablation studies confirm the need for synergistically combining these design choices.

Our extensive experiments on the demanding ERA5 weather dataset and Navier-Stokes flow simulations demonstrate ERDM's efficacy at probabilistic forecasting. It consistently outperforms relevant EDM-based baselines, particularly in terms of CRPS and calibration over extended forecast horizons. 
Although ERDM achieves these results with fewer neural function evaluations (e.g., $12$ sampling steps per snapshot on ERA5 vs. $20$ for the EDM baseline), the greater computational cost per evaluation due to its 3D architecture results in sampling times similar to next-step EDM forecasting.

ERDM offers a robust and reliable approach for probabilistic forecasting in scientific domains. Future work could focus on optimizing the computational efficiency of the 3D architecture, exploring dynamic adjustments of the window size and synergies with latent space modeling, and applying the ERDM framework to a broader range of challenging spatiotemporal dynamics. 

\section*{Acknowledgements}
S.R.C. acknowledges generous support from the summer internship and collaboration at NVIDIA. This research used resources from the National Energy Research Scientific Computing Center (NERSC), a Department of Energy User Facility, using NERSC awards DDR-ERCAP0034142, ASCR-ERCAP0033209, and EESSD-ERCAP0033799. This work was supported in part by the U.S. Army Research Office
under Army-ECASE award W911NF-07-R-0003-03, the U.S. Department Of Energy, Office of Science, IARPA HAYSTAC Program, and NSF Grants \#2205093, \#2146343, \#2134274, CDC-RFA-FT-23-0069, DARPA AIE FoundSci and DARPA YFA.


\bibliography{references}
\bibliographystyle{plainnat}

\input{sections/99appendix}
\end{document}


\usepackage{amsmath,amsfonts,bm}
\usepackage{algorithm}
\usepackage[noend]{algpseudocode}


\newcommand{\denoiser}{D_{\theta}} %
\newcommand{\Ftheta}{F_{\theta}} %
\newcommand{\sigmavec}{\boldsymbol{\sigma}} %
\newcommand{\sigmavecprog}{\bar{\boldsymbol{\sigma}}^{\text{}}} %
\newcommand{\sigmaprog}[1]{\bar{\sigma}_{#1}^{\text{}}} %
\newcommand{\varprog}{\bar{\sigma}^2} %

\newcommand{\sigmawsde}[1]{\sigma_{#1}^{\text{SDE}}} %
\newcommand{\yhatwin}{\hat{\mathbf{y}}^{\text{win}}} %
\newcommand{\yhat}{\hat{\mathbf{y}}} %
\newcommand{\diff}{\mathrm{d}} %
\newcommand{\odetime}{\tau} %
\newcommand{\smax}{\sigma_{\text{max}}}
\newcommand{\smin}{\sigma_{\text{min}}}
\newcommand{\Pmean}{P_{\text{mean}}} %
\newcommand{\Pstd}{P_{\text{std}}}   %
\newcommand{\cin}{c_{\text{in}}}     %
\newcommand{\cout}{c_{\text{out}}}   %
\newcommand{\cskip}{c_{\text{skip}}}   %
\newcommand{\cnoise}{c_{\text{noise}}} %



\newcommand{\noise}{\boldsymbol{\epsilon}}
\newcommand{\xtsigma}[2]{\vx_{#1}^{\sigma_{#2}}} %

\newcommand{\figleft}{{\em (Left)}}
\newcommand{\figcenter}{{\em (Center)}}
\newcommand{\figright}{{\em (Right)}}
\newcommand{\figtop}{{\em (Top)}}
\newcommand{\figbottom}{{\em (Bottom)}}
\newcommand{\captiona}{{\em (a)}}
\newcommand{\captionb}{{\em (b)}}
\newcommand{\captionc}{{\em (c)}}
\newcommand{\captiond}{{\em (d)}}

\newcommand{\newterm}[1]{{\bf #1}}


\def\figref#1{figure~\ref{#1}}
\def\Figref#1{Figure~\ref{#1}}
\def\twofigref#1#2{figures \ref{#1} and \ref{#2}}
\def\quadfigref#1#2#3#4{figures \ref{#1}, \ref{#2}, \ref{#3} and \ref{#4}}
\def\secref#1{section~\ref{#1}}
\def\Secref#1{Section~\ref{#1}}
\def\twosecrefs#1#2{sections \ref{#1} and \ref{#2}}
\def\secrefs#1#2#3{sections \ref{#1}, \ref{#2} and \ref{#3}}
\def\eqref#1{equation~\ref{#1}}
\def\Eqref#1{Equation~\ref{#1}}
\def\plaineqref#1{\ref{#1}}
\def\chapref#1{chapter~\ref{#1}}
\def\Chapref#1{Chapter~\ref{#1}}
\def\rangechapref#1#2{chapters\ref{#1}--\ref{#2}}
\def\algref#1{algorithm~\ref{#1}}
\def\Algref#1{Algorithm~\ref{#1}}
\def\twoalgref#1#2{algorithms \ref{#1} and \ref{#2}}
\def\Twoalgref#1#2{Algorithms \ref{#1} and \ref{#2}}
\def\partref#1{part~\ref{#1}}
\def\Partref#1{Part~\ref{#1}}
\def\twopartref#1#2{parts \ref{#1} and \ref{#2}}

\def\ceil#1{\lceil #1 \rceil}
\def\floor#1{\lfloor #1 \rfloor}
\def\1{\bm{1}}
\newcommand{\train}{\mathcal{D}}
\newcommand{\valid}{\mathcal{D_{\mathrm{valid}}}}
\newcommand{\test}{\mathcal{D_{\mathrm{test}}}}

\def\eps{{\epsilon}}


\def\reta{{\textnormal{$\eta$}}}
\def\ra{{\textnormal{a}}}
\def\rb{{\textnormal{b}}}
\def\rc{{\textnormal{c}}}
\def\rd{{\textnormal{d}}}
\def\re{{\textnormal{e}}}
\def\rf{{\textnormal{f}}}
\def\rg{{\textnormal{g}}}
\def\rh{{\textnormal{h}}}
\def\ri{{\textnormal{i}}}
\def\rj{{\textnormal{j}}}
\def\rk{{\textnormal{k}}}
\def\rl{{\textnormal{l}}}
\def\rn{{\textnormal{n}}}
\def\ro{{\textnormal{o}}}
\def\rp{{\textnormal{p}}}
\def\rq{{\textnormal{q}}}
\def\rr{{\textnormal{r}}}
\def\rs{{\textnormal{s}}}
\def\rt{{\textnormal{t}}}
\def\ru{{\textnormal{u}}}
\def\rv{{\textnormal{v}}}
\def\rw{{\textnormal{w}}}
\def\rx{{\textnormal{x}}}
\def\ry{{\textnormal{y}}}
\def\rz{{\textnormal{z}}}

\def\rvepsilon{{\mathbf{\epsilon}}}
\def\rvtheta{{\mathbf{\theta}}}
\def\rva{{\mathbf{a}}}
\def\rvb{{\mathbf{b}}}
\def\rvc{{\mathbf{c}}}
\def\rvd{{\mathbf{d}}}
\def\rve{{\mathbf{e}}}
\def\rvf{{\mathbf{f}}}
\def\rvg{{\mathbf{g}}}
\def\rvh{{\mathbf{h}}}
\def\rvu{{\mathbf{i}}}
\def\rvj{{\mathbf{j}}}
\def\rvk{{\mathbf{k}}}
\def\rvl{{\mathbf{l}}}
\def\rvm{{\mathbf{m}}}
\def\rvn{{\mathbf{n}}}
\def\rvo{{\mathbf{o}}}
\def\rvp{{\mathbf{p}}}
\def\rvq{{\mathbf{q}}}
\def\rvr{{\mathbf{r}}}
\def\rvs{{\mathbf{s}}}
\def\rvt{{\mathbf{t}}}
\def\rvu{{\mathbf{u}}}
\def\rvv{{\mathbf{v}}}
\def\rvw{{\mathbf{w}}}
\def\rvx{{\mathbf{x}}}
\def\rvy{{\mathbf{y}}}
\def\rvz{{\mathbf{z}}}

\def\erva{{\textnormal{a}}}
\def\ervb{{\textnormal{b}}}
\def\ervc{{\textnormal{c}}}
\def\ervd{{\textnormal{d}}}
\def\erve{{\textnormal{e}}}
\def\ervf{{\textnormal{f}}}
\def\ervg{{\textnormal{g}}}
\def\ervh{{\textnormal{h}}}
\def\ervi{{\textnormal{i}}}
\def\ervj{{\textnormal{j}}}
\def\ervk{{\textnormal{k}}}
\def\ervl{{\textnormal{l}}}
\def\ervm{{\textnormal{m}}}
\def\ervn{{\textnormal{n}}}
\def\ervo{{\textnormal{o}}}
\def\ervp{{\textnormal{p}}}
\def\ervq{{\textnormal{q}}}
\def\ervr{{\textnormal{r}}}
\def\ervs{{\textnormal{s}}}
\def\ervt{{\textnormal{t}}}
\def\ervu{{\textnormal{u}}}
\def\ervv{{\textnormal{v}}}
\def\ervw{{\textnormal{w}}}
\def\ervx{{\textnormal{x}}}
\def\ervy{{\textnormal{y}}}
\def\ervz{{\textnormal{z}}}

\def\rmA{{\mathbf{A}}}
\def\rmB{{\mathbf{B}}}
\def\rmC{{\mathbf{C}}}
\def\rmD{{\mathbf{D}}}
\def\rmE{{\mathbf{E}}}
\def\rmF{{\mathbf{F}}}
\def\rmG{{\mathbf{G}}}
\def\rmH{{\mathbf{H}}}
\def\rmI{{\mathbf{I}}}
\def\rmJ{{\mathbf{J}}}
\def\rmK{{\mathbf{K}}}
\def\rmL{{\mathbf{L}}}
\def\rmM{{\mathbf{M}}}
\def\rmN{{\mathbf{N}}}
\def\rmO{{\mathbf{O}}}
\def\rmP{{\mathbf{P}}}
\def\rmQ{{\mathbf{Q}}}
\def\rmR{{\mathbf{R}}}
\def\rmS{{\mathbf{S}}}
\def\rmT{{\mathbf{T}}}
\def\rmU{{\mathbf{U}}}
\def\rmV{{\mathbf{V}}}
\def\rmW{{\mathbf{W}}}
\def\rmX{{\mathbf{X}}}
\def\rmY{{\mathbf{Y}}}
\def\rmZ{{\mathbf{Z}}}

\def\ermA{{\textnormal{A}}}
\def\ermB{{\textnormal{B}}}
\def\ermC{{\textnormal{C}}}
\def\ermD{{\textnormal{D}}}
\def\ermE{{\textnormal{E}}}
\def\ermF{{\textnormal{F}}}
\def\ermG{{\textnormal{G}}}
\def\ermH{{\textnormal{H}}}
\def\ermI{{\textnormal{I}}}
\def\ermJ{{\textnormal{J}}}
\def\ermK{{\textnormal{K}}}
\def\ermL{{\textnormal{L}}}
\def\ermM{{\textnormal{M}}}
\def\ermN{{\textnormal{N}}}
\def\ermO{{\textnormal{O}}}
\def\ermP{{\textnormal{P}}}
\def\ermQ{{\textnormal{Q}}}
\def\ermR{{\textnormal{R}}}
\def\ermS{{\textnormal{S}}}
\def\ermT{{\textnormal{T}}}
\def\ermU{{\textnormal{U}}}
\def\ermV{{\textnormal{V}}}
\def\ermW{{\textnormal{W}}}
\def\ermX{{\textnormal{X}}}
\def\ermY{{\textnormal{Y}}}
\def\ermZ{{\textnormal{Z}}}

\def\vzero{{\bm{0}}}
\def\vone{{\bm{1}}}
\def\vmu{{\bm{\mu}}}
\def\vtheta{{\bm{\theta}}}
\def\va{{\bm{a}}}
\def\vb{{\bm{b}}}
\def\vc{{\bm{c}}}
\def\vd{{\bm{d}}}
\def\ve{{\bm{e}}}
\def\vf{{\bm{f}}}
\def\vg{{\bm{g}}}
\def\vh{{\bm{h}}}
\def\vi{{\bm{i}}}
\def\vj{{\bm{j}}}
\def\vk{{\bm{k}}}
\def\vl{{\bm{l}}}
\def\vm{{\bm{m}}}
\def\vn{{\bm{n}}}
\def\vo{{\bm{o}}}
\def\vp{{\bm{p}}}
\def\vq{{\bm{q}}}
\def\vr{{\bm{r}}}
\def\vs{{\bm{s}}}
\def\vt{{\bm{t}}}
\def\vu{{\bm{u}}}
\def\vv{{\bm{v}}}
\def\vw{{\bm{w}}}
\def\vx{{\bm{x}}}
\def\vy{{\bm{y}}}
\def\vz{{\bm{z}}}
\def\vxbar{\bar{\bm{x}}}

\def\evalpha{{\alpha}}
\def\evbeta{{\beta}}
\def\evepsilon{{\epsilon}}
\def\evlambda{{\lambda}}
\def\evomega{{\omega}}
\def\evmu{{\mu}}
\def\evpsi{{\psi}}
\def\evsigma{{\sigma}}
\def\evtheta{{\theta}}
\def\eva{{a}}
\def\evb{{b}}
\def\evc{{c}}
\def\evd{{d}}
\def\eve{{e}}
\def\evf{{f}}
\def\evg{{g}}
\def\evh{{h}}
\def\evi{{i}}
\def\evj{{j}}
\def\evk{{k}}
\def\evl{{l}}
\def\evm{{m}}
\def\evn{{n}}
\def\evo{{o}}
\def\evp{{p}}
\def\evq{{q}}
\def\evr{{r}}
\def\evs{{s}}
\def\evt{{t}}
\def\evu{{u}}
\def\evv{{v}}
\def\evw{{w}}
\def\evx{{x}}
\def\evy{{y}}
\def\evz{{z}}

\def\mA{{\bm{A}}}
\def\mB{{\bm{B}}}
\def\mC{{\bm{C}}}
\def\mD{{\bm{D}}}
\def\mE{{\bm{E}}}
\def\mF{{\bm{F}}}
\def\mG{{\bm{G}}}
\def\mH{{\bm{H}}}
\def\mI{{\bm{I}}}
\def\mJ{{\bm{J}}}
\def\mK{{\bm{K}}}
\def\mL{{\bm{L}}}
\def\mM{{\bm{M}}}
\def\mN{{\bm{N}}}
\def\mO{{\bm{O}}}
\def\mP{{\bm{P}}}
\def\mQ{{\bm{Q}}}
\def\mR{{\bm{R}}}
\def\mS{{\bm{S}}}
\def\mT{{\bm{T}}}
\def\mU{{\bm{U}}}
\def\mV{{\bm{V}}}
\def\mW{{\bm{W}}}
\def\mX{{\bm{X}}}
\def\mY{{\bm{Y}}}
\def\mZ{{\bm{Z}}}
\def\mBeta{{\bm{\beta}}}
\def\mPhi{{\bm{\Phi}}}
\def\mLambda{{\bm{\Lambda}}}
\def\mSigma{{\bm{\Sigma}}}
\def\msigma{{\bm{\sigma}}}

\DeclareMathAlphabet{\mathsfit}{\encodingdefault}{\sfdefault}{m}{sl}
\SetMathAlphabet{\mathsfit}{bold}{\encodingdefault}{\sfdefault}{bx}{n}
\newcommand{\tens}[1]{\bm{\mathsfit{#1}}}
\def\tA{{\tens{A}}}
\def\tB{{\tens{B}}}
\def\tC{{\tens{C}}}
\def\tD{{\tens{D}}}
\def\tE{{\tens{E}}}
\def\tF{{\tens{F}}}
\def\tG{{\tens{G}}}
\def\tH{{\tens{H}}}
\def\tI{{\tens{I}}}
\def\tJ{{\tens{J}}}
\def\tK{{\tens{K}}}
\def\tL{{\tens{L}}}
\def\tM{{\tens{M}}}
\def\tN{{\tens{N}}}
\def\tO{{\tens{O}}}
\def\tP{{\tens{P}}}
\def\tQ{{\tens{Q}}}
\def\tR{{\tens{R}}}
\def\tS{{\tens{S}}}
\def\tT{{\tens{T}}}
\def\tU{{\tens{U}}}
\def\tV{{\tens{V}}}
\def\tW{{\tens{W}}}
\def\tX{{\tens{X}}}
\def\tY{{\tens{Y}}}
\def\tZ{{\tens{Z}}}


\def\gA{{\mathcal{A}}}
\def\gB{{\mathcal{B}}}
\def\gC{{\mathcal{C}}}
\def\gD{{\mathcal{D}}}
\def\gE{{\mathcal{E}}}
\def\gF{{\mathcal{F}}}
\def\gG{{\mathcal{G}}}
\def\gH{{\mathcal{H}}}
\def\gI{{\mathcal{I}}}
\def\gJ{{\mathcal{J}}}
\def\gK{{\mathcal{K}}}
\def\gL{{\mathcal{L}}}
\def\gM{{\mathcal{M}}}
\def\gN{{\mathcal{N}}}
\def\gO{{\mathcal{O}}}
\def\gP{{\mathcal{P}}}
\def\gQ{{\mathcal{Q}}}
\def\gR{{\mathcal{R}}}
\def\gS{{\mathcal{S}}}
\def\gT{{\mathcal{T}}}
\def\gU{{\mathcal{U}}}
\def\gV{{\mathcal{V}}}
\def\gW{{\mathcal{W}}}
\def\gX{{\mathcal{X}}}
\def\gY{{\mathcal{Y}}}
\def\gZ{{\mathcal{Z}}}

\def\sA{{\mathbb{A}}}
\def\sB{{\mathbb{B}}}
\def\sC{{\mathbb{C}}}
\def\sD{{\mathbb{D}}}
\def\sF{{\mathbb{F}}}
\def\sG{{\mathbb{G}}}
\def\sH{{\mathbb{H}}}
\def\sI{{\mathbb{I}}}
\def\sJ{{\mathbb{J}}}
\def\sK{{\mathbb{K}}}
\def\sL{{\mathbb{L}}}
\def\sM{{\mathbb{M}}}
\def\sN{{\mathbb{N}}}
\def\sO{{\mathbb{O}}}
\def\sP{{\mathbb{P}}}
\def\sQ{{\mathbb{Q}}}
\def\sR{{\mathbb{R}}}
\def\sS{{\mathbb{S}}}
\def\sT{{\mathbb{T}}}
\def\sU{{\mathbb{U}}}
\def\sV{{\mathbb{V}}}
\def\sW{{\mathbb{W}}}
\def\sX{{\mathbb{X}}}
\def\sY{{\mathbb{Y}}}
\def\sZ{{\mathbb{Z}}}

\def\emLambda{{\Lambda}}
\def\emA{{A}}
\def\emB{{B}}
\def\emC{{C}}
\def\emD{{D}}
\def\emE{{E}}
\def\emF{{F}}
\def\emG{{G}}
\def\emH{{H}}
\def\emI{{I}}
\def\emJ{{J}}
\def\emK{{K}}
\def\emL{{L}}
\def\emM{{M}}
\def\emN{{N}}
\def\emO{{O}}
\def\emP{{P}}
\def\emQ{{Q}}
\def\emR{{R}}
\def\emS{{S}}
\def\emT{{T}}
\def\emU{{U}}
\def\emV{{V}}
\def\emW{{W}}
\def\emX{{X}}
\def\emY{{Y}}
\def\emZ{{Z}}
\def\emSigma{{\Sigma}}

\newcommand{\etens}[1]{\mathsfit{#1}}
\def\etLambda{{\etens{\Lambda}}}
\def\etA{{\etens{A}}}
\def\etB{{\etens{B}}}
\def\etC{{\etens{C}}}
\def\etD{{\etens{D}}}
\def\etE{{\etens{E}}}
\def\etF{{\etens{F}}}
\def\etG{{\etens{G}}}
\def\etH{{\etens{H}}}
\def\etI{{\etens{I}}}
\def\etJ{{\etens{J}}}
\def\etK{{\etens{K}}}
\def\etL{{\etens{L}}}
\def\etM{{\etens{M}}}
\def\etN{{\etens{N}}}
\def\etO{{\etens{O}}}
\def\etP{{\etens{P}}}
\def\etQ{{\etens{Q}}}
\def\etR{{\etens{R}}}
\def\etS{{\etens{S}}}
\def\etT{{\etens{T}}}
\def\etU{{\etens{U}}}
\def\etV{{\etens{V}}}
\def\etW{{\etens{W}}}
\def\etX{{\etens{X}}}
\def\etY{{\etens{Y}}}
\def\etZ{{\etens{Z}}}

\newcommand{\pmodel}{p_{\rm{model}}}
\newcommand{\Pmodel}{P_{\rm{model}}}
\newcommand{\ptildemodel}{\tilde{p}_{\rm{model}}}
\newcommand{\pencode}{p_{\rm{encoder}}}
\newcommand{\pdecode}{p_{\rm{decoder}}}
\newcommand{\precons}{p_{\rm{reconstruct}}}

\newcommand{\laplace}{\mathrm{Laplace}} %

\newcommand{\Ls}{\mathcal{L}}
\newcommand{\emp}{\tilde{p}}
\newcommand{\lr}{\alpha}
\newcommand{\reg}{\lambda}
\newcommand{\rect}{\mathrm{rectifier}}
\newcommand{\softmax}{\mathrm{softmax}}
\newcommand{\sigmoid}{\sigma}
\newcommand{\softplus}{\zeta}

\newcommand{\normlzero}{L^0}
\newcommand{\normlone}{L^1}
\newcommand{\normltwo}{L^2}
\newcommand{\normlp}{L^p}
\newcommand{\normmax}{L^\infty}

\newcommand{\parents}{Pa} %

\DeclareMathOperator{\sign}{sign}
\DeclareMathOperator{\Tr}{Tr}
\let\ab\allowbreak


\usepackage{tikz}
\usetikzlibrary{shapes,arrows,positioning,fit,backgrounds,calc,shadows}



\usepackage{amsmath}
\usepackage{graphicx}
\usepackage{pgfplots}
\pgfplotsset{compat=1.17}


\usepackage[numbers]{natbib}
\usepackage{amsmath}

\usepackage[utf8]{inputenc} %
\usepackage[T1]{fontenc}    %
\usepackage{graphicx}
\usepackage{amsmath,amssymb,amsthm,mathabx}
\usepackage{booktabs}
\usepackage{algorithm}
\usepackage[noend]{algpseudocode}
\usepackage{tikz}
\usepackage{wrapfig}
\usetikzlibrary{positioning, arrows.meta, calc}
\usepackage{acronym}
\usepackage{enumitem}
\usepackage{url}
\usepackage[
    colorlinks, 
    anchorcolor=blue, 
    citecolor=blue,
    pagebackref=true
]{hyperref}
\usepackage{setspace}
\usepackage[skip=3pt,font=small]{subcaption}
\usepackage[skip=3pt,font=small]{caption}
\usepackage{xcolor}
\usepackage[capitalise]{cleveref}
\usepackage{tabularx,colortbl,multirow,array,makecell}
\usepackage{overpic,wrapfig}

\newcommand\blfootnote[1]{\begingroup\renewcommand\thefootnote{}\footnote{#1}\addtocounter{footnote}{-1}\endgroup}


\makeatletter
\DeclareRobustCommand\onedot{\futurelet\@let@token\@onedot}
\def\@onedot{\ifx\@let@token.\else.\null\fi\xspace}
\def\eg{\emph{e.g}\onedot} 
\def\Eg{\emph{E.g}\onedot}
\def\ie{\emph{i.e}\onedot} 
\def\Ie{\emph{I.e}\onedot}
\def\cf{\emph{c.f}\onedot} 
\def\Cf{\emph{C.f}\onedot}
\def\etc{\emph{etc}\onedot} 
\def\vs{\emph{vs}\onedot}
\def\wrt{w.r.t\onedot} 
\def\dof{d.o.f\onedot}
\def\etal{\emph{et al}\onedot}
\makeatother

\newcommand{\ours}{ERDM}

\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\newcommand{\w}{\mathbf{w}}
\newcommand{\x}{\mathbf{x}}
\newcommand{\xw}{{\mathbf{x}_w}}
\newcommand{\xl}{{\mathbf{x}_l}}
\newcommand{\y}{\mathbf{y}}
\newcommand{\z}{\mathbf{z}}

\newcommand{\R}{\mathbb{R}}
\newcommand{\I}{\mathbf{I}}
\newcommand{\E}{\mathbb{E}}
\newcommand{\D}{\mathcal{D}}
\newcommand{\N}{\mathcal{N}}

\newcommand{\pref}{{p_{\rm ref}}}
\newcommand{\kl}{{\mathbb{D}_{\rm kl}}}

\RequirePackage{doi}
\definecolor{olive}{rgb}{0.5, 0.5, 0.0}
\definecolor{maroon}{rgb}{0.69, 0.19, 0.38}
\definecolor{celestialblue}{rgb}{0.29, 0.59, 0.82}
\definecolor{darkgreen}{rgb}{0.0, 0.6, 0.0}
\definecolor{grey}{rgb}{0.5,0.5,0.5}
\definecolor{darkblue}{rgb}{0.19, 0.19, 0.62}
\definecolor{silver}{rgb}{0.7,0.7,0.7}
\definecolor{darkcyan}{rgb}{0.0, 0.55, 0.55}
\definecolor{darkred}{rgb}{0.68,0.05,0.0}
\definecolor{darkpurple}{rgb}{0.47,0.09,0.29}
\definecolor{myblue}{rgb}{0.1380392156862745, 0.3027450980392157, 0.6274509803921569}
\definecolor{myred}{rgb}{0.6235294117647059, 0.13725490196078433, 0.0196078431372549}
\definecolor{lightblue}{RGB}{235,235,255}
\hypersetup{
  colorlinks = true,
  citecolor  = myblue,
  linkcolor  = myred,
  filecolor  = darkblue,
  urlcolor   = darkblue,
}
\newcommand{\TODO}[1]{{\color{red}TODO: #1}}
\newcommand{\Todo}[1]{{\color{red}TODO: #1}}
\newcommand{\TBD}[1]{{\color{red}#1}}
\newcommand{\WEAK}[1]{\color{silver}{#1}}
\newcommand{\FINAL}[1]{#1}

\def\clap#1{\hbox to 0pt{\hss #1\hss}}%
\def\initials#1{\protect\clap{\smash{\raisebox{1.4ex}{\tiny{\textsf{\textit{#1}}}}}}}%
\newcommand{\EDIT}[4][]{\strut{\color{#3}{\hspace{0pt}\initials{#2}{\color{red}\sout{#1}}{#4}}}}
\newcommand{\TK}[2][]{\protect\EDIT[#1]{TK}{olive}{#2}}
\newcommand{\TA}[2][]{\protect\EDIT[#1]{TA}{maroon}{#2}}
\newcommand{\JL}[2][]{\protect\EDIT[#1]{JL}{celestialblue}{#2}}
\newcommand{\SL}[2][]{\protect\EDIT[#1]{SL}{darkgreen}{#2}}
\newcommand{\MA}[2][]{\protect\EDIT[#1]{MA}{darkblue}{#2}}

\newcommand{\norm}[1]{\left\lVert#1\right\rVert}
\newcommand{\abs}[1]{\lvert#1\rvert}
\newcommand{\real}{\mathbb{R}}
\newcommand{\integer}{\mathbb{Z}}
\newcommand{\low}[1]{\raisebox{0pt}[0pt][0pt]{#1}}
\newcommand{\lowsqrt}[1]{\low{$\sqrt{#1}$}}

\newcommand{\bfn}{\mathbf{n}}
\newcommand{\bfx}{\mathbf{x}}
\newcommand{\bfy}{\mathbf{y}}
\newcommand{\xx}{\boldsymbol{x}}
\newcommand{\XX}{\boldsymbol{X}}
\newcommand{\xxt}{\boldsymbol{\tilde x}}
\newcommand{\xxh}{\boldsymbol{\hat x}}
\newcommand{\zz}{\boldsymbol{z}}
\newcommand{\ft}[1][t]{\boldsymbol{f}_{#1}}
\newcommand{\xt}[1][t]{\boldsymbol{x}_{#1}}   %
\newcommand{\yt}[1][t]{\boldsymbol{y}_{#1}}

\newcommand{\xth}[1][t]{\hat{\boldsymbol{x}}_{#1}}   %
\newcommand{\xtt}[1][t]{\tilde{\boldsymbol{x}}_{#1}}   %
\newcommand{\conv}{\ast}
\newcommand{\mult}{\odot}

\newcommand{\config}[1]{config~{\sc #1}}

\newcommand\undefcolumntype[1]{\expandafter\let\csname NC@find@#1\endcsname\relax}
\newcommand\forcenewcolumntype[1]{\undefcolumntype{#1}\newcolumntype{#1}}

\newcommand{\s}{\hphantom{0}}

\newcommand{\plusplus}{\raisebox{0.15ex}[0pt][0pt]{\scalebox{0.9}{++}}}
\newcommand{\ddpm}{DDPM}
\newcommand{\ddpmpp}{DDPM\plusplus}
\newcommand{\ncsnpp}{NCSN\plusplus}

\newcommand{\boldzero}{\mathbf{0}}
\newcommand{\boldi}{\mathbf{I}}
\newcommand{\sigmamax}{\sigma_{\text{max}}}
\newcommand{\scale}{s}
\newcommand{\signal}{\boldsymbol{y}}
\newcommand{\signalvar}{\boldsymbol{\hat y}^2}
\newcommand{\noisevar}{\sigma^2}
\newcommand{\wproc}{\omega_\odetime}
\newcommand{\churn}{\beta}
\newcommand{\dd}{\boldsymbol{d}}
\newcommand{\ddt}{\boldsymbol{\tilde d}}
\newcommand{\ddp}{{\boldsymbol{d}\mathrlap{\hspace*{.075em}'}}}
\newcommand{\ddh}{\boldsymbol{\hat{d}}}
\newcommand{\pdata}{p_{\rm{data}}}
\newcommand{\pdatahat}{\hat{p}_\rm{data}}
\newcommand{\lte}{\boldsymbol{\tau}}
\newcommand{\gte}{\boldsymbol{e}}
\newcommand{\ptrain}{\hat{p}_{\rm{data}}}
\newcommand{\Ptrain}{\hat{P}_{\rm{data}}}
\newcommand{\loss}{\mathcal{L}}
\newcommand{\distrand}{U([0, 1))}
\newcommand{\nablaxx}{\nabla_{\hspace{-0.5mm}\xx}}
\newcommand{\Deltaxx}{{\Delta_{\xx}}}

\newcommand{\KL}{D_{\mathrm{KL}}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\standarderror}{\mathrm{SE}}
\newcommand{\Cov}{\mathrm{Cov}}
\DeclareMathOperator{\score}{score}

\newcommand{\xhatclean}{\hat{\vy}^{\rm{clean}}}
\newcommand{\xclean}{\vy^{\rm{clean}}}
\newcommand{\xcleanw}{\xclean_w}
\newcommand{\xnoisy}{\vxbar}
\newcommand{\xnoisyw}{\vxbar_w}
\newcommand{\xin}{\vx^{\rm{in}}}
\newcommand{\sigmabar}{\bar{\sigma}}
\newcommand{\xdenoised}{\hat{\vy}}
\newcommand{\xdenoisedw}{\xdenoised_w}
\newcommand{\xcur}{\vxbar_{\rm{cur}}}
\newcommand{\xnext}{\vxbar_{\rm{next}}}
\newcommand{\tcur}{t_{\rm{cur}}}
\newcommand{\tnext}{t_{\rm{next}}}
\newcommand{\sigmainit}{\boldsymbol{\sigma}_{\rm init}}
\newcommand{\sigmacur}{\boldsymbol{\sigma}_{\rm{cur}}} %
\newcommand{\sigmanext}{\boldsymbol{\sigma}_{\rm{next}}} %

\newcommand{\stext}[1]{\text{\raisebox{0pt}[0pt][0pt]{#1}}}
\newcommand{\sdyn}{\sigma_\stext{d}}
\newcommand{\tmin}{\odetime_\stext{min}}
\newcommand{\tmax}{\odetime_\stext{max}}
\newcommand{\bmin}{\beta_\text{min}}
\newcommand{\bmax}{\beta_\text{max}}
\newcommand{\bdyn}{\beta_\text{d}}
\newcommand{\sdata}{\sigma_\text{data}}
\newcommand{\sigmaw}{\sigma_w}
\newcommand{\sigmawindow}{\boldsymbol{\sigma}}
\newcommand{\sigmaerdm}{\bar{\sigma}}
\newcommand{\sigmaerdmdot}{\dot{\sigma}^{{\text{\tiny prog}}}}
\newcommand{\sigmaerdmbold}{\boldsymbol{\bar{\sigma}}}
\newcommand{\varerdm}{\sigmaerdm^2}
\newcommand{\lambdaerdm}{\lambda_{{\text{\tiny ERDM}}}}
\newcommand{\offset}{t}
\newcommand{\offsetnext}{t'}
\newcommand{\offsetdelta}{\Delta t}
\newcommand{\nclean}{n_{\rm{clean}}}
\newcommand{\origT}[1]{u_{#1}}
\newcommand{\origTsup}[2]{u_{#1}^{#2}}
\newcommand{\origN}{M}
\newcommand{\dtime}{{t}}
\newcommand{\ttime}{\tau}
\newcommand{\ttimemax}{T_\mathrm{forecast}}
\newcommand{\Schurn}{S_\text{churn}}
\newcommand{\Stmin}{S_\text{tmin}}
\newcommand{\Stmax}{S_\text{tmax}}
\newcommand{\Snoise}{S_\text{noise}}
\newcommand{\StminStmax}{S_\text{tmin,tmax}}
\newcommand{\StminStmaxSnoise}{S_\text{tmin,tmax,noise}}
\newcommand{\nlat}{I}  %
\newcommand{\nlon}{J}  %
\newcommand{\sumij}{\sum_{i,j}}  %
\newcommand{\fracij}{\frac{1}{\nlat\nlon}}
\newcommand{\fracijens}{\frac{1}{\nens\nlat\nlon}}
\newcommand{\confhdr}[1]{\makebox[1em]{\textsc{#1}}}

\newcommand{\vparagraph}[1]{\vspace*{-1mm}\paragraph{#1}}

\newcommand{\pd}{q}
\newcommand{\condu}{\kappa}
\newcommand{\freq}{\boldsymbol{\nu}}
\newcommand{\fp}{r}
\newcommand{\ff}{\boldsymbol{f}}
\newcommand{\gb}{\boldsymbol{g}}
\newcommand{\Fargs}{\scalebox{0.85}{$\cin(\sigma) (\signal {+} \noise); \cnoise(\sigma)$}}

\pgfplotsset{xtick style={draw=none}}
\pgfplotsset{ytick style={draw=none}}
\pgfplotsset{major grid style={gray!40}}
\pgfplotsset{every axis plot/.style={thick, mark size=1.5pt}}
\pgfplotsset{legend image code/.code={\draw[mark repeat=2, mark phase=2] plot coordinates {(0cm, 0cm) (0.2cm, 0cm) (0.4cm, 0cm)};}} %

\definecolor{C0}{rgb}{0.121569, 0.466667, 0.705882}
\definecolor{C1}{rgb}{1.000000, 0.498039, 0.054902}
\definecolor{C2}{rgb}{0.172549, 0.627451, 0.172549}
\definecolor{C3}{rgb}{0.839216, 0.152941, 0.156863}
\definecolor{C4}{rgb}{0.580392, 0.403922, 0.741176}
\definecolor{C5}{rgb}{0.549020, 0.337255, 0.294118}
\definecolor{C6}{rgb}{0.890196, 0.466667, 0.760784}
\definecolor{C7}{rgb}{0.498039, 0.498039, 0.498039}
\definecolor{C8}{rgb}{0.737255, 0.741176, 0.133333}
\definecolor{C9}{rgb}{0.090196, 0.745098, 0.811765}

\newcommand{\fillbetween}[3][]{\addplot+[name path=A, draw=none, mark=none, forget plot] #2; \addplot+[name path=B, draw=none, mark=none, forget plot] #3; \addplot[#1] fill between[of=A and B]}

\newcommand{\vlabel}[3]{\makebox[0mm][l]{\rotatebox{90}{\makebox[#2][c]{#3}}}\hspace{#1}}
\newcommand{\hrlabel}[1]{\hfill\makebox[0mm]{#1}\hfill}
\newcommand{\vrlabel}[1]{\vfill\makebox[0mm]{#1}\vfill}

\newcommand{\tickYtop}[1]{\raisebox{0ex}[1ex][1ex]{#1}}
\newcommand{\tickYtopD}[1]{\raisebox{-1.5ex}[0ex][0ex]{#1}}
\newcommand{\tickFID}{\tickYtopD{FID}}
\newcommand{\tickLoss}{\tickYtopD{loss}}
\newcommand{\tickTau}{\tickYtopD{$\lVert\lte\rVert$}}
\newcommand{\tickNFE}[1]{\smash{NFE$=$}{$#1$}\hspace*{1em}}
\newcommand{\tickSchurn}[1]{$\mathllap{\smash{\Schurn{=}}}{#1}$}
\newcommand{\tickSchurnB}[1]{\smash{$\Schurn{=}$}{$#1$}\hspace*{2em}}
\newcommand{\tickRho}[1]{$\mathllap{\smash{\rho{=}}}{#1}$}
\newcommand{\tickSigma}[1]{$\mathllap{\smash{\sigma{=}}}{#1}$}

\newcommand{\atphantom}{\vphantom{${}^2$}}
\newcommand{\AProcedure}[2]{\Procedure{\smash{#1}}{\smash{#2}}}
\newcommand{\AComment}[1]{\Comment{\smash{#1}}}
\newcommand{\AState}[1]{\State{\smash{#1}}}
\newcommand{\AFor}[1]{\For{\smash{#1}}}
\newcommand{\AIf}[1]{\If{\smash{#1}}}
\newcommand{\DState}[1]{\State{#1 \vphantom{$\displaystyle\Bigg)$}}}
\newcommand{\MState}[1]{\State\raisebox{0mm}[3.2ex][1.8ex]{#1}}


\begin{figure}
    \centering
    \includegraphics[width=1\linewidth]{figures/raw/diagram-png.png}
    \caption{\small
    ERDM sampling with a window, highlighted in bold purple, of size $W=4$.
    \textbf{Top row:} \ours{} starts at diffusion time $\dtime=0$ with snapshots, $\vx_1, \dots, \vx_W$, corrupted by progressively larger noise levels, $\sigmabar_1(\offset)<\dots<\sigmabar_W(\offset)=\smax$.
    \textbf{Middle row:} After $N=2$ joint denoising steps, the sequence reaches lower noise levels at $\dtime=1$ such that $\sigmabar_1(1)=\smin$ and
    $\sigmabar_{w}(1) = \sigmabar_{w-1}(0)$ for $w>1$, as illustrated in the right-hand panel. The now fully denoised first snapshot, $\vx_1$, is returned.
    \textbf{Bottom row:} The rest of the sequence is shifted one slot to the right, and a fresh pure-noise snapshot is appended to the new window. The cycle then repeats. 
    }
    \label{fig:diagram}
\end{figure}

\begin{wrapfigure}{r}{0.439\textwidth} %
    \vspace{-3mm}
  \centering %
  \includegraphics[width=\linewidth]{figures/raw/fig_noise_schedule.png} 
  \caption{\small 
  Noise schedule comparison for a sequence length $W=6$, yielding 6 visualized segments $[\sigmaerdm_w(1), \sigmaerdm_w(0)]$. The schedules differ by their curvature parameter $\rho$: one using $\rho=7$ (default EDM) and the other $\rho=-10$ (\ours{}). Color gradients illustrate segment progression. 
  }
  \label{fig:noise_schedule}
\end{wrapfigure}

\begin{algorithm} %
    \caption{Elucidated Rolling Diffusion: Training}
    \begin{algorithmic}[1]
        \State {\bfseries Require:} Training data $\mathcal{D}_\mathrm{train}$, network $F_\theta$, $\smin, \smax, \rho, \Pmean, \Pstd$
        \Repeat
        \State Sample $\vy = (\mathbf{y}_1,\dots,\mathbf{y}_W)\in \R^{W \times D}$ from $\mathcal{D}_\mathrm{train}, \noise\sim \mathcal{N}(\boldzero, \boldi_{W \times D}), \offset \sim U([0, 1))$ 
        \State $\sigmawindow = (\sigma_{1}, \dots, \sigma_{W}) \gets \sigmaerdmbold(\offset)$
        \State $\xnoisy \gets \vy + \sigmawindow \cdot \noise$
        \Comment{Add rolling noise to data}
        \State $\xdenoised \gets \cskip(\sigmawindow) \xnoisy + \cout(\sigmawindow)  F_\theta(\cin(\sigmawindow) \xnoisy, \cnoise(\sigmawindow))$
        \State Update $\theta$ using $L_{\theta} = \frac{1}{W}\sum_{w=1}^W \mathbf{\lambda}(\sigmaw) f(\sigmaw; \Pmean, \Pstd) \lVert \vy_w - \xdenoisedw \rVert^2_2$
        \Until{Converged}
\end{algorithmic}
\label{alg:training}
\end{algorithm}



\begin{wrapfigure}{r}{0.4\textwidth}
\vspace{-10mm}
\centering
\begin{tikzpicture}[
    font=\sffamily\scriptsize, %
    node distance=0.4cm and 0.4cm,  %
    every node/.style={align=center},
    block/.style={draw, rectangle, minimum height=1cm, minimum width=2.2cm},
    smallblock/.style={draw, circle, minimum size=0.1cm},
    smallblock2/.style={draw, circle, minimum size=0.3cm},
    arrow/.style={->, line width=0.6pt, >=Stealth, shorten >=1pt, shorten <=1pt, scale=0.8}
]

\node (input) {$\bar{\bm{x}}_{1:W}$ \\ \texttt{(b, c, t, h, w)}};
\node[smallblock, below=of input] (reshape1) {\scriptsize Reshape};
\node[block, below=of reshape1] (unet) {2D U-Net Block \\ \texttt{(b*t, c, h, w)}};
\node[smallblock, below=of unet] (reshape2) {\scriptsize Reshape};
\node[block, below=of reshape2] (attention) {Causal Temporal \\ Attention Block \\ \texttt{(b*h*w, c, t)}};

\node[right=0.8cm of reshape1] (embed1) {Noise Embedding \\ \texttt{(b, c, t)}}; 
\node[smallblock2, below=of embed1] (embed2) {\scriptsize Reshape \\ \scriptsize\texttt{(b*t, c)}}; 

\draw[arrow] (input) -- (reshape1);
\draw[arrow] (reshape1) -- (unet);
\draw[arrow] (unet) -- (reshape2);
\draw[arrow] (reshape2) -- (attention);

\draw[arrow] (embed1) -- (embed2); 
\draw[arrow] (embed2.west) -- ++(-0.3,0) |- (unet.east); %

\draw[arrow] (embed1.south) -- ++(0,-0.14cm) -- ++(1.1cm,0) |- (attention.east);

\end{tikzpicture}
\caption{Sketch of one 2D U-Net block and one temporal attention block in our hybrid U-Net topology with noise embedding to both spatial and temporal paths. Dimensions \texttt{b, c, t, h, w}\; refer to batch, channel, window, height, and width. For simplicity, \texttt{c, h, w}\; do not change in the sketch.
}
\label{fig:arch_sketch}
\vspace{-2mm}
\end{wrapfigure}

\begin{algorithm}[t]
\footnotesize
\caption{\atphantom\ Elucidated Rolling Diffusion Deterministic Sampler (Euler-only)}
\begin{algorithmic}[1]
\State {\bfseries Require:} $\hat{\vy}_{1:W}, N, \ttimemax$
    \State $\offsetdelta \gets 1 / N$ \Comment{Infer step size from desired number of steps per snapshot}
    \State $\tcur \gets 0;\;\mathcal{S} \gets \emptyset$  \Comment{Initialize global diffusion time and empty generated sequence}
    \State {\bf sample} $\xcur \sim \mathcal{N}(\hat{\vy}_{1:W}, \sigmaerdmbold(\tcur)^2\boldi_{W \times D})$
    \Comment{Initialize window with snapshot-dependent rolling noise}
    \While{$\lvert\mathcal{S}\rvert < \ttimemax$} \Comment{Predict snapshot $\lvert\mathcal{S}\rvert$+1}
        \State $\tnext \gets \tcur + \offsetdelta$  \Comment{Global diffusion time after denoising}
        \State $\sigmacur \gets \sigmaerdmbold(\tcur)$ \Comment{Current noise levels}
        \State $\sigmanext \gets \sigmaerdmbold(\tnext)$ \Comment{Noise levels at the end of this iteration}
        \State $\xdenoised \gets \denoiser(\xcur, \sigmacur)$ \Comment{Denoise sequence}
        \State $\dd \gets (\xcur - \xdenoised) / \sigmacur$ \Comment{Evaluate $\diff\vxbar / \diff\dtime$ at $\tcur$}
        \State $\xnext \gets \xcur + (\sigmanext-\sigmacur)\dd$  \Comment{Euler step from $\tcur$ to $\tnext$}
        \State $\nclean \gets \lfloor \tnext \rfloor$ \Comment{Infer finished snapshots. If $0$, the following lines are a no-op}
        \State \textbf{add} first $\nclean$ snapshots of $\xdenoised$ to $\mathcal{S}$ and discard first $\nclean$ snapshots of $\xnext$
        \State {\bf sample} $\vx_\mathrm{new} \sim \mathcal{N}(\boldzero, \sigmamax^2\boldi_{\nclean \times D})$
        \State $\xcur \gets [\xnext, \vx_\mathrm{new}]$ \Comment{Concatenate fresh noisy snapshots to futuremost}
        \State $\tcur \gets \tnext - \nclean$  \Comment{Re-adjust global diffusion `time' to be in $[0, 1)$}
    \EndWhile
    \State \Return \(\mathcal{S}\)
\end{algorithmic}
\label{alg:sampling_simple}
\end{algorithm}

\begin{wrapfigure}{r}{0.415\textwidth}
    \centering
    \vspace{-8mm}
    \includegraphics[width=\linewidth]{figures/raw/spectra-main-3only.pdf}
    \caption{ 
    Normalized spectral density of 14-day forecasts, averaged over high latitudes, $[60^\circ, 90^\circ]$. Spectra are divided by the target ERA5 reanalysis spectra.
    \ours{} generates highly accurate spectra that match or slightly beat the physics-based model IFS ENS. NeuralGCM ENS underestimates energy at the mid to high frequencies. See \cref{sec:power_spectra_app} for more variables and absolute spectra.
    }
    \vspace{-9mm}
    \label{fig:era5_spectra}
\end{wrapfigure}


\addtocontents{toc}{\protect\setcounter{tocdepth}{2}}
\newpage
\appendix
\section*{Appendix}

\tableofcontents
\section{Broader Impact}
\label{sec:broader_impact}
Our research holds the potential for societal benefits, primarily through improved disaster preparedness, enhanced decision-making across sectors like agriculture or energy. However, these advancements also necessitate careful consideration of potential negative impacts, such as over-reliance on ML-based forecasts or socioeconomic disparities in access and benefit. 
Mitigating these risks requires comprehensive evaluations, communication about forecast uncertainties, and efforts to ensure equitable access to these new tools.
Any deployment of our model, or derived versions, should be accompanied by either more comprehensive evaluations than in our paper, or careful disclaimers to not blindly trust its forecasts, especially for data outside of the training data distribution.

\section{Background on Diffusion Models}
\label{sec:background_app}
Consider the data distribution represented by \(p_{{\rm data}}({\vx})\). The forward diffusion process transforms this distribution by incorporating time-dependent Gaussian noise, yielding modified distributions \(p({\vx}; \mathbf{G}(t))\). This transformation is achieved by adding Gaussian noise with zero mean and covariance \(\mathbf{G}(t)\) to the data. \(\mathbf{G}(t)\) is a \(d \times d\) positive definite noise covariance matrix that evolves over diffusion time \(t\). When the ``magnitude'' of \(\mathbf{G}(t)\) (e.g., its trace or smallest eigenvalue) is sufficiently large, the resulting distribution \(p(\vx; \mathbf{G}(t))\) approximates a pure Gaussian distribution $\mathcal{N}(\mathbf{0}, \mathbf{G}(t))$.

Conversely, the backward diffusion process operates by initially sampling noise, represented as \({\vx}_0\), from a prior distribution, typically \(\mathcal{N}(\mathbf{0}, \mathbf{G}(0))\), where \(\mathbf{G}(0)\) corresponds to the maximum noise level. The process then focuses on denoising this sample through a sequence, \({\vx}_i\), characterized by a sequence of decreasing noise covariance matrices: \(\mathbf{G}(t_0) \succ \mathbf{G}(t_1) \succ \ldots \succ \mathbf{G}(t_N) \approx \mathbf{0}\) (in the Loewner order), where $t_0 = 0$. Each intermediate sample \({\vx}_i\) is drawn from \({\vx}_i \sim p({\vx}_i; \mathbf{G}(t_i))\). The terminal sample of the backward process, \({\vx}_N\), is expected to approximate the original data distribution \(p_{{\rm data}}({\vy})\). Note that for the sake of consistency with our ERDM formulation, we reversed the diffusion time order. That is, $t=0$ ($1$) corresponds to maximal (minimal) noise levels, which is sometimes described in opposite order in the diffusion literature.

\noindent\textbf{SDE formulation}. To present the forward and backward processes rigorously, they can be captured via stochastic differential equations (SDEs). Such SDEs ensure that the sample, \(\vx(t)\), aligns with the designated data distribution, \(p(\vx; \mathbf{G}(t))\), as it evolves through diffusion time \(t\) \cite{song2021scoreSDE}.
The Elucidated Diffusion Model (EDM) by \citet{karras2022edm} provides a principled design for diffusion models, typically based on a scalar noise schedule \(\sigma(t)\) (leading to isotropic noise \(\sigma^2(t)\mathbf{I}\)). The formulation presented here extends these principles to accommodate a general, potentially anisotropic, time-varying noise covariance matrix \(\mathbf{G}(t)\).
Let \(\dot{\mathbf{G}}(t) = d\mathbf{G}(t)/dt\) be the time derivative of the noise covariance matrix. We assume \(\dot{\mathbf{G}}(t)\) is positive semi-definite for all \(t\), and let \(\mathbf{K}(t)\) be any matrix such that \(\mathbf{K}(t)\mathbf{K}(t)^T = \dot{\mathbf{G}}(t)\) (e.g., via Cholesky decomposition if \(\dot{\mathbf{G}}(t)\) is strictly positive definite, or other matrix square roots).
The forward SDE, describing the process of adding noise such that \(\vx(t) - \vx(1) \sim \mathcal{N}(\mathbf{0}, \mathbf{G}(t))\) (assuming \(\mathbf{G}(1)=\mathbf{0},\) and \(\vx(1) \sim p_{\text{data}}(\vy)\)), is:
\begin{align}
    d{\vx} = \mathbf{K}(t) d \pmb{\omega}(t),    
\end{align}
where \(\pmb{\omega}(t)\) is a standard \(d\)-dimensional Wiener process.
The corresponding backward (reverse-time) SDE is given by \cite{song2021scoreSDE}:
\begin{align}
    d{\vx} = - \dot{\mathbf{G}}(t) \nabla_{{\vx}} \log p({\vx};\mathbf{G}(t)) dt + \mathbf{K}(t) d \bar{\pmb{\omega}}(t),    \label{eq:backward_sde_correlated_general}
\end{align}
where \(\bar{\pmb{\omega}}(t)\) is a standard Wiener process in reverse time. The term `time` \(t\) here is a conceptual dimension for the denoising steps. The backward SDE comprises a deterministic drift term related to the score of the perturbed data distribution and a stochastic noise injection term.

\noindent\textbf{Denoising score matching}. An examination of the SDE in~\eqref{eq:backward_sde_correlated_general} indicates the necessity of the score function, \(\nabla_{\vx} \log p(\vx;\mathbf{G}(t))\), for sampling. This score function can be estimated using the relationship derived from Tweedie's formula. Given that \(\vx = \vy + \noise\) where \(\vy \sim p_{\text{data}}(\vy)\) and \(\noise \sim \mathcal{N}(\mathbf{0}, \mathbf{G}(t))\), the score is:
\begin{align}
    \nabla_{\vx} \log p(\vx;\mathbf{G}(t)) = (\mathbf{G}(t))^{-1} (\E[\vy | \vx, \mathbf{G}(t)] - \vx).
\end{align}
A denoising neural network, \(\denoiser(\vx; \mathbf{G}(t))\), is trained to approximate the conditional expectation \(\E[\vy | \vx, \mathbf{G}(t)] \approx \vy\). Thus, the score is approximated as \((\mathbf{G}(t))^{-1} (\denoiser(\vx; \mathbf{G}(t)) - \vx)\). The network is trained by minimizing the following objective, typically by sampling \(t\) (and thus \(\mathbf{G}(t)\)) according to a predefined schedule or distribution \(p_t(t)\) over diffusion times:
\begin{align}
    \min_{\theta} \E_{\vy \sim p_{{\rm data}}} \E_{t \sim p_t(t)} \E_{\noise \sim \mathcal{N}(\boldzero, \mathbf{G}(t))} \big[ \lVert\denoiser( \vy + \noise; \mathbf{G}(t)) - \vy\rVert^2_2  \big]. \label{eq:score_matching_correlated_general}
\end{align}
The denoiser \(\denoiser\) is now conditioned on the full noise covariance matrix \(\mathbf{G}(t)\).


\noindent\textbf{Sampling}. To generate samples from the model, one typically discretizes the backward SDE~\eqref{eq:backward_sde_correlated_general} and simulates it from \(t=0\) to \(t \approx 1\). A common discretization is the Euler-Maruyama method. Given a sequence of diffusion times \(0 = \tau_0 < \tau_1 < \ldots < \tau_N \approx 1\), and an initial sample \(\vx_{\tau_0} \sim \mathcal{N}(\mathbf{0}, \mathbf{G}(0))\), the update rule for \(i = 0, \ldots, N-1\) is:
\begin{align}
    \vx_{\tau_{i+1}} = \vx_{\tau_i} - \left( \mathbf{G}(\tau_i) - \mathbf{G}(\tau_{i+1}) \right) \hat{\nabla}_{{\vx}} \log p(\vx_{\tau_i};\mathbf{G}(\tau_i)) + \sqrt{\mathbf{G}(\tau_i) - \mathbf{G}(\tau_{i+1})} \mathbf{z}_i, \label{eq:euler_maruyama_sampling_general}
\end{align}
where \(\Delta \mathbf{G}(\tau_i) = \mathbf{G}(\tau_i) - \mathbf{G}(\tau_{i+1}) > \boldzero\), we have used the approximation \(\dot{\mathbf{G}}(\tau_i) \Delta \tau_i \approx \mathbf{G}(\tau_i) - \mathbf{G}(\tau_{i+1})\) (assuming $\Delta\mathbf{G}(\tau_i)$ is small), and \(\mathbf{z}_i \sim \mathcal{N}(\mathbf{0}, \mathbf{I})\) is a standard Gaussian noise sample. The term \(\hat{\nabla}_{{\vx}} \log p(\vx_{\tau_i};\mathbf{G}(\tau_i))\) is the score estimated using the trained denoiser:
\begin{align}
    \hat{\nabla}_{{\vx}} \log p(\vx_{\tau_i};\mathbf{G}(\tau_i)) = (\mathbf{G}(\tau_i))^{-1} (\denoiser(\vx_{\tau_i}; \mathbf{G}(\tau_i)) - \vx_{\tau_i}).
\end{align}
More sophisticated samplers, such as those proposed by \citet{karras2022edm} (e.g., second-order Heun or DPM-Solver++), can also be adapted for the general covariance case to improve sample quality or reduce the number of sampling steps \(N\). These often involve more complex update rules, potentially incorporating predictor-corrector steps or higher-order approximations of the SDE.
\paragraph{EDM preconditioning.}
Recall the denoiser network parameterization in EDM:
\begin{align}
    \denoiser(\vx; \sigma) = \cskip(\sigma)\vx + \cout(\sigma)F_\theta(\cin(\sigma)\vx, \cnoise(\sigma)),
\end{align}
where $\vx$ corresponds to data corrupted with Gaussian noise of standard deviation $\sigma$ and $F_\theta$ is the raw neural network trained to denoise $\vx$.
One of the key contributions in EDM is the careful choice of preconditioning ($\cskip, \cout, \cin$).
The preconditioning ensures unit variance for the denoiser network's inputs and targets. Consistent scales, regardless of noise levels, simplify the task for the denoiser network, which does not need to adapt to changing magnitudes for different noise standard deviations. Mathematically, these functions are defined as:
\begin{equation}
    \cin(\sigma) = {\frac{1}{\sqrt{\sigma^2 + \sdata^2}}} 
    \quad\cskip(\sigma) = {\frac{\sdata^2}{\sigma^2 + \sdata^2}} 
    \quad\cout(\sigma) = {\frac{\sigma \cdot \sdata}{\sqrt{\sigma^2 + \sdata^2}}}
\end{equation}
The transformation of the noise level is chosen empirically as $\cnoise(\sigma)=\ln(\sigma)/4$.
In ERDM, we vectorize the preconditioning to function on each snapshot independently, thus ensuring that the same EDM design principles apply to the noisy sequence that ERDM's denoiser network ingests. 
\section{\ours}
\label{sec:erdm_app}
\subsection{Probability flow ODE derivation}
\label{sec:proba_ode_app}
Recall that ERDM operated on noisy sequences $\vxbar_{1:W}$ of size $W$, with each $\vxbar_{w}\in\R^D$ corrupted according to increasing noise level standard deviations $0\approx\sigmaerdm_{1}(t)<\sigmaerdm_{2}(t)<\dots<\sigmaerdm_{W}(t)$.
The noise covariance matrix for the entire window $\vxbar_{1:W}$ at diffusion time $\dtime$ is block-diagonal:
\begin{align}
    \mathbf{G}_{\text{window}}(\dtime) = \text{diag}(\varerdm_{1}(\dtime)\boldi_D, \varerdm_{2}(\dtime)\boldi_D, \ldots, \varerdm_{W}(\dtime)\boldi_D). \label{eq:G_window}
\end{align}
Its time derivative is:
\begin{align}
    \dot{\mathbf{G}}_{\text{window}}(\dtime) = \text{diag}(2\sigmaerdm_{1}(\dtime)\dot{\sigma}_1(\dtime)\boldi_D, \ldots, 2\sigmaerdm_{W}(\dtime)\dot{\sigma}_W(\dtime)\boldi_D). \label{eq:G_dot_window}
\end{align}
Let $\mathbf{K}_{\text{window}}(\dtime)$ be a matrix such that $\mathbf{K}_{\text{window}}(\dtime)\mathbf{K}_{\text{window}}(\dtime)^T = \dot{\mathbf{G}}_{\text{window}}(\dtime)$. Specifically:
\begin{align}
    \mathbf{K}_{\text{window}}(\dtime) = \text{diag}(\sqrt{2\sigmaerdm_{1}(\dtime)\dot{\sigma}_1(\dtime)}\boldi_D, \ldots, \sqrt{2\sigmaerdm_{W}(\dtime)\dot{\sigma}_W(\dtime)}\boldi_D), \label{eq:K_window}
\end{align}
assuming $\sigmaerdm_{w}(\dtime)\dot{\sigma}_w(\dtime) \ge 0$. The forward SDE for the window $\vx(\dtime)$ is:
\begin{align}
    \diff{\vx} = \mathbf{K}_{\text{window}}(\dtime) \diff{\pmb{\omega}}(\dtime), \label{eq:forward_sde_erdm}
\end{align}
where $\pmb{\omega}(\dtime)$ is a standard $WD$-dimensional Wiener process. The corresponding backward (reverse-time) SDE, which is the primary SDE used for generation, is~\citep{song2021scoreSDE}:
\begin{align}
    \diff{\vxbar} = \left[ - \dot{\mathbf{G}}_{\text{window}}(\dtime) \nabla_{{\vxbar}} \log p(\vxbar; \mathbf{G}_{\text{window}}(\dtime)) \right] \diff t + \mathbf{K}_{\text{window}}(\dtime) \diff{\bar{\pmb{\omega}}}(\dtime). \label{eq:backward_sde_erdm}
\end{align}
The score for the $w$-th snapshot of the window, $\vxbar_{w}$, is approximated with the learned denoiser, $\denoiser$:
\begin{align}
    \nabla_{\vxbar_{w}} \log p_\theta(\vxbar; \mathbf{G}_{\text{window}}(\dtime)) = (\varerdm_{w}(\dtime))^{-1} (D_{\theta}(\vxbar; \sigmaerdmbold(\dtime))_w - \vxbar_{w}), \label{eq:score_component_erdm}
\end{align}
where $D_{\theta}(\vxbar; \sigmaerdmbold(\dtime))_w$ is the $w$-th snapshot output of the denoiser $\denoiser$, which is conditioned on the full noisy window $\vxbar$ and the vector of current noise standard deviations $\sigmaerdmbold(\dtime)$.
The probability flow ODE, which provides a deterministic path for generation, is obtained by removing the stochastic term from the backward SDE and adjusting the drift (see \citet{song2021scoreSDE}, Eq. (14) and \citet{karras2022edm}, Eq. (5)):
\begin{align}
    \diff{\vxbar} = \left[ - \dot{\mathbf{G}}_{\text{window}}(\dtime) \nabla_{{\vxbar}} \log p_\theta(\vxbar; \mathbf{G}_{\text{window}}(\dtime)) \right] \diff t.
    \label{eq:ode_flow_erdm_general}
\end{align}
Using the specific EDM parameterization where the ODE drift is $-\mathbf{D}(\dtime) \nabla_{\vxbar} \log p(\vxbar; \sigmaerdmbold(\dtime))$ with $\mathbf{D}(\dtime) = \text{diag}(\sigmaerdm_{1}(\dtime)\dot{\sigma}_1(\dtime)\boldi_D, \ldots, \sigmaerdm_{W}(\dtime)\dot{\sigma}_W(\dtime)\boldi_D)$, the ODE becomes:
\begin{align}
    \diff\vxbar &= - \text{diag}(\sigmaerdm_{1}(\dtime)\dot{\sigma}_1(\dtime)\boldi_D, \ldots, \sigmaerdm_{W}(\dtime)\dot{\sigma}_W(\dtime)\boldi_D) \nabla_{\vxbar} \log p_\theta(\vxbar; \sigmaerdmbold(\dtime)) \diff t. \label{eq:ode_erdm_user_app}
\end{align}







\subsection{Sampling}
\label{sec:sampling_app}
ERDM generates sequences with a sliding window of size \(W\). 
Generating each window requires solving the backward SDE in
\eqref{eq:backward_sde_erdm} (or ODE in \eqref{eq:ode_erdm_user_app}). 
In each iteration $k$ where the SDE is fully solved, ERDM operates on the noisy window $\vxbar^{(k)}_{1:W}$.
After solving the SDE backwards $k\ge1$ times, the first snapshot of the window, $\vxbar^{(k)}_1$ (with relative window index $w=1$), corresponds to a forecast of $\vy_k$, where $k$ is the \emph{absolute} time step of the forecast time dimension. 
Solving the SDE requires discretizing it into \(N_k\) diffusion ``time'' steps, each of size $\Delta\dtime_k:=\frac{1}{N_k}$.
In general, $N_k$ may vary across iterations. To simplify our illustrations of the sampling process in the following, we assume that $N_k$ and $\Delta\dtime_k$ are identical for all $k$, denoted as $N$ and $\Delta\dtime$ respectively. This can be achieved by setting $N$ in our sampling algorithms to a positive integer.
Note that in practice $N$ need not be an integer (e.g., we use $N=1.25$ for our Navier-Stokes experiments), which is accounted for in our sampling algorithms.
With this simplification, the SDE evolves, for all $k$, following $\sigmaerdmbold(t_1) \rightarrow\sigmaerdmbold(t_2)\rightarrow\dots\rightarrow\sigmaerdmbold(t_{N})$, where $t_{i+1} = t_{i}+\Delta\dtime$ for $i\ge1$, $t_1=0$, and $t_{N}=1$.
For example, if $N=1$ the SDE is solved from $\sigmaerdmbold(0)$ to $\sigmaerdmbold(1)$ in one sampling step, while if $N=2$ it would be solved following $\sigmaerdmbold(0) \rightarrow \sigmaerdmbold(\frac{1}{2})\rightarrow\sigmaerdmbold(1)$ in two sampling steps (see illustration in~\cref{fig:diagram}). 
At the end of the $N$ sampling steps, $\sigmaerdm_1(1)=\smin$ and the first snapshot in the window can be emitted as the forecast for lead time $k$, $\vxbar^{(k)}_1\approx\vy_k$. In the next iteration of $N$ sampling steps, the next snapshot $k+1$ will be predicted by operating on a new window, $\vxbar^{(k+1)}_{1:W}$.
Because $\sigmaerdm_{w+1}(1)=\sigmaerdm_{w}(0)$ for $w\ge1$, this new window can be constructed by shifting the noisy parts of the old window by one position.
That is, $\vxbar^{(k+1)}_{w}:=\vxbar^{(k)}_{w+1}$ for $w<W$, and a new pure-noise snapshot is appended as $\vxbar^{(k+1)}_{W}\sim\mathcal{N}(\boldzero, \smax^2\boldi)$. 
The new window is thus noised according to standard deviations $\sigmaerdmbold(0)$ and the process can be restarted and repeated indefinitely, as illustrated in~\cref{fig:diagram}.
\paragraph{Discretization.}
As mentioned above, the SDE or probability flow ODE need to be solved through discretization over $N$ diffusion steps. We denote the noisy window at diffusion step $n\in\{1, \dots, N\}$ as $\vxbar(n)$. We ignore the superscript $k$ since the ODE discretization is independent of it. Assuming that $\vxbar(0)$ corresponds to a window corrupted according to $\sigmaerdmbold(0)$, 
an Euler–Maruyama step for the probability flow ODE and the whole window is
\begin{align}
    \vxbar(n+1)
   &=\vxbar(n)
   +\frac{\sigmaerdmbold((n+1)\Delta\dtime)- \sigmaerdmbold(n\Delta\dtime)} {\sigmaerdmbold(n\Delta\dtime)}
     \bigl[\vxbar(n) - \denoiser\bigl(\vxbar(n);\sigmaerdmbold(n\Delta\dtime)\bigr)\bigr]
\end{align}
The ODE is thus solved \emph{in parallel} for all snapshots in the window.
When $\vxbar(N)$ is reached, its noise levels correspond to $\sigmaerdmbold(N\Delta\dtime) = \sigmaerdmbold(1)$. Thus, the first snapshot is emitted, and the window is shifted as describe above.
Higher-order stochastic integrators—e.g.\ the second-order Heun
scheme of EDM~\cite{karras2022edm}—can replace Euler–Maruyama as decribed in the next subsection~\ref{sec:heunsampler} below.

\paragraph{Continuous-time view of a tracked snapshot.} Focus on a single physical snapshot \(\vz\) as it is refined across windows.
Let \(k\) be the rollout step at which \(\vz\) is finalised. The snapshot first appears at rollout step \(k-W+1\) (relative position \(w=W\)) and moves to \(w=1\) at step \(k\). Define \(s_w:=\sigmaprog{w}(0)\) as the initial noise level when the snapshot is at position \(w\), and treat \(w\) as a continuous variable on the normalised interval \([0,1]\) (with \(w\!\searrow\!0\) as refinement proceeds). Let \(\vz_w\) denote the state of the tracked snapshot when its current noise level is \(s_w\). Its evolution is governed by
\begin{align}
  \diff\vz_w
    &= -\frac{1}{2}\,\frac{\diff s_w^{2}}{\diff w}\,
       \nabla_{\vz_w}\!\log p\bigl(\vz_w; s_w,\mathcal{C}_w\bigr)\,\diff w
       + \sqrt{\frac{\diff s_w^{2}}{\diff w}}\;\diff\bar{\boldsymbol{\epsilon}}_w,
       \label{eq:sde_tracked_snapshot_w_time}
\end{align}
where \(\diff w<0\) since \(w\) decreases. The context \(\mathcal{C}_w\) comprises all other snapshots in the window: The past \([0,w)\) and the future \((w,1]\).
Note that \(\tfrac{d s_w^{2}}{dw}>0\), and the score is obtained as
\[
  \nabla_{\vz_w}\!\log p\bigl(\vz_w; s_w,\mathcal{C}_w\bigr)
    = \frac{1}{s_w^{2}}\bigl(\denoiser(\vz_w; s_{1:W}, \mathcal{C}_w)_w-\vz_w\bigr),
\]
where $\denoiser(\vz_w; s_{1:W}, \mathcal{C}_w)_w$ is the fully denoised estimate of the snapshot at position \(w\). SDE in~\eqref{eq:sde_tracked_snapshot_w_time} therefore captures how a snapshot is progressively refined as it shifts forward through successive windows, with its effective noise level \(s_w\) decreasing monotonically.




\subsection{Stochastic Heun Sampler}
\label{sec:heunsampler}
In the main text, we introduced our proposed deterministic, first-order sampling algorithm (\cref{alg:sampling_simple}). We now extend this into a more sophisticated sampler, detailed in \cref{alg:sampling_stochastic}, by incorporating two key modifications: a second-order Heun correction step and optional sampling stochasticity via a ``churn'' mechanism.
Both components are inspired by EDM's stochastic sampling algorithm~\cite{karras2022edm}, but require specific adaptations for our {\ours} to work.

\paragraph{Heun correction step.}
The Heun method enhances sampling accuracy by incorporating a corrector step over the basic Euler step during the denoising of the data.
However, a complication arises because when $\tnext'\ge1$, the first snapshot of $\xnext'$ is already fully denoised (i.e., ``clean'').
Applying the denoiser to such a clean snapshot as part of the Heun step would be inappropriate.
To address this, we pad the noisy data by one extra pure-noise snapshot to the end of the sequence, resulting in a window of size $W+1$.
The core denoiser, $\denoiser$, is then selectively applied on the sub-window of size $W$, depending on whether the first snapshot is already clean or not.
That is, if $\tnext'\ge1$, the denoiser would be applied on the snapshots with indices $2, \dots, W+1$. Otherwise, it is applied, as usual, to the indices $1,\dots,W$.
To formalize this selective application, we define $\denoiser^\mathrm{pad}(\vx, \sigmaerdmbold(t))$ such that it applies the actual denoiser on all snapshots with indices $I=\{\lfloor t \rfloor+1, \dots, \lfloor t \rfloor + W\}$ and leaves all other snapshots untouched. This ensures that the denoiser evaluations within the Heun step are confined to snapshots that are indeed noisy.
\begin{align}
    \vspace{-4mm}
    \denoiser^\mathrm{pad}(\vx, \sigmaerdmbold(t))_w :=
    \begin{cases}
    \denoiser(\vx_I, \sigmaerdmbold(t)_I)_{w-\lfloor t \rfloor} & \text{if } w \in I \\
    \vx_w & \text{otherwise}
    \end{cases}
    \vspace{-8mm}
\end{align}
\paragraph{Churn mechanism adaptation.}
The second modification is the integration of a ``churn'' step, which introduces new noise at each iteration to inject stochasticity into the sampling process.
Unlike the EDM algorithm, which applies churn at the beginning of each iteration, we position it at the end of each iteration.
This change is motivated by the fact that the first snapshots of the very first initial window are noised based on low levels of noise, so that artificially injecting more noise before performing any denoising could be harmful. 
This contrasts with the EDM setup, where the noisy data at the very first iteration is typically maximally noisy.

\input{algos/sampling_stochastic_heun}
\paragraph{Experimental configuration.}
For the experiments reported in this work, the stochastic Heun sampler (\cref{alg:sampling_stochastic}) is always used with the Heun correction step. Our ablations in~\cref{sec:ablations_app} demonstrate that our second-order sampler provides better results than a first-order version (Euler-only;~\cref{alg:sampling_simple}).
However, we disable sampling stochasticity by setting the churn rate $\Schurn=0$.
In our experience, we found that small values of $\Schurn\approx0.1$ can increase the ensemble spread, albeit it had minimal impact on error metrics such as the CRPS. We defer a more comprehensive analysis of stochastic sampling to future work.
Our algorithm can, in principle, be used to fully denoise multiple snapshots at once (e.g., output two snapshots per iteration when $N=\frac{1}{2}$). This could be useful in certain modalities or problems where subsequent snapshots are highly correlated, but is left to future work too.


\subsection{Temporal noise prior}
\label{sec:noise_prior}
In our training and sampling algorithms (Alg.~\ref{alg:training}-\ref{alg:sampling_stochastic}), we use i.i.d. noise to corrupt the first window as well as append new pure-noise snapshots when sliding the window. As mentioned in the main text, it has been shown to be useful to sample \emph{temporally correlated noise} snapshots. In our experiments, we use the simple ``progressive noise model'' technique proposed by~\citet{ge2024noiseprior}, which we found to improve results on the Navier-Stokes dataset compared to using i.i.d. noise (see~\cref{sec:ablations_app}). 
This noise prior generates the noise for each snapshot autoregressively by combining a perturbed version of the previous noise with a new i.i.d. noise sample.
Mathematically, the noise for the first snapshot is random, $\noise^{(1)}\sim\mathcal{N}(\boldzero, \boldi)$, while for the following snapshots $k>1$:
\begin{align}
    \noise^{(k)} &= \frac{\alpha}{\sqrt{1+\alpha^2}}\noise^{(k-1)} + \noise^{(k)}_\mathrm{ind},
    \qquad \noise^{(k)}_\mathrm{ind}\sim \mathcal{N}(\boldzero, \frac{1}{1+\alpha^2}\boldi),
\end{align}
where $\alpha$ controls the correlation strength between snapshot noises and $\alpha=0$ corresponds to i.i.d. noise. In our experiments, we use $\alpha=1$ following the recommended practice in~\cite{ge2024noiseprior}. These temporally correlated noise samples replace the i.i.d. draws in lines 6 and 27 of~\cref{alg:sampling_stochastic}.
For example, line 6 becomes $\xcur\gets\hat{\vy}_{1:W}^\mathrm{init}+\sigmaerdmbold(\tcur)\noise_{1:W}$, where $\noise_{1:W}:=(\noise^{(1)}, \dots, \noise^{(W)})$.
Note that the noise prior is applied during both training and sampling, enabling the model to learn how to deal with temporally correlated noise samples.
A more comprehensive study of alternative noise priors in the context of \ours\ is left to future work.
\section{Experimental Details}
\label{sec:experimental_details_app}
\subsection{Dataset details}
\label{sec:datasets_app}
\paragraph{Licenses.}
We use the Navier-Stokes dataset with four obstacles introduced by~\citet{otness21nnbenchmark}. Its license is CC BY 4.0. We also use the ERA5 dataset provided by Weatherbench-2~\cite{rasp2023weatherbench2}. Its license is CC BY 4.0.

\paragraph{ERA5 variables.} 
We use the $1.5^\circ$ resolution of the data (a $240\times121$ grid) provided by~\cite{rasp2023weatherbench2}, with 69 prognostic (input and output) variables: Temperature (\texttt{t}), geopotential (\texttt{z}), specific humidity (\texttt{q}), and the u and v components of wind (\texttt{u,v}) over 13 pressure levels as well as the four surface variables 2m temperature (\texttt{2t}), mean sea level pressure (\texttt{mslp}), and u and v components of winds at 10m (\texttt{10u, 10v}). Following Weatherbench-2, the 13 levels are $\{50,100,150,200,250,300,400,500,600,700,850,925,1000\}$. Numbers after any variable abbreviation refer to the level in hPa. For example, $\texttt{z500}$ refers to geopotential at 500 hPa. Besides these prognostic variables, we also use six static maps as additional inputs: A land-sea mask, soil type, orography, and $\sin(lat), \cos(lat)\cos(lon), \cos(lat)\sin(lon)$, where $lat,lon$ are the latitude and longitude of each location.
We train our models on $12$-hourly data, as in~\cite{price2023gencast}, from 1979 to 2020 and evaluate the models based on 64 initial conditions at 00/12 UTC, evenly spaced in 2021.

\paragraph{Sizes.}
The Navier-Stokes training set consists of around $6100$ samples, and the corresponding validation set consists of 118 samples (slightly less for $W>1$).
We use the ERA5 training data set at $12$-hourly resolution with possible initial condition times in 00/06/12/18 UTC. The training data size is thus around $61000$ samples. Following Weatherbench-2, we only use 00/12 UTC start times for evaluation. We use 64 samples for evaluation. During validation, we use ensemble sizes of $M=10$ and $M=5$ for Navier-Stokes and ERA5 respectively, which we increase to $M=50$ and $M=10$ during final testing.


\subsection{Implementation details and hyperparameters}
\label{sec:impldetails}
\paragraph{Architecture.}
For all experiments and baselines that we train, we use the same basic ADM U-Net~\cite{dhariwal2021diffusion} architecture, only modified where necessary depending on the method. We always use $4$ down- and up-sampling blocks, with channel multipliers $1,2,3,4$. The base channel dimension is $64$ for Navier-Stokes and $256$ for ERA5 experiments. We use attention layers at the two lowest spatial resolutions. We use a dropout rate of 0.15 for Navier-Stokes, but disable it for ERA5.
Other architectural hyperparameters are kept to the defaults from EDM. This results in a parameter count of $29.5$ million for Navier-Stokes, and $517$ million for ERA5. For the models operating on sequence, ERDM and EDM with $W>1$, we integrate causal temporal attention layers into the 2D U-Net before each down- and up-sampling block. This increases the respective parameter counts slightly to $31.5$ and $537$ million for Navier-Stokes and ERA5, respectively. For ERA5, we use circularly padded convolutions along the longitude dimension (and zero-padding for the latitude dimension) to better respect the spherical structure of the Earth.
\paragraph{Navier-Stokes hyperparameters.}
We train each model for 300 epochs with a global effective batch size of 32, leading to almost $57,000$ gradient steps. The effective batch size is kept the same by balancing per-GPU batch size and gradient accumulation steps appropriately over different training runs. We use a cosine learning rate schedule with a linear warmup period of 10 epochs and a peak learning rate of $6\times10^{-4}$. We use the AdamW optimizer with a weight decay of $10^{-4}$. During evaluation, we use an exponential moving average (EMA)--based on an EMA decay rate of $0.995$--version of the model. Because the input spatial grid, $221\times42$, after four halvings does not result in an even number--a problem for the residual connections in the upsampling blocks--we bilinearly interpolate it to a $256\times64$ grid at the input layer (and back before the final convolution layer).
\paragraph{ERA5 hyperparameters.}
We use a cosine learning rate schedule with a linear warmup period of 5 epochs and a peak learning rate of $5\times10^{-4}$. ERDM and EDM are trained with a global effective batch size of 32 and 512, respectively. The difference in this choice was informed by how many data points we could fit into GPU memory (ERDM has higher memory needs due to operating on a window of data) and the observation that EDM benefited more strongly from higher effective batch sizes.
The maximum length of the schedule is $60$ epochs for ERDM and $1000$ epochs for EDM, resulting in a comparable number of total gradient steps.
We use the AdamW optimizer without weight decay, clipping gradients above a norm of $0.8$.
During evaluation, we use an EMA version of the model, with an EMA decay rate of $0.9999$. 
Because the input spatial grid, $240\times121$, after four halvings does not result in an even number--a problem for the residual connections in the upsampling blocks--we bilinearly interpolate it to a $240\times128$ grid at the input layer (and back before the final convolution layer).

\subsection{Baseline details}
\label{sec:baselines_app}
\paragraph{DYffusion.}
We follow the default values used by DYffusion for its Navier-Stokes experiments, but replace the interpolator and forecaster architecture with the more performant U-Net and use our optimization hyperparameters, both described above. We tuned the interpolator dropout rates and found a rate of 0.5 to produce the best results. 
The interpolator dropout rate is a key hyperparameter in DYffusion, since it is responsible for the stochasticity in the model (if 0, it reduces to a deterministic model). 
As for ERDM, we use a window size of $W=6$.
Our improved architecture and training hyperparameters jointly improved the DYffusion CRPS scores by over $3\times$ compared to the results in the original paper.
\paragraph{EDM.}
Our primary focus is to evaluate ERDM's performance relative to common EDM-derived approaches for dynamics forecasting.
Consequently, our key baseline is an EDM denoiser parameterized by $\denoiser(\vx_1 \ldots, \vx_W; \sigma, \vy_0)$, where all $\vx_w$ are corrupted according to the same noise standard deviation, $\sigma$. With $W=1$, we recover a next-step forecasting conditional EDM as in~\cite{price2023gencast, kohl2023benchmarking}. On Navier-Stokes, we tried $W\in\{1,2,4,6\}$, and found $W=4$ to work best for EDM. For computational reasons, we only report an EDM run with $W=1$ for ERA5.
To condition the denoiser network on the initial condition, $\vy_0$, we concatenate it along the channel dimension to the input, noisy window, $\vx_{1:W}$. 
When $W>1$, $\vy_0$ is thus first duplicated along the window dimension. We found this to perform better than operating on a $W+1$ window, where $\vy_0$ is inserted into the first position of the window. Preconditioning is applied before concatenation, \emph{only} to the noisy window.

\subsection{Distinctions between GenCast and our EDM model for ERA5}
\label{sec:gencastvsedm}
Our primary baseline, conditional EDM, applied to ERA5 shares several similarities with GenCast~\cite{price2023gencast}. 
A primary distinction lies in the spatial resolution: GenCast is trained on a spatial resolution of $1^\circ$ and $0.25^\circ$, whereas our model, EDM, and other baselines are exclusively trained at a $1.5^\circ$ resolution. 
The denoiser network architecture presents another significant divergence. 
GenCast uses a graph neural network, similar to the approach in~\citet{lam2023learning}, while we use a 2D U-Net for our EDM baseline borrowed from~\cite{karras2022edm, dhariwal2021diffusion}. Furthermore, GenCast incorporates SST as both an input and an output variable. sea surface temperature as an input and output, which we do not.
More minor differences between GenCast vs. our EDM are: ODE solver (DPMSolver++ vs. Heun), training noise level distribution (uniform vs. lognormal), noise distribution (spherical vs. i.i.d.), optimization hyperparameters, and residual vs. full-state prediction.

\subsection{Compute resources}
\label{sec:compute_app}
For Navier-Stokes experiments, we trained all models and baselines on $2-8$ L40S or A100 GPUs. For ERA5, we trained some models (including our final ERDM version) on 4 H200 GPUs, but performed most of our development work on $8-16$ A100 GPUs (per training run). For ERDM with $W=6$ ($W=4$), a full training run on 8 L40S GPUs took around 16 (10.5) hours. For ERA5 with $W=6$, a full training run on 4 H200 GPUs took almost 5 days.

\subsection{Metrics}
\label{sec:metrics}
Let $\vy \in \R^{\nlat \times \nlon}$ indicate the targets for a specific time step, and $\hat\vy \in \R^{M \times \nlat \times \nlon}$ the corresponding predictions with an ensemble size of $M$. $\nlat$ and $\nlon$ are the size of the height dimension and width dimension, respectively. For ERA5, these are latitude and longitude, respectively. The definitions of the metrics below follow common practices (e.g., cf. Weatherbench-2~\cite{rasp2023weatherbench2}).
\vspace{-1.3mm}
\paragraph{Weighting.} 
For ERA5 experiments, we follow the common practice of area-weighting all spatially aggregated metrics according to the size of the grid cell, ensuring that they are not biased towards the polar regions~\cite{rasp2023weatherbench2}. The unnormalized area weights are computed as $\tilde w(i) = \sin\phi_i^u-\sin\phi_i^l$, where $\phi_i^u$ and $\phi_i^l$ represent upper and lower latitude bounds, respectively, for the grid cell with latitude index $i\in\{1,2,\dots, I\}$. The normalized area weights, used in all metrics below are thus $w(i) = \frac{\tilde w(i)}{\frac{1}{I}\sum_{i=1}^I\tilde w(i)}$, where $I$ is the number of latitude indices. For the Navier-Stokes experiments, we report unweighted spatially aggregated metrics (i.e., $w(i) = 1$ for all $i$).

\paragraph{Ensemble-mean RMSE.}
In ensemble-based forecasting, the RMSE is typically computed based on the ensemble mean prediction as follows: 

\begin{equation}
    \mathrm{RMSE}_\mathrm{ens} 
    = \sqrt{\fracij\sumij w(i) (\mathrm{mean}_m(\hat\vy_{m, i,j}) - \vy_{i,j})^2},
\end{equation}
where $\mathrm{mean}_m$ refers to the average over the ensemble dimension.

\paragraph{Spread-skill ratio (SSR).}
Following \citet{fortin2014ssr}, the spread-skill ratio is defined as the ratio between the ensemble spread and the ensemble-mean RMSE. The ensemble spread is defined as the square root of the ensemble variance. Thus, the SSR is defined as:
\begin{equation}
    \mathrm{Spread} = \sqrt{\fracij\sumij w(i) \mathrm{var}_m(\hat\vy_{m,i,j})},
    \qquad
    \mathrm{SSR} = \sqrt{\frac{M+1}{M}} \frac{\mathrm{Spread}}{\mathrm{RMSE}_\mathrm{ens}},
\end{equation}
where $\mathrm{var}_m$ refers to the variance over the ensemble dimension, and $\sqrt{(M+1)/{M}}$ is a correction factor which is especially important to include for small ensemble sizes.
The SSR serves as a simple measure of the reliability of the ensemble, where values closer to 1 are better. Values smaller (larger) than 1 indicate 
underdispersion (overdispersion). In our ablations, we also report the average squared deviation, $\overline{(1 - \mathrm{SSR})^2}$ (lower is better), where the overline indicates averaging over all rollout/lead times. This is more useful than reporting $\overline{\mathrm{SSR}}$, since such a metric would be sensitive to canceling out deviations from 1 on both sides (e.g., an $\mathrm{SSR}$ of 0.5 and 1.5 on the first and last half of the rollout, respectively, would average to 1).

\paragraph{Continuous ranked probability score (CRPS).}
Following \cite{zamo2018faircrps, rasp2023weatherbench2}, we use the unbiased version of the CRPS~\cite{matheson1976crps} (lower is better)
\begin{equation}
    \mathrm{CRPS} = \fracij\sumij w(i) \left[\frac{1}{M} \sum_{m=1}^M \lvert\hat\vy_{m,i,j} - \vy_{i,j}\rvert - \frac{1}{2M(M-1)} \sum_{m=1}^M \sum_{n=1}^M \lvert\hat\vy_{m,i,j} - \hat\vy_{n,i,j}\rvert\right].
\end{equation}

\paragraph{CRPS skill score (CRPSS).}
The CRPSS is commonly used to compare the CRPS scores relative to a baseline. It is computed as $1 - \mathrm{CRPS}_\text{model} / \mathrm{CRPS}_\text{baseline}$, where $\mathrm{CRPS}_\text{model}$ and $\mathrm{CRPS}_\text{baseline}$ refer to the CRPS scores for the evaluated model and baseline, respectively.
Values larger than $0$ indicate that the model is better than the baseline model in underscore, while values smaller than 0 indicate that the model is worse than the baseline model.

\section{Ablations}
\label{sec:ablations_app}
\vspace{-1.3mm}
Starting from our proposed ERDM configuration (``Base'' in \cref{tab:ablations}), which uses the second-order Heun sampler, loss weighting with $\Pmean=0.5, \Pstd=1.2$, a temporally correlated progressive noise prior following~\cite{ge2024noiseprior} with $\epsilon_{\rm prog}:=\alpha=1$ (see~\cref{sec:noise_prior}), noise schedule parameters $\smin=0.002, \smax=200, \rho=-10$, and a window size of $W=6$, we systematically varied individual components. For sampling, we use $N=1.25$ ($\Delta t=0.8$) and $\Schurn=0$ by default. 
The first window is initialized based on predictions from the EDM $W=4$ baseline.
Our findings, summarized in \cref{tab:ablations}, highlight several critical elements for achieving optimal performance.
\input{tables/ablation_ns}

\paragraph{Heun versus Euler sampler.} 
Compared to the first-order Euler method (\cref{alg:sampling_simple}), the second-order sampling algorithm detailed in~\cref{alg:sampling_stochastic} achieved superior performance in CRPS, MSE, and SSR metrics. This improvement comes at the cost of doubling the neural function evaluations due to the correction step, which significantly reduces sampling speed. Nevertheless, we deemed this increased computational overhead justifiable for the enhanced accuracy.

\paragraph{Number of sampling steps.}
Our base ERDM model uses $N=1.25$ sampling steps ``per snapshot'', which results in a step size of $\Delta t = 1/N = 0.8$. That is, depending on the noise levels at the start of an iteration for solving the ODE, it may use 1 or 2 sampling steps before outputting the first frame and sliding the window. 
Increasing $N$ extends sampling time, but optimal performance is found at a ``sweet spot''; for example, $N=1.25$ outperforms $N=2$ for Navier-Stokes.
This was also confirmed for ERA5, where CRPS scores showed minimal improvement for $N>2$ (with an optimum around $N=2$).
The parameter $N$ is also crucial for tuning the model's spread—smaller $N$ increases spread, larger $N$ reduces it—thereby providing a control for adjusting the spread-skill ratio.
Ultimately, the ideal $N$ and step size depend on the dataset complexity and the window size, where larger window sizes generally tolerate larger step sizes (smaller $N$).
\paragraph{Loss weighting.}
As detailed in the main text, a well-tuned loss weighting can significantly improve results. In particular, using no loss reweighting based on the PDF of the lognormal distribution, $f(\sigma;\Pmean,\Pstd)$ (``No $f(\sigma)$''), degraded CRPS and MSE scores by more than $2\times$ and $5\times$, respectively. It also resulted in a worse calibration score. Similarly, choosing a poor value for the center of the lognormal distribution, e.g., $\Pmean=-1.2$, resulted in almost as suboptimal results. 
Interestingly, $\Pmean=-1.2$ corresponds to the default value in EDM, but we found values $\ge0.5$ to be necessary for \ours. We found EDM's default $\Pstd=1.2$ to work well for ERDM generally.
\paragraph{Noise prior.}
As hinted in~\cref{sec:noise_prior}, we found it beneficial to use a temporally correlated noise prior as opposed to simply sampling i.i.d. noise for each snapshot (``$\epsilon_\mathrm{indep}$''). The noise prior that we use~\cite{ge2024noiseprior} introduces one hyperparameter, $\alpha$, which balances the strength of the correlation. We generally found $\alpha\in [0.5, 1.5]$ to work best. Similarly to the findings of~\citet{ge2024noiseprior}, using overly large $\alpha$ (e.g., $\alpha=10$, which introduces large correlations between noise snapshots) is detrimental to performance and worse than simply using i.i.d. noise.
\paragraph{Other design choices.}
Surprisingly, unconditional static pre-training of the non-temporal components of the model before fine-tuning on the forecasting task was found to degrade performance (``Uncond. Pre-training''). 
Preliminary ERA5 experiments seemed to confirm this finding.
Moreover, we did not find \ours\ to be overly sensitive to the window size within a reasonable range ($W=6$ as base). However, overly large window sizes (e.g., $W=12$) significantly degrade performance, potentially due to fixed network capacity. Note that for each window size, we tuned the sampling step size from $\Delta t \in \{0.25, 0.5, 0.8, 1\}$ and report the best results only.



\subsection{\ours\ initialization}
\label{sec:init_app}
\begin{figure}[h] %
  \centering %
  \includegraphics[width=\linewidth]{figures/raw/init-ns-ablation.pdf} 
  \caption{\small 
    Impact of first-window initialization methods on \ours's Navier-Stokes forecast performance. The chosen initialization strategy significantly affects short-range skill (up to timestep 10), but its influence diminishes over longer forecast horizons. While initializing with ground truth data (an impractical approach) unsurprisingly yields the best initial results, it also produces an artificially inflated Spread-Skill Ratio (SSR) due to a near-zero initial RMSE. Notably, this early advantage dissipates throughout the rollout, leading to final CRPS and SSR scores that are largely indistinguishable regardless of the initialization method.
  }
  \label{fig:ns_erdm_init}
\end{figure}
In \cref{fig:ns_erdm_init} and the last part of \cref{tab:ablations}, we ablate possible initialization schemes for \ours. These include two neural initializations based on the EDM baselines with $W=1$ and $W=4$ (our default for Navier-Stokes being EDM $W=4$), alongside two simple non-neural approaches.
One non-neural method, Init=\texttt{truth}, initializes \ours\ using ground truth data for the first window. While clearly impractical, it serves as an informative upper bound on the achievable performance of ERDM. The other, Init=\texttt{persistence}, populates the first window by duplicating the initial condition $\vy_0$; this approach performed very poorly and is omitted from \cref{fig:ns_erdm_init} for visual clarity.
Expectedly, Init=\texttt{truth} yields optimal initial CRPS results but cannot be used in practice. More significantly, \cref{fig:ns_erdm_init} demonstrates that the choice of initialization predominantly impacts \ours's short-range forecast skill (up to approximately timestep 10). The initial performance advantages conferred by methods like Init=\texttt{truth}, or differences among other practical schemes, diminish substantially as the forecast horizon extends. 
By the end of the rollouts, both CRPS and SSR scores become largely indistinguishable across the various practical initialization methods, including our default EDM $W=4$ initialization. This convergence suggests that while the first-window initialization is crucial for early forecast accuracy, its specific choice becomes less critical for the long-range performance of \ours\ (except when using a naive initialization such as \texttt{persistence}).


\subsection{ERA5 evaluation year ablation}
\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figures/raw/2021-vs-2022-scores.png}
    \caption{
    Impact of ERA5 evaluation year on ERDM's CRPS performance. The model, trained on data up to 2020 (inclusive), generally shows improved CRPS scores when evaluated on initial conditions from 2022 compared to 2021 (the baseline year used throughout this paper), with \texttt{t50} being an exception. While most variables exhibit CRPS changes of less than $3\%$, geopotential and mean sea level pressure are significantly more sensitive to the evaluation year, particularly for lead times under 8 days (e.g., \texttt{z500} scores improve by up to $15\%$). This variability underscores the need for careful interpretation when comparing model results from different evaluation years, especially for geopotential and stratospheric fields.
    }
    \label{fig:era5_test_year_ablation}
    \vspace{-3mm}
\end{figure}
In our quantitative comparison, we evaluated ERDM, EDM, and IFS ENS on the same 64 initial conditions evenly spaced in 2021, ensuring a fair comparison. Unfortunately, we only have access to the NeuralGCM ENS and Graph-EFM results for 2020. This discrepancy highlights the importance of understanding how model scores might vary across different evaluation years. We investigate this for our ERDM model (trained on data up to 2020 inclusive) in~\cref{fig:era5_test_year_ablation}. As detailed in the figure and its caption, ERDM's CRPS performance indeed shows sensitivity to the evaluation year. For example, when evaluated on 2022 initial conditions, scores often improved relative to our 2021 baseline, though for most variables this change was less than $3\%$. However, certain variables, notably geopotential (with \texttt{z500} improving by up to $15\%$) and mean sea level pressure, exhibited significantly greater sensitivity, particularly at lead times under 8 days. These findings suggest that while our 2021-based evaluations provide a consistent benchmark for ERDM, EDM, and IFS ENS, direct comparisons with results from NeuralGCM ENS and Graph-EFM on 2020 data should be made with an awareness of this potential inter-annual performance variability.

\section{Additional Results}
\label{sec:results_app}
\begin{wrapfigure}{r}{0.43\textwidth}
\vspace{-13mm}
  \centering
  \includegraphics[width=\linewidth]{figures/raw/ns-mse-rollout.pdf} 
  \caption{ 
    Ensemble-mean MSE scores for the Navier-Stokes test rollout.
  }
  \label{fig:ns_mse_rollout}
  \vspace{-6mm}
\end{wrapfigure}
\subsection{Additional Navier-Stokes results}
\label{sec:ns_results_app}
\vspace{-1mm}
\paragraph{MSE scores.}
For completeness, we report the ensemble-mean MSE test rollout scores for all Navier-Stokes models in~\cref{fig:ns_mse_rollout}, analogously to our main CRPS and SSR results in~\cref{fig:ns_test_rollout}. The MSE scores are strongly correlated to the CRPS scores, which is why we deferred their discussion to the appendix.
Similarly to the CRPS scores, \ours\ achieves significantly better MSE scores, especially for long-range lead times. A slight difference to the CRPS results is that the relative improvement of the block-wise autoregressive EDM $W=4$ baseline over the DYffusion baseline is larger in terms of MSE than CRPS.
\paragraph{Generated videos.} 
Generated videos, the corresponding ground truth, and their lead time-dependent errors are provided at this URL: %
\href{https://drive.google.com/drive/folders/1lVp9TwiK3dcQtsvr5XKlWiBEp1nidnum?usp=sharing}{https://drive.google.com/drive/folders/1lVp9TwiK3dcQtsvr5XKlWiBEp1nidnum?usp=sharing}.

\subsection{Detailed ERA5 quantitative results}
\label{sec:era5_results_app}
\begin{figure}
    \centering
    \includegraphics[width=1\linewidth]{figures/raw/era5-crps-abs.png}
    \caption{Absolute CRPS corresponding to the relative scores shown in~\cref{fig:era5_rel}. Lower is better.
    }
    \label{fig:era5_crps_abs_appendix}
\end{figure}
\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figures/raw/timeseries_20_metrics_relative_to_EDM.png}
    \caption{CRPS relative to next-step autoregressive EDM baseline (in $\%$; lower is better) for 15-day rollouts across a more comprehensive set of variables than in~\cref{fig:era5_rel}.
    }
    \label{fig:era5_crps_rel_appendix}
\end{figure}
\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{figures/raw/era5-ssr-appendix.pdf}
    \caption{Spread Skill Ratio (SSR) computed as the ensemble spread / ensemble-mean RMSE based on 10-member ensembles for each model. Closer to 1 is better and represents a well-calibrated ensemble. Among the ML-based methods, \ours\ tends to produce the best calibration in terms of SSR, together with IFS ENS, except for the first few lead times of \texttt{z500} and \texttt{z700}  where it is overcalibrated. NeuralGCM and EDM tend to be severely underdispersed, while IFS ENS tends to be slightly overdispersed, in the short range.
    }
    \label{fig:era5_ssr_appendix}
\end{figure}
A comprehensive quantitative assessment of forecast skill is presented in~\cref{fig:era5_crps_rel_appendix}, which displays the CRPS for \ours, IFS ENS, NeuralGCM, and Graph-EFM, relative to the EDM baseline. These relative CRPS values are reported as percentages (where lower indicates better performance) for numerous atmospheric variables (extending the set shown in~\cref{fig:era5_rel}) over the full 15-day forecast horizon. These results underscore that \ours\ consistently achieves significantly lower CRPS values than the EDM baseline. The corresponding absolute CRPS scores are visualized in~\cref{fig:era5_crps_abs_appendix}. A full relative comparison between ERDM and EDM is presented in the form of a scorecard in~\cref{fig:era5_scorecard_edm_erdm}.
A quantitative assessment of forecast calibration is presented in ~\cref{fig:era5_ssr_appendix}, which displays the spread-skill ratio for the same set of models and variables. \ours{} generally generates well-calibrated forecasts on par or better than IFS ENS, and consistently more calibrated forecasts than the EDM baseline or NeuralGCM.
\begin{figure}
    \centering
    \includegraphics[width=1\linewidth]{figures/raw/era5_scorecard_edm_erdm.png}
    \caption{
    Scorecard of ERDM's CRPS relative to next-step autoregressive EDM baseline (in $\%$; lower is better) for all predicted variables (except \texttt{v}, which is correlated with \texttt{u}), pressure levels, and lead times (up to 15 days).
    ERDM consistently outperforms the EDM baseline, especially for geopotential and high-altitude levels. 
    }
    \label{fig:era5_scorecard_edm_erdm}
\end{figure}


\subsection{Comprehensive ERA5 power spectra}
\label{sec:power_spectra_app}

To assess the physical realism of the generated forecasts, we analyze their spectral properties, as presented in \cref{fig:era5_spectra_app}. This figure shows the spectral density of 14-day forecasts for several key atmospheric variables, averaged over high latitudes ($[60^\circ, 90^\circ]$). The absolute spectra (subfigure (a)) and spectra normalized by ERA5 reanalysis (subfigure (b)) consistently demonstrate the high fidelity of \ours{}. Our model's spectra closely align with the ERA5 target across a broad range of zonal wavenumbers and frequently match or even slightly outperform those from the operational physics-based IFS ENS model. This accurate representation indicates that \ours{} effectively captures the energy distribution from large planetary scales down to smaller synoptic scales for diverse atmospheric fields. In contrast, NeuralGCM ENS exhibits a marked underestimation of energy at mid to high frequencies for several variables, evident by its spectra decaying more rapidly in the absolute plots and falling below unity in the normalized plots at these scales. The robust spectral characteristics of \ours{} across multiple variables underscore its capability to produce physically consistent and realistic long-range weather forecasts.
\begin{figure}[htbp]
    \centering
    \begin{subfigure}{\linewidth}
        \centering
        \includegraphics[width=\linewidth]{figures/raw/spectra-abs-app.pdf}
        \caption{\small Absolute spectral density.}
        \label{fig:era5_spectra_abs_sub} %
    \end{subfigure}

    \begin{subfigure}{\linewidth}
        \centering
        \includegraphics[width=\linewidth]{figures/raw/spectra-rel-app.pdf}
        \caption{\small Normalized spectral density (relative to ERA5 reanalysis).}
        \label{fig:era5_spectra_rel_sub} %
    \end{subfigure}

    \caption{ 
    Spectral density of 14-day forecasts, averaged over high latitudes ($[60^\circ, 90^\circ]$). 
    \ours{} generates highly accurate spectra that match or even slightly beat the physics-based model IFS ENS, while NeuralGCM ENS underestimates energy at mid to high frequencies. 
    Subfigure (a) displays the absolute spectral density, and subfigure (b) presents the normalized spectral density (absolute spectra divided by the target ERA5 spectra). 
    }
    \label{fig:era5_spectra_app} %
\end{figure}

\subsection{ERA5 forecast visualizations}
\label{sec:era5_viz}
See~\cref{fig:era5_quali1} and~\cref{fig:era5_quali2} for visualizations of example forecasts by \ours\ for a specific initial condition (2022-01-01) and several lead times.
As suggested by the good power spectra (see section above), \ours\ produces realistic forecasts, including at long lead times (e.g., 15 days).
\begin{figure}[htbp]
    \centering
    \begin{subfigure}{\linewidth}
        \centering
        \includegraphics[width=\linewidth]{figures/raw/forecast_ERDM_2m_temperature_2022-01-01_0000.pdf}
        \caption{2 meter temperature (\texttt{2t}).}
        \label{fig:qual1a} %
    \end{subfigure}

    \begin{subfigure}{\linewidth}
        \centering
        \includegraphics[width=\linewidth]{figures/raw/forecast_ERDM_10m_u_component_of_wind_2022-01-01_0000.pdf}
        \caption{10m u component of wind (\texttt{10u}).}
        \label{fig:qual1b} %
    \end{subfigure}
    \caption{ 
    Example visualizations of ERDM (second row), the corresponding ground truth (first row), and the bias (last row) for two example variables and forecast lead times $3, 7,10,$ and $15$ days.
    }
    \label{fig:era5_quali1} 
\end{figure}
\begin{figure}[htbp]
    \centering
    \begin{subfigure}{\linewidth}
        \centering
        \includegraphics[width=\linewidth]{figures/raw/forecast_ERDM_geopotential_500_2022-01-01_0000.pdf}
        \caption{Geopotential at 500 hPa (\texttt{z500}).}
        \label{fig:qual2a} %
    \end{subfigure}

    \begin{subfigure}{\linewidth}
        \centering
        \includegraphics[width=\linewidth]{figures/raw/forecast_ERDM_specific_humidity_700_2022-01-01_0000.pdf}
        \caption{Specific humidity at 700 hPa (\texttt{q700}).}
        \label{fig:qual2b} %
    \end{subfigure}
    \caption{ 
    Example visualizations of ERDM (second row), the corresponding ground truth (first row), and the bias (last row) for two example variables and forecast lead times $3, 7,10,$ and $15$ days.
    }
    \label{fig:era5_quali2} 
\end{figure}




\begin{algorithm}[t]
\footnotesize
\caption{\atphantom\ Elucidated Rolling Diffusion Stochastic Heun Sampler}
\begin{algorithmic}[1]
\State {\bfseries Require:} $\hat{\vy}_{1:W}^\mathrm{init}, N, \ttimemax, \Schurn=0, \Snoise=1$ \Comment{Choose $\Schurn>0$ for sampling stochasticity}
\State\textcolor{gray}{\# Initialization}
\State $\offsetdelta \gets 1 / N$ \Comment{Infer step size from given number of steps per snapshot}
\State $\offsetdelta' \gets \offsetdelta/ (1-\Schurn)$ 
\Comment{Size of larger, initial step, before backtracking (if $\Schurn >0$) to $\offsetdelta$}
\State $\tcur \gets 0;\;\mathcal{S} \gets \emptyset$  \Comment{Initialize global diffusion time and empty generated sequence}
\State {\bf sample} $\xcur \sim \mathcal{N}(\hat{\vy}_{1:W}^\mathrm{init}, \sigmaerdmbold(\tcur)^2\boldi_{W \times D})$
\Comment{Initialize window with snapshot-dependent rolling noise}
\State {\bf concatenate} $n_\mathrm{pad}:=\lfloor \offsetdelta' \rfloor+1$ fully noisy snapshots to the end of $\xcur$ \Comment{Necessary for Heun step}
\While{$\lvert\mathcal{S}\rvert < T_{\rm forecast}$} \Comment{Predict snapshot $\lvert\mathcal{S}\rvert$+1}
\State $\tnext' \gets \tcur + \offsetdelta'$ \Comment{Global diffusion time after denoising, before churn}
\State $\tnext \gets \tcur + \offsetdelta$ \Comment{Global diffusion time after churn ($\tnext\leq \tnext'$)}
\State $\sigmacur \gets \sigmaerdmbold(\tcur)$ \Comment{Current noise levels}
\State $\sigmanext' \gets \sigmaerdmbold(\tnext')$ \Comment{Noise levels before churn}
\State $\sigmanext \gets \sigmaerdmbold(\tnext)$ \Comment{Noise levels at the end of this iteration}
\State\textcolor{gray}{\# Euler step}
\State $\xdenoised \gets \denoiser^\mathrm{pad}(\xcur, \sigmacur)$ \Comment{Denoise sequence}
\State $\dd \gets (\xcur - \xdenoised) / \sigmacur$ \Comment{Evaluate $\diff\xx / \diff\odetime$ at $\tcur$}
\State $\xnext' \gets \xcur + (\sigmanext'-\sigmacur)\dd$ \Comment{Euler step from $\tcur$ to $\tnext'$}
\State\textcolor{gray}{\# Heun step (2nd-order correction)}
\State $\dd'\gets (\xnext' - \denoiser^\mathrm{pad}(\xnext', \sigmanext')) / \sigmanext'$ \Comment{Denoise at $\tnext'$}
\State $\xnext'\gets\xcur + \frac{1}{2}(\sigmanext' - \sigmacur)(\dd + \dd')$ \Comment{$2^\text{nd}$ order Heun correction step}
\State\textcolor{gray}{\# Churn step (stochastic backtrack)}
\State {\bf sample} $\noise \sim \mathcal{N}(\boldzero, \Snoise^2\boldi_{W \times D})$ \Comment{Sample churn noise}
\State $\xnext \gets \xnext' + \sqrt{\sigmanext^2 - \sigmanext'^2} \cdot \noise$  \Comment{Backtrack to $\tnext$ noise levels}
\State\textcolor{gray}{\# Sliding-window shift and forecast snapshot extraction}
\State $\nclean \gets \lfloor \tnext \rfloor$ \Comment{Infer finished snapshots. If $0$, the following lines are a no-op}
\State \textbf{add} first $\nclean$ snapshots of $\xdenoised$ to $\mathcal{S}$ and discard first $\nclean$ snapshots of $\xnext$
\State {\bf sample} $\vx_\mathrm{new} \sim \mathcal{N}(\boldzero, \sigmamax^2\boldi_{\nclean \times D})$
\State $\xcur \gets [\xnext, \vx_\mathrm{new}]$ \Comment{Concatenate fresh noisy snapshots to futuremost}
\State $\tcur \gets \tnext - \nclean$ \Comment{Re-adjust global diffusion `time' to be in $[0, 1)$}

\EndWhile
\State \Return \(\mathcal{S}\)
\end{algorithmic}
\label{alg:sampling_stochastic}
\end{algorithm}

\begin{table}[htbp]
  \centering
  \caption{
Ablation study results on the Navier-Stokes dataset. The most important design choices, without which \ours{} would degrade
to worse performance than the EDM baseline (highlighted in red), are 1) a proper 3D architecture; 2) an appropriate noise schedule (in particular,
not EDM's default $\rho=7$); 3) our proposed loss weighting; 4) a fixed training schedule.
CRPSS refers to the CRPS skill score, where $> 0$ means that the model is better than the baseline model in underscore. 
CRPSS values smaller than 0 indicate that the model is worse than the baseline model.
For all other metrics, lower is better.
}
  \label{tab:ablations}
  \begin{tabular}{l c c c c c}
    \toprule
    Ablation & $\overline{\mathrm{CRPS}}_{\times 10^{2}}$ & $\overline{\mathrm{MSE}}_{\times 10^{3}}$ & $\overline{\mathrm{CRPSS}_{\scriptstyle \mathrm{ERDM}}}$ & $\overline{\mathrm{CRPSS}_{\scriptstyle \mathrm{EDM}}}$ & $\overline{(1 - \mathrm{SSR})^2}$ \\
    \midrule
    Base & 0.904 & \textbf{0.588} & 0.000 & 0.432 & 0.052 \\
    \midrule
    \textbf{Sampling} &  &  &  &  &  \\
    Euler Sampler & 1.160 & 0.752 & -0.279 & 0.288 & 0.227 \\
    $\Delta t=1$ & 1.270 & 0.810 & -0.399 & 0.224 & 0.343 \\
    $\Delta t=0.9$ & 1.067 & 0.823 & -0.160 & 0.349 & 0.056 \\
    $\Delta t=0.7$ & 0.986 & 0.668 & -0.084 & 0.389 & 0.080 \\
    $\Delta t=0.5$ & 1.042 & 0.648 & -0.159 & 0.349 & 0.121 \\
    \midrule
    \textbf{Loss Weighting} &  &  &  &  &  \\
    No $f(\sigma)$ & 2.168 & 2.987 & -1.296 & \textcolor{red}{-0.243} & 0.244 \\
    $P_{\rm mean}=1$ & 1.082 & 0.783 & -0.181 & 0.338 & 0.057 \\
    $P_{\rm mean}=0$ & 1.431 & 1.517 & -0.489 & 0.177 & 0.119 \\
    $P_{\rm mean}=-1.2$ & 2.001 & 2.694 & -1.086 & \textcolor{red}{-0.134} & 0.259 \\
    \midrule
    \textbf{Noise Prior ($\epsilon$)} &  &  &  &  &  \\
    $\epsilon_{\rm{indep}}$ & 1.176 & 0.977 & -0.240 & 0.309 & 0.134 \\
    $\epsilon_{\rm{prog}=2}$ & 1.060 & 0.715 & -0.151 & 0.357 & 0.065 \\
    $\epsilon_{\rm{prog}=10}$ & 1.404 & 1.353 & -0.474 & 0.190 & 0.098 \\
    \midrule
    \textbf{Noise schedule} &  &  &  &  &  \\
    Random training sched. & 1.904 & 1.998 & -1.065 & \textcolor{red}{-0.130} & 0.347 \\
    $\sigma_{\rm min}=0.001$ & 1.040 & 0.758 & -0.102 & 0.383 & 0.062 \\
    $\sigma_{\rm min}=0.01$ & 1.511 & 1.342 & -0.692 & 0.052 & 0.219 \\
    $\sigma_{\rm max}=80$ & 1.094 & 0.878 & -0.175 & 0.346 & 0.080 \\
    $\sigma_{\rm max}=800, \rho=-20$ & 1.249 & 1.086 & -0.355 & 0.244 & 0.056 \\
    $\rho=-5$ & 1.529 & 1.569 & -0.615 & 0.121 & 0.189 \\
    $\rho=-20$ & 1.060 & 0.695 & -0.180 & 0.339 & \textbf{0.045} \\
    $\rho=7$ & 1.982 & 2.382 & -1.273 & \textcolor{red}{-0.265} & 0.098 \\
    \midrule
    \textbf{Window Size ($W$)} &  &  &  &  &  \\
    $W=4$ & 1.411 & 1.308 & -0.487 & 0.185 & 0.104 \\
    $W=8$ & 1.258 & 1.134 & -0.313 & 0.276 & 0.196 \\
    $W=12$ & 1.709 & 1.841 & -0.765 & 0.041 & 0.422 \\
    \midrule
    \textbf{Architecture} &  &  &  &  &  \\
    Uncond. Pre-training & 1.475 & 1.732 & -0.528 & 0.157 & 0.074 \\
    2D arch. & 3.931 & 2.979 & -3.202 & \textcolor{red}{-1.209} & 3.476 \\
    \midrule
    \textbf{Initialization} &  &  &  &  &  \\
    Init=EDM $W=1$ & 1.176 & 0.977 & -0.240 & 0.309 & 0.134 \\
    Init=\texttt{persistence} & 14.789 & 172.233 & -25.361 & \textcolor{red}{-20.412} & 0.765 \\
    Init=\texttt{truth} & \textbf{0.899} & 0.593 & \textbf{0.053} & \textbf{0.492} & 0.286 \\
    \bottomrule
  \end{tabular}
\end{table}

\begin{thebibliography}{55}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
  \providecommand{\doi}[1]{doi: #1}\else
  \providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi

\bibitem[Andrae et~al.(2025)Andrae, Landelius, Oskarsson, and Lindsten]{andrae2025continuous}
Martin Andrae, Tomas Landelius, Joel Oskarsson, and Fredrik Lindsten.
\newblock Continuous ensemble weather forecasting with diffusion models.
\newblock \emph{International Conference on Learning Representations}, 2025.

\bibitem[Bauer et~al.(2015)Bauer, Thorpe, and Brunet]{bauer2015thequiet}
Peter Bauer, Alan Thorpe, and Gilbert Brunet.
\newblock The quiet revolution of numerical weather prediction.
\newblock \emph{Nature}, 525\penalty0 (7567):\penalty0 47--55, Sep 2015.
\newblock ISSN 1476-4687.
\newblock \doi{10.1038/nature14956}.

\bibitem[Bevacqua et~al.(2023)Bevacqua, Suarez-Gutierrez, J{\'e}z{\'e}quel, Lehner, Vrac, Yiou, and Zscheischler]{bevacqua2023smiles}
Emanuele Bevacqua, Laura Suarez-Gutierrez, Agla{\'e} J{\'e}z{\'e}quel, Flavio Lehner, Mathieu Vrac, Pascal Yiou, and Jakob Zscheischler.
\newblock Advancing research on compound weather and climate events via large ensemble model simulations.
\newblock \emph{Nature Communications}, 14\penalty0 (1):\penalty0 2145, April 2023.

\bibitem[Bi et~al.(2023)Bi, Xie, Zhang, Chen, Gu, and Tian]{bi2023accurate}
Kaifeng Bi, Lingxi Xie, Hengheng Zhang, Xin Chen, Xiaotao Gu, and Qi~Tian.
\newblock Accurate medium-range global weather forecasting with {3D} neural networks.
\newblock \emph{Nature}, 619\penalty0 (7970):\penalty0 533--538, 2023.
\newblock \doi{10.1038/s41586-023-06185-3}.

\bibitem[Blattmann et~al.(2023)Blattmann, Dockhorn, Kulal, Mendelevitch, Kilian, Lorenz, Levi, English, Voleti, Letts, Jampani, and Rombach]{blattmann2023stablevideodiffusion}
Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, Varun Jampani, and Robin Rombach.
\newblock Stable video diffusion: Scaling latent video diffusion models to large datasets.
\newblock 2023.
\newblock URL \url{https://arxiv.org/abs/2311.15127}.

\bibitem[Bodnar et~al.(2024)Bodnar, Bruinsma, Lucic, Stanley, Allen, Brandstetter, Garvan, Riechert, Weyn, Dong, Gupta, Thambiratnam, Archibald, Wu, Heider, Welling, Turner, and Perdikaris]{bodnar2024aurora}
Cristian Bodnar, Wessel~P. Bruinsma, Ana Lucic, Megan Stanley, Anna Allen, Johannes Brandstetter, Patrick Garvan, Maik Riechert, Jonathan~A. Weyn, Haiyu Dong, Jayesh~K. Gupta, Kit Thambiratnam, Alexander~T. Archibald, Chun-Chieh Wu, Elizabeth Heider, Max Welling, Richard~E. Turner, and Paris Perdikaris.
\newblock Aurora: A foundation model for the earth system, 2024.
\newblock URL \url{https://arxiv.org/abs/2405.13063}.

\bibitem[Bouallègue et~al.(2024)Bouallègue, Clare, Magnusson, Gascón, Maier-Gerber, Janoušek, Rodwell, Pinault, Dramsch, Lang, Raoult, Rabier, Chevallier, Sandu, Dueben, Chantry, and Pappenberger]{bouallegue2024rise}
Zied~Ben Bouallègue, Mariana C.~A. Clare, Linus Magnusson, Estibaliz Gascón, Michael Maier-Gerber, Martin Janoušek, Mark Rodwell, Florian Pinault, Jesper~S. Dramsch, Simon T.~K. Lang, Baudouin Raoult, Florence Rabier, Matthieu Chevallier, Irina Sandu, Peter Dueben, Matthew Chantry, and Florian Pappenberger.
\newblock The rise of data-driven weather forecasting: A first statistical assessment of machine learning–based weather forecasts in an operational-like context.
\newblock \emph{Bulletin of the American Meteorological Society}, 105\penalty0 (6):\penalty0 E864 -- E883, 2024.
\newblock \doi{10.1175/BAMS-D-23-0162.1}.

\bibitem[Chang et~al.(2024)Chang, Tang, Gross, and Azevedo]{chang2024warpednoise}
Pascal Chang, Jingwei Tang, Markus Gross, and Vinicius~C. Azevedo.
\newblock How i warped your noise: a temporally-correlated noise prior for diffusion models.
\newblock \emph{International Conference on Learning Representations}, 2024.
\newblock URL \url{https://openreview.net/forum?id=pzElnMrgSD}.

\bibitem[Chen et~al.(2024)Chen, Monso, Du, Simchowitz, Tedrake, and Sitzmann]{chen2024diffusionforcing}
Boyuan Chen, Diego~Marti Monso, Yilun Du, Max Simchowitz, Russ Tedrake, and Vincent Sitzmann.
\newblock Diffusion forcing: Next-token prediction meets full-sequence diffusion.
\newblock \emph{Advances in Neural Information Processing Systems}, 2024.

\bibitem[Chen et~al.(2023)Chen, Du, Hu, Wang, and Wang]{chen2023swinrdm}
Lei Chen, Fei Du, Yuan Hu, Zhibin Wang, and Fan Wang.
\newblock Swinrdm: integrate swinrnn with diffusion model towards high-resolution and high-quality weather forecasting.
\newblock In \emph{Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence}, 2023.
\newblock \doi{10.1609/aaai.v37i1.25105}.

\bibitem[Clark et~al.(2001)Clark, Carpenter, Barber, Collins, Dobson, Foley, Lodge, Pascual, Pielke, Pizer, Pringle, Reid, Rose, Sala, Schlesinger, Wall, and Wear]{clark2001ecology}
James~S. Clark, Steven~R. Carpenter, Mary Barber, Scott Collins, Andy Dobson, Jonathan~A. Foley, David~M. Lodge, Mercedes Pascual, Roger Pielke, William Pizer, Cathy Pringle, Walter~V. Reid, Kenneth~A. Rose, Osvaldo Sala, William~H. Schlesinger, Diana~H. Wall, and David Wear.
\newblock Ecological forecasts: An emerging imperative.
\newblock \emph{Science}, 293\penalty0 (5530):\penalty0 657--660, 2001.
\newblock \doi{10.1126/science.293.5530.657}.

\bibitem[Dhariwal and Nichol(2021)]{dhariwal2021diffusion}
Prafulla Dhariwal and Alex Nichol.
\newblock Diffusion models beat {GAN}s on image synthesis.
\newblock \emph{Advances in Neural Information Processing Systems}, 2021.
\newblock \doi{10.48550/arxiv.2105.05233}.

\bibitem[ECMWF(2019)]{ifs-manual-cy46r1-ens}
ECMWF.
\newblock \emph{IFS Documentation CY46R1 - Part V: Ensemble Prediction System}.
\newblock 2019.
\newblock \doi{10.21957/38yug0cev}.

\bibitem[Fortin et~al.(2014)Fortin, Abaza, Anctil, and Turcotte]{fortin2014ssr}
V.~Fortin, M.~Abaza, F.~Anctil, and R.~Turcotte.
\newblock Why should ensemble spread match the rmse of the ensemble mean?
\newblock \emph{Journal of Hydrometeorology}, 15\penalty0 (4):\penalty0 1708 -- 1713, 2014.
\newblock \doi{https://doi.org/10.1175/JHM-D-14-0008.1}.

\bibitem[Ge et~al.(2024)Ge, Nah, Liu, Poon, Tao, Catanzaro, Jacobs, Huang, Liu, and Balaji]{ge2024noiseprior}
Songwei Ge, Seungjun Nah, Guilin Liu, Tyler Poon, Andrew Tao, Bryan Catanzaro, David Jacobs, Jia-Bin Huang, Ming-Yu Liu, and Yogesh Balaji.
\newblock Preserve your own correlation: A noise prior for video diffusion models.
\newblock \emph{ICCV}, 2024.

\bibitem[Gneiting and Katzfuss(2014)]{gneiting2014Probabilistic}
Tilmann Gneiting and Matthias Katzfuss.
\newblock Probabilistic forecasting.
\newblock \emph{Annual Review of Statistics and Its Application}, 1\penalty0 (1):\penalty0 125–151, January 2014.
\newblock ISSN 2326-831X.
\newblock \doi{10.1146/annurev-statistics-062713-085831}.

\bibitem[Harvey et~al.(2022)Harvey, Naderiparizi, Masrani, Weilbach, and Wood]{harvey2022flexiblevideos}
William Harvey, Saeid Naderiparizi, Vaden Masrani, Christian Weilbach, and Frank Wood.
\newblock Flexible diffusion modeling of long videos.
\newblock \emph{Advances in Neural Information Processing Systems}, 2022.
\newblock \doi{10.48550/arxiv.2205.11495}.

\bibitem[Ho et~al.(2020)Ho, Jain, and Abbeel]{ho2020ddpm}
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
\newblock Denoising diffusion probabilistic models.
\newblock \emph{Advances in Neural Information Processing Systems}, 2020.
\newblock \doi{10.48550/arxiv.2006.11239}.

\bibitem[Ho et~al.(2022)Ho, Chan, Saharia, Whang, Gao, Gritsenko, Kingma, Poole, Norouzi, Fleet, and Salimans]{ho2022imagenvideo}
Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik~P. Kingma, Ben Poole, Mohammad Norouzi, David~J. Fleet, and Tim Salimans.
\newblock Imagen video: High definition video generation with diffusion models.
\newblock \emph{arXiv}, 2022.
\newblock \doi{10.48550/arxiv.2210.02303}.

\bibitem[Huang et~al.(2024)Huang, Gianinazzi, Yu, Dueben, and Hoefler]{huang2024diffda}
Langwen Huang, Lukas Gianinazzi, Yuejiang Yu, Peter~Dominik Dueben, and Torsten Hoefler.
\newblock Diff{DA}: a diffusion model for weather-scale data assimilation.
\newblock \emph{International Conference on Machine Learning}, 2024.
\newblock URL \url{https://openreview.net/forum?id=vhMq3eAB34}.

\bibitem[Hyv{{\"a}}rinen(2005)]{hyvarinen05score}
Aapo Hyv{{\"a}}rinen.
\newblock Estimation of non-normalized statistical models by score matching.
\newblock \emph{Journal of Machine Learning Research}, 6\penalty0 (24):\penalty0 695--709, 2005.
\newblock URL \url{http://jmlr.org/papers/v6/hyvarinen05a.html}.

\bibitem[Karlbauer et~al.(2024)Karlbauer, Maddix, Ansari, Han, Gupta, Wang, Stuart, and Mahoney]{karlbauer2024comparing}
Matthias Karlbauer, Danielle~C. Maddix, Abdul~Fatir Ansari, Boran Han, Gaurav Gupta, Yuyang Wang, Andrew Stuart, and Michael~W. Mahoney.
\newblock Comparing and contrasting deep learning weather prediction backbones on navier-stokes and atmospheric dynamics.
\newblock 2024.
\newblock URL \url{https://arxiv.org/abs/2407.14129}.

\bibitem[Karras et~al.(2022)Karras, Aittala, Aila, and Laine]{karras2022edm}
Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine.
\newblock Elucidating the design space of diffusion-based generative models.
\newblock \emph{Advances in Neural Information Processing Systems}, 2022.
\newblock \doi{10.48550/arxiv.2206.00364}.

\bibitem[Kim et~al.(2024)Kim, Kang, Choi, and Han]{kim2024fifodiffusion}
Jihwan Kim, Junoh Kang, Jinyoung Choi, and Bohyung Han.
\newblock {FIFO}-diffusion: Generating infinite videos from text without training.
\newblock \emph{Advances in Neural Information Processing Systems}, 2024.
\newblock URL \url{https://openreview.net/forum?id=uikhNa4wam}.

\bibitem[Kochkov et~al.(2024)Kochkov, Yuval, Langmore, Norgaard, Smith, Mooers, Klöwer, Lottes, Rasp, Düben, Hatfield, Battaglia, Sanchez-Gonzalez, Willson, Brenner, and Hoyer]{kochkov2023neural}
Dmitrii Kochkov, Janni Yuval, Ian Langmore, Peter Norgaard, Jamie Smith, Griffin Mooers, Milan Klöwer, James Lottes, Stephan Rasp, Peter Düben, Sam Hatfield, Peter Battaglia, Alvaro Sanchez-Gonzalez, Matthew Willson, Michael~P. Brenner, and Stephan Hoyer.
\newblock Neural general circulation models for weather and climate.
\newblock \emph{Nature}, 632\penalty0 (8027):\penalty0 1060–1066, July 2024.
\newblock ISSN 1476-4687.
\newblock \doi{10.1038/s41586-024-07744-y}.

\bibitem[Kohl et~al.(2023)Kohl, Chen, and Thuerey]{kohl2023benchmarking}
Georg Kohl, Liwei Chen, and Nils Thuerey.
\newblock Benchmarking autoregressive conditional diffusion models for turbulent flow simulation.
\newblock 2023.

\bibitem[Lam et~al.(2023)Lam, Sanchez-Gonzalez, Willson, Wirnsberger, Fortunato, Alet, Ravuri, Ewalds, Eaton-Rosen, Hu, Merose, Hoyer, Holland, Vinyals, Stott, Pritzel, Mohamed, and Battaglia]{lam2023learning}
Remi Lam, Alvaro Sanchez-Gonzalez, Matthew Willson, Peter Wirnsberger, Meire Fortunato, Ferran Alet, Suman Ravuri, Timo Ewalds, Zach Eaton-Rosen, Weihua Hu, Alexander Merose, Stephan Hoyer, George Holland, Oriol Vinyals, Jacklynn Stott, Alexander Pritzel, Shakir Mohamed, and Peter Battaglia.
\newblock Learning skillful medium-range global weather forecasting.
\newblock \emph{Science}, 382\penalty0 (6677):\penalty0 1416–1421, 2023.
\newblock ISSN 1095-9203.
\newblock \doi{10.1126/science.adi2336}.

\bibitem[Li et~al.(2024)Li, Carver, Lopez-Gomez, Sha, and Anderson]{li2024genemulation}
Lizao Li, Robert Carver, Ignacio Lopez-Gomez, Fei Sha, and John Anderson.
\newblock Generative emulation of weather forecast ensembles with diffusion models.
\newblock \emph{Science Advances}, 10\penalty0 (13):\penalty0 eadk4489, 2024.
\newblock \doi{10.1126/sciadv.adk4489}.
\newblock URL \url{https://www.science.org/doi/abs/10.1126/sciadv.adk4489}.

\bibitem[Lippe et~al.(2023)Lippe, Veeling, Perdikaris, Turner, and Brandstetter]{lippe2023pderefiner}
Phillip Lippe, Bastiaan~S. Veeling, Paris Perdikaris, Richard~E Turner, and Johannes Brandstetter.
\newblock {PDE-Refiner}: Achieving accurate long rollouts with temporal neural pde solvers.
\newblock \emph{Advances in Neural Information Processing Systems}, 2023.
\newblock \doi{10.48550/arxiv.2308.05732}.

\bibitem[Liu and Vahdat(2025)]{liu2025equivdm}
Chao Liu and Arash Vahdat.
\newblock Equivdm: Equivariant video diffusion models with temporally consistent noise.
\newblock 2025.

\bibitem[Lopez-Gomez et~al.(2025)Lopez-Gomez, Wan, Zepeda-Núñez, Schneider, Anderson, and Sha]{lopez2025dynamical}
Ignacio Lopez-Gomez, Zhong~Yi Wan, Leonardo Zepeda-Núñez, Tapio Schneider, John Anderson, and Fei Sha.
\newblock Dynamical-generative downscaling of climate model ensembles.
\newblock \emph{Proceedings of the National Academy of Sciences}, 122\penalty0 (17):\penalty0 e2420288122, 2025.
\newblock \doi{10.1073/pnas.2420288122}.

\bibitem[Manshausen et~al.(2025)Manshausen, Cohen, Harrington, Pathak, Pritchard, Garg, Mardani, Kashinath, Byrne, and Brenowitz]{manshausen2025generative}
Peter Manshausen, Yair Cohen, Peter Harrington, Jaideep Pathak, Mike Pritchard, Piyush Garg, Morteza Mardani, Karthik Kashinath, Simon Byrne, and Noah Brenowitz.
\newblock Generative data assimilation of sparse weather station observations at kilometer scales.
\newblock 2025.
\newblock URL \url{https://arxiv.org/abs/2406.16947}.

\bibitem[Mardani et~al.(2025)Mardani, Brenowitz, Cohen, Pathak, Chen, Liu, Vahdat, Nabian, Ge, Subramaniam, Kashinath, Kautz, and Pritchard]{mardani2025corrdiff}
Morteza Mardani, Noah Brenowitz, Yair Cohen, Jaideep Pathak, Chieh-Yu Chen, Cheng-Chin Liu, Arash Vahdat, Mohammad~Amin Nabian, Tao Ge, Akshay Subramaniam, Karthik Kashinath, Jan Kautz, and Mike Pritchard.
\newblock Residual corrective diffusion modeling for km-scale atmospheric downscaling.
\newblock \emph{Communications Earth \& Environment}, 6\penalty0 (1), February 2025.
\newblock ISSN 2662-4435.
\newblock \doi{10.1038/s43247-025-02042-5}.

\bibitem[Matheson and Winkler(1976)]{matheson1976crps}
James~E. Matheson and Robert~L. Winkler.
\newblock Scoring rules for continuous probability distributions.
\newblock \emph{Management Science}, 22\penalty0 (10):\penalty0 1087--1096, 1976.

\bibitem[Nguyen et~al.(2023)Nguyen, Brandstetter, Kapoor, Gupta, and Grover]{nguyen2023climax}
Tung Nguyen, Johannes Brandstetter, Ashish Kapoor, Jayesh~K Gupta, and Aditya Grover.
\newblock Clima{X}: A foundation model for weather and climate.
\newblock \emph{International Conference on Machine Learning}, 2023.
\newblock \doi{10.48550/arxiv.2301.10343}.

\bibitem[Nguyen et~al.(2024)Nguyen, Shah, Bansal, Arcomano, Madireddy, Maulik, Kotamarthi, Foster, and Grover]{nguyen2023stormer}
Tung Nguyen, Rohan Shah, Hritik Bansal, Troy Arcomano, Sandeep Madireddy, Romit Maulik, Veerabhadra Kotamarthi, Ian Foster, and Aditya Grover.
\newblock Scaling transformer neural networks for skillful and reliable medium-range weather forecasting.
\newblock \emph{Advances in Neural Information Processing Systems}, 2024.
\newblock \doi{10.48550/arxiv.2312.03876}.

\bibitem[Oskarsson et~al.(2024)Oskarsson, Landelius, Deisenroth, and Lindsten]{oskarsson2024probabilistic}
Joel Oskarsson, Tomas Landelius, Marc~Peter Deisenroth, and Fredrik Lindsten.
\newblock Probabilistic weather forecasting with hierarchical graph neural networks.
\newblock \emph{Advances in Neural Information Processing Systems}, 2024.
\newblock URL \url{https://openreview.net/forum?id=wTIzpqX121}.

\bibitem[Otness et~al.(2021)Otness, Gjoka, Bruna, Panozzo, Peherstorfer, Schneider, and Zorin]{otness21nnbenchmark}
Karl Otness, Arvi Gjoka, Joan Bruna, Daniele Panozzo, Benjamin Peherstorfer, Teseo Schneider, and Denis Zorin.
\newblock An extensible benchmark suite for learning to simulate physical systems.
\newblock \emph{Advances in Neural Information Processing Systems Track on Datasets and Benchmarks}, 2021.

\bibitem[Palmer(2002)]{palmer2002economic}
T.~N. Palmer.
\newblock The economic value of ensemble forecasts as a tool for risk assessment: From days to decades.
\newblock \emph{Quarterly Journal of the Royal Meteorological Society}, 128\penalty0 (581):\penalty0 747--774, 2002.
\newblock \doi{https://doi.org/10.1256/0035900021643593}.

\bibitem[Pathak et~al.(2022)Pathak, Subramanian, Harrington, Raja, Chattopadhyay, Mardani, Kurth, Hall, Li, Azizzadenesheli, Hassanzadeh, Kashinath, and Anandkumar]{pathak2022fourcastnet}
Jaideep Pathak, Shashank Subramanian, Peter Harrington, Sanjeev Raja, Ashesh Chattopadhyay, Morteza Mardani, Thorsten Kurth, David Hall, Zongyi Li, Kamyar Azizzadenesheli, Pedram Hassanzadeh, Karthik Kashinath, and Anima Anandkumar.
\newblock {FourCastNet: Accelerating global high-resolution weather forecasting using adaptive Fourier neural operators}.
\newblock \emph{Proceedings of the National Academy of Sciences (PNAS)}, 119\penalty0 (48):\penalty0 e2208655119, 2022.

\bibitem[Pathak et~al.(2024)Pathak, Cohen, Garg, Harrington, Brenowitz, Durran, Mardani, Vahdat, Xu, Kashinath, and Pritchard]{pathak2024stormcast}
Jaideep Pathak, Yair Cohen, Piyush Garg, Peter Harrington, Noah Brenowitz, Dale Durran, Morteza Mardani, Arash Vahdat, Shaoming Xu, Karthik Kashinath, and Michael Pritchard.
\newblock Kilometer-scale convection allowing model emulation using generative diffusion modeling, 2024.
\newblock URL \url{https://arxiv.org/abs/2408.10958}.

\bibitem[Price et~al.(2024)Price, Sanchez-Gonzalez, Alet, Andersson, El-Kadi, Masters, Ewalds, Stott, Mohamed, Battaglia, Lam, and Willson]{price2023gencast}
Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet, Tom~R. Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson.
\newblock Probabilistic weather forecasting with machine learning.
\newblock \emph{Nature}, 637\penalty0 (8044):\penalty0 84–90, December 2024.
\newblock ISSN 1476-4687.
\newblock \doi{10.1038/s41586-024-08252-9}.
\newblock URL \url{http://dx.doi.org/10.1038/s41586-024-08252-9}.

\bibitem[Rasp et~al.(2024)Rasp, Hoyer, Merose, Langmore, Battaglia, Russell, Sanchez‐Gonzalez, Yang, Carver, Agrawal, Chantry, Ben~Bouallegue, Dueben, Bromberg, Sisk, Barrington, Bell, and Sha]{rasp2023weatherbench2}
Stephan Rasp, Stephan Hoyer, Alexander Merose, Ian Langmore, Peter Battaglia, Tyler Russell, Alvaro Sanchez‐Gonzalez, Vivian Yang, Rob Carver, Shreya Agrawal, Matthew Chantry, Zied Ben~Bouallegue, Peter Dueben, Carla Bromberg, Jared Sisk, Luke Barrington, Aaron Bell, and Fei Sha.
\newblock {WeatherBench} 2: A benchmark for the next generation of data‐driven global weather models.
\newblock \emph{Journal of Advances in Modeling Earth Systems}, 16\penalty0 (6), June 2024.
\newblock ISSN 1942-2466.
\newblock \doi{10.1029/2023ms004019}.

\bibitem[Rombach et~al.(2022)Rombach, Blattmann, Lorenz, Esser, and Ommer]{rombach2022stablediffusion}
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj{\"o}rn Ommer.
\newblock High-resolution image synthesis with latent diffusion models.
\newblock pages 10684--10695, 2022.

\bibitem[Rozet and Louppe(2023)]{rozet2023scorebased}
Fran{\c{c}}ois Rozet and Gilles Louppe.
\newblock Score-based data assimilation.
\newblock \emph{Advances in Neural Information Processing Systems}, 2023.
\newblock URL \url{https://openreview.net/forum?id=VUvLSnMZdX}.

\bibitem[Ruhe et~al.(2024)Ruhe, Heek, Salimans, and Hoogeboom]{ruhe2024rolling}
David Ruhe, Jonathan Heek, Tim Salimans, and Emiel Hoogeboom.
\newblock Rolling diffusion models.
\newblock \emph{International Conference on Machine Learning}, 2024.
\newblock URL \url{https://openreview.net/forum?id=a9bzTv9SzO}.

\bibitem[R{\"u}hling~Cachay et~al.(2023)R{\"u}hling~Cachay, Zhao, Joren, and Yu]{cachay2023dyffusion}
Salva R{\"u}hling~Cachay, Bo~Zhao, Hailey Joren, and Rose Yu.
\newblock {DYffusion}: A dynamics-informed diffusion model for spatiotemporal forecasting.
\newblock \emph{Advances in Neural Information Processing Systems}, 2023.
\newblock \doi{10.48550/arxiv.2306.01984}.

\bibitem[R{\"u}hling~Cachay et~al.(2024)R{\"u}hling~Cachay, Henn, Watt-Meyer, Bretherton, and Yu]{cachay2024probemulation}
Salva R{\"u}hling~Cachay, Brian Henn, Oliver Watt-Meyer, Christopher~S. Bretherton, and Rose Yu.
\newblock Probabilistic emulation of a global climate model with {Spherical DYffusion}.
\newblock \emph{Advances in Neural Information Processing Systems}, 2024.

\bibitem[Sohl-Dickstein et~al.(2015)Sohl-Dickstein, Weiss, Maheswaranathan, and Ganguli]{sohldickstein2015deepunsupervised}
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli.
\newblock Deep unsupervised learning using nonequilibrium thermodynamics.
\newblock \emph{International Conference on Machine Learning}, 2015.
\newblock \doi{10.48550/arxiv.1503.03585}.

\bibitem[Song et~al.(2021{\natexlab{a}})Song, Meng, and Ermon]{song2021ddim}
Jiaming Song, Chenlin Meng, and Stefano Ermon.
\newblock Denoising diffusion implicit models.
\newblock \emph{International Conference on Learning Representations}, 2021{\natexlab{a}}.

\bibitem[Song et~al.(2021{\natexlab{b}})Song, Sohl-Dickstein, Kingma, Kumar, Ermon, and Poole]{song2021scoreSDE}
Yang Song, Jascha Sohl-Dickstein, Diederik~P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.
\newblock Score-based generative modeling through stochastic differential equations.
\newblock \emph{International Conference on Learning Representations}, 2021{\natexlab{b}}.
\newblock \doi{10.48550/arxiv.2011.13456}.

\bibitem[Srivastava et~al.(2024)Srivastava, Yang, Kerrigan, Dresdner, McGibbon, Bretherton, and Mandt]{srivastava2024precip}
Prakhar Srivastava, Ruihan Yang, Gavin Kerrigan, Gideon Dresdner, Jeremy~J McGibbon, Christopher~S. Bretherton, and Stephan Mandt.
\newblock Precipitation downscaling with spatiotemporal video diffusion.
\newblock \emph{Advances in Neural Information Processing Systems}, 2024.

\bibitem[Vahdat et~al.(2021)Vahdat, Kreis, and Kautz]{vahdat2021ldm}
Arash Vahdat, Karsten Kreis, and Jan Kautz.
\newblock Score-based generative modeling in latent space.
\newblock \emph{Advances in Neural Information Processing Systems}, 2021.
\newblock URL \url{https://proceedings.neurips.cc/paper_files/paper/2021/file/5dca4c6b9e244d24a30b4c45601d9720-Paper.pdf}.

\bibitem[Wu et~al.(2023)Wu, Fan, Liu, Gong, Shen, Jiao, Zheng, Li, Wei, Guo, Duan, and Chen]{wu2023ardiffusion}
Tong Wu, Zhihao Fan, Xiao Liu, Yeyun Gong, Yelong Shen, Jian Jiao, Hai-Tao Zheng, Juntao Li, Zhongyu Wei, Jian Guo, Nan Duan, and Weizhu Chen.
\newblock {AR-Diffusion:} auto-regressive diffusion model for text generation.
\newblock \emph{Advances in Neural Information Processing Systems}, 2023.

\bibitem[Zamo and Naveau(2018)]{zamo2018faircrps}
Micha{\"e}l Zamo and Philippe Naveau.
\newblock Estimation of the continuous ranked probability score with limited information and applications to ensemble weather forecasts.
\newblock \emph{Mathematical Geosciences}, 50\penalty0 (2):\penalty0 209--234, 2018.
\newblock \doi{10.1007/s11004-017-9709-7}.

\end{thebibliography}