An adaptive and evolvable deep reinforcement learning framework for weather prediction

The framework maintains a library of pretrained models $\mathcal{M}=\{M^{(1)},M^{(2)},\ldots,M^{(N_{\mathrm{pool}})}\}$ drawn from diverse architectural families: Transformer-based models (Pangu-Weather[9], Aurora[17], FuXi[12]), graph-based models (GraphCast[10], AIFS[19]), diffusion-based probabilistic models (GenCast[14]), and neural-operator or spectral-operator models (FourCastNet3[15], SFNO[51]). At any time an active subset $\mathcal{M}_{\mathrm{active}}\subseteq\mathcal{M}$ of $N$ models participates in forecast generation. Each model is treated as a black-box forecaster through a common input–output interface, so the eight pretrained models can be evaluated and combined without modifying or retraining their internal parameters.

The Weight-Agent produces fusion weights conditioned on the expert predictions, the atmospheric state, and the forecast lead time. At initialisation time $t$ for target lead time $\tau$, each active model $M^{(i)}\in\mathcal{M}_{\mathrm{active}}$ produces a forecast $\hat{X}^{(i)}_{t+\tau}$ from the initial conditions $X_{t}$. The Weight-Agent receives the concatenated expert predictions $\{\hat{X}^{(1)}_{t+\tau},\ldots,\hat{X}^{(N)}_{t+\tau}\}$ together with $X_{t}$ and assigns a weight $W^{(i)}$ to each model. Weight generation is framed as a reinforcement learning problem: the agent observes the expert forecasts and the ERA5 verification state, selects weight configurations as actions, receives a reward based on forecast skill, and updates its policy to maximise expected return. A convolutional encoder captures multiscale atmospheric patterns and inter-model discrepancies; a Softmax layer enforces the simplex constraint so that weights sum to one at every grid point. An asynchronous caching mechanism stores precomputed model outputs, decoupling policy updates from the differing inference speeds of constituent models. FTAE-Weather supports two fusion granularities.

The Evolve-Agent updates $\mathcal{M}_{\mathrm{active}}$ on a slower timescale to keep the model pool current. Its signal comes directly from the Weight-Agent: models that consistently receive high weights demonstrate practical value, while those assigned low weights contribute little. After each evaluation epoch, the Evolve-Agent ranks models by their fitness score $\mathcal{F}_{i}$ (Eq. 2) and applies a prune-and-inject protocol that removes the lowest-ranked models and replaces them with candidates from the reserve pool $\mathcal{M}\setminus\mathcal{M}_{\mathrm{active}}$. This feedback loop ties tactical weight allocation to strategic model curation, allowing the system to retire outdated models and incorporate newly released architectures without manual intervention.

At the 24 h lead time, Aurora achieves the lowest RMSE across all evaluated variables (Fig. 2). For geopotential height, its errors for z500 and z850 are clearly lower than those of Pangu6, FuXi, and the two direct FTAE variants. Similar behavior is observed for temperature, wind, and humidity variables. In contrast, FTAE-Unify exhibits relatively large errors at this short lead time, especially for z500, z850, and wind-related variables. FTAE-Refine reduces the error relative to FTAE-Unify but still does not outperform the best individual model. This indicates that, at short forecast horizons, model errors have not yet diverged sufficiently for fusion to provide clear additional benefits.

At the 72 h lead time, the model ranking becomes more variable-dependent (Fig. 3). Pangu6 performs well for geopotential height and several thermodynamic variables, whereas FTAE-Cascade begins to achieve lower errors for wind and humidity variables, including u10m, v10m, q500, u500, and v500. FTAE-Refine also substantially reduces the errors of FTAE-Unify and becomes competitive with the baseline models for some variables. However, its advantage is not yet consistent across all variables. These results suggest that model complementarity starts to emerge at this lead time, but the best individual model remains highly competitive for geopotential height and temperature-related variables.

At the 120 h lead time, the benefit of fusion becomes more evident (Fig. 4). FTAE-Refine improves upon the best baseline model for most variables, indicating that the refined fusion strategy can exploit complementary information among different models. FTAE-Cascade further strengthens this advantage and achieves the lowest RMSE for 9 of the 10 evaluated variables, with t2m being the only exception. Its improvement is particularly clear for geopotential height, wind, and humidity variables. This suggests that, at approximately five days of forecast lead time, the advantage of any single model becomes less stable, while cascade fusion can more effectively integrate variable-dependent model strengths.

At the 168 h lead time, the advantage of cascade-based fusion becomes more pronounced (Fig. 5). Among the baseline models, FuXi performs relatively well for most variables, while AIFS and Pangu6 remain competitive for selected variables. In contrast, FTAE-Unify and FTAE-Refine do not consistently outperform the best baseline model at this lead time. In particular, FTAE-Refine shows notably larger errors for geopotential height, suggesting that direct fusion or one-step refined fusion may still be affected by systematic model biases and accumulated forecast errors at 168 h. By contrast, FTAE-Cascade achieves the lowest RMSE across all evaluated variables. Its errors for z500 and z850 are substantially lower than those of both the baseline models and the direct fusion variants, and it also shows clear advantages for temperature, wind, and humidity variables. These results indicate that, at the 7-day forecast range, model error structures have become sufficiently differentiated, allowing cascade fusion to more effectively exploit inter-model complementarity and suppress error growth in both dynamical and thermodynamic fields.

At the 240 h lead time, errors in the baseline models increase substantially, especially for geopotential height and upper-level wind variables (Fig. 6). Among the baseline models, FuXi remains relatively competitive for several variables, but it is consistently outperformed by FTAE-Cascade. Cascade achieves the lowest RMSE across all 10 evaluated variables, including z500, z850, temperature, humidity, and wind variables. FTAE-Refine also improves upon the best baseline model for most variables, although its error reduction remains smaller than that of Cascade. These results indicate that cascade-based fusion provides stronger control of accumulated forecast errors at long lead times.

At the 360 h lead time, the performance gap between FTAE-Cascade and the other methods becomes more pronounced (Fig. 7). Although FuXi remains one of the stronger baseline models, it does not achieve the lowest RMSE for any evaluated variable. FTAE-Cascade maintains the lowest errors for geopotential height, temperature, humidity, and wind variables. By comparison, FTAE-Unify and FTAE-Refine do not consistently outperform the best baseline model at this lead time, suggesting that direct fusion alone may still be affected by accumulated error propagation and systematic model biases in ultra-long-range forecasts.

Across different forecast lead times, the evaluation results reveal a clear lead-time dependence in the benefits of multi-model fusion. At the short lead time of H24, the best individual model, Aurora, remains dominant, indicating that a single high-performing model still benefits from strong initial-condition constraints and short-range stability. By H72, inter-model complementarity begins to emerge: FTAE-Cascade shows advantages for wind and humidity variables, while lower errors in geopotential height and temperature variables are still mainly achieved by the best baseline models. After H120, the benefits of fusion become more evident, with FTAE-Cascade outperforming the baseline models for most variables. This suggests that, as the forecast horizon extends, the error structures of different models become increasingly differentiated, allowing fusion strategies to more effectively exploit complementary information across models.

At the H168 lead time, the advantage of cascade fusion further increases. FTAE-Cascade achieves the lowest RMSE across all evaluated variables, with particularly clear error reduction for geopotential height, wind, and humidity variables. When the lead time is further extended to H240 and H360, the errors of the baseline models continue to increase, whereas FTAE-Cascade maintains the lowest RMSE for all variables, indicating more stable error control in medium- to long-range forecasting.

These results indicate that multi-model fusion does not necessarily outperform the best individual model at all forecast lead times. Its primary advantage is concentrated in medium- to long-range prediction. As the forecast horizon increases, different models exhibit increasingly distinct error-growth characteristics and systematic biases across variables. The cascade fusion strategy can better exploit this complementarity, thereby suppressing accumulated errors in multi-variable forecasts and improving the overall stability of long-range prediction.

The global atmospheric state at time $t$ is represented as $X_{t}\in\mathbb{R}^{H\times W\times V}$ on a $0.25^{\circ}$ latitude–longitude grid ($H{=}721$, $W{=}1440$, $V{=}69$ variables). Given initialisation time $t_{0}$, the goal is to predict $X_{t_{0}+\tau}$ at lead time $\tau\in\{6,12,24,72,120,168,240,360\}$ h. We assume access to a pool of pretrained forecasters $\mathcal{M}=\{M^{(1)},\dots,M^{(N_{\mathrm{pool}})}\}$, each mapping $\mathbb{R}^{H\times W\times V}\rightarrow\mathbb{R}^{H\times W\times V}$. An active subset $\mathcal{M}_{\mathrm{active}}\subseteq\mathcal{M}$ with $N=|\mathcal{M}_{\mathrm{active}}|$ models participates at any given time. The fusion is parameterised by $\Phi=\{W^{(1)},\dots,W^{(N)},b\}$, where $W^{(i)}\in\mathbb{R}^{H\times W\times V}$ is the weight field for model $i$ and $b\in\mathbb{R}^{H\times W\times V}$ is an additive bias correction. The fused forecast is

where $\odot$ denotes element-wise multiplication, $W^{(i)}_{t_{0}}(h,w,v)\geq 0$ for all spatial locations $(h,w)$ and variables $v$, with the simplex constraint $\sum_{i=1}^{N}W^{(i)}_{t_{0}}(h,w,v)=1$ enforced at each grid point and variable.

Optimisation proceeds on two timescales. At the tactical level, weight generation is cast as an MDP in which a policy network $\pi_{\phi}$ produces $\Phi$ conditioned on the atmospheric state and receives verification-based rewards. At the strategic level, $\mathcal{M}_{\mathrm{active}}$ is updated periodically. Model utility is measured by the tactical controller’s revealed preference, that is, the accumulated weight allocation over an evaluation window $T_{\mathrm{eval}}$, which defines a fitness score

where $\langle W^{(i)}_{t}\rangle=\frac{1}{V}\sum_{v=1}^{V}\bar{W}^{(i)}_{t}(v)$ denotes the variable-averaged weight assigned to expert $i$ at time $t$. This fitness metric drives a prune-and-inject protocol that updates the active expert set.

All experiments use the ERA5 reanalysis[7] at $0.25^{\circ}$ resolution. Upper-air variables (geopotential height, temperature $T$, specific humidity $q$, wind components $u$, $v$) are included at 13 pressure levels ($p\in\{50,100,150,200,250,300,400,500,600,700,850,925,1000\}$ hPa), alongside surface fields ($T_{2m}$, $u_{10}$, $v_{10}$, mean sea-level pressure). Data are split chronologically: 1980–2017 for training, 2018–2019 for validation, and 2020–present for testing. ACC is computed against a daily climatology over 1991–2020, smoothed with a 7-day centred moving average. All input fields are standardised to zero mean and unit variance using training-period statistics only.

Forecast skill is measured by latitude-weighted RMSE and ACC[52, 53]. For forecast $\hat{Y}\in\mathbb{R}^{H\times W}$ and reference $Y\in\mathbb{R}^{H\times W}$, the latitude-weighted RMSE is

where $\phi_{h}$ denotes the latitude at grid row $h$. ACC is the latitude-weighted Pearson correlation between forecast anomalies $\hat{Y}^{\prime}=\hat{Y}-\bar{Y}$ and observed anomalies $Y^{\prime}=Y-\bar{Y}$ (with $\bar{Y}$ the climatological mean):

Each model in $\mathcal{M}$ is accessed as a black-box forecaster through a common input–output interface. In this study, the pool consists of eight pretrained models: GenCast, GraphCast, Pangu-Weather, Aurora, FuXi, AIFS, FourCastNet3, and SFNO. These models cover several representative families of AI weather forecasters, including Transformer-based models (Pangu-Weather[9], Aurora[17], FuXi[12]), graph-based models (GraphCast[10], AIFS[19]), diffusion models (GenCast[14]), and neural-operator or spectral-operator models (FourCastNet3[15], SFNO[51]). All outputs are interpolated to the common ERA5 grid ($0.25^{\circ}$) and aligned to a shared variable set, pressure-level set, lead-time convention, and unit system before fusion. For probabilistic models such as GenCast, deterministic summaries such as the ensemble mean are used when RMSE- or ACC-based fusion and evaluation are performed. Predictions are cached asynchronously during policy training, so that Weight-Agent updates are decoupled from repeated execution of the slowest constituent model.

The Weight-Agent is an off-policy continuous-control agent that predicts dense weight fields $W^{(i)}\in\mathbb{R}^{H\times W\times V}$ rather than scalar mixture coefficients. A lightweight CNN encoder maps the high-resolution atmospheric input ($721\times 1440$ grid points) to a compact latent representation, and a deterministic actor–critic architecture[45, 48] with experience replay drives learning. Table 1 lists the main components.

The Evolve-Agent manages $\mathcal{M}_{\mathrm{active}}$ on a slower timescale, using the Weight-Agent’s revealed preferences rather than a separate neural policy.