Sequential Bayesian parameter-state estimation in dynamical systems with noisy and incomplete observations via a variational framework

Extracted source text · arXiv 2512.25056

\documentclass[preprint,3p,times]{elsarticle}

\usepackage{amssymb}
\usepackage{amsmath}
\usepackage{xcolor}
\newcommand{\numpy}{{\tt numpy}} 

\DeclareMathOperator{\E}{\mathbb{E}}
% \DeclareMathOperator{\Var}{\mathbb{V}ar}
% \DeclareMathOperator{\Cov}{\mathbb{C}ov}
\DeclareMathOperator{\Var}{Var}
\DeclareMathOperator{\Cov}{Cov}
\DeclareMathOperator{\KL}{KL}
\DeclareMathOperator{\Y}{\mathcal{Y}}
\DeclareMathOperator{\U}{\mathcal{U}}
\DeclareMathOperator{\N}{\mathcal{N}}
\DeclareMathOperator*{\argmin}{\arg\!\min}
\DeclareMathOperator*{\argmax}{\arg\!\max}
\newcommand{\norm}[1]{\left\lVert#1\right\rVert}
\newcommand{\reals}{\mathbb{R}}
\DeclareMathOperator{\Tr}{Tr}
\newcommand{\pdense}{p}
\usepackage{comment}
\newtheorem{task}{Task}
\renewcommand{\thetask}{}
\newtheorem{theorem}{Theorem}
\newtheorem{prop}[theorem]{Proposition}
\newtheorem{corollary}{Corollary}
\newtheorem{remark}{Remark}

%%%%%%%%%%proof environment
\newcommand{\BlackBox}{\rule{1.5ex}{1.5ex}} 
\ifdefined\proof
    \renewenvironment{proof}{\par\noindent{\bf Proof\ }}{\hfill\BlackBox\\[2mm]}
\else
    \newenvironment{proof}{\par\noindent{\bf Proof\ }}{\hfill\BlackBox\\[2mm]}
\fi
%%%%%%%%%%%%%%
\usepackage{graphicx}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{float}
\usepackage{pifont}
\usepackage{tabularx}

\usepackage[linesnumbered,ruled,vlined]{algorithm2e}


\begin{document}

\begin{frontmatter}

\title{Sequential Bayesian parameter–state estimation in dynamical systems with noisy and incomplete observations via a variational framework}

\author[label1]{Liliang Wang}
\author[label1]{Alex Gorodetsky}

\affiliation[label1]{organization={Department of Aerospace Engineering, University of Michigan},
            city={Ann Arbor},
            postcode={48109},
            state={MI},
            country={USA}}

\begin{abstract}
Online joint estimation of a dynamical model’s unknown parameters and states with uncertainty quantification is crucial in many applications. For example, digital twins dynamically update their knowledge of model parameters and states to support prediction and decision-making. Reliability and computational speed are vital for DTs. Online parameter–state estimation ensures computational efficiency, while uncertainty quantification is essential for making reliable predictions and decisions. In parameter-state estimation, the joint distribution of the state and model parameters conditioned on the data, termed the \textit{joint posterior}, provides accurate uncertainty quantification. Because the joint posterior is generally intractable to compute, this paper presents an online variational inference framework to compute its approximation at each time step. The approximation is factorized into a marginal distribution over the model parameters and a state distribution conditioned on the parameters. This factorization enables recursive updates through a two-stage procedure: first, the parameter posterior is approximated via variational inference; second, the state distribution conditioned on the parameters is computed using Gaussian filtering based on the estimated parameter posterior. The algorithmic design is supported by a theorem establishing upper bounds on the joint posterior approximation error. Numerical experiments demonstrate that the proposed method (i) matches the performance of the joint particle filter in low-dimensional problems, accurately inferring both unobserved states and unknown parameters of dynamical and observation models; (ii) remains robust under noisy, partial observations and model discrepancies in a chaotic Lorenz 96 system; and (iii) scales effectively to a high-dimensional convection–diffusion system, where it outperforms the joint ensemble Kalman filter.
\end{abstract}

\begin{keyword}
Online data assimilation \sep Variational inference \sep Joint parameter-state estimation \sep Incomplete information
\end{keyword}

\end{frontmatter}

\section{Introduction}
\label{sec:introduction}
In many data-driven techniques, there is a great need to continuously update knowledge of a system’s dynamics and states with proper uncertainty quantification in an online manner. A prominent example arises in the context of digital twins (DTs) of operating assets. A digital twin is a virtual representation of a physical system that is continuously updated using data from its physical counterpart~\cite{XiuTartakovsky2025DigitalTwins}. One key consideration in DT applications is the limitation of computational resources~\cite{dt_gap}. In such settings, online inference methods are particularly well suited. Unlike offline approaches which process data in batches, online methods operate recursively: at each time step $k$, they update the inference results from the previous step $k-1$ by assimilating only the newly received data point $y_k$. This recursive structure removes the need to store and reprocess historical data, offering substantially higher computational efficiency compared with offline approaches. In DTs, models and data work together to enable monitoring, prediction and decision-making~\cite{dt_gap}.
However, real-world data are normally noisy and often incomplete. Therefore, DTs must not only update model parameters but also estimate the underlying system states to effectively achieve monitoring and prediction. Moreover, there is often a mismatch between the inference model and the true system, either due to limited knowledge or the complexity of the real dynamics. The noisy, incomplete data together with the model misspecification introduce significant uncertainty. This makes uncertainty quantification essential for establishing trust in models and supporting risk-aware decision-making~\cite{pash2025predictive, dt_gap}. In parameter-state estimation, the \textit{joint posterior}, defined as the joint distribution of state and parameters conditioned on the data, provides accurate uncertainty quantification. It captures not only the individual uncertainties of the states and parameters but also their dependencies. However, the joint posterior is typically intractable to compute in practice. Therefore, we instead consider approximating it online. This task is formalized below.

\begin{task}
\label{task}
Given sequentially arriving data points $y_1,y_2,\cdots$, the goal is to compute a distribution of the current state $X_k$ and unknown model parameters $\theta$, denoted by $q_k(X_k,\theta)$, that approximates the joint posterior $\pdense(X_k,\theta \mid \Y_k)$, defined as the distribution of $X_k$ and $\theta$ conditioned on all observations up to time $k$, $\Y_k=(y_1,y_2,\cdots,y_k)$. At each time step $k$, we use only the approximation from the previous step, $q_{k-1}(X_{k-1},\theta)$, together with the newly received data point $y_k$, to compute the current approximation $q_k(X_k,\theta)$. This task is illustrated in Figure~\ref{fig:task}.
\end{task}
\begin{figure}
     \centering
     \includegraphics[width=0.8\linewidth]{alg_visual/goal_online.pdf}
     \caption{Visualization of the task considered in this paper. A prior distribution $p_0(X_0,\theta)$ that incorporates prior knowledge of the unknown initial state and model parameters is given. The objective is to sequentially approximate the joint posterior $p(X_k,\theta \mid \Y_k)$ by another distribution $q_k(X_k,\theta)$, as new data arrive. At each time step $k$, the approximation $q_k(X_k,\theta)$ is computed recursively using only the newly received data $y_k$ and the previous approximation $q_{k-1}(X_{k-1},\theta)$, without reprocessing past data.}
    \label{fig:task}
\end{figure}


In recent years, some methods have been developed to address this task, namely, to approximate the joint posterior in an online manner. Among these methods, the most commonly used approaches are based on state augmentation. They augment the state vector with the unknown parameters and apply standard filtering methods to the augmented system. Examples of such methods include joint Gaussian filters (e.g., joint unscented Kalman filter (UKF)~\cite{joint_ukf2,joint_ukf3} and joint ensemble Kalman filter (EnKF)~\cite{joint_enkf}) and joint particle filters (PFs)~\cite{joint_pf_1,joint_pf_2}. Joint Gaussian filters approximate the joint posterior with a Gaussian distribution. Let $Y_k$ denote the observation at time step $k$, with the data $y_k$ as its realization. Joint Gaussian filters assume that the joint distribution $p(X_k, \theta, Y_k \mid \Y_{k-1})$ is Gaussian. Under this assumption, the joint posterior $p(X_k,\theta \mid \Y_k)$ remains Gaussian and can be updated recursively. This yields computationally efficient algorithms, but the Gaussian assumption is overly restrictive. It can introduce large approximation errors when the relation between $\theta$ and $X_k$ is highly nonlinear, a situation that can arise in both linear and nonlinear systems. Joint PFs, by contrast, avoid Gaussian assumptions by representing the joint posterior with an empirical distribution constructed from weighted particles, achieving significantly higher approximation accuracy for low-dimensional systems. However, the presence of static parameters in the augmented state amplifies particle weight degeneracy~\cite{joint_pf_issue}, a well-known issue of PFs. As a result, particle weights degenerate faster than in standard PFs, and the number of particles required to counteract this issue grows rapidly with the dimensions of the state and parameters~\cite{Snyder2008Obstacles}. Introducing artificial dynamics for the parameters~\cite{joint_pf_1} mitigates this issue to some extent, but at the cost of introducing estimation bias that is difficult to quantify~\cite{joint_pf_issue}. Consequently, joint PFs may perform poorly even for moderately sized problems~\cite{fbovi}. 

To avoid the severe weight degeneracy resulting from state augmentation, another PF-based method for approximating the joint posterior online, SMC$^2$~\cite{Chopin2013SMC2}, was developed. SMC$^2$ employs a nested filtering structure to compute an approximation of the joint posterior. For each sample $\theta^{(i)}$ drawn from the previous approximate parameter posterior, an inner-loop particle filter estimates the conditional state posterior $p(X_k \mid \Y_k,\theta^{(i)})$. The conditional state posterior estimates are then used by an outer-loop filter to update the approximate parameter posterior. While effective, SMC$^2$ has a major drawback: its per-step computational cost grows with time, making it unsuitable for online tasks.

In contrast to PFs, variational inference (VI)~\cite{vi_def}, which approximates the posterior distribution by minimizing the Kullback-Leibler (KL) divergence of the approximate posterior from the true posterior, is widely regarded as scalable. Moreover, unlike Gaussian filters, VI is not restricted to a specific distribution family~\cite{online_variational_filtering}, making it a promising tool for accurate joint posterior approximation. Nevertheless, most of the existing VI methods for joint posterior approximation are designed for offline settings (e.g.,~\cite{IFT,Ryder2018BBVI_SDE,variational_state_parameter}). Only a limited number of VI-based approaches have been proposed for online joint posterior approximation~\cite{online_rl, online_VI_linear}, and these methods apply only to linear dynamical and observation systems. Furthermore, these methods typically approximate the joint posterior as the product of marginal distributions over parameters and states. Such factorization neglects the intrinsic dependence between model parameters and states and often results in inaccurate joint posterior approximations.

It is noteworthy that another class of online parameter–state estimation methods focuses on computing only the approximate marginal posteriors of the states and parameters by running two filters—a state filter and a parameter filter. Examples include the United Filter~\cite{UnitedFilter} and classical dual Gaussian filters such as the dual UKF~\cite{dual_ukf_1} and dual EnKF~\cite{dual_enkf}. However, since the outputs of these methods do not capture the dependency between parameter and state uncertainties, they often fail to provide accurate uncertainty quantification. 

Beyond the aforementioned limitations of existing posterior approximation-based methods for online parameter-state estimation, few of them provide theoretical guarantees on their approximation accuracy. However, theoretical foundations are essential for establishing trustworthiness, since numerical experiments cannot capture the full range of practical scenarios.

In summary, there is a clear need for an online approach that (i) provides accurate joint posterior approximations across diverse problems and scales, and (ii) is supported by theoretical guarantees of effectiveness. In this paper, we introduce a flexible and scalable variational inference framework to address the task of computing an approximate joint posterior online using noisy and incomplete data. We refer to this framework as \textit{factorization-based online variational inference} (FBOVI). FBOVI factorizes the approximate joint posterior $q_k(X_k,\theta)$ into a parameter marginal $\nu_k(\theta)$ and a conditional state distribution $\rho_k(X_k \mid \theta)$. Guided by a theorem establishing approximation error bounds, the framework proceeds in two stages: Stage~1 computes $\nu_k(\theta)$ via VI, while Stage~2 computes $\rho_k(X_k \mid \theta)$ using a filtering method based on $\nu_k(\theta)$. The family of $\nu_k(\theta)$ can be chosen flexibly, while $\rho_k(X_k \mid \theta)$ is represented as a Gaussian with mean and covariance depending on $\theta$, enabling the use of Gaussian filtering in Stage~2. Although this paper primarily focuses on Gaussian representations for $\rho_k(X_k \mid \theta)$ and using Gaussian filtering at Stage 2, our framework also supports non-Gaussian representations and filtering methods not in the family of Gaussian filters.

This work is an extension and expansion of our previous study~\cite{fbovi}, which also factorizes $q_k(X_k,\theta)$ and employs a two-stage procedure. The present work, however, introduces several key advances: (i) the framework supports a wide range of Gaussian filtering methods, including the EnKF, thus enhancing scalability; (ii) it provides a simplified procedure for linear systems by integrating the Kalman filter; (iii) it establishes theoretical results on approximation accuracy; and (iv) it validates the method with extensive numerical experiments, including cases with partially unknown observation models, model misspecification, and a high-dimensional system described by a partial derivative equation (PDE), whereas~\cite{fbovi} focuses only on low-dimensional problems with correct model forms. 

The main contributions of this paper are summarized as follows:
\begin{enumerate}
\item We propose a flexible variational inference framework for online joint posterior approximation. By avoiding restrictive assumptions on the structure of the joint posterior, the framework enables higher approximation accuracy. A variety of Gaussian filtering methods (including EnKF) can be integrated into the proposed framework, which allows practitioners to balance inference accuracy and computational efficiency.
\item We establish a theoretical result (Theorem~\ref{thm:error_bound}) that provides an upper bound on the error between the true joint posterior and the approximate joint posterior. This result demonstrates the effectiveness of the factorization-based strategy and informs our algorithmic design. Building on this, we further derive Theorem~\ref{thm:error_bound_linear}, which gives estimable error bounds of our method for linear systems.
\item Numerical experiments show that the proposed method (i) achieves estimation and prediction performance comparable to joint PF in low-dimensional systems, successfully inferring unobserved states and parameters in partially unknown dynamical and observation models, (ii) exhibits robustness when noisy and incomplete data, model misspecification, and chaotic dynamics occur simultaneously, and (iii) scales well to a high-dimensional convection–diffusion system.
\end{enumerate}

The contributions of this work can support the advancement of emerging data-driven techniques that rely on online parameter–state estimation. For example, the proposed framework can assist DTs in improving monitoring and predictive performance, and providing reliable uncertainty quantification and theoretical guarantees that contribute to achieving trustworthiness.


The remainder of this paper is organized as follows. Section~\ref{sec:background} introduces the notation, formulates the online parameter-state estimation problem, and reviews the Kullback-Leibler divergence and general framework of Gaussian filtering. Section~\ref{sec:methodology} presents the proposed factorization-based online variational inference framework. In Section~\ref{sec:experiments}, we evaluate the method through numerical experiments and compare its performance with several online Bayesian parameter-state methods. Finally, conclusions are given in Section~\ref{sec:conclusion}. 

\section{Background/Setting}
\label{sec:background}
In this section, we formally present the notation and problem formulation, followed by an introduction to Kullback-Leibler divergence. Lastly, we introduce the fundamental concepts of Gaussian filtering.

\subsection{Notation}
Let $\mathbb{R}$ denote the set of real numbers and $\mathbb{Z}_{\geq 0}$ the set of non-negative integers. For a symmetric matrix $M$, let $\lvert M \rvert$ denote its determinant. Let the triple $(\Omega,\mathcal{F},\mathbb{P})$ denote a probability space, where $\mathbb{P}$ is a probability measure with density $\pdense$. The notation $p(X = x \mid Y = y)$ denotes the value of the probability density function at $x$, conditioned on the event that the random variable $Y$ takes the value $y$. For simplicity, we use $p(x \mid y)$ to represent $p(X = x \mid Y = y)$ throughout this paper. Let $\E[X]$, $\Var[X]$, and $\Cov[X, Y]$ denote the expectation and variance of the random variable $X$, and the covariance between the random variables $X$ and $Y$, respectively. Similarly, let $\E[X \mid Z]$, $\Var[X \mid Z]$, and $\Cov[X, Y \mid Z]$ denote the conditional expectation and variance of $X$, and the conditional covariance between $X$ and $Y$, given $Z$.

We use $\mathcal{N}(\mu,R)$ to denote a Gaussian distribution with mean $\mu$ and covariance $R$. The probability density function of this distribution evaluated at $x$ is written as $\pdense_\mathcal{N}(x;\mu,R)$, that is,
\[
\pdense_\mathcal{N}(x;\mu,R) := \frac{1}{(2\pi)^{d_x/2}\lvert R \rvert^{1/2}} 
\exp\!\left(-\tfrac{1}{2}(x-\mu)^\top R^{-1}(x-\mu)\right),
\]
where $d_x$ is the dimension of $x$. 


\subsection{Problem formulation}
\label{sec:problem}
Consider the following discrete-time system with partially observable states:
\begin{equation}\label{sys:general}
\begin{split}
X_{k+1} &= \Phi(X_k;\theta)+W_k, \quad W_k \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,\Sigma(\theta)), \\
Y_k &= h(X_k;\theta)+V_k, \quad V_k \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,\Gamma(\theta)),
\end{split}
\end{equation}
where $k \in \mathbb{Z}_{\geq 0}$ denotes a time index; and $X_k \in \reals^n$, $Y_k \in \reals^r$, $W_k \in \reals^n$, $V_k \in \reals^r$ represent the state, observation, process noise, and measurement noise at time step $k$, respectively. Here $\theta$ represents the vector of unknown system parameters. The functions $\Phi(X_k;\theta)$ and $h(X_k;\theta)$ are parameterized by the parameters $\theta$. The process noise covariance $\Sigma(\theta)$ and the measurement noise covariance $\Gamma(\theta)$ are determined by $\theta$. The quantities $X_k$, $Y_k$, $W_k$, $V_k$, and $\theta$ are random variables and assumed to be continuous.

Let $y_k$ denote the observed value of $Y_k$. Coupled with system~\eqref{sys:general}, we have a set $\Y_k = (y_1,\ldots,y_k)$ of all observations up to time $k$. The \textit{joint posterior} at time step $k$ is defined as the joint distribution of state $X_k$ and parameters $\theta$ conditioned on $\Y_k$, represented by the probability density $\pdense(X_k,\theta \mid \Y_k)$. The goal is to compute the joint posterior $\pdense(X_k,\theta \mid \Y_k)$ at each time step $k$, using only the previous joint posterior $\pdense(X_{k-1},\theta \mid \Y_{k-1})$ and the newly received data point $y_k$, without reprocessing all historical data $\Y_{k-1}$. 


Theoretically, the expression of the current joint posterior $\pdense(X_k,\theta \mid \Y_k)$ in terms of the previous joint posterior $\pdense(X_{k-1},\theta \mid \Y_{k-1})$ and the newly received data $y_k$ can be derived through a prediction–update procedure. Specifically, at each time step $k$, the joint posterior satisfies
\begin{equation*}
\pdense(X_k,\theta \mid \Y_k) = \pdense(X_k \mid \theta, \Y_k)\,\pdense(\theta \mid \Y_k),
\end{equation*}
where $\pdense(X_k \mid \theta, \Y_k)$ denotes the filtering distribution conditioned on the parameter $\theta$, and $\pdense(\theta \mid \Y_k)$ represents the posterior of $\theta$. During the prediction step, the predictive distribution $p(X_k \mid \theta, \Y_{k-1})$ is obtained by propagating the previous conditional filtering distribution $p(X_{k-1} \mid \theta, \Y_{k-1})$ through the system dynamics of~\eqref{sys:general}. During the update step, the newly observed data $y_k$ is assimilated to update both the predictive distribution $p(X_k \mid \theta, \Y_{k-1})$ and the previous parameter posterior $\pdense(\theta \mid \Y_{k-1})$, resulting in the conditional filtering distribution $\pdense(X_k \mid \theta, \Y_k)$ and the updated parameter posterior $\pdense(\theta \mid \Y_k)$~\cite{fbovi}. This prediction–update mechanism is illustrated in Figure~\ref{fig:pred_update}. The subsequent subsections provide a detailed description of each step.

\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.6\linewidth]{alg_visual/procedure_ana_new.pdf}
    \caption{Illustration of the prediction–update procedure for determining the conditional filtering distribution $p(X_k \mid \theta, \Y_k)$ and the parameter posterior $p(\theta \mid \Y_k)$. The predictive distribution $p(X_k \mid \theta, \Y_{k-1})$ is obtained by propagating the previous filtering distribution $p(X_{k-1} \mid \theta, \Y_{k-1})$ through the dynamics. The data $y_k$ is then assimilated to update the predictive distribution, yielding $p(X_k \mid \theta, \Y_k)$. The distribution $p(X_k \mid \theta, \Y_{k-1})$ is also used for determining the parameter posterior $p(\theta \mid \Y_k)$.}
    \label{fig:pred_update}
\end{figure}

\subsubsection{Prediction step}
\label{sec:pred_ana}
The predictive distribution $p(X_k \mid \theta, \Y_{k-1})$ is obtained by propagating the conditional filtering distribution from the previous step:
\begin{align}
p(X_k \mid \theta, \Y_{k-1})
&= \int \pdense(X_k, X_{k-1} \mid \theta, \Y_{k-1})\,dX_{k-1} \notag\\
&= \int \pdense_{\mathcal{N}}\!\left(X_k; \Phi(X_{k-1};\theta), \Sigma(\theta)\right)\pdense(X_{k-1} \mid \theta, \Y_{k-1})\,dX_{k-1}.
\end{align}
Let $\mathcal{T}$ denote the operator that maps $\pdense(X_{k-1} \mid \theta, \Y_{k-1})$ to the predictive distribution $\pdense(X_k \mid \theta, \Y_{k-1})$. We refer to $\mathcal{T}$ as the \textit{prediction operator}. For example, the distribution $\pdense(X_k \mid \theta, \Y_{k-1})$ can be represented as $\pdense(X_k \mid \theta, \Y_{k-1}) = \mathcal{T}p(X_{k-1} \mid \theta, \Y_{k-1})$. 


\subsubsection{Update step}
Given the predictive distribution $\pdense(X_k \mid \theta, \Y_{k-1})$, we now derive the posterior distributions $\pdense(X_k \mid \theta, \Y_k)$ and $\pdense(\theta \mid \Y_k)$.  By Bayes’ rule, the conditional filtering distribution is
\begin{equation}
\pdense(X_k \mid \theta, \Y_k)
= \frac{1}{Z_{X,k}(\theta)}\,\pdense_{\mathcal{N}}\!\left(y_k; h(X_k;\theta), \Gamma(\theta)\right)
\pdense(X_k \mid \theta, \Y_{k-1}),
\end{equation}
where
\[
Z_{X,k}(\theta)
= \int \pdense_{\mathcal{N}}\!\left(y_k; h(X_k;\theta), \Gamma(\theta)\right)
\pdense(X_k \mid \theta, \Y_{k-1})\,dX_k
\]
is a normalization constant for a given $\theta$.  
Let $\mathcal{F}_k^X$ denote the operator mapping $\pdense(X_{k-1} \mid \theta, \Y_{k-1})$ to $\pdense(X_k \mid \theta, \Y_k)$.

Similarly, by Bayes’ rule, the parameter posterior can be written as
\begin{equation}
\pdense(\theta \mid \Y_k)
= \frac{1}{Z_k}\,\pdense(\theta \mid \Y_{k-1})\,p(y_k \mid \theta, \Y_{k-1})
= \frac{1}{Z_k}\,\pdense(\theta \mid \Y_{k-1})
\int \pdense_{\mathcal{N}}\!\left(y_k; h(X_k;\theta), \Gamma(\theta)\right)
\pdense(X_k \mid \theta, \Y_{k-1})\,dX_k,
\end{equation}
where
\[
Z_k
= \int \pdense(\theta \mid \Y_{k-1})
\left (\int \pdense_{\mathcal{N}}\!\left(y_k; h(X_k;\theta), \Gamma(\theta)\right)
\pdense(X_k \mid \theta, \Y_{k-1})\,dX_k\right )d\theta
\]
is a normalizing constant.  
Let $\mathcal{F}_k$ denote the operator that maps the pair $\big(\pdense(X_{k-1} \mid \theta, \Y_{k-1}),\, \pdense(\theta \mid \Y_{k-1})\big)$ to the updated parameter posterior $\pdense(\theta \mid \Y_k)$.


\subsection{Kullback-Leibler divergence}
\label{sec:metrics_dist}
This section introduces the Kullback–Leibler (KL) divergence, which is employed in our framework described in detail in Section~\ref{sec:methodology}. Let $\mu_1$ and $\mu_2$ be two probability measures with corresponding probability density functions $f_1(x)$ and $f_2(x)$. The KL divergence of $\mu_1$ relative to $\mu_2$ is defined as:
\begin{equation}
\KL(\mu_1 \lVert \mu_2) := \int f_1(x)\log \frac{f_1(x)}{f_2(x)}dx.
\end{equation}



\subsection{Gaussian filtering}
Gaussian filtering will be employed in the proposed framework. In this section, we present the fundamental concept of Gaussian filtering. Gaussian filtering assumes that the joint distribution of the current state $X_k$ and the current observation $Y_k$, conditioned on past observations $\Y_{k-1}$, namely $\pdense \left(X_k,Y_k \mid \Y_{k-1}\right)$, is Gaussian. Under this assumption, the posterior $p(X_k \mid \Y_k)$ is also Gaussian and can be written as
\begin{equation*}
p(X_k \mid \Y_k) = p_{\mathcal{N}}(X_k; m_k, C_k),
\end{equation*}
with mean $m_k$ and covariance $C_k$ given by
\begin{align}
m_k &= m_k^- + U_k S_k^{-1}(y_k - \mu_k), \qquad 
C_k = C_k^- - U_k S_k^{-1} U_k^\top,
\label{eq:post_mean_cov}
\end{align}
where $m_k^-$, $C_k^-$, $\mu_k^-$, $U_k$, and $S_k$ are defined as
\begin{equation}\label{eq:GF_eqs}
\begin{split}
m_k^- &:= \E[X_k \mid \Y_{k-1}], \quad 
C_k^- := \Var[X_k \mid \Y_{k-1}], \quad 
\mu_k := \E[Y_k \mid \Y_{k-1}], \\
U_k &:= \Cov[X_k,Y_k \mid \Y_{k-1}], \quad 
S_k := \Var[Y_k \mid \Y_{k-1}].
\end{split}
\end{equation}
Various Gaussian filtering methods differ in how they approximate the quantities $m_k^-$, $C_k^-$, $\mu_k$, $U_k$, and $S_k$ \cite{Gaussian_filter}. 


\section{Methodology}
\label{sec:methodology}
In this section, we introduce our proposed framework, FBOVI, which aims to obtain the joint posterior $p(X_k,\theta \mid \Y_k)$ in an online fashion.

As noted in Section~\ref{sec:introduction}, computing the joint posterior $\pdense(X_k,\theta \mid \Y_k)$ is typically intractable. We therefore approximate $\pdense(X_k,\theta \mid \Y_k)$ at each time step $k$ by another distribution $q_k(X_k,\theta)$, referred to as the \emph{approximate joint posterior}. The approximation is computed recursively: $q_k(X_k,\theta)$ is obtained using only the newly received data $y_k$ and the previous approximation $q_{k-1}(X_{k-1},\theta)$. We factorize $q_k$ as the product of a conditional distribution of $X_k$ given $\theta$ and a marginal distribution over $\theta$:
\begin{equation*}
q_k(X_k,\theta) := \rho_k(X_k \mid \theta)\,\nu_k(\theta).
\end{equation*}
In principle, one might try to directly approximate the exact factorization $\pdense(X_k,\theta \mid \Y_k)=\pdense(X_k \mid \theta,\Y_k)\,\pdense(\theta \mid \Y_k)$ by setting $\rho_k(X_k \mid \theta) \approx \pdense(X_k \mid \theta,\Y_k)$ and $\nu_k(\theta) \approx \pdense(\theta \mid \Y_k)$. However, as described in Section~\ref{sec:problem}, the expressions of $\pdense(X_k \mid \theta,\Y_k)$ and $\pdense(\theta \mid \Y_k)$ depend on the \emph{previous} conditional filtering distribution $\pdense(X_{k-1}\mid \theta,\Y_{k-1})$ and the \emph{previous} parameter posterior $\pdense(\theta \mid \Y_{k-1})$, both of which are unavailable in closed forms. This makes a direct approximation infeasible.

Our solution to this issue is to introduce ``online targets". We let the distribution $\rho_k(X_k \mid \theta)$ approximate the \emph{online conditional filtering distribution} $\rho_k^*(X_k \mid \theta)$ and let $\nu_k(\theta)$ approximate the \emph{online parameter posterior} $\nu^*_k(\theta)$. The online conditional filtering distribution $\rho_k^*(X_k \mid \theta)$ is the resulting $\pdense(X_k \mid \theta, \Y_k)$ obtained by replacing the previous conditional filtering distribution $\pdense(X_{k-1} \mid \theta, \Y_{k-1})$ with its approximation $\rho_{k-1}(X_{k-1} \mid \theta)$. The online parameter posterior $\nu_k^*(\theta)$ represents the resulting $\pdense(\theta \mid \Y_k)$ when the distributions $\pdense(X_{k-1} \mid \theta, \Y_{k-1})$ and $\pdense(\theta \mid \Y_{k-1})$ are replaced by their approximations $\rho_{k-1}(X_{k-1} \mid \theta)$ and $\nu_{k-1}(\theta)$, respectively. Formally, $\rho^*_k$ and $\nu_k^*$ are defined as
\begin{equation*}
\rho_k^* := \mathcal{F}_k^X \rho_{k-1}, \qquad 
\nu_k^* := \mathcal{F}_k(\rho_{k-1},\nu_{k-1}),
\end{equation*}
where $\mathcal{F}_k^X$ and $\mathcal{F}_k$ are the operators introduced in Section~\ref{sec:problem}. Generally, the distributions $\rho_k^*(X_k \mid \theta)$ and $\nu_k^*(\theta)$ cannot be computed analytically. The core idea of our framework is therefore to use $\rho_k(X_k \mid \theta)$ to approximate $\rho_k^*(X_k \mid \theta)$, and $\nu_k(\theta)$ to approximate $\nu_k^*(\theta)$. This procedure is illustrated in Figure~\ref{fig:strategy}.

\begin{figure}[htbp]
    \centering
    \includegraphics[width=0.8\linewidth]{alg_visual/procedure_ours.pdf}
    \caption{Core idea of the proposed framework: the online conditional filtering distribution $\rho_k^*(X_k \mid \theta)$ is approximated by $\rho_k(X_k \mid \theta)$, and the online parameter posterior $\nu_k^*(\theta)$ is approximated by $\nu_k(\theta)$. Their product, $\rho_k(X_k \mid \theta)\nu_k(\theta)$, forms the approximate joint posterior $q_k(X_k,\theta)$. The blue arrows represent the maps $\mathcal{F}_k^X$ and $\mathcal{F}_k$. For comparison, the transformations of the exact solutions are shown on the right. The distributions shown in gray are those that are intractable to compute.}
    \label{fig:strategy}
\end{figure}


As \( \rho_k^*(X_k \mid \theta) \) and \( \nu_k^*(\theta) \) normally differ from \(p(X_k \mid \theta, \Y_k)\) and \(p(\theta \mid \Y_k)\), respectively, one may ask whether approximating these online targets is sufficient to effectively approximate the true joint posterior $\pdense(X_k,\theta \mid \Y_k)$. The following theorem, which establishes an upper bound on the error between the true and approximate joint posteriors, demonstrate the effectiveness of our strategy and offers principled guide for algorithmic design:

\begin{theorem}
\label{thm:error_bound}
Assume $\inf_{\theta} \big \lvert \Gamma(\theta) \big \rvert = \tilde{C} > 0$. 
The total variation and Hellinger distances between $p(X_k,\theta \mid \Y_k)$ and $q_k(X_k,\theta)$ satisfy:
\begin{align}
d(p(X_k,\theta \mid \Y_k), q_k(X_k,\theta)) &\leq \sum_{j=1}^{k-1}C(\Y_k,\tilde{C},j,k)\sqrt{\E_{\theta \sim \nu_j(\cdot)}\left[\KL\left(\rho_j(\cdot \mid \theta) \lVert \rho_j^*(\cdot \mid \theta) \right) \right] +\KL\left(\nu_j \lVert \nu_j^*\right)} \nonumber \\
&\qquad  + \frac{1}{\sqrt{2}} \sqrt{\E_{\theta \sim \nu_k(\cdot)}\left[\KL\left(\rho_k(\cdot \mid \theta) \lVert \rho_k^*(\cdot \mid \theta) \right) \right] +\KL\left(\nu_k \lVert \nu_k^*\right)},
\label{ieq:learning_error_bound_tv}
\end{align}
where $C(\Y_k,\tilde{C},j,k)=\frac{(2\pi)^{-\frac{r(k-j)}{2}}\tilde{C}^{-\frac{k-j}{2}}}{\sqrt{2}p(\Y_{j+1:k}\mid\Y_j)}$ when $d$ is the total variation distance, and $C(\Y_k,\tilde{C},j,k)=\frac{2^{(k-j)}(2\pi)^{-\frac{r(k-j)}{4}}\tilde{C}^{-\frac{k-j}{4}}}{\sqrt{2p(\Y_{j+1:k}\mid\Y_j)}}$ when $d$ is the Hellinger distance. The notation $\Y_{j+1:k}$ denotes the data received from time step $j+1$ through $k$.
\end{theorem}
The total variation (TV) and Hellinger distances are two commonly used metrics for measuring the difference between two probability distributions. Their definitions are provided in~\ref{app:metrics_dist}. 

\begin{proof}
See~\ref{app:proof_thm}.
\end{proof}

The terms $C(\Y_k,\tilde{C},j,k)$ depend only on the observational data and model quantities and are thus constant with respect to the choices for $\rho_j$ and $\nu_j$ for $j \in [1,k]$. The controllable terms in the bounds are the divergences
\[
\E_{\theta \sim \nu_j(\cdot)}\!\left[\KL\!\left(\rho_j(\cdot \mid \theta)\,\big\|\, \rho_j^*(\cdot \mid \theta)\right)\right]
\quad \text{and} \quad
\KL\!\left(\nu_j \,\big\|\, \nu_j^*\right), \qquad j=1,\dots,k,
\]
which measure how well the approximate conditional filtering distribution and the approximate parameter posterior track their online targets. Hence, to reduce the error between the true and approximate joint posteriors, we adopt the following two-stage strategy at each time step $k$:
\begin{enumerate}
    \item \textbf{Minimize} \( \KL(\nu_k  \| \nu_k^*) \) to obtain \( \nu_k(\theta) \).
    \item \textbf{Minimize} \( \E_{\theta \sim \nu_k(\cdot)}[\KL(\rho_k(\cdot \mid \theta) \| \rho_k^*(\cdot\mid \theta))] \) to obtain \( \rho_k(X_k \mid \theta) \), using the updated \( \nu_k(\theta) \).
\end{enumerate}
%
We allow \( \nu_k(\theta) \) to be chosen from an arbitrary variational family, while \( \rho_k(X_k \mid \theta) \) is modeled as a conditional Gaussian:
\[
\rho_k(X_k \mid \theta) = \pdense_{\mathcal{N}}(X_k; m_k(\theta), C_k(\theta)).
\]
We now describe how each of these two stages is performed in detail.

\subsection{Stage 1: computation of $\nu_k(\theta)$}
\label{sec:step1}
To compute $\nu_k(\theta)$ by minimizing $\KL(\nu_k \Vert \nu_k^*)$, we first derive the expression of $\nu_k^*(\theta)$. Let $\rho_k^{*-}(X_k \mid \theta)$ denote the corresponding predictive distribution $p(X_k \mid \theta, \Y_{k-1})$ when the previous conditional filtering distribution $\pdense(X_{k-1} \mid \theta, \Y_{k-1})$ is replaced with $\rho_{k-1}(X_{k-1} \mid \theta)$, i.e., $\rho_k^{*-}:=\mathcal{T}(\rho_{k-1})$. We can express $\rho_k^{*-}(X_k \mid \theta)$ as 
\begin{equation*} \rho_k^{*-}(X_k \mid \theta) := \int \pdense_{\mathcal{N}}(X_k; \Phi(X_{k-1};\theta), \Sigma(\theta))\rho_{k-1}(X_{k-1} \mid \theta)dX_{k-1}. \end{equation*} 
We refer to $\rho_k^{*-}(X_k \mid \theta)$ as the \emph{online conditional predictive distribution}. Throughout this paper, the superscript ‘$-$’ indicates quantities associated with predictive distributions.

By Bayes' rule, the online parameter posterior $\nu_k^*(\theta)$ is given by:
\begin{equation}
\nu_k^*(\theta) = \frac{1}{\hat{Z}_{k}} \nu_{k-1}(\theta) \int \pdense_\mathcal{N}(y_k; h(X_k;\theta), \Gamma(\theta))\, \rho_k^{*-}(X_k \mid \theta)\, dX_k,
\label{eq:nu_star}
\end{equation}
where \( \hat{Z}_{k} \) is a normalizing constant.
Given Equation~\eqref{eq:nu_star}, the KL divergence $\KL(\nu_k  \| \nu_k^*)$ can be written as
\begin{equation}
\KL(\nu_k  \| \nu_k^*) = \log \hat{Z}_{k} - \left( \E_{\theta \sim \nu_k(\cdot)} \left[\int \pdense_\mathcal{N}(y_k; h(X_k;\theta), \Gamma(\theta))\, \rho_k^{*-}(X_k \mid \theta)\, dX_k \right]- \KL(\nu_k \| \nu_{k-1})\right).
\label{eq:kl_step1}
\end{equation}
Since the first term on the right-hand side of Equation~\eqref{eq:kl_step1}, $\log \hat{Z}_{k}$, is constant, VI minimizes the KL divergence $\KL(\nu_k \| \nu_k^*)$ by maximizing the negative of the remaining terms on the right-hand side of Equation~\eqref{eq:kl_step1}, which is defined as the evidence lower bound (ELBO) $\mathcal{L}_k(\nu_k)$:
\begin{equation}
\mathcal{L}_k(\nu_k) := \E_{\theta \sim \nu_k(\cdot)}\!\left[I(\theta)\right] - \KL\!\left(\nu_k\,\|\, \nu_{k-1}\right),
\label{eq:elbo_step1}
\end{equation}
where 
\begin{equation*}
I(\theta) := \int \pdense_\mathcal{N}(y_k; h(X_k;\theta), \Gamma(\theta))\, \rho_k^{*-}(X_k \mid \theta)\, dX_k.
\end{equation*}
In the following subsections, we describe how the integral $I(\theta)$ in Equation~\eqref{eq:elbo_step1} can be computed for both linear and nonlinear systems.

\subsubsection{Linear case}
If the dynamical and observation models in system~\eqref{sys:general} are linear in the state $X_k$ and have the following forms
\begin{equation}\label{sys:linear}
\begin{split}
    X_{k+1} &= A(\theta)X_k + W_k, \qquad W_k \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,\Sigma(\theta)), \\
    Y_k &= H(\theta)X_k + V_k, \qquad V_k \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,\Gamma(\theta)),
\end{split}
\end{equation}
where $A(\theta) \in \mathbb{R}^{n \times n}$ and $H(\theta) \in \mathbb{R}^{r \times n}$ are matrices determined by the parameters $\theta$, the online conditional predictive distribution $\rho^{*-}_k$ is Gaussian and given by
\begin{equation*}
\rho^{*-}_k(X_k \mid \theta) =  \pdense_{\mathcal{N}}\left(X_k; m_k^{*-}(\theta), C_k^{*-}(\theta)\right),
\end{equation*}
where
\begin{equation}
m_k^{*-}(\theta) = A(\theta)m_{k-1}(\theta), \qquad C_k^{*-}(\theta)=A(\theta)C_{k-1}(\theta)A(\theta)^T+\Sigma(\theta).
\label{eq:mean_cov_pred_linear}
\end{equation}
Thus the integral $I(\theta)$ has a closed-form expression:
\begin{align}
&I(\theta) = \pdense_{\mathcal{N}}\left(y_k;H(\theta)m_k^{*-}(\theta), H(\theta)C_k^{*-}(\theta)H(\theta)^T+\Gamma(\theta)\right).
\end{align}
The procedure of computing the distribution $\nu_k(\theta)$ for the linear case is illustrated in Figure~\ref{fig:stage1_linear}. 
\begin{figure}
     \centering
     \includegraphics[width=0.6\linewidth]{alg_visual/stage1_linear.pdf}
     \caption{Computation of the distribution $\nu_k(\theta)$ for linear systems. The predictive distribution $\rho_k^{*-}(X_k \mid \theta)$ can be computed in the linear case. The prediction operator $\mathcal{T}$ is introduced in Section~\ref{sec:pred_ana}. It represents the operator that maps the previous conditional filtering distribution $\pdense(X_{k-1} \mid \theta, \Y_{k-1})$ to the predictive distribution $\pdense(X_k \mid \theta, \Y_{k-1})$. }
    \label{fig:stage1_linear}
\end{figure}


\subsubsection{Nonlinear case}
When the dynamical model is nonlinear in the state variable $X_k$, closed-form expressions of the online conditional predictive distribution $\rho_k^{*-}(X_k \mid \theta)$ are generally unavailable, rendering the computation of the ELBO $\mathcal{L}_k(\nu_k)$ intractable. In such cases, we approximate $\rho_k^{*-}(X_k \mid \theta)$ with a Gaussian distribution:
\begin{equation}
\rho_k^{*-}(X_k \mid \theta) \approx \pdense_{\mathcal{N}}(X_k; m_k^-(\theta), C_k^-(\theta)),
\label{eq:predictive_nonlinear}
\end{equation}
where the mean $m_k^-(\theta)$ and the covariance $m_k^-(\theta)$ are defined as
\begin{align}
m_k^-(\theta) &:= \int \Phi(X_{k-1};\theta)\pdense_{\mathcal{N}}(X_{k-1}; m_{k-1}(\theta), C_{k-1}(\theta))dX_{k-1},
\label{eq:int_pred_mean} \\
C_k^-(\theta) &:= \int \left(\Phi(X_{k-1};\theta)-m_k^-(\theta)\right)\left(\Phi(X_{k-1};\theta)-m_k^-(\theta)\right)^T\pdense_{\mathcal{N}}(X_{k-1}; m_{k-1}(\theta), C_{k-1}(\theta))dX_{k-1}+\Sigma(\theta).
\label{eq:int_pred_cov}
\end{align}
The integrals in Equations~\eqref{eq:int_pred_mean} and~\eqref{eq:int_pred_cov} can be evaluated using the unscented transform or Gaussian quadrature~\cite{bayesian_filter}.

With the Gaussian approximation $\pdense_{\mathcal{N}}\left(X_k;m_k^-(\theta), C_k^-(\theta)\right)$ for $\rho_{k}^{*-}(X_{k} \mid \theta)$, we approximate the integral $I(\theta)$ by
\begin{equation}
I(\theta) \approx \int \pdense_{\mathcal{N}}(y_k;h(X_k;\theta),\Gamma(\theta))\pdense_{\mathcal{N}}\left(X_k;m_k^-(\theta), C_k^-(\theta)\right)dX_k.
\label{eq:int_approx}
\end{equation}
The right hand side of Equation~\eqref{eq:int_approx} can be evaluated by unscented transform or Gaussian quadrature. In addition, when the observation model is linear in the state $X_k$:
\begin{equation}
Y_k = H(\theta)X_k + V_k, \qquad V_k \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,\Gamma(\theta)),
\label{eq:obs_linear}
\end{equation}
the right hand side of Equation~\eqref{eq:int_approx} has a closed-form expression:
\begin{align}
\int \pdense_{\mathcal{N}}(y_k;h(X_k;\theta),\Gamma(\theta))\pdense_{\mathcal{N}}\left(X_k;m_k^-(\theta), C_k^-(\theta)\right)dX_k &= \pdense_{\mathcal{N}}\left(y_k;H(\theta)m_k^-(\theta), H(\theta)C_k^-(\theta)H(\theta)^T+\Gamma(\theta)\right).
\end{align}
The procedure of computing the distribution $\nu_k(\theta)$ for the nonlinear case is illustrated in Figure~\ref{fig:stage1_nonlinear}.
\begin{figure}
     \centering
     \includegraphics[width=0.7\linewidth]{alg_visual/stage1_nonlinear.pdf}
     \caption{Computation of the distribution $\nu_k(\theta)$ for nonlinear systems. The predictive distribution $\rho_k^{*-}(X_k \mid \theta)$ is intractable to compute and thus approximated by a Gaussian distribution $\mathcal{N}(X_k;m_k^-(\theta),C_k^-(\theta))$ first.}
    \label{fig:stage1_nonlinear}
\end{figure}
The choice of approximating the distribution $\rho^{*-}_k(X_k \mid \theta)$ with a Gaussian distribution is to convert the integral $I(\theta)$ into an integral of the likelihood $\pdense_{\mathcal{N}}(y_k;h(X_k;\theta)$ with respect to a Gaussian measure. This conversion enables the use of existing computationally efficient numerical methods for evaluating integrals under Gaussian measures and thereby ensures the overall computational efficiency of the proposed method. If we don't choose to approximate the distribution $\rho^{*-}_k(X_k \mid \theta)$ with a Gaussian, we can still estimate the integral $I(\theta)$ using the Monte Carlo method:
\begin{equation}
I(\theta) \approx \frac{1}{N}\sum_{i=1}^N\pdense_{\mathcal{N}}\left(y_k;h(X_k^{(i)};\theta),\Gamma(\theta)\right),
\end{equation}
where $X_k^{(i)}$, $i=1,2,\cdots,N$ are samples following the distribution $\rho_k^{*-}(X_k \mid \theta)$ constructed by
\begin{equation*}
X_k^{(i)} = \Phi(X_{k-1}^{(i)};\theta) + \eta^{(i)}, \qquad X_{k-1}^{(i)} \sim \rho_{k-1}(X_{k-1} \mid \theta), \quad \eta^{(i)} \sim \Sigma(\theta).
\end{equation*}
When the integral $I(\theta)$ can be evaluated, we can compute the distribution $\nu_k(\theta)$ by maximizing the ELBO $\mathcal{L}_k(\nu_k)$. Stage 1 ends with $\nu_k(\theta)$ obtained and we proceed to Stage 2.

\subsection{Stage 2: computation of \( \rho_k(X_k \mid \theta) \)}
\label{sec:step2}
With $\nu_k(\theta)$ computed, we focus on computing the conditional distribution \( \rho_k(X_k \mid \theta) \) that minimizes
\[
\E_{\theta \sim \nu_k(\cdot)}\left[ \KL\!\left(\rho_k(\cdot\mid \theta) \,\|\, \rho_k^*(\cdot \mid \theta) \right) \right].
\]
In the following two subsections, we describe how \( \rho_k(X_k \mid \theta) \) is computed for both linear and nonlinear systems, given the marginal distribution $\nu_k(\theta)$.

\subsubsection{Linear case}
When the dynamical and observation models are linear and described by Equation~\eqref{sys:linear}, the online conditional filtering distribution \( \rho_k^*(X_k \mid \theta) \) is Gaussian:
\[
\rho_k^*(X_k \mid \theta) = \mathcal{N}\left(X_k; m_k^*(\theta), C_k^*(\theta)\right),
\]
with the mean \( m_k^*(\theta) \) and covariance \( C_k^*(\theta) \) given by the Kalman filter:
\begin{align}
m_k^*(\theta) =& m_k^{*-}(\theta)+C_k^{*-}(\theta)H(\theta)^T\left(H(\theta)C_k^{*-}(\theta)H(\theta)^T+\Gamma(\theta)\right)^{-1}\left(y_k-H(\theta)m_k^{*-}(\theta)\right), 
\label{eq:kf_mean_post} \\
C_k^*(\theta) =& C_k^{*-}(\theta) - C_k^{*-}(\theta)H(\theta)^T\left(H(\theta)C_k^{*-}(\theta)H(\theta)^T+\Gamma(\theta)\right)^{-1}H(\theta)C_k^{*-}(\theta),
\label{eq:kf_var_post}
\end{align}
where $m_k^{*-}(\theta)$ and $C_k^{*-}(\theta)$ are defined in Equation~\eqref{eq:mean_cov_pred_linear}.

While using \( m_k(\theta) = m_k^*(\theta) \) and \( C_k(\theta) = C_k^*(\theta) \) would minimize the KL divergence exactly, this is computationally infeasible in practice. As shown in Equations~\eqref{eq:kf_mean_post} and~\eqref{eq:kf_var_post}, the complexity of \( m_k^*(\theta) \) and \( C_k^*(\theta) \) increases over time due to recursive dependence on past distributions. To maintain constant computational and storage costs over time, we represent \( m_k(\theta) \) and \( C_k(\theta) \) using neural networks with fixed architectures. This design ensures that the inference cost does not grow with time, while leveraging the universal approximation capability of neural networks to reduce error. The procedure of computing the distribution $\rho_k(X_k \mid \theta)$ for the linear case is illustrated in Figure~\ref{fig:stage2_linear}.
\begin{figure}
     \centering
     \includegraphics[width=0.7\linewidth]{alg_visual/stage2_linear.pdf}
     \caption{Procedure of computing the distribution $\rho_k(X_k \mid \theta)$ for linear systems. The online conditional filtering distribution $\rho_k^*(X_k \mid \theta)$ is a Gaussian distribution $\mathcal{N}(X_k;m_k^*(\theta),C_k^*(\theta))$ given by the Kalman filter. We compute the distribution $\rho_k(X_k \mid \theta)$ using the mean function $m_k^*(\theta)$, the covariance function $C_k^*(\theta)$, and the distribution $\nu_k(\theta)$ which is inferred at Stage 1.}
    \label{fig:stage2_linear}
\end{figure}

The following theorem demonstrates the joint posterior approximation accuracy of our method, establishing an upper bound on the error between the true joint posterior $p(X_k,\theta \mid \Y_k)$ and the approximate joint posterior $q_k(X_k,\theta)$ obtained by our method, for linear system~\eqref{sys:linear}. 

\begin{theorem}
\label{thm:error_bound_linear}
Assume $\inf_{\theta} \big \lvert \Gamma(\theta) \big \rvert = \tilde{C} > 0$. Suppose the ELBO $\mathcal{L}_i(\nu_i)$ satisfies $\mathcal{L}_i(\nu_i) \geq \epsilon_i$ for all $i \in [1,k]$.
The total variation and Hellinger distances between $p(X_k,\theta \mid \Y_k)$ and $q_k(X_k,\theta)$ satisfy:
\begin{align}
&d(p(X_k,\theta \mid \Y_k), q_k(X_k,\theta)) \nonumber \\
&\quad \leq \sum_{j=1}^{k-1}C^\prime(\Y_k,\tilde{C},j,k, m_{j:k}, C_{j:k}, \nu_{j:k})\sqrt{\E_{\theta \sim \nu_j(\cdot)}\left[ \Psi_j(C_j(\theta), m_j(\theta), C_{j-1}(\theta), m_{j-1}(\theta), \theta) \right] -\frac{r}{2}\log(2\pi)-\frac{1}{2}\log \tilde{C} - \epsilon_j} \nonumber \\
&\qquad \qquad  + \frac{1}{\sqrt{2}} \sqrt{\E_{\theta \sim \nu_k(\cdot)}\left[\Psi_k(C_k(\theta), m_k(\theta), C_{k-1}(\theta), m_{k-1}(\theta), \theta) \right] -\frac{r}{2}\log(2\pi)-\frac{1}{2}\log \tilde{C} - \epsilon_k},
\label{eq:bound_linear}
\end{align}
where $m_{j:k}$, $C_{j:k}$, and $\nu_{j:k}$ denote the sequences $\{m_i\}_{i=j}^k$, $\{C_i\}_{i=j}^k$, and $\{\nu_i\}_{i=j}^k$, respectively. For any integer $j \in [1,k]$, $\Psi_j$ is defined as:
\begin{align*}
&\Psi_j(C_j(\theta), m_j(\theta), C_{j-1}(\theta), m_{j-1}(\theta), \theta) \\
& \qquad := \log \big \lvert F_c(C_{j-1}(\theta), m_{j-1}(\theta), \theta) \big \rvert - \log \big \lvert C_j(\theta) \big \rvert - n + \Tr\left(\left(F_c(C_{j-1}(\theta), m_{j-1}(\theta), \theta)\right)^{-1}C_j(\theta)\right) \\
&\qquad \qquad + \left(m_j(\theta)-F_m(m_{j-1}(\theta), C_{j-1}(\theta),\theta)\right)^T\left(F_c(C_{j-1}(\theta), m_{j-1}(\theta), \theta)\right)^{-1}\left(m_j(\theta)-F_m(m_{j-1}(\theta), C_{j-1}(\theta),\theta)\right),
\end{align*}
with $F_m$ and $F_c$ defined as
\begin{align*}
F_m(C_{j-1}(\theta), m_{j-1}(\theta), \theta) :=& m_j^{*-}(\theta)+C_j^{*-}(\theta)H(\theta)^T\left(H(\theta)C_j^{*-}(\theta)H(\theta)^T+\Gamma(\theta)\right)^{-1}\left(y_j-H(\theta)m_j^{*-}(\theta)\right), \nonumber \\
F_c(C_{j-1}(\theta), m_{j-1}(\theta), \theta) :=& C_j^{*-}(\theta) - C_j^{*-}(\theta)H(\theta)^T\left(H(\theta)C_j^{*-}(\theta)H(\theta)^T+\Gamma(\theta)\right)^{-1}H(\theta)C_j^{*-}(\theta),
\end{align*}
where
\begin{equation*}
m_j^{*-}(\theta):= A(\theta)m_{j-1}(\theta), \quad C_j^{*-}(\theta) := A(\theta)C_{j-1}(\theta)A(\theta)^T+\Sigma(\theta).
\end{equation*}
The term $C^\prime(\Y_k,\tilde{C},j,k, m_{j:k}, C_{j:k}, \nu_{j:k})$ in  Equation~\eqref{eq:bound_linear} is defined as $C^\prime(\Y_k,\tilde{C},j,k, m_{j:k}, C_{j:k}, \nu_{j:k})=\frac{(2\pi)^{-\frac{r(k-j)}{2}}\tilde{C}^{-\frac{k-j}{2}}}{\sqrt{2}\prod_{i=j+1}^kZ_i(m_{i-1}, C_{i-1},\nu_{i-1})}$ when $d$ is the total variation distance, and $C^\prime(\Y_k,\tilde{C},j,k, m_{j:k}, C_{j:k}, \nu_{j:k})=\frac{2^{(k-j)}(2\pi)^{-\frac{r(k-j)}{4}}\tilde{C}^{-\frac{k-j}{4}}}{\sqrt{2\prod_{i=j+1}^kZ_i(m_{i-1}, C_{i-1},\nu_{i-1})}}$ when $d$ is the Hellinger distance. Here $Z_i(m_{i-1}, C_{i-1},\nu_{i-1})$ is defined as $Z_i(m_{i-1}, C_{i-1},\nu_{i-1}):=\E_{\theta \sim \nu_{i-1}(\cdot)}[\pdense_{\mathcal{N}}(y_i;H(\theta)m_i^{*-}(\theta), H(\theta)C_i^{*-}(\theta)H(\theta)^T+\Gamma(\theta))]$.
\end{theorem}

\begin{proof}
See~\ref{app:proof_thm_linear}.
\end{proof}


\begin{remark}
The upper bound provided by Theorem~\ref{thm:error_bound_linear} on the approximation error $d(p(X_k,\theta) \mid \Y_k), q_k(X_k, \theta))$ is determined by the outputs of FBOVI: $m_i(\theta), C_i(\theta)$ and $\nu_i(\theta)$ for $i=1,2,\cdots, k$. In practice, this upper bound can be estimated using the outputs of FBOVI by Monte Carlo method:
\begin{align*}
&\E_{\theta \sim \nu_j(\cdot)}\left[ \Psi_j(C_j(\theta), m_j(\theta), C_{j-1}(\theta), m_{j-1}(\theta), \theta) \right] \approx \frac{1}{N}\sum_{i=1}^N \Psi_j(C_j(\theta^{(i)}), m_j(\theta^{(i)}), C_{j-1}(\theta^{(i)}), m_{j-1}(\theta^{(i)}), \theta^{(i)}), \quad \theta^{(i)} \overset{\mathrm{i.i.d.}}{\sim} \nu_j(\cdot), \\
&\qquad \qquad Z_j(m_{j-1}, C_{j-1}, \nu_{j-1}) \approx \frac{1}{N^\prime}\sum_{i=1}^{N^\prime} \pdense_{\mathcal{N}}(y_j;H(\theta^{(i)})m_j^{*-}(\theta^{(i)}), H(\theta^{(i)})C_j^{*-}(\theta^{(i)})H(\theta^{(i)})^T+\Gamma(\theta^{(i)})), \quad \theta^{(i)} \overset{\mathrm{i.i.d.}}{\sim} \nu_{j-1}(\cdot).
\end{align*}

\end{remark}

\subsubsection{Nonlinear case}
When the dynamical model or the observation model is nonlinear, we use Gaussian filtering to approximate \( \rho_k^*(X_k \mid \theta) \) by a Gaussian:
\[
\rho_k^*(X_k \mid \theta) \approx \mathcal{N}\left(X_k; m_k^*(\theta), C_k^*(\theta)\right),
\]
where \( m_k^*(\theta) \) and \( C_k^*(\theta) \) are obtained via Gaussian filtering. Again, to avoid the complexity growth of time-dependent filtering operations, we represent \( m_k(\theta) \) and \( C_k(\theta) \) using neural networks with time-invariant architectures and train them to approximate \( m_k^*(\theta) \) and \( C_k^*(\theta) \). The procedure of computing the distribution $\rho_k(X_k \mid \theta)$ for the nonlinear case is illustrated in Figure~\ref{fig:stage2_nonlinear}.
\begin{figure}
     \centering
     \includegraphics[width=0.9\linewidth]{alg_visual/stage2_nonlinear.pdf}
     \caption{Procedure of computing the distribution $\rho_k(X_k \mid \theta)$ for nonlinear systems. The online conditional filtering distribution $\rho_k^*(X_k \mid \theta)$ is infeasible to compute and approximated by a Gaussian distribution $\mathcal{N}(X_k;m_k^*(\theta),C_k^*(\theta))$. We compute the distribution $\rho_k(X_k \mid \theta)$ using the mean function $m_k^*(\theta)$, the covariance function $C_k^*(\theta)$, and the distribution $\nu_k(\theta)$ which is computed at Stage 1.}
    \label{fig:stage2_nonlinear}
\end{figure}
A schematic overview of the complete procedure is presented in Figure~\ref{fig:procedure_onlineVI_2}.

\begin{figure}[t]
     \centering
     \includegraphics[width=0.7\linewidth]{alg_visual/alg_procedure.pdf}
     \caption{Scheme of FBOVI. Functions $F_m$ and $F_C$ are given by Kalman filter or Gaussian filtering.}
    \label{fig:procedure_onlineVI_2}
\end{figure}

\subsubsection*{Training Strategy for $m_k$ and $C_k$}
In principle, the neural networks $m_k$ and $C_k$ should be trained by directly minimizing the expected KL divergence:
\begin{align}
&\E_{\theta \sim \nu_k(\cdot)}\left[\KL\left(\rho_k(\cdot  \mid \theta) \,\|\, \rho_k^*(\cdot\mid \theta)\right)\right] \approx \frac{1}{N} \sum_{i=1}^N \KL\left(\rho_k(\cdot\mid \theta^{(i)}) \,\|\, \rho_k^*(\cdot \mid \theta^{(i)})\right), \nonumber \\
&\quad  = \frac{1}{2N} \sum_{i=1}^N\log \big \lvert C_k^*(\theta^{(i)}) \big \rvert - \log \big \lvert C_k(\theta^{(i)}) \big \rvert - n + \Tr\left(\left(C_k^*(\theta^{(i)})\right)^{-1}C_k(\theta^{(i)})\right) + \delta(\theta^{(i)})^T\left(C_k^*(\theta^{(i)})\right)^{-1}\delta(\theta^{(i)}),
\label{eq:kl_linear_step2}
\end{align}
with \( \theta^{(i)} \sim \nu_k(\cdot) \) and $\delta(\theta^{(i)}) = m_k(\theta^{(i)})-m_k^*(\theta^{(i)})$. 

From Equation~\eqref{eq:kl_linear_step2}, we can see that evaluating the expected KL divergence $\E_{\theta \sim \nu_k(\cdot)}\left[\KL\left(\rho_k(\cdot \mid \theta) \,\|\, \rho_k^*(\cdot \mid \theta)\right)\right]$ requires evaluating the determinants of the covariances $C_k^*(\theta^{(i)})$ and $C_k(\theta^{(i)})$, as well as the inverse of the covariance $C_k^*(\theta^{(i)})$, for each samples of $\theta$, $\theta^{(i)}$. This can be computationally expensive, especially for high state dimension. When the requirement for computational efficiency is high, the following two surrogate losses can be employed for the training of neural networks used to represent $m_k(\theta)$ and $C_k(\theta)$:
\begin{align}
\frac{1}{N} \sum_{i=1}^N \big  \lVert m_k(\theta^{(i)}) - m_k^*(\theta^{(i)})\big \rVert_2^2, \quad
\frac{1}{N} \sum_{i=1}^N \big  \lVert C_k(\theta^{(i)}) - C_k^*(\theta^{(i)})\big \rVert_F^2, \quad \theta^{(i)} \sim \nu_k(\cdot).
\end{align}


\subsubsection{Incorporation of ensemble Kalman filter}
In high-dimensional nonlinear systems, many Gaussian filtering methods, including the Unscented Kalman Filter (UKF) and the Gauss–Hermite Kalman Filter (GHKF), encounter significant scalability challenges. For instance, in an $n$-dimensional state space, evaluating the integrals in Equation~\eqref{eq:GF_eqs} using the UKF requires $2n+1$ function evaluations, and, more critically, the UKF’s covariance estimates become increasingly unstable as the dimensionality grows. Similarly, the GHKF requires $l^n$ function evaluations when employing an $l$-th-order Hermite polynomial, causing the computational cost to rise drastically with $n$. Consequently, directly incorporating these Gaussian filtering methods into Stage~2 of our framework would also constrain the overall scalability of the proposed approach. To overcome this, the proposed framework supports the use of the EnKF, which is known for its scalability, at Stage 2. 

A distinctive aspect of our framework is that only the approximate parameter posterior \( \nu_k(\theta) \), and the function-based representations \( m_k(\theta) \) and \( C_k(\theta) \), are propagated to the next time step. State particles are not passed forward. As a result, the standard EnKF is not directly applicable. Instead, we propose an \textit{every-step resampling} procedure. For a given sample \( \theta^{(i)} \), we first draw \( M \) state particles from the previous approximate conditional posterior \( \rho_{k-1}(X_{k-1} \mid \theta^{(i)}) \):
\begin{equation*}
X_{k-1}^{(j)} \sim \mathcal{N}\left(m_{k-1}(\theta^{(i)}), C_{k-1}(\theta^{(i)})\right), \quad j = 1,\dots,M.
\end{equation*}
These samples \( \{X_{k-1}^{(j)}\}_{j=1}^M \) are then propagated using the standard EnKF update to generate an ensemble \( \{X_k^{(j)}\}_{j=1}^M \), which approximates the online conditional filtering distribution \( q_k^*(X_k \mid \theta^{(i)}) \). Finally, the sample mean and covariance of \( \{X_k^{(j)}\}_{j=1}^M \) are computed and assigned to \( m_k^*(\theta^{(i)}) \) and \( C_k^*(\theta^{(i)}) \), respectively. The full procedure is outlined in Algorithm~\ref{alg:enkf_update}.

\begin{algorithm}[t]
\caption{Single-step EnKF-based computation for $m_k^*(\theta)$ and $C_k^*(\theta)$}
\label{alg:enkf_update}
\KwIn{
    \\
    $\nu_k(\theta)$: parameter posterior at time $k$ \\
    $m_{k-1}(\theta)$, $C_{k-1}(\theta)$: mean and covariance functions of the previous conditional posterior $\rho_{k-1}(X_{k-1} \mid \theta)$\\
    $y_k$: current observation \\
    $\Phi(\cdot\,;\theta)$: state transition function \\
    $h(\cdot;\theta)$: observation model\\
    $\Sigma(\theta)$, $\Gamma(\theta)$: process noise covariance and measurement noise covariance \\
    $M$: number of ensemble members \\
    $N$: number of samples $\theta^{(i)} \sim \nu_k(\theta)$
}
\KwOut{
    Training data $\{ \theta^{(i)}, m_k^*(\theta^{(i)}), C_k^*(\theta^{(i)}) \}_{i=1}^N$
}

\For{$i = 1$ \KwTo $N$}{
    Sample $\theta^{(i)} \sim \nu_k(\theta)$ \\
    
    \For{$j = 1$ \KwTo $M$}{
        Sample $X_{k-1}^{(j)} \sim \mathcal{N}(m_{k-1}(\theta^{(i)}), C_{k-1}(\theta^{(i)}))$ {\color{red} $\leftarrow$ Re-sampling happens here}\\
        Sample $\xi_k^{(j)} \sim \mathcal{N}(0, \Sigma(\theta^{(i)}))$ \\
        Compute $X_k^{(j)-} \leftarrow \Phi(X_{k-1}^{(j)};\theta^{(i)}) + \xi_k^{(j)}$
    }

    Compute $m_k^-(\theta^{(i)}) \leftarrow \frac{1}{M} \sum_{j=1}^M X_k^{(j)-}$ \\
    Compute $C_k^-(\theta^{(i)}) \leftarrow \frac{1}{M-1} \sum_{j=1}^M (X_k^{(j)-} - m_k^-(\theta^{(i)})) (X_k^{(j)-} - m_k^-(\theta^{(i)}))^T$

    \For{$j = 1$ \KwTo $M$}{
        Sample $\eta_k^{(j)} \sim \mathcal{N}(0, \Gamma(\theta^{(i)}))$ \\
        Define perturbed observation $y_k^{(j)} \leftarrow y_k - \eta_k^{(j)}$
    }

    Compute $\mu_y(\theta^{(i)}) \leftarrow \frac{1}{M} \sum_{j=1}^M (h(X_k^{(j)-};\theta^{(i)})  + \eta_k^{(j)})$ \\
    Compute $S(\theta^{(i)}) \leftarrow \frac{1}{M-1} \sum_{j=1}^M (h(X_k^{(j)-};\theta^{(i)}) + \eta_k^{(j)} - \mu_y(\theta^{(i)}))(h(X_k^{(j)-};\theta^{(i)}) + \eta_k^{(j)} - \mu_y(\theta^{(i)}))^T$ \\
    Compute $U(\theta^{(i)}) \leftarrow \frac{1}{M-1} \sum_{j=1}^M (X_k^{(j)-} - m_k^-(\theta^{(i)}))(h(X_k^{(j)-};\theta^{(i)}) + \eta_k^{(j)} - \mu_y(\theta^{(i)}))^T$

    \For{$j = 1$ \KwTo $M$}{
        Compute $X_k^{(j)} \leftarrow X_k^{(j)-} + U(\theta^{(i)}) S(\theta^{(i)})^{-1} (y_k^{(j)} - h(X_k^{(j)-};\theta^{(i)}))$
    }

    Compute $m_k^*(\theta^{(i)}) \leftarrow \frac{1}{M} \sum_{j=1}^M X_k^{(j)}$ \\
    Compute $C_k^*(\theta^{(i)}) \leftarrow \frac{1}{M-1} \sum_{j=1}^M (X_k^{(j)} - m_k^*(\theta^{(i)})) (X_k^{(j)} - m_k^*(\theta^{(i)}))^T$
    
}

\Return $\{ \theta^{(i)}, m_k^*(\theta^{(i)}), C_k^*(\theta^{(i)}) \}_{i=1}^N$
\end{algorithm}

Once the pairs \( \{ \theta^{(i)}, m_k^*(\theta^{(i)}), C_k^*(\theta^{(i)}) \} \) are computed for all \( i = 1,\dots,N \), they are used as training data to fit the neural networks representing \( m_k(\theta) \) and \( C_k(\theta) \). These learned functions are then passed to the next time step \( k+1 \). It can be seen that, unlike standard EnKF approaches, which propagate state particles forward in time, the incorporated EnKF in our framework is only used as a training data generating procedure to generate the target mean $m_k^*(\theta^{(i)})$ and target covariance $C_k^*(\theta^{(i)})$, for each sample $\theta^{(i)}$.

It is noteworthy that our framework can also operate when a non-Gaussian distribution is used to represent the conditional distribution $\rho_k(X_k \mid \theta)$. The details of computing $\nu_k(\theta)$ and $\rho_k(X_k \mid \theta)$ in this setting are provided in~\ref{app:non_Gaussian}.

\section{Numerical Experiments}
\label{sec:experiments}
In this section, we implement the proposed FBOVI algorithm across three numerical experiments and compare its performance against three online joint posterior approximation methods: the joint PF, joint UKF and joint EnKF. All these methods assign artificial dynamics to the unknown model parameters and augment the system state with them. Standard PF, UKF, and EnKF are then applied to the augmented system, respectively. The joint PF tested in this section employs the bootstrap PF. Details of the hyperparameters used for the joint PF, joint UKF, and joint EnKF in the experiments are provided in~\ref{app:hyper_param}. Each of these methods has its own strengths: the joint PF can provide accurate joint posterior approximations in low-dimensional systems; the joint EnKF is effective for large-scale problems; and the joint UKF is widely adopted for its balance between scalability and computational efficiency. The experimental setups are designed to capture several of the most common and significant challenges encountered in recent application scenarios of parameter–state estimation methods, such as DTs:
\begin{enumerate}
\item \textit{Observation-related challenges}: First, as previously discussed, observational data are typically noisy and often incomplete. Second, the observation model may be partially unknown, adding additional challenges to the inference process.
\item \textit{Model misspecification}: The dynamical models used by parameter-state estimation methods are often inaccurate due to limited knowledge or the inherent complexity of real-world physical systems. Furthermore, online parameter-state estimation methods frequently rely on simplified, low-fidelity models that differ from the true physical systems in order to reduce computational burden and allow rapid updates. This is quite often the case in e.g., digital twins.
\item \textit{Chaotic dynamics}: Many physical systems encountered in parameter–state estimation problems exhibit inherently chaotic behavior, wherein small perturbations in initial conditions or inputs can cause significant changes in system trajectories. This sensitivity poses a major challenge for data-driven estimation and prediction.
\item \textit{High dimensionality}: parameter-state estimation methods must remain scalable in order to provide reliable guidance for high-dimensional systems, a requirement that is particularly demanding for online inference algorithms.
\end{enumerate}

To evaluate the estimation and prediction capabilities of FBOVI under these conditions and illustrate how different Gaussian filtering methods can be integrated into FBOVI depending on the problem settings, we present the following three experiments:

\begin{enumerate}
\item In Section~\ref{sec:low_dimension}, we consider a two-dimensional linear pendulum system with partially unknown dynamical and observation models. As the underlying system is linear, we use the Kalman filter within FBOVI to compute the filtering distribution conditioned on the parameters. 
\item In Section~\ref{sec:Lorentz}, we study a ten-dimensional Lorenz-96 system in a chaotic regime. FBOVI is implemented in two cases: one with model error and one without. The UKF is employed within FBOVI to balance computational efficiency and estimation accuracy. 
\item In Section~\ref{sec:Burger}, FBOVI is applied to a nonlinear convection–diffusion transport model. After spatial discretization, the system becomes 51-dimensional. To ensure scalability in this high-dimensional setting, we integrate EnKF into FBOVI. 
\end{enumerate}

All systems considered in these experiments are partially observable. Different evaluation metrics and validation strategies are employed to assess the estimation and prediction performance of FBOVI. The detailed experimental configurations, criteria, and results are provided in the subsequent subsections. 

\subsection{Single linear pendulum system}
\label{sec:low_dimension}
In this experiment, we consider a two-dimensional linear pendulum system governed by
\begin{align}
\begin{bmatrix}\dot{x}_1 \\ \dot{x}_2 \end{bmatrix} = 
\begin{bmatrix}x_2 \\ -\tfrac{g}{l}x_1 \end{bmatrix} + w,
\end{align}
where $g=9.8\,\mathrm{m/s^2}$ is the gravitational constant, $l=1.2\,\mathrm{m}$ denotes the pendulum length, and $w$ represents the random perturbations in real world. The initial state is chosen as $x_0=\begin{bmatrix}0.5 & 0.5\end{bmatrix}^T$. Reference trajectories are generated with the forward Euler scheme using a step size of $\Delta t=0.1\,\mathrm{s}$. Specifically, the reference trajectories are generated according to
\begin{equation}
X_{k+1} = \begin{bmatrix}0.9594 & 0.0986 \\ -0.8056 & 0.9594 \end{bmatrix}X_k + \eta_k, \qquad \eta_k \sim \mathcal{N}(0, 0.01I_{2\times2}).
\end{equation}
Noisy measurements of the states are collected according to the model $Y_k = HX_k + V_k, \quad V_k \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,0.01)$. Throughout this section, the learning model does not have access to the true initial state. Instead, the prior distribution for $X_0$ is taken to be $\mathcal{N}([
    3, 4.5
]^T, 4I_{2\times2})$.
\subsubsection{System with partially unknown dynamical model}
\label{sec:pendulum_part1}
In this section, we focus on learning the parameters in a partially unknown dynamical model and learning partially observable states. We show that approximate posteriors obtained by FBOVI are close to the true posteriors at most time steps. 

The dynamical model used for learning is parameterized as:
\begin{equation}
X_{k+1} = \begin{bmatrix}\theta_1&0.0986\\ \theta_2&\theta_1\end{bmatrix}X_k + W_k, \qquad W_k \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,0.01I_{2\times2}),
\end{equation}
where $\theta=(\theta_1,\theta_2)$ represent the unknown parameters with a prior ${\mathcal{N}}(0,I_{2\times2})$. The observation matrix $H=[1 \quad 0]$, the process noise covariance and the measurement noise covariance are known.
The simulation is conducted over a span of 50 time steps. Three online parameter-state estimation algorithms, FBOVI, joint PF, and joint UKF, are employed to approximate the joint posterior distribution $P(X_k,\theta \mid \Y_k)$ at each time step $k$.  FBOVI utilizes a Gaussian distribution to represent the distribution $\nu_k(\theta)$, while the joint PF utilizes $1\times 10^4$ particles to represent the joint posterior. The rest of the experiment settings can be found in~\ref{app:ep1}.

Figure~\ref{fig:marginal_dist_eg1} depicts the marginal distributions of the parameters and state components produced by different algorithms at each time step for a single data realization. The results show that FBOVI yields marginal distributions that closely track the ground truth across most time steps. Although the joint PF employs a large number of particles, the mean estimated by FBOVI remains closer to the true posterior mean most of the time.

As time progresses, the means of the marginal distributions of parameters and states obtained by FBOVI gradually converge to their true values. However, during the time from around step $11$ to $25$, FBOVI tends to overestimate the uncertainty of $\theta_2$, as shown in Figure~\ref{fig:dist_theta2}. This can be attributed to the fact that although the prior of $\theta$ is Gaussian, the posterior of $\theta$ is typically highly non-Gaussian in the early stages due to the limited amount of data points incorporated. Consequently, a Gaussian distribution may not adequately capture the posterior of $\theta$. Moreover, as online inference progresses, approximation errors accumulate over time. As more data are assimilated, the posterior is expected to become increasingly Gaussian, and FBOVI regains its ability to provide accurate uncertainty estimation. This highlights FBOVI's capability to yield an accurate posterior approximation online, starting from an initially inaccurate approximation, provided that the variational distribution family used for the parameters can adequately represent the parameter posterior. In contrast, the joint UKF fails to demonstrate this capability, as evidenced by the divergence of its approximated posterior when more data points are assimilated, and the true parameter posterior approaches Gaussian distribution. This suggests that the joint UKF may struggle to provide a good approximation of the posterior when given an inaccurate prior, or that the approximation error of the joint UKF in the early stages is too substantial, and more data points especially noisy data points cannot provide sufficient information to mitigate the accumulated error within the framework of the joint UKF. 

\begin{figure}
    \centering
    \begin{subfigure}[b]{0.98\linewidth}
        \centering
        \includegraphics[width=\textwidth]{numerical_examples/example1/legend_marginal_part1_e1.pdf}
    \end{subfigure}

    \vspace{0.8em}
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/example1/theta1_e1.pdf}
         \caption{Distribution of $\theta_1$}
         \label{fig:dist_theta1}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/example1/theta2_e1.pdf}
         \caption{Distribution of $\theta_2$}
         \label{fig:dist_theta2}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/example1/x1_e1.pdf}
         \caption{Distribution of $x_1(k)$}
         \label{fig:dist_x1}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/example1/x2_e1.pdf}
         \caption{Distribution of $x_2(k)$ }
         \label{fig:dist_x2}
     \end{subfigure}
         \caption{\textbf{Single pendulum:} Approximate marginal distributions parameters at each time step $k$. FBOVI closely follows the true posteriors for most time steps, although it overestimates the uncertainty of $\theta_2$ between steps $10$ and $25$.}
         \label{fig:marginal_dist_eg1}
\end{figure}

Next, we evaluate the three online inference methods using $50$ independent data realizations. The estimation performance is assessed using the root mean square error (RMSE), which measures the average discrepancy between the true values of the states or parameters and the means of their corresponding marginal approximate posterior distributions over different realizations. Specifically, the RMSE for estimating the $i$-th component of the parameter $\theta$ and the $i$-th component of the state $X_k$ at time step $k$ are defined as 
\begin{align}
RMSE_{\theta_i}(k) := & \sqrt{\frac{1}{N}\sum_{j=1}^N\left(\hat{\theta}_{i}^{(j)}(k)-\bar{\theta}_i\right)^2}, %\label{eq:rmse_theta} \\ 
\quad RMSE_{x_i}(k)  :=  \sqrt{\frac{1}{N}\sum_{j=1}^N\left(\hat{x}_{i}^{(j)}(k)-\bar{x}_{i}^{(j)}(k)\right)^2}, \label{eq:rmse_state}
\end{align}
where $N=50$ is the total number of data realizations. Here, $\hat{\theta}_{i}^{(j)}(k)$ and $\hat{x}_{i}^{(j)}(k)$ denote the means of the marginal approximate posterior distributions for the $i$-th elements of $\theta$ and $X_k$, respectively, obtained from the $j$-th data realization at time step $k$. The true parameter value is denoted by $\bar{\theta}_i$, and $\bar{x}_{i}^{(j)}(k)$ represents the true value of the $i$-th state component at time step $k$ for the $j$-th realization.

The resulting $RMSE_{\theta_i}(k)$ and $RMSE_{x_i}(k)$ for $i=1,2$ and $k=1,2,\ldots,50$ are shown in Figure~\ref{fig:RMSE}.
Overall, FBOVI achieves the lowest RMSE for both parameter and state estimation across most time steps, outperforming the joint PF and joint UKF.

\begin{figure}[t]
    \centering
    \begin{subfigure}[b]{0.4\linewidth}
        \centering
        \includegraphics[width=\textwidth]{numerical_examples/example1_new/legend_mse_e1_new.pdf}
    \end{subfigure}

    \vspace{0.8em}
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/example1/theta1_mse_e1.pdf}
         \caption{$RMSE_{\theta_1}$ vs. time}
         \label{fig:rmse_theta1}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/example1/theta2_mse_e1.pdf}
         \caption{$RMSE_{\theta_2}$ vs. time}
         \label{fig:rmse_theta2}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/example1/x1_mse_e1.pdf}
         \caption{$RMSE_{x_1}$ vs. time}
         \label{fig:rmse_x1}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/example1/x2_mse_e1.pdf}
         \caption{$RMSE_{x_2}$ vs. time}
         \label{fig:rmse_x2}
     \end{subfigure}
         \caption{\textbf{Single pendulum:} Average estimation error of states and parameters over time. The figure compares the performance of FBOVI, joint PF, and joint UKF across $50$ independent data realizations. FBOVI achieves the lowest estimation error at most time steps.}
         \label{fig:RMSE}
\end{figure}

Next, we assess the predictive ability of the three online parameter–state estimation methods by computing the prediction RMSE over the same $50$ data realizations.
The prediction RMSE at time step $k$ is defined as
\begin{equation}
RMSE_{pred}(k) := \sqrt{\frac{1}{N}\sum_{j=1}^N\E_{(X_{k-1},\theta) \sim q_{k-1}^{(j)}\left(X_{k-1},\theta \right), W_{k-1} \sim \mathcal{N}\left(0,\Sigma(\theta)\right)}\left[\lVert (\Phi(X_{k-1};\theta)+W_{k-1}(\theta))-\bar{x}^{(j)}(k)\rVert _2^2\right]}, \label{eq:rmse_pred}
\end{equation}
where $q_{k-1}^{(j)}(X_{k-1},\theta)$ and $\bar{x}^{(j)}(k)$ denote the approximate joint posterior distribution at time step $k-1$ and the true state at time step $k$, respectively, corresponding to the $j$-th data realization. 


Figure~\ref{fig:rmse_pred} presents the prediction errors of different algorithms at each time step. FBOVI achieves the best predictive accuracy after time step $18$, although it underperforms the other two methods between steps $10$ and $14$. This temporary degradation can be partly attributed to its overestimation of uncertainty during that short period, as illustrated in Figure~\ref{fig:marginal_dist_eg1}. Since the prediction error is computed based on the joint posterior, FBOVI’s strong predictive performance demonstrates its ability to obtain accurate approximations of the joint posterior.
\begin{figure}
\centering
    \begin{subfigure}[b]{0.4\linewidth}
        \centering
        \includegraphics[width=\textwidth]{numerical_examples/example1_new/legend_mse_e1_new.pdf}
    \end{subfigure}

    \vspace{0.4em}
     \includegraphics[width=0.3\textwidth]{numerical_examples/example1/pred_e1.pdf}
     \caption{\textbf{Single pendulum:} time-step prediction error for the system with a partially unknown dynamical model. The figure shows prediction errors calculated using results from FBOVI, joint PF and joint UKF. FBOVI begins to achieve the most accurate predictions among the three methods after step $18$.}
    \label{fig:rmse_pred}
\end{figure}


\subsubsection{System with partially unknown dynamical and observation models}
In this section, we aim at learning the unknown parameters of the dynamical and observation model simultaneously. We show that the proposed method is able to yield good estimates of the dynamical and observation models as well as the states with both dynamical and observation models partially unknown.

The dynamical and observation models used by the learning agent are as follows:
\begin{equation*}
X_{k+1} = \begin{bmatrix}0.9594&\theta_1\\ -0.8056&0.9594\end{bmatrix}X_k + W_k, \quad W_k \sim \mathcal{N}(0, 0.01I_{2\times2}), \qquad Y_k = \begin{bmatrix}\theta_2&0\\0&1\end{bmatrix}X_k + V_k, \quad V_k \sim \mathcal{N}(0,0.01I_{2\times2}).
\end{equation*}
We assume that the parameters $\theta = (\theta_1,\theta_2)$ are unknown to the learning model. The prior of $\theta$ is set to $\mathcal{N}([1 \quad 0]^T, I_{2\times 2})$. The covariances of the process and measurement noises are assumed to be known. 

To evaluate the efficacy of the proposed algorithm, data is generated with $H=I_{2\times2}$ for 100 steps. Next, three online inference algorithms are implemented: FBOVI, joint PF and joint UKF. This problem is more challenging than the one discussed in Section~\ref{sec:pendulum_part1}. The joint PF with $10^4$ particles collapses and we increase the number of particles to $1\times 10^6$. 

The marginal distributions of $\theta$ and states obtained by different algorithms at each time step are shown in Figure \ref{fig:marginal_dist_eg3}. Remarkably, the means of FBOVI and joint PF for all parameters converge to the truth rapidly despite the inherent difficulty in learning $\theta_2$ as it constitutes an entry in the observation matrix $H$. In contrast, the entire distribution of $\theta_2$ obtained by the joint UKF deviates significantly from the truth for most of the duration. It can be seen that FBOVI and joint PF are both able to track the true states closely while the joint UKF tends to overestimate the amplitude of the trajectory of $x_1$, corresponding to an underestimation of $\theta_2$.
\begin{figure}
     \centering
    \begin{subfigure}[b]{0.95\linewidth}
        \centering
        \includegraphics[width=\textwidth]{numerical_examples/pendulum_part2/legend_e1_part2.pdf}
    \end{subfigure}

    \vspace{0.8em}
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/pendulum_part2/theta1_e1_part2.pdf}
         \caption{Distribution of $\theta_1$}
         \label{fig:dist_theta1_eg3}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/pendulum_part2/theta2_e1_part2.pdf}
         \caption{Distribution of $\theta_2$}
         \label{fig:dist_theta2_eg3}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/pendulum_part2/x1_e1_part2.pdf}
         \caption{Distribution of $x_1(k)$}
         \label{fig:dist_x1_eg3}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.24\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/pendulum_part2/x2_e1_part2.pdf}
         \caption{Distribution of $x_2(k)$}
         \label{fig:dist_x2_eg3}
     \end{subfigure}
         \caption{\textbf{Single pendulum:} approximate marginal distributions parameters and states for the single pendulum system with partially unknown dynamics and partially unknown observation model. For detailed analysis, the zoom-in views from time step $20$ to $100$ are provided in~\ref{fig:dist_theta1_eg3} and ~\ref{fig:dist_x1_eg3}. The estimates produced by FBOVI and the joint UKF converge to the truth rapidly.}
         \label{fig:marginal_dist_eg3}
\end{figure}

\subsection{Chaotic system: Lorenz 96 model}
\label{sec:Lorentz}
In this example, we evaluate the performance of the proposed method for parameter-state estimation in chaotic, partially observable dynamical systems, where learning is performed using both accurate models and models subject to structural errors. 

We consider a generalized Lorenz 96 model~\cite{Lorentz96_generalization}\footnote{The common version of Lorentz 96 model is a low-fidelity approximation of this generalized formulation.} with the state vector $x=[x_{1},x_{2},\cdots, x_{M}]^T$. The governing equations are given by
\begin{equation}
\dot{x_i} = \alpha x_{i-1}\left(x_{i+1}-x_{i-2}\right)-\beta x_{i}+F, \quad \forall i\in [1,M], 
\label{eq:Lorentz_dynamics_ct}
\end{equation}
where $x_{-1}:=x_{M-1}$, $x_{0}:=x_{M}$, and $x_{M+1}:=x_{1}$. The parameters $\alpha=1.1$ and $\beta=0.9$ are unknown. The external forcing $F=8$ is known to the learning agent and the system dimension is $M=10$. Following~\cite{Lorentz96_initial_condition}, the initial condition is set as $x_i = \cos(2\pi i/M), \quad \forall i \in [1,M]$, which is assumed to be unknown to the learning agent.
The system is simulated using a time step $\Delta t=0.01$. Noisy measurements of the odd-numbered states $x_1,x_3,x_5,x_7$, and $x_9$ are collected every 5 simulation steps. The measurement noise follows a Gaussian distribution with zero mean and covariance $I_{5\times5}$. This measurement model is known to the learning agent. To ensure that the learning process is performed when the system is operating in the chaotic regime, the learning process begins after the $3000$th simulation step. The 4th-order runge-kutta (RK4) method is used to generate reference solution trajectories and data.

Let $\theta = (\alpha, \beta)$ denote the vector of unknown parameters. We assign $\theta$ a Gaussian prior $\mathcal{N}(0_2, I_{2\times2})$. The discrete-time dynamical model used by the learning agent is as follows:
\begin{equation}
X_{k+1} = \Phi(X_k; \theta) + V_k, \quad V_k \sim \mathcal{N}(0, \sigma^2 I_{M \times M}),
\label{eq:learning_model}
\end{equation}
where the integer $k \geq 0$ represents the time instance, and each time step corresponds to a duration of $5\Delta t$. The function $\Phi$ is a low-fidelity, discrete-time approximation of the continuous dynamics in Equation~\eqref{eq:Lorentz_dynamics_ct}, obtained by applying the forward Euler method or RK4 over a time interval of length $5\Delta t$, with simulation time step $\Delta t$. We present the results of our method applying both the forward Euler method and RK4 method to computing the function $\Phi$, as detailed in Section~\ref{sec:correct_model} and Section~\ref{sec:wrong_model}. The joint PF with $10^6$ particles collapses in this example due to the increasing dimensionality, and we therefore compare the performance of FBOVI with the joint UKF.
\subsubsection{Correct model}
\label{sec:correct_model}
In this case, the RK4 method is used to obtain the function $\Phi(X_k;\theta)$. Therefore, the deterministic part of the model used for learning, $\Phi$, matches the dynamical system which is used to generate the reference solutions. The variance of the process noise is chosen to be $\sigma^2=0.5$. 

Figure~\ref{fig:param_accurateModel} presents the approximate marginal distributions of the parameters obtained by FBOVI and joint UKF. The FBOVI estimates converges rapidly to the truth parameter values, while joint UKF struggles to learn the parameters. Moreover, the ability to accurately estimate unobserved states is particularly noteworthy, given that only half of the state variables are observed and the model is partially unknown—further complicated by the system's operation in a chaotic regime.
\begin{figure}
     \centering
    \begin{subfigure}[b]{0.9\linewidth}
        \centering
        \includegraphics[width=\textwidth]{numerical_examples/Lorentz/legend.pdf}
    \end{subfigure}

    \vspace{0.8em}
   
     \begin{subfigure}[b]{0.25\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/alpha_e2.pdf}
         \caption{Approximate marginal distribution of $\alpha$}
         \label{fig:alpha_e2_1}
     \end{subfigure}
     \hspace{0.1\linewidth}
     \begin{subfigure}[b]{0.25\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/beta_e2.pdf}
         \caption{Approximate marginal distribution of $\beta$}
         \label{fig:beta_e2_1}
     \end{subfigure}
     \caption{\textbf{Lorentz 96 system with correct model:}
     approximate marginal posterior distributions of the parameters $\alpha$ and $\beta$ when the deterministic part of the model used by the learning agent matches the true model. FBOVI rapidly converges to the true parameter values.}
     \label{fig:param_accurateModel}
\end{figure}

\begin{figure}[t]
     \centering
    \begin{subfigure}[b]{1.0\linewidth}
        \centering
        \includegraphics[width=\textwidth]{numerical_examples/Lorentz/legend_data.pdf}
    \end{subfigure}

     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/x1_e2.pdf}
         \caption{Distribution of $x_1(k)$}
         \label{fig:x1_e2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/x2_e2.pdf}
         \caption{Distribution of $x_2(k)$}
         \label{fig:x2_e2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/x3_e2.pdf}
         \caption{Distribution of $x_3(k)$}
         \label{fig:x3_e2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/x4_e2.pdf}
         \caption{Distribution of $x_4(k)$ }
         \label{fig:x4_e2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/x5_e2.pdf}
         \caption{Distribution of $x_5(k)$}
         \label{fig:x5_e2}
     \end{subfigure}
     \vfill
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/x6_e2.pdf}
         \caption{Distribution of $x_6(k)$ }
         \label{fig:x6_e2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/x7_e2.pdf}
         \caption{Distribution of $x_7(k)$ }
         \label{fig:x7_e2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/x8_e2.pdf}
         \caption{Distribution of $x_8(k)$}
         \label{fig:x8_e2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/x9_e2.pdf}
         \caption{Distribution of $x_9(k)$ }
         \label{fig:x9_e2}
    \end{subfigure}
    \hspace{0.01\linewidth}
    \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/correct_model/x10_e2.pdf}
         \caption{Distribution of $x_{10}(k)$ }
         \label{fig:x10_e2}
     \end{subfigure}
         \caption{\textbf{Lorentz 96 system with correct model:} approximate marginal posterior distributions of the state variables when the deterministic part of the model used by the learning agent matches the true model. Shaded regions represent the $95\%$ credible intervals. FBOVI accurately tracks both observed and unobserved states.}
         \label{fig:state_accurateModel}
\end{figure}

Figure~\ref{fig:state_pred_accurateModel} shows the distribution of the predicted state $X_k$, generated using the approximate joint posterior obtained at the previous learning step. Specifically, we draw a set of $N$ samples $\{(X_{k-1}^{(i)}, \theta^{(i)})\}_{i=1}^N$ from the approximate joint posterior $q_{k-1}(X_{k-1},\theta)$. For each sample $(X_{k-1}^{(i)}, \theta^{(i)})$, we compute a predicted state $\hat{X}_k^{(i)}$ using the model in Equation~\eqref{eq:learning_model}:
\begin{equation*}
\hat{X}_k^{(i)} = \Phi(X_{k-1}^{(i)};\theta^{(i)}) + V_{k-1}, \quad V_{k-1} \sim \mathcal{N}(0,\sigma^2 I_{M \times M}).
\end{equation*}
The resulting set $\{\hat{X}_k^{(i)}\}_{i=1}^N$ represents the predicted distribution of $X_k$. As illustrated in the figure, FBOVI achieves accurate predictions for most time steps.
\begin{figure}[t]
     \centering
    \begin{subfigure}[b]{0.9\linewidth}
        \centering
        \includegraphics[width=\textwidth]{numerical_examples/Lorentz/legend.pdf}
    \end{subfigure}

     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/correct_model/x1_e2_pred.pdf}
         \caption{Distribution of the predicted $x_1(k)$}
         \label{fig:x1_e2_pred}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/correct_model/x2_e2_pred.pdf}
         \caption{Distribution of the predicted $x_2(k)$}
         \label{fig:x2_e2_pred}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/correct_model/x3_e2_pred.pdf}
         \caption{Distribution of the predicted $x_3(k)$}
         \label{fig:x3_e2_pred}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/correct_model/x4_e2_pred.pdf}
         \caption{Distribution of the predicted $x_4(k)$ }
         \label{fig:x4_e2_pred}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/correct_model/x5_e2_pred.pdf}
         \caption{Distribution of the predicted $x_5(k)$}
         \label{fig:x5_e2_pred}
     \end{subfigure}
     \vfill
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/correct_model/x6_e2_pred.pdf}
         \caption{Distribution of the predicted $x_6(k)$ }
         \label{fig:x6_e2_pred}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/correct_model/x7_e2_pred.pdf}
         \caption{Distribution of the predicted $x_7(k)$ }
         \label{fig:x7_e2_pred}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/correct_model/x8_e2_pred.pdf}
         \caption{Distribution of the predicted $x_8(k)$}
         \label{fig:x8_e2_pred}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/correct_model/x9_e2_pred.pdf}
         \caption{Distribution of the predicted $x_9(k)$ }
         \label{fig:x9_e2_pred}
    \end{subfigure}
    \hspace{0.01\linewidth}
    \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/correct_model/x10_e2_pred.pdf}
         \caption{Distribution of the predicted $x_{10}(k)$ }
         \label{fig:x10_e2_pred}
     \end{subfigure}
         \caption{\textbf{Lorentz 96 system with correct model (prediction):} the distributions of the predicted state variables. The prediction is based on the approximate joint posterior of the state and parameters obtained at the previous learning step. The deterministic part of the model used by the learning agent matches the true system. Shaded regions indicate the $95\%$ credible intervals. FBOVI accurately forecasts all state components.}
         \label{fig:state_pred_accurateModel}
\end{figure}

In the early stages, the joint distribution in nonlinear systems is often non-Gaussian. To demonstrate that FBOVI is able to capture such non-Gaussian features, we present the approximate joint posteriors obtained by FBOVI and the joint UKF. Figure~\ref{fig:joint_dist_e2} shows representative examples of these approximate joint posterior distributions. The results show that FBOVI is capable of capturing significant non-Gaussian features in both parameter-state and state-state joint distributions. Importantly, although FBOVI employs Gaussian filtering to approximate the conditional distribution of the state given the parameters, the absence of a mean-field assumption allows the resulting joint distributions to preserve their inherent non-Gaussian structure. As a result, the marginal posterior distributions of the states, after integrating out the parameters, can still exhibit rich, non-Gaussian behavior.
\begin{figure}[t]
     \centering
    \begin{subfigure}[b]{0.8\linewidth}
        \centering
        \includegraphics[width=\textwidth]{numerical_examples/Lorentz/joint_dist/legend_joint_dist_e2.pdf}
    \end{subfigure}
    
     \begin{subfigure}[b]{0.3\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/joint_dist/alpha_x1_step5.pdf}
         \caption{ joint distribution of $x_1$ and $\alpha$}
         \label{fig:alpha_x1_alpha}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.3\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/joint_dist/alpha_x2_step5.pdf}
         \caption{joint distribution of $x_2$ and $\alpha$}
         \label{fig:alpha_x2_alpha}
     \end{subfigure}
     \vfill
     \begin{subfigure}[b]{0.3\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/joint_dist/beta_x10_step5.pdf}
         \caption{joint distribution of $x_{10}$ and $\beta$}
         \label{fig:beta_x10_alpha}
     \end{subfigure}
    \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.3\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/joint_dist/x2_x4_step5.pdf}
         \caption{joint distribution of $x_2$ and $x_4$}
         \label{fig:x2_x4_alpha}
     \end{subfigure}
     \caption{\textbf{Lorentz 96 system with correct model:} representative examples of approximate joint posterior distributions at simulation step $25$ for the Lorenz 96 system under the correct model. Contour lines indicate density levels of the joint distributions. FBOVI successfully captures the non-Gaussian characteristics in both parameter-state and state-state joint distributions. Notably, although Gaussian filtering is used to approximate the conditional distributions of the state given parameters, the overall joint distributions remain expressive due to the absence of mean-field assumptions.}
         \label{fig:joint_dist_e2}
\end{figure}

\subsubsection{Incorrect model}
\label{sec:wrong_model}
Next, we assess whether the proposed method can still accurately estimate the states when the model used for learning deviates from the true system dynamics. Specifically, we use the forward Euler method to compute the deterministic component of the learning model, $\Phi$. Consequently, $\Phi$ differs from the true model. To compensate for the model-form error, the process noise variance is increased to $\sigma^2=2$. In this setting, the notion of ``true values'' for the parameters no longer applies, as the model form used for learning does not match the true dynamics. We evaluate the performance of the proposed method based on its ability to estimate the current states, with particular focus on the unobserved ones, and its predictive accuracy for the subsequent time step.

Figure~\ref{fig:state_inaccurateModel} shows the approximate marginal posterior distributions of the state variables. Despite the use of an inaccurate model form, FBOVI is able to track all states reasonably well. However, in comparison to the results obtained under the correct model form, the estimation quality around the peaks and troughs of the trajectories slightly degrades. This degradation is likely attributable to the limited expressiveness of the incorrect model used for learning. 
\begin{figure}[t]
     \centering
    \begin{subfigure}[b]{1.0\linewidth}
        \centering
        \includegraphics[width=\textwidth]{numerical_examples/Lorentz/legend_data.pdf}
    \end{subfigure}

     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/incorrect_model/x1_e2_2.pdf}
         \caption{Distribution of $x_1(k)$}
         \label{fig:x1_e2_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/incorrect_model/x2_e2_2.pdf}
         \caption{Distribution of $x_2(k)$}
         \label{fig:x2_e2_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/incorrect_model/x3_e2_2.pdf}
         \caption{Distribution of $x_3(k)$}
         \label{fig:x3_e2_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/incorrect_model/x4_e2_2.pdf}
         \caption{Distribution of $x_4(k)$ }
         \label{fig:x4_e2_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/incorrect_model/x5_e2_2.pdf}
         \caption{Distribution of $x_5(k)$}
         \label{fig:x5_e2_2}
     \end{subfigure}
     \vfill
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/incorrect_model/x6_e2_2.pdf}
         \caption{Distribution of $x_6(k)$ }
         \label{fig:x6_e2_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/incorrect_model/x7_e2_2.pdf}
         \caption{Distribution of $x_7(k)$ }
         \label{fig:x7_e2_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/incorrect_model/x8_e2_2.pdf}
         \caption{Distribution of $x_8(k)$}
         \label{fig:x8_e2_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/incorrect_model/x9_e2_2.pdf}
         \caption{Distribution of $x_9(k)$ }
         \label{fig:x9_e2_2}
    \end{subfigure}
    \hspace{0.001\linewidth}
    \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/marginal_dist/incorrect_model/x10_e2_2.pdf}
         \caption{Distribution of $x_{10}(k)$ }
         \label{fig:x10_e2_2}
     \end{subfigure}
         \caption{\textbf{Lorentz 96 system with model form error:} approximate marginal posterior distributions of the states when the deterministic part of the model used for learning differs from the true system. The shaded regions represent the $95\%$ credible intervals. Despite model-form inaccuracies, FBOVI provides robust state estimates across all components. }
         \label{fig:state_inaccurateModel}
\end{figure}

Figure~\ref{fig:state_pred_inaccurateModel} presents the distributions of the predicted states. As described in Section~\ref{sec:correct_model}, the prediction at each learning step is based on the approximate joint posterior obtained at the previous learning step.  Despite the presence of model-form errors, FBOVI achieves reasonably accurate predictions. Compared to the results under the correct model form (shown in Figure~\ref{fig:state_pred_accurateModel}), the predictive accuracy around the peaks and troughs is noticeably lower. This lower predictive accuracy is expected, as structural model inaccuracies inherently limit the predictive capability of the learned model and subsequently affect the accuracy of state prediction.
\begin{figure}[t]
     \centering
    \begin{subfigure}[b]{0.9\linewidth}
        \centering
        \includegraphics[width=\textwidth]{numerical_examples/Lorentz/legend.pdf}
    \end{subfigure}

     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/incorrect_model/x1_e2_pred_2.pdf}
         \caption{Distribution of the predicted $x_1(k)$}
         \label{fig:x1_e2_pred_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/incorrect_model/x2_e2_pred_2.pdf}
         \caption{Distribution of the predicted $x_2(k)$}
         \label{fig:x2_e2_pred_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/incorrect_model/x3_e2_pred_2.pdf}
         \caption{Distribution of the predicted $x_3(k)$}
         \label{fig:x3_e2_pred_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/incorrect_model/x4_e2_pred_2.pdf}
         \caption{Distribution of the predicted $x_4(k)$ }
         \label{fig:x4_e2_pred_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/incorrect_model/x5_e2_pred_2.pdf}
         \caption{Distribution of the predicted $x_5(k)$}
         \label{fig:x5_e2_pred_2}
     \end{subfigure}
     \vfill
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/incorrect_model/x6_e2_pred_2.pdf}
         \caption{Distribution of the predicted $x_6(k)$ }
         \label{fig:x6_e2_pred_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/incorrect_model/x7_e2_pred_2.pdf}
         \caption{Distribution of the predicted $x_7(k)$ }
         \label{fig:x7_e2_pred_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/incorrect_model/x8_e2_pred_2.pdf}
         \caption{Distribution of the predicted $x_8(k)$}
         \label{fig:x8_e2_pred_2}
     \end{subfigure}
     \hspace{0.01\linewidth}
     \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/incorrect_model/x9_e2_pred_2.pdf}
         \caption{Distribution of the predicted $x_9(k)$ }
         \label{fig:x9_e2_pred_2}
    \end{subfigure}
    \hspace{0.01\linewidth}
    \begin{subfigure}[b]{0.18\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Lorentz/state_pred/incorrect_model/x10_e2_pred_2.pdf}
         \caption{Distribution of the predicted $x_{10}(k)$ }
         \label{fig:x10_e2_pred_2}
     \end{subfigure}
         \caption{\textbf{Lorentz 96 system with model form error (prediction):} distributions of the predicted state variables when the deterministic part of the model used by the learning agent differs from the true system. The prediction for the state at each time step is based on the approximate joint posterior obtained at the previous learning step. Shaded regions denote the $95\%$ credible intervals. FBOVI achieves relatively high prediction accuracy despite the presence of model-form errors.}
         \label{fig:state_pred_inaccurateModel}
\end{figure}


\subsection{Nonlinear convection--diffusion transport model}
\label{sec:Burger}
In this section, we consider a one-dimensional nonlinear convection--diffusion transport model given by
\begin{equation}
\frac{\partial u}{\partial t} + \alpha u \frac{\partial u}{\partial x} = \nu \frac{\partial^2 u}{\partial x^2},
\label{eq:convection-diffusion}
\end{equation}
for $u = u(x,t)$ with $x \in [0,1]$. Here, $\alpha$ denotes the nonlinear convection coefficient and $\nu$ the diffusion coefficient. Throughout this study, we set $\alpha = 1$\footnote{Equation~\eqref{eq:convection-diffusion} reduces to Burgers’ equation when $\alpha = 1$.} and $\nu = 0.01$. The initial condition is prescribed as
\begin{equation*}
u(x,0) = \sin(2\pi x) + 0.5\cos(4\pi x) + 10,
\end{equation*}
together with periodic boundary conditions $u(0,t) = u(1,t)$.


The reference solution is computed using the forward Euler method with time interval $\Delta t = 0.001$ on a uniform grid with mesh size $h = 0.02$. This discretization yields 51 equally spaced grid points, with the first at $x=0$ and the last at $x=1$. Based on this reference solution, measurements are taken at the $(3n+1)$-th and $(3n+2)$-th grid points for $n=0,1,\ldots,16$. At each time step $t_k = k \Delta t$, the measurements are corrupted by Gaussian noise $W_k \sim \mathcal{N}(0, I_{34 \times 34})$, so that noisy observations are available at $2/3$ of the spatial grid points.

For online inference, we employ the following $51$-dimensional dynamical model:
\begin{equation}
X_{k+1} = \Phi(X_k;\theta) + V_k, 
\quad V_k \sim \mathcal{N}(0, 0.01 I_{51 \times 51}), 
\label{eq:dynamical_model_e3}
\end{equation}
where the state $X_k \in \mathbb{R}^{51}$ represents the solution values at the 51 grid points at time $t_k$. The mapping $\Phi$ denotes the numerical propagation of $X_k$ to $X_{k+1}$, obtained using the same temporal and spatial discretization scheme as in the reference solution for \eqref{eq:convection-diffusion}. It is important to note that the numerical method used to generate the reference solution differs from that used in $\Phi$. For the reference solution, the upwind scheme is employed to compute the spatial derivative $\frac{\partial u}{\partial x}$, which is stable when the viscosity $\nu$ is small—this is the case here. In contrast, the model $\Phi$ uses the simple central difference scheme for computing the derivative $\frac{\partial u}{\partial x}$, which is less stable under this condition. The parameter vector $\theta = (\alpha, \nu)$ is treated as unknown, with prior distribution $\mathcal{N}([0,1]^T, 4I_{2 \times 2})$. Because of the model-form error, there are no “true” parameter values. The initial state $X_0$ is assigned a prior $\mathcal{N}(\mu_0, 4I_{51 \times 51})$, where the prior mean $\mu_0$ is randomly drawn according to $\mu_0 \sim \mathcal{N}(\mathbf{10}_{51}, 9I_{51 \times 51})$, with $\mathbf{10}_{51}$ denoting the 51-dimensional all-10 vector.

As the dynamical system~\eqref{eq:dynamical_model_e3} used for learning is high-dimensional, we employ the EnKF,  which is effective for large-scale problems, within the framework of FBOVI in Stage 2. To demonstrate the effectiveness of FBOVI, we compare its performance with that of joint EnKF, a parameter-state estimation method commonly applied to large-scale state–parameter estimation problems.  

The means of the marginal distributions of the solution $u(x,t)$ obtained by FBOVI and joint EnKF, along with the reference solution, are shown in Figure~\ref{fig:state_e3}. We can see that the mean computed by FBOVI closely follows the reference solution. This demonstrates that, although the prior for the initial solution value is far from the reference, FBOVI quickly adapts and begins to track the reference within a short time. In contrast, the discrepancy between the reference and the joint EnKF mean is considerably larger during the early stage ($t < 0.02$).  

\begin{figure}[t]
     \centering
     \begin{subfigure}[b]{0.3\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Burger/state_reference_e3.pdf}
         \caption{Reference solution $u(x,t)$}
         \label{fig:refer_e3}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.3\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Burger/state_FBOVI_e3.pdf}
         \caption{FBOVI mean for $u(x,t)$}
         \label{fig:fbovi_e3}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.3\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Burger/state_enkf_e3.pdf}
         \caption{Joint EnKF mean for $u(x,t)$}
         \label{fig:enkf_e3}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.052\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Burger/legend_e3_all.pdf}
     \end{subfigure}
         \caption{\textbf{Nonlinear convection--diffusion transport model}: reference solution $u(x,t)$ at different spatial locations $x$ and times $t$, together with the means of the distributions of $u(x,t)$ obtained by FBOVI and joint EnKF. FBOVI rapidly approaches the reference solution (from $t \approx 0.003$ onward), while the joint EnKF mean are significantly biased during the initial phase.}
         \label{fig:state_e3}
\end{figure}


Figure~\ref{fig:pred_e3} presents the means of the predictions produced by FBOVI and joint EnKF, where the predicted values are computed in the same way as described in Section~\ref{sec:Lorentz}. The prediction generated by FBOVI is relatively accurate, reflecting the high quality of its joint posterior approximation. On the contrary, the Gaussian assumption for the joint distribution used by the joint EnKF limits its predictive capability. This deficiency is most obvious in the early stage, when only a small number of observations have been incorporated and the true joint posterior tend to be highly non-Gaussian.  

\begin{figure}
     \centering
     \begin{subfigure}[b]{0.3\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Burger/state_reference_pred_e3.pdf}
         \caption{Reference solution $u(x,t)$}
         \label{fig:refer_pred_e3}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.3\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Burger/state_fbovi_pred_e3.pdf}
         \caption{Mean of FBOVI prediction for $u(x,t)$}
         \label{fig:fbovi_pred_e3}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.3\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Burger/state_enkf_pred_e3.pdf}
         \caption{Mean of joint EnKF prediction for $u(x,t)$}
         \label{fig:enkf_pred_e3}
     \end{subfigure}
     \hfill
     \begin{subfigure}[b]{0.047\linewidth}
         \centering
         \includegraphics[width=\textwidth]{numerical_examples/Burger/legend_e3_pred_all.pdf}
     \end{subfigure}
         \caption{\textbf{Nonlinear convection--diffusion transport model (prediction)}: reference solution $u(x,t)$ at different spatial locations $x$ and times $t$, along with prediction means from FBOVI and joint EnKF. FBOVI provides accurate predictions, while joint EnKF predictions deviate substantially from the reference, particularly when $t<0.02$.}
         \label{fig:pred_e3}
\end{figure}


\section{Conclusions}
\label{sec:conclusion}
In this paper, we have presented an online variational inference (VI) framework for joint parameter-state estimation of dynamical systems, particularly under conditions of noisy and incomplete data. The core idea of our approach is to approximate the joint posterior distribution of model parameters and system states through a structured factorization: a flexible variational distribution over parameters and a conditional distribution over states given those parameters. Unlike approaches that rely on restrictive assumptions about the joint posterior structure, our method preserves the structure between states and parameters, leading to improved estimation and predictive accuracy. The modular nature of the framework allows for integration with different filtering techniques, broadening its applicability across domains. A theoretical error bound for the approximation has been established, which not only quantifies inference accuracy but also informs algorithmic design.

Through a series of numerical experiments, we demonstrated that the proposed method maintains strong performance in both low- and high-dimensional settings and remains robust under noisy, partial observations and model discrepancies. Together, these results highlight the reliability, scalability, and adaptability of the approach, underscoring its potential as a powerful tool in challenging application scenarios, including DT systems.

\section*{Acknowledgements}
This work was supported by an NSF CAREER Award, grant number CMMI-2238913.


\bibliographystyle{elsarticle-num} 
\bibliography{ref}

\newpage

\appendix
\section{Metrics of difference between probability measures}
\label{app:metrics_dist}
In this section, we introduce two widely used metrics for quantifying differences between probability distributions: the total variation (TV) distance and the Hellinger distance. 

Let $\mu_1$ and $\mu_2$ be two probability measures with corresponding probability density functions $f_1(x)$ and $f_2(x)$. The total variation distance between $\mu_1$ and $\mu_2$ is defined as:
\begin{equation}
d_{TV}(\mu_1,\mu_2) := \frac{1}{2}\int \big \lvert f_1(x) - f_2(x) \big \rvert dx.
\end{equation}
The Hellinger distance between $\mu_1$ and $\mu_2$ is defined by:
\begin{equation}
d_{H}(\mu_1,\mu_2) := \sqrt{\frac{1}{2}\int \left( \sqrt{f_1(x)} - \sqrt{f_2(x) }\right)^2 dx}.
\end{equation}

\section{Proof of Theorem~\ref{thm:error_bound}}
\label{app:proof_thm}
The following theorem is used for proving Theorem~\ref{thm:error_bound}.
\begin{theorem}[Theorem 16 in~\cite{wang_gorodetsky_2025_lipschitz_stability}]
\label{thm:learning_bound}
For any step $k \geq 2$, given a sequence of data $\Y_k=(y_1,y_2,\cdots,y_k)$, suppose that for any step $i \in [2,k]$, we have
\begin{equation}
\bar{C}(y_i;i) := \sup_{X_{i-1},\theta}\int p(y_i \mid X_i,\theta)p(X_i \mid X_{i-1},\theta)dX_i < \infty.
\label{ieq:assumption}
\end{equation}
The total variation distance between $p(X_k,\theta \mid \Y_k)$ and $q(X_k,\theta)$, namely $d_{TV}\left(p(X_k,\theta \mid \Y_k), q_k(X_k,\theta)\right)$, satisfies:
\begin{align}
d_{TV}\left(p(X_k,\theta \mid \Y_k), q_k(X_k,\theta)\right) &\leq \sum_{j=1}^{k-1}\frac{\prod_{i=j+1}^k\bar{C}(y_i;i)}{p(\Y_{j+1:k}\mid \Y_{1:j})}d_{TV}\left(q_j, q_j^*\right) + d_{TV}\left(q_k, q_k^*\right).
\end{align}
The Hellinger distance between $p(X_k,\theta \mid \Y_k)$ and $q(X_k,\theta)$, namely $d_{H}(p(X_k,\theta \mid \Y_k), q(X_k,\theta))$, satisfies:
\begin{align}
d_{H}(p(X_k,\theta \mid \Y_k), q_k(X_k,\theta)) &\leq \sum_{j=1}^{k-1}\frac{2^{(k-j)}\prod_{i=j+1}^k\sqrt{\bar{C}(y_i;i)}}{\sqrt{p(\Y_{j+1:k}\mid \Y_{1:j})}}d_{H}\left(q_j, q_j^*\right) + d_{H}\left(q_k, q_k^*\right).
\end{align}

\end{theorem}
\begin{proof}
We begin by confirming that the assumption in Theorem~\ref{thm:learning_bound} holds.

For any $i \in [2, k]$, if $\inf_{\theta} |\Gamma(\theta)| = \tilde{C} > 0$, then
\begin{align*}
\pdense(y_i \mid X_i,\theta) &= \pdense_{\mathcal{N}}(y_i ; h(X_i;\theta), \Gamma(\theta)) = (2\pi)^{-\frac{r}{2}}\big \lvert \Gamma(\theta)\big \rvert^{-\frac{1}{2}} \exp \left(-\frac{1}{2}\left(y_i - h(X_i;\theta)\right)^T\Gamma(\theta)^{-1}\left(y_i - h(X_i;\theta)\right)\right) \nonumber \\
&\leq (2\pi)^{-\frac{r}{2}} \big \lvert \Gamma(\theta) \big \rvert^{-\frac{1}{2}} \leq (2\pi)^{-\frac{r}{2}}\tilde{C}^{-\frac{1}{2}}.
\end{align*}
Thus,
\begin{align}
\bar{C}(y_i;i) &=\sup_{X_{i-1},\theta}\int p(y_i \mid X_i,\theta)p(X_i \mid X_{i-1},\theta)dX_i \nonumber \\
&\leq (2\pi)^{-\frac{r}{2}}\tilde{C}^{-\frac{1}{2}}\sup_{X_{i-1},\theta}\int p(X_i \mid X_{i-1},\theta)dX_i  \nonumber \\
&= (2\pi)^{-\frac{r}{2}}\tilde{C}^{-\frac{1}{2}} < \infty,
\label{ieq:vi:type1:C}
\end{align}
and hence the assumption is satisfied.

We now derive an upper bound on the total variation distance between $q_i(X_i,\theta)$ and $q^*_i(X_i,\theta)$. The KL divergence of the distribution $q_i$ with respect to the distribution $q^*_i$ can be written as
\begin{align}
\KL(q_i \lVert q^*_i) &= \E_{(X_i,\theta) \sim q_i(\cdot)}\left[\log \left(\frac{q_i(X_i,\theta)}{q^*_i(X_i,\theta)}\right) \right] \nonumber \\
&= \E_{(X_i,\theta) \sim q_i(\cdot)}\left[\log \left(\frac{\rho_i(X_i \mid \theta)\nu_i(\theta)}{\rho^*_i(X_i \mid \theta)\nu^*_i(\theta)}\right)\right] \nonumber \\
&= \E_{(X_i,\theta) \sim q_i(\cdot)}\left[\log \left(\frac{\rho_i(X_i \mid \theta)}{\rho^*_i(X_i \mid \theta)}\right)\right] + \E_{(X_i,\theta) \sim q_i(\cdot)}\left[\log \left(\frac{\nu_i(\theta)}{\nu^*_i(\theta)}\right)\right].
\label{eq:kl}
\end{align}
The first term on the right hand side in Equation~\eqref{eq:kl} can be expressed as
\begin{equation}
\E_{(X_i,\theta) \sim q_i(\cdot)}\left[\log \left(\frac{\rho_i(X_i \mid \theta)}{\rho^*_i(X_i \mid \theta)}\right)\right] = \E_{\theta \sim \nu_i(\cdot)}\left[\E_{X_i \sim \rho_i(\cdot \mid \theta)}\left[\log \left(\frac{\rho_i(X_i \mid \theta)}{\rho^*_i(X_i \mid \theta)}\right)\right]\right] = \E_{\theta \sim \nu_i(\cdot)}\left[\KL\left(\rho_i(\cdot \mid \theta) \lVert \rho_i^*(\cdot \mid \theta) \right) \right].
\label{eq:term1_kl}
\end{equation}
Since the term $\log \left(\frac{\nu_i(\theta)}{\nu^*_i(\theta)}\right)$ does not depend on $X_i$, the second term on the right hand side in Equation~\eqref{eq:kl} becomes
\begin{equation}
\E_{(X_i,\theta) \sim q_i(\cdot)}\left[\log \left(\frac{\nu_i(\theta)}{\nu^*_i(\theta)}\right)\right] = \E_{\theta \sim \nu_i(\cdot)}\left[\log \left(\frac{\nu_i(\theta)}{\nu^*_i(\theta)}\right)\right] = \KL\left(\nu_i \lVert \nu_i^*\right).
\label{eq:term2_kl}
\end{equation}
Plugging Equations~\eqref{eq:term1_kl} and ~\eqref{eq:term2_kl} into Equation~\eqref{eq:kl} yields
\begin{align}
\KL\left(q_i \lVert q^*_i \right) = \E_{\theta \sim \nu_i(\cdot)}\left[\KL\left(\rho_i(\cdot \mid \theta) \lVert \rho_i^*(\cdot \mid \theta) \right) \right] +\KL\left(\nu_i \lVert \nu_i^*\right).
\label{eq:kl_rewritten}
\end{align}
Consequently, we get
\begin{align}
d_{TV}(q_i,q_i^*) & \leq \frac{1} {\sqrt{2}}\sqrt{\KL\left(q_i \lVert q^*_i\right)} \nonumber \\
&= \frac{1}{\sqrt{2}}\sqrt{\E_{\theta \sim \nu_i(\cdot)}\left[\KL\left(\rho_i(\cdot\mid \theta) \lVert \rho_i^*(\cdot \mid \theta) \right) \right] +\KL\left(\nu_i \lVert \nu_i^*\right)}.
\label{ieq:vi:type1:tv_approx}
\end{align}
An analogous inequality holds for the Hellinger distance:
\begin{align}
d_{H}(q_i,q_i^*) & \leq \frac{1}{\sqrt{2}}\sqrt{\KL\left(q_i \lVert q^*_i\right)} \nonumber \\
&= \frac{1}{\sqrt{2}}\sqrt{\E_{\theta \sim \nu_i(\cdot)}\left[\KL\left(\rho_i(\cdot \mid \theta) \lVert \rho_i^*(\cdot \mid \theta) \right) \right] +\KL\left(\nu_i \lVert \nu_i^*\right)}.
\label{ieq:vi:type1:H_approx}
\end{align}
Lastly, combining Theorem~\ref{thm:learning_bound} with
Equations~\eqref{ieq:vi:type1:C}, \eqref{ieq:vi:type1:tv_approx}, and \eqref{ieq:vi:type1:H_approx}
completes the proof of Theorem~\ref{thm:error_bound}.
\end{proof}

\section{Proof of Theorem~\ref{thm:error_bound_linear}}
\label{app:proof_thm_linear}
The following theorem is used for the proof of Theorem~\ref{thm:error_bound_linear}.
\begin{theorem}[Theorem 16 in~\cite{wang_gorodetsky_2025_lipschitz_stability}]
\label{thm:learning_bound_z}
For any step $k \geq 2$, given a sequence of data $\Y_k=(y_1,y_2,\cdots,y_k)$, suppose that for any step $i \in [2,k]$, we have
\begin{equation}
\bar{C}(y_i;i) := \sup_{X_{i-1},\theta}\int p(y_i \mid X_i,\theta)p(X_i \mid X_{i-1},\theta)dX_i < \infty.
\label{ieq:assumption_z}
\end{equation}
The total variation distance between $p(X_k,\theta \mid \Y_k)$ and $q(X_k,\theta)$ satisfies:
\begin{align}
d_{TV}\left(p(X_k,\theta \mid \Y_k), q_k(X_k,\theta)\right) &\leq \sum_{j=1}^{k-1}\frac{\prod_{i=j+1}^k\bar{C}(y_i;i)}{\prod_{i=j+1}^k Z_i(q_{i-1})}d_{TV}\left(q_j, q_j^*\right) + d_{TV}\left(q_k, q_k^*\right).
\end{align}
The Hellinger distance between $p(X_k,\theta \mid \Y_k)$ and $q(X_k,\theta)$, $d_{H}(p(X_k,\theta \mid \Y_k), q(X_k,\theta))$ satisfies:
\begin{align}
d_{H}(p(X_k,\theta \mid \Y_k), q_k(X_k,\theta)) &\leq \sum_{j=1}^{k-1}\frac{2^{(k-j)}\prod_{i=j+1}^k\sqrt{\bar{C}(y_i;i)}}{\prod_{i=j+1}^k\sqrt{ Z_i(q_{i-1})}}d_{H}\left(q_j, q_j^*\right) + d_{H}\left(q_k, q_k^*\right).
\end{align}
Here $Z_i(q_{i-1})$ is defined as $Z_i(q_{i-1}):= \int p(y_i \mid X_i,\theta)p(X_i \mid X_{i-1},\theta)q_{i-1}(X_{i-1},\theta)dX_{i-1}dX_id\theta$.
\end{theorem}
\begin{proof}
Similar to the proof of Theorem~\ref{thm:error_bound}, we can show that the assumption in Theorem~\ref{thm:learning_bound_z} holds under the condition $\inf_{\theta} \lvert \Gamma(\theta) \rvert = \tilde{C} > 0$ and $\bar{C}(y_i; i)$ satisfies 
\begin{equation}
\bar{C}(y_i;i) \leq (2\pi)^{-\frac{r}{2}}\tilde{C}^{-\frac{1}{2}}.
\label{eq:C_thm2}
\end{equation}
Next, we prove that $Z_{i}(q_{i-1})= \E_{\theta \sim \nu_{i-1}(\cdot)}[\pdense_{\mathcal{N}}(y_i;H(\theta)m_i^{*-}(\theta), H(\theta)C_i^{*-}(\theta)H(\theta)^T+\Gamma(\theta))]$. According to the definition of $Z_{i}(q_{i-1})$, it can be written as 
\begin{align}
Z_{i}(q_{i-1}) &= \int p_\mathcal{N}(y_i; H(\theta)X_i, \Gamma(\theta))p_\mathcal{N}(X_i; A(\theta)X_{i-1},\Sigma(\theta))p_\mathcal{N}(X_{i-1};m_{i-1}(\theta), C_{i-1}(\theta))\nu_{i-1}(\theta)dX_{i-1}dX_id\theta \nonumber \\
&= \E_{\theta \sim \nu_{i-1}(\cdot)}\left[\int p_\mathcal{N}(y_i; H(\theta)X_i, \Gamma(\theta))p_\mathcal{N}(X_i; A(\theta)X_{i-1},\Sigma(\theta))p_\mathcal{N}(X_{i-1};m_{i-1}(\theta), C_{i-1}(\theta))dX_{i-1}dX_i\right] \nonumber \\
&= \E_{\theta \sim \nu_{i-1}(\cdot)}[\pdense_{\mathcal{N}}(y_i;H(\theta)m_i^{*-}(\theta), H(\theta)C_i^{*-}(\theta)H(\theta)^T+\Gamma(\theta))].
\label{eq:z}
\end{align}
By Equations~\eqref{ieq:vi:type1:tv_approx} and ~\eqref{ieq:vi:type1:H_approx}, the terms $d_{TV}(q_j,q_j^*)$ and $d_{H}(q_j,q_j^*)$ satisfy
\begin{align}
d_{TV}(q_j,q_j^*) & \leq \frac{1} {\sqrt{2}}\sqrt{\KL\left(q_j \lVert q^*_j\right)} = \frac{1}{\sqrt{2}}\sqrt{\E_{\theta \sim \nu_j(\cdot)}\left[\KL\left(\rho_j(\cdot\mid \theta) \lVert \rho_j^*(\cdot \mid \theta) \right) \right] +\KL\left(\nu_j \lVert \nu_j^*\right)},
\label{ieq:vi:type1:tv_approx_2}
\end{align}
and
\begin{align}
d_{H}(q_j,q_j^*) & \leq \frac{1}{\sqrt{2}}\sqrt{\KL\left(q_j \lVert q^*_j\right)} = \frac{1}{\sqrt{2}}\sqrt{\E_{\theta \sim \nu_j(\cdot)}\left[\KL\left(\rho_j(\cdot \mid \theta) \lVert \rho_j^*(\cdot \mid \theta) \right) \right] +\KL\left(\nu_j \lVert \nu_j^*\right)}.
\label{ieq:vi:type1:H_approx_2}
\end{align}
The KL divergence $\KL\left(\nu_j \lVert \nu_j^*\right)$ satisfies
\begin{align}
\KL(\nu_j  \| \nu_j^*) &= \log\left( \int \pdense_\mathcal{N}(y_j; h(X_j;\theta), \Gamma(\theta))\, \rho_j^{*-}(X_j \mid \theta)\,\nu_{j-1}(\theta)  dX_jd\theta \right) - \mathcal{L}_j(\nu_j) \nonumber \\
&\leq \log \left((2\pi)^{-\frac{r}{2}}\tilde{C}^{-\frac{1}{2}}\int \rho_j^{*-}(X_j \mid \theta)\nu_{j-1}(\theta) \, dX_jd\theta \right) - \mathcal{L}_j(\nu_j) \nonumber \\
& =\log \left((2\pi)^{-\frac{r}{2}}\tilde{C}^{-\frac{1}{2}}\right) - \mathcal{L}_j(\nu_j) \nonumber \\
& \leq -\frac{r}{2}\log(2\pi) - \frac{1}{2}\log \tilde{C} - \epsilon_j.
\label{ieq:kl_theta}
\end{align}
We next derive the expression of the KL divergence $\KL\left(\rho_j(\cdot \mid \theta) \lVert \rho_j^*(\cdot \mid \theta) \right)$.
The analytical expression of $\rho_j^*(X_j \mid \theta)$ is given by Kalman filter:
\begin{equation}
\rho_j^*(X_j \mid \theta) = \pdense_{\mathcal{N}}(X_j; m_j^*(\theta), C_j^*(\theta)).
\end{equation}
Recall that $\rho_j(X_j \mid \theta) = \pdense_{\mathcal{N}}(X_j; m_j(\theta), C_j(\theta))$. By employing the formula for the KL divergence between two Gaussian distributions, we obtain
\begin{equation}
\KL\left(\rho_j(\cdot \mid \theta) \lVert \rho_j^*(\cdot \mid \theta) \right) = \Psi_j(C_j(\theta), m_j(\theta), C_{j-1}(\theta), m_{j-1}(\theta), \theta). 
\label{eq:kl_state}
\end{equation}
Finally, combining Equations~\eqref{eq:C_thm2},~\eqref{eq:z}, \eqref{ieq:vi:type1:tv_approx_2}, \eqref{ieq:vi:type1:H_approx_2}, \eqref{ieq:kl_theta}, and \eqref{eq:kl_state}, together with Theorem~\ref{thm:learning_bound_z}, completes the proof.
\end{proof}


\section{Non-Gaussian approximation for $\rho_k(X_k \mid \theta)$}
\label{app:non_Gaussian}
In this section, we introduce how the marginal distribution $\nu_k(\theta)$ and the conditional distribution $\rho_k(X_k \mid \theta)$ can be computed in the setting where $\rho_k(X_k \mid \theta)$ is represented by a non-Gaussian distribution and the corresponding filtering method which is integrated into our framework does not belong to the family of Gaussian filtering.

\subsection{Stage 1: Computation of $\nu_k(\theta)$}
\label{sec:step1_eg1}
When a non-Gaussian distribution is used to represent $\rho_k(X_k \mid \theta)$, the following procedure can be applied to compute the integral $I(\theta)$ in Equation~\eqref{eq:elbo_step1}, for evaluating the ELBO $\mathcal{L}_k(\nu_k)$.  

For a given $\theta$, draw $M$ samples from the previous conditional distribution $\rho_{k-1}(X_{k-1} \mid \theta)$ and $M$ samples of the process noise:
\begin{equation*}
X_{k-1}^{(i)} \overset{\mathrm{i.i.d.}}{\sim} \rho_{k-1}(X_{k-1} \mid \theta), \qquad 
\xi^{(i)} \overset{\mathrm{i.i.d.}}{\sim} \mathcal{N}(0,\Sigma(\theta)). 
\end{equation*}
The samples of the predictive distribution $\rho_k^*(X_k \mid \theta)$ can then be constructed by
\begin{equation*}
X_k^{(i)-} = \Phi(X_{k-1}^{(i)};\theta) + \xi^{(i)}.
\end{equation*}
As a result, the integral 
\(I(\theta)\) can be approximated by
\begin{equation}
I(\theta)
\approx \frac{1}{M}\sum_{i=1}^M \pdense_{\mathcal{N}}(y_k;h(X_k^{(i)-};\theta), \Gamma(\theta)).
\label{eq:elbo_eg1}
\end{equation}

\subsection{Stage 2: Computation of $\rho_k(X_k \mid \theta)$}
Following Stage 1, a variational filtering method is employed to approximate the conditional posterior $\rho_k^*(X_k \mid \theta)$ with a distribution $\hat{\rho}_k(X_k \mid \theta)$. The parametric family used to represent $\rho_k(X_k \mid \theta)$ is chosen to be the same with that of $\hat{\rho}_k(X_k \mid \theta)$, and the same filtering method is applied at every time step. Let $\phi_k(\theta)$ and $\phi_k^*(\theta)$ denote the corresponding hyperparameters used to represent the distributions $\rho_k(X_k \mid \theta)$ and $\hat{\rho}_k(X_k \mid \theta)$, respectively. If we directly set $\rho_k(X_k \mid \theta) = \hat{\rho}_k(X_k \mid \theta)$, i.e., $\phi_k(\theta) = \phi_k^*(\theta)$, then the complexity of the functional representation of $\phi_k(\theta)$ will grow with time step $k$. To maintain constant computational and storage costs over time, we therefore re-approximate $\hat{\rho}(X_k \mid \theta)$ with $\rho(X_k \mid \theta)$ by minimizing the expected KL divergence $\E_{\theta \sim \nu_k(\cdot)}[\KL\left(\rho_k(\cdot \mid \theta) \lVert \hat{\rho}_k(\cdot \mid \theta)\right)]$. For a given $\theta$, the KL divergence $\KL\left(\rho_k(\cdot \mid \theta) \lVert \hat{\rho}_k(\cdot \mid \theta)\right)$ can be approximated using the Monte Carlo method with samples generated from $\rho_k(X_k \mid \theta)$:
\begin{equation*}
\KL\left(\rho_k(\cdot \mid \theta) \lVert \hat{\rho}_k(\cdot \mid \theta)\right) \approx \frac{1}{N_x}\sum_{i=1}^{N_x} \left(\log \rho_k(X_k^{(i)} \mid \theta) - \log \hat{\rho}_k(X_k^{(i)} \mid \theta) \right), \qquad X_k^{(i)} \overset{\mathrm{i.i.d.}}{\sim} \rho_k(\cdot \mid \theta).
\end{equation*}  
When computational resources are highly constrained, the training loss can be simplified to  
\begin{equation*}
\frac{1}{N}\sum_{i=1}^N \big\lVert \phi_k(\theta^{(i)}) - \phi_k^*(\theta^{(i)}) \big\rVert_2^2, 
\qquad \theta^{(i)} \overset{\mathrm{i.i.d.}}{\sim} \nu_k(\cdot).
\end{equation*}
Finally, the distribution $\nu_k(\theta)$ together with the function $\phi_k(\theta)$ are passed forward to the next time step $k+1$.

\section{Detailed settings for hyperparameters of the methods FBOVI is compared with in Section~\ref{sec:experiments}}
\label{app:hyper_param}
The hyperparameters of the unscented transform used by the joint UKF in Section~\ref{sec:low_dimension} and Section~\ref{sec:Lorentz} are set to $\alpha = 0.5$, $\beta = 2$, and $\kappa = 0$, following the settings in~\cite{joint_ukf2}. Specifically, $\alpha$ is typically chosen within the range $[10^{-4}, 1]$~\cite{joint_ukf2}. For Gaussian distributions, $\beta = 2$ is generally regarded as an optimal choice~\cite{julier2002scaled}. It is also common practice to set $\kappa = 0$.

The joint PF, joint UKF, and joint EnKF assign random-walk dynamics with zero-mean Gaussian noise to the model parameters. For the joint PF, the noise covariance is set to $10^{-4}I_{d_\theta \times d_\theta}$ to balance estimation accuracy and the mitigation of weight degeneracy. The joint UKF and joint EnKF use a smaller noise covariance of $10^{-8}I_{d_\theta \times d_\theta}$. Here $d_\theta$ represents the dimension of the parameters.


\section{Detailed settings for the experiment of the single pendulum system}
\label{app:ep1}
The ground truth of the posterior $\pdense(X_k,\theta \mid \mathcal{Y}_k)$  is obtained by the delayed rejection adaptive Metropolis (DRAM), recognized as a state-of-the-art Markov Chain Monte Carlo (MCMC) method. Notably, MCMC operates in an offline manner, incorporating the entire history of observations $\mathcal{Y}_k={y_1,y_2,\cdots,y_k}$ at once for each time step $k$. The hyper-parameters of DRAM used are as follows. The adaption threshold for the adaptive Metropolis algorithm is set to $100$. The 2-tier delayed rejection method is used and the scaling factor for the proposal covariance of the second tier is $\gamma=0.5$. DRAM generates $3 \times 10^{6}$ samples from the joint distribution $\pdense(X_k, \theta \mid \mathcal{Y}_k)$ at each time step $k$. After discarding the first $50\%$ of samples as burn-in, half of the remaining samples, i.e., $7.5 \times 10^{5}$, are used to represent the posterior.

\end{document}