Online Subspace Learning on Flag Manifolds for System Identification
Extracted source text · arXiv 2511.06416
% \documentclass{l4dc2026}
% If your paper is accepted, please use the "final" option
% for the camera-ready version
% The following packages will be automatically loaded:
% amsmath, amssymb, natbib, graphicx, url, algorithm2e
\documentclass{l4dc2026}
% \usepackage{amsthm}
\usepackage{acro}
\usepackage{mathtools}
\usepackage{float}
\usepackage{amsfonts}
% \usepackage[ruled,vlined]{algorithm2e}
\usepackage{physics}
% \usepackage{musicography}
\usepackage{lilyglyphs}
\usepackage{tcolorbox}
\usepackage[dvipsnames]{xcolor}
\usepackage{epigraph}
\usepackage{setspace}
\usepackage{matlab-prettifier}
\usepackage{textcomp}
\usepackage{mathrsfs}
% \usepackage{tabularx}
\usepackage{booktabs}
\usepackage{stmaryrd}
\usepackage{bm}
\usepackage{cite}
\usepackage{enumitem}
\usepackage{pgfplots}
\usepackage{pgfplotstable} % for reading CSV files
\usepackage{booktabs}
\usepackage{placeins} % for \FloatBarrier
\usepackage{float} % for [H]
\usepackage{wrapfig}
\usepackage{subcaption}
\usepackage[skip=10pt]{caption}
\makeatletter
\let\algorithm\relax
\let\endalgorithm\relax
\let\listofalgorithms\relax
% also undefine keywords that conflict with algpseudocode
\let\For\relax
\let\EndFor\relax
\let\State\relax
\let\Require\relax
\makeatother
\usepackage{algorithm}
\usepackage[noend]{algpseudocode}
\algrenewcommand\algorithmicindent{1.0em}
\usepackage[framemethod=default]{mdframed}
\captionsetup{
font=footnotesize, labelfont=footnotesize,
aboveskip=2pt, belowskip=2pt
}
% optional spacing tweaks
\setlength{\intextsep}{6pt} % vertical space above/below wrapped floats
\setlength{\columnsep}{6pt} % gap between wrapped float and text
\usepgfplotslibrary{groupplots} % enables groupplot
\usepgfplotslibrary{dateplot}
\pgfplotsset{compat=1.18}
\usepgfplotslibrary{fillbetween}
\usetikzlibrary{calc} % needed for ($(A)!t!(B)$)
% \usepackage[colorlinks=true, linkcolor=black, citecolor=black, urlcolor=black]{hyperref}
% \usepackage[dvipsnames]{xcolor}
\input{user_commands}
%============ colors ========================
\definecolor{bluecomment}{RGB}{18, 48, 174}
\definecolor{redcomment}{RGB}{217, 22, 86}
\definecolor{eqnColor}{RGB}{199, 62, 58}
\definecolor{boxColor}{RGB}{210, 224, 243}
\definecolor{myteal}{RGB}{42,160,135} % deeper teal
\definecolor{myorange}{RGB}{236,112,72} % deeper muted orange
\definecolor{mypurple}{RGB}{115,140,200}% deeper lavender/blue
\definecolor{mygreen}{RGB}{120,190,60} % deeper soft green
\definecolor{myblue}{RGB}{90,140,210}
\definecolor{myred}{RGB}{232, 99, 99}
\definecolor{mycyan}{RGB}{103, 196, 181}
\definecolor{myviolet}{RGB}{170, 149, 214}
\definecolor{PeachRed}{RGB}{231, 143, 129}
\definecolor{coolgray}{RGB}{180,180,180}
\definecolor{mutedorange}{RGB}{230,160,90}
\definecolor{dustyrose}{RGB}{219, 64, 40}
\definecolor{deepblue}{RGB}{90,140,210}
\definecolor{slategreen}{RGB}{120,190,60}
\definecolor{seateal}{RGB}{95,175,190}
\definecolor{sandyellow}{RGB}{240, 88, 122}
\definecolor{plumviolet}{RGB}{165,125,190}
%============ acryonyms =================
% \usepackage[acronym]{glossaries} % REMOVE
% \makeglossaries % REMOVE
% Define acronyms once
\DeclareAcronym{LTI}{short=LTI,long=Linear Time-Invariant}
\DeclareAcronym{LTV}{short=LTV,long=Linear Time-Varying}
\DeclareAcronym{DeePC}{short=DeePC,long=Data-enabled Predictive Control}
\DeclareAcronym{SVD}{short=SVD,long=singular value decomposition}
\DeclareAcronym{PCA}{short=PCA,long=principal component analysis}
\DeclareAcronym{FRONT}{short=FRONT,long=Flag Recursive Online Tracking}
\DeclareAcronym{GREAT}{short=GREAT,long=Grassmannian Recursive Algorithm for Tracking}
\DeclareAcronym{RGD}{short=RGD,long=Riemannian gradient descent}
\DeclareAcronym{SISO}{short=SISO,long=single-input single-output}
\DeclareAcronym{ARX}{short=ARX,long=AutoRegressive model with eXogenous inputs}
\DeclareAcronym{FROSL}{short=FROSL,long=Flag Recursive Online Subspace Learning}
\DeclareAcronym{PAST}{short=PAST,long=projection approximation subspace tracking}
\DeclareAcronym{NSR}{short=NSR,long=noise-to-signal ratio}
%============= comments and notes ============
\def\bblue{\color{bluecomment}}
\def\eblue{\protect\normalcolor}
\def\bred{\color{redcomment}}
\def\ered{\protect\normalcolor}
%% Notes
\newcommand{\jcmargin}[1]{\marginpar{\color{redcomment}\tiny\ttfamily{Jeremy:} #1}}
\newcommand{\djmargin}[1]{\marginpar{\color{redcomment}\tiny\bf\ttfamily{Dian:} #1}}
\title{Online Subspace Learning on Flag Manifolds for System Identification}
\usepackage{times}
% Use \Name{Author Name} to specify the name.
% If the surname contains spaces, enclose the surname
% in braces, e.g. \Name{John {Smith Jones}} similarly
% if the name has a "von" part, e.g \Name{Jane {de Winter}}.
% If the first letter in the forenames is a diacritic
% enclose the diacritic in braces, e.g. \Name{{\'E}louise Smith}
% Two authors with the same address
% \coltauthor{\Name{Author Name1} \Email{abc@sample.com}\and
% \Name{Author Name2} \Email{xyz@sample.com}\\
% \addr Address}
% Three or more authors with the same address:
% \coltauthor{\Name{Author Name1} \Email{an1@sample.com}\\
% \Name{Author Name2} \Email{an2@sample.com}\\
% \Name{Author Name3} \Email{an3@sample.com}\\
% \addr Address}
% Authors with different addresses:
\author{%
\Name{Dian Jin} \Email{djin38@wisc.edu}\\
\addr University of Wisconsin-Madison
\AND
\Name{Jeremy Coulson} \Email{jeremy.coulson@wisc.edu}\\
\addr University of Wisconsin-Madison%
}
\begin{document}
\maketitle
\begin{abstract}%
Data-driven control methods based on subspace representations are powerful but are often limited to linear time-invariant systems where the model order is known. A key challenge is developing online data-driven control algorithms for time-varying systems, especially when the system's complexity is unknown or changes over time. To address this, we propose a novel online subspace learning framework that operates on flag manifolds. Our algorithm leverages streaming data to recursively track an ensemble of nested subspaces, allowing it to adapt to varying system dimensions without prior knowledge of the true model order. We show that our algorithm is a generalization of the Grassmannian Recursive Algorithm for Tracking. The learned subspace models are then integrated into a data-driven simulation framework to perform prediction for unknown dynamical systems. The effectiveness of this approach is demonstrated through a case study where the proposed adaptive predictor successfully handles abrupt changes in system dynamics and outperforms several baselines.%
\end{abstract}
\begin{keywords}%
System identification; Subspace learning; Optimization on manifolds
\end{keywords}
\input{v1/paper_v1}
\bibliography{l4dc2026-sample}
\end{document}
\newcommand{\Flag}{\mathrm{Flag}}
\newcommand{\true}{\mathrm{true}}
\newcommand{\im}{\mathrm{im~}}
\newcommand{\opt}{\mathrm{opt}}
\newcommand{\ini}{\mathrm{ini}}
\newcommand{\rmRef}{\mathrm{ref}}
\newcommand{\Flagqp}{\mathrm{Flag}(q_{1:d}, p)}
\newcommand{\multi}{\mathrm{multi}}
\newcommand{\snatural}{\text{\natural}}
\newcommand{\qf}{\mathrm{qf}}
\renewcommand{\grad}{\mathrm{grad}}
\newcommand{\rB}[2]{\mathcal{B}|_{[#1,#2]}}
\newcommand{\rmd}{\mathrm{d}}
\newcommand{\rmp}{\mathrm{p}}
\newcommand{\rmf}{\mathrm{f}}
\newcommand{\rmE}{\mathrm{E}}
\newcommand{\SE}{\mathrm{SE}}
\newcommand{\DeePC}{\mathrm{DeePC}}
\newcommand{\FRONT}{\mathrm{FRONT}}
\newcommand{\Tf}{T_\mathrm{f}}
\newcommand{\Tini}{T_\mathrm{ini}}
\newcommand{\Tsim}{T_\mathrm{sim}}
\newcommand{\Tswitch}{T_\text{switch}}
\newcommand{\LTI}{\mathsf{LTI}}
\newcommand{\pred}{\text{pred}}
\newcommand{\NSR}{\text{NSR}}
% \theoremstyle{remark}
% \newtheorem{theorem}{Theorem}
% \newtheorem*{remark}{Remark}
% \newtheorem{proposition}{Proposition}[section]
% \newtheorem{definition}{Definition}[section]
% \newtheorem{example}{Example}[section]
\newtheorem{problem}{Problem}[section]
% \newcounter{example}[section]
% \newenvironment{example}[1][]{
% \refstepcounter{example}\par\medskip
% \noindent {\color{brown}\textsf{Example~\theexample} \textsc{#1}}
% }{}
% \newcounter{problem}[section]
% \newenvironment{problem}[1][]{
% \refstepcounter{problem}\par\medskip
% \noindent {\color{brown}\textsf{Problem~\theexample} \textsc{#1}}
% }{}
% ==== marking the end of an environment ====
\newcommand{\xqed}[1]{%
\leavevmode\unskip\penalty9999 \hbox{}\nobreak\hfill
\quad\hbox{#1}}
\newcommand{\egqed}{\xqed{$\crotchetRest$}}
% ============= summation, product =============
\newcommand{\Sum}[2]{\sum_{#1}^{#2}}\newcommand{\Prod}[2]{\prod_{#1}^{#2}}
% ============ Greek =================
\newcommand{\eps}{\varepsilon}
\newcommand{\lam}{\lambda}
\newcommand{\Lam}{\Lambda}
\newcommand{\s}{\sigma}
\newcommand{\omg}{\omega}
\newcommand{\Omg}{\Omega}
\newcommand{\Ga}{\Gamma}
\newcommand{\ga}{\gamma}
\newcommand{\vphi}{\varphi}
\newcommand{\phe}{\varphi}
\newcommand{\cta}{\theta}
% ============ mathbb ================
\newcommand{\E}{\mathbb{E}}
\newcommand{\F}{\mathbb{F}}
\newcommand{\bbL}{\mathbb{L}}
\newcommand{\bbP}{\mathbb{P}}
\newcommand{\Q}{\mathbb{Q}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\C}{\mathbb{C}}
\newcommand{\N}{\mathbb{N}}
\newcommand{\Z}{\mathbb{Z}}
\newcommand{\Zp}{\mathbb{Z}_{\geq 0}}
%==========================
\newcommand{\U}{\mathbb{U}}
\newcommand{\bbU}{\mathbb{U}}
\newcommand{\W}{\mathbb{W}}
\newcommand{\bbS}{\mathbb{S}}
\newcommand{\T}{\mathbb{T}}
\newcommand{\bbT}{\mathbb{T}}
\newcommand{\D}{\mathbb{D}}
\newcommand{\B}{\mathbb{B}}
\newcommand{\bbH}{\mathbb{H}}
\newcommand{\bbX}{\mathbb{X}}
\newcommand{\X}{\mathbb{X}}
\newcommand{\bbY}{\mathbb{Y}}
\newcommand{\Y}{\mathbb{Y}}
% ============ mathrm ==================
\newcommand{\rmA}{\mathrm{A}}
\newcommand{\rmB}{\mathrm{B}}
\newcommand{\rmO}{\mathrm{O}}
% ============ bold ====================
\newcommand{\bmU}{\mathbf{U}}
\newcommand{\bmH}{\mathbf{H}}
\newcommand{\bmV}{\mathbf{V}}
\newcommand{\bmW}{\mathbf{W}}
\newcommand{\bmA}{\mathbf{A}}
\newcommand{\bmB}{\mathbf{B}}
% ============ calligraphy =============
\newcommand{\Cal}[1]{\mathcal{#1}}
\newcommand{\calA}{\mathcal{A}}
\newcommand{\calB}{\mathcal{B}}
\newcommand{\calC}{\mathcal{C}}
\newcommand{\calD}{\mathcal{D}}
\newcommand{\calE}{\mathcal{E}}
\newcommand{\calF}{\mathcal{F}}
\newcommand{\calG}{\mathcal{G}}
\newcommand{\calH}{\mathcal{H}}
\newcommand{\calI}{\mathcal{I}}
\newcommand{\calJ}{\mathcal{J}}
\newcommand{\calK}{\mathcal{K}}
\newcommand{\calL}{\mathcal{L}}
\newcommand{\calM}{\mathcal{M}}
\newcommand{\calN}{\mathcal{N}}
\newcommand{\calO}{\mathcal{O}}
\newcommand{\calP}{\mathcal{P}}
\newcommand{\calQ}{\mathcal{Q}}
\newcommand{\calR}{\mathcal{R}}
\newcommand{\calS}{\mathcal{S}}
\newcommand{\calT}{\mathcal{T}} % topology
\newcommand{\calU}{\mathcal{U}}
\newcommand{\calV}{\mathcal{V}}
\newcommand{\calX}{\mathcal{X}}
\newcommand{\calY}{\mathcal{Y}}
\newcommand{\frakX}{\mathfrak{X}}
\newcommand{\frakY}{\mathfrak{Y}}
% ============ category theory: sans-serif ================
\newcommand{\sfC}{\mathsf{C}}
\newcommand{\sfD}{\mathsf{D}}
\newcommand{\sfSet}{\mathsf{Set}}
\newcommand{\sfTop}{\mathsf{Top}}
\newcommand{\sfMan}{\mathsf{Man}}
\newcommand{\sfDiff}{\mathsf{Diff}}
\newcommand{\sfVec}{\mathsf{Vec}}
\newcommand{\sfGrp}{\mathsf{Grp}}
\newcommand{\sfAb}{\mathsf{Ab}}
\newcommand{\sfRng}{\mathsf{Rng}}
\newcommand{\sfCRng}{\mathsf{CRng}}
\newcommand{\Ob}{\mathrm{Ob}}
\newcommand{\Hom}{\mathrm{Hom}}
% ============= braces ==============
\newcommand{\curlBrace}[1]{\left\{#1\right\}}
\newcommand{\Brace}[1]{\left(#1\right)}
\renewcommand{\hat}{\widehat}
\newcommand{\br}[1]{\left(#1\right)}
\newcommand{\cBr}[1]{\left\{#1\right\}}
\newcommand{\vBr}[1]{\left[#1\right]}
\newcommand{\floor}[1]{\lfloor #1 \rfloor}
% ============= colors ==============
\newcommand{\brown}[1]{\textcolor{brown}{#1}}
\newcommand{\red}[1]{\textcolor{red}{#1}}
\newcommand{\cyan}[1]{\textcolor{cyan}{#1}}
\newcommand{\NvBlue}[1]{\textcolor{NavyBlue}{#1}}
\newcommand{\OrangeRed}[1]{\textcolor{OrangeRed}{#1}}
\newcommand{\RedOrange}[1]{\textcolor{RedOrange}{#1}}
\newcommand{\BrickRed}[1]{{\color{BrickRed}#1}}
\newcommand{\Blue}[1]{{\color{Blue}#1}}
\newcommand{\Emerald}[1]{{\color{Emerald}#1}}
\newcommand{\Mahogany}[1]{{\color{Mahogany}#1}}
\newcommand{\Bittersweet}[1]{{\color{Bittersweet}#1}}
\definecolor{WiscRed}{RGB}{180, 42, 26}
\newcommand{\WiscRed}[1]{{\color{WiscRed}#1}}
\definecolor{PeachRed}{RGB}{231, 143, 129}
\newcommand{\PeachRed}[1]{{\color{PeachRed}#1}}
% http://tug.ctan.org/info/symbols/comprehensive/symbols-a4.pdf
% ============== math terminology =============
\newcommand{\inv}[1]{#1^{-1}}
\newcommand{\Lim}[2]{\lim_{#1 \to #2}}
\newcommand{\osc}{\mathrm{osc}} % oscillation
\newcommand{\supp}{\mathrm{supp~}} % support of a function
%\newcommand{\op}{\mathrm{op}} % operator norm subscript
\newcommand{\sgn}{\mathrm{sgn~}}
\newcommand{\Span}{\mathrm{span}}
%\newcommand{\osc}{\mathrm{osc}}
\newcommand{\sign}{\mathrm{sign}}
\newcommand{\range}{\mathrm{range}}
\newcommand{\Null}{\mathrm{null~}}
\newcommand{\codim}{\mathrm{codim~}}
\newcommand{\id}{\mathrm{id}}
\newcommand{\Aut}{\mathrm{Aut}}
\newcommand{\Sym}{\mathrm{Sym}}
\newcommand{\Alt}{\mathrm{Alt}}
\newcommand{\GL}{\mathrm{GL}}
% ============== special functions, class of functions =============
% indicator/characteristic function
\newcommand{\Char}[1]{\chi_{#1}}
\newcommand{\Lap}{\Delta}
\newcommand{\Lip}{\mathrm{Lip}} % Lipschitz
\newcommand{\AC}{\mathrm{AC}} % absolutely continuous
% Lebesgue integral on a set
\newcommand{\Lint}[1]{\int_{#1}}
\newcommand{\loc}{\mathrm{loc}} % locally integrable
% ============= algebra ================
% list of vectors
\newcommand{\List}[2]{#1_{1}, \cdots, #1_{#2}}
\DeclareMathOperator*{\Ann}{Ann} % annhilator
\DeclareMathOperator*{\diag}{diag} % diagonal
%\DeclareMathOperator*{\Tr}{Tr} % trace
% ============= optimization ===============
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\newcommand{\dom}{\mathrm{dom}}
\newcommand{\Riemgrad}{\mathrm{grad~}}
% ============= matrix manifolds ================
\newcommand{\St}{\mathrm{St}} % Stiefel
\newcommand{\Gr}{\mathrm{Gr}} % Grassmannian
% ============== manifolds examples ==============
\newcommand{\RP}{\mathbb{RP}} % real projective spaces
\newcommand{\lift}{\mathrm{lift}}
\newcommand{\Exp}{\mathrm{Exp}}
\newcommand{\Log}{\mathrm{Log}}
% ============= music symbol ================
\section{Introduction}
% subspace representation in system theory ->
% Where are subspaces used? subspace identification
% recently data-driven control (Hankel matrix structure)
% For time-varying systems: [...] identify the system by recursively update A_t, B_t, C_t, D_t matrices.
% GREAT by estimating input-output trajectories, which is called restricted behavior.
Subspace representations are becoming increasingly important in the field of systems and control, especially with the recent revival of behavioral systems theory \citep{willems1986time} in data-driven control. A key concept in this area is Willems' Fundamental Lemma, which provides a direct data-driven non-parametric \emph{subspace} representation for finite-length trajectories of linear time-invariant systems. This has spurred the development of effective subspace identification techniques \citep{favoreel1999spc, van2012subspace}, novel data-driven simulation and control methods \citep{markovsky2008data, coulson2021distributionally, de2019formulas, berberich2020data, verhoek2021data, dorfler2022bridging},
which have been shown to perform remarkably well across a wide range of application domains~\citep{ elokda2021data, huang2021quadratic, huang2021decentralized, kerkhof2021predictive, ames2022quad}.
See \citep{markovsky2021behavioral} and references therein.
Despite these advances, many existing identification and control methods are limited to \ac{LTI} systems. A significant open challenge is the design of online adaptive approaches and recursive algorithms that can handle systems that change over time. While some subspace tracking methods have been developed for online state-space identification \citep{verhaegen2007filtering, van1994n4sid, oku2002recursive, zhang2019online, sasfi2025GREAT}, they typically require the system order to be known and constant. In many real-world scenarios, however, the true system order is unknown and may vary, for instance, due to component failures.
In this work, we propose a novel framework for adaptive, data-driven identification of unknown, time-varying dynamical systems based on subspace learning techniques.
% However, these identification and control methods are often limited to linear time-invariant systems.
% Recent surveys~\citep{markovsky2021behavioral,berberich2024overview} emphasize the open challenge of designing online adaptive data-driven approaches and recursive algorithms for online computation.
% Subspace-based methods for state-space system identification (4SID) have long been studied \citep{gustafsson1997recursive},
% and subspace tracking methods have been applied to 4SID by identifying state-space matrices online.
% Instead of estimating state-space matrices, \citep{sasfi2025GREAT} directly \jcmargin{put picture in intro or near algorithm if space allows}estimates the subspace of finite length input-output trajectories.
% However, these methods are all based on the assumption that the system order is known and fixed. In many practical settings, the true system order is unknown and may even vary over time -- for example, actuator/sensor failures. This motivates the need for an adaptive framework that not only tracks evolving subspaces but also selects the appropriate dimension online.
%==========================================
%==========================================
Subspace learning (sometimes termed tracking), has long been studied in signal processing \citep{yang1995projectionPAST, delmas2010subspace, balzano2010GROUSE, balzano2015local, he2012incremental}.
Broadly speaking, subspace learning algorithms can be classified into algebraic and geometric methods \citep{balzano2018streamingPCA}. The geometric methods optimize a certain loss function via gradient descent on a matrix manifold such as the Grassmannian, a set consisting of all subspaces of a particular dimension. Such methods have a solid theoretical background rooted in optimization on manifolds \citep{edelman1998geometry, absil2004riemannian, boumal2023introduction}.
Running these algorithms on Grassmannians requires choosing the dimensionality \emph{a priori}. This poses a significant challenge in practical applications where the true underlying dimension is unknown or may change over time.
%==========================================
%==========================================
\paragraph{Contribution.}
We propose a novel framework for online system identification and data-driven simulation for unknown time-varying dynamical systems. The proposed framework consists of the \ac{FRONT} algorithm on flag manifolds that leverages streaming data and yields a nested hierarchy of subspaces.
To the best of our knowledge, this is the first online algorithm that optimizes over flag manifolds with streaming data.
We formally show that \ac{GREAT}~\citep{sasfi2025GREAT} is a special case of our algorithm when the true subspace dimension is known. When the dimension is unknown, we propose a dimensionality ensemble strategy inspired by ensemble learning \citep{dietterich2000ensemble}.
Finally, the \ac{FRONT} algorithm is leveraged as a subspace model in data-driven simulation \citep{markovsky2008data} to perform trajectory prediction for a time and complexity-varying unknown system.
The framework is showcased on an \ac{ARX} system simulation and is shown to outperform non-adaptive subspace prediction \citep{favoreel1999spc} and other adaptive benchmarks.
The rest of the paper is organized as follows. Section~\ref{sec:preliminaries} contains preliminaries on subspace geometry.
We formulate the problem in Section \ref{sec:problem-setup}.
Section~\ref{sec:flag_tracking_algorithm} introduces the online subspace learning algorithm.
Section \ref{sec:numrical-study} contains numerical studies.
Section~\ref{sec:conclusion} concludes the paper.
%======================================================
%======================================================
\section{Preliminaries}\label{sec:preliminaries}
We use $\Zp$ to denote non-negative integers.
Given $i,j\in\Zp$, with $i\leq j$, we use $[i, j]$ to denote the discrete interval $[i, j]\cap \Zp$.
We use $I_{p \times m}$ to denote the $p \times m$ matrix whose entries are $1$ on the main diagonal (i.e., $(i, i)$-entry for $i=1,\dots,\min(p, m)$), and all other entries are $0$.
The diagonal matrix with diagonal elements $\ga_1,\cdots,\ga_d$ is denoted $\diag(\ga_1,\cdots,\ga_d)$.
Given a function $v\colon \Zp\to\R^q$ and $i,j\in\Zp$, with $i\leq j$, we define $v_{[i,j]} = (v(i), v(i+1), \dots, v(j)) \in \R^{q(j-i+1)}$. We use boldface $\bmU$ to denote a subspace of $\R^p$ and plain $U$ to denote a matrix such that $\im U = \bmU$. The orthogonal projector onto the subspace $\bmU$ is denoted by $\Pi_{\bmU}$, and is used interchangeably with $\Pi_{U} = UU^\top$, where $U$ is an orthonormal matrix whose columns span $\bmU$.
The Moore–Penrose inverse of a matrix $A$ is denoted $A^\dag$.
The Gaussian distribution with mean $0$ and covariance $\Sigma \in \R^{p \times p}$ is denoted $\calN(0, \Sigma)$.
%==========================
\subsection{Subspace geometry}
A Grassmannian is the set of all subspaces in $\R^p$ with dimension $d$ denoted by $\Gr(p, d)\coloneqq \{\bmU $ a linear subspace of $\R^p: \dim \bmU = d \}.$
Flag manifolds generalize the Grassmannians by considering sequences of nested subspaces of increasing dimension.
Let $p \geq 2$ be an integer and $q_{1:d} \coloneqq (q_1, \dots, q_d)$ with $0 < q_1 < \cdots < q_d < p.$
A flag of signature $(p, q_{1:d})$ is a nested sequence of linear subspaces $\{\bmU_i\}_{i=1}^{d}$ of $\R^p$ satisfying
$\{0\} \subset \bmU_1 \subset \cdots \subset \bmU_d \subset \R^p$
with $\dim \bmU_j = q_j, j=1,\dots,d$. We denote this flag by $\bmU_{1:d}$.
The set of all such flags forms a smooth manifold $\Flag(p, q_{1:d})$ \citep{ye2022optimization}.
We also denote $q_0 = 0, q = q_d, q_{d+1} = p$.
We can represent flags as points on the Stiefel manifold $\St(p, q) = \{U \in \R^{p \times q}: U^\top U = I\}$ consisting of orthonormal matrices.
Define for each $k =1,\dots,d$, $U_k \in \St(p, q_{k}-q_{k-1})$ such that $[U_1 \cdots U_k]$ is an orthonormal basis of $\bmU_k$. We must note that under these notations, $\bmU_k$ is not the subspace spanned by $U_k$.
Then $U_{1:d} \coloneqq [U_1 \cdots U_d]$ is a representative of $\bmU_{1:d}$, called Stiefel representative. Throughout this paper we will abuse the notation to write $U_{1:d} \in \Flag(p, q_{1:d})$.
If the signature is $(p, q_1)$, then $\Flag(p, (q_1)) = \Gr(p, q_1)$.
The chordal distance of two subspaces $\bmA \in \Flag(p, (m)), \bmB \in \Flag(p, (n))$ is defined by $\rmd(\bmA, \bmB) = (\sum_{i=1}^{\min(m,n)} \sin^2 \theta_i)^{1/2}$ \citep[Theorem 12]{ye2016schubert}, where $\theta_i$ is the $i^{\text{th}}$ principal angle \citep{golub2013matrix}
between $\bmA$ and $\bmB$.
We present a simple example to illustrate the difference between flags and subspaces.
\begin{example}
Consider the canonical basis $e_1=(1, 0, 0, 0), e_2 = (0, 1, 0, 0), e_3 = (0, 0, 1, 0), e_4 = (0, 0, 0, 1)$ of $\R^4$. Let $\bmU_1 = \Span \{e_1\}, \bmU_2 = \Span \{e_1, e_2\}, \bmU_3 = \Span \{e_1, e_2, e_3\}$, then $\bmU_{1:3}\in \Flag(4, (1, 2, 3))$, and a Stiefel representation of this flag is the matrix $U = \begin{bmatrix} e_1 & e_2 & e_3 \end{bmatrix}$.
Note that even if two bases span the same subspace, the hierarchical structures of their flags may be different. Let $V = \begin{bmatrix} e_1 & e_3 & e_2 \end{bmatrix}$, then clearly $\im V = \im U$. However, $\bmV_1 = \Span\{e_1\}, \bmV_2 = \Span\{e_1, e_3\}, \bmV_3 = \Span\{e_1, e_2, e_3\}$, which implies $\bmV_2 \neq \bmU_2$, hence $\bmV_{1:d}\neq \bmU_{1:d}$.
% This is an important subtlety that will be leveraged to perform dimension selection.
\end{example}
%==========================================
%==========================================
\section{Problem Setup \& Motivation}\label{sec:problem-setup}
The goal of this paper is to estimate an unknown, time-varying subspace $\bmU^t$ from streaming data. More precisely, for each $t\in\Zp$, let
$\bmU^t \in \bigcup_{j=1}^{d} \Gr(p, q_j),$
where $0<q_1<\cdots<q_d<p$. We assume that we have access to an upper bound $q_d$ on the dimension of all subspaces $\bmU^t$. Note that the upper bound $q_d$ may not be tight in the sense that there may be no time instance $t\in\Zp$ where $\dim \bmU^t = q_d$. We assume that at each $t\in\Zp$, we measure the data $w_t=v_t+\eta_t,$
where $v_t\in \bmU^t$, and $\eta_t$ is noise.
We wish to obtain estimates $\hat{\bmU}^t$ such that
% \begin{equation}\label{eq:distance-bound}
% d(\hat{\bmU}^t,\bmU^t)\leq\epsilon_{\textup{tol}},
% \end{equation}
$\rmd(\hat{\bmU}^t,\bmU^t)\leq\epsilon_{\textup{tol}}$
for some user-defined tolerance $\epsilon_{\textup{tol}}>0$, i.e., we seek estimates that are ``close'' to the true subspaces.
We refer to this problem as \emph{learning} $\bmU^t$ or \emph{tracking} $\bmU^t$.
In the case when $d=1$ and $q_d$ is known, there are efficient algorithms~\citep{yang1995projectionPAST, balzano2010GROUSE, sasfi2025GREAT} for robustly learning $\bmU^t$. However, in many practical settings the true dimension of the underlying subspaces are \emph{unknown} and may even vary over time (e.g., online dynamic mode decomposition for time-varying dynamical systems~\citep{zhang2019online}, direction of arrival analysis \citep{rabideau1996fast}).
Solving this problem is critical for online system identification and control: a fixed, mis-specified subspace dimension yields biased estimates, slow adaptation, and brittle decisions \citep{ljung1998system} (see Figure \ref{fig:y_prediction_two}). Learning a nested hierarchy of subspaces (flag) online gives rank-on-demand models that track changing complexity, enabling efficient prediction, reliable change-point detection, without prior knowledge of the true order \citep{szwagier2025nestedsubspacelearningflags}.
% ---------------------------------------------------
% ---------------------------------------------------
\subsection{Motivating Example: Online System Identification}\label{sec:motivation}
Consider the \ac{LTV} dynamical system
\begin{equation}\label{eq:LTV}
\begin{aligned}
x(t+1) &= A_t x(t) + B_t u(t), \\
y(t) &= C_t x(t) + D_t u(t),
\end{aligned}
\end{equation}
where $A_t\in\R^{n\times n}$, $B_t\in\R^{n\times m}$, $C_t\in\R^{r\times n}$, and $D_t\in\R^{r\times m}$ are time varying matrices. Assume that the model parameters $(A_t,B_t,C_t,D_t)$ are unknown, and we only have access to input-output data of~\eqref{eq:LTV}.
A typical problem in system identification is to use input-output data to find a model that captures the input-output behavior of~\eqref{eq:LTV}. The main challenge in this setting is that the model parameters are \emph{time-varying}. Furthermore, we do not have access to state measurements. Hence, the underlying state-space dimension (system complexity) is unknown.
Let $L \geq 1,$ for each $t \geq L-1$, the $L$-length input-output trajectories of~\eqref{eq:LTV} on the interval $[t-L+1,t]$ are given by
\begin{equation*}
v_t := \begin{bmatrix}
u_{[t-L+1,t]} \\ y_{[t-L+1,t]}
\end{bmatrix} = \underbrace{\begin{bmatrix}
0 & I_{mL} \\
\mathcal{O}_{[t-L+1,t]} & \mathcal{T}_{[t-L+1,t]}
\end{bmatrix}}_{=: U^t}
\begin{bmatrix}
x(t-L+1) \\ u_{[t-L+1,t]}
\end{bmatrix}, \text{ where}
\end{equation*}
\[
\resizebox{\columnwidth}{!}{$\calO_{[t-L+1,t]} = \begin{bmatrix}
C_{t-L+1} \\
C_{t-L+2} A_{t-L+1} \\
\vdots \\
C_{t} A_{t-1}\cdots A_{t-L+1}
\end{bmatrix},
\calT_{[t-L+1,t]} =\begin{bmatrix}
D_{t-L+1} & 0 & 0 & \cdots & 0 \\
C_{t-L+2} B_{t-L+1} & D_{t-L+2} & 0 & \cdots & 0 \\
C_{t-L+3} A_{t-L+2} B_{t-L+1} & C_{t-L+3} B_{t-L+2} & D_{t-L+3} & \cdots & 0 \\
\vdots & \vdots & \vdots & \ddots & \vdots \\
C_{t} A_{t-1} \cdots A_{t-L+2}B_{t-L+1} & \cdots & \cdots & \cdots & D_{t}
\end{bmatrix}.$}
\]
Instead of learning the state-space matrices $(A_t, B_t, C_t, D_t)$ through (recursive) state-space system identification methods~\citep{verhaegen1992subspace, van1994n4sid, oku2002recursive}, we directly learn the subspaces $\bmU^t \in \bigcup_{j=1}^d \Gr(p, q_j)$ based on streaming data $w_t = v_t + \eta_t$, where $\eta_t$ is measurement noise. Note that $\dim \bmU^t$ is unknown and may vary depending on the rank of $\calO_{[t-L+1,t]}$.
The subspace $\bmU^t$ is known in behavioral system theory \citep{willems1986time, markovsky2021behavioral} as the \textit{restricted behavior} \citep{markovsky2022identifiability} and serves as the backbone of several powerful nonparametric control methods~\citep{coulson2019data, de2019formulas, markovsky2021behavioral, markovsky2008data}.
This online learning problem is studied in \citep{sasfi2025GREAT} under the assumption that $\dim \bmU^t$ is constant which may not hold in practical settings in which the complexity of the system varies over time (see Section \ref{subsec:geodesic_tracking}).
% We wish to design control inputs $u(t)$ that drive the unknown dynamical system~\eqref{eq:LTV} along a desired reference trajectory, while satisfying input and output safety constraints. When the parameters are time-invariant, powerful methods exist~\citep{coulson2019data, verhoek2021data, tanaskovic2017data}.
% For the case of \ac{LTI} systems, we can perform control tasks using a fixed data matrix \citep{coulson2019data}. However, for \ac{LTV} systems,
% the subspace of finite-length trajectories (restricted behavior) is changing which can cause poor control performance.
In what follows, we propose a method for learning $\bmU_t$ without assuming $\dim \bmU_t$ is constant, only assuming an upper bound on the subspace dimensions, and use the learned subspaces for a data-driven prediction task (see Section \ref{sec:ARX-system}).
% ---------------------------------------
% ---------------------------------------
\section{Online subspace learning on flag manifolds}\label{sec:flag_tracking_algorithm}
Inspired by recent advances in subspace learning for online system identification \citep{sasfi2025GREAT}, we propose a method for solving the learning problem in Section \ref{sec:problem-setup}, which consists of estimating the evolving subspace $\bmU^t$ using optimization on the flag manifold. Each element of the flag manifold is an ensemble of subspaces that can be used for downstream tasks such as data-driven prediction. Thus, our algorithm contributes to a data-to-prediction pipeline that continuously adapts to streaming data and the changing complexity of the system (see section \ref{sec:ARX-system}).
%===================================================
%===================================================
\subsection{Gradient descent on flag manifolds with streaming data}\label{subsec:why_flag}
Let $\{\bmU^t\}_{t \in \Zp}$ be a sequence of subspaces with each $\bmU^t \in \bigcup_{j=1}^{d} \Gr(p, q_j)$. At each time $t \in \Zp$ we observe a data sample $w_t = v_t + \eta_t$, where $v_t \in \bmU^t$ and $\eta_t$ is noise.
Let $T \geq 1$. For each $t \geq T-1$, let $W_t = [w_{t-T+1}~\cdots~w_t] \in \R^{p \times T}$ be the window of the most recent $T$ data samples.
For each $t \in \Zp$ and $j\in\{1,\dots,d\}$, we define the cost function $g_{W_t}^j \colon \Gr(p,q_j)\to\R$ by
\begin{equation}\label{eq:grass_cost_function}
g_{W_t}^j(\bmU) = \norm{W_{t} - \Pi_{\bmU}W_{t}}_F^2,
\end{equation}
where $\| \cdot \|_F$ is the Frobenius norm. Minimizing~\eqref{eq:grass_cost_function} over $\bmU \in \Gr(p, q_j)$ is known as \ac{PCA}~\citep[Section 2.4]{boumal2023introduction} and entails finding the $q_j$-dimensional subspace that best fits the data in the sense of minimizing the projection error. It is well-known that \ac{PCA} admits a closed-form solution~\citep{eckart1936approximation} given by the top $q_j$ singular vectors (principal vectors) of $W_t$.
% However, minimizing~\eqref{eq:grass_cost_function} requires searching over subspaces of a particular dimension $q_j$, which in our setting is unknown or even time-varying\citep{rabideau1996fast}.
One approach to solving the proposed problem would be running $d$ separate optimizations on each $\Gr(p, q_k)$ and select a suitable dimension, but this is computationally expensive and would yield optimal subspaces $\bmV_j^t := \argmin_{\bmU \in \Gr(p, q_j)} g_{W_t}^j(\bmU)$ that are not nested for $j=1, \dots,d$.
The lack of nestedness has been shown \citep{szwagier2025nestedsubspacelearningflags} to cause poor performance when using subspaces for tasks such as robust subspace recovery.
Instead, we propose optimizing over $\Flag(p,q_{1:d})$.
For each $t\in\Zp$ we define
\begin{equation}\label{eq:flag_cost_function}
f_{W_t}(\bmU_{1:d}) = \norm{W_{t} - \frac{1}{d}\sum_{i=1}^{d}\Pi_{\bmU_i}W_{t}}_F^2
\end{equation}
for all $\bmU_{1:d} \in \Flag(p, q_{1:d})$.
One key difference of optimizing~\eqref{eq:flag_cost_function} instead of~\eqref{eq:grass_cost_function} and is that we obtain a nested sequence of subspaces which can be leveraged for downstream tasks (e.g., data-driven prediction) via ensembling methods~\citep{dietterich2000ensemble} (see Section~\ref{sec:ARX-system}).
The operator $\frac{1}{d}\sum_{i=1}^{d}\Pi_{\bmU_i}$ demonstrates an ensemble of projections onto subspaces of different dimensions, which is called the ``flag trick'' in \citep[Definition 5]{szwagier2025nestedsubspacelearningflags}. Similar to~\eqref{eq:grass_cost_function}, minimizers of \eqref{eq:flag_cost_function} can be computed in closed form by computing the top-$q_d$ principal vectors of $W_t$~\citep[Theorem 4]{szwagier2025nestedsubspacelearningflags}.
% The optimal flag $\bmV_{1:d}^*$ thus consists of a sequence of subspaces $\bmV_1^* \subset \cdots \subset \bmV_d^*$, where each $\bmV_j^*$ is the span of the top $q_j$ principal vectors of $W_t$.
However, storing the sample covariance $W_t W_t^\top$ and compute the top-$q_d$ eigenvectors at every $t$
leads to expensive memory and time complexities for large-scale problems \citep{balzano2018streamingPCA},
motivating an online recursive approach.
Furthermore, one could consider minimizing different functions or modifications of~\eqref{eq:flag_cost_function} (e.g., with regularization) for which no closed-form solutions exist, further motivating iterative updates.
Based on results in optimization on flag manifolds \citep{ye2022optimization, zhu2024practical}, we perform \ac{RGD} \citep[Section 4.3]{boumal2023introduction} for \eqref{eq:flag_cost_function}.
The goal of \ac{RGD} is to seek a minimizer of the function \eqref{eq:flag_cost_function} by iterative updates. In particular, for each new data sample $w_t$, we perform $K>0$ steps along the gradient of~\eqref{eq:flag_cost_function}. Suppose that the initial point is $\hat{\bmU}^{T-1}_{1:d}$ (we use the superscript $T-1$ since we have to collect $T$ data points to form the first window $W_{T-1}$). For each fixed $t \geq T-1$,
we update
\begin{equation}\label{eq:flag_update}
\begin{aligned}
\hat{\bmU}^{t, k+1}_{1:d} = \Exp_{\hat{\bmU}^{t, k}_{1:d}}
\left(-\alpha_t \grad~f_{W_t}(\hat{\bmU}^{t, k}_{1:d})\right),
\end{aligned}
\end{equation}
where $k=0,\dots,K-1$, $\alpha_t > 0$ is the step size, and we denote
$\hat{\bmU}^{T-1,0}_{1:d} = \hat{\bmU}^{T-1}_{1:d}$,
$\hat{\bmU}^{t+1}_{1:d} \coloneqq \hat{\bmU}^{t, K}_{1:d}$,
$\hat{\bmU}^{t+1,0}_{1:d} = \hat{\bmU}^{t+1}_{1:d}$, for all $t \geq T-1$.
The Riemannian gradient $\grad~f_{W_t}$ is a vector field on the manifold, and $\Exp_{\hat{\bmU}^{t, k}_{1:d}}$ is the exponential map, which maps the tangent vector $-\alpha_t \grad~f_{W_t}(\hat{\bmU}^{t, k}_{1:d})$ back to the manifold.
Riemannian gradient and exponential map can be computed using only matrix algebra \citep{ye2022optimization}.
\subsection{Learning time- and dimension-varying subspaces}\label{subsec:flag_tracking_algorithm}
Having established the necessary foundations,
we propose the \textbf{F}lag \textbf{R}ecursive \textbf{ON}line \textbf{T}racking (FRONT) algorithm given by
\begin{figure}[tbp]
\centering
\begin{minipage}{0.48\linewidth}
\begin{algorithm}[H]
\small
\caption{FRONT}\label{alg:flag}
\begin{algorithmic}[1]
\Require Window length $T \geq 1$; initial estimate $\hat{\bmU}^{T-1}_{1:d}$; samples $\{w_t\}_{t \in \Zp}$; step sizes $\alpha_t$
\For{$t = T-1,T,\dots$}
\State Construct $W_t=[w_{t-T+1}\cdots w_t]$
\Statex \quad Initialize gradient descent: $\hat{\bmU}^{t,0}_{1:d} = \hat{\bmU}^{t}_{1:d}$
\For{$k = 0,1,\dots,K-1$}
\Statex \qquad Gradient step:
\Statex \qquad $\hat{\bmU}^{t,k+1}_{1:d} =
\Exp_{\hat{\bmU}^{t,k}_{1:d}}\!\left(-\alpha_t \grad f_{W_t}(\hat{\bmU}^{t,k}_{1:d})\right)$
\EndFor
\State \textbf{Update:} $\hat{\bmU}^{t+1}_{1:d} = \hat{\bmU}^{t,K}_{1:d}$
\EndFor
\end{algorithmic}
\end{algorithm}
\end{minipage}
\hfill
\begin{minipage}{0.48\linewidth}
\begin{algorithm}[H]
\small
\caption{GREAT}\label{alg:GREAT}
\begin{algorithmic}[1]
\Require Window length $T \geq 1$; initial estimate $\hat{\bmU}^{T-1}$; samples $\{w_t\}_{t \in \Zp}$; step sizes $\alpha_t$
\For{$t = T-1,T,\dots$}
\State Construct $W_t=[w_{t-T+1}\cdots w_t]$
\Statex \quad Initialize gradient descent: $\hat{\bmU}^{t,0} = \hat{\bmU}^{t}$
\For{$k = 0,1,\dots,K-1$}
\Statex \qquad Gradient step:
\Statex \qquad $\hat{\bmU}^{t,k+1} = \Exp_{\hat{\bmU}^{t,k}}\!\left(-\alpha_t\grad f_{W_t}(\hat{\bmU}^{t,k})\right)$
\EndFor
\State \textbf{Update:} $\hat{\bmU}^{t+1} = \hat{\bmU}^{t,K}$
\EndFor
\end{algorithmic}
\end{algorithm}
\end{minipage}
\end{figure}
Algorithm \ref{alg:flag}. FRONT is similar in structure to \ac{GREAT} \citep{sasfi2025GREAT} (see Algorithm \ref{alg:GREAT}), but generalizes Algorithm \ref{alg:GREAT} by learning an ensemble of models with different complexity and it does not assume that the true data-generating subspaces have fixed dimension.
Moreover, when $d=1$ so that $\Flag(p, (q_1)) = \Gr(p, q_1)$. Algorithm \ref{alg:flag} is equivalent to \ac{GREAT}. To prove this, we make a definition on algorithm equivalence.
\begin{definition}\label{def:algorithm_equivalence}
Let $M$ be a Riemannian manifold.
We define an algorithm $\calA$ on $M$ as an update mapping: $\calA: M \to M$ by $U^{t+1} = \calA(U^t).$
\end{definition}
\begin{definition}
Let $\calA_1, \calA_2$ be two algorithms on $M$ such that $U^{t+1}=\calA_1(U^t)$ and $V^{t+1}=\calA_2(V^t)$, respectively.
We say that $\calA_1$ is equivalent to $\calA_2$ if
$U^0=V^0 \text{ implies } U^t = V^t \text{ for all } t \geq 1. $
\end{definition}
\begin{proposition}
Let $\calA_1: \Flag(p, (q)) \to \Flag(p, (q))$ be defined by (\ref{eq:flag_update}), and $\calA_2:\Gr(p,q) \to \Gr(p,q)$ be defined by the update rule in Algorithm \ref{alg:GREAT}.
Then, $\calA_1$ is equivalent to $\calA_2$.
\end{proposition}
\begin{proof}
By definition, $\Flag(p, (q)) = \Gr(p, q)$, so $\calA_1$ and $\calA_2$ are defined on the same manifold.
Let $\bmU^0$ be the initial point of $\calA_1$ and $\bmV^0$ be the initial point of $\calA_2$. Suppose $\bmU^0 = \bmV^0$.
The cost function (\ref{eq:flag_cost_function}) becomes
$f_{W_t}(\hat{\bmU})=\|W_t - \Pi_{\hat{\bmU}}W_t\|_F^2$, which coincides with $g_{W_t}(\hat{\bmU})$. For each $t = T-1, T, \dots$ and $k \in \{0, \dots,K-1\}$,
let $\bmU^{t,k}$ and $\bmV^{t,k}$ be the points obtained by
the gradient step in Algorithm \ref{alg:flag} and Algorithm \ref{alg:GREAT}, respectively.
Since $\Flag(p, (q))$ and $\Gr(p, q)$ are the same manifold, their intrinsic geometric structures coincide. Thus, the exponential map and the Riemannian gradients are identical. This can also be verified by the formulas in \citep{ye2022optimization} for $\Flag(p,(q))$ with those in \citep{edelman1998geometry} for $\Gr(p,q)$.
Therefore, we have $\hat{\bmU}^{t, k} = \hat{\bmV}^{t, k}$ for all $t \in \{T-1,T,\dots\}$ and $k \in \{0,\cdots,K-1\}$, hence $\calA_1$ and $\calA_2$ are equivalent.
\end{proof}
Under assumptions in \citep[Assumptions 1-4]{sasfi2025GREAT}, all convergence guarantees and finite-sample analysis of \citep[Theorem 1]{sasfi2025GREAT} directly extend to Algorithm \ref{alg:flag} on $\Flag(p, (q))$. In particular, \citep {sasfi2025GREAT} derives finite-sample upper bounds on the distance $d(\hat{\bmU}^t,\bmU^t)$ in terms of subspace change rate, and singular values of the data window $W_t$. These results provide guaranteed convergence rates to the true underlying subspace and therefore give a quantitative method for determining conditions on data needed to satisfy $d(\hat{\bmU}^t,\bmU^t) < \epsilon_{\text{tol}}$ for a given $\epsilon_{\text{tol}}$. In the future work, we wish to study finite-sample convergence guarantees for Algorithm~\ref{alg:flag} in the more general case when the flag does not have a trivial signature $(p, (q))$.
To evaluate the effectiveness of Algorithm \ref{alg:flag}, we present two numerical case studies.
% --------------------------------------
% --------------------------------------
\section{Numerical case studies}\label{sec:numrical-study}
Throughout all numerical examples\footnote{The code is available at \url{https://github.com/DianJin-Frederick/FRONT-experiments}.}, when implementing Algorithm \ref{alg:flag}, we use the Python class \texttt{Flag} developed in \citep{szwagier2025nestedsubspacelearningflags} and use the \texttt{SteepestDescent} optimizer from Pymanopt \citep{pymanopt}, which uses backtracking line-search~\citep{boumal2023introduction} to specify the step sizes $\alpha_t$ used for the gradient steps in Algorithm \ref{alg:flag}. The number of gradient steps is $K=5$.
% ---------------------------
% ---------------------------
\subsection{Geodesic tracking}\label{subsec:geodesic_tracking}
We generate a sequence of time-varying ground-truth subspaces
on $\Flag(10, (1,\dots,5))$. The initial flag $\bmU^0$ has Stiefel representation
$
U^{0} = I_{10\times 5}.
$
For each $t \in \Zp$ we update $\bmU^{t+1} = \Exp_{\bmU^t}(\alpha \bmH^t),$
where $\bmH^t$ is a random tangent vector \citep[Section 8.3]{boumal2023introduction} at $\bmU^t$
and $\alpha = 5 \cdot 10^{-5}$.
At time \(T_{\text{switch}}=100\), we increase the subspace dimension by setting
$
U^{\Tswitch} = I_{10 \times 6}
$
and generate subsequent flags in the same way on $\Flag(10, (1,\dots,6))$.
At each time $t$ we observe a data point $w_t = U^t a_t + \eta_t$, where $a_t \sim \calN(0, I)$ and $\eta_t \sim \calN(0, 10^{-4}I)$ is noise.
For each $t \geq T-1$, the data window is constructed as $W_t = [w_{t-T+1} \dots w_t]$.
The subspace estimate is updated online by running Algorithm \ref{alg:flag} on $\Flag(10, (5, 6))$.
We set the initial estimate $\hat{\bmU}^T$ to be random and run with data window size $T \in \{1, 20, 50\}$. For each $T \in \{1, 20, 50\}$, we run 100 experiments across different random data sets $\{w_t\}_{t=1}^{200}$.
% -------------------------
% -------------------------
\begin{wrapfigure}[8]{r}{0.45\linewidth}
\vspace{-10pt} % adjust vertical spacing
\centering
\begin{tikzpicture}
\begin{axis}[
% xlabel={$t$},
xmin=0, xmax=200, xtick={0,20, 50,100,150,200},
ymin=0, ymax=1.30,
width=1.0\linewidth,
height=0.5\linewidth,
tick label style={font=\scriptsize},
label style={font=\scriptsize},
legend style={
draw=gray!50,
font=\scriptsize,
at={(0.5,1.0)}, anchor=south,
legend columns=3,
/tikz/every even column/.append style={column sep=6pt}
},
grid=both,
cycle list={
{myteal, mark=o},
{myorange, dashed, mark=square*},
{mypurple, dashed, mark=triangle*}
}
]
% Window = 1
\addplot[thick, color=myteal, line width=1.0pt] table[
col sep=comma,
header=false,
y index=0,
x expr=\coordindex
]{geodesic_tracking_data/chordal_distances_1_flag_5to6.csv};
\addlegendentry{$T=1$}
% Window = 20
\addplot[thick, color=myorange, line width=1.0pt] table[
col sep=comma,
header=false,
y index=0,
x expr=\coordindex+20
]{geodesic_tracking_data/chordal_distances_20_flag_5to6.csv};
\addlegendentry{$T=20$}
% Window = 50
\addplot[thick, solid, color=mypurple, line width=1.3pt] table[
col sep=comma,
header=false,
y index=0,
x expr=\coordindex+50
]{geodesic_tracking_data/chordal_distances_50_flag_5to6.csv};
\addlegendentry{$T = 50$}
\end{axis}
\end{tikzpicture}
\captionsetup{width=0.85\linewidth}
\caption{Average chordal distance for different window sizes across 100 experiments.}
\label{fig:chordal_distance}
\end{wrapfigure}
% -----------------------------------------
% -----------------------------------------
Figure \ref{fig:chordal_distance} shows how $\rmd(\hat{\bmU}^{t}, \bmU^t)$ evolve over time $t=T-1,\dots,200$ for different window sizes $T \in \{1, 20, 50\}$. For all $T\in\{1, 20, 50\}$, the distance decreases as more data are available, indicating that the estimates gradually align with the true subspaces. At time $\Tswitch$, the distance sharply increase due to the sudden change in subspace dimension, after which Algorithm \ref{alg:flag} readapts and the distance decreases again.
Larger data window sizes ($T=20, 50$) yield better estimates.
% To capture the dimension of the true underlying subspace $\bmU^t$, we introduce the truncated reconstruction error given by
% \[\rmE_t(k) \coloneqq \norm{W_t - \Pi_{\hat{\bmU}_k^t}W_t}_F^2 \text{ for } k=1,\dots,d.\]
% Recall that the flag $\hat{\bmU}^t_{1:d}$ is a sequence of nested subspaces $\hat{\bmU}^t_1 \subset \cdots \hat{\bmU}^t_d$. By projecting the data window $W_t$ onto those subspaces, we clearly see when adding a dimension does not reduce the reconstruction error.
% \begin{figure}[tbp]
% \centering
% \begin{tikzpicture}
% \pgfplotstableread[col sep=comma, header=false, trim cells=true]{geodesic_tracking_data/multi_level_errors_20_flag_5to6.csv}\datatable
% \begin{groupplot}[
% group style={group size=1 by 6, vertical sep=8pt},
% width=0.5\linewidth,
% height=0.15\linewidth,
% ymin=0, ymax=1, ytick={0, 0.5, 1},
% xmin=20, xmax=200,
% xtick={20, 100, 200},
% grid=both,
% tick label style={font=\scriptsize},
% label style={font=\scriptsize},
% ylabel style={font=\scriptsize, rotate=270},
% legend style={at={(0.5,-0.3)}, anchor=north, legend columns=-1}, % horizontal legend
% legend cell align={left}
% ]
% \nextgroupplot[ylabel={$\rmE_t(4)$}, xticklabels={}]
% \addplot+[no marks, color=myteal, line width=1.3pt] table[x expr=\coordindex+20, y index=3]{\datatable};
% \addplot[orange,dashed,line width=1pt] coordinates {(100,0) (100,1)};
% % U5
% \nextgroupplot[ylabel={$\rmE_t(5)$}, xticklabels={}]
% \addplot+[no marks, color=myteal, line width=1.3pt] table[x expr=\coordindex+20, y index=4]{\datatable};
% \addplot[orange,dashed,line width=1pt] coordinates {(100,0) (100,1)};
% % U6 (with x label)
% \nextgroupplot[ylabel={$\rmE_t(6)$}, xticklabels={}]
% \addplot+[no marks, color=myteal, line width=1.3pt] table[x expr=\coordindex+20, y index=5]{\datatable};
% \addplot[myorange,dashed,line width=1pt] coordinates {(100,0) (100,1)};
% \end{groupplot}
% \end{tikzpicture}
% \caption{\footnotesize{Average truncated reconstruction errors ($T=20$) across 100 experiments.
% }}
% \label{fig:multi_level_errors}
% \end{figure}
% In Fig. \ref{fig:multi_level_errors},
% $\rmE_t(4)$ is notably larger than $\rmE_t(5)$. Around $t = \Tswitch$, $\rmE_t(5)$ suddenly increases from near zero and remains high, while $\rmE_t(6)$ decreases. These observations are in accordance with the true dimensions of $\bmU^t$ being 5 for $t<\Tswitch$ and 6 for $t\geq \Tswitch$.
These results demonstrate that Algorithm \ref{alg:flag} effectively adapts to time-varying subspaces of different dimensions.
%-------------------------------------
%-------------------------------------
\subsection{Online data-driven prediction for a switched system}\label{sec:ARX-system}
The data-driven prediction problem posed in \citep[Section 4]{markovsky2008data} aims to find the output sequence of an unknown dynamical system corresponding to an input sequence and a past input-output trajectory collected from the system. For \ac{LTI} systems, data-driven prediction can be performed using a subspace representation of the set of finite-length trajectories (restricted behavior)~\citep{favoreel1999spc}. Motivated by the subspace representations of dynamical systems in Section~\ref{sec:motivation}, we will use Algorithm~\ref{alg:flag} to learn the subspace of finite-length trajectories of a time- and complexity-varying system and perform data-driven prediction.
We consider the data-driven prediction task for a \ac{SISO} \ac{ARX} system, whose dynamics is given by
\begin{equation}\label{eq:ARX}
\begin{cases}
y_t = 0.3 y_{t-1} - 0.02 y_{t-2} + 0.6 u_{t-1} + 0.2 u_{t-2}, & t<\Tswitch, \\
y_t = 1.5 y_{t-1} - 0.74 y_{t-2} + 0.12 y_{t-3} + 0.6 u_{t-1} + 0.2 u_{t-2} + 0.05 u_{t-3}, & t\geq \Tswitch,
\end{cases}
\end{equation}
The input dimension is $m=1$ and output dimension is $n=1$.
Note that the complexity of the dynamics increases at time $\Tswitch$ due to the increased time delay. Let $\Tsim=300$ and $\Tswitch=100$. Given a random input sequence $u_{[0,\Tsim-1]}$ and a prediction horizon $\Tf \in \Zp$, we wish to predict the corresponding future output trajectory $y_{[0,\Tsim-1]}$ of~\eqref{eq:ARX}. We assume that the measurement of output at time $t$ is corrupted by the independent noise $\eta_t$ for $t\in [0,\Tsim - 1]$.
The \ac{NSR} at time $t \in [0, \Tsim - 1]$ is defined as $\sigma_t \coloneqq \|\eta_t\|_2 / \|y_t\|_2$.
In our experiments, we fix this ratio to a constant $\sigma > 0$ for all $t$. The measurement noise is generated as $\eta_t \sim \mathcal{N}\!\left(0,\, (\sigma \|y_t\|_2)^2 I\right)$.
% ------------------------------
% \subsubsection{Offline Data Collection}\label{sec:offline_data_collection}
\paragraph{Offline Data Collection.}
Assume that we have access to a noisy $T_\rmd$-length input-output trajectory of~\eqref{eq:ARX} of length $T_\rmd=30$ denoted by $(u^\rmd_{[0,T_\rmd-1]}, y^\rmd_{[0,T_\rmd-1]} + \eta_{[0,T_\rmd]}^\rmd)$, where $\eta_t^\rmd \sim \mathcal{N}\!\left(0,\, (\sigma \|y_t^\rmd\|_2)^2 I\right)$ for $t \in [0, T_\rmd-1]$.
Let $\Tini=4$, $\Tf=4$, and $L=\Tini+\Tf=8$.
We sort the offline input-output data into a Hankel matrix of depth $L$ and partition it as
\[
H \coloneqq \begin{bmatrix}
u^\rmd_{[0, L-1]} & \cdots & u^\rmd_{[T_\rmd-L, T_\rmd-1]} \\[5pt]
y^\rmd_{[0, L-1]} + \eta_{[0,L-1]}^\rmd & \cdots & y^\rmd_{[T_\rmd-L, T_\rmd-1]} + \eta_{[T_\rmd-L, T_\rmd-1]}^\rmd
\end{bmatrix} \eqqcolon \begin{bmatrix}
U_\rmp \\
U_\rmf \\
Y_\rmp \\
Y_\rmf
\end{bmatrix},
\]
where
$U_\rmp \in \R^{m\Tini \times (T_\rmd-L+1)}, U_\rmf \in \R^{m\Tf \times (T_\rmd-L+1)}, Y_\rmp \in \R^{n\Tini \times (T_\rmd-L+1)}, Y_\rmf \in \R^{n\Tf \times (T_\rmd-L+1)}$.
The \ac{SVD} of $H$ is given by $H = X \Sigma Y^\top$, where we denote $X=[x_1 \cdots x_{(m+n)L}]$. Then we set the initial point of Algorithm \ref{alg:flag} on $\Flag((m+n)L, (q_1,\dots,q_d))$ to be the flag $\bmV_{1:d}^0 = \{\bmV^0_{k}\}_{k=1}^d$, where $\bmV^0_{k} = \Span\{x_1, \cdots, x_{q_k}\}$ for $k \in [1,d]$.
% --------------------------------------
% --------------------------------------
\paragraph{Subspace Predictor.}
The predicted output at time $t$ can be obtained by the subspace predictor \citep{favoreel1999spc},
which yields a $\Tf$-length prediction
\begin{equation}\label{eq:offline-subspace-prediction}
y^{\text{pred}}_t =
\begin{bmatrix}
y^{\text{pred}}_t(0) \\ \vdots \\ y^{\text{pred}}_t(\Tf-1)
\end{bmatrix}
= Y_\rmf
\begin{bmatrix}
U_\rmp \\
U_\rmf \\
Y_\rmp
\end{bmatrix}^\dag
\begin{bmatrix}
u_{[t-T_\ini, t-1]} \\ u_{[t,t+\Tf-1]} \\
y_{[t-T_\ini, t-1]} + \eta_{[t-\Tini, t-1]}
\end{bmatrix} \in \R^{n\Tf}.
\end{equation}
% In a receding horizon fashion, we only keep the first element of the $\Tf$-length prediction $y^{\text{pred}}_t$ and denote it by $\hat{y}_t$.
% Fig.~\ref{fig:trajectory} plots the predictions $\hat{y}_t$ for every $t\in[0,\Tsim-1]$. We observe that the predictive performance after $\Tswitch$ is poor since the subspace predictor~\eqref{eq:subspace-predictor} does not accurately represent the dynamics~\eqref{eq:ARX} as it is constructed with offline data.
In order to cope with the changing dynamics of~\eqref{eq:ARX}, we use Algorithm~\ref{alg:flag} to update the subspace (restricted behavior of \eqref{eq:ARX}: see Section \ref{sec:motivation}) based on streaming data $w_t \coloneqq (u_{[t-T+1, t]}, y_{[t-T+1, t]} + \eta_{[t-T+1, t]})$ and initial point $\bmV_{1:d}^0$, where $\eta_t \sim \calN(0, (\sigma \|y_t\|_2)^2 I)$. We then update the subspace predictor $Y_\rmf([U_\rmp^\top ~U_\rmf^\top~ Y_\rmp^\top]^\top)^\dag$ in~\eqref{eq:offline-subspace-prediction} based on the learned flag.
Let $\hat{\bmV}_{1:d}^t = \{\hat{\bmV}_k^t\}_{k=1}^d$ be the $t^\text{th}$ flag estimate produced by Algorithm~\ref{alg:flag} on $\Flag(16, (q_1,\dots,q_d))$.
The Stiefel representative $\hat{V}_{1:k}^t$ of $\hat{\bmV}_k^t$ is obtained by taking the first $q_k$ columns of the Stiefel representative $\hat{V}_{1:d}^t$ of $\hat{\bmV}_{1:d}^t$. Then we partition the rank-$q_k$ matrix $\hat V_{1:k}^{t}$ as
$
\hat V_{1:k}^{t} = [U_{\rmp,k}^\top ~ U_{\rmf,k}^\top ~
Y_{\rmp,k}^\top ~ Y_{\rmf,k}^\top]^\top
$
where $
U_{\rmp,k} \in \R^{m\Tini \times q_k},~
U_{\rmf,k} \in \R^{m\Tf \times q_k},~
Y_{\rmp,k} \in \R^{n\Tini \times q_k},~
Y_{\rmf,k} \in \R^{n\Tf \times q_k}.
$
For each $k$, we form the $\Tf$-step prediction
\begin{equation}\label{eq:subspace_prediction}
y_{t,k}^\text{pred}
=
\begin{bmatrix}
y^{\text{pred}}_{t,k}(0) \\ \vdots \\ y^{\text{pred}}_{t,k}(\Tf-1)
\end{bmatrix}
=
Y_{\rmf,k}
\begin{bmatrix}
U_{\rmp,k} \\
U_{\rmf,k} \\
Y_{\rmp,k}
\end{bmatrix}^{\!\dag}
\begin{bmatrix}
u_{[t-T_\ini, t-1]} \\
u_{[t, t+\Tf-1]} \\
y_{[t-T_\ini, t-1]} + \eta_{[t-\Tini, t-1]}
\end{bmatrix} \in \R^{n\Tf},
\end{equation}
and keep only its first component $y^{\text{pred}}_{t,k}(0)$, simplifying its notation as $\hat y_t(k) \in \R^{n}$.
Thus, by leveraging the nestedness of a flag, at each time $t$, we obtain $d$ different predictions $\hat{y}_t(1), \dots, \hat{y}_t(d)$ from matrices of different ranks based on only one matrix $\hat{V}_{1:d}^t$.
Linking back to the motivation made in Section \ref{subsec:why_flag}, we also conduct a baseline simulation by running Algorithm \ref{alg:flag} on $\Gr(16, q_k)$ for $k=1,\dots,d$ and use the individually learned subspace $\hat{\bmU}_k^t \in \Gr(16, q_k)$ to perform prediction by the subspace predictor constructed in the same fashion as in \eqref{eq:subspace_prediction}.
We measure the prediction performance by the cumulative prediction error over the whole simulation horizon $\Tsim-\Tf$ given by
$\sum_{t=0}^{\Tsim-\Tf-1} \|y_t - \hat{y}_t\|_2^2,$
where $\hat{y}_t$ is the prediction at time $t$ from the corresponding model. We set $\Tini = \Tf = 4$ and the window size $T=20$.
\begin{wrapfigure}[13]{r}{0.45\linewidth}
\vspace{-15pt}
\centering
\begin{tikzpicture}[scale=0.65]
\begin{axis}[
width=1.6\linewidth,
height=0.95\linewidth,
ybar,
bar width=12pt,
xlabel={Subspace Dimension},
xtick=data,
ymin=0,
legend style={
draw=none,
at={(0.5, 1.05)},
anchor=south,
legend columns=-1,
font=\large,
column sep=6pt
},
legend image code/.code={
\draw[draw=none, fill=#1] (0cm,-0.08cm) rectangle (0.35cm,0.18cm);
},
tick label style={font=\large},
label style={font=\large},
grid=both
]
% --- Flag (median + IQR) ---
\addplot[
fill=orange!60, draw=none,
error bars/.cd,
y dir=both,
y explicit,
]
table[
x index=0,
y index=1,
y error expr=\thisrowno{2}/2,
col sep=comma, header=false,
]{data_prediction_experiment/flag_vs_gr_median.csv};
\addlegendentry{Nested subspaces in flag};
% --- Grassmann (median + IQR) ---
\addplot+[
fill=myteal!40, draw=none,
error bars/.cd,
y dir=both,
y explicit,
]
table[
x index=0,
y index=3,
y error expr=\thisrowno{4}/2,
col sep=comma, header=false
]{data_prediction_experiment/flag_vs_gr_median.csv};
\addlegendentry{Grassmannians};
\end{axis}
\end{tikzpicture}
\captionsetup{width=0.85\linewidth}
\caption{Median cumulative prediction errors with interquartile whiskers: a single flag evaluated at different ranks (orange) versus individually learned Grassmann subspaces (teal).}
\label{fig:flag_vs_gr}
\end{wrapfigure}
Figure ~\ref{fig:flag_vs_gr} shows the median cumulative prediction errors of both approaches, where we set $d=8$ and $(q_1,\dots,q_d) = (8,\dots,15)$. The \ac{NSR} is set to be $\sigma=0.02$.
All models were evaluated over 100 independent trials, and in each trial, all the models use the same offline input-output data $(u^\rmd_{[0,T_\rmd-1]}, y^\rmd_{[0,T_\rmd-1]} + \eta_{[0,T_\rmd]}^\rmd)$, input sequence $u_{[0, \Tsim-1]}$ and measurement noise $\eta_{[0,\Tsim-1]}$.
The results demonstrate that the nested subspace predictions from a single flag outperforms individually optimized Grassmannians for many dimensions in terms of median.
The interquartile range between the $30^\text{th}$ and $70^\text{th}$ percentiles is shown as whiskers in Figure \ref{fig:flag_vs_gr}.
% ------------------------------------
% ------------------------------------
\subsubsection{Ensemble of Nested Subspace Prediction}
Since the true system order is unknown,
relying on a single subspace from the flag estimate may lead to biased predictions. To mitigate this, inspired by ensembling methods~\citep{dietterich2000ensemble}, we aggregate information from all candidate subspaces $\{\hat{\bmV}_{k}^t\}_{k=1}^d$. Specifically, we compute an ensemble of predictions by averaging $\hat{y}_t(k)$ obtained from \eqref{eq:subspace_prediction} across $k=1,\dots,d$, yielding
$
\hat{y}_t
\coloneqq
\frac{1}{d}\sum_{k=1}^d \hat{y}_t(k).
$
This ensemble approach leverages the hierarchical structure of the flag to balance the contribution of lower- and higher-dimensional subspaces, yielding more robust online predictions. We name this model as \textbf{ensemble-$(q_1,\dots,q_d)$ prediction.}
% --------------------------------
We choose an ensemble of dimension $q_1 = 9$ and $q_2 = 10$ based on validation and evaluate the ensemble-$(9, 10)$ prediction against several baseline models: no learning, N4SID, \ac{PAST}.
The \emph{no learning} model uses \eqref{eq:offline-subspace-prediction} to predict future outputs without any adaptive updates to the subspace predictor.
The N4SID method~\citep{van1994n4sid} is implemented in an online fashion: at each time $t$ we re-identify a state-space model $(A_t, B_t, C_t, D_t)$ from the current input-output data window $W_t=[w_{t-T+1} \cdots w_t]$. The state is estimated by a Kalman filter.
Then the estimated state-space model is used to compute $\hat{y}_t$.
\begin{figure}[tbp]
\centering
\pgfplotsset{compat=1.18}
% title, true-file, pred-file, color
\newcommand{\trajplot}[4]{%
\begin{minipage}{0.48\linewidth}
\centering
\begin{tikzpicture}
\begin{axis}[
width=\linewidth, height=3.5cm,
xmin=0, xmax=300,
ymin=-8, ymax=8,
% ytick={-1,0,1},
grid=both,
title={\footnotesize #1},
tick label style={font=\scriptsize},
label style={font=\scriptsize},
legend style={at={(0.5,0.95)}, anchor=south, legend columns=-1,
draw=none, fill=none, font=\scriptsize},
]
% true trajectory
\addplot[thick, color=gray]
table[col sep=comma, header=false, x expr=\coordindex, y index=0]
{#2};
% predicted trajectory
\addplot[thick, color=#4]
table[col sep=comma, header=false, x expr=\coordindex, y index=0]
{#3};
\legend{True, Prediction}
\end{axis}
\end{tikzpicture}
\end{minipage}%
}
% --- paths wrapped in \detokenize{...} to allow spaces/parentheses ---
\trajplot{No learning}
{\detokenize{data_prediction_experiment/y_true_at_No_learning.csv}}
{\detokenize{data_prediction_experiment/y_pred_No_learning.csv}}
{myteal}
\hfill
\trajplot{$\mathrm{Flag}(16,(9,10))$}
{data_prediction_experiment/y_true_at_Flag_9_10.csv}
{data_prediction_experiment/y_pred_Flag_9_10.csv}
{orange}
\caption{True and predicted trajectories.
Left: no learning baseline. Right: the ensemble-$(9, 10)$ prediction model adapts and tracks the system after $t=\Tswitch$ quickly. }
\label{fig:y_prediction_two}
\end{figure}
The \ac{PAST} algorithm~\citep{yang1995projectionPAST} performs recursive subspace tracking based on each new $w_t$, where the estimated subspace is used in \eqref{eq:subspace_prediction} for prediction.
In addition, we evaluate the ensemble-$(8,\dots,15)$ prediction on $\Flag(16, (8, \dots, 15))$ to examine the performance when combining subspaces with high cumulative prediction errors (see Figure \ref{fig:flag_vs_gr}).
To study the effect of noise, we vary the \ac{NSR} level as $\sigma \in \{0.01, 0.02, 0.05, 0.1, 0.2\}$.
For each \ac{NSR} level, we conduct 100 independent trials.
All models are evaluated using the same 100 offline input-output data, identical input sequences, and the same noise realizations across all \ac{NSR} levels to ensure a fair comparison.
\subsubsection{Results}
% ----------------------------------------
\begin{wrapfigure}[12]{r}{0.5\linewidth}
\vspace{-8pt}
\centering
\pgfplotsset{compat=1.18}
\begin{tikzpicture}[scale=0.8]
\begin{axis}[
width=1.2\linewidth, height=5cm,
xmode=log, log basis x=10,
% xlabel={Noise-to-signal ratio (NSR)},
xtick = {0.01, 0.02, 0.05, 0.1, 0.2},
xticklabels={0.01,0.02,0.05,0.1,0.2},
xmin=0.01, xmax=0.2,
ymin=4, ymax=16,
ytick = {5, 10, 15, 20},
grid=both,
legend columns=2,
legend style={
anchor=south,
at={(0.5, 1.02)},
draw=none,
font=\footnotesize,
},
tick label style={font=\footnotesize},
label style={font=\footnotesize},
]
% --- read CSV by index (no header parsing) ---
\pgfplotstableread[
col sep=comma,
header=false,
skip first n=1, % <-- drop the CSV header row
trim cells=true
]{data_prediction_experiment/nsr_cpe_med_pct.csv}\datatable
% ---- helper macros ----
\newcommand{\plotbandidx}[4]{%
\addplot[name path=up, draw=none, forget plot]
table[x index=#1, y index=#3]{\datatable};
\addplot[name path=down, draw=none, forget plot]
table[x index=#1, y index=#2]{\datatable};
\addplot[fill opacity=0.15, draw=none, forget plot]
fill between[of=up and down];
\addplot[fill=#4, fill opacity=0.2, draw=none, forget plot]
fill between[of=up and down];
}
\newcommand{\plotmedianidx}[5]{%
\addplot+[
line width=1.2pt,
mark=*, mark size=1.7pt,
color=#3,
mark options={solid, fill=#3, draw=#3},
#5
]
table[x index=#1, y index=#2]{\datatable};
\addlegendentry{#4}
}
% ---- colors ----
% \colorlet{NoLearn}{coolgray}
\colorlet{N4SID}{gray}
\colorlet{PAST}{mycyan}
\colorlet{Gr9}{slategreen}
\colorlet{Gr10}{myblue}
\colorlet{Gr11}{seateal}
\colorlet{FlagBig}{dustyrose}
\colorlet{Flag910}{myorange!80}
% \plotbandidx{0}{5}{6}{N4SID}
\plotmedianidx{0}{4}{N4SID}{N4SID}{mark=square*}
% \plotbandidx{0}{8}{9}{PAST}
\plotmedianidx{0}{7}{PAST}{PAST}{mark=triangle*}
% \plotbandidx{0}{14}{15}{Gr9}
\plotmedianidx{0}{13}{Gr9}{$\Gr(9)$}{mark=diamond}
% \plotbandidx{0}{17}{18}{Gr10}
\plotmedianidx{0}{16}{Gr10}{$\Gr(10)$}{mark=diamond*}
% \plotbandidx{0}{20}{21}{Gr11}
% \plotmedianidx{0}{19}{Gr11}{Gr(11)}{solid}
% \plotbandidx{0}{23}{24}{FlagBig}
\plotmedianidx{0}{22}{FlagBig}{$\Flag(16, (8,\dots,15))$}{mark=square}
% \plotbandidx{0}{11}{12}{orange!80}
\plotmedianidx{0}{10}{Flag910}{$\Flag(16, (9,10))$}{solid}
\end{axis}
\end{tikzpicture}
\captionsetup{width=0.9\linewidth}
\caption{Median cumulative prediction error ($y$-axis) versus varying \ac{NSR} ($x$-axis).}
\label{fig:NSR-CPE}
\end{wrapfigure}
% ---------- comparison of all models -------------
% -------------------------------------------------
Figure \ref{fig:y_prediction_two} shows, for the no learning baseline and ensemble-$(9, 10)$ prediction model,
the predicted trajectory and the corresponding ground truth from the trial whose cumulative prediction error is closest to the median over 100 trials.
Figure \ref{fig:NSR-CPE} reports the median cumulative prediction error versus the \ac{NSR} for all models. The ensemble-$(9, 10)$ prediction model achieves nearly the same median error as the Grassmann models $\Gr(16, 9)$ and $\Gr(16, 10)$. Combining models with different dimensional subspaces provides a robust prediction when the true complexity of the system is varying and uncertain.
At higher \ac{NSR} levels, the ensemble-$(8,\dots,15)$ prediction model yields slightly larger errors than the ensemble-$(9, 10)$ prediction model but outperforms N4SID. The dimensions in the ensemble-$(8,\dots,15)$ prediction model are chosen from the range $[\Tini+\Tf, (m+n)L-1]$ without any prior knowledge of the true system dimension.
This result indicates that even when the subspace dimensions are guessed broadly, the flag structure can still produce competitive predictions when \ac{NSR} is high.
\vspace{-8pt}
% ----------------------------------------
\section{Conclusion}\label{sec:conclusion}
We proposed the \ac{FRONT} algorithm for online system identification and data-driven prediction that adapts to time-varying systems by recursively updating a nested hierarchy of subspaces. This enables the predictor to accommodate unknown or changing system dimensions without prior model information.
We proved that FRONT is a natural generalization of \ac{GREAT}.
Numerical studies demonstrated that the proposed predictor effectively tracks abrupt changes in dynamics and consistently achieves better prediction error than most baselines even under high \ac{NSR}.
Future work focuses on finite sample convergence of Algorithm \ref{alg:flag} for nontrivial flag signatures and integrating the proposed flag-based learning framework into an adaptive data-to-control pipeline.