A Combinatorial Characterization of Supervised Online Learnability
Extracted source text · arXiv 2307.03816
\newcount\Comments
\Comments=1
%\PassOptionsToPackage{linesnumbered}{algorithm2e}
\documentclass{article}
\usepackage{tcolorbox}
\usepackage{bbm}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{algorithm}
\usepackage{mathtools}
\usepackage{amsthm}
\usepackage{comment}
\usepackage{natbib}
\usepackage{authblk}
\PassOptionsToPackage{algo2e,ruled, linesnumbered}{algorithm2e}
\RequirePackage{algorithm2e}
\usepackage[letterpaper,top=2cm,bottom=2cm,left=3cm,right=3cm,marginparwidth=1.75cm]{geometry}
\usepackage[colorlinks=true, allcolors=blue]{hyperref}
\DeclarePairedDelimiter\ceil{\lceil}{\rceil}
\DeclarePairedDelimiter\floor{\lfloor}{\rfloor}
\newcommand{\reals}{\mathbb{R}}
\newcommand{\naturals}{\mathbb{N}}
\newcommand{\integers}{\mathbb{Z}}
\newcommand{\complex}{\mathbb{C}}
\usepackage{color}
\definecolor{darkgreen}{rgb}{0,0.5,0}
\definecolor{purple}{rgb}{1,0,1}
% \kibitz{color}{comment} inserts a colored comment in the text
\newcommand{\kibitz}[2]{\ifnum\Comments=1\textcolor{#1}{#2}\fi}
% add yourself here:
\newcommand{\vinod}[1]{\kibitz{red} {[VIN: #1]}}
\newcommand{\ambuj}[1] {\kibitz{blue} {[AMB: #1]}}
\newcommand{\unique}[1] {\kibitz{green} {[UNQ: #1]}}
\newcommand{\Acal}{\mathcal{A}}
\newcommand{\Bcal}{\mathcal{B}}
\newcommand{\Dcal}{\mathcal{D}}
\newcommand{\Fcal}{\mathcal{F}}
\newcommand{\Gcal}{\mathcal{E}}
\newcommand{\Hcal}{\mathcal{H}}
\newcommand{\Ical}{\mathcal{I}}
\newcommand{\Ncal}{\mathcal{N}}
\newcommand{\Mcal}{\mathcal{M}}
\newcommand{\Scal}{\mathcal{S}}
\newcommand{\Tcal}{\mathcal{T}}
\newcommand{\Zcal}{\mathcal{Z}}
\newcommand{\Xcal}{\mathcal{X}}
\newcommand{\Ycal}{\mathcal{Y}}
\newcommand{\Rcal}{\mathcal{R}}
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\renewcommand{\qedsymbol}{$\blacksquare$}
\DeclareMathOperator*{\expect}{\operatorname{\mathbb{E}}}
\newcommand{\var}{\operatorname{var}}
\newcommand{\cov}{\operatorname{cov}}
\newcommand{\svar}{\operatorname{\widehat{var}}}
\newcommand{\scor}{\operatorname{\widehat{cor}}}
\newcommand{\scov}{\operatorname{\widehat{cov}}}
\newcommand{\prob}{\mathbb{P}}
\newcommand{\trace}{\operatorname{tr}}
\newcommand{\indicator}{\mathbbm{1}}
\newcommand{\onevec}{\mathbf{1}}
\newcommand{\identity}{\mathbf{I}}
\newcommand{\vc}{\text{vc}}
\newcommand{\norm}[1]{\left\lVert#1\right\rVert}
\newcommand{\inner}[2]{\left\langle #1, #2 \right\rangle}
% \newtheorem{theorem}{Theorem}
% \newtheorem{proposition}[theorem]{Proposition}
% \newtheorem{lemma}[theorem]{Lemma}
% \newtheorem{corollary}[theorem]{Corollary}
% \newtheorem{assumption}{Assumption}
% \newtheorem{example}[theorem]{Example}
% \newtheorem*{theorem*}{Theorem}
% \newtheorem*{remark*}{Remark}
\usepackage{natbib}
\tcbuselibrary{theorems}
\newtcbtheorem
[]% init options
{definition}% name
{Definition}% title
{%
colback=black!5,
colframe=black!35!black,
fonttitle=\bfseries,
}% options
{def}% prefix
\newtcbtheorem
[]% init options
{theorem}% name
{Theorem}% title
{%
colback=black!5,
colframe=black!35!black,
fonttitle=\bfseries,
}% options
{thm}% prefix
\newtcbtheorem
[]% init options
{lemma}% name
{Lemma}% title
{%
colback=black!5,
colframe=black!35!black,
fonttitle=\bfseries,
}% options
{lem}% prefix
\newtcbtheorem
[]% init options
{corollary}% name
{Corollary}% title
{%
colback=black!5,
colframe=black!35!black,
fonttitle=\bfseries,
}% options
{cor}% prefix
\newtcbtheorem
[]% init options
{proposition}% name
{Proposition}% title
{%
colback=black!5,
colframe=black!35!black,
fonttitle=\bfseries,
}% options
{prop}% prefix
\newtcbtheorem
[]% init options
{openquestion}% name
{Open Question}% title
{%
colback=black!5,
colframe=black!35!black,
fonttitle=\bfseries,
}% options
{openq}% prefix
\newtcbtheorem
[]% init options
{contributions}% name
{Main Technical Contributions}% title
{%
colback=black!5,
colframe=black!35!black,
fonttitle=\bfseries,
}% options
{contributions}% prefix
\newtcbtheorem
[]% init options
{guidingquestion}% name
{Guiding Question}% title
{%
colback=black!5,
colframe=black!35!black,
fonttitle=\bfseries,
}% options
{guiding}% prefix
\title{A Combinatorial Characterization of Online Learning Games with Bounded Losses}
%Online learnability with bounded losses
%A general theory of online learnability with bounded losses
\author[ ]{Vinod Raman, Unique Subedi, Ambuj Tewari}
\affil[ ]{Department of Statistics, University of Michigan}
\affil[ ]{\texttt{\{vkraman, subedi, tewaria\}@umich.edu}}
\date{}
\begin{document}
\maketitle
\begin{abstract}
We study the online learnability of hypothesis classes with respect to arbitrary, but bounded, loss functions. We give a new scale-sensitive combinatorial dimension, named the sequential Minimax dimension, and show that it gives a tight quantitative characterization of online learnability. As applications, we give the first quantitative characterization of online learnability for two natural learning settings: vector-valued regression and multilabel classification.
\end{abstract}
\section{Introduction}
The learnability of a hypothesis class $\mathcal{H}$ is a fundamental question in online learning theory. In the standard supervised model, a learner plays a repeated game against an adversary over $T \in \mathbb{N}$ rounds. In each round $t \in [T]$, an adversary picks a labeled example $(x_t, y_t) \in \mathcal{X} \times \mathcal{Y}$ and reveals $x_t$ to the learner. The learner observes $x_t$, picks a probability measure $\mu_t$ over $\mathcal{Z}$, and then makes a randomized prediction $z_t \sim \mu_t$. Finally, the adversary reveals the true label $y_t$ and the learner suffers the loss $\ell(y_t, z_t)$, where $\ell:
\mathcal{Y} \times \mathcal{Z} \rightarrow \mathbb{R}_{\geq 0}$ is some pre-specified, bounded loss function. At the end of $T$ rounds, the expected regret
% $$R(T) := \mathbb{E}\left[\sum_{t=1}^T \ell(y_t, z_t) - \inf_{h \in \mathcal{H}} \sum_{t=1}^T\ell(y_t, h(x_t))\right],$$
\noindent is the expected difference between the cumulative loss of the learner's predictions and that of the best fixed hypothesis in hindsight. We say a tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$ is \textit{online learnable} if there exists a learner such that its expected regret is a sublinear function of $T$, for any strategy of the adversary.
Mathematically, we can reformulate the notion of online learnability in terms of the minimax value of the game \citep{rakhlin2015online}, defined as
\begin{equation*}
\mathcal{V}_{T}(\Hcal, \ell) := \sup_{x_1 \in \Xcal}\inf_{\mu_1 \in \Pi(\Zcal)} \sup_{y_1 \in \Ycal} \expect_{z_1 \sim \mu_1} \ldots \sup_{x_T \in \Xcal}\inf_{\mu_T \in \Pi(\Zcal)} \sup_{y_T \in \Ycal} \expect_{z_t \sim \mu_T}\left[\sum_{t=1}^T \ell(y_t, z_t) - \inf_{h \in \mathcal{H}} \sum_{t=1}^T\ell(y_t, h(x_t)) \right],
\end{equation*}
where $\Pi(\Zcal)$ is the set of all probability measures over a measure space $(\Zcal, \sigma(\Zcal))$. Equivalently, we say a tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$ is online learnable if and only if the minimax value of the associated game satisfies
\[\limsup_{T \to \infty} \frac{\mathcal{V}_{T}(\Hcal, \ell)}{T} \leq 0. \]
In other words, we require $\mathcal{V}_{T}(\Hcal, \ell)$ to be a sublinear function of $T$.
In this work, we are interested in characterizing online learnability in full generality. The only known sequential complexity measure for an arbitrary tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$ is the sequential Rademacher complexity of the loss class $\ell \circ \Hcal := \{(x, y) \mapsto \ell(y, h(x)): h \in \mathcal{H}\}$. \cite{rakhlin2015online} show that this complexity measure upper bounds the minimax value of the game, thus providing a sufficient condition for online learnability in general. In particular, \cite{rakhlin2015online} show that if the sequential Rademacher complexity of the loss class $\ell \circ \mathcal{H}$ is sublinear in the time horizon $T$, then the tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$ is online learnable. However, \cite{hanneke2023multiclass} prove that even for the natural problem of online multiclass classification, sublinear sequential Rademacher complexity is not \textit{necessary} for online learnability. Accordingly, we investigate the following question.
\begin{guidingquestion}{}{}
Given any tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$, what are necessary and sufficient conditions for $\mathcal{H}$ to be online learnable with respect to $\ell$?
\end{guidingquestion}
In many settings, the online learnability of a tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$ is characterized in terms of a combinatorial dimension. For instance, when $\mathcal{Z} = \mathcal{Y}$ and $\ell(y, z) = \mathbbm{1}\{y \neq z\}$, the Littlestone dimension of $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$, denoted $\text{Ldim}(\mathcal{H})$, characterizes its online learnability. That is, a hypothesis class $\mathcal{H}$ is online learnable with respect to the 0-1 loss if and only if $\text{Ldim}(\mathcal{H})$ is finite \citep{Littlestone1987LearningQW, DanielyERMprinciple, hanneke2023multiclass}. When $\mathcal{Z} = \mathcal{Y}$ is a bounded subset of $\mathbb{R}$, the \textit{sequential} fat-shattering dimension of a real-valued function class $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$, denoted $\text{seq-fat}_{\gamma}(\mathcal{H})$, characterizes the online learnability of $\mathcal{H}$ with respect to the absolute value loss.
% Unlike the Littlestone dimension, the sequential fat-shattering dimension is scale-sensitive. That is, $\text{seq-fat}_{\gamma}(\mathcal{H})$ is defined at every scale $\gamma > 0$. Accordingly,
\cite{rakhlin2015online} show that a real-valued function class $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$ is online learnable with respect to the absolute value loss if and only if its sequential fat-shattering dimension is finite at every scale $\gamma > 0$.
Remarkably, the Littlestone and sequential fat-shattering dimension also provide a tight \textit{quantitative} characterization of online learnability, appearing in both the lower and upper bounds of their respective minimax values. In addition, the finiteness of these dimension allows us to design generic online learners for arbitrary hypothesis classes in their respective settings. This motivates the next question we investigate.
\begin{guidingquestion}{}{}
Given any tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$, is there a combinatorial dimension that provides a tight quantitative characterization of the online learnability of $\mathcal{H}$ with respect to $\ell$? If so, does its finiteness lead to a generic online learner?
\end{guidingquestion}
% The proof techniques in online learning are constructive and use the finiteness of combinatorial dimensions to design generic online learners for arbitrary hypothesis classes. In this spirit, \cite{rakhlin_relax_n_randomize} design a generic online learner for any tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$ whose sequential Rademacher complexity of $\ell \circ \mathcal{H}$ is sublinear in $T$. This motivates the question whether there exists a generic online learner for any tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$.
% We say that a hypothesis class $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$ is online learnable if there exists a learning algorithm such that no matter what stream the adversary picks, the (expected) cumulative loss of the learner's predictions is not too far from that of the best-fixed hypothesis in hindsight.
% We can ma
% Like in PAC learning, combinatorial dimensions play an important role in characterizing when a hypothesis class $\mathcal{H}$ is online learnable under different settings of $\mathcal{Z}$, $\mathcal{Y}$, and $\ell: \mathcal{Y} \times \mathcal{Z} \rightarrow \mathbbm{R}_{\geq 0}$. For instance, when $\mathcal{Z} = \mathcal{Y}$ and $\ell$ is the 0-1 loss, the Littlestone dimension of $\mathcal{H} \subset \mathcal{Z}^{\mathcal{X}}$, denoted $\text{Ldim}(\mathcal{H})$, characterizes its online learnability. That is, a hypothesis class $\mathcal{H}$ is online learnable with respect to the 0-1 loss if and only if $\text{Ldim}(\mathcal{H})$ is finite \citep{Littlestone1987LearningQW, DanielyERMprinciple, hanneke2023multiclass}. When $\mathcal{Z} = \mathcal{Y}$ is a bounded subset of $\mathbb{R}$, the \textit{sequential} fat-shattering dimension of a real-valued function class $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$, denoted $\text{seq-fat}_{\gamma}(\mathcal{H})$, characterizes the online learnability of $\mathcal{H}$ with respect to the absolute value loss \citep{rakhlin2015online}. Unlike the Littlestone dimension, the sequential fat-shattering dimension is a scale-sensitive dimension. That is, $\text{seq-fat}_{\gamma}(\mathcal{H})$ is defined at every scale $\gamma > 0$. Accordingly, a real-valued function class $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$ is learnable with respect to the absolute value loss if and only if its sequential fat-shattering dimension is finite at \textit{every} scale $\gamma > 0$ \citep{rakhlin2015online}.
% In this paper, we go beyond online classification and scalar-valued regression and study online learnability in full generality. In particular, we allow $\mathcal{Z} \neq \mathcal{Y}$ and $\ell: \mathcal{Y} \times \mathcal{Z} \rightarrow \mathbb{R}_{\geq0}$ to be any bounded loss function. Such a mismatch in $\mathcal{Z}$ and $\mathcal{Y}$ is natural, for example, in online multilabel ranking where the learner predicts a permutation but only receives a binary relevance score vector as feedback \citep{learning_to_rank, raman2023learnability}. Another example is list online classification, where $\mathcal{Z} = \{S: S\subset \mathcal{Y}, |S| \leq k\}$ and $\ell(y, z) = \mathbbm{1}\{y \notin z\}$ \citep{moran2023list}.
% One useful tool for providing \textit{sufficient} conditions for the online learnability of a tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$ is the \textit{sequential} Rademacher complexity of the loss class $\ell \circ \mathcal{H}$ \citep{rakhlin2015online}. Specifically, \cite{rakhlin2015online} show that if the sequential Rademacher complexity of the loss class $\ell \circ \mathcal{H}$ is sublinear in the time horizon $T$, then the tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$ is online learnable. However, \cite{hanneke2023multiclass} show that even for online multiclass classification, sublinear sequential Rademacher complexity is not \textit{necessary} for online learnability. Accordingly, in this work, we pose the following question.
% \begin{guidingquestion}{}{}
% Given any tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$, what are necessary and sufficient conditions for $\mathcal{H}$ to be online learnable with respect to $\ell$?
% \end{guidingquestion}
% Beyond a \textit{qualitative} characterization of the online learnability of a tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$, we are also interested in obtaining a \textit{quantitative} characterization of online learnability. For example, in classification and scalar-valued regression, combinatorial dimensions like the Ldim and seq-fat also provide a sharp quantitative characterization of online learnability. Accordingly, this motivates the following question:
% \begin{guidingquestion}{}{}
% Given any tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$, is there a \textit{combinatorial dimension} that provides a tight quantitative characterization of the online learnability of $\mathcal{H}$ with respect to $\ell$?
% \end{guidingquestion}
Guided by these two questions, we make the following contributions.
\begin{contributions*}{}{}
\begin{enumerate}
\item We give a new scale-sensitive dimension, named the sequential Minimax dimension (SMdim), that gives a tight quantitative characterization of online learnability for any tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$.
\item Using SMdim, we construct a generic online learner for any tuple $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$.
\item We show that the SMdim reduces to existing combinatorial dimensions in the case where $\mathcal{Z} = \mathcal{Y}$, like the Ldim and seq-fat, as well as the case where $\mathcal{Z} \neq \mathcal{Y}$, like the $(k+1)$-Ldim from \cite{moran2023list} and MSdim from \cite{raman2023online}.
\item We give the first quantitative characterization of online learnability for two natural learning settings: vector-valued regression and multilabel classification.
\end{enumerate}
\end{contributions*}
% We note that the combinatorial dimension we give is a function of the \textit{entire} tuple, $(\mathcal{X}, \mathcal{Y}, \mathcal{Z}, \mathcal{H}, \ell)$. This does not contradict the recent results by [cite], since [need to verify].
\subsection{Related Works}
Characterizing learnability in terms of complexity measures has a long rich history in statistical learning theory, originating from the seminal work of \cite{vapnik1971uniform}. In online learning, \cite{Littlestone1987LearningQW} showed that a combinatorial parameter, later named as Littlestone dimension, provides a quantitative characterization of online binary classification in the realizable setting. Twenty-two years later, \cite{ben2009agnostic} proved that the Littlestone dimension also provides a tight quantitative characterization of online binary classification in the agnostic setting. \cite{DanielyERMprinciple} generalized the Littlestone dimension to multiclass classification and showed that it fully characterizes online learnability when the label space is finite. Recently, \cite{hanneke2023multiclass} proved that the multiclass extension of the Littlestone dimension characterizes multiclass learnability under the $0$-$1$ loss even when the label space is unbounded. In a parallel line of work, \cite{rakhlin2015online, rakhlin2015sequential} defined the sequential fat-shattering dimension and showed that it tightly characterizes the online learnability of scalar-valued regression with respect to the absolute value loss. In addition, they defined a general complexity measure called the sequential Rademacher complexity and proved that it upper bounds the minimax value of any online game. In a similar spirit, we define a complexity measure that upper and lower bounds the minimax value of any online game.
The proof techniques in online learning are generally constructive and result in beautiful algorithms such as Follow The (Regularized) Leader, Hedge, Multiplicative Weights, Online Gradient Descent, and so forth. In online binary classification, \cite{Littlestone1987LearningQW} proposed the Standard Optimal Algorithm and proved its optimality in the realizable setting. \cite{DanielyERMprinciple} and \cite{rakhlin2015online} generalize this algorithm to multiclass classification and scalar-valued regression respectively. The ideas of the Standard Optimal Algorithm is foundational in online learning and still appears in more recent works by \cite{moran2023list, filmus2023optimal} and \cite{raman2023online}. A common theme in these variants of the Standard Optimal Algorithm is their use of combinatorial dimensions to make predictions. Similarly, \cite{rakhlin_relax_n_randomize} use the sequential Rademacher complexity to directly construct a generic online learner in the agnostic setting. However, their online learner requires the sequential Rademacher complexity of the loss class to be sublinear in $T$, and thus does not work for arbitrary tuples $(\Xcal, \Ycal, \Zcal, \Hcal, \ell)$. Closing this gap, we define a new scale-sensitive dimension, named the sequential Minimax dimension, and use it to give a generic online learner for any tuple $(\Xcal, \Ycal, \Zcal, \Hcal, \ell)$.
\section{Preliminaries}
\subsection{Notation}
Let $\Xcal$ denote the instance space, $\Ycal$ denote the label space, and $\Zcal$ denote the prediction space. For a sigma algebra $\sigma(\Zcal)$ on the prediction space $\Zcal$, define $\Pi(\Zcal)$ to be the set of all distributions on $(\Zcal, \sigma(\Zcal))$. For any set $S \in \sigma(\mathcal{Z})$, let $S^c$ denote its complement. Let $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$ denote an arbitrary hypothesis class consisting of predictors $h: \mathcal{X} \rightarrow \mathcal{Z}$ that maps an instance to a prediction. Given any prediction $z \in \Zcal$ and a label $y \in \Ycal$, we consider a loss function $\ell : \Ycal \times \Zcal \to \reals_{\geq 0}$. We put no restrictions on the loss function $\ell$, except that it is bounded, $\sup_{y, z} \ell(y, z) \leq c$ for some $c \in \reals_{>0}$. In particular, the loss can asymmetric, and therefore we reserve the first argument for the label and the second argument for the prediction. Finally, we let $[N]$ denote the set of positive integers $\{1, 2, \ldots, N\}$.
\subsection{Online Learning}
In the online setting, an adversary plays a sequential game with the learner over $T$ rounds. In each round $t \in [T]$, the adversary selects a labeled instance $(x_t, y_t) \in \mathcal{X} \times \Ycal$ and reveals $x_t$ to the learner. The learner picks a probability measure $\mu_t \in \Pi(\mathcal{Z})$ and then makes a randomized prediction $z_t \sim \mu_t$. Finally, the adversary reveals the feedback $y_t$, and the learner suffers the loss $\ell(y_t, z_t)$.
% Given a hypothesis class $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$, the goal of the learner is to output randomized predictions $z_t$ such that its expected cumulative loss is close to the smallest possible cumulative loss over hypotheses in $\mathcal{H}$.
As is standard in online learning, we consider an adaptive adversary that can use learner's measure $\mu_t$ to select the label $y_t$, but not its prediction $z_t$. To formally define an online learning algorithm, we follow the convention in Chapter 4 of \cite{cesa2006prediction}.
\begin{definition}{Online Learning Algorithm }{}
An online learning algorithm is a deterministic mapping $\Acal : (\Xcal \times \Ycal )^{\star} \times \Xcal \to \Pi(\Zcal)$ that maps past examples and the newly revealed instance $x \in \Xcal$ to a probability measure $\mu \in \Pi(\Zcal)$. The learner then randomly samples $z \sim \mu$ to make a prediction.
\end{definition}
\noindent Although $\Acal$ is a deterministic mapping, the prediction $z \sim \mu$ is random. Restricting the range of $\Acal$ to be the set of Dirac measures on $\Zcal$ yields a deterministic online learner. When the context is clear, with a slight abuse of notation, we use $\Acal(x)$ to denote the random sample $z$ drawn from the distribution that $\Acal$ outputs.
We say that $\mathcal{H}$ is online learnable with respect to $\ell$ if there exists an online learning algorithm such that for any sequence of labeled examples, $(x_1, y_1), ..., (x_T, y_T)$, the difference in expected cumulative loss between its predictions and the predictions of the best-fixed hypothesis in $\mathcal{H}$ is small. We formalize this notion in the following definition.
\begin{definition} {Online Agnostic Learnability}{agnOL}
A hypothesis class $\Hcal \subseteq \Zcal^{\Xcal}$ is online learnable if there exists an algorithm $\mathcal{A}$ such that for any adaptively chosen sequence of labeled examples $(x_t, y_t) \in \mathcal{X} \times \mathcal{Y}$, the algorithm outputs $\mathcal{A}(x_t) \in \mathcal{Z}$ at every iteration $t \in [T]$ such that
$$\mathbb{E}\left[\sum_{t=1}^T \ell(y_t, \Acal(x_t)) - \inf_{h \in \mathcal{H}}\sum_{t=1}^T \ell(y_t, h(x_t))\right] \leq R_{\mathcal{A}}(T) $$
where the expectation is taken with respect to the randomness of $\mathcal{A}$ and that of the possibly adaptive adversary, and $R_{\mathcal{A}}(T): \mathbb{N} \rightarrow \mathbb{R}^+$ is a non-decreasing, sub-linear function of $T$ referred to as the regret of $\Acal$.
\end{definition}
\noindent Note that the existence of an online learner with sublinear regret is equivalent to requiring $\limsup_{T \to \infty} \frac{\mathcal{V}_{T}(\Hcal, \ell)}{T} \leq 0$.
A sequence of labeled examples $(x_1, y_1), ..., (x_T, y_T)$ is said to be \textit{realizable} by $\Hcal$ if there exists a hypothesis $h^{\star} \in \Hcal$ such that $\ell(y_t, h^{\star}(x_t)) = 0$ for all $t \in [T]$.
In such case, we have $ \inf_{h \in \Hcal} \sum_{t=1}^T \ell(y_t, h(x_t)) = 0$, and the setting is typically referred to as the realizable setting.
\section{Combinatorial Dimensions in Online Learning}
Combinatorial dimensions play an important role in understanding when a hypothesis class $\mathcal{H} \subseteq \mathcal{Y}^{\mathcal{X}}$ is online learnable. As discussed in the introduction, two important examples are the Littlestone and the sequential fat-shattering dimension, which characterize the online learnability of classification and scalar-valued regression respectively. Common to both the Littlestone and sequential fat-shattering dimension is their definition in terms of \textit{trees}, a basic combinatorial object that captures the temporal dependence in online learning. Accordingly, we start this section by formally defining the notion of a tree. Our notation is borrowed from \cite{rakhlin2015online} and \cite{raman2023online}.
Given an instance space $\mathcal{X}$ and a (potentially uncountable) set of objects $\mathcal{M}$, a $\mathcal{X}$-valued, $\mathcal{M}$-ary tree $\mathcal{T}$ of depth $T$ is a complete rooted tree such that (1) each internal node $v$ is labeled by an instance $x \in \mathcal{X}$ and (2) for every internal node $v$ and object $m \in \mathcal{M}$, there is an outgoing edge $e^m_{v}$ indexed by $m$. Such a tree can be identified by a sequence $(\mathcal{T}_1, ..., \mathcal{T}_T)$ of labeling functions $\mathcal{T}_t:\mathcal{M}^{t-1} \rightarrow \mathcal{X}$ which provide the labels for each internal node. A path of length $T$ is given be a sequence of objects $m = (m_1,..., m_T) \in \mathcal{M}^T$. Then, $\mathcal{T}_t(m_1, ..., m_{t-1})$ gives the label of the node by following the path $(m_1, ..., m_{t-1})$ starting from the root node, going down the edges indexed by the $m_t$'s. We let $\mathcal{T}_1 \in \mathcal{X}$ denote the instance labeling the root node. For brevity, we define $m_{<t} = (m_1, ..., m_{t-1})$ and therefore write $\mathcal{T}_t(m_1, ..., m_{t-1}) = \mathcal{T}_t(m_{<t})$. Analogously, we let $m_{\leq t} = (m_1, ..., m_{t})$.
Often, it is useful to label the edges of a tree with some \textit{auxiliary} information. Given a $\mathcal{X}$-valued, $\mathcal{M}$-ary tree $\mathcal{T}$ of depth $T$ and a (potentially uncountable) set of objects $\mathcal{N}$, we can formally label the edges of $\mathcal{T}$ using objects in $\mathcal{N}$ by considering a sequence $(f_1, ..., f_T)$ of edge-labeling functions $f_t: \mathcal{M}^{t} \rightarrow \mathcal{N}$. For each depth $t \in [T]$, the function $f_t$ takes as input a path $m_{\leq t}$ of length $t$ and outputs an object in $\mathcal{N}$. Accordingly, we can think of the object $f_t(m_{\leq t})$ as labeling the edge indexed by $m_t$ after following the path $m_{< t}$ down the tree. We now use this notation to rigorously define existing combinatorial dimensions in online learning.
\subsection{Existing Combinatorial Dimensions}
We start of with the Littlestone dimension, first proposed by \cite{Littlestone1987LearningQW} for binary classification, and then generalized by \cite{DanielyERMprinciple}
to multiclass classification. In classification, we take $\mathcal{Z} = \mathcal{Y}$ and $\ell(y, z) = \mathbbm{1}\{y \neq z\}$.
\begin{definition} {Littlestone dimension}{ldim}
Let $\mathcal{T}$ be a complete, $\mathcal{X}$-valued, $\{\pm 1\}$-ary tree of depth $d$. The tree $\mathcal{T}$ is shattered by $\mathcal{H} \subseteq \Zcal^{\Xcal}$ if there exists a sequence $(f_1, ..., f_d)$ of edge-labeling functions $f_t: \{\pm 1\}^{t} \rightarrow \mathcal{Y}$ such that for every path $\sigma = (\sigma_1, ..., \sigma_d) \in \{\pm 1\}^d$, there exists a hypothesis $h_{\sigma} \in \mathcal{H}$ such that for all $t \in [d]$, $h_{\sigma}(\mathcal{T}_t(\sigma_{<t})) = f_t(\sigma_{\leq t})$ and $f_t((\sigma_{< t}, -1)) \neq f_t((\sigma_{< t}, +1))$. The Littlestone dimension of $\mathcal{H}$, denoted $\text{Ldim}(\mathcal{H})$, is the maximal depth of a tree $\mathcal{T}$ that is shattered by $\mathcal{H}$. If there exists shattered trees of arbitrarily large depth, we say $\text{Ldim}(\mathcal{H}) = \infty$.
\end{definition}
It is well known that the Littlestone dimension characterizes online learnability with respect to the 0-1 loss \citep{Littlestone1987LearningQW, ben2009agnostic, DanielyERMprinciple, hanneke2023multiclass}. For online regression, we take $\mathcal{Z} = \mathcal{Y} = [-1, 1]$, $\ell(y, z) = |y - z|$, and define the sequential-fat shattering dimension.
\begin{definition} {Sequential Fat-shattering dimension}{sfat}
Let $\mathcal{T}$ be a complete, $\mathcal{X}$-valued, $\{\pm 1\}$-ary tree of depth $d$ and fix $\gamma \in (0, 1]$. The tree $\mathcal{T}$ is $\gamma$-shattered by $\mathcal{H} \subseteq \mathcal{Z}^{\Xcal}$ if there exists a sequence $(f_1, ..., f_d)$ of edge-labeling functions $f_t: \{\pm 1\}^{t} \rightarrow \mathcal{Y}$ such that for every path $\sigma = (\sigma_1, ..., \sigma_d) \in \{\pm 1\}^d$, there exists a hypothesis $h_{\sigma} \in \mathcal{H}$ such that for all $t \in [d]$, $\sigma_t(h_{\sigma}(\mathcal{T}_t(\sigma_{<t})) - f_t(\sigma_{\leq t})) \geq \gamma$ and $f_t((\sigma_{< t}, -1)) = f_t((\sigma_{< t}, +1))$. The sequential fat-shattering dimension of $\mathcal{H}$ at scale $\gamma$, denoted $\text{seq-fat}_{\gamma}(\mathcal{H})$, is the maximal depth of a tree $\mathcal{T}$ that is $\gamma$-shattered by $\mathcal{H}$. If there exists $\gamma$-shattered trees of arbitrarily large depth, we say that $\text{seq-fat}_{\gamma}(\mathcal{H}) = \infty$.
\end{definition}
Unlike the Littlestone dimension, the two outgoing edges from every internal node in the sequential fat-shattering tree are labeled by the \textit{same} real number in $\mathcal{Y}$. These scalars labeling the outgoing edges in the sequential fat-shattering tree are commonly referred to as \textit{witnesses} to shattering in literature. \cite{rakhlin2015online} showed that $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$ is online learnable with respect to the absolute value loss if and only if $\text{seq-fat}_{\gamma}(\mathcal{H})$ is finite at every scale $\gamma > 0$.
We now move to the more recent setting of list online classification \citep{moran2023list}. Here, we take $\mathcal{Z} = \{S: S\subset \mathcal{Y}, |S| \leq k\}$ and $\ell(y, z) = \mathbbm{1}\{y \notin z\}$. That is, the learner is allowed to play a set of labels of size at most $k$ and is penalized only if the true label is not in this set. \cite{moran2023list} show that the $(k+1)$- Littlestone dimension, characterizes online learnability of a hypothesis class $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$.
\begin{definition}{$(k+1)\text{-Littlestone dimension}$}{kldim}
Let $\mathcal{T}$ be a complete, $\mathcal{X}$-valued, $[k+1]$-ary tree of depth $d$. The tree $\mathcal{T}$ is shattered by $\mathcal{H} \subseteq \mathcal{Z}^{\Xcal}$ if there exists a sequence $(f_1, ..., f_d)$ of edge-labeling functions $f_t: [k+1]^{t} \rightarrow \mathcal{Y}$ such that for every path $p = (p_1, ..., p_d) \in [k+1]^d$, there exists a hypothesis $h_{p} \in \mathcal{H}$ such that for all $t \in [d]$, $f_t(p_{\leq t})) \in h_{\sigma}(\mathcal{T}_t(\sigma_{<t}))$ and for all distinct $i, j \in [k+1]$, $f_t((p_{< t}, i)) \neq f_t((p_{< t}, j))$. The $(k+1)$-Littlestone dimension of $\mathcal{H}$ denoted $\text{Ldim}_{k+1}(\mathcal{H})$, is the maximal depth of a tree $\mathcal{T}$ that is shattered by $\mathcal{H}$. If there exists shattered trees of arbitrarily large depth, we say that $\text{Ldim}_{k+1}(\mathcal{H}) = \infty$.
\end{definition}
At a high-level, the $(k+1)$-Littlestone tree is a $[k+1]$-ary tree where every internal node has $k+1$ outgoing edges indexed by natural numbers in $[k+1]$ and \textit{labeled} by different labels in $\mathcal{Y}$. In this way, when $k = 1$, the $(k+1)$-Littlestone dimension reduces to the original Littlestone dimension. Finally, we consider the ``flip" of list online classification where now $\mathcal{Y} \subset \sigma(\mathcal{Z})$ is some collection of measurable subsets of $\mathcal{Z}$ and $\ell(y, z) = \mathbbm{1}\{z \notin y\}$. In this setting, \cite{raman2023online} show that the Measure shattering dimension characterizes online learnability of a hypothesis class $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$ under set-valued feedback.
\begin{definition} {Measure shattering dimension}{}
Let $\mathcal{T}$ be a complete $\mathcal{X}$-valued, $\Pi(\mathcal{Z})$-ary tree of depth $d$, and fix $\gamma \in (0,1] $. The tree $\mathcal{T}$ is $\gamma$-shattered by $\mathcal{H} \subseteq \Zcal^{\Xcal}$ if there exists a sequence $(f_1, ..., f_d)$ of edge-labeling set-valued functions $f_t: \Pi(\mathcal{Z})^{t} \rightarrow \mathcal{Y}$ such that for every path $\mu = (\mu_1, ..., \mu_d) \in \Pi(\mathcal{Z})^d$, there exists a hypothesis $h_{\mu} \in \mathcal{H}$ such that for all $t \in [d]$, $h_{\mu}(\mathcal{T}_t(\mu_{<t})) \in f_t(\mu_{\leq t})$ and $\mu_t(f_t(\mu_{\leq t})) \leq 1 - \gamma$. The Measure Shattering dimension of $\mathcal{H}$ at scale $\gamma$, denoted $\text{MSdim}_{\gamma}(\mathcal{H}, \mathcal{Y})$, is the maximal depth of a tree $\mathcal{T}$ that is $\gamma$-shattered by $\mathcal{H}$. If there exists $\gamma$-shattered trees of arbitrarily large depth, we say $\text{MSdim}_{\gamma}(\mathcal{H}, \mathcal{Y}) = \infty$.
\end{definition}
Note that unlike the $(k+1)$-Littlestone dimension, the Measure shattering dimension is a scale-sensitive dimension and has infinite-width. \cite{raman2023online} show that the MSdim also characterizes online learnability for a wide range of practical problems, like online multilabel ranking, online multilabel classification with $\varepsilon$-sensitive losses, and online interval learning.
\subsection{A New Combinatorial Dimension}
In this work, we are interested in a combinatorial dimension that characterizes the online learnability for an arbitrary tuple $(\Xcal, \Ycal, \Zcal, \Hcal, \ell)$. Inspired by the minimax value of the game, we define a dimension that provides an adversary with a strategy against every possible move of the learner. Since the learner plays measures in $\Pi(\mathcal{Z})$, this amounts to defining a tree where each internal node has an outgoing edge for every measure in $\Pi(\mathcal{Z})$ labeled by an element of $\mathcal{Y}$. For any prediction $\mu \in \Pi(\mathcal{Z})$ of the learner, the label on the edge corresponding to $\mu$ gives the element $y \in \mathcal{Y}$ that the adversary should play. Definition \ref{def:SMdim} makes this precise.
\begin{definition} {Sequential Minimax dimension}{SMdim}
Let $\mathcal{T}$ be a complete $\mathcal{X}$-valued, $\Pi(\mathcal{Z})$-ary tree of depth $d$, and fix $\gamma > 0$. The tree $\mathcal{T}$ is $\gamma$-shattered by $\mathcal{H} \subseteq \Zcal^{\Xcal}$ with respect to $\ell: \mathcal{Y} \times \mathcal{Z} \rightarrow \mathbb{R}_{\geq0}$ if there exists a sequence $(f_1, ..., f_d)$ of edge-labeling functions $f_t: \Pi(\mathcal{Z})^{t} \rightarrow \mathcal{Y}$ such that for every path $\mu = (\mu_1, ..., \mu_d) \in \Pi(\mathcal{Z})^d$, there exists a hypothesis $h_{\mu} \in \mathcal{H}$ such that for all $t \in [d]$, $\mathbb{E}_{z \sim \mu_t}\left[\ell(f_t(\mu_{\leq t}), z) \right] \geq \ell(f_t(\mu_{\leq t}), h_{\mu}(\mathcal{T}_t(\mu_{<t}))) + \gamma.$ The sequential Minimax dimension of $\mathcal{H}$ at scale $\gamma$, denoted $\text{SMdim}_{\gamma}(\mathcal{H}, \ell)$, is the maximal depth of a tree $\mathcal{T}$ that is $\gamma$-shattered by $\mathcal{H}$. If there exists $\gamma$-shattered trees of arbitrarily large depth, we say $\text{SMdim}_{\gamma}(\mathcal{H}, \ell) = \infty$. Analogously, we can define $ \text{SMdim}_{0}(\mathcal{H}, \ell)$ by requiring strict inequality, $ \mathbb{E}_{z \sim \mu}\left[\ell(f_t(\mu_{\leq t}), z) \right] > \ell(f_t(\mu_{\leq t}), h_{\mu}(\mathcal{T}_t(\mu_{<t})))$.
\end{definition}
One can also view the SMdim as a generalization of the MSdim proposed in \cite{raman2023online}. It is not too hard to see that the finiteness of SMdim at every scale is an obvious necessary condition. Often in mathematics, the obvious necessary condition also turns out to be sufficient. It is no different here as, in Section \ref{sec:agnostic}, we show that the finiteness of SMdim at every scale is also sufficient for online learnability. We note that the SMdim is a function of both the hypothesis class $\mathcal{H}$ and the loss function $\ell$. However, when it is clear from context, we drop the dependence of $\ell$ and only write $\text{SMdim}_{\gamma}(\mathcal{H})$. As with most scale-sensitive dimensions, SMdim has a monotonicity property, namely, $\text{SMdim}_{\gamma_1}(\mathcal{H}) \leq \text{SMdim}_{\gamma_2}(\mathcal{H}) $ for any $\gamma_2 \leq \gamma_1$. In Section \ref{sec:relations}, we show that the SMdim reduces to all four dimensions defined above.
\section{$\varepsilon_t$-Realizable Setting}
Before we study online learnability with bounded losses in the agnostic setting, we consider a slightly easier setting, which we term the $\varepsilon_t$-realizable setting. In this setting, an adversary plays a sequential game with the learner over $T$ rounds. In each round $t \in [T]$, the adversary selects a \textit{thresholded} labeled instance $(x_t, (y_t, \varepsilon_t)) \in \mathcal{X} \times (\mathcal{Y} \times [0, c])$ and reveals $x_t$ to the learner. The learner selects a measure $\mu_t \in \Pi(\mathcal{Z})$ and makes a randomized prediction $z_t \sim \mu_t$. Finally, the adversary reveals both the true label $y_t$ and the threshold $\varepsilon_t$ and the learner suffers the loss $\ell(y_t, z_t)$. A sequence of thresholded labeled examples $\{(x_t, (y_t, \varepsilon_t))\}_{t=1}^T$ is called $\varepsilon_t$-realizable if there exists a hypothesis $h^* \in \mathcal{H}$ such that $\ell(y_t, h^*(x_t)) \leq \varepsilon_t$ for all $t \in [T]$. Given any $\varepsilon_t$-realizable stream, the goal of the learner is to output predictions such that the \textit{difference} $\mathbb{E}\left[\sum_{t=1}^T \ell(y_t, z_t) \right] - \sum_{t=1}^T \varepsilon_t$ is sublinear in $T$. We can think of the thresholds $\varepsilon_t$ as the adversary additionally revealing the loss that the best fixed hypothesis in hindsight suffers on the labeled instance $(x_t, y_t)$. This intuition is critical to our construction of an agnostic learner in Section \ref{sec:agnostic}. Note that if it is guaranteed ahead of time that $\varepsilon_t = 0$ for all $t \in [T]$, then this setting boils down to the standard realizable setting. Theorem \ref{thm:real} shows that the finiteness of $\text{SMdim}_{\gamma}(\mathcal{H})$ at every scale $\gamma > 0$ is sufficient for $\mathcal{H}$ to be online learnable in $\varepsilon_t$-realizable setting.
\begin{theorem}{$\varepsilon_t$-Realizable Learnability}{real}
For any tuple $(\Xcal, \Ycal, \Zcal, \Hcal, \ell)$,
there exists a randomized online learner whose expected cumulative loss on any $\varepsilon_t$-realizable stream $\{(x_t, (y_t, \varepsilon_t))\}_{t=1}^T$ is at most
$$ \sum_{t=1}^T\varepsilon_t + \inf_{\gamma > 0 }\, \Big\{\gamma T
+ c \cdot \text{SMdim}_{\gamma}(\mathcal{H}) \Big\} .$$
\end{theorem}
The key idea to prove Theorem \ref{thm:real} is to modify the Randomized Standard Optimal Algorithm from \cite{raman2023online} to account for the additional information $\varepsilon_t$ that the adversary reveals in each round. Algorithm \ref{alg:rand_SOA} makes this idea more precise.
\begin{figure}[H]
\begin{tcolorbox} [colback=white!5,]
\begin{center}
{\bf Minimax Randomized Standard Optimal Algorithm }
\end{center}
\textbf{Input:} $\mathcal{H}$, Target accuracy $\gamma > 0$
\textbf{Initialize:} $V_0 = \mathcal{H}$
For $t=1,\ldots,T:$
\begin{enumerate}
\item Receive unlabeled example $x_t \in \mathcal{X}$.
\item For all $(y, \varepsilon) \in \mathcal{Y} \times [0, c]$, define $V_{t-1}(y, \varepsilon) := \{h \in V_{t-1} \, \mid \, \ell(y, h(x_t)) \leq \varepsilon\}$.
\item Let $\mathcal{C}_t:= \{(y, \varepsilon) \in \mathcal{Y} \times [0, c]: \exists h \in V_{t-1} \text{ such that } \ell(y, h(x_t)) \leq \varepsilon\}.$
\item If $\text{SMdim}_{\gamma}(V_{t-1}) = 0$, let $\mu_t \in \Pi(\Zcal)$ be such that for all $(y, \varepsilon) \in \mathcal{C}_t$ we have $\mathbb{E}_{z \sim \mu_t}\left[\ell(y, z)\right] < \varepsilon + \gamma$
\item Else, compute
\[\mu_t = \argmin_{\mu \in \Pi(\Zcal)} \,\max_{\substack{(y, \varepsilon) \in \mathcal{Y} \times [0, c] \\ \mathbb{E}_{z \sim \mu}\left[\ell(y, z)\right] \geq \varepsilon + \gamma}} \, \text{SMdim}_{\gamma}(V_{t-1}(y, \varepsilon)).\]
\item Predict $z_t \sim \mu_t.$
\item Receive feedback $(y_t, \varepsilon_t)$.
\item Update $V_t = V_{t-1}(y_t, \varepsilon_t).$
\end{enumerate}
\end{tcolorbox}
\caption{Minimax Randomized Standard Optimal Algorithm (MRSOA)}
\label{alg:rand_SOA}
\end{figure}
We now prove that the expected cumulative loss of Algorithm \ref{alg:rand_SOA} satisfies the upperbound given in Theorem \ref{thm:real}.
\begin{proof}
We show that given any target accuracy $\gamma > 0$, the expected cumulative loss of Algorithm \ref{alg:rand_SOA} is at most $\sum_{t=1}^T \varepsilon_t + c \, d_{\gamma} + \gamma T$ on any $\varepsilon_t$-realizable stream, where $d_{\gamma} = \text{SMdim}_{\gamma}(\mathcal{H})$. In fact, we show that Algorithm \ref{alg:rand_SOA} achieves an even \textit{stronger} guarantee, namely that on any $\varepsilon_t$-realizable sequence $\{(x_t, (y_t, \varepsilon_t))\}_{t=1}^T$, Algorithm \ref{alg:rand_SOA} computes distributions $\mu_t \in \Pi(\mathcal{Z})$ such that
\begin{equation}\label{eq:randSOA}
\sum_{t=1}^T \mathbbm{1}\{\mathbb{E}_{z \sim \mu_t}\left[\ell(y_t, z)\right] \geq \gamma + \varepsilon_t\} \leq d_{\gamma}.
\end{equation}
From here, it follows that $\mathbbm{E}\left[\sum_{t=1}^T \ell(y_t, z_t) \right] \leq c \, d_{\gamma} + \gamma T + \sum_{t=1}^T \varepsilon_t.$ To see this, observe that
\begin{align*}
\mathbbm{E}\left[\sum_{t=1}^T \ell(y_t, z_t) \right] &= \sum_{t=1}^T \mathbbm{E}_{z_t \sim \mu_t}\left[\ell(y_t, z_t) \right]\\
&= \sum_{t=1}^T \mathbbm{E}_{z_t \sim \mu_t}\left[\ell(y_t, z_t)\right]\mathbbm{1}\{\mathbb{E}_{z_t \sim \mu_t}\left[\ell(y_t, z_t)\right] \geq \gamma + \varepsilon_t \} + \mathbbm{E}_{z_t \sim \mu_t}\left[\ell(y_t, z_t)\right]\mathbbm{1}\{\mathbb{E}_{z_t \sim \mu_t}\left[\ell(y_t, z_t)\right] < \gamma + \varepsilon_t \}\\
&\leq c \sum_{t=1}^T\mathbbm{1}\{\mathbb{E}_{z_t \sim \mu_t}\left[\ell(y_t, z_t)\right] \geq \gamma + \varepsilon_t \} + \gamma T + \sum_{t=1}^T \varepsilon_t \\
&\leq c\,d_{\gamma} + \gamma T + \sum_{t=1}^T \varepsilon_t.
\end{align*}
We now show that the outputs of Algorithm \ref{alg:rand_SOA} satisfy Equation \eqref{eq:randSOA}. It suffices to show that (1) on any round where $\mathbb{E}_{z_t \sim \mu_t}\left[\ell(y_t, z_t) \right] \geq \gamma + \varepsilon_t$ and $\text{SMdim}_{\gamma}(V_{t-1}) > 0$, we have $\text{SMdim}_{\gamma}(V_t) \leq \text{SMdim}_{\gamma}(V_{t-1}) - 1$, and (2) if $\text{SMdim}_{\gamma}(V_{t-1}) = 0$ there always exists distribution $\mu_t \in \Pi(\mathcal{Z})$ such that $\mathbbm{E}_{z_t \sim \mu_t}\left[\ell(y_t, z_t)\right] < \gamma + \varepsilon_t.$
Let $t \in [T]$ be a round where $\mathbb{E}_{z_t \sim \mu_t}\left[\ell(y_t, z_t) \right] \geq \gamma + \varepsilon_t$ and $\text{SMdim}_{\gamma}(V_{t-1}) > 0$. For the sake contradiction, suppose that $\text{SMdim}_{\gamma}(V_t) = \text{SMdim}_{\gamma}(V_{t-1}) = d$. Then, by the min-max computation in line (4) of Algorithm \ref{alg:rand_SOA}, for every measure $\mu \in \Pi(\mathcal{Z})$, there exists a pair $(y_{\mu}, \varepsilon_{\mu}) \in \mathcal{Y} \times [0, 1]$ such that $\mathbb{E}_{z \sim \mu}\left[\ell(y_{\mu}, z)\right] \geq \varepsilon_{\mu} + \gamma $ and $\text{SMdim}_{\gamma}(V_{t-1}(y_{\mu}, \varepsilon_{\mu})) = d$. Now construct a tree $\mathcal{T}$ with $x_t$ labeling the root node. For each measure $\mu \in \Pi(\mathcal{Z})$, construct an outgoing edge from $x_t$ indexed by $\mu$ and labeled by $y_{\mu}$. Append the tree of depth $d$ associated with the version space $V_{t-1}(y_{\mu}, \varepsilon_{\mu})$ to the edge indexed by $\mu$. Note that the depth of $\mathcal{T}$ must be $d+1$. Furthermore, observe that for every hypothesis $h \in V_{t-1}(y_{\mu}, \varepsilon_{\mu})$, we have that $\mathbb{E}_{z \sim \mu}\left[\ell(y_{\mu}, z) \right] \geq \ell(y_{\mu}, h(x_t)) + \gamma$, matching the shattering condition in Definition \ref{def:SMdim}. Therefore, by definition of SMdim, we have that $\text{SMdim}_{\gamma}(V_{t-1}) = d+1$, a contradiction. Thus, it must be the case that $\text{SMdim}_{\gamma}(V_t) \leq \text{SMdim}_{\gamma}(V_{t-1}) - 1$.
Now, suppose $t \in [T]$ is a round such that $\text{SMdim}_{\gamma}(V_{t-1}) = 0$. We show that there always exist a distribution $\mu_t \in \Pi(\mathcal{Z})$ such that for all $(y, \varepsilon) \in \mathcal{C}_t$ , we have $\mathbb{E}_{z_t \sim \mu_t}\left[\ell(y, z_t) \right] < \gamma + \varepsilon$. Since we are in the $\varepsilon_t$-realizable setting, it must be the case that $(y_t, \varepsilon_t) \in \mathcal{C}_t$. To see why such a $\mu_t$ must exist, suppose for the sake of contradiction that it does not exist. Then, for all $\mu \in \Pi(\mathcal{Z})$, there exists a pair $(y_{\mu}, \varepsilon_{\mu}) \in \mathcal{C}_t$ such that $\mathbb{E}_{z \sim \mu}\left[\ell(y_{\mu}, z)\right] \geq \gamma + \varepsilon_{\mu} $. As before, consider a tree with root node labeled by $x_t$. For each measure $\mu \in \Pi(\mathcal{Z})$, construct an outgoing edge from $x_t$ indexed by $\mu$ and labeled by $y_{\mu}$. Since $(y_{\mu}, \varepsilon_{\mu}) \in \mathcal{C}_t$, there exists a hypothesis $h_{\mu} \in V_{t-1}$ such that $\ell(y_{\mu}, h_{\mu}(x_t)) \leq \varepsilon_{\mu}$. Therefore, we have $\mathbb{E}_{z \sim \mu}\left[\ell(y_{\mu}, z) \right] \geq \ell(y_{\mu}, h_{\mu}(x_t)) + \gamma$ . By definition of SMdim, this implies that $\text{SMdim}_{\gamma}(V_{t-1}) \geq 1$, which contradicts the fact that $\text{SMdim}_{\gamma}(V_{t-1}) = 0$. Thus, there must be a distribution $\mu_t \in \Pi(\mathcal{Z})$ such that for for all $(y, \varepsilon) \in \mathcal{C}_t$ , we have $\mathbb{E}_{z \sim \mu_t}\left[\ell(y, z) \right] < \gamma + \varepsilon$. Since this is precisely the distribution that Algorithm \ref{alg:rand_SOA} plays in step (3) and since $\text{SMdim}_{\gamma}(V_{t'}) \leq \text{SMdim}_{\gamma}(V_{t-1})$ for all $t' \geq t$, the algorithm no longer suffers expected loss more than $\gamma + \varepsilon_{t'}$. This completes the proof.
\end{proof}
\section{Agnostic Setting}\label{sec:agnostic}
In this section, we go beyond the $\varepsilon_t$-realizable setting and consider the more general agnostic setting where the adversary does not reveal $\varepsilon_t$ at the end of each round. Our main Theorem shows that the finiteness of SMdim at every scale $\gamma > 0$ is both a necessary and sufficient condition for agnostic online learnability.
\begin{theorem}{Agnostic Learnability}{agn}
Consider a tuple $(\Xcal, \Ycal, \Zcal, \Hcal, \ell)$. Then,
\begin{itemize}
\item[(i)] There exists a randomized online learner whose expected regret on any stream is at most
$$\inf_{\gamma > 0 }\, \left\{c\cdot\text{SMdim}_{\gamma}(\mathcal{H}) + \gamma T + 1 + 2c\sqrt{\,\text{SMdim}_{\gamma} (\mathcal{H})\,T \ln(2cT)}\right\},$$
and this upper bound is tight up to logarithmic factors in $T$.
\item[(ii)] Suppose $\sup_{\gamma > 0 } \text{SMdim}_{\gamma}(\mathcal{H}) > 0$. Then, for every randomized online learner, there exists a stream such that the expected regret of the learner is at least
$$\sup_{\gamma > 0 } \, \gamma\, \text{SMdim}_{\gamma}(\mathcal{H})$$
and this lower bound cannot be improved in general.
\end{itemize}
\end{theorem}
\noindent The condition $\sup_{\gamma > 0 } \text{SMdim}_{\gamma}(\mathcal{H}) > 0$ in (ii) is necessary to get a non-negative lower bound. \cite{raman2023online} provide an example of a tuple $(\Xcal, \Ycal, \Zcal, \Hcal, \ell)$ where $\sup_{\gamma > 0 } \text{SMdim}_{\gamma}(\mathcal{H}) =0 $ but there is an algorithm whose regret has a negative upper bound.
A primary approach to proving online agnostic upperbounds involves defining a set of experts that exactly covers the hypothesis class and then running multiplicative weights using these experts. This technique originated in \cite{ben2009agnostic} for binary classification and was later generalized by \cite{DanielyERMprinciple} to multiclass classification. \cite{DanielyERMprinciple}'s generalization involves simulating all possible labels in $\Ycal$ to update the experts, thus making their upper bound vacuous when $|\Ycal|$ is unbounded. Recently, \cite{hanneke2023multiclass} removed $|\Ycal|$ from the upperbound by (1) constructing an approximate cover of the hypothesis class instead of an exact cover and (2) using the feedback in the stream to update experts rather than simulating all possible labels. Our proof of the upperbound in Theorem \ref{thm:agn} combines the ideas of both \cite{DanielyERMprinciple} and \cite{hanneke2023multiclass}. In particular, following \cite{hanneke2023multiclass}, we construct an approximate cover of the hypothesis class but follow \cite{DanielyERMprinciple} in simulating all possible \textit{loss values}.
\begin{proof}(of (i) in Theorem \ref{thm:agn})
Let $(x_1, y_1), \ldots, (x_T, y_T) $ be the data stream. Here, we only consider an oblivious adversary since a standard reduction (Chapter 4 in \cite{cesa2006prediction}) allows us to convert oblivious regret bounds to adaptive regret bounds. Let $h^{\star} = \argmin_{h \in \mathcal{H}} \sum_{t=1}^T \ell(y_t, h(x_t))$ be an optimal function in hind-sight. For a target accuracy $\gamma > 0$, let $d_{\gamma} = \text{SMdim}_{\gamma}(\Hcal)$. Given time horizon $T$, let $L_T = \{L \subset [T]; |L| \leq d_{\gamma}\}$ denote the set of all possible subsets of $[T]$ with size at most $d_{\gamma}$. For $\alpha \in [0,1]$, let $\{0, \alpha, \ldots, \ceil{\frac{c}{\alpha}}\alpha\}$ be an $\alpha$-cover of the loss space $[0,c]$. For every $L \in L_T$, define $\Phi_{L} = \{0, \alpha, \ldots, \ceil{\frac{c}{\alpha}}\alpha\}^L$ to be the set of all functions from $L $ to the $\alpha$-cover of $[0,c]$. Given $L \in L_T$ and $\phi_L \in \Phi_L$, define an expert $E_{L}^{\phi_L}$ such that
\[E_L^{\phi_L}(x_t) := \text{MRSOA}_{\gamma}\Big(x_t \mid \{i, \phi_L(i)\}_{i \in L \cap [t-1]} \Big),\]
where $ \text{MRSOA}_{\gamma}\Big(x_t \mid \{i, \phi_L(i)\}_{i \in L \cap [t-1]} \Big)$ is the prediction of the Minimax Randomized Standard Optimal Algorithm (MRSOA) running at scale $\gamma$ that has updated on thresholded labeled examples $\{(x_i, (y_i, \phi_L(i))\}_{i \in L \cap [t-1]}$.
Let $\mathcal{E} = \bigcup_{L \in L_T} \bigcup_{\phi_L \in \Phi_L} \{E^{\phi_L}_{L}\}$ denote the set of all Experts. Note that $|\mathcal{E}| = \sum_{i=0}^{d_{\gamma}} \left(\frac{2c}{\alpha} \right)^{i}\binom{T}{i} \leq \left(\frac{2cT}{\alpha}\right)^{d_{\gamma}}$. Finally, given our set of experts $\mathcal{E}$, we run the Multiplicative Weights Algorithm (MWA), denoted hereinafter as $\mathcal{A}$, over the stream $(x_1, y_1), ..., (x_T, y_T)$ with a learning rate $\eta = \sqrt{2 \ln(|\mathcal{E}|)/T}$. Let $B$ denote the random variable denoting the randomized prediction of all MRSOA's. Then, conditioned on $B$, Theorem 21.11 of \cite{ShwartzDavid} tells us that
\begin{align*}
\sum_{t=1}^T \mathbb{E}\left[ \ell(y_t, \Acal(x_t)) \mid B\right] &\leq \inf_{E \in \mathcal{E}} \sum_{t=1}^T \ell(y_t, E(x_t)) + c\,\sqrt{2T\ln(|\mathcal{E}|)}\\
&\leq \inf_{E \in \mathcal{E}} \sum_{t=1}^T \ell(y_t, E(x_t)) + c\,\sqrt{2d_{\gamma}T\ln\left(\frac{2cT}{\alpha} \right)},
\end{align*}
where the second inequality follows because $|\mathcal{E}| \leq \left(\frac{2cT}{\alpha}\right)^{d_{\gamma}}$. Taking expectations on both sides, we obtain
\begin{equation}\label{eq:agn_intermediate}
\mathbb{E}\left[\sum_{t=1}^T \ell(y_t, \Acal(x_t)) \right] \leq \expect \left[\inf_{E \in \mathcal{E}} \sum_{t=1}^T \ell(y_t, E(x_t)) \right] + c\,\sqrt{2d_{\gamma}T\ln\left(\frac{2cT}{\alpha} \right)}.
\end{equation}
Next, we show that the expected loss of the optimal expert is at most the loss of $h^{\star}$ plus a sublinear quantity. Define $\varepsilon_t := \ell(y_t, h^{\star}(x_t))$ to be the loss of the optimal hypothesis in hindsight on each round $t$. We use $\varepsilon_t$ to define a notion of mistake and use the mistake bound guarantee provided by Theorem \ref{thm:real}.
Define $\mu_t = \mu$$\text{-MRSOA}_{\gamma}\big(x_t \mid \{i, \phi_L(i)\}_{i \in L \cap [t-1]}\big)$ to be the measure returned by $\text{MRSOA}_{\gamma}$, as described in steps 4 and 5 of Algorithm \ref{alg:rand_SOA}, given that the algorithm has updated on thresholded labeled examples $\{(x_i, (y_i, \phi_L(i)))\}_{i\in L \cap [t-1]}$. We say that $\mu\text{-MRSOA}_{\gamma}$ makes a mistake on round $t$ if $\expect_{z_t \sim \mu_t}\left[\ell(y_t, z_t) \right] \geq \ceil*{\frac{\varepsilon_t}{\alpha}}\alpha + \gamma $. Because $ \ceil*{\frac{\varepsilon_t}{\alpha}}\alpha \geq \varepsilon_t$, the entire stream $ (x_1,( y_1, \ceil*{\frac{\varepsilon_t}{\alpha}}\alpha)), \ldots, (x_T,( y_T, \ceil*{\frac{\varepsilon_t}{\alpha}}\alpha))$ is $\ceil*{\frac{\varepsilon_t}{\alpha}}\alpha$-realizable. Thus, with this notion of the mistake, Equation \ref{eq:randSOA} tells us that $\text{MRSOA}_{\gamma}$ makes at most $d_{\gamma}$ mistakes on the stream $ (x_1,( y_1, \ceil*{\frac{\varepsilon_t}{\alpha}}\alpha)), \ldots, (x_T,( y_T, \ceil*{\frac{\varepsilon_t}{\alpha}}\alpha))$.
Since $\mu\text{-MRSOA}_{\gamma}$ is a deterministic mapping from the past examples to a probability measure in $\Pi(\Zcal)$, we can procedurally define and select a sequence of time points where $\mu\text{-MRSOA}_{\gamma}$, had it run exactly on this sequence of time points, would make mistakes at each time point. To that end, let
$$t_1 = \min \Big\{\, t \in [T]: \expect_{z_t \sim \mu_t}\left[\ell(y_t, z_t) \right] \geq \ceil*{\frac{\varepsilon_t}{\alpha}}\alpha + \gamma \text{ where } \mu_t=\mu\text{-MRSOA}_{\gamma}\big(x_t|\,\{\}\big) \Big\}$$
be the earliest time point, where a fresh, unupdated copy of $\mu\text{-MRSOA}_{\gamma}$ makes a mistake if it exists. Given $t_1$, we recursively define $t_i$ for $i > 1$ as
$$t_i = \min \left\{\,t > t_{i-1}: \expect_{z_t \sim \mu_t}\left[\ell(y_t, z_t) \right] \geq \ceil*{\frac{\varepsilon_t}{\alpha}}\alpha + \gamma \text{ where } \mu_t =\mu\text{-MRSOA}_{\gamma}\left(x_t \Big|\, \left\{t_j, \ceil*{\frac{\varepsilon_{t_j}}{\alpha}}\alpha\right\}_{j=1}^{i-1}\right) \right\}$$
if it exists. That is, $t_i$ is the earliest timepoint in $[T]$ after $t_{i-1}$ where $\mu\text{-MRSOA}_{\gamma}$ having updated only on the sequence $\{(x_{t_j}, (y_{t_j}, \ceil*{\frac{\varepsilon_{t_j}}{\alpha}}\alpha))\}_{j=1}^{i-1}$ makes a mistake. We stop this process when we reach an iteration where no such time point in $[T]$ can be found where $\mu\text{-MRSOA}_{\gamma} $ makes a mistake.
Using the definitions above, let $t_1, ..., t_N$ denote the sequence of timepoints in $[T]$ selected via this recursive procedure. Define $L^{\star} = \{t_1, ..., t_N\}$ and $\phi_{L^{\star}}$ be the function such that $\phi_{L^{\star}}(t) = \ceil*{\frac{\varepsilon_t}{\alpha}}\alpha$ for each $t \in L^{\star}$. Let $E_{L^{\star}}^{\phi_{L^{\star}}}$ be the expert parametrized by the pair $(L^{\star}, \phi_{L^{\star}})$. The expert $E_{L^{\star}}^{\phi_{L^{\star}}}$ exists because Equation \eqref{eq:randSOA}
implies that $N \leq d_{\gamma}$. By definition of the expert, we have $E_{L^{\star}}^{\phi_{L^{\star}}}(x_t) = \text{MRSOA}_{\gamma}\Big(x_t \mid \{i, \phi_{L^{\star}}(i)\}_{i \in L^{\star} \cap [t-1]} \Big)$ for all $t \in [T]$. Let us define $\mu_t^{\star} = \mu\text{-MRSOA}_{\gamma}\Big(x_t \mid \{i, \phi_{L^{\star}}(i)\}_{i \in L^{\star} \cap [t-1]} \Big)$. Using the guarantee of Algorithm \ref{alg:rand_SOA}, we obtain
\begin{equation*}
\begin{split}
\expect \left[ \sum_{t=1}^T \ell\Big(y_t, E_{L^{\star}}^{\phi_{L^{\star}}}(x_t)\Big) \right] &\leq c\,d_{\gamma}+ \gamma T + \sum_{t=1}^T \, \ceil*{\frac{\varepsilon_t}{\alpha}}\alpha \\
&\leq c\,d_{\gamma} + \gamma T + \sum_{t=1}^T \varepsilon_t + \alpha T\\
&\leq c\,d_{\gamma} + \gamma T + \inf_{h \in \Hcal}\sum_{t=1}^T \ell(y_t, h(x_t)) + \alpha T,
\end{split}
\end{equation*}
where the last step above follows from the definition of $\varepsilon_t$'s. Plugging the bound above to Equation \eqref{eq:agn_intermediate}, we obtain
\begin{equation*}
\begin{split}
\mathbb{E}\left[\sum_{t=1}^T \ell(y_t, \Acal(x_t)) \right] &\leq \inf_{h \in \Hcal}\sum_{t=1}^T \ell(y_t, h(x_t)) + c\,d_{\gamma }+ \gamma T + \alpha T+
c\, \sqrt{2d_{\gamma}T\ln\left(\frac{2cT}{\alpha} \right)} \\
&\leq \inf_{h \in \Hcal}\sum_{t=1}^T \ell(y_t, h(x_t)) + c\,d_{\gamma} + \gamma T+ 1 + 2c\,\sqrt{ d_{\gamma} T \ln(2cT)},
\end{split}
\end{equation*}
where the final step follows upon picking $\alpha = \frac{1}{T}$. Since $\gamma > 0 $ is arbitrary, this completes our proof.
\end{proof}
We highlight that the upper bound $\inf_{\gamma > 0 }\, \left\{c\cdot \text{SMdim}_{\gamma}(\mathcal{H}) + \gamma T + 1 + 2c\,\sqrt{ \text{SMdim}_{\gamma}(\mathcal{H}) T \ln(2cT)}\right\}$ can be tight up to logarithmic factors in $T$. To see this, consider the case where $\mathcal{Y} = \Zcal = [-1,1]$, $\ell(y, z) = |y-z|$, and $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$ such that $\text{seq-fat}_{\gamma}(\mathcal{H}) = d_{\gamma}$. Then, by Proposition \ref{prop:fateq}, we have that $\text{SMdim}_{\gamma}(\mathcal{H}) \leq \text{seq-fat}_{\gamma^{\prime}}(\mathcal{H}) $ for all $\gamma^{\prime} < \gamma$ and therefore the upperbound on the expected regret given by (i) in Theorem \ref{thm:agn} is at most $2\,\text{seq-fat}_{\gamma^{\prime}}(\Hcal) + 2\gamma^{\prime} T + 1+ 4\sqrt{\, \text{seq-fat}_{\gamma^{\prime}}(\Hcal)\, T\ln(4T)}$ for any $\gamma/2 \leq \gamma^{\prime} < \gamma$. However, we have a known lower bound $\frac{\gamma^{\prime}}{4 \sqrt{2}}\sqrt{\text{seq-fat}_{\gamma^{\prime}}(\Hcal)T}$ for scalar valued regression \citep{rakhlin2015online} . Thus, the upper bound given by (i) in Theorem \ref{thm:agn} is tight up to $O(\ln(T))$. We now move on to prove the lower bound.
\begin{proof}(of (ii) in Theorem \ref{thm:agn})
We now prove the lower bound given by (ii) in Theorem \ref{thm:agn}. Fix $\gamma > 0$ and $d_{\gamma} := \text{SMdim}_{\gamma}(\mathcal{H}) $. By definition of SMdim, there exists a $\mathcal{X}$-valued, $\Pi(\mathcal{Z})$-ary tree $\mathcal{T}$ of depth $d_{\gamma}$ shattered by $\mathcal{H}$. Let $(f_1, ..., f_d)$ be the sequence of edge-labeling functions $f_t: \Pi(\mathcal{Z})^t \rightarrow \Ycal$ associated with $\mathcal{T}$. Let $\mathcal{A}$ be any randomized learner for $\mathcal{H}$. Our goal will be to use $\mathcal{T}$ and its edge-labeling functions $(f_1, ..., f_d)$ to construct a difficult stream for $\mathcal{A}$ such that on every round, the expected loss of $\mathcal{A}$ is at least $ \gamma$ more than the loss of the optimal hypothesis in hindsight. This stream is obtained by traversing $\mathcal{T}$ adapting to the sequence of distributions output by $\mathcal{A}$.
To that end, for every round $t \in [d_{\gamma}]$, let $\mu_t$ denote the distribution that $\mathcal{A}$ computes before making its prediction $z_t \sim \mu_t$. Consider the stream $\{\left(\mathcal{T}_t(\mu_{<t}), f_t(\mu_{\leq t})\right)\}_{t=1}^{d_{\gamma}}$, where $\mu =
(\mu_1, \ldots, \mu_{d_{\gamma}})$ denotes the sequence of distributions output by $\Acal$. This stream is obtained by starting at the root of $\mathcal{T}$, passing $\mathcal{T}_1$ to $\mathcal{A}$, observing the distribution $\mu_1$ computed by $\mathcal{A}$, passing the label $f_t(\mu_{\leq 1})$ to $\mathcal{A}$, and then finally moving along the edge indexed by $\mu_1$. This process then repeats $d_{\gamma} -1$ times until the end of the tree $\mathcal{T}$ is reached. Note that we can observe and use the distribution computed by $\mathcal{A}$ on round $t$ to generate the label because $\mathcal{A}$ \textit{deterministically} maps a sequence of labeled instances to a distribution.
Recall that the shattering condition implies that there exists a $h_{\mu} \in \mathcal{H}$ such that $ \expect_{z_t \sim \mu_t}[\ell(f_t(\mu_{\leq t}), z_t)] \geq \ell(f_t(\mu_{\leq t}), h_{\mu}(\mathcal{T}_t(\mu_{<t})) + \gamma $ for all $t \in [d_{\gamma}]$. Therefore, the regret of $\Acal$ on the stream described above is at least
\begin{equation*}
\begin{split}
R_{\Acal}(T) \geq \sum_{t=1}^{d_{\gamma}} \expect_{z_t \sim \mu_t}[\ell(f_t(\mu_{\leq t}), z_t)] - \sum_{t=1}^{d_{\gamma}} \ell(f_t(\mu_{\leq t}), h_{\mu}(\mathcal{T}_t(\mu_{<t})) \geq \sum_{t=1}^{d_{\gamma}}\gamma = \gamma d_{\gamma}.
\end{split}
\end{equation*}
Since our choice of $\gamma$ and the randomized algorithm $\mathcal{A}$ is arbitrary, this holds true for any $\gamma > 0$ and randomized online learner. This completes the proof of part (ii).
\end{proof}
In full generality, the lower bound of $\sup_{\gamma >0}\, \gamma\, \text{SMdim}_{\gamma}(\Hcal)$ cannot be improved. In Section 5 of their work, \cite{raman2023online} provide a tuple $(\Xcal, \Ycal, \Zcal, \Hcal, \ell)$ demonstrating the tightness of this lower bound in the context of set-valued feedback. Another example demonstrating the tightness of the lower bound up to $O(\ln(T))$ is provided in Section \ref{sec:vec_reg} following the proof of Corollary \ref{cor:vec_cor}.
\subsection{When is a lower bound of $\Omega(\sqrt{T})$ possible?}
In many natural problems, one typically gets an agnostic lower bound that scales with $\Omega(\sqrt{T})$. This includes problem settings such as classification with $0$-$1$ loss, scalar-valued regression with absolute loss, and multilabel ranking with $0$-$1$ ranking loss \citep{ben2009agnostic, DanielyERMprinciple, rakhlin2015online, raman2023online}. All of the aforementioned settings exploit the problem structure such as the symmetry between the prediction and the label space or loss-specific properties such as being a distance metric. The following Lemma generalizes these properties and derives a sufficient condition to obtain the lower bound of $\Omega(\sqrt{T})$.
\begin{lemma}{Sufficient condition for $\Omega(\sqrt{T})$ regret}{suff_sqrt}
Consider a tuple $(\Xcal, \Ycal, \Zcal, \Hcal, \ell)$. Suppose there exists two distinct hypotheses $h_{-1}, h_{+1} \in \Hcal$, an instance $x \in \Xcal$, and a pair of labels $y_{-1}, y_{+1} \in \Ycal$ such that
\begin{itemize}
\item[(i)] $\eta:= \min_{\sigma \in \{\pm1\}} \Big(\ell(y_{-\sigma}, h_{\sigma}(x)) - \ell(y_{\sigma}, h_{\sigma}(x))\Big) > 0$
\item[(ii)] $ \inf_{z \in \Zcal}\Big( \ell(y_{-1}, z) + \ell(y_{+1}, z)\Big) \geq \frac{1}{2} \sum_{\sigma_1, \sigma_2} \ell(y_{\sigma_1}, h_{\sigma_2}(x))$
\end{itemize}
Then, for every randomized online learner, there exists a stream such that the expected regret of the learner is at least $\eta \, \sqrt{\frac{T}{8}}$.
\end{lemma}
\begin{comment}
\begin{lemma}{}{suff_sqrt}
Consider a tuple $(\Xcal, \Ycal, \Zcal, \Hcal, \ell)$. Suppose there exists $\eta>0$, distinct hypotheses $h_1, h_2 \in \Hcal$, an instance $x \in \Xcal$, and pair of labels $y_1, y_2 \in \Ycal$ that satisfies the following properties:
\begin{itemize}
\item[(i)] $\min_{i \neq j}\ell(y_i, h_j(x)) \geq \eta $
\item[(ii)] $\ell(y_i, h_i(x)) = 0$ for $i \in \{1,2\}$
\item[(iii)] $ \inf_{z \in \Zcal}\Big( \ell(y_1, z) + \ell(y_2, z)\Big) \geq \frac{1}{2} \Big( \ell(y_1, h_2(x)) + \ell(y_2, h_1(x))\Big)$
\end{itemize}
Then, for every randomized online learner, there exists a stream such that the expected regret of the learner is at least $\eta \sqrt{\frac{T}{8}}$.
\end{lemma}
\end{comment}
In the case when $\Ycal = \Zcal$ and the loss $\ell$ is a distance metric, the conditions above reduce to requiring two distinct hypotheses $h_{-1}, h_{+1} \in \Hcal$.
\begin{proof}(of Lemma \ref{lem:suff_sqrt})
Let $(\Xcal, \Ycal, \Zcal, \Hcal, \ell)$ be such that there exists $ h_{-1}, h_{+1} \in \Hcal$, $x \in \Xcal$, and $y_{-1}, y_{+1} \in \Ycal$ that satisfies the aforementioned properties. Let $\eta= \min_{\tau \in \{\pm1\}} \Big(\ell(y_{-\tau}, h_{\tau}(x)) - \ell(y_{\tau}, h_{\tau}(x))\Big) > 0$. For an odd $T \in \naturals$, consider the stream $\{x, y_{\sigma_t}\}_{t=1}^T$ where $\sigma_t \sim \text{Unif}(\{\pm1\})$. The expected loss of any randomized algorithm $\Acal$ on this stream is
\[\expect \left[\sum_{t=1}^T \ell(y_{\sigma_t}, \Acal(x)) \right] = \expect\left[\sum_{t=1}^T \frac{1}{2} \Big( \ell(y_{-1}, \Acal(x)) +\ell(y_{+1}, \Acal(x)) \Big)\right] \geq \frac{T}{4} \sum_{\tau_1, \tau_2 } \ell(y_{\tau_1}, h_{\tau_2}(x)),\]
where the first equality follows because $\sigma_t$'s are sampled uniformly at random and the second inequality follows due to condition (ii) in Lemma \ref{lem:suff_sqrt}.
Let $\sigma = \text{sign}\left( \sum_{t=1}^T \sigma_t\right)$. Then,
\begin{equation*}
\begin{split}
\expect\left[\min_{h \in \Hcal} \sum_{t=1}^T \ell(y_{\sigma_t}, h(x)) \right] &\leq \expect\left[ \sum_{t=1}^T \ell(y_{\sigma_t}, h_{\sigma}(x)) \right] \\
&= \expect\left[ \sum_{t=1}^T \Big(\ell(y_{\sigma_t}, h_{\sigma}(x))
\indicator\{\sigma = \sigma_t\} +\ell(y_{\sigma_t}, h_{\sigma}(x)) \indicator\{\sigma = -\sigma_t\} \Big)\right] \\
&= \sum_{t=1}^T \expect\left[\ell(y_{\sigma}, h_{\sigma}(x)) \, \left(\frac{1+\sigma \sigma_t}{2} \right)+ \ell(y_{-\sigma}, h_{\sigma}(x)) \, \left( \frac{1-\sigma \sigma_t}{2} \right)\right] \\
&= \frac{T}{2} \Big( \expect\left[\ell(y_{\sigma}, h_{\sigma}(x))\right] + \expect\left[\ell(y_{-\sigma}, h_{\sigma}(x))\right]\Big) \, - \frac{1}{2} \expect\left[\Big(\ell(y_{-\sigma}, h_{\sigma}(x)) - \ell(y_{\sigma}, h_{\sigma}(x)) \Big)\sum_{t=1}^T \sigma\, \sigma_t\right] \\
&= \frac{T}{4} \sum_{\tau_1, \tau_2 } \ell(y_{\tau_1}, h_{\tau_2}(x)) \, - \frac{1}{2} \expect\left[\Big(\ell(y_{-\sigma}, h_{\sigma}(x)) - \ell(y_{\sigma}, h_{\sigma}(x)) \Big) \left|\sum_{t=1}^T \sigma_t \right|\right] \\
&\leq \frac{T}{4} \sum_{\tau_1, \tau_2 } \ell(y_{\tau_1}, h_{\tau_2}(x)) - \frac{\eta}{2} \expect\left[\left|\sum_{t=1}^T \sigma_t \right|\right],
\end{split}
\end{equation*}
where the final equality follows upon using the fact that $\sigma \sim \text{Unif}(\{-1,1\})$ and the final inequality follows from condition (i).
Thus, the expected regret of $\Acal$ is
\begin{equation*}
\begin{split}
\expect \left[\sum_{t=1}^T \ell(y_{\sigma_t}, \Acal(x)) \right] - \expect\left[\min_{h \in \Hcal} \sum_{t=1}^T \ell(y_{\sigma_t}, h_{\sigma}(x)) \right]
&\geq \frac{\eta}{2}\expect\left[\left|\sum_{t=1}^T \sigma_t \right|\right] \\
&\geq\eta \sqrt{\frac{T}{8}},
\end{split}
\end{equation*}
where the final step follows upon using Khinchine's inequality \cite[Page 364]{cesa2006prediction}. This completes our proof.
\end{proof}
\section{Relations Between Combinatorial Dimensions}\label{sec:relations}
In this section, we show that SMdim reduces to existing combinatorial dimensions. We start with Proposition \ref{prop:GMS=Ldim}, which shows that SMdim $\equiv$ Ldim.
\begin{proposition}{SMdim $\equiv$ Ldim}{GMS=Ldim}
Let $\mathcal{Y} = \mathcal{Z}$, $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$, and $\ell(y, z) = \mathbbm{1}\{y \neq z \}$ be the 0-1 loss function. Then, $\text{SMdim}_{\gamma}(\mathcal{H}) = \text{Ldim}(\mathcal{H})$ for all $\gamma \in [0, \frac{1}{2}]$.
\end{proposition}
\begin{proof}
Fix $\gamma \in (0, \frac{1}{2}]$. We first show that $\text{SMdim}_{\gamma}(\mathcal{H}) \leq \text{Ldim}(\mathcal{H})$. Let $\mathcal{T}$ be a $\mathcal{X}$-valued, $\Pi(\mathcal{Z})$-ary tree of depth $d_{\gamma} = \text{SMdim}_{\gamma}(\mathcal{H})$ shattered by $\mathcal{H}$. Let $v$ be the root node of $\mathcal{T}$ and $x$ denote the instance labeling the node. Recall that $v$ has an outgoing edge for each measure $\mu \in \Pi(\mathcal{Z})$. Let $\{y_{\mu}\}_{\mu \in \Pi(\mathcal{Z})}$ be the set of elements in $\mathcal{Y}$ that label the outgoing edges from $v$. We first claim that there at least two distinct elements in the set $\{y_{\mu}\}_{\mu \in \Pi(\mathcal{Z})}$. For the sake of contradiction, suppose this was not the case. That is, there is only one distinct element that labels the outgoing edges from $v$. Let $y $ denote the element that labels the outgoing edges from $v$. That is, $y_{\mu} = y$ for all $\mu \in \Pi(\mathcal{Z})$. Consider the Dirac measure $\delta_y$ that puts all mass on $y$. Note that $\delta_y \in \Pi(\mathcal{Z})$ and therefore there exists an outgoing edge from $v$ indexed by $\delta_y$ and labeled by $y$. However, it must be the case that $\mathbb{P}_{z \sim \delta_y}\left[y \neq z \right] = 0$. Since $\gamma > 0$, the shattering condition required by Definition \ref{def:SMdim} cannot be met, which is a contradiction. Accordingly, there is at least two distinct elements in the set $\{y_{\mu}\}_{\mu \in \Pi(\mathcal{Z})}$.
Let $y_{-1}, y_{+1}$ be the distinct elements of the set $\{y_{\mu}\}_{\mu \in \Pi(\mathcal{Z})}$, and $\mu_{-1}, \mu_{+1}$ be the indices of the edges labeled by $y_1$ and $y_2$ respectively. Let $\mathcal{H}_{-1} = \{h_\mu: \mu \in \Pi(\mathcal{Z})^d, \mu_1 = \mu_{-1}\}$ denote the set of shattering hypothesis that corresponds to following a path down $\mathcal{T}$ that takes the outgoing edge indexed $\mu_{-1}$ from the root node. Likewise define $\mathcal{H}_{+1}$. Keep the edges indexed by $\mu_{-1}$ and $\mu_{+1}$ and remove all other outgoing edges along with their corresponding subtree. Reindex the two edges using $\{\pm 1\}$. The root node $v$ should now have two outgoing edges, indexed by $\{\pm 1\}$ and labeled by a distinct element of $\mathcal{Y}$, matching the first constraint of a $\text{Ldim}$ tree. As for the second constraint, observe that for all $h_{-1} \in \mathcal{H}_{-1}$ and $h_{+1} \in \mathcal{H}_{+1}$ the shattering condition from Definition \ref{def:SMdim} implies that $\mathbb{P}_{z \sim \mu_{-1}}\left[y_{-1} \neq z \right] \geq \mathbbm{1}\{y_{-1} \neq h_{-1}(x)\} + \gamma$ and $\mathbb{P}_{z \sim \mu_{+1}}\left[y_{+1} \neq z \right] \geq \mathbbm{1}\{y_{+1} \neq h_{+1}(x)\} + \gamma.$ However, this can only be true if both $\mathbbm{1}\{y_{-1} \neq h_{-1}(x)\} = 0 \implies y_{-1} = h_{-1}(x)$ and $\mathbbm{1}\{y_{+1} \neq h_{+1}(x)\} = 0 \implies y_{+1} = h_{+1}(x)$. Accordingly, the hypotheses that shatters the edges indexed by $\mu_{-1}$ and $\mu_{+1}$ in the original tree according to Definition \ref{def:SMdim} also shatters the newly re-indexed edges according to Definition \ref{def:ldim}. Recursively repeating the above procedure on the subtrees following the two reindexed edges results in a $\text{Ldim}$ tree shattered by $\mathcal{H}$ of depth $d$. Thus, $\text{SMdim}_{\gamma}(\mathcal{H}) \leq \text{Ldim}(\mathcal{H})$ for $\gamma \in (0, \frac{1}{2}]$. The case when $\gamma = 0$ follows similarly and uses the fact that when $\gamma = 0$, we define the shattering condition in SMdim with a strict inequality (see last sentence in Definition \ref{def:SMdim}).
We now prove the inequality that $\text{SMdim}_{\gamma}(\mathcal{H}) \geq \text{Ldim}(\mathcal{H})$. Fix $\gamma \in [0, \frac{1}{2}]$. Let $\mathcal{T}$ be a $\mathcal{X}$-valued, $\{\pm1\}$-ary tree of depth $d = \text{Ldim}(\mathcal{H})$ shattered by $\mathcal{H}$ according to Definition \ref{def:ldim}. Our goal will be to expand $\mathcal{T}$ into a $\Pi(\mathcal{Z})$-ary tree that is $\gamma$-shattered by $\mathcal{H}$ according to Definition \ref{def:SMdim}. Let $v$ be the root node of $\mathcal{T}$, $x$ be the instance that labels the root node, and $y_{-1}, y_{+1}$ denote the distinct elements of $\mathcal{Y}$ that label the left and right outgoing edges from $v$ respectively. Let $\mathcal{H}_{-1} = \{h_\sigma: \sigma \in \{\pm1\}^d, \sigma_1 = -1\} \subset \mathcal{H}$ denote the set of shattering hypothesis that correspond to following a path down $\mathcal{T}$ that takes the edge indexed by $-1$ in the first level. Define $\mathcal{H}_{+1}$ analogously. Then, for all $h_{-1} \in \mathcal{H}_{-1}$ and $h_{+1} \in \mathcal{H}_{+1}$, the shattering condition implies that $h_{-1}(x) = y_{-1}$ and $h_{+1}(x) = y_{+1}$.
For every measure $\mu \in \Pi(\mathcal{Z})$, we claim that there exists a $\sigma_{\mu} \in \{\pm1\}$ such that $\mathbb{P}_{z \sim \mu}\left[y_{\sigma_{\mu}} \neq z \right] = \mu(\{y_{\sigma_{\mu}}\}^c) \geq \gamma$. Suppose for the sake of contradiction that this is not true. Then, there exists a measure $\mu \in \Pi(\mathcal{Z})$ such that for both $\sigma \in \{\pm1\}$, we have $\mu(\{y_{\sigma}\}^c) < \gamma$. Then, $1 = \mu(\mathcal{Z}) = \mu(\{y_{-1}\}^c \cup \{y_{+1}\}^c) < 2\gamma < 1$, a contradiction. Thus, for every measure $\mu \in \Pi(\mathcal{Z})$ there exists a $\sigma_{\mu} \in \{\pm1\}$ such that $\mathbb{P}_{z \sim \mu}\left[y_{\sigma_{\mu}} \neq z \right] \geq \gamma$. Combining this with the fact that for any $h_{-1} \in \mathcal{H}_{-1}$ and $h_{+1} \in \mathcal{H}_{+1}$, we have $y_{-1} = h_{-1}(x)$ and $y_{+1} = h_{+1}(x)$, gives that, for every measure $\mu \in \Pi(\mathcal{Z})$, there exists a $\sigma_{\mu} \in \{\pm1\}$ such that for all $h_{\sigma_{\mu}} \in \mathcal{H}_{\sigma_{\mu}}$ ,we have $\mathbb{P}_{z \sim \mu}\left[y_{\sigma_{\mu}} \neq z \right] \geq \mathbbm{1}\{y_{\sigma_{\mu}} \neq h_{\sigma_{\mu}}(x)\} + \gamma$. Note that if we take $y_{\sigma_{\mu}}$ to be the label on an edge indexed by $\mu$, then the inequality above matches the shattering condition required by Definition \ref{def:SMdim}.
To that end, for every measure $\mu \in \Pi(\mathcal{Z})$, add an outgoing edge from $v$ indexed by $\mu$ and labeled by the $y_{\sigma_{\mu}}$, where $\sigma_{\mu}$ is the index as promised by the analysis above. Grab the sub-tree in $\mathcal{T}$ following the original outgoing edge from $v$ indexed by $\sigma_{\mu}$, and append it to the newly constructed outgoing edge from $v$ indexed by $\mu$. Remove the original outgoing edges from $v$ indexed by $\{\pm1\}$ and their corresponding subtrees. Recursively repeat the above procedure on the subtrees following the newly created edges indexed by measures. Upon repeated this process for every internal node in $\mathcal{T}$, we obtain a $\Pi(\mathcal{Z})$-ary tree that is $\gamma$-shattered by $\mathcal{H}$ of depth $d$. Thus, we have that $\text{Ldim}(\mathcal{H}) \leq \text{SMdim}_{\gamma}(\mathcal{H})$ for $\gamma \in [0, \frac{1}{2}]$.
\end{proof}
% \begin{proof}
% Fix $\gamma \in (0, \frac{1}{2}]$. We first prove that $\text{MSdim}_{\gamma}(\mathcal{H}) \leq \text{Ldim}(\mathcal{H})$. Let $\mathcal{T}$ be a $\mathcal{X}$-valued, $\Pi(\mathcal{Z})$-ary tree of depth $d_{\gamma} = \text{MSdim}_{\gamma}(\mathcal{H})$ shattered by $\mathcal{H}$. We first claim that for every internal node $v$, at least two of its outgoing edges are labeled by different elements of $\mathcal{Z}$. Suppose this was the not the case, that is, there exists a $z \in \mathcal{Z}$ such that for every $\mu \in \Pi(\mathcal{Z})$, the edge indexed by $\mu$ is labeled by $z \in \mathcal{Z}$. Consider the edge indexed by the Dirac measure $\delta_{z}$ that places all mass on $z$. Then, $\mathbb{E}_{z \sim \delta_{z}}\left[\ell(z, z \right] = 0$. Since $\gamma > 0$, the shattering cannot be satisfied by any hypothesis $h \in \mathcal{H}$, leading to a contradiction. Next, since $\ell$ is the 0-1 loss function, the only way for the shattering condition to hold is if for every path $\mu \in \Pi(\mathcal{Z})^{d_{\gamma}}$, the shattering hypothesis $h_{\mu}$ at the end of the path satisfies $\ell(h_{\mu}(\mathcal{T}_t(\mu_{<t})), f_t(\mu_{\leq t})) = 0 \implies h_{\mu}(\mathcal{T}_t(\mu_{<t}) = f_t(\mu_{\leq t})$. In other words, for every path $\mu \in \Pi(\mathcal{Z})^{d_{\gamma}}$, the shattering hypothesis $h_{\mu}$ when evaluated on the sequence of instance corresponding to the path, outputs labels that exactly agrees with the elements of $\mathcal{Z}$ labeling the edges corresponding to the path. Note that this is exactly the shattering condition required by Ldim in Definition \ref{def:ldim}. Using these two facts, we now construct a Ldim tree.
% Let $v$ be the root node of $\mathcal{T}$ and $\{z_{\mu}\}_{\mu \in \Pi(\mathcal{Z})} \subset \mathcal{Z}$ be the sequence of elements in $\mathcal{Z}$ \textit{labeling} the outgoing edges from $v$ (recall that $\mathcal{Z} = \mathcal{Y}$). Find two edges that are labeled by different elements of $\mathcal{Z}$. Let $z_1, z_2 \in \mathcal{Z}$ denote the labels on these two edges. Keep these two edges and remove all other outgoing edges along with their corresponding subtree. Reindex the two edges using $\{\pm1\}$ and relabel the two edges using the elements $z_1$ and $z_2$ respectively. Recursively repeat this process on the two substrees following the two edges labeled by $z_1$ and $z_2$. Completing this process on every internal node returns a Ldim tree shattered by $\mathcal{H}$ of depth $d_{\gamma}$. The case when $\gamma = 0$ follows identically and uses the fact that when $\gamma = 0$, we define the shattering condition in MSdim with a strict inequality (see last sentence in Definition \ref{def:SMdim}).
% We now show the reverse direction - $\text{Ldim}(\mathcal{H}) \leq \text{MSdim}_{\gamma}(\mathcal{H})$. Fix $\gamma \in [0, \frac{1}{2}]$. Let $\mathcal{T}$ be a $\mathcal{X}$-valued, $\{\pm1\}$-ary tree shattered by $\mathcal{H}$ of depth $d = \text{Ldim}(\mathcal{H})$. Let $v$ be the root node and let $z_{-1}$ and $z_{+1}$ be the elements of $\mathcal{Z}$ that label the left and right edges respectively. To transform $\Tcal$ to a $\Pi(\Zcal)$-ary tree, we now construct an outgoing edge for each measure. Fix a measure $\mu \in \Pi(\Zcal)$. Then, there must exist an $\sigma_{\mu} \in \{\pm1\}$ such that $\mathbb{E}_{z \sim \mu}\left[\ell(z_{\sigma_{\mu}}, z)\right] = \mu(\{z_{\sigma}\}^c) \geq \gamma$. Suppose, for the sake of contradiction, this is not true. That is, $\mu(\{z_{\sigma}\}^c) < \gamma$ for both $ \sigma \in \{\pm1\}$. Then, $\mu(\mathcal{Z}) = \mu(\{z_{-1}\}^c \cup \{z_{+1}\}^c) < 2\gamma < 1$, a contradiction. Therefore, for every $\mu$, there exists a $\sigma_{\mu} \in \{\pm 1\}$ such that $\mathbb{E}_{z \sim \mu}\left[\ell(z_{\sigma_{\mu}}, z)\right] \geq \gamma$. For every measure $\mu \in \Pi(\mathcal{Z})$, add an outgoing edge from $v$ indexed by $\mu$ and labeled by $z_{\sigma_{\mu}}$. Grab the sub-tree in $\mathcal{T}$ following the outgoing edge from $v$ indexed by $\sigma_{\mu}$ and append it to the newly constructed outgoing edge from $v$ indexed by $\mu$. Remove the original two outgoing edges from $v$ indexed by $\sigma \in \{\pm 1\}$ along with their corresponding subtrees. Upon repeating this process for every internal node $v$ in $\Tcal$, we obtain a $\Pi(\Zcal)$-ary tree. Since Littlestone dimension requires the shattering hypothesis to exactly match the elements of $\mathcal{Z}$ labeling the edges, and because we never add any new edges, but only reused the original, existing ones in $\mathcal{T}$ to expand the tree, it follows that the newly constructed tree is $\gamma$-shattered by $\mathcal{H}$. Thus, we have $\text{MSdim}_{\gamma}(\mathcal{H}) \geq \text{Ldim}(\mathcal{H})$.
% \end{proof}
Next, we show an equivalence between SMdim and seq-fat.
\begin{proposition}{SMdim $\equiv$ seq-fat}{fateq}
Let $\mathcal{Y} = \mathcal{Z} = [-1, 1]$, $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$, and $\ell(y, z) = |y - z|$ be the absolute value loss function. Then for every $\gamma \in (0, 1]$,
$$\text{seq-fat}_{\gamma}(\mathcal{H}) \leq \text{SMdim}_{\gamma}(\mathcal{H}) \leq \text{seq-fat}_{\gamma'}(\mathcal{H})$$
where $\gamma' < \gamma. $
\end{proposition}
\begin{proof}
We first prove the upper bound. Let $\gamma \in (0, 1]$ and $\gamma' < \gamma$. Let $\mathcal{T}$ be a $\mathcal{X}$-valued, $\Pi(\mathcal{Z})$-ary tree of depth $d_{\gamma} = \text{SMdim}_{\gamma}(\mathcal{H})$ shattered by $\mathcal{H}$. Let $v$ be the root node of $\mathcal{T}$ and $x$ denote the instance labeling the node. Recall that $v$ has an outgoing edge for each measure $\mu \in \Pi(\mathcal{Z})$. In particular, this means that $v$ has outgoing edges corresponding to the Dirac measures on $\mathcal{Z}$, which we denote by $\{\delta_z\}_{z \in \mathcal{Z}}$. Fix a $z \in \mathcal{Z}$ and consider the outgoing edge from $v$ indexed by $\delta_z$. Let $y_{z} \in \mathcal{Y}$ be the element that labels the outgoing edge indexed by $\delta_{z}$. Let $\mathcal{H}_{z} = \{h_{\mu}: \mu \in \Pi(\mathcal{Z})^d, \mu_1 = \delta_{z}\} \subset \mathcal{H}$ denote the set of shattering hypothesis that corresponds to following a path down $\mathcal{T}$ that takes the edge $\delta_z$ in the root node. Then, for all $h \in \mathcal{H}_{z}$ the shattering condition from Definition \ref{def:SMdim} implies that
$$|z - y_{z}| \geq |h(x) - y_{z}| + \gamma > |h(x) - y_{z}| + \gamma'.$$
Taking the supremum on both sides, gives that:
\begin{equation}\label{eq:relation_ineq}
|z - y_{z}| > \sup_{h \in \mathcal{H}_{z}}|h(x) - y_{z}| + \gamma' = r_z + \gamma'.
\end{equation}
where we let $r_z = \sup_{h \in \mathcal{H}_{z}}|h(x) - y_{z}|$.
% $$|z - \tilde{z}| > \sup_{h \in \mathcal{H}_{\delta_{z}}}|h(x) - \tilde{z}| + \gamma'.$$
Let $I_z := \left[y_{z} - (r_z + \gamma'), y_{z} + (r_z + \gamma')\right] \subset [-3, 3]$ denote an interval corresponding to $z$. The inequality above implies that $z \notin I_z$ (note that $I_z$ changes depending on $z$). Since $z \in \mathcal{Z}$ was arbitrary, it must be the case that $z \notin I_z$ for all $z \in \mathcal{Z}$. This means that $\bigcap_{z \in \mathcal{Z}} I_z = \emptyset$. Since $[-3, 3]$ is compact and $\{I_z\}_{z \in \mathcal{Z}}$ is a family of intervals whose intersection is empty, the celebrated Helly's theorem states that there exists two intervals in $\{I_z\}_{z \in \mathcal{Z}}$ that are disjoint \citep{ECKHOFF1993389, radon1921mengen}. Accordingly, let $z_1, z_2$ be such that $I_{z_1} \cap I_{z_2} = \emptyset$. As before, let $y_{z_1}$ and $y_{z_2}$ be the labels on the outgoing edges from $v$ indexed by the Dirac measures $\delta_{z_1}$ and $\delta_{z_2}$ respectively. Without loss of generality, let $y_{z_1} < y_{z_2}$ (we have strict inequality because we are guaranteed that $I_{z_1}$ and $I_{z_2}$ are disjoint). By inequality \ref{eq:relation_ineq}, for all $h_{z_1} \in \mathcal{H}_{z_1}$ and $h_{z_2} \in \mathcal{H}_{z_2}$ we have that
$$h_{z_1}(x) \in \left[y_{z_1} - r_{z_1}, y_{z_1} + r_{z_1} \right] \quad\text{and}\quad h_{z_2}(x) \in \left[y_{z_2} - r_{z_2}, y_{z_2} + r_{z_2} \right].$$
Let $s = \frac{y_{z_1} + r_{z_1} + y_{z_2} - r_{z_2}}{2} \in [-1, 1]$ be a witness. Then, for all $h_{z_1} \in \mathcal{H}_{z_1}$ and $h_{z_2} \in \mathcal{H}_{z_2}$, we have that $s - h_{z_1}(x) \geq \gamma'$ and $h_{z_2}(x) - s \geq \gamma'$. Relabel the two edges indexed by $\delta_{z_1}$ and $\delta_{z_2}$ with the same witness $s$. Reindex the two edges indexed by $\delta_{z_1}$ and $\delta_{z_2}$ with $-1$ and $+1$ respectively. Remove all other edges indexed by measures and their corresponding subtrees. There should now only be two outgoing edges from $v$, each labeled by the same witness. Next, recall that for all $h_{z_1} \in \mathcal{H}_{z_1}$ and $h_{z_2} \in \mathcal{H}_{z_2}$ we have that $s - h_{z_1} \geq \gamma'$ and $h_{z_2} - s \geq \gamma'$. Accordingly, the hypotheses that shatter the edges indexed by $\delta_{z_1}$ and $\delta_{z_2}$ in the original tree according to Definition \ref{def:SMdim} also shatter the newly re-indexed and relabeled edges according to Definition \ref{def:sfat}. Recursively repeating the above procedure on the subtrees following the two newly reindexed and relabeled edges results in a seq-fat tree $\gamma'$-shattered by $\mathcal{H}$ of depth $d_{\gamma}$. Thus,
$\text{SMdim}_{\gamma}(\mathcal{H}) \leq \text{seq-fat}_{\gamma'}(\mathcal{H})$ for $\gamma' < \gamma$.
We now move on to prove the lower bound. Let $\gamma \in (0, 1]$ and $\mathcal{T}$ be a $\mathcal{X}$-valued, $\{\pm1\}$-ary tree of depth $d_{\gamma} = \text{seq-fat}_{\gamma}(\mathcal{H})$ shattered by $\mathcal{H}$ according to Definition \ref{def:sfat}. Our goal will be expand $\mathcal{T}$ into a $\Pi(\mathcal{Z})$-ary tree that is $\gamma$-shattered by $\mathcal{H}$ according to Definition \ref{def:SMdim}. Let $v$ be the root node, $x$ the instance that labels the root node, and $s$ be the witness that labels the two outgoing edges of $v$. Let $\mathcal{H}_{-1} = \{h_{\sigma}: \sigma \in \{\pm1\}^{d_{\gamma}}, \sigma_1 = -1\} \subset \mathcal{H}$ denote the set of shattering hypothesis that corresponds to following a path down $\mathcal{T}$ that takes the outgoing edge indexed by $-1$ from the root node. Likewise define $\mathcal{H}_{+1}$. Then, for all $h_{-1} \in \mathcal{H}_{-1}$ and $h_{+1} \in \mathcal{H}_{+1}$, the shattering condition implies that $ s - h_{-1}(x) \geq \gamma$ and $h_{+1}(x) - s \geq \gamma$ respectively.
% Defining $I_{-1} = [-1, s+ \gamma]$ and $I_{+1} = [s+\gamma, 1]$, then gives that for all $h_{-1} \in \mathcal{H}_{-1}$ and $h_{+1} \in \mathcal{H}_{+1}$, we have $h_{-1}(x) \in I_{-1}$ and $h_{+1}(x) \in I_{+1}$.
For every measure $\mu \in \Pi(\mathcal{Z})$, we claim that there exist a $\sigma_{\mu} \in \{-1, 1\}$ such that $\mathbb{E}_{z \sim \mu}\left[|\sigma_{\mu} - z| \right] \geq |s - \sigma_{\mu}|$. Suppose for the sake of contradiction that this is not true. That is, there exists $\mu \in \Pi(\mathcal{Z})$ such that for all $\tau \in \{-1, 1\}$ we have that $\mathbb{E}_{z \sim \mu}\left[|\tau - z| \right] < |s - \tau|$. Then, when $\tau = -1$, we have that $\mathbb{E}_{z \sim \mu}\left[z\right] < |s + 1| - 1$ and when $\tau = 1$, we have $1 - |s - 1| < \mathbb{E}_{z \sim \mu}\left[z\right]$, using the fact that $|\tau - z| = 1 -\tau z $. Combining the two inequalities together and using the fact that $s \in [-1, 1]$ gives that $2 < |s + 1| + |s - 1| = 2$, which is a contradiction. Accordingly, for every measure $\mu \in \Pi(\mathcal{Z})$, there exists a $\sigma_{\mu} \in \{-1, 1\}$ such that $\mathbb{E}_{z \sim \mu}\left[|\sigma_{\mu} - z| \right] \geq |s - \sigma_{\mu}|$. Next, crucially note that for any $\tau \in \{\pm1\}$ and any $h_{\tau} \in \mathcal{H}_{\tau}$, we have $|h_{\tau}(x) - \tau| = |s - \tau| - |h_{\tau}(x) - s| \leq |s - \tau| - \gamma$ by the seq-fat shattering condition from Definition \ref{def:sfat}. Therefore, for every measure $\mu \in \Pi(\mathcal{Z})$, there exists $\sigma_{\mu} \in \{\pm1\}$ such that for all $h_{\sigma_{\mu}} \in \mathcal{H}_{\sigma_{\mu}}$, we have that $\mathbb{E}_{z \sim \mu}\left[|\sigma_{\mu} - z| \right] \geq |\sigma_{\mu} - h_{\sigma_{\mu}}(x)| + \gamma$. Note that if we take $\sigma_{\mu}$ to be the label on a edge indexed by $\mu$, then $\mathbb{E}_{z \sim \mu}\left[|\sigma_{\mu} - z| \right] \geq |\sigma_{\mu} - h_{\sigma_{\mu}}(x)| + \gamma$ exactly matches the shattering condition required by Definition \ref{def:SMdim}.
To that end, for every measure $\mu \in \Pi(\mathcal{Z})$, add an outgoing edge from $v$ indexed by $\mu$ and labeled by the $\sigma_{\mu} \in \{\pm1\}$ promised in the analysis above. Grab the sub-tree in $\mathcal{T}$ following the original outgoing edge from $v$ indexed by $\sigma_{\mu}$, and append it to the newly constructed outgoing edge from $v$ indexed by $\mu$. Remove the original outgoing edges from $v$ indexed by $-1$ and $+1$ and their corresponding subtrees. Recursively repeat the above procedure on the subtrees following the newly created edges indexed by measures. Upon repeating this process for every internal node in $\mathcal{T}$, we obtain a $\Pi(\mathcal{Z})$-ary tree that is $\gamma$-shattered by $\mathcal{H}$ of depth $d_{\gamma}$. Thus, we have that $\text{seq-fat}_{\gamma}(\mathcal{H}) \leq \text{SMdim}_{\gamma}(\mathcal{H})$.
\end{proof}
% \begin{proof}
% We first prove the upper bound. Let $\gamma \in (0, 1]$ and $\gamma' < \gamma$. Let $\mathcal{T}$ be a $\mathcal{X}$-valued, $\Pi(\mathcal{Z})$-ary tree of depth $d_{\gamma} = \text{SMdim}_{\gamma}(\mathcal{H})$ shattered by $\mathcal{H}$. Let $v$ be the root node of $\mathcal{T}$ and $x$ denote the instance labeling the node. Recall that $v$ has an outgoing edge for each measure $\mu \in \Pi(\mathcal{Z})$. In particular, this means that $v$ has outgoing edges corresponding to the Dirac measures on $\mathcal{Z}$, which we denote by $\{\delta_z\}_{z \in \mathcal{Z}}$. Fix a $z \in \mathcal{Z}$ and consider the outgoing edge from $v$ indexed by $\delta_z$. Let $\tilde{z} \in \mathcal{Z}$ be the element that labels the outgoing edge indexed by $\delta_{z}$. Let $\mathcal{H}_{\delta_{z}} = \{h_{\mu}: \mu_1 = \delta_{z}\} \subset \mathcal{H}$ denote the set of shattering hypothesis that corresponds to following a path down $\mathcal{T}$ that takes the edge $\delta_z$ in the first level. Then, for all $h \in \mathcal{H}_{\delta_{z}}$ the shattering condition implies that
% $$|z - \tilde{z}| \geq |h(x) - \tilde{z}| + \gamma > |h(x) - \tilde{z}| + \gamma'$$
% which, after taking the supremum on both sides, further means that
% \begin{equation}\label{eq:relation_ineq}
% |z - \tilde{z}| > \sup_{h \in \mathcal{H}_{\delta_{z}}}|h(x) - \tilde{z}| + \gamma'.
% \end{equation}
% % $$|z - \tilde{z}| > \sup_{h \in \mathcal{H}_{\delta_{z}}}|h(x) - \tilde{z}| + \gamma'.$$
% Let $I_z := \left[\tilde{z} - (\sup_{h \in \mathcal{H}_{\delta_{z}}}|h(x) - \tilde{z}| + \gamma'), \tilde{z} + (\sup_{h \in \mathcal{H}_{\delta_{z}}}|h(x) - \tilde{z}| + \gamma')\right] \subset [-1, 1]$ denote an interval corresponding to $z$. The inequality above implies that $z \notin I_z$ (note that $I_z$ changes depending on $z$). Since $z \in \mathcal{Z}$ was arbitrary, it must be the case that $z \notin I_z$ for all $z \in \mathcal{Z}$. This means that $\bigcap_{z \in \mathcal{Z}} I_z = \emptyset$. Since $[-1, 1]$ is compact and $\{I_z\}_{z \in \mathcal{Z}}$ is a family of intervals whose intersection is empty, the celebrated Helly's theorem states that there exists two intervals in $\{I_z\}_{z \in \mathcal{Z}}$ that are disjoint \cite{ECKHOFF1993389, radon1921mengen}. Accordingly, let $z_1, z_2$ be such that $I_{z_1} \cap I_{z_2} = \emptyset$. As before, let $\tilde{z}_1$ and $\tilde{z}_2$ be the labels on the outgoing edges from $v$ indexed by the Dirac measures $\delta_{z_1}$ and $\delta_{z_2}$ respectively. Without loss of generality, let $\tilde{z}_1 < \tilde{z}_2$ (we have strict inequality because we are guaranteed that $I_{z_1}$ and $I_{z_2}$ are disjoint). By inequality \ref{eq:relation_ineq}, for all $h_{\delta_{z_1}} \in \mathcal{H}_{\delta_{z_1}}$ and $h_{\delta_{z_2}} \in \mathcal{H}_{\delta_{z_2}}$ we have that
% $$h_{\delta_{z_1}}(x) \in \left[\tilde{z}_1 - \sup_{h \in \mathcal{H}_{\delta_{z_1}}}|h(x) - \tilde{z}_1|, \tilde{z}_1 + \sup_{h \in \mathcal{H}_{\delta_{z_1}}}|h(x) - \tilde{z}_1| \right] := \tilde{I}_{z_1}$$
% and
% $$h_{\delta_{z_2}}(x) \in \left[\tilde{z}_2 - \sup_{h \in \mathcal{H}_{\delta_{z_2}}}|h(x) - \tilde{z}_2|, \tilde{z}_2 + \sup_{h \in \mathcal{H}_{\delta_{z_2}}}|h(x) - \tilde{z}_2| \right] := \tilde{I}_{z_2}$$
% respectively. Note that $\tilde{I}_{z_1} \subset I_{z_1}$ and $\tilde{I}_{z_2} \subset I_{z_2}$. Let $s = \frac{\tilde{z}_1 + \tilde{z}_2}{2}$ be a witness. Then, we have that $\min_{x \in \tilde{I}_{z_1}}|x - s| = \gamma'$ and $\min_{x \in \tilde{I}_{z_2}}|x - s| = \gamma'$. That is, $s$ is distance $\gamma'$ away from both intervals $\tilde{I}_{z_1}$ and $\tilde{I}_{z_2}$. Relabel the two edges indexed by $\delta_{z_1}$ and $\delta_{z_2}$ with the same witness $s$. Reinindex the two edges indexed by $\delta_{z_1}$ and $\delta_{z_2}$ with $-1$ and $+1$ respectively. Remove all other edges indexed by measures and their corresponding substrees. There should now only be two outgoing edges from $v$, each labeled by the same witness. Next, observe that for all $h_{\delta_{z_1}} \in \mathcal{H}_{\delta_{z_1}}$ and $h_{\delta_{z_2}} \in \mathcal{H}_{\delta_{z_2}}$ we have that $s - h_{\delta_{z_1}} \geq \gamma'$ and $h_{\delta_{z_2}} - s \geq \gamma'$. Accordingly, the hypotheses that shatter the edges indexed by $\delta_{z_1}$ and $\delta_{z_2}$ in the original tree according to Definition \ref{def:SMdim} also shatter the newly re-indexed and relabeled edges according to Definition \ref{def:sfat}. Recursively repeating the above procedure on the subtrees following the two newly reindexed and relabeled edges results in a seq-fat tree $\gamma'$-shattered by $\mathcal{H}$ of depth $d_{\gamma}$. Thus,
% $\text{SMdim}_{\gamma}(\mathcal{H}) \leq \text{seq-fat}_{\gamma'}(\mathcal{H})$ for $\gamma' < \gamma$.
% We now move on to prove the lower bound. Let $\gamma \in (0, 1]$ and $\mathcal{T}$ be a $\mathcal{X}$-valued, $\{\pm1\}$-ary tree of depth $d_{\gamma} = \text{seq-fat}_{\gamma}(\mathcal{H})$ shattered by $\mathcal{H}$ according to Definition \ref{def:sfat}. Our goal will be expand $\mathcal{T}$ into a $\pi(\mathcal{Z})$-ary tree that is $\gamma$-shattered by $\mathcal{H}$ according to Definition \ref{def:SMdim}. Let $v$ be the root node, $x$ the instance that labels the root node, and $s$ be the witness that labels the two outgoing edges of $v$. Let $\mathcal{H}_{-1} = \{h_{\sigma}: \sigma_1 = -1\} \subset \mathcal{H}$ denote the set of shattering hypothesis that corresponds to following a path down $\mathcal{T}$ that takes the edge index by $-1$ in the first level. Likewise, define $\mathcal{H}_{+1}$. Then, for all $h_{-1} \in \mathcal{H}_{-1}$ and $h_{+1} \in \mathcal{H}_{+1}$, the shattering condition implies that $ s - h_{-1}(x) \geq \gamma$ and $h_{+1}(x) - s \geq \gamma$ respectively. Defining $I_{-1} = [-1, s+ \gamma]$ and $I_{+1} = [s+\gamma, 1]$, then gives that for all $h_{-1} \in \mathcal{H}_{-1}$ and $h_{+1} \in \mathcal{H}_{+1}$, we have $h_{-1}(x) \in I_{-1}$ and $h_{+1}(x) \in I_{+1}$.
% For every measure $\mu \in \Pi(\mathcal{Z})$, we claim that there exist a $\sigma \in \{-1, 1\}$ such that $\mathbb{E}_{z \sim \mu}\left[|\sigma - z| \right] \geq |s - \sigma|$. Suppose for the sake of contradiction that this is not true. That is, there exists $\mu \in \Pi(\mathcal{Z})$ such that for all $\sigma \in \{-1, 1\}$ we have that $\mathbb{E}_{z \sim \mu}\left[|\sigma - z| \right] < |s - \sigma|$. Then, when $\sigma = -1$, we have that $\mathbb{E}_{z \sim \mu}\left[z\right] < |s + 1| - 1$ and when $\sigma = 1$, we have $1 - |s - 1| < \mathbb{E}_{z \sim \mu}\left[z\right]$. Combining the two inequalities together gives that $2 < |s + 1| + |s - 1| = 2$, a contradiction. Accordingly, for every measure $\mu \in \Pi(\mathcal{Z})$, there exists a $\sigma \in \{-1, 1\}$ such that $\mathbb{E}_{z \sim \mu}\left[|\sigma - z| \right] \geq |s - \sigma|$. Next, crucially note that for any $\sigma \in \{\pm1\}$ and any $h_{\sigma} \in \mathcal{H}_{\sigma}$, we have $|h_{\sigma}(x) - \sigma| = |s - \sigma| - |h_{\sigma}(x) - s| \leq |s - \sigma| - \gamma$ by the seq-fat shattering condition from Definition \ref{def:sfat}. Therefore, for every measure $\mu \in \Pi(\mathcal{Z})$, there exists $\sigma \in \{\pm1\}$ such that for all $h_{\sigma} \in \mathcal{H}_{\sigma}$, we have that $\mathbb{E}_{z \sim \mu}\left[|\sigma - z| \right] \geq |\sigma - h_{\sigma}(x)| + \gamma$. Note that if we take $\sigma$ to be a label on the edge indexed by $\mu$, then $\mathbb{E}_{z \sim \mu}\left[|\sigma - z| \right] \geq |\sigma - h_{\sigma}(x)| + \gamma$ exactly matches the shattering condition required by Definition \ref{def:SMdim}.
% To that end, for every measure $\mu \in \Pi(\mathcal{Z})$, add an outgoing edge from $v$ indexed by $\mu$ and labeled by the $\sigma_{\mu} \in \{\pm1\}$ promised in the analysis above. Grab the sub-tree in $\mathcal{T}$ following the original outgoing edge from $v$ indexed by $\sigma_{\mu}$, and append it to the newly constructed outgoing edge from $v$ indexed by $\mu$. Remove the original outgoing edges from $v$ indexed by $-1$ and $+1$ and their corresponding subtrees. Recursively repeat the above procedure on the subtrees following the newly created edges indexed by measures. Upon repeating this process for every internal node in $\mathcal{T}$, we obtain a $\Pi(\mathcal{Z})$-ary tree that is $\gamma$-shattered by $\mathcal{H}$ of depth $d_{\gamma}$. Thus, we have that $\text{seq-fat}_{\gamma}(\mathcal{H}) \leq \text{SMdim}_{\gamma}(\mathcal{H})$.
% \end{proof}
Next, we show that SMdim reduces to $(k+1)$-Ldim from \cite{moran2023list}.
\begin{proposition}{SMdim $\equiv$ $\text{Ldim}_{k+1}$}{}
Let $\mathcal{Z} = \{S: S\subset \mathcal{Y}, |S| \leq k\}$, $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$, and $\ell(y, z) = \mathbbm{1}\{y \notin z\}$. Then for every $\gamma \in [0, \frac{1}{k+1}]$, we have $\text{SMdim}_{\gamma}(\mathcal{H}) = \text{Ldim}_{k+1}(\mathcal{H}).$
\end{proposition}
\begin{proof}
Fix $\gamma \in (0, 1]$. We first show that $\text{SMdim}_{\gamma}(\mathcal{H}) \leq \text{Ldim}_{k+1}(\mathcal{H})$. Let $\mathcal{T}$ be a $\mathcal{X}$-valued, $\Pi(\mathcal{Z})$-ary tree of depth $d_{\gamma} = \text{SMdim}_{\gamma}(\mathcal{H})$ shattered by $\mathcal{H}$. Let $v$ be the root node of $\mathcal{T}$ and $x$ denote the instance labeling the node. Recall that $v$ has an outgoing edge for each measure $\mu \in \Pi(\mathcal{Z})$. Let $\{y_{\mu}\}_{\mu \in \Pi(\mathcal{Z})}$ be the set of elements in $\mathcal{Y}$ that label the outgoing edges from $v$. We first claim that there at least $k+1$ distinct elements in the set $\{y_{\mu}\}_{\mu \in \Pi(\mathcal{Z})}$. For the sake of contradiction, suppose this was not the case. That is, there are only $k$ distinct elements that label the outgoing edges from $v$. Let $y_1, ..., y_k$ denote the $k$ distinct elements that label the outgoing edges from $v$, Consider the measure $\tilde{\mu}$ that puts all mass on $\{y_1, ..., y_k\}$. Note that $\tilde{\mu} \in \Pi(\mathcal{Z})$ and let $\tilde{y} \in \{y_1, ..., y_k\}$ be the label on the outgoing edge from $v$ indexed by $\tilde{\mu}$. By definition of $\tilde{\mu}$ and $\tilde{y}$, it must be the case that $\mathbb{P}_{z \sim \mu}\left[\tilde{y} \notin z \right] = 0$. Since $\gamma > 0$, the shattering condition required by Definition \ref{def:SMdim} cannot be met, which is a contradiction. Accordingly, there at least $k+1$ distinct elements in the set $\{y_{\mu}\}_{\mu \in \Pi(\mathcal{Z})}$.
Let $y_1, ..., y_{k+1}$ be the distinct elements of the set $\{y_{\mu}\}_{\mu \in \Pi(\mathcal{Z})}$, and $\mu_1, ..., \mu_{k+1}$ be the indices of the edges labeled by $y_1, ..., y_{k+1}$ respectively. For $\mu_i \in \{\mu_1, ..., \mu_{k+1}$\}, let $\mathcal{H}_{\mu_i}$ denote the set of shattering hypothesis that corresponds to following a path down $\mathcal{T}$ that takes the outgoing edge $\mu_i$ from the root node. Keep the edges indexed by $\mu_1, ..., \mu_{k+1}$, and remove all other outgoing edges along with their corresponding subtree. Reindex the $k+1$ edges using distinct numbers in $[k+1]$. The root node $v$ should now have $k+1$ outgoing edges, each indexed by a different natural number in $[k+1]$ and labeled by a distinct element of $\mathcal{Y}$, matching the first constraint of a $\text{Ldim}_{k+1}$ tree. As for the second constraint, observe that for all $h \in \mathcal{H}_{\mu_i}$ the shattering condition implies that $\mathbb{P}_{z \sim \mu_i}\left[y_i \notin z \right] \geq \mathbbm{1}\{y_i \notin h(x)\} + \gamma.$ However, this can only be true if $\mathbbm{1}\{y_i \notin h(x)\} = 0 \implies y_i \in h(x)$. Accordingly, the hypotheses that shatter the edges indexed by $\mu_1, ..., \mu_{k+1}$ in the original tree according to Definition \ref{def:SMdim} also shatter the newly re-indexed edges according to Definition \ref{def:kldim}. Recursively repeating the above procedure on the subtrees following the $k+1$ reindexed edges results in a $\text{Ldim}_{k+1}$ tree shattered by $\mathcal{H}$ of depth $d_{\gamma}$. Thus, $\text{SMdim}_{\gamma}(\mathcal{H}) \leq \text{Ldim}_{k+1}(\mathcal{H})$ for $\gamma \in (0, 1]$. The case when $\gamma = 0$ follows similarly and uses the fact that when $\gamma = 0$, we define the shattering condition in SMdim with a strict inequality (see last sentence in Definition \ref{def:SMdim}).
We now prove the inequality that $\text{SMdim}_{\gamma}(\mathcal{H}) \geq \text{Ldim}_k(\mathcal{H})$. Fix $\gamma \in [0, \frac{1}{k+1}]$. Let $\mathcal{T}$ be a $\mathcal{X}$-valued, $[k+1]$-ary tree of depth $d = \text{Ldim}_{k+1}(\mathcal{H})$ shattered by $\mathcal{H}$ according to Definition \ref{def:kldim}. Our goal will be to expand $\mathcal{T}$ into a $\Pi(\mathcal{Z})$-ary tree that is $\gamma$-shattered by $\mathcal{H}$ according to Definition \ref{def:SMdim}. Let $v$ be the root node of $\mathcal{T}$, $x$ be the instance that labels the root node, and $\{y_i\}_{i=1}^{k+1}$ denote the distinct elements of $\mathcal{Y}$ that label the $k+1$ outgoing edges from $v$. For each $i \in [k+1]$, let $\mathcal{H}_i = \{h_p: p \in [k+1]^{d}, p_1 = i\} \subset \mathcal{H}$ denote the set of shattering hypothesis that corresponds to following a path down $\mathcal{T}$ that takes the outgoing edge indexed by $i$ from $v$. Then, for all $i \in [k+1]$ and $h_i \in \mathcal{H}_i$, the shattering condition implies that $y_i \in h_i(x) \implies \mathbbm{1}\{y_i \notin h_i(x)\} = 0$.
For every measure $\mu \in \Pi(\mathcal{Z})$, we claim that there exists a $i_{\mu} \in [k+1]$ such that $\mathbb{P}_{z \sim \mu}\left[y_{i_{\mu}} \notin z \right] \geq \gamma$. Suppose for the sake of contradiction that this is not true. Then, there exists a measure $\mu \in \Pi(\mathcal{Z})$ such that for all $i \in [k+1]$, we have $\mathbb{P}_{z \sim \mu}\left[y_{i} \notin z \right] < \gamma$. This implies that $\mathbb{P}_{z \sim \mu}\left[\exists i \in [k+1] \text{ such that } y_{i} \notin z \right] < (k+1)\gamma < 1$. However, since $\mu$ is supported only over subsets of $\mathcal{Y}$ of size at most $k$, we have $\mathbb{P}_{z \sim \mu}\left[\exists i \in [k+1] \text{ such that } y_{i} \notin z \right] = 1$, a contradiction. Thus, for every measure $\mu \in \Pi(\mathcal{Z})$ there exists a $i_{\mu} \in [k+1]$ such that $\mathbb{P}_{z \sim \mu}\left[y_{i_{\mu}} \notin z \right] \geq \gamma$. Combining this with the fact that for every $i \in [k+1]$ and $h_i \in \mathcal{H}_i$ we have that $y_i \in h_i(x)$ gives that, for every measure $\mu \in \Pi(\mathcal{Z})$, there exists a $i_{\mu} \in [k+1]$ such that for all $h_{i_{\mu}} \in \mathcal{H}_{i_{\mu}}$, we have $\mathbb{P}_{z \sim \mu}\left[y_{i_{\mu}} \notin z \right] \geq \mathbbm{1}\{y_{i_{\mu}} \notin h_{i_{\mu}}(x)\} + \gamma$. Note that if we take $y_{i_{\mu}}$ to be the label on an edge indexed by $\mu$, then the inequality above matches the shattering condition required by Definition \ref{def:SMdim}.
To that end, for every measure $\mu \in \Pi(\mathcal{Z})$, add an outgoing edge from $v$ indexed by $\mu$ and labeled by the $y_{i_{\mu}}$, where $i_{\mu}$ is the index as promised by the analysis above. Grab the sub-tree in $\mathcal{T}$ following the original outgoing edge from $v$ indexed by $i_{\mu}$, and append it to the newly constructed outgoing edge from $v$ indexed by $\mu$. Remove the original outgoing edges from $v$ indexed by numbers in $[k+1]$ and their corresponding subtrees. Recursively repeat the above procedure on the subtrees following the newly created edges indexed by measures. Upon repeating this process for every internal node in $\mathcal{T}$, we obtain a $\Pi(\mathcal{Z})$-ary tree of depth $d$ that is $\gamma$-shattered by $\mathcal{H}$. Thus, we have that $\text{Ldim}_{k+1}(\mathcal{H}) \leq \text{SMdim}_{\gamma}(\mathcal{H})$ for $\gamma \in [0, \frac{1}{k+1}]$.
\end{proof}
Finally, we show that the SMdim $\equiv$ MSdim.
\begin{proposition}{SMdim $\equiv$ MSdim}{}
Let $\mathcal{Y} \subset \sigma(\mathcal{Z})$, $\mathcal{H} \subseteq \mathcal{Z}^{\mathcal{X}}$, and $\ell(y, z) = \mathbbm{1}\{z \notin y\}$. Then for every $\gamma \in [0, 1]$, we have $\text{SMdim}_{\gamma}(\mathcal{H}) = \text{MSdim}_{\gamma}(\mathcal{H}).$
\end{proposition}
\begin{proof}
The equality follows directly from the fact that $\mathbb{E}_{z \sim \mu}\left[\ell(y, z)\right] = \mu(y^c)$ and the fact that $\mathbb{E}_{z \sim \mu_t}\left[\ell(z, f_t(\mu_{\leq t})) \right] \geq \ell(h_{\mu}(\mathcal{T}_t(\mu_{<t})), f_t(\mu_{\leq t})) + \gamma \iff h_{\mu}(\mathcal{T}_t(\mu_{<t})) \in f_t(\mu_{\leq t}) \text{ and } \mu_t(f_t(\mu_{\leq t})) \leq 1 - \gamma$.
\end{proof}
% \subsection{Combinatorial dimensions when $\mathcal{Z} = \mathcal{Y}$}
% The case when $\mathcal{Z} = \mathcal{Y}$ is a fundamental setting in online learning. Many practical applications, including classification and regression, fall under this category. In this section, we study the SMdim when $\mathcal{Z} = \mathcal{Y}$. We prove that the SMdim can simplify to an equivalent, but arguably more concrete, combinatorial dimension. At the heart of this reduction is the Helly number, which for a set system, quantifies the property that every sequence of sets with empty intersection has a small sub-subsequence with empty intersection. Definition \ref{def:helly_num} provides the formal definition of the Helly number.
% \begin{definition}{Helly Number}{helly_num}
% Let $S$ be a family of sets. The Helly number of $S$, denoted $\mathbb{H}(S)$, is the smallest number $p \in \mathbbm{N}$ such that for any collection of sets $C \subseteq S$ whose intersection is empty, there is a subset $C' \subset C$ of size at most $p$ whose intersection is empty.
% \end{definition}
% The Helly number allows us to define the notion of a dimension for an abstract label space. Concretely, given a label space $\mathcal{Y}$ and a loss $\ell: \mathcal{Y} \times \mathcal{Y} \rightarrow [0, 1]$, let $\mathcal{B}_{\ell}(y, r) := \{y' \in \mathcal{Y}: \ell(y, y') \leq r\}$ denote the ball of radius $r$ centered at $y$ induced by the loss $\ell$ and $\mathcal{B}_{\ell}(\mathcal{Y}) := \{\mathcal{B}_{\ell}(y, r): y \in \mathcal{Y}, r \in [0, 1] \}$ to be the set of all such balls in $\mathcal{Y}$. The Helly number of $\mathcal{B}_{\ell}(\mathcal{Y})$ gives us a natural notion of dimension for $\mathcal{Y}$, which we define below.
% \begin{definition}{Helly Space}{hell_space}
% Let $\mathcal{Z} = \mathcal{Y}$, $\ell: \mathcal{Y} \times \mathcal{Y} \rightarrow [0, 1]$, and $\mathcal{B}_{\ell}(\mathcal{Y}) := \{\mathcal{B}_{\ell}(y, r): y \in \mathcal{Y}, r \in [0, 1] \}$. Then, we say $\mathcal{Y}$ is a Helly space with respect to $\ell$ if and only if $\mathbb{H}(\mathcal{B}_{\ell}(\mathcal{Y})) < \infty$. Moreover, we define the Helly number of the label space $\mathcal{Y}$ with respect to $\ell$ as $\mathbb{H}_{\ell}(\mathcal{Y}) := \mathbb{H}(\mathcal{B}_{\ell}(\mathcal{Y}))$.
% \end{definition}
% A label space $\mathcal{Y}$ being a Helly space with respect to $\ell$ allows us to effectively ``compress" the infinite-width, $\Pi(\mathcal{Z})$-ary tree from the definition of SMdim into a finite width, $\left[\mathbb{H}_{\ell}(\mathcal{Y})\right]$-ary tree. To formalize this idea, it will be useful to define the notion of an expansion of a ball. Namely, given any ball $S \in \mathcal{B}_{\ell}(\mathcal{Y})$ and $\gamma \geq 0$, we let $\mathcal{E}(S, \gamma)$ denote the $\gamma$-expansion of the ball $S$ around its center. That is, $\mathcal{E}(S, \gamma) \in \mathcal{B}_{\ell}(\mathcal{Y})$ is the ball that shares the same center as $S$, but has a $\gamma$ larger radius. Using this notion of expansion, Definition \ref{def:bldim} gives the Ball Littlestone dimension which compresses the $\Pi(\mathcal{Z})$-ary tree from the definition of SMdim into a $\left[\mathbb{H}_{\ell}(\mathcal{Y})\right]$-ary tree when $\mathcal{Y}$ is a Helly space with respect to $\ell$.
% \begin{definition}{Ball Littlestone dimension}{bldim}
% Let $\ell: \mathcal{Y} \times \mathcal{Y} \rightarrow [0, 1]$ be a loss function and suppose $\mathcal{Y}$ is a Helly space with respect to $\ell$. Let $\mathcal{T}$ be a complete $\mathcal{X}$-valued, $\left[\mathbb{H}_{\ell}(\mathcal{Y})\right]$-ary tree of depth $d$, and fix $\gamma \in (0,1] $. The tree $\mathcal{T}$ is $\gamma$-shattered by $\mathcal{H} \subseteq \Zcal^{\Xcal}$ if there exists a sequence $(f_1, ..., f_d)$ of edge-labeling functions $f_t: \left[\mathbb{H}_{\ell}(\mathcal{Y})\right]^{t} \rightarrow \mathcal{B}_{\ell}(\mathcal{Y})$ such that for every path $p = (p_1, ..., p_d) \in \left[\mathbb{H}_{\ell}(\mathcal{Y})\right]^d$, there exists a hypothesis $h_{p} \in \mathcal{H}$ such that for all $t \in [d]$, $h_{p}(\mathcal{T}_t(p_{<t})) \in f_t(p_{\leq t})$ and $\bigcap_{i \in [\mathbb{H}(\mathcal{Y})]} \mathcal{E}(f_t((p_{< t},i)), \gamma) = \emptyset$. The Ball Littlestone dimension of $\mathcal{H}$ at scale $\gamma$, denoted $\text{BLdim}_{\gamma}(\mathcal{H}, \ell)$, is the maximal depth of a tree $\mathcal{T}$ that is $\gamma$-shattered by $\mathcal{H}$. If there exists $\gamma$-shattered trees of arbitrarily large depth, we say $\text{BLdim}_{\gamma}(\mathcal{H}, \ell) = \infty$.
% \end{definition}
% At a high-level, the BLdim tree is a $[\mathbb{H}_{\ell}(\mathcal{Y})]$-ary tree where each edge is labeled by a ball from $\mathcal{B}_{\ell}(\mathcal{Y})$ but for any internal node $v$, the mutual intersection of the $\gamma$-expanded balls labeling its outgoing edges is empty. Finally, Proposition \ref{prop:rel_SMdim_bldim} relates the SMdim and BLdim of a hypothesis class.
% % In this sense, the SMdim ``reduces" to the BLdim when $\mathcal{Y}$ is a Helly space. In Section [blank], we show that for many natural problems, $\mathcal{Y}$ is a Helly space with respect to $\ell$. As a result, the BLdim also provides a characterizes of online learnability for these settings.
% \begin{proposition}{Relations between SMdim and BLdim}{rel_SMdim_bldim}
% Suppose that $\mathcal{Z} = \mathcal{Y}$ and that $\mathcal{Y}$ is a Helly space with respect to $\ell: \mathcal{Y} \times \mathcal{Y} \rightarrow [0,1]$. Then,
% \begin{itemize}
% \item[(i)] $\text{SMdim}_{\gamma}(\mathcal{H}) \leq \text{BLdim}_{\gamma'}(\mathcal{H})$ for all $\gamma' < \gamma$
% \item[(ii)] If $\ell$ is a metric and $\mathbb{H}_{\ell}(\mathcal{Y}) = 2$, then $\text{BLdim}_{\gamma}(\mathcal{H}) \leq \text{SMdim}_{\gamma}(\mathcal{H})$
% \end{itemize}
% \end{proposition}
% \begin{proof} (of (i) in Proposition \ref{prop:rel_SMdim_bldim})
% Let $p = \mathbb{H}_{\ell}(\mathcal{Y})$. Fix $\gamma \in (0, 1]$ and $\gamma' < \gamma$. Let $\mathcal{T}$ be a $\mathcal{X}$-valued, $\Pi(\mathcal{Z})$-ary tree of depth $d_{\gamma} = \text{SMdim}_{\gamma}(\mathcal{H})$ shattered by $\mathcal{H}$ Let $v$ be the root node of $\mathcal{T}$ and $x$ denote the instance labeling $v$. Recall that $v$ has an outgoing edge for each measure $\mu \in \Pi(\mathcal{Z})$. In particular, this means that $v$ has outgo edges corresponding to the Dirac measures on $\mathcal{Z}$, which we denote by $\{\delta_z\}_{z \in \mathcal{Z}}$. Fix a $z \in \mathcal{Z}$ and consider the outgoing edge from $v$ indexed by $\delta_z$. Let $y_z \in \mathcal{Y}$ be the element that labels the outgoing edge indexed by $\delta_z$. Let $\mathcal{H}_{\delta_z} = \{h_{\mu}: \mu \in \Pi(\mathcal{Z})^{d_{\gamma}}, \mu_1 = \delta_z\} \subset \mathcal{H}$ denote the set of shattering hypothesis that corresponds to following a path down $\mathcal{T}$ that takes the edge $\delta_z$ in the first level. Then, for all $h \in \mathcal{H}_{\delta}$, the shattering condition implies that
% $$\ell(y_z, z) \geq \ell(y_z, h(x)) + \gamma > \ell(y_z, h(x)) + \gamma'.$$
% Taking the supremum on both sides further gives that
% $$\ell(y_z, z) > \sup_{h \in \mathcal{H}_{\delta_z}}\ell(y_z, h(x)) + \gamma'.$$
% Let $B_z := \mathcal{B}(y_z, \sup_{h \in \mathcal{H}_{\delta_z}}\ell(y_z, h(x)) + \gamma') \in \mathcal{B}_{\ell}(\mathcal{Y})$ be the ball centered around $y_z$ of radius $\sup_{h \in \mathcal{H}_{\delta_z}}\ell(y_z, h(x)) + \gamma'$. The inequality above implies that $z \notin B_z$ (note that $B_z$ changes depending on $z$). Since $z \in \mathcal{Z}$ was arbitrary, it must be the case that $z \notin B_z$ for all $z \in \mathcal{Z}$. This means that $\bigcap_{z \in \mathcal{Z}}B_z = \emptyset$. Then, using the fact that $\mathcal{Y}$ is a Helly space with Helly number $p$, there exists $p$ balls in $\{B_z\}_{z \in \mathcal{Z}}$ such that their collection-wise intersection is also empty. Accordingly, let $z_1, ..., z_p$ be such that $\bigcap_{i=1}^p B_{z_i} = \emptyset$. As before, for every $i \in [p]$, let $y_{z_i}$ denote the label on the outgoing edge from $v$ indexed by the Dirac measure $\delta_{z_i}$. By definition, for all $i \in [p]$ and $h_{\delta_{z_i}} \in \mathcal{H}_{\delta_{z_i}}$ we have that
% $$h_{\delta_{z_i}}(x) \in \mathcal{B}(y_{z_i}, \sup_{h \in \mathcal{H}_{\delta_{z_i}}}\ell(y_{z_i}, h(x))) := \tilde{B}_{z_i}$$
% Note that $B_{z_i}$ is the $\gamma'$ expansion of $\tilde{B}_{z_i}$. That is, $B_{z_i} = \mathcal{E}(\tilde{B}_{z_i}, \gamma')$. For each $i \in [p]$, relabel the outgoing edge from $v$ indexed by $\delta_{z_i}$ with the ball $\tilde{B}_{z_i}$. For each $i \in [p]$, reindex the outgoing edge from $v$ indexed by $\delta_{z_i}$ with $i$. Remove all other edges indexed by measures and their corresponding subtrees. There should now only be $p$ outgoing edges from $v$, each indexed by a number $i \in [p]$ and labeled by the ball $\tilde{B}_{z_i} \in \mathcal{B}_{\ell}(\mathcal{Y})$. Note that $\bigcap_{i=1}^p \mathcal{E}(\tilde{B}_{z_i}, \gamma') = \bigcap_{i=1}^p B_{z_i} = \emptyset$, which matches the second constraint imposed by Definition \ref{def:bldim}. As for the first constraint on shattering, note that for all $i \in [p]$ and all $h_{\delta_{z_i}} \in \mathcal{H}$, we have that $h_{\delta_{z_i}}(x) \in \tilde{B}_{z_i}$. Thus, the hypothesis that shatters the edges indexed by $\delta_{z_i}$ in the original tree according to Definition \ref{def:SMdim} also shatter the newly re-indexed and relabeled edges according to Definition \ref{def:bldim}. Thus, for the root node $v$, both constraints imposed by Definition \ref{def:bldim} are met. Recursively repeating the above procedure on the subtrees following the $p$ newly reindexed and relabeled edges results in a BLdim tree $\gamma'$-shattered by $\mathcal{H}$ of depth $d_{\gamma}$. Thus, $\text{SMdim}_{\gamma}(\mathcal{H}) \leq \text{BLdim}_{\gamma'}(\mathcal{H})$ for $\gamma' < \gamma$.
% \end{proof}
% \begin{proof}(of part (ii) in Proposition \ref{prop:rel_SMdim_bldim}) We now move on to prove the lower bound. Suppose $\ell$ is a metric and $p = \mathbb{H}_{\ell}(\mathcal{Y}) = 2$. Let $\gamma \in (0, 1]$ and $\mathcal{T}$ be a $\mathcal{X}$-valued, $[2]$-ary tree of depth $d_{\gamma} = \text{BLdim}_{\gamma}(\mathcal{H})$ shattered by $\mathcal{H}$ according to Definition \ref{def:bldim}. Our goal will be to expand $\mathcal{T}$ into a $\Pi(\mathcal{Z})$-ary tree that is $\gamma$-shattered by $\mathcal{H}$ according to Definition \ref{def:SMdim}. Let $v$ be the root node and $x$ be the instance that labels the root node. For every $i \in [2]$, $\mathcal{B}(y_i, r_i) \in \mathcal{B}_{\ell}(\mathcal{Y})$ be the ball that labels the outgoing edge from $v$ indexed by $i$. Here, we are letting $y_i$ and $r_i$ denote the center and radius of the ball labeling the outgoing edge from $v$ indexed by $i$. For each $i \in [2]$, let $\mathcal{H}_i \subset \mathcal{H}$ denote the set of shattering hypothesis that corresponds to following a path down $\mathcal{T}$ that takes the edge indexed by $i$ in the first level. Then, for all $h_i \in \mathcal{H}_i$, the shattering condition implies that $h_i(x) \in \mathcal{B}(y_i, r_i)$ but $\mathcal{E}(\mathcal{B}(y_1, r_1), \gamma) \cap \mathcal{E}(\mathcal{B}(y_2, r_2), \gamma) = \emptyset$. Note that this implies that $\ell(y_1, y_2) \geq r_1 + r_2 + 2\gamma$.
% For every measure $\mu \in \Pi(\mathcal{Z})$, we claim that there exists a $i \in [2]$ such that $\mathbb{E}_{z \sim \mu}\left[\ell(y_i, z) \right] \geq r_i + \gamma$. Suppose for the sake of contradiction that this is not the case. Then, there exists $\mu \in \Pi(\mathcal{Z})$ such that for all $i \in [2]$, we have that $\mathbb{E}_{z \sim \mu}\left[\ell(y_i, z) \right] < r_i + \gamma$. Combining the two cases gives that $\mathbb{E}_{z \sim \mu}\left[\ell(y_1, z) + \ell(y_2, z) \right] < r_1 + r_2 + 2\gamma$. However, since $\ell$ is a metric, by the reverse triangle inequality, we have that $\ell(y_1, y_2) \leq \ell(y_1, z) + \ell(y_2, z)$. Therefore, we get that $\ell(y_1, y_2) < r_1 + r_2 + 2\gamma$, which contradicts the fact that $\mathcal{E}(\mathcal{B}(y_1, r_1), \gamma) \cap \mathcal{E}(\mathcal{B}(y_2, r_2), \gamma) = \emptyset$. Accordingly, for every measure $\mu \in \Pi(\mathcal{Z})$, there exists a $i_{\mu} \in [2]$ such that $\mathbb{E}_{z \sim \mu}\left[\ell(y_{i_{\mu}}, z) \right] \geq r_{i_{\mu}} + \gamma$. Next, note that for any $i \in [2]$ and any $h_i \in \mathcal{H}_i$, we have that $\ell(y_i, h_i(x)) \leq r_i$ by the BLdim shattering condition from Definition \ref{def:bldim}. Therefore, for every measure for every measure $\mu \in \Pi(\mathcal{Z})$, there exists a $i_{\mu} \in [2]$ such that for all $h_{i_{\mu}} \in \mathcal{H}_{i_{\mu}}$, we have that $\mathbb{E}_{z \sim \mu}\left[\ell(y_{i_{\mu}}, z) \right] \geq \ell(y_{i_{\mu}}, h_{i_{\mu}}(x)) + \gamma$. Thus, if we take $y_{i_{\mu}}$ to be the label on an edge indexed by $\mu$, then $\mathbb{E}_{z \sim \mu}\left[\ell(y_{i_{\mu}}, z) \right] \geq \ell(y_{i_{\mu}}, h_{i_{\mu}}(x)) + \gamma$ exactly matches the shattering condition required by Definition \ref{def:SMdim}.
% To that end, for every measure $\mu \in \Pi(\mathcal{Z})$, add an outgoing edge from $v$ indexed by $\mu$ and labeled by the $i_{\mu} \in [2]$ promised in the analysis above. Grab the sub-tree in $\mathcal{T}$ following the original outgoing edge from $v$ indexed by $i_{\mu}$, and append it to the newly constructed outgoing edge from $v$ indexed by $\mu$. Remove the original outgoing edges from $v$ indexed by $1$ and $2$ and their corresponding subtrees. Recursively repeat the above procedure on the subtrees following the newly created edges indexed by measures. Upon repeating this process for every internal node in $\mathcal{T}$, we obtain a $\Pi(\mathcal{Z})$-ary tree that is $\gamma$-shattered by $\mathcal{H}$ of depth $d_{\gamma}$. Thus, we have that $\text{BLdim}_{\gamma}(\mathcal{H}) \leq \text{SMdim}_{\gamma}(\mathcal{H})$. \end{proof}
% Taking (i) and (ii) in Proposition \ref{prop:rel_SMdim_bldim} together implies that when $\ell$ is a metric and $\mathbb{H}_{\ell}(\mathcal{Y}) = 2$, we have that BLdim $\equiv$ SMdim. In other words, when $\ell$ is a metric and $\mathbb{H}_{\ell}(\mathcal{Y}) = 2$, we can compress the infinite-width tree that is the SMdim tree into a binary tree that is the BLdim tre. We highlight that both classification with the 0-1 loss and scalar-valued regression with the absolute value loss are both cases where $\ell$ is a metric and $\mathbb{H}_{\ell}(\mathcal{Y}) = 2$. In fact, its not hard to see that the BLdim reduces to both the Ldim and seq-fat in classification and regression respectively. In this sense, the binary nature of the Ldim and seq-fat can perhaps be explained by the fact that in both cases, the Helly number of its label space, with respect to the appropriate loss, is two.
\section{Applications}
In this section, we discuss some applications of our general result presented in Section \ref{sec:agnostic}. For instance, taking $\Ycal = \Zcal$ and $\ell(y, z)
= \indicator\{y \neq z\}$ defines standard multiclass classification. As pointed out in Proposition \ref{prop:GMS=Ldim}, the sequential Minimax dimension reduces to the Littlestone dimension. Similarly, taking $\Ycal = \Zcal = [-1,1]$ and $\ell(y,z) = |y-z|$, Proposition \ref{prop:fateq} shows the equivalence between SMdim and sequential fat shattering dimension. Quantitative characterizations of multiclass classification and scalar-valued regression are well established in the literature. However, our results in Section \ref{sec:agnostic} can also be used to fully characterize the online learnability of vector-valued regression and multilabel classification, which was not known previously.
We point out that \cite{raman2023characterization} consider the problem of multioutput regression with $L_p$ norm losses and multilabel classification with normalized Hamming loss. For multioutput regression, they consider the case $\Ycal =[-1,1]^K $. Given any $\Hcal \subseteq \Ycal^{\Xcal}$, they show that the finiteness of the sequential fat-shattering dimension of each coordinate output restriction $\Hcal_i \subseteq [-1,1]^{\Xcal}$ is necessary and sufficient for learnability. This coordinate-wise characterization results in both their upper and lower bounds on regret depending on $K$, thus preventing their techniques from generalizing to infinite dimensional label spaces. Accordingly, they left it open to characterize the learnability when $\Ycal$ is infinite-dimensional. Moreover, they asked whether there exists a combinatorial dimension of the multioutput function class that provides a sharp quantitative characterization of learnability. We resolve both open problems in this work and settle the question of learnability for vector-valued regression. As for multilabel classification with Hamming loss, \cite{raman2023characterization} characterize learnability of $\Hcal \subseteq(\{0,1\}^K)^{\Xcal}$ in terms of the Littlestone dimension of each $\Hcal_i \subseteq \{0,1\}^{\Xcal}$. Their characterization only works for small $K$, namely when $K = o(\sqrt{T})$. Furthermore, \cite{raman2023online} studies multilabel classification with the thresholded Hamming loss, $\ell(y,z) = \indicator\left\{\sum_{i=1}^K \indicator\{y^i \neq z^i\} \geq q \right\}$ for $q \in [K-1]$, and provides a $K$-independent characterization of learnability for this $q$-insensitive model. Extending \cite{raman2023characterization, raman2023online}, this work resolves the question of learnability for multilabel classification with Hamming loss for arbitrary $K \in \naturals$.
\subsection{ Online Vector-Valued Regression}\label{sec:vec_reg}
Let $(\mathcal{V}, \norm{\cdot})$ be a normed vector space and define $\Zcal = \Ycal := \{v \in \mathcal{V}: \, \norm{v} \leq 1\}$ to be the unit ball centered around origin in $\mathcal{V}$. Note that $\Ycal$ is infinite-dimensional if $\mathcal{V}$ is infinite-dimensional. We consider the loss function to be $\ell(y, z) := \norm{y-z}^p$ for $p \in [1, \infty]$. Clearly, $\sup_{y, z \in \Ycal} \ell(y,z) \leq 2^p $. The problem of vector-valued regression is an extremely general framework capturing a wide range of regression problems. For instance, it captures multioutput regression on $\reals^K$ with $\norm{\cdot}_p$ norms as well as their $p^{th}$ power \citep{raman2023characterization} as well as function-valued regression \citep{morris2015functional}. As an example, consider $ (\mathcal{V}, \norm{\cdot}) = ( C_{\reals}(\Omega), \norm{\cdot}_{\infty})$, a Banach space of all continuous real-valued functions defined on a topological space $\Omega$ equipped with the norm $\norm{f}_{\infty} = \sup_{\omega \in \Omega}|f(w)|$. One may also consider the problem of operator learning when both $\Xcal$ and $\Ycal$
are subsets of vector spaces and the hypothesis class is a subset of linear operators between $\Xcal$ and $\Ycal$ \citep{tabaghi2019learning, sridharan2010convex}.
Theorem \ref{thm:agn} immediately implies that a vector-valued hypothesis class $\Hcal \subseteq \Ycal^{\Xcal}$ is learnable if and only if $\text{SMdim}_{\gamma}(\Hcal) < \infty$ for all $\gamma > 0$. Next, we use Theorem \ref{thm:agn} and Lemma \ref{lem:suff_sqrt} to derive quantitative regret bounds.
\begin{corollary}{Characterization of Learnability for Vector-Valued Regression }{vec_cor}
Let $\Hcal \subseteq \Ycal^{\Xcal}$ and $\ell(y, z) = \norm{y-z}^p$. Then,
\begin{itemize}
\item[(i)] There exists a randomized online learner whose expected regret on any stream is at most
$$\inf_{\gamma > 0}\, \left\{2^p\, \cdot\text{SMdim}_{\gamma}(\mathcal{H}) + \gamma T + 1 + 2^{p+1}\sqrt{\,\text{SMdim}_{\gamma} (\mathcal{H})\,T \ln(2^{p+1}\,T)}\right\}.$$
\item[(ii)] Define $\eta := \max_{(x, h_1, h_2) \in \Xcal \times \Hcal \times \Hcal} \,\,\ell(h_1(x), h_2(x))$. For every randomized online learner, there exists a stream such that the expected regret of the learner is at least
\[\begin{cases}
\max\left(\eta \sqrt{\frac{T}{8}}\, , \, \sup_{\gamma > 0 } \, \gamma\, \text{SMdim}_{\gamma}(\mathcal{H}) \right)\quad & \quad \text{ if } p=1,\\
\sup_{\gamma > 0 } \, \gamma\, \text{SMdim}_{\gamma}(\mathcal{H}) \quad & \quad \text{ otherwise}.
\end{cases}\]
\end{itemize}
\end{corollary}
\noindent Note that (i) follows immediately from Theorem \ref{thm:agn}. So, we only focus on proving (ii) here.
\begin{proof}(of (ii) in Corollary \ref{cor:vec_cor})
The lower bound of $\sup_{\gamma > 0} \, \gamma \text{SMdim}_{\gamma}(\Hcal)$ is immediate from Theorem \ref{thm:agn} as well. Thus, it suffices to establish the lower bound of $\eta \sqrt{\frac{T}{8}}$ for $p=1$. In view of Lemma \ref{lem:suff_sqrt}, it amounts to finding $y_{-1}, y_{+1}$ such that the conditions outlined in the Lemma are satisfied. Let $h_{-1}, h_{+1}, x$ be such that $\ell(h_{-1}(x), h_{+1}(x)) = \eta > 0$. We can always find $h_{-1}, h_{+1}, x$ for any non-trivial hypothesis class where $|\Hcal| \geq 2$. As $\Ycal = \Zcal$, we can pick $y_{-1} = h_{-1}(x)$ and $y_{+1} = h_{+1}(x)$. Then, condition (i) is satisfied because
\[\min_{\sigma \in \{\pm1\}} \Big(\ell(y_{-\sigma}, h_{\sigma}(x)) - \ell(y_{\sigma}, h_{\sigma}(x))\Big) = \ell(h_{-1}(x), h_{+1}(x)) = \eta > 0.\]
As for condition (ii), we can use the triangle inequality to obtain
\[\inf_{z \in \Zcal}\Big( \ell(y_{-1}, z) + \ell(y_{+1}, z)\Big) \geq \ell(y_{-1}(x), y_{+1}(x)) = \frac{1}{2} \sum_{\sigma_1, \sigma_2} \ell(y_{\sigma_1}, h_{\sigma_2}(x)), \]
where the last equality follows because $\ell(y_{\sigma_i}, h_{\sigma_i}(x)) =0$ for $\sigma_i \in \{-1,1\}$ and $\ell(y_{-\sigma_i}, h_{\sigma_i}(x)) = \ell(y_{\sigma_i}, h_{-\sigma_i}(x)) = \ell(y_{-1}(x), y_{+1}(x))$ for $\sigma_i \in \{-1,1\}$.
This completes our proof.
\end{proof}
For general $p$ (when the loss is not a distance metric), we provide an example showing that the lower bound of $\sup_{\gamma > 0} \gamma \,\text{SMdim}_{\gamma}(\Hcal)$ is tight up to logarithmic factors in $T$. This also shows that the lower bound of $\Omega(\sqrt{T})$ is not always possible. Let $\mathcal{V}$ be a Hibert space and $\Ycal=\Zcal$ be the usual unit ball in $\mathcal{V}$. Define $\Hcal = \{v \in \mathcal{V}: \norm{v} \leq 1\}$ to be a set of all constant functions mapping to the vectors in the unit ball, and consider the loss function $\ell(y,z) = \norm{y-z}^2$. Let $e_1, e_2 $ be two orthonormal bases in $\mathcal{V}$. Since $\norm{e_i} =1$, we must have $e_i \in \Ycal$. It is easy to see that $\text{SMdim}_{\frac{1}{2}}(\Hcal) \geq 1$. This is true because, for every $\mu \in \Pi(\Zcal)$, there exists $y \in \{e_1, e_2\}$ such that $\expect_{z \sim \mu}\left[\norm{y - z}\right] \geq \frac{1}{\sqrt{2}}$. Using Jensen's, we further obtain $\expect_{z \sim \mu}\left[\norm{y - z}^2\right] \geq \left(\expect_{z \sim \mu}\left[\norm{y - z}\right] \right)^2 \geq \frac{1}{2}$. Suppose, for the sake of contradiction, there exists $\mu \in \Pi(\Zcal)$ such that $\expect_{z \sim \mu}[\norm{e_1-z}] < \frac{1}{\sqrt{2}}$ and $\expect_{z \sim \mu}[\norm{e_2-z}] < \frac{1}{\sqrt{2}}$. Upon adding these two inequalities and using reverse triangle inequality, we obtain $\norm{e_1 - e_2} < \sqrt{2}$, a contradiction as $\norm{e_1 - e_2} = \sqrt{2}$. Therefore, Theorem \ref{thm:agn} implies the lower bound of $\sup_{\gamma > 0} \gamma \,\text{SMdim}_{\gamma}(\Hcal) \geq \frac{1}{2}$ for this problem. However, it is well known that the regret of the Follow The Leader algorithm is $\leq 8(1+ \ln(T))$ (see \cite[Page 43]{cesa2006prediction} for the proof), showing that our lower bound is tight up to $O(\ln(T))$.
Next, consider a particular Hilbert space $\mathcal{V} = \reals$ equipped with inner product $\langle y, z \rangle := yz$ and take $\Ycal = \Zcal = [-1,1]$. Then, the squared norm loss reduces to $ \norm{y-z}^2 = \inner{y-z}{y-z} = (y-z)^2$, the usual squared loss for scalar-valued regression. Thus, the result in \cite{cesa2006prediction} shows that one cannot obtain a lower bound of $\Omega(\sqrt{T})$ even for scalar-valued regression with squared loss. This is in contrast to scalar-valued regression with absolute value loss where one can obtain a lower bound of $\Omega(\sqrt{T})$ for any non-trivial hypothesis class where $|\Hcal| \geq 2$. Moreover, one can verify that the squared loss does not satisfy the conditions of Lemma \ref{lem:suff_sqrt} whereas the absolute value loss does. This highlights how sensitive quantitative characterizations are to the loss function.
% \subsubsection{Examples}
% To be written.
\subsection{Online Multilabel Classification}
For $K \in \naturals$, let $\Ycal = \Zcal = \{0,1\}^K$. Define $\ell(y,z) := \frac{1}{K}\sum_{i=1}^K \indicator\{y^i \neq z^i\}$, the normalized Hamming distance between $y$ and $z$. As before, Theorem \ref{thm:agn} immediately implies that a multilabel hypothesis class $\Hcal \subseteq \Ycal^{\Xcal}$ is learnable if and only if $\text{SMdim}_{\gamma}(\Hcal) < \infty$ for all $\gamma > 0$. We now use Theorem \ref{thm:agn} and Lemma \ref{lem:suff_sqrt} to derive quantitative regret bounds.
\begin{corollary}{Characterization of Learnability for Multilabel Classification}{vec_cor}
Let $\Hcal \subseteq \Ycal^{\Xcal}$ and $\ell(y, z) = \frac{1}{K}\sum_{i=1}^K \indicator\{y^i \neq z^i\}$. Then,
\begin{itemize}
\item[(i)] There exists a randomized online learner whose expected regret on any stream is at most
$$\inf_{\gamma > 0 }\, \left\{\text{SMdim}_{\gamma}(\mathcal{H}) + \gamma T + 1 + 2\,\sqrt{\text{SMdim}_{\gamma} (\mathcal{H})\,T \ln(T)}\right\}.$$
\item[(ii)] Define $\eta := \sup_{(x, h_1, h_2) \in \Xcal \times \Hcal \times \Hcal} \,\,\ell(h_1(x), h_2(x))$. For every randomized online learner, there exists a stream such that the expected regret of the learner is at least
\[\max\left\{\eta \sqrt{\frac{T}{8}}\, , \, \sup_{\gamma > 0} \, \gamma\, \text{SMdim}_{\gamma}(\mathcal{H}) \right\}\]
\end{itemize}
\end{corollary}
\begin{proof}
The proof of (i) and the lower bound of $ \sup_{\gamma > 0 } \, \gamma\, \text{SMdim}_{\gamma}(\mathcal{H})$ in (ii) is immediate from Theorem \ref{thm:agn}. The lower bound of $\eta \sqrt{\frac{T}{8}}$ is identical to the proof of Corollary \ref{cor:vec_cor} and uses the fact that $\Ycal = \Zcal $ and $\ell$ is a distance metric. We leave the details to the reader.
\end{proof}
% \subsubsection{Examples}
% To be written.
\bibliographystyle{plainnat}
\bibliography{references}
\end{document}
\begin{thebibliography}{19}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
\providecommand{\doi}[1]{doi: #1}\else
\providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi
\bibitem[Ben-David et~al.(2009)Ben-David, P{\'a}l, and
Shalev-Shwartz]{ben2009agnostic}
Shai Ben-David, D{\'a}vid P{\'a}l, and Shai Shalev-Shwartz.
\newblock Agnostic online learning.
\newblock In \emph{COLT}, volume~3, page~1, 2009.
\bibitem[Cesa-Bianchi and Lugosi(2006)]{cesa2006prediction}
Nicolo Cesa-Bianchi and G{\'a}bor Lugosi.
\newblock \emph{Prediction, learning, and games}.
\newblock Cambridge university press, 2006.
\bibitem[Daniely et~al.(2011)Daniely, Sabato, Ben-David, and
Shalev-Shwartz]{DanielyERMprinciple}
Amit Daniely, Sivan Sabato, Shai Ben-David, and Shai Shalev-Shwartz.
\newblock Multiclass learnability and the erm principle.
\newblock In Sham~M. Kakade and Ulrike von Luxburg, editors, \emph{Proceedings
of the 24th Annual Conference on Learning Theory}, volume~19 of
\emph{Proceedings of Machine Learning Research}, pages 207--232, Budapest,
Hungary, 09--11 Jun 2011. PMLR.
\bibitem[Eckhoff(1993)]{ECKHOFF1993389}
Jürgen Eckhoff.
\newblock Helly, {R}adon, and {C}arathéodory {T}ype {T}heorems.
\newblock In \emph{Handbook of Convex Geometry}, pages 389--448. North-Holland,
Amsterdam, 1993.
\newblock ISBN 978-0-444-89596-7.
\newblock URL
\url{https://www.sciencedirect.com/science/article/pii/B9780444895967500171}.
\bibitem[Filmus et~al.(2023)Filmus, Hanneke, Mehalel, and
Moran]{filmus2023optimal}
Yuval Filmus, Steve Hanneke, Idan Mehalel, and Shay Moran.
\newblock Optimal prediction using expert advice and randomized littlestone
dimension, 2023.
\bibitem[Hanneke et~al.(2023)Hanneke, Moran, Raman, Subedi, and
Tewari]{hanneke2023multiclass}
Steve Hanneke, Shay Moran, Vinod Raman, Unique Subedi, and Ambuj Tewari.
\newblock Multiclass online learning and uniform convergence.
\newblock \emph{Proceedings of the 36th Annual Conference on Learning Theory
(COLT)}, 2023.
\bibitem[Littlestone(1987)]{Littlestone1987LearningQW}
Nick Littlestone.
\newblock Learning quickly when irrelevant attributes abound: A new
linear-threshold algorithm.
\newblock \emph{Machine Learning}, 2:\penalty0 285--318, 1987.
\bibitem[Moran et~al.(2023)Moran, Sharon, Tsubari, and
Yosebashvili]{moran2023list}
Shay Moran, Ohad Sharon, Iska Tsubari, and Sivan Yosebashvili.
\newblock List online classification.
\newblock \emph{arXiv preprint arXiv:2303.15383}, 2023.
\bibitem[Morris(2015)]{morris2015functional}
Jeffrey~S Morris.
\newblock Functional regression.
\newblock \emph{Annual Review of Statistics and Its Application}, 2:\penalty0
321--359, 2015.
\bibitem[Radon(1921)]{radon1921mengen}
Johann Radon.
\newblock Mengen konvexer k{\"o}rper, die einen gemeinsamen punkt enthalten.
\newblock \emph{Mathematische Annalen}, 83\penalty0 (1-2):\penalty0 113--115,
1921.
\bibitem[Rakhlin et~al.(2015{\natexlab{a}})Rakhlin, Sridharan, and
Tewari]{rakhlin2015online}
Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari.
\newblock Online learning via sequential complexities.
\newblock \emph{J. Mach. Learn. Res.}, 16\penalty0 (1):\penalty0 155--186,
2015{\natexlab{a}}.
\bibitem[Rakhlin et~al.(2015{\natexlab{b}})Rakhlin, Sridharan, and
Tewari]{rakhlin2015sequential}
Alexander Rakhlin, Karthik Sridharan, and Ambuj Tewari.
\newblock Sequential complexities and uniform martingale laws of large numbers.
\newblock \emph{Probability theory and related fields}, 161:\penalty0 111--153,
2015{\natexlab{b}}.
\bibitem[Rakhlin et~al.(2012)Rakhlin, Shamir, and
Sridharan]{rakhlin_relax_n_randomize}
Sasha Rakhlin, Ohad Shamir, and Karthik Sridharan.
\newblock Relax and randomize : From value to algorithms.
\newblock In F.~Pereira, C.J. Burges, L.~Bottou, and K.Q. Weinberger, editors,
\emph{Advances in Neural Information Processing Systems}, volume~25. Curran
Associates, Inc., 2012.
\newblock URL
\url{https://proceedings.neurips.cc/paper_files/paper/2012/file/53adaf494dc89ef7196d73636eb2451b-Paper.pdf}.
\bibitem[Raman et~al.(2023{\natexlab{a}})Raman, Subedi, and
Tewari]{raman2023characterization}
Vinod Raman, Unique Subedi, and Ambuj Tewari.
\newblock A characterization of multioutput learnability.
\newblock \emph{arXiv cs.{LG}}, 2023{\natexlab{a}}.
\newblock preprint arXiv:2303.17716.
\bibitem[Raman et~al.(2023{\natexlab{b}})Raman, Subedi, and
Tewari]{raman2023online}
Vinod Raman, Unique Subedi, and Ambuj Tewari.
\newblock Online learning with set-valued feedback.
\newblock \emph{arXiv preprint arXiv:2306.06247}, 2023{\natexlab{b}}.
\bibitem[Shalev-Shwartz and Ben-David(2014)]{ShwartzDavid}
Shai Shalev-Shwartz and Shai Ben-David.
\newblock \emph{Understanding Machine Learning: From Theory to Algorithms}.
\newblock Cambridge University Press, USA, 2014.
\bibitem[Sridharan and Tewari(2010)]{sridharan2010convex}
Karthik Sridharan and Ambuj Tewari.
\newblock Convex games in banach spaces.
\newblock In \emph{COLT}, pages 1--13. Citeseer, 2010.
\bibitem[Tabaghi et~al.(2019)Tabaghi, de~Hoop, and
Dokmani{\'c}]{tabaghi2019learning}
Puoya Tabaghi, Maarten de~Hoop, and Ivan Dokmani{\'c}.
\newblock Learning schatten--von neumann operators.
\newblock \emph{arXiv preprint arXiv:1901.10076}, 2019.
\bibitem[Vapnik and Chervonenkis(1971)]{vapnik1971uniform}
Vladimir~Naumovich Vapnik and Aleksei~Yakovlevich Chervonenkis.
\newblock On uniform convergence of the frequencies of events to their
probabilities.
\newblock \emph{Teoriya Veroyatnostei i ee Primeneniya}, 16\penalty0
(2):\penalty0 264--279, 1971.
\end{thebibliography}