Adversarial Tradeoffs in Robust State Estimation

Extracted source text · arXiv 2111.08864

% \documentclass[twoside]{article}
\documentclass[11pt]{article}
\parskip 0.1in

\usepackage{geometry}
\geometry{margin=1in}
\setlength\parindent{0pt}

% \usepackage{aistats2022}
% If your paper is accepted, change the options for the package
% aistats2022 as follows:
%
%\usepackage[accepted]{aistats2022}
%
% This option will print headings for the title of your paper and
% headings for the authors names, plus a copyright note at the end of
% the first column of the first page.

% If you set papersize explicitly, activate the following three lines:
%\special{papersize = 8.5in, 11in}
%\setlength{\pdfpageheight}{11in}
%\setlength{\pdfpagewidth}{8.5in}

% If you use natbib package, activate the following three lines:
%\usepackage[round]{natbib}
%\renewcommand{\bibname}{References}
%\renewcommand{\bibsection}{\subsubsection*{\bibname}}

% If you use BibTeX in apalike style, activate the following line:
%\bibliographystyle{apalike}


\usepackage{Shorthands}

\title{Adversarial Tradeoffs in Linear Inverse Problems and Robust State Estimation}
% \author[1]{Bruce Lee}
% \author[1]{Thomas Zhang}
% \author[1]{Hamed Hassani}
% \author[1]{Nikolai Matni}

\author{Bruce D.\ Lee\thanks{Equal contribution} , Thomas T.C.K.\ Zhang\footnotemark[1] , Hamed Hassani, and Nikolai Matni}

\date{Department of Electrial and Systems Engineering, University of Pennsylvania}


\begin{document}

% If your paper is accepted and the title of your paper is very long,
% the style will print as headings an error message. Use the following
% command to supply a shorter title of your paper so that it can be
% used as headings.
%
%\runningtitle{I use this title instead because the last one was very long}

% If your paper is accepted and the number of authors is large, the
% style will print as headings an error message. Use the following
% command to supply a shorter version of the authors names so that
% they can be used as headings (for example, use only the surnames)
%
%\runningauthor{Surname 1, Surname 2, Surname 3, ...., Surname n}


%\onecolumn

% \twocolumn[
% \aistatstitle{Adversarial Tradeoffs in Linear Inverse Problems and Robust State Estimation}
% % \aistatstitle{Adversarial Tradeoffs in Robust State Estimation}

% \aistatsauthor{ Bruce Lee \And Thomas Zhang \And  Hamed Hassani \And Nikolai Matni }

% \aistatsaddress{ Institution 1 \And  Institution 2 \And Institution 3 }]

\maketitle

% \aistatstitle{Adversarial Tradeoffs in Robust State Estimation}

% \aistatsauthor{ Bruce Lee \And Thomas Zhang \And  Hamed Hassani \And Nikolai Matni }

% \aistatsaddress{ Institution 1 \And  Institution 2 \And Institution 3 } ]

%YO MOMMA SO FAT THOMAS HAD TO GET RID OF THE TWO COLUMN FORMAT TO FIT HER IN HERE


\begin{abstract}
    %Despite how fundamental linear inverse problems are across science and engineering, general results about their behavior under adversarial perturbations are far and few between. In particular, the resulting minimax optimization does not admit a closed-form solution, thus it is difficult to gauge the risk incurred by introducing adversaries, let alone design a solution robust to them. 
    Adversarially robust training has been shown to reduce the susceptibility of learned models to targeted input data perturbations.  However, it has also been observed that such adversarially robust models suffer a degradation in accuracy when applied to unperturbed data sets, leading to a robustness-accuracy tradeoff.  In this paper, we provide sharp and interpretable characterizations of such robustness-accuracy tradeoffs for linear inverse problems.  In particular, we provide an algorithm to find the optimal adversarial perturbation given data, and develop tight upper and lower bounds on the adversarial loss in terms of the standard (non-adversarial) loss and the spectral properties of the resulting estimator.  Further, motivated by the use of adversarial training in reinforcement learning, we define and analyze the \emph{adversarially robust Kalman Filtering problem.}  We apply a refined version of our general theory to this problem, and provide the first characterization of robustness-accuracy tradeoffs in a setting where the data is generated by a dynamical system.  In doing so, we show a natural connection between a filter's robustness to adversarial perturbation and underlying control theoretic properties of the system being observed, namely the spectral properties of its observability gramian.
    
    %Our bounds are then applied to a familiar instance of linear inverse problems: state estimation of a dynamical system. 
    % We propose adversarial training as a means to design robust state estimators. As opposed to traditional robust state estimation, this approach avoids the conservatism resulting from completely disregarding distributional assumptions. 
    % In particular, we provide analytical and empirical comparisons of the performance of state estimators optimized for nominal settings, versus state estimators optimized for adversarial settings with Gaussian noise \emph{as well as} worst-case norm bounded perturbations. We demonstrate that key system properties, such as observability, control the performance degradation of nominal state estimators when faced with adversarial measurements.
    
    % We propose adversarial training as a means to design robust state estimators. As opposed to traditional robust state estimation, this approach avoids the conservatism resulting from completely disregarding distributional assumptions. We study tradeoffs for this adversarial state estimation problem as a special case of those arising in adversarial least squares. In particular, we provide analytical and empirical comparisons of the performance of state estimators which are optimized to perform well in ``nominal'’ settings with Gaussian noise, versus state estimators optimized to perform well in ``adversarial'’ settings with Gaussian noise \emph{along} with worst-case norm bounded perturbations. We demonstrate that key system properties, such as observability, control the performance degradation of state estimators optimized for the nominal setting when faced with adversarial measurements. Along the way, we develop general, sharp upper and lower bounds on the adversarial loss in least squares, which may be of independent interest. 
\end{abstract}


% To-Do
% Clean up abstract, intro to balance linear inverse narrative with state estimation.
% 


\input{intro_v2}
% \input{commentedLS}
\input{LSproblem-results}
\input{kalman}
\input{techresults}
\input{kalmanBounds}
\input{numericalResults}
\input{discussion}
\input{conclusion}

\section*{Acknowledgements}
Bruce Lee is supported by the Department of Defense through the National Defense Science \& Engineering Graduate Fellowship Program. The research of Hamed Hassani is supported by NSF Grants 1837253, 1943064, 1934876, AFOSR Grant FA9550-20-1-0111, and DCIST-CRA. Nikolai Matni is funded by NSF awards CPS-2038873, CAREER award ECCS-2045834, and a Google Research Scholar award. 

\bibliographystyle{abbrvnat}
\bibliography{refs}

\onecolumn
% \bibliography{refs}{}


\input{appendix}

%\vfill





\end{document}
\section{Introduction}

% The concept of ``adversarial robustness'' has become increasingly prominent in the machine learning community over recent years. From a practical perspective, achieving adversarial robustness is born out of necessity: 

It has been demonstrated across various application areas that contemporary learned models, despite their impressive nominal performance, can be extremely susceptible to small, adversarially designed input perturbations \citep{carlini2016defensive, goodfellow2014explaining, szegedy2013intriguing, huang2017adversarial}.  In order to mitigate the effects of such attacks, various adversarially robust training algorithms \citep{carlini2016defensive, carlini2017towards, madry2017towards, xie2020smooth, deka2020dynamically} have been developed.  However, it was soon noticed that while adversarial training could be used to improve model robustness, it often came with a corresponding decrease in accuracy on nominal (unperturbed) data.  Further, various simplified theoretical models \citep{tsipras2018robustness,zhang2019theoretically,nakkiran2019adversarial,raghunathan2019adversarial,chen2020more, javanmard2020precise}, have been used to explain this phenomena, and to argue that such \emph{robustness-accuracy tradeoffs} are unavoidable.

% % Despite the great deal of effort dedicated to to coming up with adjustments to learning algorithms to achieve adversarial robustness \citep{carlini2016defensive, carlini2017towards, madry2017towards, xie2020smooth, deka2020dynamically} (\textbf{more citations to come}), which has seen mixed success. In general, even when one is able to train an adversarially robust model, its nominal performance often sees a noticeable decrease \citep{tsipras2018robustness}. This brings us to the other side of the coin: robust optimization.
% Fundamentally, adversarial training is a form of robust optimization, since it aims to minimize the maximum loss an adversary can induce. However, whereas the origin of adversarial training comes from the notion of defending against ``adversarial attacks'' \citep{szegedy2013intriguing}, in which one wants to protect their model from malicious end-users designing perturbed data to maximally fool the model. On the other hand, the perturbation set in robust optimization is more often intended as a margin given to the \textit{uncertainty} in the problem framework \citep{ben2009robust}. It is also well-known that, similar to the decrease in performance of adversarially trained models \citep{carlini2016defensive, carlini2017towards, madry2017towards, xie2020smooth, deka2020dynamically}, optimal robust solutions often perform suboptimally in practice, for example $H^\infty$ control \textbf{CITE}. On the practical side, the utility of robust optimization usually comes in the form of guidance to finding a high-performing solution that also preserves some degree of robustness like in the aforementioned $H^\infty$ control. On the theoretical side, analyzing the tradeoff between robustness and nominal performance can reveal key properties of the problem framework that practitioners can then adjust for when deploying their model.

In this paper, we continue the study of such robustness-accuracy tradeoffs, with a focus on linear inverse problems.  Such problems are ubiquitous in machine learning and engineering, with application areas spanning medical imaging \citep{ribes2008linear}, systems biology \citep{engl2009inverse}, econometrics \citep{carrasco2007linear}, recommender systems \citep{recht2010guaranteed}, communications \citep{candes2005decoding}, and others.  We show that due to the convexity of both the nominal and adversarial linear inverse problems, robustness-accuracy tradeoffs can be sharply and interpretably characterized in terms of the spectral properties of a given linear model. We then define and analyze an \emph{adversarially robust Kalman Filtering problem} by applying a refined version of our general tools.  Motivated by applications of adversarial robustness in the reinforcement literature \citep{lutter2021robust,pinto2017robust,mandlekar2017adversarially}, we provide the first theoretical analysis of robustness-accuracy tradeoffs in a setting where data is generated by a \emph{dynamical system}, and in doing so establish connections to natural control theoretic properties of the underlying system.

Our specific contributions can be summarized as follows:
% Given data $x_i \in \R^n$ and noisy measurements $y_i \in \R^p$ of the form $y_i = A_\star x_i + w_i$, $i = 1,\dots,N$, one aims to learn a matrix $A \in \R^{p \times n}$ that minimizes the mean-square loss: $\Ex\brac{\norm{y - Ax}}_2^2$. Despite its simplicity, there are surprisingly few general theoretical results about the performance of least-squares linear inverse problems under simple $\ell^2$ norm-bounded perturbations. Javanmard et al.\ \citep{javanmard2020precise} recently characterized the adversarial risk and maximal perturbation in the case of linear regression, which can be seen as a scalar measurement linear inverse problem, through a recursive formula. Our contributions toward the general linear inverse problem are two-fold:
\begin{itemize}
    \item We propose a simple and computationally efficient algorithm that provably finds the worst-case optimal $\ell^2$ norm-bounded adversarial perturbation for a given linear model and data-set. This allows us to efficiently compute and explore the Pareto-optimal robustness-accuracy tradeoff curve.
    
    %, which only requires the data pair $\curly{x,y}$ and the candidate model $\hat{A}$.
    
    \item We derive interpretable upper and lower bounds on the gap between the adversarial and standard (unperturbed) risks in terms of the spectral properties of a given linear model. We also show that our bounds are %{\color{red} Regarding Nik's comment ``Is this true''. I think this is correct, depending on what exactly tight means... Thomas convinced me it makes sense. }
    tight in the one-dimensional setting, recovering the results of \citep{javanmard2020precise}, and for matrices with full column rank and orthogonal columns.
    %These bounds allow us to characterize the aforementioned tradeoff between standard and adversarial risks.
    
    \item We define and analyze an \emph{adversarially robust Kalman Filtering} problem, and show that upper and lower bounds on the gap between the adversarial and standard (unperturbed) risk can be controlled in terms of the spectral properties of the \emph{observability gramian} \citep{zhou1998robust} of the underlying system.

%% Decided to kill this, it's a bit weak as a contribution, and I couldn't find compatible data-augmentation references.    
    % \item We show, perhaps counter-intuitively, that designing a nominal Kalman Filter with artificially increased measurement noise leads to a decrease in the robustness-accuracy gap, providing justification for commonly used data-augmentation techniques in the adversarial learning literature \citep{TODO}.
    
    \item We empirically demonstrate through extensive numerical experiments that our results qualitatively and quantitatively predict robustness-accuracy tradeoffs in linear inverse problems as a function of underlying spectral properties of the optimal nominal solution.
\end{itemize}

%% Note this paragraph can be deleted if we are short on space, but we will want it for the arXiv version.
The rest of this paper is organized as follows: in Section~\ref{sec:LS}, we present the general linear inverse problem along with an algorithm that solves the problem in an adversarial setting. We then derive tight upper and lower bounds on the adversarial risk, providing an interpretable characterization of the optimal tradeoff between robustness and accuracy. In Section~\ref{sec: Kalman}, we pose the adversarially robust Kalman Filtering problem as a linear inverse problem, and refine the general bounds developed in the previous section, revealing that for the setting where data is generated by a dynamical system, robustness-accuracy tradeoffs are dictated by natural control theoretic properties of the underlying system (namely, the observability gramian). In Section~\ref{sec: numerical}, we provide empirical evidence to support the trends predicted by our bounds, and we end with conclusions and a discussion of future work in Section~\ref{sec:conclusion}.

% \textbf{A couple of comments: (1) I think we should have a separate related work section where we cover related ideas from adversarial robustness, robust control and estimation, robust optimization, etc.  Trying to dump all of that into the intro makes it too long and not clear.  I've copied below this related work sections robustness-accuracy tradeoffs from a proposal I submitted that can probably be used with minimal modification. (2) Just as we have specialized bounds in filtering setting for when we plug in the KF (solution to nominal solution), I think we should have bounds the general setting when you plug in $A_{nominal} = pinv(X)*y$.  This shouldn't be too hard to derive quickly, and will highlight the importance of the spectral properties of the data gram Matrix $X'X$.}

\subsection{Related Work}
Our work makes connections between adversarial robustnesss and robust estimation and control.  We now provide a brief overview of work most directly related to ours from these two areas.

\textbf{Robustness-accuracy tradeoffs:} 
%a key component of this paper is understanding the fundamental robustness/performance tradeoffs that result from estimating the states of dynamical systems with various levels of observability. 
We draw inspiration from recent work offering theoretical characterizations of robustnesss-accuracy tradeoffs: \citet{tsipras2018robustness} and \citet{zhang2019theoretically} posit that high standard accuracy is fundamentally at odds with high robust accuracy by considering classification problems, whereas \citet{nakkiran2019adversarial} suggests an alternative explanation that classifiers that are simultaneously robust and accurate are complex, and may not be contained in current function classes. However, \citet{raghunathan2019adversarial} shows that the tradeoff is not due to optimization or representation issues by showing that such tradeoffs exist even for a problem with a convex loss where the optimal predictor achieves 100\% standard and robust accuracy.  In contrast to previous work, we provide sharp and interpretable characterizations of the robustness-accuracy tradeoffs that may arise in regression problems, albeit restricted to linear models.  Most closely related to our work, \citet{javanmard2020precise} derive a formula for the exact tradeoff between standard and robust accuracy in the linear regression setting, which can be viewed as a 1-d linear inverse problem.  We generalize these results to the matrix-valued setting, and further apply these tools to the adversarially robust Kalman Filtering problem, wherein data is generated by a dynamical system.

\textbf{Robust estimation and control:}
 Robustness in estimation and control has traditionally been studied from a worst-case induced gain perspective \citep{hassibi1999,zhou1998robust}. When perturbations are restricted to be $\ell^2$-bounded, this gives rise to so-called $\calH_\infty$ estimation and control problems. Although widely known, and celebrated for their applications in robust control, $\calH_\infty$-based methods are often overly conservative. This conservatism can be reduced by using mixed $\calH_2/\calH_\infty$ methods \citep{khargonekar1996}, which blend gaussian disturbance assumptions with worst case disturbances.  While such an approach is related to the adversarially robust Kalman Filtering problem that we pose, we note that it decouples the adversarial and stochastic inputs during design, leading to a fundamentally different tradeoff. We leave characterizing a connection between traditional and adversarial robustness to future work. More recently, \citet{al2020accuracy} have considered the robustness-accuracy tradeoff in data-driven perception-based control. However, the adversary in \citep{al2020accuracy}  perturbs the covariance of the noise distribution, which is assumed to remain gaussian, whereas our adversary additively attacks each measurement, which is more aligned to the standard adversarial perturbations considered in machine learning contexts. Additionally, \citep{al2020accuracy} does not quantitatively analyze the severity of tradeoffs, but rather proves the existence of tradeoffs. 
 %While an effective means of reducing the conservatism of $\calH_\infty$ state estimation, there are no effective analyses of the performance tradeoffs arising via these methods. We propose a state estimator similar to mixed $\calH_2/\calH_\infty$ state estimation, but whose fundamental tradeoff properties can be analyzed simiarly to those arising in traditional machine learning. 

% Having derived tradeoffs in the general linear inverse problem setting, we demonstrate that applying them to a particular inverse problem reveals the properties salient to robustness. State estimation is a classical instance of an inverse problem, where one is given noisy observations of a dynamical system's state, and the goal is to recover an estimate of the underlying state. State estimation is central to many control tasks such as those arising in []. A conventional approach to state estimation is to assume that disturbances to the system state and observations are gaussian. In this case, the optimal state estimator for linear systems is linear and can be derived exactly--this is known as the Kalman estimator \citep{kalman1961new, kalman1960general}. While Kalman estimation has been extraordinarily successful, it can fail under some circumstances, such as when the disturbances contain noise processes that are highly non-gaussian \Thomas{CITE}. In such cases, one might seek to improve performance by instead considering estimators designed under the assumption that all disturbances are worst-case under a norm bound--this is the aforementioned $H^\infty$ approach to estimation. This approach, however, often turns out to be overly conservative \Thomas{CITE AGAIN}. 

% We propose using adversarial training as a means to design robust state estimators without entirely disregarding the stochasticity of the disturbances. Indeed, the observations in many modern state estimation applications come in the form of noisy image data \Thomas{CITE PERCEPTION+CONTROL PAPERS}, which is precisely the kind of data that motivated adversarial training in the first place. We study trade-offs between robustness and performance of this adversarial state estimation problem as an application of the trade-offs arising in the general linear inverse setting. In particular, we analytically and empirically compare the performance of the Kalman estimator which is optimized to perform well in nominal settings with purely gaussian noise versus state estimators optimized to perform well in adversarial settings with gaussian noise as well as worst-case norm-bounded perturbations to the observations. We demonstrate that key system properties, such as observability, inform the performance degradation of state estimators operating under adversaries when optimized for the nominal setting.
% Along the way, we develop general results for understanding trade-offs in adversarial least squares, which may be of independent interest. 

% Other works have turned to online learning in an attempt to overcome the conservatism of existing robust state estimation approaches. In particular, \citep{goel2020regret} \Bruce{cite the one about state estimation and control} formulates the problem of robust state estimation as a regret minimization problem, where the baseline is a non-causal, clairvoyant state estimator with access to the true disturbance sequence. Others have focused on related control analogs of this problem. We note that our motivation is different from than theirs in that one of our key goals is to understand the system properties which make adversarial training particularly important. 

% The reason that we turn to adversarial training as an approach to design and analyze state estimators is the vast number of results on adversarial training methods and tradeoff analyses which have recently arisen in this area, and may be applied to our problem. In particular, 

% In the second part of our paper, we consider a powerful application of ordinary least-squares: the analysis of Linear Time Invariant (LTI) dynamical systems. In particular, we can cast a state estimation problem as a large least squares problem, where the adversaries enter through the observations. Intuitively, we are given observations of a possibly well-understood system, but we want to account for disturbances to the observations that may not follow standard distributions.  We demonstrate that our lower bounds on the adversarial risk can be applied to the state estimation bound, from which we can extract control-theoretic terms. These terms provide a concrete connection between system properties and robustness in a learning context.


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% BACKUP


% State estimation is central to many control tasks such as those arising in []. A conventional approach to state estimation is to assume that perturbations to the process and measurements are Gaussian. In this case, the optimal state estimator for linear systems can be derived exactly, and is known as the Kalman filter. While Kalman filtering has been extraordinarily successful, it may fail under some circumstances, such as when the noise distributions are not close to Gaussian. In such cases, we may improve performance by instead considering estimators designed under the assumption of worst case bounded disturbances. This approach, however, often turns out to be overly conservative. 

% We propose using adversarial training as a means to design robust state estimators without entirely disregarding distributional assumptions on the disturbances. We study trade-offs for this adversarial state estimation problem as a special case of the trade-offs arising in adversarial least squares. In particular, we analytically and numerically compare the performance of state estimators which are optimized to perform well in ``nominal'’ settings with Gaussian noise versus state estimators optimized to perform well in ``adversarial'’ settings with Gaussian noise along with worst-case bounded perturbations. We demonstrate that key system properties, such as observability, can serve as indicators to the performance degradation of state estimators optimized for the nominal setting when operating in the adversarial setting. Along the way, we develop general results for understanding trade-offs in adversarial least squares, which may be of independent interest. 

% Other works have turned to online learning in an attempt to overcome the conservatism of existing robust state estimation approaches. In particular, \citep{goel2020regret} \Bruce{cite the one about state estimation and control} formulates the problem of robust state estimation as a regret minimization problem, where the baseline is a non-causal, clairvoyant state estimator with access to the true disturbance sequence. Others have focused on related control analogs of this problem. We note that our motivation is different from than theirs in that one of our key goals is to understand the system properties which make adversarial training particularly important. 

% The reason that we turn to adversarial training as an approach to design and analyze state estimators is the vast number of results on adversarial training methods and tradeoff analyses which have recently arisen in this area, and may be applied to our problem. In particular, the concept of ``adversarial robustness'' has become increasingly prominent in the machine learning community over recent years. From a practical perspective, achieving adversarial robustness is borne out of necessity; it has been demonstrated repeatedly across various applications that modern learning models, especially in the overparameterized regime, can be extremely fragile to small, specifically designed perturbations to the data \citep{carlini2016defensive, goodfellow2014explaining, szegedy2013intriguing, huang2017adversarial}. Much effort has been dedicated to adjusting learning algorithms for adversarial robustness \citep{carlini2016defensive, carlini2017towards, madry2017towards, xie2020smooth, deka2020dynamically} (\textbf{more citations to come}), which has met with mixed success. In general, even when one is able to train an adversarially robust model, its \textit{nominal} performance often sees a noticeable decrease \citep{tsipras2018robustness}. This brings us to the other side of the coin: robust optimization.

% At its core, adversarial training is a form of robust optimization, where we are minimizing across our domain the maximum loss an adversary can induce. However, in robust optimization, the maximization across a perturbation set often carries a different intent than the adversaries in these machine learning contexts. Whereas the origins of adversarial training comes from the notion of defending against ``adversarial attacks'' \citep{szegedy2013intriguing}, in which one wants to protect their model from malicious end-users designing perturbed data to maximally fool the model, the perturbation set in robust optimization is more often intended as a margin given to the \textit{uncertainty} in the problem framework \citep{ben2009robust}. It is also well-known that, similar to the decrease in performance of adversarially trained models, robust solutions often perform suboptimally in practice, for example $H^\infty$ control \textbf{CITE}. On the practical side, their utility usually comes in the form of guidance to finding a good-performing solution like in the aforementioned $H^\infty$ control, while on the theoretical side, analyzing the trade-off between robustness and nominal performance can reveal key properties of the problem framework that practitioners can then adjust for when deploying their model.

% In this paper, we study the robustness-performance trade-off in perhaps the most fundamental learning model: linear inverse problems, where we are given data $x_i \in \R^n$ and noisy measurements $y_i \in \R^p$ that are generated $y_i = A_\star x_i + w_i$, $i = 1,\dots,N$, and one aims to learn a matrix $A \in \R^{p \times n}$ that minimizes the mean-square loss: $\Ex\brac{\norm{y - Ax}}_2^2$. Despite its simplicity, there are surprisingly few general theoretical results about the performance of least-squares under simple $\ell^2$ norm-bounded perturbations. Javanmard et al.\ \citep{javanmard2020precise} recently characterized the adversarial risk and maximal perturbation in the case of linear regression through a recursive formula in what can be seen as the 1-D measurement version of the OLS problem. Our contributions here are two-fold:
% \begin{itemize}
%     \item We propose a simple deterministic algorithm that leverages quadratically-constrained quadratic programming (QCQP) strong duality \citep{boyd2004convex} to find the maximal $\ell^2$ norm-bounded adversarial perturbation at a given data point, which only requires the data pair $\curly{x,y}$ and the candidate model $\hat{A}$.
    
%     \item We then derive lower-bounds on the gap between the adversarial versus standard risks, whose definitions we will introduce in the next section. We demonstrate that not only do the lower bounds match the exact value in the one-dimensional case from \citep{javanmard2020precise}, but are actually sharp in general for tall systems ($p \geq n$).
% \end{itemize}

% In the second part of our paper, we consider a powerful application of ordinary least-squares: the analysis of Linear Time Invariant (LTI) dynamical systems. In particular, we can cast a state estimation problem as a large least squares problem, where the adversaries enter through the observations. Intuitively, we are given observations of a possibly well-understood system, but we want to account for disturbances to the observations that may not follow standard distributions. Indeed, many modern state estimation applications use image data as observations--the same kind of data that motivated ``adversarial robustness'' in the first place. We demonstrate that our lower bounds on the adversarial risk can be applied to the state estimation bound, from which we can extract control-theoretic terms. These terms provide a concrete connection between system properties and robustness in a learning context.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

% 
% We study the a powerful application of least squares comes in the form of analyzing Linear Time Invariant (LTI) dynamical systems.


% there is a trust-region based adversarial attack (might be converse of what we are doing)

% outline:
% Recently ML cares a lot about adversarial robustness. On one hand a lot of the motivation may come from security: models with many parameters tend to suffer from fragility to small perturbations of the inputs -- intuitively well-separated objects may not be well-separated in the distance metric (panda-gibbon example). On the other hand, from an optimization standpoint, adversarial robustness can also be thought of as an intermediary between traditional deterministic robust optimization, and the stochastic optimization, on which much of modern ML is built on, where we want to give some budget to our fundamental uncertainty about the disturbance distribution, but still want to make use of the statistical and concentration properties inherent to most realistic learning problems.

% We start with arguably the simplest model: ordinary least squares. Surprisingly few general results for eps-bounded least squares for both computing the adversarial risk as well as an analytic form for the adversary (need lit review here). A recent result has characterized the adversarial risk as well as the analytic form of the adversarial perturbation for a standard linear regression setting (which can be thought of as a 1-D OLS).



\section{The Linear Inverse Problem}
\label{sec:LS}
% \subsection{Linear Inverse Problem}

We consider the following linear inverse problem, where given data $x \in \R^n$, measurements $y \in \R^p$ are generated by
\begin{equation}
    \label{eq:linearInverse}
    y = A_\star x + w
\end{equation}
where $w \in \R^p$ is a noise vector and $A_{\star}$ is an unknown ground-truth matrix. Our goal is to estimate the ground truth matrix $A_{\star}$ through an estimator $\hat{A}$. 

In the following subsection, we introduce the fundamental tradeoff between accuracy of the resulting estimator $\hat{A}$ under an assumed noise distribution, and the robustness of this estimator to adversarial attacks. In \S\ref{sec: inner max}, we show how to solve for the worst-case adversarial perturbations, which provides a means of computing the performance of the estimator in the face of adversarial attacks. \S\ref{sec: LIP AR SR bounds} provides upper and lower bounds on the gap between the robust performance and the nominal performance, which allow for prediction of the severity of the tradeoffs based on properties of the underlying problem.


\subsection{Optimal Robustness-Accuracy Tradeoffs}
 \label{sec: linInvTradeoffs}
  When the noise vector $w$ in equation \eqref{eq:linearInverse} is assumed to be stochastic, then the minimum mean square estimator can be found by solving the following stochastic optimization problem:
\begin{align}\label{eq: SR def}
    \hat{A} &= \argmin_{A \in \R^{p \times n}} \; \Ex_{x,w}\brac{\norm{y - Ax}_2^2} =: \argmin_{A \in \R^{p \times n}}  \; \SR(A). %\nonumber
\end{align}
where $\norm{\cdot}_2$ is the Euclidean norm, and we use $\SR(A)$ to denote the \emph{standard risk of $A$}, i.e., $\SR(A)$ is the mean squared error incurred by a matrix $A$. We then consider the same problem but with the inclusion of an $\ell^2$-norm bounded adversary:
\begin{align}\label{eq: AR def}
    \hat{A}(\varepsilon) &=  \argmin_{A \in \R^{p \times n}} \; \Ex_{x,w}\brac{\max_{\norm{\delta}_2 \leq \varepsilon} \norm{y - A(x + \delta)}_2^2} =:  \argmin_{A \in \R^{p \times n}} \; \AR(A), %\nonumber
\end{align}
where $\AR(A)$ denotes the \emph{adversarial risk of $A$}.
% More generally, we care not only about the empirical solution, but the generalization properties of our solution. We quantify this by defining the standard and adversarial risks, which we denote $\SR(\hat{A})$ and $\AR(\hat{A})$, as the expected mean-squared prediction error:
% \begin{align*}
%     \SR(A) &= \Ex_{x,w}\brac{\norm{y - Ax}_2^2} \\
%     \AR(A) &= \Ex_{x,w}\brac{\max_{\norm{\delta}_2 \leq \varepsilon} \norm{y - A(x + \delta)}_2^2},
% \end{align*}

% The two objectives in \eqref{eq: SR def} and \eqref{eq: AR def} are fundamentally at conflict with each other, i.e.\ decreasing one may lead to increasing the other. 
Our objective in this section is to precisely characterize the optimal robustness-accuracy tradeoff curve defined by the adversarial and standard risks.  We refer to the set of all points $(\SR(A), \AR(A))$ over all $A \in \mathbb{R}^{p\times n}$ as the $(\SR,\AR)$ region. The optimal tradeoff between
standard and adversarial risks is characterized via the so-called Pareto boundary of this region, which we denote $\curly{(\SR(A_\lambda),\AR(A_\lambda)): \lambda \geq 0}$. Using standard results in multi-objective optimization, $A_\lambda$ are computed by solving the regularized optimization problem
\begin{align}\label{eq:reg-opt}
    A_\lambda &:= \argmin_{A}\; \SR(A) + \lambda \AR(A).
\end{align}
Varying the regularization parameter $\lambda$ in problem \eqref{eq:reg-opt} thus allows us to characterize the aforementioned Pareto boundary by interpolating between the solution to the standard (i.e., $A_0$) and adversarial (i.e., $A_\infty$) linear inverse problems. Our first contribution is to show that the solution $A_\lambda$ to the regularized optimization problem \eqref{eq:reg-opt} can be computed efficiently using stochastic optimization. %\Bruce{XXX: Include pareto plot here for one such linear inverse problem}. 

%does not admit a closed form solution, Thus, it is imperative that we are capable of optimizing for $A_\lambda$, and that we can extract the relevant quantities of the underlying problem that control the tradeoff, i.e.\ control the standard-adversarial gap $\AR(A) - \SR(A)$.


In what follows, we assume that the noise $w \sim{} \calN(0,\Sigma_w)$, $\Sigma_w \succ 0$, and that the data $x \sim{} \calN(0,\Sigma_x)$, $\Sigma_x \succ 0$. %This is in line with standard assumptions in the analysis of least-squares. 
It is simple to verify that under these assumptions, the standard risk $\SR(A)$ is given by
\begin{align}\label{eq: standard risk deterministic form}
    \SR(A) &= \Ex_{x,w}\brac{\norm{(y-A_{\star}x) + (A_{\star} - A)x}_2^2} = \tr(\Sigma_w) + \tr\paren{(A - A_{\star}) \Sigma_x (A - A_{\star})^\top}
\end{align}

However, due to the inner maximization in the adversarial risk in equation \eqref{eq: standard risk deterministic form}, no such closed-form expression exists for $\AR(A)$. We show next that despite the non-convexity of the the inner maximization problem, it can be solved efficiently.  This allows us to apply stochastic gradient descent to solve  $\min_A \AR(A)$. In particular, note that $\AR(A)$ is the linear combination of a point-wise supremum of convex functions in $A$, and hence convex in $A$ itself \citep{boyd2004convex}. Next observe that we can draw samples of $x\sim \calN(0,\Sigma_x)$ and $w\sim \calN(0,\Sigma_w)$, and apply the solution to the inner maximization problem to solve for realizations of $\max_{\norm{\delta}_2 \leq \epsilon} \norm{y - A(x+\delta)}_2^2$. Taking the gradient of these realizations with respect to $A$ provides a stochastic descent direction. As the overall expression is convex in $A$, stochastic gradient descent with an appropriately decaying stepsize converges to the optimal solution \citep{bottou2018first}. %\Bruce{XXX: Add citation}

%we can then incorporate the solution to the inner-maximization problem into a stochastic gradient descent algorithm that provably converges to the adversarial risk minimizer. %allowing us to design a practical iterative procedure to minimize the adversarial risk. This enables us to compute the Pareto tradeoff curve. 

%Then, in \ref{sec: LIP AR SR bounds}, we provide global lower and upper bounds on the gap between $\AR(A)$ and $\SR(A)$ that allow us to understand, for example, the risk of relying on the optimal nominal solution under the presence of adversaries.

\subsection{Solving the Inner Maximization}
\label{sec: inner max}

To the best of our knowledge, no closed-form expression exists for the adversarial risk: indeed, even in the scalar case studied in \citep{javanmard2020precise}, it is characterized by a recursive relationship. Further, the techniques used to derive that recursion do not extend to the multi-variable case.
% However, given $A$, and instances of $\curly{x_i, w_i}_{i=1}^N$, we are both able to solve for the maximization inside the expectation and retrieve the maximizing perturbations $\curly{\delta_i}_{i=1}^N$. We are then able to numerically solve for the optimal $A$ by observing $\AR(A)$ is a convex function, as it is an expectation of a maximization over convex functions in $A$. Thus, given access to many independent samples of $\curly{x_i, w_i}$, we may apply stochastic gradient descent to numerically minimize $\AR(A)$. Furthermore, no part of the optimization procedure requires any prior knowledge of the underlying parameters of the random vectors, this can be seen as a data-driven approach to finding the optimal solution of minimizing the adversarial risk. 

To address this challenge, we show how to efficiently compute solutions to the inner maximization of the adversarial risk $\AR(A)$.  We observe that the inner maximization $\max_{\norm{\delta}_2 \leq \varepsilon} \norm{y - A(x + \delta)}_2^2$ can be expanded and re-written as the following (non-convex) quadratically-constrained quadratic maximization problem:
\begin{align*}\label{eq:P}
    \maximize_{\delta \in \R^n}&\quad \delta^\top A^\top A \delta - 2\delta^\top A^\top b \tag{P}\\
    \subjectto &\quad \delta^\top \delta \leq \varepsilon^2,
\end{align*}
where we set $b := y - Ax$.  Let $A = U\Sigma V^\top \in \R^{p \times n}$ be the full singular-value decomposition of $A$, with $U \in \R^{p\times p}$, $\Sigma \in \R^{p \times n}$, $V\in \R^{n \times n}$, and $\Sigma = \mathrm{diag}(\sigma_1,\dots,\sigma_{\min\curly{n,p}})$, $\sigma_1 \geq \cdots \geq \sigma_{\min\curly{n,p}} \geq 0$ the nonzero singular values of $A$. We also denote the columns of $U$ and $V$ by $u_i$ and $v_i$, respectively. It is known that \eqref{eq:P} satisfies strong duality \citep{boyd2004convex} and the optimal primal-dual pair $(\delta^*,\lambda^*)$ can be characterized by the KKT conditions:
\begin{align*}
    2(\lambda^* I-A^\top A)\delta^* + 2A^\top b &= 0 \\
    \lambda^* ({\delta^*}^\top \delta^* - \varepsilon^2) &= 0 \\
    (\lambda^* I -A^\top A) &\succeq 0.
\end{align*}
The KKT conditions can then be leveraged to solve for the optimal dual solution $\lambda^*$ and subsequently the optimal perturbation $\delta^*$. The full procedure is summarized in Algorithm~\ref{alg:getPerturbation}.  

We note that \citet{boyd2004convex} show how to solve (P) via semidefinite programming. Algorithm~\ref{alg:getPerturbation}, however, allows us to recycle the SVD of $A$ to solve (P) for various values of $b$ simply by solving a root finding problem for each $b$.  This enables efficient batching when applying SGD to the outer minimization problem. 
% The third condition implies that $\lambda^* \geq \sigma_1^2$. We assume $\sigma_1 > 0$, otherwise the problem is trivial. Then using the SVD of $A$ to re-arrange the first stationarity condition, we get
% \begin{align*}
%     (\lambda^* I-\Sigma^\top \Sigma) V^\top \delta^* &= \Sigma^\top U^\top b.
% \end{align*}
% Maximizing a convex function over a convex set achieves its maximum on the boundary; it suffices to search over $\delta^\top \delta = \varepsilon^2$. We now consider two cases: $\lambda^* > \sigma_1^2$ and $\lambda^* = \sigma_1^2$. In the first case, we know $\lambda^* I-\Sigma^\top \Sigma$ must be invertible, and thus
% \begin{align*}
%     \delta^* &= V (\lambda^* I-\Sigma^\top \Sigma)^{-1} \Sigma^\top U^\top b \\
%     \varepsilon^2 = {\delta^*}^\top \delta^* &=  b^\top U \Sigma (\lambda^* I - \Sigma^\top \Sigma)^{-2} \Sigma^\top U^\top b \\
%     &= \sum_{i=1}^{\min\curly{n,p}} \frac{(b^\top u_i)^2 \sigma_i^2}{(\lambda^* - \sigma_i^2)^2},
% \end{align*}
% where $u_i$ are the columns of $U$. Observe that
% \begin{align*}
%     f(\lambda) &:= \sum_{i=1}^{\min\curly{n,p}} \frac{(b^\top u_i)^2 \sigma_i^2}{(\lambda - \sigma_i^2)^2},
% \end{align*}
% is a strictly monotonically decreasing function when $\lambda > \sigma_1^2$, and converges to $0$ when $\lambda \to \infty$. This implies there is a unique $\lambda^*$ such that $f(\lambda^*) = \varepsilon^2$, which can be numerically solved for in various ways, such as bisection.

% Now we consider the case where $\lambda^* = \sigma_1^2$. In this case, $\delta^*$ will no longer be unique, and will come in the form
% \begin{align*}
%     \delta^* &= V (\sigma_1^2 I-\Sigma^\top \Sigma)^{\dagger} \Sigma^\top U^\top b + cv,
% \end{align*}
% where $^\dagger$ denotes the Moore-Penrose pseudoinverse, and $v$ is any unit vector lying in the null-space of $\paren{\sigma_1^2 I -\Sigma^2 }V^\top$, which is precisely characterized in this case by
% \begin{align*}
%     \ker\paren{\paren{\sigma_1^2 I-\Sigma^2}V^\top} &= \Span\paren{\curly{v_i : \sigma_i^2 = \sigma_1^2}},
% \end{align*}
% with $v_i$ denoting the $i$th column of $V$. To find the appropriate scaling $c$, we observe
% \begin{align*}
%     {\delta^*}^\top \delta^* &=  b^\top U \Sigma \paren{(\sigma_1^2 I - \Sigma^\top \Sigma)^{\dagger}}^2 \Sigma^\top U^\top b + c^2 \\
%     &= \sum_{i:\sigma_i < \sigma_1} \frac{(b^\top u_i)^2 \sigma_i^2}{(\sigma_1^2 - \sigma_i^2)^2} =\varepsilon^2 \\
%     c &= \sqrt{\varepsilon^2 - \sum_{i:\sigma_i < \sigma_1} \frac{(b^\top u_i)^2 \sigma_i^2}{(\sigma_1^2 - \sigma_i^2)^2}}.
% \end{align*}
% Combining our precise characterization of $\ker\paren{\paren{\sigma_1^2 I-\Sigma^\top\Sigma}V^\top}$ using the columns of $V$, and the formula for $c$, we can extract an optimal perturbation vector $\delta^*$. Therefore, we have demonstrated that (P) as well as extracting its optimal solution can be numerically solved to arbitrary precision.


\begin{algorithm}
    \caption{Inner Maximization Solution}
    \label{alg:getPerturbation}
    \begin{algorithmic}
        \State \textbf{given} $A=U\Sigma V^\top \in \R^{p\times n}$, $b\in\R^p$, perturbation bound $\varepsilon > 0$
        \If {$\sum_{i:\sigma_i < \sigma_1} \frac{(b^\top u_i)^2 \sigma_i^2}{(\sigma_1^2 - \sigma_i^2)^2} < \varepsilon^2$}
            \State $c = \sqrt{\varepsilon^2 - \sum_{i:\sigma_i < \sigma_1} \frac{(b^\top u_i)^2 \sigma_i^2}{(\sigma_1^2 - \sigma_i^2)^2}}$
            \State Set $v$ as any unit vector lying in the null-space of $\paren{\sigma_1^2 I -\Sigma^\top\Sigma }V^\top$, i.e.\ $v\in\Span\curly{v_i: \sigma_i = \sigma_1}$
            \State $\delta^* = -V (\sigma_1^2 I-\Sigma^\top\Sigma)^{\dagger} \Sigma^\top U^\top b + cv$
        \Else 
            \State solve $\sum_{i=1}^m \frac{(b^\top u_i)^2 \sigma_i^2}{(\lambda - \sigma_i^2)^2}=\varepsilon^2 $ for $\lambda$, e.g.\ by Newton's method
            \State $\delta^* = -V (\lambda^* I-\Sigma^\top\Sigma)^{-1} \Sigma^\top U^\top b$
        \EndIf
        \State \Return $\delta^*$
    \end{algorithmic}
\end{algorithm}

The proof of correctness for Algorithm \ref{alg:getPerturbation} is detailed in Appendix~\ref{ss:proof of inner max algorithm}. %With the ability to solve the inner maximization in hand, we make the following remark regarding optimizing $\AR(A)$.

% \begin{remark}\label{rem: minimizing AR}
% We observe that
% \begin{align*}
%     \AR(A) :=\Ex_{x,w}\brac{\max_{\norm{\delta}_2 \leq \varepsilon} \norm{y - A(x + \delta)}_2^2},
% \end{align*}
% is an expectation of a pointwise supremum of a convex function in $A$, and thus $\AR(A)$ is a convex function in $A$. Algorithm~\ref{alg:getPerturbation} can be used to evaluate the exact value of the inner maximization. Thus, stochastic gradient descent or its variants can be used to precisely evaluate the optimal adversarially robust estimator $\hat{A}(\varepsilon)$ and the corresponding optimal adversarial risk $\AR(\hat{A}(\varepsilon))$.

% \end{remark}



\subsection{Lower and Upper Bounds on $\AR(A) - \SR(A)$}
\label{sec: LIP AR SR bounds}

% As previously mentioned, it is analytically intractable to find a closed-form expression for the exact gap $\AR(A) - \SR(A)$. 
Although no closed-form expression exists for the adversarial risk $\AR(A)$, we show now that interepretable upper and lower bounds on the robustness-accuracy tradeoff, as characterized by the gap $\AR(A) - \SR(A)$, can be derived.  Such bounds predict the severity of the robustness-accuracy traeoff based upon underlying properties of specific linear inverse problems. We further show that these bounds are tight in the sense that they are exact for certain classes of matrices $A$, and strong in the sense that the lower and upper bounds differ only in higher-order terms with respect to the adversarial budget $\varepsilon$.

% Before moving to the main bounds, we mention that a direct duality and semidefinite relaxation approach on (P) yields a lower bound that matches the exact formula for $\AR(A)$ in the 1-D linear regression case given in \citep{javanmard2020precise}.

% \begin{proposition}
% % Let the measurements $y \in \R^p$ be generated according to $y = A_{\star} x_i + w_i$, where $x \in \R^n$, $A_{\star} \in \R^{p \times n}$. Given an arbitrary matrix $A \in \R^{p \times n}$, 

% We observe that 
% \[
% \max_{\norm{\delta}_2 \leq \varepsilon}\norm{y - A(x+\delta)}_2^2 - \norm{y - Ax}_2^2
% \]
% is identical to the quadratic maximization problem (P), setting $b := y - Ax$. The optimal value of (P) can be lower bounded by
% \begin{equation*}
%     2\frac{\varepsilon}{\sqrt{p}}\norm{A^\top (y - Ax)}_2 + \frac{\varepsilon^2}{p} \norm{A}_F^2,
% \end{equation*}
% where $\norm{\cdot}_F$ denotes the Frobenius norm, which yields the bound
% \begin{align}\label{eq: duality lower bound}
%     &\AR(A) - \SR(A) \nonumber \\
%     &= \Ex\brac{\max_{\norm{\delta}_2 \leq \varepsilon}\norm{y - A(x+\delta)}_2^2} - \Ex\brac{\norm{y - Ax}_2^2} \nonumber \\
%     &\geq 2\frac{\varepsilon}{\sqrt{p}}\Ex_{x,w}\brac{\norm{A^\top (y - Ax)}_2} + \frac{\varepsilon^2}{p} \norm{A}_F^2.
% \end{align}
% \end{proposition}

% The proof of this bound can be found in the supplement \Thomas{}. Note that plugging in $A = \hat{\theta}^\top \in \R^{1 \times n}$, $p = 1$, and the appropriate data and noise gaussian distribution parameters, the lower bound \eqref{eq: duality lower bound} becomes the exact characterization of $\AR$ provided in Lemma 3.1 of \citep{javanmard2020precise}, which implies this lower bound is actually exactly equal to $\AR(A) - \SR(A)$ when $p = 1$. However, we demonstrate the dimension dependence on $p$ can be removed with some finer analysis, which results in the following bounds.

\begin{theorem}
\label{thm:generalUBLB}
Given any $A \in \R^{p \times n}$, we have the following lower bound on $\AR(A) - \SR(A)$:
\begin{align}\label{eq: s-lemma lower bound}
    &\AR(A) - \SR(A) \geq 2\varepsilon\; \Ex_{x,w}\brac{\norm{A^\top (y-Ax)}_2} + \varepsilon^2 \lambda_{\min} A^\top A),
\end{align}
and a corresponding upper bound
\begin{align}\label{eq: s-lemma upper bound}
    &\AR(A) - \SR(A)  \leq 2\varepsilon\; \Ex_{x,w}\brac{\norm{A^\top (y-Ax)}_2} + \varepsilon^2 \lambda_{\max}(A^\top A),
\end{align}
where $\lambda_{\min}(A^\top A)$ and $\lambda_{\max}(A^\top A)$ are the minimum and maximum eigenvalues of $A^\top A$, respectively.
\end{theorem}

The proof of these bounds relies on turning the inner maximization of the adversarial risk into various equivalent optimization problems, and utilizing the properties of Schur complements and the S-lemma. The full proof is relegated to Appendix~\ref{ss:proof of general UBLB}.

We note that when $A^\top\in\R^{n}$, equation \eqref{eq: s-lemma upper bound} recovers the exact characterization of the gap $\AR(A)-\SR(A)$ provided in \citep[Lemma 3.1]{javanmard2020precise}; thus when $p=1$, the inequality in equation \eqref{eq: s-lemma upper bound} is in fact an equality.
%by plugging in $A = \hat{\theta}^\top \in \R^{1 \times n}$, $p = 1$, and the appropriate data and noise gaussian distribution parameters, the upper bound \eqref{eq: s-lemma upper bound} becomes the exact characterization of $\AR$ provided in Lemma 3.1 of \citep{javanmard2020precise}, which implies the upper bound is actually exactly equal to $\AR(A) - \SR(A)$ when $p = 1$.
% Plugging in $A = \hat{\theta}^\top \in \R^{1 \times n}$, $p=1$ and appropriate gaussian data and noise distributions, the \textit{upper bound} \eqref{eq: s-lemma upper bound} is precisely the exact gap presented in \citep{javanmard2020precise}, using the fact that $A = \hat{\theta}^\top$ is rank-one, and thus $\lambda_{\max}(\hat{\theta}\hat{\theta}^\top) = ||\hat{\theta}||_2^2$.
We also note that the upper and lower bounds differ only in the  $\calO(\varepsilon^2)$ terms. This leads immediately to the following corollary.

\begin{corollary}
If $p \geq n$ and $A$ has orthogonal columns, then bounds \eqref{eq: s-lemma lower bound} and \eqref{eq: s-lemma upper bound} match.
\end{corollary}
\begin{proof}
If $p \geq n$ and $A$ has orthogonal columns, then $\lambda_{\min}(A^\top A)=\lambda_{\max}(A^\top A)$.
\end{proof}

The terms involving the eigenvalues of $A^\top A$ in our bounds also support the intuition that adversarial robustness is a form of implicit regularization, which is visualized in Figure~\ref{fig:heatmap}. In the one-dimensional linear classification setting, this phenomenon is well-understood \citep{tsipras2018robustness, dobriban2020provable}, where robustness to adversarial perturbations prevent a robust feature vector from relying on an aggregate of small features.  
% We see an analogous phenomenon implied by our bounds, wherein robustness to adversarial perturbations prevent a robust linear model from over-relying on a particular singular-vector-space for regression.  

% \Thomas{Should put Heatmap of advrobust $A$ vs standard $A$ as Figure}

\begin{figure}
    \centering
    \includegraphics[width=0.7\textwidth]{heatmaps2.png}
    \vspace*{-0.5cm}
    \caption{Optimizing for adversarial robustness induces an implicit regularization, which is visualized in this heatmap of a $5\times5$ nominal solution $A_\star$ (left) and adversarially robust solution $\hat{A}(\varepsilon)$ (right), where $\varepsilon= 0.5$, $x \sim \calN(0, 0.5I)$, and $w \sim \calN(0,0.1I)$.}  %{\color{red} specify epsilon, and the max magnitude of A and hat A}}
    
    \label{fig:heatmap}
\end{figure}

Leveraging bounds \eqref{eq: s-lemma lower bound} and \eqref{eq: s-lemma upper bound} from Theorem \ref{thm:AR-SR gap lower bound}, we can bound both the susceptibility and robustness of a particular linear model to adversarial perturbations.  In particular, the solution to the standard risk problem, which is $A_\star$ (as seen from \eqref{eq: standard risk deterministic form}), provides key insights into the properties of the problem.% easily see arising from using estimators optimized for the nominal risk $\SR$ that reveal the dependence of adversarial robustness to the ``conditioning'' of the measurements.

\begin{corollary}\label{cor: Astar adv tradeoff}
Assume that $w \sim{} \calN(0,\sigma_w^2 I_p)$.  Then
    \begin{align*}
        2\varepsilon \sigma_w \sqrt{\frac{2}{\pi p}} \tr\paren{\paren{A_{\star}^\top A_{\star}}^{\frac{1}{2}}} + \varepsilon^2\lambda_{\min}(A_{\star}^\top A_{\star}) 
        \leq\; \AR(A_\star) - \SR(A_\star) 
        \leq\; 2 \varepsilon \sigma_w \sqrt{\tr\paren{A_{\star}^\top A_{\star}}} + \varepsilon^2 \lambda_{\max}(A_{\star}^\top A_{\star}).
    \end{align*}
\end{corollary}
%{\color{red} Loosen bound above to get something symmetric?}
Corollary \ref{cor: Astar adv tradeoff} highlights that the spectral properties of the ground truth model $A_\star$ influences both the susceptibility and robustness of a nominal model to adversarial perturbations. A more refined analysis akin to Corollary~\ref{cor: Astar adv tradeoff} will form the basis of our analysis for the adversarially robust Kalman Filtering problem posed in the next section. 

% In general, evaluating, or more likely bounding $\Ex_{x,w}\brac{\norm{A^\top (y-Ax)}_2}$ becomes problem and structure dependent. We observe that if noises $w$ and data $x$ are zero-mean gaussian, then the expression is simply the expected norm of some zero-mean gaussian random vector. This forms the basis of our bounds in the Kalman state estimation setting.

\section{Adversarially Robust Kalman Filtering}
\label{sec: Kalman}

We consider a modification to the standard Kalman Filtering problem (reviewed in \S\ref{ss:state estimation}) to incorporate adversarial robustness.  We then extend and refine the tools developed in \S\ref{sec: LIP AR SR bounds} to this setting, with the goal of relating control theoretic properties of the underlying linear dynamical system to the robustness-accuracy tradeoffs that it induces.

\subsection{State Estimation and Observability}
\label{ss:state estimation}
State estimation, which is central to control theory, can be viewed as an inverse problem: the goal is to recover an estimate of the underlying state of a dynamical system given noisy partial observations thereof. 
A particularly interesting example of a state estimator is the Kalman Filter, which is designed for the setting where the underlying dynamical system is linear, and disturbances are Gaussian. In particular, consider a linear-time-invariant (LTI) autonomous system with state and measurement disturbances: let $x_t \in \R^n$ be the system state, $w_t \in \R^n$ the process noise, $y_t \in \R^p$ the measurement, and $v_t \in \R^p$ the measurement noise. The initial condition, process noise, and measurement noise are assumed to be i.i.d.\ zero-mean Gaussians: $x_0 \sim{} \calN(0,\Sigma_0)$, $w_t \overset{\mathrm{i.i.d.}}{\sim} \calN(0,\Sigma_w)$, $v_t \overset{\mathrm{i.i.d.}}{\sim} \calN(0,\Sigma_v)$. The LTI system is then defined by the following equations:
\begin{equation}\label{eq:ss}
    \begin{aligned}
     x_{t+1} &= Ax_t + w_t, \\
     y_t &= C x_t + v_t.
\end{aligned}
\end{equation}
Finite horizon state estimation determines an estimate for the state of the system at time $k$ given some sequence of measurements $y_0, \dots, y_N$.  This problem encompasses smoothing ($k<N$), filtering ($k=N$), and prediction ($k>N$). When the measurement and process noise satisfy the assumptions above, the optimal state estimator is the celebrated Kalman Filter (or smoother/predictor), which produces state estimates that are a linear function of the observations.  Therefore, the optimal estimate $\hat{x}_k$ of the state $x_k$ at time $k$ can be written as\footnote{We note that state-space representations for the Kalman Filter (see Appendix \ref{ss: kalman state space}) also exist \citet{hassibi1999}, but for our purposes it is more convenient to view it as a linear map.} $\hat{x}_k:= L Y_N$, where $L\in \R^{n\times p(N+1)}$ is some matrix and $Y_N$ is a vector of stacked observations
\[
    Y_N := \bmat{y_0 \\ \vdots \\y_N } \in \R^{p(N+1)}.
\]
We similarly define the stacked process and measurement noise vectors as
\[
    W_N := \begin{bmatrix} w_0 \\ \vdots \\ w_{N-1} \end{bmatrix} \in \R^{nN} \mbox{ and }  V_N := \bmat{v_0 \\ \vdots \\ v_N} \in \R^{p(N+1)}.
\]
Furthermore, suppose $k \leq N$ and let
\begin{align*}
    \calO_N &= \bmat{ C \\ CA \\ \vdots \\ CA^N} \in \R^{p(N+1)\times n} , \quad \tau_N = \bmat{0 & & & & \\ C & 0  & & \\  CA & C &  && \\ \vdots  & & \ddots & 0 & \\ CA^{N-1} & \ldots & & C & 0 }\in \R^{p(N+1)\times nN}, \\
    \Gamma_k &= \bmat{A^{k-1} & A^{k-2} & \ldots & I & 0 & \ldots & 0} \in \R^{n\times n(N+1)}
\end{align*}
so that $Y_N=\calO_N x_0 + \tau_N W_N$ and $x_k = A^k x_0 + \Gamma_k W_N$. Here $\calO_N$ is the $N$-step observability matrix, which is a quantity of interest in our analysis. 

We now introduce some background on the observability of a LTI system and its implications on state estimation.
% The name `extended observability matrix' arises from the following definition and subsequent lemma. 
\begin{definition}
    The LTI system \eqref{eq:ss} is \textnormal{observable} if when the disturbances are zero, i.e. $v_t = 0$ and $w_t=0$ for all $t\geq 0$, it is possible to recover the initial state $x_0$ from a sequence of $n$ measurements $y_0, y_1, \dots y_{n-1}$. 
\end{definition}
\begin{lemma}
\label{lem:observability}
A system of the form \eqref{eq:ss} is \emph{observable} if and only if the observability matrix $\calO_{n-1}$ has rank $n$. 
% For any $N\geq n$, an estimator which recovers the intial state from a sequence of $N+1$ measurements of an observable system without disturbances is given by $L = \paren{\calO_N^\top \calO_N}^{-1} \calO_N^\top$.
\end{lemma}
Throughout the remainder of the paper we make the following assumption:
% that enable the existence of perfect estimators in the absence of disturbances.
\begin{assumption}
    The LTI system \eqref{eq:ss} is observable and $N \geq n-1$. 
\end{assumption}

As stated, observability is a binary notion that determines whether state consistent estimation is possible. However, observability does not capture the conditioning of the linear inverse problem defining the Kalman Filter.
% however it does not inform the engineer of the difficulty of state estimation in the presence of disturbances. In the presence of disturbances, we are interested in how close our state estimate is to the true state. In particular, we evaluate a particular state estimator using the squared estimation error: $\norm{x_k - \hat{x}_k}_2^2$. Under the assumption that the noise and initial state are stochastic for instance, we would consider the mean-squared estimation error: $\Ex\brac{\norm{x_k - \hat{x}_k}_2^2}$. If we assume that the disturbances and initial state are bounded, we might consider the worst case squared estimation error, such as $\sup \norm{x_k - \hat{x}_k}_2^2$. %\Bruce{Not exactly what I want this to say, just want to set up to be able to capture both kalman estimation and $H_\infty$ estimation. }
%Even if a system is observable, the estimation error could be large due, for instance, to small amounts of information in the measurements relative the noise. 
A more refined, non-binary notion of observability can be defined in terms of the \emph{observability gramian} of a system.
% defined below, rather than whether or not system is observable. 
\begin{definition}
    The \emph{$N$-step observability gramian} is defined as $W_o(N) := \calO_N^\top \calO_N$. If the spectral radius of $A$ is less than one, then taking the limit as $N \to \infty$ results in the \emph{observability gramian} $W_o(\infty) = \sum_{t=0}^\infty \paren{A^t}^\top C^\top C A^t$. 
\end{definition}
%The $N$-step observability gramian can be written in terms of the d observability matrix: $W_o(N) = \calO_N^\top \calO_N$. It should not be surprising then, that we can determine whether or not the system is observable using the observability gramian, as in the following lemma.
\begin{lemma}
    An LTI system \eqref{eq:ss} is observable if and only if $W_o(\infty)$ is positive definite. 
\end{lemma}
%Thus the observability gramian informs whether the system is observable, and the size of the singular values inform how observable each mode of the system is. In particular, the size of the singular values relative to the disturbances can be thought of as a signal to noise ratio. If a singular value of $W_o(N)$ is small, there will likely be substantial error in estimating the states related to the corresponding singular vector. 
The observability gramian provides significantly more information about the difficulty of state estimation than the rank condition of Lemma~\ref{lem:observability}. In particular, the ellipsoid $\curly{x \, \vert \, x^\top W_o(\infty)x \leq 1}$ contains the initial states $x$ that lead to measurement signals with $\ell^2$ norm bounded by $1$ in the absense of process and measurement noise. In particular, letting $x_0 = x$, we have $x_0^\top W_o(\infty)x_0 = \sum_{t=0}^\infty x_0^\top \paren{A^t}^\top C^\top C A^t x_0 = \sum_{t=0}^\infty x_t^\top C^\top C x_t = \sum_{t=0}^\infty \norm{y_t}_2^2$. As such, small singular values of the observability gramian imply that a large subset of the state space leads to relatively small impacts on future measurements. This makes it difficult to use measurements to  distinguish states in this region in the presence of process and measurement noise, as high-gain estimators are required. %\Bruce{XXX Add another sentence or two of explanation}

% \Bruce{This should be made more precise, or a reference should be supplied}. 
%We will show that analysis of the observability gramian can also indicate the degree to which 

%We will show that the observability gramian can also indicate the degree to which .
%Analysis of the observability gramian can do more than indicate the difficulty of state estimation for a particular system; it can also inform approaches for estimator design. Notably, a good choice for the state estimator depends heavily upon the specific conditions for which it is being designed. For instance, if the engineer assumes an incorrect distribution for the disturbances, or if the training and test distributions are different, the estimator may perform quite poorly when deployed. To avoid making such assumptions in estimator design, a common approach in control is to design an estimator for ``worst-case'' performance assuming some bounds on the disturbances. Such approaches are often overly conservative. A less conservative approach is to maintain the assumption of some distribution on the disturbances, bounded or otherwise, and consider an adversary that perturbs the measurements. While such methods can improve the performance of estimators when the distributional assumptions are incorrect, they are often more computationally intensive to design, and could decrease the performance of an estimator in nominal settings. We argue that depending on the observability properties of the system, we can determine the importance of adversarially robust estimators. 

%\Bruce{The point I'm trying to make above is that in scenarios where observability is high, finding state estimators robust to adversarial perturbations may not be worth the effort. When observability is low, it may be more important to either better model the disturbance setting, or use adversarial training to lessen worst case risk.}

%Below, we summarize two common assumptions in the estimator design. One set of assumptions is chosen to design estimators that perform better on average for statistically likely disturbances and initial conditions, and the other is designed to perform better in the the worst case, potentially at the cost of performing poorer on average. We also introduce a new set of assumptions on the design that result from adversarial training in machine learning. 

%In some cases, we will have such limited knowledge of the system that we will not even try to model it under the above assumptions, and instead try to learn a filter directly from trajectory data \Bruce{Not sure how we want to frame this entire motivation/introduction, but I think we want something highlighting the fact that our analysis will hold for both learned and hand designed filters}. 

%We show both analytically and experimentally that the additional error occurring from an estimator designed to perform well in nominal setting to a setting with adversarial disturbances and vice versa increases as a function of decreasing observability of the system. In particular, we demonstrate a bounds on the gap between $\AR$ and $\SR$ where the upper bound is inversely proportional to the minimum singular value of the observability gramian, $\sigma_{min}\paren{W_o(N)}$ and the lower bound is inversely proportional to the maximum singular value of the observability gramian $\sigma_{max}\paren{W_o(N)}$. 

%The bounds supply heuristic information for the dependency of the gap on various properties, and experiments with linear systems demonstrate that these dependencies arise in practice as well. Though the theory is specific to linear systems, we demonstrate experimentally that dependencies arise in other systems as well. \Bruce{?}

\subsection{Kalman Filtering and Smoothing}
\label{ss:kalman}

% Recall the conditions of Section~\ref{ss:state estimation}, i.e. the state progresses according to dynamics \eqref{eq:ss} with $w_t \overset{\mathrm{i.i.d.}}{\sim} \calN(0, \Sigma_w)$ and $v_t \overset{\mathrm{i.i.d.}}{\sim} \calN(0,\Sigma_v)$ for all $t \geq 0$. Under the additional assumption that there is a gaussian prior on the initial condition, $x_0 \sim \calN(0, \Sigma_0)$, there is a simple and effective state estimator, attributed to Kalman, which we fill refer to as the Kalman estimator. In this section, we pose this estimator as the solution to a standard risk minimization problem.
We begin by reviewing relevant results from standard Kalman Filtering and Smoothing.  We make the following simplifying assumption for presentation purposes going forward. 
%\Bruce{Update this assumption and subsequent results to use only the assumption that noise is scaled isotropic. State the general results, and further down, add the assumption that A is scaled orthogonal, and specify the results to this case. }
\begin{assumption}\label{as:noisepd}
    % $\Sigma_w$, $\Sigma_0$, and $\Sigma_v$ are positive definite.
    $\Sigma_0 = \sigma_0^2 I $, $\Sigma_w = \sigma_w^2 I$, $\Sigma_v = \sigma_v^2 I$. We further assume that the system matrix $A = \rho Q$, $\rho \in \R$ is a scaled orthogonal matrix, such that $\rho$ controls the stability of the system.
\end{assumption} 

A generalization of our subsequent results to generic dynamics $A$ and positive definite covariance matrices can be found in Appendix~\ref{ss: kf general statements and proofs}: although more notationally cumbersome and difficult to interpret, they nevertheless convey the same overall trends. %and  heorems and proofs for full generality convey the same message, albeit with denser notation, and can be found in the Supplement. 

\paragraph{Standard State Estimation} Under Assumption \ref{as:noisepd}, we define the minimum mean square estimator for the state $x_k$ as
\begin{align}\label{eq:sr-kf}
    \hatL_k = \argmin_{L \in \R^{n\times p(N+1)}} \Ex \brac{\norm{x_k - L Y_N}_2^2}.
\end{align}
We note that the optimal solution to this problem is precisely the Kalman Filter ($k=N$) or Smoother ($k<N$). % Note that $\hatL_k$ admits a state-space representation (see supplementary material), but treating it as a matrix is more easily generalizable to the adversarial setting. 
%We will often compare this to the minimum norm estimator in the noiseless case, where $\Sigma_w,\;\Sigma_v = 0$. In this case, it is easy to verify the estimator is simply
%\begin{align*}
%    L_k = A_k (\calO_N^\top \calO_N)^{-1} \calO_N^\top.
%\end{align*}
 We explicitly solve for the minimum mean square estimator $\hatL_k$ in the following lemma, which is standard, but included for completeness. %For simplicity, we assume that $k\leq N$, however the solution is similar for $k > N$.
%\Bruce{Will we actually make use of the minimum norm estimator now that we have reasonable bounds for the Kalman estimator?} 

\begin{lemma}
    \label{lem:kf}
    Suppose $k \leq N$. The finite horizon Kalman state estimator is the solution to optimization problem \eqref{eq:sr-kf}, and is given by 
    % \begin{align*}
    %     \hatL_k &= \paren{A^k \Sigma_0 \calO_N\T + \Gamma_k \Sigma_w \tau_N\T} \\
    %     \cdot & \paren{\calO_N \Sigma_0 \calO_N\T+\tau_N\paren{I_N \otimes \Sigma_w} \tau_N\T +\paren{I_{N+1} \otimes \Sigma_v}}\I.
    % \end{align*}
    \begin{align*}
        \hatL_k = &\paren{\sigma_0^2 A^k \calO_N^\top + \sigma_w^2 \Gamma_k \tau_N^\top} \paren{\sigma_0^2 \calO_N  \calO_N^\top +
        \sigma_w^2 \tau_N \tau_N^\top + \sigma_v^2 I}\I.
    \end{align*}
\end{lemma}

% Introducing additional notation to express the Kalman estimator will be helpful in subsequent sections. Let
% \begin{align}\label{eq:lbnotation}
%     \bar{\Sigma} &:= \bmat{\Sigma_0 & \\ & I_N \otimes \Sigma_w} \\ %\in \R^{n(N+1)\times n(N+1)} \\
%     \nonumber
%     H &:=  \bmat{I &  & & & \\ A  & I & & & \\ \vdots && \ddots && \\ A^N & A^{N-1} & \hdots & I}\bar{\Sigma}^{1/2} \\
%     %\in \R^{n(N+1)\times n(N+1)} \\
%     \nonumber
%     H_k &:= E_k^\top H =  \bmat{A^k& A^{k-1} & \dots & I & 0 & \dots & 0}\bar{\Sigma}^{1/2} \\%\in \R^{n\times n(N+1)} \\ 
%     \nonumber
%     M &:= H^\top \paren{C^\top \otimes I} %\\
%     %\\= \bmat{\Sigma_0^{1/2} C^\top & \Sigma_0^{1/2} A^\top C^\top & \ldots & \Sigma_0^{1/2} \paren{A^{N}}^\top C^\top  \\  & \Sigma_w^{1/2} C^\top  &\ldots & \Sigma_w^{1/2} \paren{A^{N-1}}^\top C^\top  \\  && \ddots & \\  &  &  & \Sigma_w^{1/2} C^\top} \\
%     %\nonumber
%     = \bar{\Sigma}^{1/2} \bmat{ \calO_N^\top \\  \paren{\calZ \calO_N}^\top \\ \vdots \\ \paren{\calZ^N \calO_N}^\top}, %\in \R^{n(N+1)\times p(N+1)}.
% \end{align}
% where $\calZ \in \R^{p(N+1) \times p(N+1)}$ is a block downshift operator, with blocks of size $m$. With this notation, the Kalman estimator given in Lemma \ref{lem:kf} may be rewritten more compactly as
% \[
%     \hat{L}_k = H_k M\paren{M^\top M + \Sigma_v}^{-1}.
% \]

%\Bruce{Subsection on $\calH_\infty$ can go in if we want more background on it.}
% \subsection{$\calH_\infty$ State Estimation}
% The assumptions on the distributions of disturbances and the prior on the initial condition made by Kalman filtering is not particularly realistic, and in situations where it is not satisfied, can lead to very poor estimator performance. A control theoretic approach to robustify the filter is to choose filters that perform well in the fact of worst case choices of the initial condition, process disturbance, and measurement disturbance. This gives rise to an $\calH_\infty$ estimator:
% \begin{align*}
%     L_\infty := \argmin_{L \in \R^{n\times p(N+1)}} \max_{\norm{\bmat{x_0 \\ W_N \\ V_N}}_2 \leq \varepsilon}\norm{x_k - L Y_N}_2^2,
% \end{align*}
% where $\varepsilon$ is some positive number. Given a fixed $L$, we observe that we can rewrite the above norm expression into
% \begin{align*}
%     \norm{x_k - L Y_N}_2^2 &= \norm{\paren{A^k - L\calO_N}x_0 + \paren{\Gamma_k - L\tau_N}W_N - LV_N}_2^2 \\
%     &= \norm{\brac{\begin{array}{c|c|c} A^k - L\calO_N & \Gamma_k - L\tau_N & -L \end{array} } \bmat{x_0 \\ W_N \\ V_N} }_2^2.
% \end{align*}
% Re-defining $u := [x_0^\top \; W_N^\top \; V_N^\top]^\top$, and $M(L) := \brac{\begin{array}{c|c|c} A^k - L\calO_N & \Gamma_k - L\tau_N & -L \end{array} }$, the inner maximization can be written into
% \begin{align*}
%     \max_{\norm{u}_2 \leq \varepsilon} u^\top M(L)^\top M(L) u &= \varepsilon^2 \sigma_{\max}\paren{M(L)}^2,
% \end{align*}
% where we used the min-max characterization of singular values. This also implies the worst-case disturbance $u$ is the right singular vector corresponding to $\sigma_{\max}\paren{M(L)}$ scaled to norm $\varepsilon$. We observe that $\norm{x_k - LY_N}_2^2$ is a convex function with respect to $L$, and therefore 
% \begin{align*}
%     \min_{L} \max_{\norm{\bmat{x_0 \\ W_N \\ V_N}}_2 \leq \varepsilon}\norm{x_k - L Y_N}_2^2
% \end{align*}
% is a convex optimization problem. %\Thomas{A closed-form solution $L_\infty$ can be derived using first-order optimality conditions (?)
% %observing that
% %\begin{align*}
% %    M(L) := \brac{\begin{array}{c|c|c} A^k & \Gamma_k & 0 \end{array} } - L \brac{\begin{array}{c|c|c}\calO_N & \tau_N & -I \end{array} }
% %\end{align*}
% %is affine function of $L$, and the largest singular value is a matrix-convex function. }\Bruce{} 


% \begin{lemma}
%     The $\calH_\infty$ state estimator is given by the solution to the following SDP
%     \begin{align*}
%         \minimize_{L\in \R^{n\times p(N+1)}, \lambda\in\R}: &\quad \lambda \\
%         \subjectto: & \quad \bmat{\lambda &  \brac{\begin{array}{c|c|c} A^k & \Gamma_k & 0 \end{array} } - L \brac{\begin{array}{c|c|c}\calO_N & \tau_N & -I \end{array} } \\ \paren{\brac{\begin{array}{c|c|c} A^k & \Gamma_k & 0 \end{array} } - L \brac{\begin{array}{c|c|c}\calO_N & \tau_N & -I \end{array} }}\T & I} \succeq 0
%     \end{align*}
% \end{lemma}


%As discussed in the introduction, the $\calH_\infty$ state estimator is a type of adversarial state estimator in the sense that the initial condition, disturbance, and process noise are adversarially chosen subject to bounded energy. A less conservative approach to
%It is well known that adversarial attacks can pose great difficulty for classification tasks. There has been little study, however, of adversarial attacks on data generated from a partially observed dynamical system. We argue that considering robustness to such attacks is extraordinarily important. 

\paragraph{Adversarially Robust State Estimation}
We now modify the standard filtering problem \eqref{eq:sr-kf} to allow adversarial perturbations to enter through sensor measurements.\footnote{We choose to restrict our attention to adversarial sensor measurements because it is a more direct analog to the traditional adversarial robustness literature, which considers perturbations to image data, and not to the image data-generating distribution \citep{szegedy2013intriguing, goodfellow2014explaining, carlini2016defensive}.} In particular, for some $\varepsilon>0$, the adversarially robust state estimator is defined by
\begin{align*}
    \hatL_k(\varepsilon):= \argmin_{L \in \R^{n\times p(N+1)}} \Ex \brac{\max_{\norm{\delta}_2 \leq \varepsilon} \norm{x_k - L (Y_N + \delta)}_2^2},
\end{align*}
where the assumptions on the initial condition, process disturbance and measurement disturbance are as in Assumption \ref{as:noisepd}. % We again have that $\varepsilon$ is some positive number.  

\subsection{Robustness-Accuracy Tradeoffs in Kalman Filtering}
The Kalman state estimation problem and adversarial state estimation problems can be viewed as standard and adversarial linear inverse problems by defining
\begin{align*}
    \SR(L) &:= \Ex \brac{\norm{x_k - L Y_N}_2^2}, \\
    \AR(L) &:= \Ex \brac{\max_{\norm{\delta}_2 \leq \varepsilon} \norm{x_k - L(Y_N + \delta)}_2^2}.
\end{align*}
% As discussed in Section \ref{sec:LS}, there is a trade-off between optimizing $\AR(L)$ and $\SR(L)$. 

As in \S\ref{sec:LS}, our goal is to characterize robustness-accuracy trade-offs for this linear inverse problem. As previewed in \S\ref{sec:LS}, we show that a refined analysis allows for the the gap $\SR(L)-\AR(L)$ to be bounded in terms of the spectral properties of the observability gramian of the system, establishing a natural connection to the robust control and estimation literature \citep{hassibi1999,zhou1998robust}. In particular, our results indicate the robustness-accuracy tradeoff is more severe for systems with ``uniformly low observability,'' as characterized by the Frobenius norm of the observability gramian.   %Recent work has closely examined these tradeoffs in the case of scalar regression. Analyzing the tradeoff between $\AR$ and $\SR$ above requires us to consider the vector case. We will be interested in determining bounds on the gap between $\AR$ and $\SR$ as a function of system properties, such as observability and stability. 
%This observation allows us to draw upon recent results in machine learning literature which analyzes the tradeoffs that occur between standard risk and adversarial risk in showing our hypothesis. In particular, we will extend and apply the tools from \citep{javanmard2020precise} to (1) find  upper and lower bounds on the gap between the quantities for specific estimators $L$, and (2) numerically trace out Pareto curves for various systems as well as $\SR$ vs $\AR$ curves for specific types of estimator as parameters of the system determining observability are varied. Both our theoretical result and numerical results support our hypothesis that the gap between adversarial risk and standard risk of reasonable estimators tends to increase as a function of observability. 

To begin, we present a closed form for the standard risk $\SR(L)$. 
\begin{lemma} \label{lem:SR closed form}
    For any $L\in\R^{n\times p(N+1)}$, our standard risk is 
    % \begin{align*}
    %     \SR(L) &= \Ex \brac{\norm{x_k - L Y_N}_2^2} = \sigma_0^2\.\norm{A^k - L \calO_N}_F^2 \\
    %     &+ \norm{\Gamma_k - L \tau_N\paren{I_N \otimes \Sigma_w}^{1/2}}_F^2 \\
    %     &+ \norm{L \paren{I_{N+1} \otimes \Sigma_v}^{1/2}}_F^2.
    % \end{align*}
    \begin{align*}
        &\SR(L) = \Ex \brac{\norm{x_k - L Y_N}_2^2} = \sigma_0^2\norm{A^k - L \calO_N}_F^2  + \sigma_w^2\norm{\Gamma_k - L \tau_N}_F^2 + \sigma_v^2\norm{L}_F^2.
    \end{align*}
\end{lemma}

Lemma \ref{lem:SR closed form} makes clear that the noise terms act as a regularizer: if $\sigma_w^2=\sigma_v^2=0$, then $\min_L \SR(L)=0$ and is achieved by $L=A^k(W_o(N))^{-1}\calO_N^\top$. This closed-form expression suggests that the spectral properties of $W_o(N)$ may play an important role in the robustness-accuracy tradeoffs satisfied by an LTI system \eqref{eq:ss}.  We formalize this intuition next.
% A proof of this Lemma may be found in the Appendix.

% Note that Lemma \ref{lem:kf} follows by taking the derivative of the above quantity with respect to $L$ \citep{petersen2008matrix}, and setting it equal to zero. Since $\SR(L)$ is a convex function with respect to $L$, the resulting $L$ is the minimizer.

%As with the least squares setting, a closed form expression for the adversarial risk is not known, which leads us to instead consider upper and lower bounds on the gap between the standard risk and adversarial risk. To achieve these bounds, we will leverage the general results presented in Section \ref{sec:LS}. 




%  \subsection{Adversarial Risk}

%  Deriving a closed-form expression for the adversarial risk is intractable in comparison to the standard risk. Iindeed, in the scalar case studied in \cite{javanmard2020precise}, it is characterized by a recursive relationship. The techniques used to derive that recursion do not extend to the multi-variable case. However, given $L$, and instances of $x_0, W_N, V_N$, we are both able to solve for the maximization inside the expectation and retrieve the maximizing perturbation $\delta$. We are then able to numerically solve for the optimal $L$ by observing $\AR(L)$ is a convex function (maximization of convex functions), independently sampling many $x_0, W_N, V_N$, and applying stochastic gradient descent. In fact, no part of the optimization procedure requires any prior knowledge of the underlying parameters of the random vectors, and thus this can be seen as a learning problem from trajectory data. We now discuss how to characterize the solutions to the inner maximization in a numerically tractable way. 


% Consider the following (non-convex) quadratically-constrained quadratic maximization problem:
% \begin{align*}
%     \maximize_{\delta \in \R^M}: \quad& \delta^\top L^\top L \delta - 2b^\top L\delta \tag{P}\\
%     \subjectto \quad& \delta^\top \delta \leq \epsilon^2,
% \end{align*}
% where $L = U\Sigma V^\top \in \R^{m \times M}$, $m < M$ is a given matrix and its singular-value decomposition with singular values $\sigma_1 \geq \cdots \geq \sigma_m \geq 0$, $U \in \R^{m\times m}$, $\Sigma \in \R^{m \times m}$, $V\in \R^{M \times m}$ and $b \in \R^m$. Then, (P) satisfies strong duality \cite{boyd2004convex} and the optimal primal-dual solutions $\delta^*,\lambda^*$ are characterized by the KKT conditions:
% \begin{align*}
%     2(\lambda^* I-L^\top L)\delta^* - 2L^\top b &= 0 \\
%     \lambda^* ({\delta^*}^\top \delta^* - \epsilon^2) &= 0 \\
%     (\lambda^* I -L^\top L) &\succeq 0.
% \end{align*}
% The third condition implies that $\lambda^* \geq \sigma_1^2$. We assume $\sigma_1 > 0$, otherwise the problem is trivial. Then rewriting using the SVD of L and the first stationarity condition, we get
% \begin{align*}
%     (\lambda^* I-\Sigma^2) V^\top \delta^* &= \Sigma U^\top b.
% \end{align*}
% Using the fact that maximizing a convex function over a convex set achieves its maximum on the boundary, it suffices to search over $\delta^\top \delta = \epsilon^2$. We now consider two cases: $\lambda^* > \sigma_1^2$ and $\lambda^* = \sigma_1^2$. In the first case, we know $-\Sigma^2 + \lambda^* I$ must be invertible, and thus we know
% \begin{align*}
%     \delta^* &= V (\lambda^* I-\Sigma^2)^{-1} \Sigma U^\top b \\
%     {\delta^*}^\top \delta^* &=  b^\top U \Sigma (\lambda^* I - \Sigma^2)^{-2} \Sigma U^\top b \\
%     &= \sum_{i=1}^m \frac{(b^\top u_i)^2 \sigma_i^2}{(\lambda^* - \sigma_i^2)^2} \\
%     &= \epsilon^2,
% \end{align*}
% where $u_i$ are the columns of $U$. Observe that
% \begin{align*}
%     f(\lambda) &:= \sum_{i=1}^m \frac{(b^\top u_i)^2 \sigma_i^2}{(\lambda - \sigma_i^2)^2},
% \end{align*}
% is a strictly monotonically decreasing function when $\lambda > \sigma_1^2$, and converges to $0$ when $\lambda \to \infty$. This implies there is a unique $\lambda^*$ such that $f(\lambda^*) = \epsilon^2$, which can be numerically solved for in various ways, such as bisection.

% Now we consider the case where $\lambda^* = \sigma_1^2$. In this case, $\delta^*$ will no longer be unique, and will come in the form
% \begin{align*}
%     \delta^* &= V (\sigma_1^2 I-\Sigma^2)^{\dagger} \Sigma U^\top b + cw,
% \end{align*}
% where $^\dagger$ denotes the Moore-Penrose pseudoinverse, and $w$ is any unit vector lying in the null-space of $\paren{\sigma_1^2 I -\Sigma^2 }V^\top$, which is precisely characterized in this case by
% \begin{align*}
%     \ker\paren{\paren{\sigma_1^2 I-\Sigma^2}V^\top} &= \Span\paren{\curly{v_i : \sigma_i^2 = \sigma_1^2}},
% \end{align*}
% with $v_i$ denoting the $i$th column of $V$. To find the appropriate scaling $c$, we observe
% \begin{align*}
%     {\delta^*}^\top \delta^* &=  b^\top U \Sigma \paren{(\sigma_1^2 I - \Sigma^2)^{\dagger}}^2 \Sigma U^\top b + c^2 \\
%     &= \sum_{i:\sigma_i < \sigma_1} \frac{(b^\top u_i)^2 \sigma_i^2}{(\sigma_1^2 - \sigma_i^2)^2} =\epsilon^2 \\
%     c &= \sqrt{\epsilon^2 - \sum_{i:\sigma_i < \sigma_1} \frac{(b^\top u_i)^2 \sigma_i^2}{(\sigma_1^2 - \sigma_i^2)^2}}.
% \end{align*}
% Thus, combining our precise characterization of $\ker\paren{\paren{\sigma_1^2 I-\Sigma^2}V^\top}$ using the columns of $V$, and the formula for $c$, we can extract an optimal perturbation vector $\delta^*$. Therefore, we have demonstrated that (P) along with extracting its optimal solution can be numerically solved. The procedure is summarized in Alg~\ref{alg:getPerturbation}.

% \begin{algorithm}
%     \caption{Adversarially Robust State Estimator}
%     \label{alg:getPerturbation}
%     \begin{algorithmic}
%         \State \textbf{given} $L=U\Sigma V^\top \in \R^{m\times M}$, $b\in\R^M$ 
%         \If {$\sum_{i:\sigma_i < \sigma_1} \frac{(b^\top u_i)^2 \sigma_i^2}{(\sigma_1^2 - \sigma_i^2)^2} < \epsilon^2$}
%             \State $c = \sqrt{\epsilon^2 - \sum_{i:\sigma_i < \sigma_1} \frac{(b^\top u_i)^2 \sigma_i^2}{(\sigma_1^2 - \sigma_i^2)^2}}$
%             \State Set $w$ as any unit vector lying in the null-space of $\paren{\sigma_1^2 I -\Sigma^2 }V^\top$, i.e.\ $w\in\Span\curly{v_i: \sigma_i = \sigma_1}$
%             \State $\delta^* = V (\sigma_1^2 I-\Sigma^2)^{\dagger} \Sigma U^\top b + cw$
%         \Else 
%             \State solve $\sum_{i=1}^m \frac{(b^\top u_i)^2 \sigma_i^2}{(\lambda - \sigma_i^2)^2}=\epsilon^2 $ for $\lambda$, e.g.\ by bisection
%             \State $\delta^* = V (\lambda^* I-\Sigma^2)^{-1} \Sigma U^\top b$
%         \EndIf
%         \State \Return $\delta^*$
%     \end{algorithmic}
% \end{algorithm}

% Applying this to the inner maximization of the adversarial risk function, we set $b = \paren{A^k - L\calO_N}x_0 + \paren{\Gamma_k - L\tau_N}W_N - LV_N$.



\subsection{Bounding $\AR - \SR$ for State Estimation}

\label{ss:bounding state estimation error}
As in \S\ref{sec:LS}, we do not have a closed-form for the adversarial risk. Upper and lower bounds on the gap between the adversarial risk and standard risk, however, still highlight the role control theoretic quantities play in robustness-accuracy tradeoffs.

%Applying a specialized analysis to the lower bound in Theorem \ref{thm:generalUBLB}, we derive the following lower bound on the gap $\AR(L)-\SR(L)$. 

\begin{lemma}\label{thm:AR-SR gap lower bound}
    For any $L \in \R^{n \times p(N+1)}$, the gap between $\AR(L)$ and $\SR(L)$ admits the following lower bound:
    % \begin{align}\label{eq:AR-SR gap no exp}
    %      %2 & \varepsilon\; \Ex\brac{\norm{L^\top \paren{x_k - LY_N}}_2} \nonumber \\ 
    %      \AR(&L) - \SR(L)
    %      \geq  2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}}  \tr\bigg(\bigg( L^\top \Big(S \Sigma_0 S^\top \\
    %      &+ T \paren{I_N \otimes\Sigma_w} T^\top \nonumber + L\paren{I_{N+1} \otimes \Sigma_v} L^\top\Big)L\bigg)^{1/2}\bigg),
    % \end{align}
    % where $S := A^k - L\calO_N$, $T := \Gamma_k - L\tau_N$.
    \begin{align}\label{eq:AR-SR gap no exp}
         %2 & \varepsilon\; \Ex\brac{\norm{L^\top \paren{x_k - LY_N}}_2} \nonumber \\ 
         \AR(L) - \SR(L)
         \geq 2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}} \sigma_v \norm{L}_F^2
    \end{align}
    
    
\end{lemma}


We now turn our attention to studying the tradeoffs enjoyed by the Kalman Filter/Smoother $L = \hat{L}_k$ defined in Lemma \ref{lem:kf}. This choice is made because the Kalman estimator is the optimal estimator in the nominal setting, and is commonly used in practice. To simplify notation in the subsequent results, we will denote $\sigma_{\vee}^2 = \max\curly{\sigma_0^2, \sigma_w^2}$, $\sigma_{\wedge}^2 = \min\curly{\sigma_0^2, \sigma_w^2}$ and
    \begin{equation}
        r_k(\rho) = \begin{cases} k, \quad& \rho = 1 \\ \frac{1 - \rho^{2(k+1)}}{1 - \rho^2}, \quad & \rho \neq 1.
    \end{cases}
    \end{equation}
With these definitions, we have the following theorem. 

\begin{theorem}
     Suppose that $\hat{L}_k$ is the Kalman estimator from Lemma~\ref{lem:kf}. 
    %Let us denote $\sigma_{\vee}^2 = \max\curly{\sigma_0^2, \sigma_w^2}$, $\sigma_{\wedge}^2 = \min\curly{\sigma_0^2, \sigma_w^2}$ and
    % \begin{equation}
    %     r_k(\rho) = \begin{cases} k, \quad& \rho = 1 \\ \frac{1 - \rho^{2(k+1)}}{1 - \rho^2}, \quad & \rho \neq 1.
    % \end{cases}
    % \end{equation}
    %assume that %assume that $\sqrt{\norm{\Sigma_v}_2} \leq (N+1)\norm{\sigma_{\vee}}_2 \sigma_{min}\curly{\calO_N^\top}$.
    We have the following bound on the gap between $\AR$ and $\SR$. 
    % \begin{align}\label{eq:lb}
    %     \AR(\hat{L}_k) - \SR(\hat{L}_k) \geq 2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}}\sigma_{min}\left(\Sigma_v  \right) \norm{C}_F^2 \\ \nonumber
    %     \cdot \paren{ \frac{\sigma_{min} \paren{A^k \Sigma_0 {A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w {A^{k-i}}^\top} }{(N+1) \norm{\sigma_{\vee}}_2^2 \norm{W_o(N)}_F + \norm{\Sigma_v}_2} }^2. 
    % \end{align}
    % \begin{align}\label{eq:lb0}
    %     &\AR(\hat{L}_0) - \SR(\hat{L}_0) \geq 2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}}\sigma_v \\ \nonumber &\paren{\frac{\norm{{W_o(N)}}_F^{1/2}}{(N+1) \norm{\sigma_{\vee}}_2^2 \norm{W_o(N)}_F + \norm{\Sigma_v}_2}}^2
    % \end{align}
    \begin{align}\label{eq:lb}
        \AR(\hat{L}_k)& - \SR(\hat{L}_k) \geq 2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}}\sigma_v \norm{C}_F^2  \paren{ \frac{\rho^{2k} \sigma_0^2 + r_k(\rho)\; \sigma_w^2 }{(N+1) \sigma_{\vee}^2\norm{W_o(N)}_F + \sigma_v^2 } }^2. 
    \end{align}
    %Furthermore, when $k=0$, i.e. for initial state $x_0$ estimation, we obtain the simplified bound
    %\begin{align}\label{eq:lb0}
    %    \AR(\hat{L}_0)& - \SR(\hat{L}_0) \geq 2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}}\sigma_v \\ \nonumber &\times\paren{\frac{\norm{{W_o(N)}}_F^{1/2}}{(N+1)\sigma_{\vee}^2 \norm{W_o(N)}_F + \sigma_v^2 }}^2.
    %\end{align}
    \label{thm:lb}
\end{theorem}
We see that the lower bound increases as the Frobenius norm of the observability gramian decreases. This indicates that as observability becomes uniformly low, i.e., if all eigenvalues of $W_o(N)$ are small, then a nominal state estimator $\hat L_k$ will have a large gap $\AR(\hat L_k)-\SR(\hat L_k)$. Observe that increasing $\sigma_w$ will increase the lower bound shown above when $\sigma_w \leq \sigma_0$. 
% \subsection{Upper bounding the gap between $\SR$ and $\AR$}  

We now derive an upper bound on the gap between the standard and adversarial risk for any given $L$. This bound follows from the upper bound in Theorem \ref{thm:generalUBLB}.
\begin{lemma}\label{thm:general upper bound}
    For any $L \in \R^{n \times p(N+1)}$, the following bound holds
    \begin{align*}
        \AR(L) -  \SR(L) \leq 2\varepsilon  \norm{L}_2 \norm{\Sigma^{1/2}}_F + \varepsilon^2 \norm{L}_2^2
    \end{align*}
    where $\Sigma^{1/2}$ is the symmetric square root of the covariance of $x_k - LY_N$. 
\end{lemma}

Again, we consider how this upper bound looks for the Kalman estimator $\hat{L}_k$. %The following theorem upper bounds the gap between the adversarial and standard risk for the Kalman filter. 
%The following Lemma will be helpful.
%\begin{lemma}\label{lem:minimum nonzero singular value}
%    Suppose $A \in \R^{n \times N}$, where $N\geq n$ and $\operatorname{Rank}(A) = m$. Then
%    for any $c \in \R_{++}$, 
%    \begin{align*}
%        \norm{A \paren{A\T A +cI}^{-1}}_2 \leq \frac{1}{\sigma_{m}\paren{A}}.
%    \end{align*}
%\end{lemma}
%\Bruce{Change variables in the above lemma} 
%The proof of this Lemma may be found in the appendix. 
%We now state the result for the upper bound on the gap between $\AR$ and $\SR$ of the kalman filter.  
\begin{theorem}
    % Suppose that $L_k$ is the Kalman state estimator given by Lemma \ref{lem:kf}. Then the gap between $\AR(L_k)$ and $\SR(L_k)$ is upper bounded by
    % \begin{align*}
    %     \varepsilon & \paren{\frac{{\norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2}}{\sigma_{min}(W_o(N))^{1/2} \sigma_{min} \paren{\bar\Sigma}}} \\
    %     & \quad \cdot \left(2 \sqrt{n}
    %     %\paren{ \norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2
    %         \paren{\norm{\sigma_{\vee}}_2 + \paren{\frac{\norm{\Sigma_v}_2}{\sigma_{min}(W_o(N))^{1/2} \sigma_{min} \paren{\bar\Sigma}}}^2 }^{1/2} \right. \\ & \quad \quad \left.   
    %     %\paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}} {\paren{\norm{\bar{\Sigma}}_2 \norm{H_k}_2^2 + \norm{\Sigma_v}_2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2 }^{1/2}}  \\
    %     %+ \varepsilon^2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2.
    %     +\varepsilon \paren{ \frac{1}{\sigma_{min}(W_o(N))^{1/2} \sigma_{min} \paren{\bar\Sigma}}}\right).
    % \end{align*}
    
    Suppose that $\hat{L}_k$ is the Kalman state estimator from Lemma \ref{lem:kf}. Then
    % \begin{align*}
    %     &\AR(\hat{L}_k) - \SR(\hat{L}_k) \\
    %     &\leq \varepsilon \paren{\frac{ \rho^{2k} \sigma_0^2 + r_k(\rho)\; \sigma_w^2}{\sigma_{\min}(W_o(N))^{1/2} \min\curly{\sigma_0^2, \sigma_w^2} }} \\
    %     & \quad \times \Bigg[2 \sqrt{n}
    %     %\paren{ \norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2
    %         \paren{\sigma_{\vee}^2 + \paren{\frac{\sigma_v^2}{\sigma_{\vee}^2\sigma_{\min}(W_o(N))^{1/2} }}^2 }^{1/2} \\ & \quad \quad   
    %     %\paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}} {\paren{\norm{\bar{\Sigma}}_2 \norm{H_k}_2^2 + \norm{\Sigma_v}_2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2 }^{1/2}}  \\
    %     %+ \varepsilon^2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2.
    %     +\varepsilon \paren{ \frac{1}{\sigma_{\vee}^2 \sigma_{\min}(W_o(N))^{1/2} }}\Bigg].
    % \end{align*}
    \begin{align*}
        \AR(\hat{L}_k) - \SR(\hat{L}_k) &\leq \varepsilon \paren{\frac{ \rho^{2k} \sigma_0^2 + r_k(\rho)\; \sigma_w^2}{\sigma_{\wedge}^2 \lambda_{\min}(W_o(N))^{1/2}}} \\
        &\quad \times \Bigg[2 \sqrt{n}
        %\paren{ \norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2
            \paren{\sigma_{\vee}^2 + \paren{\frac{\sigma_v}{\sigma_{\wedge}^2\lambda_{\min}(W_o(N))^{1/2} }}^2 }^{1/2} 
        %\paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}} {\paren{\norm{\bar{\Sigma}}_2 \norm{H_k}_2^2 + \norm{\Sigma_v}_2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2 }^{1/2}}  \\
        %+ \varepsilon^2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2.
        +\varepsilon \paren{ \frac{1}{\sigma_{\wedge}^2 \lambda_{\min}(W_o(N))^{1/2} }}\Bigg].
    \end{align*}
    
    Furthermore, when $\lambda_{\min}(W_o(N))^{1/2} \geq \frac{\sigma_v}{\sigma_{\wedge}^2}$, we have
    \begin{align*}
        \AR(\hat{L}_k) - \SR(\hat{L}_k) &\leq \varepsilon \paren{\frac{\sigma_{\wedge}^2 \lambda_{\min}(W_o(N))^{1/2}}{\sigma_{\wedge}^4 \lambda_{\min}(W_o(N)) + \sigma_v^2 }\paren{ \rho^{2k} \sigma_0^2 + r_k(\rho)\; \sigma_w^2} } \\
        & \quad \times \Bigg[2 \sqrt{n}
        %\paren{ \norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2
            \paren{\sigma_{\vee}^2 + \sigma_v^2 \paren{\frac{\sigma_{\wedge}^2 \lambda_{\min}(W_o(N))^{1/2}}{\sigma_{\wedge}^4 \lambda_{\min}(W_o(N)) + \sigma_v^2 }}^2 }^{1/2} \\ & \quad \quad   \quad
        %\paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}} {\paren{\norm{\bar{\Sigma}}_2 \norm{H_k}_2^2 + \norm{\Sigma_v}_2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2 }^{1/2}}  \\
        %+ \varepsilon^2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2.
        +\varepsilon \paren{\frac{\sigma_{\wedge}^2 \lambda_{\min}(W_o(N))^{1/2}}{\sigma_{\wedge}^4 \lambda_{\min}(W_o(N)) + \sigma_v^2 }}\Bigg].
    \end{align*}
    
   % \Bruce{Alternatively, we could say: Suppose that $\sigma_{min}(\calO_N) \geq \frac{\sigma_{min}(\Sigma_v)}{\sigma_{min}(\bar \Sigma}$. Then the gap between $\AR(L_k)$ and $\SR(L_k)$ is upper bounded by 
   % the same as above, but with all occurrences of 
   % \[
   %     \frac{1}{\sigma_{min}(W_o(N))^{1/2} \sigma_{min} (\bar\Sigma)}        
   % \]
   % replaced by 
   %   \[
   %     \frac{\sigma_{min}(W_o(N))^{1/2} \sigma_{min} (\bar\Sigma)}{\paren{\sigma_{min}(W_o(N))^{1/2} \sigma_{min} (\bar\Sigma)}^2 + \sigma_{min}(\Sigma_v)}
   % \]
   % This might be more convoluted than necessary, particularly given the two column format, although if the gains in terms of comparison to the lower bound are worth it, we can write it that way. 
   % }
    \label{thm:ub}
\end{theorem}
The upper bound on the gap decreases as the minimum eigenvalue of the observability gramian increases. This indicates that as the observability of the system becomes uniformly large, the gap between standard and adversarial risk for the nominal Kalman estimator will decrease. Perhaps counter-intuitively, when observability is poor in some direction, i.e. when $\lambda_{\min}\paren{W_o(N)}$ is small, increasing the sensor noise $\sigma_v$ will actually \emph{decrease} the above upper bound, as long as $\lambda_{\min}\paren{W_o(N)} \geq \frac{\sigma_v}{\sigma_\wedge^2}$. This aligns with results demonstrating that injecting artificial noise can improve the robustness of state observers \citep{doyle1979robustness}, and is further consistent with our interpretation of noise as a regularizer following Lemma \ref{lem:SR closed form}.

%In both the upper and lower bound, there is an interesting dependence on the stability of the system, measured by $\rho$ for our scaled orthogonal systems, and the state estimate time $k$. In particular, both the upper and lower bounds increase with $\rho^k$. %This suggests that when filtering in explosive systems, there will be a very large gap between the standard and adversarial risk. This is reasonable, because small adversarial attacks can be amplified over time.    

%\Thomas{Is there supposed to be a plus sign? leq sign?} \Bruce{multiplication, didn't fit on one line and couldn't figure out how to put a linebreak in parens}

% \begin{align*}
%     \norm{L_k}_2 &= \norm{\paren{A^k \Sigma_0 \calO_N\T + \Gamma_k \paren{I_N \otimes \Sigma_w} \tau_N\T} \paren{\calO_N \Sigma_0 \calO_N\T+\tau_N\paren{I_N \otimes \Sigma_w} \tau_N\T +\paren{I_{T+1} \otimes \Sigma_v}}\I}_2 \\
%     &\leq  \norm{A^k \Sigma_0 \calO_N\T \paren{\calO_N \Sigma_0 \calO_N\T+\sum_{i=1}^T \calZ^i \calO_N \Sigma_w \calO_N\T \calZ\T^i +\paren{I_{T+1} \otimes \Sigma_v}}\I}_2 \\ 
%     & \quad \quad + \sum_{i=1}^k \norm{  A^{k-i} \Sigma_w  \calO_N\T \calZ\T^i \paren{\calO_N \Sigma_0 \calO_N\T+\sum_{i=1}^T \calZ^i \calO_N \Sigma_w \calO_N\T \calZ\T^i +\paren{I_{T+1} \otimes \Sigma_v}}\I}_2 \\
%     &\leq \norm{A^k \Sigma_0^{1/2}}_2 \norm{ \Sigma_0^{1/2} \calO_N\T \paren{\calO_N \Sigma_0 \calO_N\T+\sum_{i=1}^T \calZ^i \calO_N \Sigma_w \calO_N\T \calZ\T^i +\paren{I_{T+1} \otimes \Sigma_v}}\I}_2 \\ 
%     & \quad \quad + \sum_{i=1}^k \norm{  A^{k-i} \Sigma_w^{1/2}}_2 \norm{  \Sigma_w^{1/2} \calO_N\T \calZ\T^i \paren{\calO_N \Sigma_0 \calO_N\T+\sum_{i=1}^T \calZ^i \calO_N \Sigma_w \calO_N\T \calZ\T^i +\paren{I_{T+1} \otimes \Sigma_v}}\I}_2 \\
%     & \leq \norm{A^k \Sigma_0^{1/2}}_2 \norm{ \Sigma_0^{1/2} \calO_N\T \paren{\calO_N \Sigma_0 \calO_N\T+\sum_{i=1}^T \calZ^i \calO_N \Sigma_w \calO_N\T \calZ\T^i +\sigma_{min}(\Sigma_v) I }\I}_2 \\ 
%     & \quad \quad + \sum_{i=1}^k \norm{  A^{k-i} \Sigma_w^{1/2}}_2 \norm{  \Sigma_w^{1/2} \calO_N\T \calZ\T^i \paren{\calO_N \Sigma_0 \calO_N\T+\sum_{i=1}^T \calZ^i \calO_N \Sigma_w \calO_N\T \calZ\T^i +\sigma_{min}(\Sigma_v) I}\I}_2 \\
%      & \leq \norm{A^k \Sigma_0^{1/2}}_2 \norm{ \Sigma_0^{1/2} \calO_N\T \paren{\calO_N \Sigma_0 \calO_N\T +\sigma_{min}(\Sigma_v) I }\I}_2 \\
%     & \quad \quad + \sum_{i=1}^k \norm{  A^{k-i} \Sigma_w^{1/2}}_2 \norm{  \Sigma_w^{1/2} \calO_N\T \calZ\T^i \paren{ \calZ^i \calO_N \Sigma_w \calO_N\T \calZ\T^i +\sigma_{min}(\Sigma_v) I}\I}_2 
%     \end{align*}
    
%    Applying Lemma \ref{lem:minimum nonzero singular value} with $\Sigma_0^{1/2} \calO_N\T$ and $\Sigma_w^{1/2} \calO_N\T \calZ\T^i$, we see that components on the right of each product in our above bound are bounded by
%    \[1/\sqrt{\lambda_{n}\paren{\Sigma_0^{1/2} \calO\T \calO \Sigma_0^{1/2}}}\]
%    and  
%    \[1/\sqrt{\lambda_{min pos}\paren{\Sigma_w^{1/2} \calO\T \calZ\T^i\calZ^i \calO \Sigma_w^{1/2}}}.\]
%    Note, however, that for any $\calZ^i$ where $i \leq T$, if $A$ has singular values $\sigma_1\geq ...\geq \sigma_m \geq 0 = ... =0$, then $A \calZ^i \T$ has singular values $\sigma_1\geq ...\geq \sigma_k \geq 0 = ... =0$ where $k \leq m$. This can be seen by the fact that if $A = U \bmat{\Sigma & 0} V\T$, then 
%    \begin{align*}
%        A \calZ^i \T \calZ^i A\T  = U \Sigma V\T \bmat{I & \\ & 0} V \Sigma U\T = U \Sigma  \bmat{I & \\ & 0} \Sigma U\T
%    \end{align*}
%    Then 
%    \[1/\sqrt{\lambda_{min pos}\paren{\Sigma_w^{1/2} \calO\T \calZ\T^i\calZ^i \calO \Sigma_w^{1/2}}} \leq 1/\sqrt{\lambda_{min pos}\paren{\Sigma_w^{1/2} \calO\T \calO \Sigma_w^{1/2}}}\]
%    Then the above quantity is bounded by
%    \begin{align*}
%        \frac{\norm{A^k \Sigma_0^{1/2}}_2} {\sqrt{\lambda_{min}\paren{\Sigma_0^{1/2} \calO\T \calO \Sigma_0^{1/2}}}} &+ \sum_{i=1}^k \frac{\norm{  A^{k-i} \Sigma_w^{1/2}}_2}{\sqrt{\lambda_{min}\paren{\Sigma_w^{1/2}  \calO\T \calO \Sigma_w^{1/2}}}} \\
%    &\leq \frac{\norm{A^k \Sigma_0^{1/2}}_2} {\sqrt{\lambda_{min}\paren{\calO\T \calO}} \sigma_{min}\curly{\Sigma_0}^{1/2}} + \sum_{i=1}^k \frac{\norm{  A^{k-i} \Sigma_w^{1/2}}_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}} \sigma_{min}\curly{\Sigma_w}{1/2}} \\
%    &\leq \kappa\paren{\Sigma_0^{1/2}} \frac{\norm{A^k}_2} {\sqrt{\lambda_{min}\paren{\calO\T \calO}} } + \kappa \paren{\Sigma_w}^{1/2} \sum_{i=1}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}
%    \end{align*}

%Combining these bounds, we have that the Kalman estimator satisifies 
%\begin{align*}
%    \AR(L)-\SR(L) \leq \varepsilon \norm{L}_2 \paren{2 \sqrt{n}\paren{\norm{\bar{\Sigma}}_2 \norm{H_k}_2^2 + \norm{\Sigma_v}_2 \norm{L}_2^2}^{1/2}  + \varepsilon \norm{L}_2}
%\end{align*}
%
%Let $\kappa = \max\curly{\kappa{\Sigma_0}, \kappa{\Sigma_w}}$. Substituting our upper bound for $\norm{L}_2$, 
%\begin{align*}
%    \AR(L) - \SR(L) &\leq \varepsilon \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}} \\
%    & (2 \sqrt{n} \paren{\norm{\bar{\Sigma}}_2 \norm{H_k}_2^2 + \norm{\Sigma_v}_2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2 }^{1/2}  \\
%    & + \varepsilon \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}})
%\end{align*}




\section{Numerical Results}
\label{sec: numerical}




\begin{figure}
    \centering
    \includegraphics[width = 0.5\textwidth]{arSrTradeoff_linearInverse.pdf}
    \caption{Approximation of Pareto boundaries of $(\SR,\AR)$ arising in the general linear inverse problem for random normalized matrices of different condition numbers, $\kappa$. As the condition number increases, the tradeoff becomes more severe.} %{\color{red} Try to adjust parameters in learning this to make the curves smoother.}} %{\color{red} Need to say what this?}}
    \label{fig:liproblem}
\end{figure}


\begin{figure*}[t]
    \centering
    \makebox[\textwidth][c]{
        \subfloat[][Initial state estimation $k=0$]{
        \includegraphics[width = 0.5 \textwidth]{arSrTradeoff_smoothing_orthogonal.pdf}
        }
        \subfloat[][Final state estimation $k=N$]{
        \includegraphics[width = 0.5 \textwidth]{arSrTradeoff_filtering_orthogonal.pdf}
        }
    }

    \caption{Pareto boundaries of $(\SR, \AR)$ for initial and final state estimation. The observability of a the system is determined by $\alpha$. When $\alpha$ approaches one, observability decreases and the tradeoff between $\SR$ and $\AR$ becomes more severe.}
    
    %Plots are not finalized. The first plot has the tradeoff curves with a linear filter, while the second plot has the tradeoff curves with  a NN. We can see that they look quite similar. Note that the NN plot uses an under approximation for AR, and thus can fall below the linear tradeoff curves WITHOUT disproving Nik's conjecture that linear is optimal. The third has tradeoff curves for filtering rather than initial state estimation. Working out optimization hyperparameters for the third plot }
    \label{fig:state estimation tradeoffs}
\end{figure*}

% \begin{figure}[]
%     \makebox[\textwidth][c]{
%         \subfloat[][$\gamma = 1/2$, $n = 100,\, 500,\, 5000$]{
%         \includegraphics[width=0.6\textwidth]{./figs/Gamma2het.png}
%         }
%         \hspace{-1.3cm}
%         \subfloat[][$\gamma = 1/5$, $n = 100,\, 500,\, 5000$]{
%         \includegraphics[width=0.6\textwidth]{./figs/Gamma5het.png}
%         }
%     }
%     \caption{}
%     \label{}
% \end{figure}


We now demonstrate that the theoretical results shown in the previous two sections predict the tradeoffs arising in linear inverse problems. %Additional experiments showing that similar trends arise when using nonlinear estimators can be found in Appendix~\ref{ss:additional experiments}.

\begin{figure}
    \centering
    \includegraphics[width=0.5\textwidth]{kfVsAdvkfVaryingObservability.pdf}
    \caption{$\SR$ vs $\AR$ for a nominal Kalman smoother and an adversarially robust smoother, where observability of the underlying system increases in the direction of the arrows. For any fixed value of the standard risk, the adversarially robust smoother achieves a  lower adversarial risk than the nominal smoother. On the right side of the plot, when observability is low, the adversarial risk of the adversarially robust smoother is much lower than that of the nomimal smoother. This difference shrinks as we move to the left and observability increases.} %{\color{red} Make plot easier to read, chage lengeds, have the arrows correspond to some data points}
    \label{fig:kf vs adv kf}
\end{figure}

\paragraph{Adversarial Linear Inverse Problems} In Figure~\ref{fig:liproblem}, we approximate the $(\SR, \AR)$ Pareto boundary for a linear inverse problem, where $A_\star \in \R^{4\times 4}$ is generated randomly by multiplying a prescribed positive diagonal matrix on the left and right by random orthogonal matrices, such that it has condition number $1$, $3$, or $10$. The positive diagonal matrices are normalized to have $\ell^2$ norm $1$ such that the resulting $A_\star$ all have the same Frobenius norm, regardless of their condition number. We also set $\Sigma_x = I$,  $\Sigma_w = 0.1 I$, and provide the adversary a budget of $\varepsilon = 0.5$. We see that the severity of the tradeoff depends on the condition number of the nominal solution $A_\star$. 

\paragraph{Pareto Curves for Adversarial Kalman Filtering} In this experiment we compute the Pareto-optimal frontier for adversarially robust  Kalman filtering on systems with varying observability. We consider the system defined by the tuple
%or initial and final state estimation for systems with varying observability for the system 
$\paren{A, C, \Sigma_0, \Sigma_w, \Sigma_v, N} = \paren{\bmat{\alpha & \beta \\ -\beta & \alpha}, \bmat{1 & 0}, I, 0.1 I, 0.1, 5}$ where $\alpha^2 + \beta^2 =1$, and we vary $\alpha$. As $\alpha$ approaches one the minimum eigenvalues of the observability gramian become small. In particular,  for $\alpha = 0.95$, $\alpha=0.98$, $\alpha=0.99$, the minimum eigenvalues of the observability gramian are given by $1.22, 0.81$ and $0.58$ respectively. The adversarial budget is set to $\varepsilon = 0.5$. Figure~\ref{fig:state estimation tradeoffs} shows the resulting tradeoff curves which demonstrate that as observability decreases, both $\SR$ and $\AR$ increase, as do the distance between the extremes of the Pareto curve. 
The results therefore support \S\ref{ss:bounding state estimation error}, where we showed shrinking the eigenvalues of $W_o(N)$ increases the severity of the tradeoff between $\SR$ and $\AR$.  %Although the theory was developed only for linear estimators, we see that similar trends hold for nonlinear function approximators, such as neural networks. This suggests that the phenomena arising in the bounds are not limited by the assumption that the estimators are linear. 



\paragraph{Tradeoffs of Kalman Smoother versus Adversarially Robust Kalman Smoother} In Figure \ref{fig:kf vs adv kf}, we demonstrate the impact of adversaries on the risk incurred by an estimator optimized for $\SR$ versus $\AR$.
 We consider initial state estimation of a system defined by the tuple $(A, C, \Sigma_0, \Sigma_w, \Sigma_v, N) = \paren{\bmat{1 & \rho \\ 0 & 1}, \bmat{1 & 0}, I, 0.1 I, 0.1, 5}$, where we vary $\rho$ from $0.1$ to $\sqrt{10}$, such that the eigenvalues of $W_o(N)$ increase as $\rho$ increases. The adversarial budget is fixed at $\varepsilon = 0.5$.  Evaluating the nominal and robust estimators on this class of systems, we see that the adversarially robust smoother has significantly smaller adversarial risk compared to the nominal Kalman smoother when observability is low. As observability increases, this advantage shrinks. This suggests that considering the tradeoffs from adversarially robust state estimation is most important when observability is low. %{\color{red} Carefully considering tradeoffs is most important when observability is low?}
 
 \paragraph{Tradeoffs with Nonlinear State Estimators}
 In Figure~\ref{fig:NN tradeoffs}, we demonstrate empirically that the fundamental tradeoffs are not overcome by using nonlinear state estimators. In particular, we consider the system defined by  $(A, \Sigma_0, \Sigma_w, \Sigma_v, N) = \paren{\bmat{1 & 1 \\ 0 & 1}, I, 0.1 I, 0.1, 5}$, and $C$ as in the legend of the plots. We set the adversarial budget to $\epsilon = 0.5$. We solve for the Pareto boundary for the linear estimator as in Figure~\ref{fig:state estimation tradeoffs}. We also consider a two layer network with ten neurons per layer and ReLU activation functions. To train this network, we perform an SGD procedure similar to that used in the linear case, with the exception being that we do not solve the exact adversary corresponding to each data point, but rather apply 100 steps of gradient ascent to find the adversarial perturbation. Once the neural network is trained, this same approach to find the adversary is used again to estimate the adversarial risk of the resultant estimator. As a result, the tradeoff curves shown for the neural network are an under approximation to the true values of $\AR$ achieved   . As such, it appears that the neural network is getting roughly the same tradeoff curves as the linear estimator, suggesting that linear estimators may be optimal for minimizing the adversarial risk.


 

%from five data points. In particular, we consider the system defined by \eqref{eq:ss} where $A$ is a simple integrator, i.e. $A = \bmat{1 & 1\\ 0 1& 1}$, and $C$ takes the values specified in the legend of the leftmost plot. When paired with $A$, these values of $C$ result in decreasing observability as we move down the legend. In addition to the linear filtering state estimation discussed in the theory, we train neural networks to estimate the state of the system.  

%https://www.overleaf.com/project/61509e385f12bed011702d72 
 \begin{figure*}[t]
    \centering
    \makebox[\textwidth][c]{
        \subfloat[][Linear estimator]{
        \includegraphics[width = 0.5 \textwidth]{arSrTradeoff_nonorthogonal/arSrTradeoff_smoothing.pdf}
        }
        \subfloat[][Neural Network]{
        \includegraphics[width = 0.5 \textwidth]{arSrTradeoff_nonorthogonal/arSrTradeoff_smoothing_NNFilter.pdf}
        }
    }

    \caption{Pareto boundaries of $(\SR, \AR)$ for initial state estimation for a variety of measurement matrices $C$ by both a linear state estimator and a neural network (Lower bound on $\AR$ plotted for NN). As the first entry of $C$ decreases, the tradeoff curve becomes more severe. The trade-offs are not alleviated by a nonlinear estimator.} %{\color{red} In the right plot, change it to estiamte of $\AR$. Also make colors match up with the previous plots.}
    
    %Plots are not finalized. The first plot has the tradeoff curves with a linear filter, while the second plot has the tradeoff curves with  a NN. We can see that they look quite similar. Note that the NN plot uses an under approximation for AR, and thus can fall below the linear tradeoff curves WITHOUT disproving Nik's conjecture that linear is optimal. The third has tradeoff curves for filtering rather than initial state estimation. Working out optimization hyperparameters for the third plot }
    \label{fig:NN tradeoffs}
\end{figure*}
 
% could be interesting to see how adversarial risk compares to H-infinity cost. Want to say adversarial robustness is a nice intermediary that can leverage statistical and concentration properties of stochastic optimization while ensuring robustness where it matters -- want to show that AR is (much?) less conservative than full robust optimization H-infinity.
\section{Conclusion}
\label{sec:conclusion}

We analyzed the robustness-accuracy tradeoffs arising in linear inverse problems. We did this in two parts. We first provided an algorithm to solve for the optimal adversarial perturbation, which can be used to trace out the Pareto boundary. We then bounded the gap between the adversarial and standard risk in terms of the spectral properties of the underlying linear model. These bounds generalize the robustness-accuracy tradeoffs arising in the classification and linear regression settings. We then specialized these general results to study the tradeoffs arising in adversarial Kalman Filtering for linear systems, where we demonstrated that the accuracy decrease of the nominal Kalman estimator under adversarial measurements can be bounded by the eigenvalues of the observability gramian, a control-theoretic object. An interesting avenue of future work is to have an analytic characterization of the solutions $A_\lambda$ along the Pareto boundary of the linear inverse problem, such that we can not only bound the fragility of the nominal solution, but also the conservatism of adversarially robust solutions.
% The application of the general results to study dynamical systems provides the a\textbf{} characterization of robustness-accuracy tradeoffs for state estimation.
On the control side, we pave the way for numerous avenues for future work including the analysis of robustness-performance tradeoffs in LQR control. It would also be interesting to study the tradeoffs arising in estimation from an infinite horizon perspective. 
\appendix

%\section*{Supplementary Material}


\section{Proofs from Section \ref{sec:LS}}

\subsection{Proof of Correctness for Algorithm \ref{alg:getPerturbation}} \label{ss:proof of inner max algorithm}

Recalling the optimization problem 
\begin{align*}
    \maximize_{\delta \in \R^n}&\quad \delta^\top A^\top A \delta - 2\delta^\top A^\top b \tag{P}\\
    \subjectto &\quad \delta^\top \delta \leq \varepsilon^2,
\end{align*}
and the corresponding KKT conditions:
\begin{align*}
    2(\lambda^* I-A^\top A)\delta^* + 2A^\top b &= 0 \\
    \lambda^* ({\delta^*}^\top \delta^* - \varepsilon^2) &= 0 \\
    (\lambda^* I -A^\top A) &\succeq 0.
\end{align*}
The third condition implies that $\lambda^* \geq \sigma_1^2$. We assume $\sigma_1 > 0$, otherwise the problem is trivial. Then using the SVD of $A$ to re-arrange the first stationarity condition, we get
\begin{align*}
    (\lambda^* I-\Sigma^\top \Sigma) V^\top \delta^* &= \Sigma^\top U^\top b.
\end{align*}
Maximizing a convex function over a convex set achieves its maximum on the boundary; it suffices to search over $\delta^\top \delta = \varepsilon^2$. We now consider two cases: $\lambda^* > \sigma_1^2$ and $\lambda^* = \sigma_1^2$. In the first case, we know $\lambda^* I-\Sigma^\top \Sigma$ must be invertible, and thus
\begin{align*}
    \delta^* &= -V (\lambda^* I-\Sigma^\top \Sigma)^{-1} \Sigma^\top U^\top b \\ %\implies
    \varepsilon^2 = {\delta^*}^\top \delta^* &=  b^\top U \Sigma (\lambda^* I - \Sigma^\top \Sigma)^{-2} \Sigma^\top U^\top b \\
    &= \sum_{i=1}^{\min\curly{n,p}} \frac{(b^\top u_i)^2 \sigma_i^2}{(\lambda^* - \sigma_i^2)^2},
\end{align*}
where $u_i$ are the columns of $U$. Observe that
\begin{align*}
    f(\lambda) &:= \sum_{i=1}^{\min\curly{n,p}} \frac{(b^\top u_i)^2 \sigma_i^2}{(\lambda - \sigma_i^2)^2},
\end{align*}
is a strictly monotonically decreasing function when $\lambda > \sigma_1^2$, and converges to $0$ when $\lambda \to \infty$. This implies there is a unique $\lambda^*$ such that $f(\lambda^*) = \varepsilon^2$, which can be numerically solved for in various ways, such as bisection.

Now we consider the case where $\lambda^* = \sigma_1^2$. In this case, $\delta^*$ will no longer be unique, and will come in the form
\begin{align*}
    \delta^* &= -V (\sigma_1^2 I-\Sigma^\top \Sigma)^{\dagger} \Sigma^\top U^\top b + cv,
\end{align*}
where $^\dagger$ denotes the Moore-Penrose pseudoinverse, and $v$ is any unit vector lying in the null-space of $\paren{\sigma_1^2 I -\Sigma^2 }V^\top$, which is precisely characterized in this case by
\begin{align*}
    \ker\paren{\paren{\sigma_1^2 I-\Sigma^2}V^\top} &= \Span\paren{\curly{v_i : \sigma_i^2 = \sigma_1^2}},
\end{align*}
with $v_i$ denoting the $i$th column of $V$. To find the appropriate scaling $c$, we observe
\begin{align*}
    {\delta^*}^\top \delta^* &=  b^\top U \Sigma \paren{(\sigma_1^2 I - \Sigma^\top \Sigma)^{\dagger}}^2 \Sigma^\top U^\top b + c^2 \\
    &= \sum_{i:\sigma_i < \sigma_1} \frac{(b^\top u_i)^2 \sigma_i^2}{(\sigma_1^2 - \sigma_i^2)^2} =\varepsilon^2 \\
    c &= \sqrt{\varepsilon^2 - \sum_{i:\sigma_i < \sigma_1} \frac{(b^\top u_i)^2 \sigma_i^2}{(\sigma_1^2 - \sigma_i^2)^2}}.
\end{align*}
Combining our precise characterization of $\ker\paren{\paren{\sigma_1^2 I-\Sigma^\top\Sigma}V^\top}$ using the columns of $V$, and the formula for $c$, we can extract an optimal perturbation vector $\delta^*$. Therefore, we have demonstrated that (P), as well as extracting its optimal solution, can be solved to arbitrary precision.



\subsection{Proof of Theorem \ref{thm:generalUBLB}} \label{ss:proof of general UBLB}
 First recall the definitions of SR and and AR:
\begin{align*}
    \SR(A) &= \Ex\brac{\norm{y - Ax}_2^2} \\
    \AR(A) &= \Ex\brac{\max_{\norm{\delta}_2 \leq \varepsilon}\norm{y - A(x+\delta)}_2^2}.
\end{align*}
Given fixed $y, x$, let us define the quantity $d(A) := y - Ax$. Writing out the inner maximization of AR we have:
\begin{align*}
    \max_{\norm{\delta}_2 \leq \varepsilon}\norm{y - A(x+\delta)}_2^2 &= \max_{\norm{\delta}_2 \leq \varepsilon}\norm{d(A) - A\delta}_2^2.
\end{align*}
Observe that this is equivalent to the problem
\begin{align*}
    \minimize_{s} \quad& s \tag{P1}\\
    \subjectto \quad& s - \norm{d(A) - A\delta}_2^2 \geq 0 \text{ for all } \delta^\top \delta \leq \varepsilon^2.
\end{align*}

We now recall the S-lemma for quadratic functions.
\begin{lemma}[S-lemma] \label{lemma: S-lemma}
    Given quadratic functions $p(x), q(x) : \R^n \to \R$, suppose there exists $x$ such that $p(x) > 0$. Then,
    \[
    p(x) \geq 0 \implies q(x) \geq 0 \text{ for all }x
    \]
    if and only if
    \[
    \exists t \geq 0 \text{ such that }q(x) \geq tp(x)  \text{ for all }x.
    \]
\end{lemma}

Using this lemma, we set $p(\delta) = \varepsilon^2 - \delta^\top \delta$, $q(\delta) = s - \norm{d(A) - A\delta}_2^2 \geq 0$. We observe that trivially, there exists $\delta = 0$ such that $p(\delta) > 0$. Now given feasible $s$ for (P1), we observe that by our constraints, any $\delta$ such that $p(\delta) \geq 0$ immediately implies $q(\delta) \geq 0$. By the S-lemma, this is equivalent to the existence of some $t \geq 0$ such that
\[
q(\delta) - tp(\delta) = s - \norm{d(A) - A\delta}_2^2 - t(\varepsilon^2 - \delta^\top \delta) \geq 0
\]
for all $\delta$. Therefore, we can re-write the optimization problem (P1) into
\begin{align*}  
    \minimize_{s} \quad& s \tag{P2}\\
    \subjectto \quad& \exists t \geq 0 \text{ s.t. }s - \norm{d(A) - A\delta}_2^2 - t(\varepsilon^2 - \delta^\top \delta) \geq 0 \text{ for all }\delta.
\end{align*}
Re-arranging the terms in the quadratic expression, we get:
\begin{align*}
    s - \norm{d(A) - A\delta}_2^2 - t(\varepsilon^2 - \delta^\top \delta) &= s - \paren{\delta^\top A^\top A \delta - 2d(A)^\top A \delta + \norm{d(A)}_2^2} - t(\varepsilon^2 - \delta^\top \delta) \\
    &= \delta^\top \paren{tI - A^\top A}\delta + 2d(A)^\top A\delta + \paren{s - t\varepsilon^2 - \norm{d(A)}_2^2}.
\end{align*}
Now we recall a property of Schur complements.

\begin{lemma}[Schur Complement]
    Given $p(x) = x^\top P x + b^\top x + c$, we have
    \begin{align*}
    p(x) \geq 0 \;\forall\; x &\iff \bmat{P & b \\ b^\top & c} \succeq 0 \\
    &\iff P \succeq 0, \; c - b^\top P^\dagger b \geq 0.
    \end{align*}
\end{lemma}

Applying this to (P2), we see the constraints can be re-written
\begin{align*}
    &\exists t \geq 0 \text{ s.t. }s - \norm{d(A) - A\delta}_2^2 - t(\varepsilon^2 - \delta^\top \delta) \geq 0 \text{ for all }\delta \\
    \iff\quad& \exists t \geq 0, \; tI - A^\top A \succeq 0, \; s - t\varepsilon^2 - \norm{d(A)}_2^2 - d(A)^\top A (tI - A^\top A)^\dagger A^\top d(A) \geq 0 \\
    \iff\quad& \exists t\geq \lambda_{\max}(A^\top A),\;s - t\varepsilon^2 - \norm{d(A)}_2^2 - d(A)^\top A (tI - A^\top A)^\dagger A^\top d(A) \geq 0.
\end{align*}
Therefore, we get the optimization problem
\begin{align*}
    \minimize_{s,t} \quad& s \\
    \subjectto \quad& t\geq \lambda_{\max}(A^\top A)\\
    &s - t\varepsilon^2 - \norm{d(A)}_2^2 - d(A)^\top A (tI - A^\top A)^\dagger A^\top d(A) \geq 0.
\end{align*}
However, this is clearly equivalent and has the same optimal value as the following problem
\begin{align*}
    \minimize_{t} \quad& t\varepsilon^2 + \norm{d(A)}_2^2 + d(A)^\top A (tI - A^\top A)^\dagger A^\top d(A) \\
    \subjectto \quad& t\geq \lambda_{\max}(A^\top A).
\end{align*}
Notice that so far we are simply considering equivalent formulations to the original optimization. The ensuing step is where the lower and upper bounds \ref{eq: s-lemma lower bound} and $\ref{eq: s-lemma upper bound}$ arise. Recall the Neumann series, where since $t \geq \lambda_{\max}(A^\top A)$, we have
\begin{align*}
    (tI - A^\top A)^{-1} &= \frac{1}{t}\paren{I - \frac{1}{t}A^\top A }^{-1} \\
    &= \frac{1}{t}\paren{I + \frac{1}{t}A^\top A + \frac{1}{t^2}\paren{A^\top A}^2 + \cdots }.
\end{align*}
From the Neumann series, we see that we can upper and lower bound the inverse using geometric series of the largest and smallest eigenvalues of $A^\top A$, respectively,
\begin{align*}
    \frac{1}{t - \lambda_{\min}(A^\top A)} I \preceq (tI - A^\top A)^{-1} \preceq \frac{1}{t - \lambda_{\max}(A^\top A)} I
\end{align*}
From now on, we will deal with the inverse, since instead of the pseudo-inverse we can take the infimum of the above problem, which is bounded from below. Let us consider the lower bound first--the upper bound follows using the exact same techniques. We have
\begin{align*}
    t\varepsilon^2 + \norm{d(A)}_2^2 + d(A)^\top A (tI - A^\top A)^{-1} A^\top d(A) &\geq t\varepsilon^2 + \norm{d(A)}_2^2 + d(A)^\top A \paren{\frac{1}{t - \lambda_{\min}(A^\top A)} I } A^\top d(A) \\
    &= t\varepsilon^2 + \norm{d(A)}_2^2 + \frac{1}{t - \lambda_{\min}(A^\top A)}\norm{A^\top d(A)}_2^2.
\end{align*}
Therefore, we have
\begin{align*}
    \max_{\norm{\delta}_2 \leq \varepsilon}\norm{x_k - A(Y_T+\delta)}_2^2 &\geq \norm{d(A)}_2^2 + \min_{t > \lambda_{\max}(A^\top A)} t\varepsilon^2  + \frac{1}{t - \lambda_{\min}(A^\top A)}\norm{A^\top d(A)}_2^2.
\end{align*}
We now make a second relaxation:
\begin{align*}
    \norm{d(A)}_2^2 + \min_{t > \lambda_{\max}(A^\top A)} t\varepsilon^2  + \frac{1}{t}\norm{A^\top d(A)}_2^2 &\geq \norm{d(A)}_2^2 + \min_{t \geq 0} t\varepsilon^2  + \frac{1}{t - \lambda_{\min}(A^\top A)}\norm{A^\top d(A)}_2^2 \\
    &= \norm{d(A)}_2^2 + 2\varepsilon \norm{A^\top d(A)}_2 + \varepsilon^2 \lambda_{\min}(A^\top A),
\end{align*}
which we get by deriving the unconstrained minimizer $t^* = \frac{\norm{A^\top d(A)}_2}{\varepsilon} + \lambda_{\min}(A^\top A)$. Now putting expectations on both sides of the inequality, we get
\begin{align*}
    \AR(A) &\geq \SR(A) + 2\varepsilon\;\Ex\brac{\norm{A^\top (x_k - AY_T)}_2} + \varepsilon^2 \lambda_{\min}(A^\top A).
\end{align*}
\hfill$\blacksquare$

\subsection{Proof of Corollary \ref{cor: Astar adv tradeoff}}

We note that the upper and lower bounds boil down to upper and lower bounding $\Ex\brac{\norm{A_\star^\top (y - A_\star x)}_2}$. We first observe $A_\star^\top (y - A_\star x) = A_\star^\top w \sim \calN(0, \sigma_w^2 A_\star^\top A_\star)$. The upper bound therefore comes from an application of Jensen's inequality:
\begin{align*}
    \Ex\brac{\norm{A_\star^\top w}_2} &= \Ex\brac{\sqrt{\norm{A_\star^\top w}_2^2}} \\
    &\leq \sqrt{\Ex\brac{\norm{A_\star^\top w}^2_2}} \\
    &= \sqrt{\tr\paren{A_\star^\top A_\star}}.
\end{align*}
For the lower bound, we first consider the full SVD of $A_\star = U\Sigma V^\top$, where $U, V$ are orthogonal matrices. Therefore, we have
\begin{align*}
    \norm{A^\top w}_2^2 &= \norm{\Sigma U^\top w}_2^2 \\
    &= \sum_{i=1}^{p} \sigma_i^2 w_i^2.
\end{align*}
Using the $\ell^2$-$\ell^1$ equivalence of norms, we have
\begin{align*}
    \sqrt{\sum_{i=1}^{} \sigma_i^2 w_i^2} \geq \frac{1}{\sqrt{p}} \sum_{i=1}^p \sigma_i \abs{w_i}.
\end{align*}
Putting this together, we have
\begin{align*}
    \Ex\brac{\norm{A_\star^\top w}} &\geq \frac{1}{\sqrt{p}} \Ex\brac{\sum_{i=1}^p \sigma_i \abs{w_i}} = \frac{1}{\sqrt{p}} \Ex\brac{\abs{w_1}} \sum_{i=1}^p \sigma_i,
\end{align*}
where $w_1 \sim \calN(0,1)$. The quantity $\Ex\brac{\abs{w_1}}$ is the mean of a folded standard normal, and we may express $\sum_{i=1}^p \sigma_i = \tr\paren{(A_\star^\top A_\star)^{1/2}}$, which yields us the desired lower bound. \hfill $\blacksquare$

\section{Kalman Filtering State Space Solution} \label{ss: kalman state space}
Consider the setting defined in \S\ref{ss:state estimation}, i.e.\ we have a dynamical system which progresses according to 
\begin{align*}
    x_{t+1} &= A x_t + w_t \\
    y_t &= C x_t + v_t
\end{align*}
with $x_0 \sim{} \calN(0,\Sigma_0)$, $w_t \overset{\mathrm{i.i.d.}}{\sim} \calN(0,\Sigma_w)$, $v_t \overset{\mathrm{i.i.d.}}{\sim} \calN(0,\Sigma_v)$. 
Consider estimating state $x_k$ given measurements $y_0, \dots, y_k$. The minimum mean square estimator is given by
\[
    \hat x_k = \min_z \Ex \brac{\norm{z - x_k}_2^2 | y_0, \dots, y_k}.
\]
This can be written as an integral
\[
 \min_z \int_{\R^n} \norm{z - x_k}_2^2 f(x_k | y_0, \dots, y_k) dx_k
\]
where $f$ denotes the conditional density of $x_k$ given the measurements. The objective is convex in $z$, and thus we can find the minimizer by setting the gradient with respect to $z$ to zero. In particular, we have
\begin{align*}
    \frac{d}{dz} \int_{\R^n} \norm{z - x_k}_2^2 f(x_k | y_0, \dots, y_k) &=  \int_{\R^n} \frac{d}{dz} \norm{z - x_k}_2^2 f(x_k | y_0, \dots, y_k) dx_k\\
    &=\int_{\R^n} 2 (z - x_k) f(x_k | y_0, \dots, y_k) dx_k \\
    &=0
\end{align*}
where dominated convergence theorem permits the exchange of integration and differentiation. Therefore, the state estimate may be expressed
\[
    \hat x_k = \int_{\R^n} x_k f(x_k | y_0, \dots, y_k) dx_k = \Ex\brac{x_k | y_0, \dots y_k}.
\]
Thus we can determine the state estimates $\hat x_k$ as the mean of the conditional distribution $f(x_k|y_0, \dots y_k)$. This may be computed recursively. In particular, let $x_k|y_{0:k}$ be the random variable with probability density function $f(x_k|y_0, \dots y_k)$. Then for all $k$, we have that $x_k|y_{0:k}= \calN(\hat x_k^+, P_k^+)$ where 
\begin{equation}
\begin{aligned}
\label{eq:KFequations}
    x_0^- &= 0\\ % &= \paren{C \Sigma_0 C^\top + \Sigma_v}^{-1} \Sigma_0 C^\top y_0 \\
    P_0^- &= \Sigma_0 \\ %- \Sigma_0 C^\top \paren{C \Sigma_0 C^\top + \Sigma_v}^{-1} C\Sigma_0
    \hat x_k^+ &= x_k^- + P_k^- C^\top \paren{C P_k^- C^\top + \Sigma_v}^{-1} \paren{y_k - C\hat x_k^-} \\
    P_{k}^+ & = P_{k}^- - P_k^- C^\top \paren{C P_k^- C^\top + \Sigma_v} C P_k^- \\
    \hat x_{k+1}^- &= A \hat x_k^+\\
    P_{k+1}^- &= A^\top  P_k^+ A + \Sigma_w \\
\end{aligned}
\end{equation}
To see that this is the case, recall that $x_0 \sim \calN(0, \Sigma_0)$, by assumption. Now suppose that $x_k|y_{0:k-1} \sim \calN(\hat x_k^-, P_k^-)$. Observe that 
\[
    \bmat{ x_k|y_{0:k-1} \\ y_k} \sim \calN\paren{ \bmat{\hat x_k^- \\ C \hat x_k^-}, \bmat{P_k^- & P_k^- C^\top \\ C P_k^- & C P_k^- C^\top + \Sigma_v}}
\]
Thus
\begin{align*}
    x_k|y_{0:k} &\sim \calN\paren{x_k^- + P_k^- C^\top \paren{C P_k^- C^\top + \Sigma_v}^{-1} \paren{y_k - C\hat x_k^-} ,P_{k}^- - P_k^- C^\top \paren{C P_k^- C^\top + \Sigma_v} C P_k^-} \\
    &= \calN(x_k^+, P_k^+)
\end{align*}
Now, given $x_k| y_{0:k} \sim \calN(x_k^+, P_k^+)$, observe that $x_{k+1}|y_{0:k} = A x_k|y_{0:k} + w_k \sim \calN(A^\top x_k^+,  A^\top P_k^+ A + \Sigma_w) = \calN(x_{k+1}^-, P_k^-)$.

Using the equations in \eqref{eq:KFequations}, we can write the Kalman filter as a state space system with inputs $y_t$. In particular, if $P_k^-$ and $P_k^+$ are defined as in \eqref{eq:KFequations}, we can let $K_k = P_k^- C^\top \paren{C P_k^- C^\top + \Sigma_v}^{-1}$. Then we may express our state estimates using the following time varying system.
\begin{align*}
    \hat x_{k+1} &= (A - K_k C A) \hat x_k + K_k y_k.
\end{align*}



\section{General Statements and Proofs from Section \ref{sec: Kalman}} \label{ss: kf general statements and proofs}

\subsection{Lemma \ref{lem:kf}}
\textbf{Lemma~\ref{lem:kf}:}\;\:Suppose $k \leq N$. The finite horizon Kalman state estimator is the solution to optimization problem \eqref{eq:sr-kf}, and is given by 
    \begin{align*}
        \hatL_k &= \paren{A^k \Sigma_0 \calO_N\T + \Gamma_k \Sigma_w \tau_N\T}  \paren{\calO_N \Sigma_0 \calO_N\T+\tau_N\paren{I_N \otimes \Sigma_w} \tau_N\T +\paren{I_{N+1} \otimes \Sigma_v}}^{-1}.
    \end{align*}
\textit{Proof:} We know $\SR(L)$ is convex in $L$, thus we may take the derivative of $\SR(L)$ with respect to $L$ and set it to $0$ to solve for $\hatL_k$. Matrix derivatives can be found in \citet{petersen2008matrix}.
    % \begin{align*}
    %     \hatL_k = &\paren{\sigma_0^2 A^k \calO_N^\top + \sigma_w^2 \Gamma_k \tau_N^\top} \paren{\sigma_0^2 \calO_N  \calO_N^\top +
    %     \sigma_w^2 \tau_N \tau_N^\top + \sigma_v^2 I}\I.
    % \end{align*}

\subsection{Proof of Lemma \ref{lem:SR closed form}}
\textit{Proof:} This follows simply by expanding the norm inside the expectation, and noticing that since $x_0$, $W_N$, $V_N$ are defined to be zero-mean Gaussian random vectors, their cross terms vanish. More precisely, we have
\begin{align*}
    \Ex \brac{\norm{x_k - L Y_N}_2^2} &= \Ex\brac{\tr\paren{(x_k - LY_N)(x_k - LY_N)^\top}}\\
    &= \Ex\bigg[\tr\bigg( \paren{A^k - L\calO_N}x_0x_0^\top \paren{A^k - L\calO_N}^\top + \paren{\Gamma_k - L\tau_N}W_NW_N^\top (\Gamma_k - L\tau_N)^\top \\
    &\quad + LV_NV_N^\top L^\top \bigg) \bigg] + 0 \\
    &= \norm{\paren{A^k - L \calO_N}\Sigma_0^{1/2}}_F^2 + \norm{\paren{\Gamma_k - L \tau_N}\paren{I_N \otimes \Sigma_w}^{1/2}}_F^2 + \norm{L \paren{I_{N+1} \otimes \Sigma_v}^{1/2}}_F^2.
\end{align*}
\hfill$\blacksquare$

\subsection{Lemma \ref{thm:AR-SR gap lower bound}}
\textbf{Lemma~\ref{thm:AR-SR gap lower bound}.}\;\:\emph{For any $L \in \R^{n \times p(N+1)}$, the gap between $\AR(L)$ and $\SR(L)$ admits the following lower bound:
    \begin{align}\label{eq:AR-SR gap no exp}
         %2 & \varepsilon\; \Ex\brac{\norm{L^\top \paren{x_k - LY_N}}_2} \nonumber \\ 
         \AR(L) - \SR(L)
         &\geq  2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}}  \tr\bigg(\bigg( L^\top \Big(S \Sigma_0 S^\top + T \paren{I_N \otimes\Sigma_w} T^\top \nonumber + L\paren{I_{N+1} \otimes \Sigma_v} L^\top\Big)L\bigg)^{1/2}\bigg) \\
         &\geq 2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}} \sigma_{\min}(\Sigma_v)^{1/2} \norm{L}_F^2
    \end{align}
    where $S := A^k - L\calO_N$, $T := \Gamma_k - L\tau_N$.
} 


\textit{Proof:} Applying the lower bound~\eqref{eq: s-lemma lower bound}, we have
 \begin{align*}
    \AR(L) &\geq \SR(L) + 2\varepsilon\;\Ex\brac{\norm{L^\top (x_k - LY_T)}_2}.
\end{align*}
 
Then to derive the lower bound \eqref{eq:AR-SR gap no exp}, we observe that the random vector  \begin{align*}
    z &= L^\top (x_k - LY_T) \\
    &= L^\top \paren{S x_0 -T W_T + LV_T)},
\end{align*}
is a zero-mean gaussian with covariance
\begin{align*}
    \Sigma &= L^\top \paren{S\Sigma_0S^\top + T \Sigma_w T^\top + L \Sigma_v L^\top}L.
\end{align*}

We can also write $z = \Sigma^{1/2} w$ where $w \sim \calN(0,I)$.  Consider the diagonalization of $\Sigma^{1/2} = VSV^\top$. Then 
\begin{align*}
    \Ex \brac{\norm{z}_2} &= \Ex\brac{\norm {V S V^\top w}_2} \\
    &= \Ex\brac{\norm{\sum_i s_i w_i v_i }_2},
\end{align*}
where $v_i$ is the $i$th row of $V$ and $s_i$ is the $i$th singular value of $\Sigma^{1/2}$. We have that 
\begin{align*}
    \norm{\sum_i s_i w_i v_i }_2^2 = \sum_{i} s_i^2 w_i^2 v_i^\top v_i = \sum_{i} s_i^2 w_i^2
\end{align*}
We have by equivalence of norms, $\sqrt{\sum_{i=1}^n x_i^2} \geq n^{-1/2} \sum_{i=1}^n \abs{x_i}$. Therefore,
\begin{align*}
    \sqrt{\sum_{i=1}^n s_i^2 w_i^2} \geq \frac{1}{\sqrt{n}} \sum_{i=1}^n s_i \abs{w_i},
\end{align*}
and thus
\begin{align*}
    \Ex \brac{\norm{z}_2} &\geq \frac{1}{\sqrt n} \Ex \brac{\sum_{i=1}^n s_i |w|_i} = \frac{1}{\sqrt n} \Ex\brac{\abs{w}} \sum_{i=1}^n s_i 
\end{align*}
where $w \sim \calN(0,1)$. The quantity $\Ex\brac{\abs{w}}$ is the expected value of a folded standard normal, which is $\sqrt{\frac{2}{\pi}}$, while $\sum_{i=1}^n s_i  = \tr\paren{\Sigma^{1/2}}$. Putting this together, we have that
\begin{align*}
    \AR(L) - \SR(L)
         &\geq  2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}}  \tr\bigg(\bigg( L^\top \Big(S \Sigma_0 S^\top + T \paren{I_N \otimes\Sigma_w} T^\top \nonumber + L\paren{I_{N+1} \otimes \Sigma_v} L^\top\Big)L\bigg)^{1/2}\bigg).
\end{align*}
From the above bound, we may now derive a cruder lower bound from which we can observe a dependence on the singular values of the observability grammian, $W_o(N)$. In particular, begin with the expression above, and note that the terms involving $\Sigma_0$ and $\Sigma_w$ are positive definite to achieve a lower bound in terms of $L$:
\begin{align*}
    \quad &2  \sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}} \tr \left(\left(L\T \left(S \Sigma_0 S\T + T \paren{I_N \otimes \Sigma_w} T\T + L \paren{I_{N+1} \otimes \Sigma_v} L\T \right)L\right)^{1/2}\right) \\
    \geq \;&2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}} \sigma_{\min}\left(\Sigma_v  \right)^{1/2} \tr \paren{\paren{L\T L L\T L}^{1/2}} \\
    %    &=  2 \sqrt{\frac{2}{\pi}} \frac{\varepsilon}{\sqrt{n}} \sigma_{\min}\left(\Sigma_v  \right) \tr \paren{L\T L } \\
    = \;&2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}} \sigma_{\min}\left(\Sigma_v  \right)^{1/2} \norm{L}_F^2. 
\end{align*} 
which completes the proof of inequality \eqref{eq:AR-SR gap no exp}. 
\hfill $\blacksquare$




   





Introducing additional notation to express the Kalman estimator will be helpful in subsequent sections. Let
\begin{align}\label{eq:lbnotation}
    \bar{\Sigma} &:= \bmat{\Sigma_0 & \\ & I_N \otimes \Sigma_w} \\ %\in \R^{n(N+1)\times n(N+1)} \\
    \nonumber
    H &:=  \bmat{I &  & & & \\ A  & I & & & \\ \vdots && \ddots && \\ A^N & A^{N-1} & \hdots & I}\bar{\Sigma}^{1/2} \\
    %\in \R^{n(N+1)\times n(N+1)} \\
    \nonumber
    H_k &:= E_k^\top H =  \bmat{A^k& A^{k-1} & \dots & I & 0 & \dots & 0}\bar{\Sigma}^{1/2} \\%\in \R^{n\times n(N+1)} \\ 
    \nonumber
    M &:= H^\top \paren{C^\top \otimes I} %\\
    %\\= \bmat{\Sigma_0^{1/2} C^\top & \Sigma_0^{1/2} A^\top C^\top & \ldots & \Sigma_0^{1/2} \paren{A^{N}}^\top C^\top  \\  & \Sigma_w^{1/2} C^\top  &\ldots & \Sigma_w^{1/2} \paren{A^{N-1}}^\top C^\top  \\  && \ddots & \\  &  &  & \Sigma_w^{1/2} C^\top} \\
    %\nonumber
    = \bar{\Sigma}^{1/2} \bmat{ \calO_N^\top \\  \paren{\calZ \calO_N}^\top \\ \vdots \\ \paren{\calZ^N \calO_N}^\top}, %\in \R^{n(N+1)\times p(N+1)}.
\end{align}
where $\calZ \in \R^{p(N+1) \times p(N+1)}$ is a block downshift operator, with blocks of size $m$. With this notation, the Kalman estimator given in Lemma \ref{lem:kf} may be rewritten more compactly as
\[
    \hat{L}_k = H_k M\paren{M^\top M + \Sigma_v}^{-1}.
\]



% \section{Proof of Theorem \ref{thm:lbsmalldim}}

% The adverserial risk is given by
%     \begin{align*}
%         \Ex\brac{\max_{\norm{\delta}_2 \leq \varepsilon} \norm{x_0-L (Y_T+\delta)}_2^2}.
%     \end{align*}
%     Note that $\norm{x_0 - L(Y_T + \delta)}_2^2 = \norm{x_0 - L(Y_T+\delta)}_2^2 = \norm{x_0 - LY_T - L\delta}_2^2 = \norm{x_0 - LY_T}_2^2 - 2 (x_0 - LY_T)\T L \delta + \delta\T L\T L\delta$. Consider the maximization problem
%     \begin{align*}
%         \maximize: &\quad \delta\T L\T L \delta - 2 (x_0 - LY_T)\T L \delta \\
%         \subjectto: & \quad \delta\T \delta \leq \varepsilon^2.
%     \end{align*}
%     The Lagrangian is $L(\delta, \lambda) = \delta\T L\T L \delta - 2 (x_0 - LY_T)\T L \delta - \lambda \delta\T \delta + \lambda \varepsilon^2$, so the dual function is given by
%     \begin{align*}
%         g(\lambda) &= \sup_\delta L(\delta, \lambda) \\
%         &= \sup_\delta \delta\T L\T L \delta - 2 (x_0 - LY_T)\T L \delta - \lambda \delta\T \delta + \lambda \varepsilon^2 \\
%         &= \begin{cases}
%         \lambda \varepsilon^2 + (x_0 - LY_T)\T L (\lambda I - L\T L)^{-1}
%         L\T (x_0 - LY_T) & \lambda I \succeq L\T L \\
%     \infty & \mbox{otherwise}. \end{cases}
%     \end{align*}
%     Then the dual problem is
%     \begin{align*}
%         \minimize: &\quad \lambda \varepsilon^2 +(x_0 - LY_T)\T L (\lambda I - L\T L)^{-1}
%         L\T (x_0 - LY_T) \\
%         \subjectto: &\quad \lambda I \succeq L\T L.
%     \end{align*}
%     Note that this is equivalent to
%     \begin{align*}
%         \minimize: &\quad \trace(\lambda I) \varepsilon^2/n +(x_0 - LY_T)\T L (\lambda I - L\T L)^{-1}
%         L\T (x_0 - LY_T) \\
%         \subjectto: &\quad \lambda I \succeq L\T L.
%     \end{align*}
%     If we relax the feasible set, and replace $\lambda I$ by a symmetric variable $\Lambda$, we have made the feasible set larger, so the optimal value of the above problem is greater than or equal to the optimal value of
%     \begin{align*}
%         \minimize: &\quad \trace(\Lambda) \varepsilon^2/n +(x_0 - LY_T)\T L (\Lambda - L\T L)^{-1}
%         L\T (x_0 - LY_T) \\
%         \subjectto: &\quad \Lambda \succeq L\T L.
%     \end{align*}
%     We can further relax the problem by removing the constraint. Then the following problem provides a lower bound on the optimal value:
%     \begin{align*}
%         \minimize: &\quad \trace(\Lambda) \varepsilon^2/n +(x_0 - LY_T)\T L (\Lambda - L\T L)^{-1} L\T (x_0 - LY_T).
%     \end{align*}
%     To find the minimizer for this problem, we take the derivative with respect to $\Lambda$ and set it to zero. In particular,
%     \begin{align*}
%         \varepsilon^2/n I - (\Lambda - L\T L)^{-1} L\T (x_0 - LY_T) (x_0 - LY_T)\T L (\Lambda - L\T L)^{-1} = 0.
%     \end{align*}
%     Then
%     \begin{align*}
%         \varepsilon/\sqrt n (\Lambda - L\T L)= \sqrt{L\T (x_0 - LY_T) (x_0 - LY_T) L }
%     \end{align*}
%     so
%     \begin{align*}
%         \Lambda = L\T L + \sqrt n/\varepsilon \sqrt{L\T (x_0 - LY_T) (x_0 - LY_T) L }.
%     \end{align*}
%     Substituting this back into the objective, we find that our lower bound is
%     \begin{align*}
%          \varepsilon^2/n \norm{L}_F^2 + & \varepsilon/\sqrt n \trace(\sqrt{L\T (x_0 - LY_T) (x_0 - LY_T)\T L })  \\
%              &+ \varepsilon/\sqrt n (x_0 - LY_T)\T L \paren{\sqrt{L\T (x_0 - LY_T) (x_0 - LY_T) L }}^{-1} L\T (x_0 - LY_T) \\
%          = \varepsilon^2/n\norm{L}_F^2 + &  \varepsilon/\sqrt n\trace(\sqrt{L\T (x_0 - LY_T) (x_0 - LY_T)\T L })  \\
%              &+ \varepsilon/\sqrt n \trace \paren{(x_0 - LY_T)\T L \paren{\sqrt{L\T (x_0 - LY_T) (x_0 - LY_T) L }}^{-1} L\T (x_0 - LY_T)} \\
%         =  \varepsilon^2/n \norm{L}_F^2 + &  \varepsilon/\sqrt n\trace(\sqrt{L\T (x_0 - LY_T) (x_0 - LY_T)\T L })  \\
%              &+ \varepsilon/\sqrt n \trace \paren{\paren{\sqrt{L\T (x_0 - LY_T) (x_0 - LY_T) L }}^{-1} L\T (x_0 - LY_T) (x_0 - LY_T)\T L} \\
%         =  \varepsilon^2/n \norm{L}_F^2 +& 2\varepsilon/\sqrt{n} \trace(\sqrt{L\T (x_0 - LY_T) (x_0 - LY_T)\T L }) \\
%         %{\color{red}?}
%     = \varepsilon^2/n \norm{L}_F^2 + & 2\varepsilon/\sqrt{n} \norm{L\T (x_0 - LY_T)}_2, \end{align*} as we wanted to show. \hfill $\blacksquare$  

\subsection{Theorem \ref{thm:lb}}

\textbf{Theorem \ref{thm:lb}.}\;\:\emph{Suppose that $\hat{L}_k$ is the Kalman estimator from Lemma~\ref{lem:kf}.
    %assume that %assume that $\sqrt{\norm{\Sigma_v}_2} \leq (N+1)\norm{\sigma_{\vee}}_2 \sigma_{\min}\curly{\calO_N^\top}$.
    Then we have the following bound on the gap between $\AR$ and $\SR$. 
    \begin{align*}
        \AR(\hat{L}_k) - \SR(\hat{L}_k) \geq 2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}}\sigma_{\min}\left(\Sigma_v  \right) \norm{C}_F^2\paren{ \frac{\sigma_{\min} \paren{A^k \Sigma_0 {A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w {A^{k-i}}^\top} }{(N+1) \norm{\bar{\Sigma}}_2^2 \norm{W_o(N)}_F + \norm{\Sigma_v}_2} }^2. 
    \end{align*}
    % \begin{align}\label{eq:lb0}
    %     &\AR(\hat{L}_0) - \SR(\hat{L}_0) \geq 2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}}\sigma_v \\ \nonumber &\paren{\frac{\norm{{W_o(N)}}_F^{1/2}}{(N+1) \norm{\sigma_{\vee}}_2^2 \norm{W_o(N)}_F + \norm{\Sigma_v}_2}}^2
    % \end{align}
    % \begin{align}\label{eq:lb}
    %     \AR(\hat{L}_k)& - \SR(\hat{L}_k) \geq 2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}}\sigma_v \norm{C}_F^2  \paren{ \frac{\rho^{2k} \sigma_0^2 + r_k(\rho)\; \sigma_w^2 }{(N+1) \sigma_{\vee}^2\norm{W_o(N)}_F + \sigma_v^2 } }^2. 
    % \end{align}
}

% \begin{align}\label{eq:lb}
    %     \AR(\hat{L}_k) - \SR(\hat{L}_k) \geq 2\sqrt{\frac{2 }{\pi}} \frac{\varepsilon}{\sqrt{n}}\sigma_{min}\left(\Sigma_v  \right) \norm{C}_F^2 \\ \nonumber
    %     \cdot \paren{ \frac{\sigma_{min} \paren{A^k \Sigma_0 {A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w {A^{k-i}}^\top} }{(N+1) \norm{\sigma_{\vee}}_2^2 \norm{W_o(N)}_F + \norm{\Sigma_v}_2} }^2. 
    % \end{align}

\emph{Proof:}
We begin by writing the Kalman estimator using the notation defined in \eqref{eq:lbnotation}
\[L_k = H_k M\paren{M^\top M + I_{N+1} \otimes \Sigma_v}^{-1}.\]
Suppose the rank of $M$ is $m$. Then the singular value decomposition of $M$ can be taken to be 
\[ U \bmat{S & 0 \\ 0 & 0} V^\top = M\]
where $S = \diag\paren{\bmat{s_1 & s_2 & \dots s_{m}}}$ with $s_1 \geq s_2 \geq \dots \geq s_m \geq 0$, while $U\in\R^{n(N+1)\times n(N+1)}$ and $V\in\R^{p(N+1)\times p(N+1)}$. We can now lower bound the Frobenius norm of $L_k$ as follows.
\begin{align*}
    \norm{L_k}_F^2 \geq \norm{H_k M}_F^2 \sigma_{\min}\curly{\paren{M^\top M + \Sigma_v I}^{-1}}^2.
\end{align*}
Note that $\sigma_{\min}\curly{\paren{M^\top M + \Sigma_v I}^{-1}} \geq \sigma_{\min}\curly{\paren{M^\top M + \norm{\Sigma_v}_2 I}^{-1}}$. Therefore 
\begin{equation}\label{eq:fnlb}
\begin{aligned}
    \norm{L_k}_F^2  &\geq \sigma_{\min} \curly{\paren{\bmat{S^2 & \\ & 0} + \norm{\Sigma_v}_2 I}^{-2}} \norm{H_k M}_F^2 \\
    &= \sigma_{\min} \curly{\paren{\bmat{S^2 & \\ &0} + \norm{\Sigma_v} I}^{-2}} \norm{H_kH^\top \paren{C^\top \otimes I}}_F^2 \\
    = \sigma_{\min} \curly{\paren{\bmat{S^2 & \\ &0} + \norm{\Sigma_v}_2 I}^{-2}} & \norm{\bmat{A^k \Sigma_0 C^\top & \ldots & A^k \Sigma_0 \paren{A^{N}}^\top C^\top+ \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{N-i}}^\top C^\top }}_F^2
\end{aligned}
\end{equation}

Now observe that 
\begin{equation}\label{eq:smlb}
    \sigma_{\min} \curly{\paren{\bmat{S^2 & \\ & 0} + \norm{\Sigma_v} I}^{-2}} = \frac{1}{\paren{s_1^2 + \norm{\Sigma_v}_2}^2}
\end{equation}    %We also observe that  $\sigma_{\min} \curly{\paren{S^2 + \sigma_v I}^{-2}} = \sigma_{\min} \curly{S^{-2} S^2 \paren{S^2 + \sigma_v I}^{-2}} \geq \sigma_{\min} \curly{S^{-2}} \sigma_{\min} \curly{S^2 \paren{S^2 + \sigma_v I}^{-2}} = \frac{1}{s_1^2}  \sigma_{\min} \curly{S^2 \paren{S^2 + \sigma_v I}^{-2}}$. We may see that $s_{p(N+1)} \geq (N+1)\norm{\bar{\Sigma}}_2 \sigma_{\min}\curly{\calO_N^\top} \sqrt{\sigma_v}$. Then $\sigma_{\min} \curly{S^2 \paren{S^2 + \sigma_v I}^{-2}} = \paren{\frac{s_{p(N+1)}}{s_{p(N+1)}^2+\sigma_v}}^2$. We also have that 
Also note that $s_1 = \norm{M}_2 \leq \norm{\bar \Sigma^{1/2}}_2 \norm{ \bar \Sigma^{-1/2} M}_F$. We have that 
\begin{align*}
    \norm{\bar \Sigma^{-1/2} M}_F^2 = \norm{ \bmat{ \calO_N^\top \\  \paren{\calZ \calO_N}^\top \\ \vdots \\ \paren{\calZ^N \calO_N}^\top}}_F^2 \leq \sum_{i=0}^N \norm{\calZ^i \calO_N}_F^2 \leq (N+1) \norm{\calO_N}_F^2.  
\end{align*}
Then 
\begin{align}\label{eq:s1lb}
s_1 \leq \norm{\bar \Sigma^{1/2}}_2 \sqrt{N+1} \norm{\calO_N}_F
\end{align}
% \begin{equation}\label{eq:s1lb}
%     s_1 = \norm{M}_2 \leq \norm{M}_F= \norm{\bar{\Sigma} \paren{I_{N+1} \otimes \calO_N^\top}\bmat{I \\ \calZ^\top \\ \vdots \\ \paren{\calZ^N}^\top}}_2 \leq \norm{\bar{\Sigma}}_2 \norm{\calO_N}_2 \norm{\bmat{I \\ \calZ^\top \\ \vdots \\ \paren{\calZ^N}^\top}}_2=(N+1)\norm{\bar{\Sigma}}_2 \norm{\calO_N}_2.  \\
% %    &= \sigma_i \curly{\bar{\Sigma} \paren{I_{N+1} \otimes \calO_N^\top} \bmat{I \\ \calZ^\top \\ \vdots \\ \paren{\calZ^N}^\top}} \\
% %    &\leq \sigma_{\max} \curly{\bar{\Sigma}} \sigma_i \curly{I_{N+1} \otimes \calO_N^\top} \sigma_{\max} \curly{\bmat{I \\ \calZ^\top \\ \vdots \\ \paren{\calZ^N}^\top}}\\
% %    &\leq \sigma_{\max} \curly{\bar{\Sigma}} \sigma_i \curly{I_{N+1} \otimes \calO_N^\top} \paren{N+1}
%\end{equation} 

%Also note that $\sigma_{p(N+1)}\curly{I_{N+1} \otimes \calO_N^\top}=\sigma_{\min}\curly{\calO_N^\top}$ and $\sigma_1\curly{I_{N+1} \otimes \calO_N^\top}=\sigma_1 \curly{ \calO_N^\top}$

% Then to get the bound in \eqref{eq:lb0}, we combine the inequalities in \eqref{eq:fnlb}, \eqref{eq:smlb}, and \eqref{eq:s1lb} and note that for $k=0$, 
% \[\norm{\bmat{A^k \Sigma_0 C^\top & \ldots & A^k \Sigma_0 \paren{A^{N}}^\top C^\top+ \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{T-i}}^\top C^\top }}_F^2 = \norm{\calO_N}_F^2.\]
When $k\geq 0$, we have 
\begin{align*}
    \norm{\bmat{A^k \Sigma_0 C^\top & \ldots & A^k \Sigma_0 \paren{A^{N}}^\top C^\top+ \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{T-i}}^\top C^\top }}_F^2 \\
    \geq \norm{\paren{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top}C^\top}_F^2 \\
    \geq\sigma_{\min} \paren{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top}^2 \norm{C}_F^2. 
\end{align*}
In conjunction with \eqref{eq:fnlb}, \eqref{eq:smlb}, and \eqref{eq:s1lb}, this leads to \eqref{eq:lb}. \hfill $\blacksquare$




%\section{Proof of Lemma \ref{lem:minimum nonzero singular value}}
%\textit{Proof:}
%        Let the singular value decomposition of $A$ be $U \bmat{\Sigma & 0} V\T$. Then 
%        \begin{align*}
%            \norm{A \paren{A\T A +cI}^{-1}}_2  &= \norm{U\bmat{\Sigma & 0} V\T \paren{V \bmat{ \Sigma^2 & 0 \\ 0 & 0} V\T +cI}^{-1}}_2 \\
%            &=  \norm {\bmat{\Sigma & 0} V\T \paren{V \paren{ \bmat{ \Sigma^2 & 0 \\ 0 & 0} + cI} V\T}^{-1}}_2 \\
%            &= \norm{\bmat{\Sigma & 0} \paren{ \bmat{\Sigma^2 & 0 \\ 0 & 0} + cI}^{-1}}_2 \\
%            &= \max_{i\in[n]} \frac{\sigma_i}{\sigma_i^2 + c} \\
%            &= \max_{i\in[m]} \frac{\sigma_i}{\sigma_i^2 + c} \\
%            &\leq \max_{i\in[m]} \frac{\sigma_i}{\sigma_i^2} \\
%            &= \max_{i\in[m]} \frac{1}{\sigma_i} \\
%            &= \frac{1}{\sigma_m}
%        \end{align*}
%    \hfill $\blacksquare$    

\subsection{Proof of Lemma \ref{thm:general upper bound}}
\textbf{Lemma \ref{thm:general upper bound}.}\;\:\emph{For any $L \in \R^{n \times p(N+1)}$, the following bound holds
    \begin{align*}
        \AR(L) -  \SR(L) \leq 2\varepsilon  \norm{L}_2 \norm{\Sigma^{1/2}}_F + \varepsilon^2 \norm{L}_2^2
    \end{align*}
    where $\Sigma^{1/2}$ is the symmetric square root of the covariance of $x_k - LY_N$. }

\textit{Proof:} By Theorem \ref{thm:generalUBLB}, 
\begin{align*}
    \AR(L) - \SR(L) \leq 2 \varepsilon  \Ex \brac{\norm{L(x_t - LY_N)}_2} + \varepsilon^2 \lambda_{\max}\paren{L^\top L} \leq 2 \varepsilon \norm{L}_2  \Ex \brac{\norm{x_t - LY_N}_2} + \varepsilon^2 \norm{L}_2^2
\end{align*}
    % As with the proof of Theorem, we may apply a similar idea to the proof of theorem one to show that
    % \begin{align*}
    %     \AR(L) - \SR(L) \geq 2\varepsilon \norm{L}_2 \Ex \brac{\norm{x_t - LY_N}_2} + \varepsilon^2 \norm{L}_2^2.
    % \end{align*}
    % The proof is brief, and will be repeated here for this specific setting.
%     For any $x_k \in \R^n$, $L \in \R^{n\times p(N+1)}$, and $Y_T \in \R^{p(N+1)}$, note that 
% \begin{align*}
%     \max_{\norm{\delta}_2 \leq \varepsilon} \norm{x_t - L(Y_N + \delta)}_2^2 &\leq \max_{\norm{\delta}_2 \leq \varepsilon} (\norm{x_t - LY_N}_2 + \norm{L\delta}_2)^2 \\
%     &= (\norm{x_t - LY_N}_2 + \varepsilon \norm{L}_2)^2 \\
%     &= \norm{x_t - LY_N}_2^2 + 2\varepsilon \norm{L}_2 \norm{x_t - LY_N}_2 + \varepsilon^2 \norm{L}_2^2
% \end{align*}
% Then, by taking expectations of each side, 
% \begin{align*}
%     \AR(L) &\leq \SR(L) + 2\varepsilon \norm{L}_2 \Ex \brac{\norm{x_t - LY_N}_2} + \varepsilon^2 \norm{L}_2^2.
% \end{align*}
We can upper bound $\Ex\brac{\norm{x_t - LY_N}_2}$ by bounding the expectation of the euclidean norm of a normal random variable. In particular, let $w$ be a $n$ dimensional standard normal random variable, so that 
$\Ex\brac{\norm{x_t - LY_N}_2} =\Ex\brac{\norm{ \Sigma^{1/2} w}_2}$, where $\Sigma$ is defined as the covariance of $x_k - LY_N$, and $\Sigma^{1/2}$ is its symmetric square root. 
Let $US^{1/2} U\T := \Sigma^{1/2}$ be the eigenvalue decomposition of $\Sigma^{1/2}$ so that
\begin{align*}
    \norm{\Sigma^{1/2} z}_2 &= \norm{U S^{1/2} U\T w}_2
\end{align*}
Now define $z = U\T w$. We have that $z \sim N(0, I)$. Then the above quantity equals $\sqrt{\norm{U S^{1/2} w}_2^2}$. Jensen's inequality tells us that
\begin{align*}
    \Ex \brac{\sqrt{\norm{US^{1/2}  w}_2^2}} \leq \sqrt{\Ex \brac{\norm{U S^{1/2} w}_2^2}} = \sqrt{\Ex \brac{w\T S^{1/2} w}} = \norm{S^{1/2}}_F = \norm{\Sigma^{1/2}}_F,
\end{align*}
from which the theorem follows. \hfill $\blacksquare$

\subsection{Theorem \ref{thm:ub}}

\textbf{Theorem~\ref{thm:ub}.}\;\:\emph{Suppose that $\hatL_k$ is the Kalman state estimator given by Lemma \ref{lem:kf}. Then the gap between $\AR(\hatL_k)$ and $\SR(\hatL_k)$ is upper bounded by
\begin{align*}
    \AR(\hatL_k) - \SR(\hatL_k) &\geq \epsilon  \paren{\frac{{\norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2}}{\sigma_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}}} \\
    & \quad \times \left(2 \sqrt{n}
    %\paren{ \norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2
        \paren{\norm{\bar{\Sigma}}_2 + \paren{\frac{\sqrt{\norm{\Sigma_v}_2}}{\sigma_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}}}^2 }^{1/2}  
    %\paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}} {\paren{\norm{\bar{\Sigma}}_2 \norm{H_k}_2^2 + \norm{\Sigma_v}_2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2 }^{1/2}}  \\
    %+ \epsilon^2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2.
    +\epsilon \paren{ \frac{1}{\sigma_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}}}\right).
\end{align*}
Furthermore, if $\lambda_{\min}(W_o(N))^{1/2} \geq \sigma_v/\sigma_{\min}\paren{\bar\Sigma}$, and defining $\kappa = \frac{\lambda_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}}{\lambda_{\min}(W_o(N)) \sigma_{\min} \paren{\bar\Sigma}^2 + \sigma_v}$, we get the bound
\begin{align*}
    \AR(\hat{L}_k) - \SR(\hat{L}_k) &\leq \varepsilon \paren{\kappa \sqrt{\norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2}  } \\
    &\quad \times \paren{2\sqrt{n}\paren{\sigma_{\max}(\bar{\Sigma})^2 + \sigma_{\min}(\Sigma_v) \kappa^2}^{1/2} + \varepsilon \kappa }
\end{align*}
}

\textit{Proof:} By Lemma \ref{thm:general upper bound}, upper bounding the gap between $\AR(L_k)$ and $\SR(L_k)$ reduces to upper bounding $\norm{\Sigma^{1/2}}_F$ and $\norm{L_k}_2$. First consider $\norm{\Sigma^{1/2}}_F$. Equivalence of norms tells us that
\begin{align}\label{eq:norm equivalence}
    \norm{\Sigma^{1/2}}_F \leq \sqrt{n} \norm{\Sigma^{1/2}}_2 = \sqrt{n}\norm{\Sigma}_2^{1/2}.
\end{align}
Recalling the notation defined in \eqref{eq:lbnotation}, the Kalman filter may be expressed as $L_k = H_k M \paren{M^\top M + \Sigma_v}^{-1}$. We may also  express $\Sigma$ in terms of this notation: $\Sigma = \paren{H_k - L_k M} \bar{\Sigma} \paren{H_k - L_k M}^\top + L_k \paren{I \otimes \Sigma_v} L_k\T$. To upper bound the spectral radius of this, we can leverage triangle inequality and submultiplicativity
\begin{align*}
    \norm{\Sigma}_2
    &\leq  \norm{\paren{H_k - L_k M} \bar{\Sigma} \paren{H_k - L_kM}^\top}_2 + \norm{L_k \paren{I \otimes \Sigma_v} L_k\T}_2 \\
    &\leq  \norm{\bar{\Sigma}}_2 \norm{H_k - L_kM}_2^2 + \norm{\Sigma_v}_2 \norm{L_k}_2^2.
\end{align*}
Note that $H_k - L_kM = H_k - H_k M\paren{M^\top M + \Sigma_v}^{-1} M = H_k \paren{I - M \paren{M^\top M + \Sigma_v}^{-1} M^\top}$. Then by submultiplicativity, 
\begin{align*}
    \norm{H_k \paren{I - M \paren{M^\top M + \Sigma_v}^{-1} M^\top}}_2 \leq \norm{H_k}_2 \norm{I - M \paren{M^\top M + \Sigma_v}^{-1} M^\top}_2 \leq \norm{H_k}_2.
\end{align*}
We can further upper bound $\norm{H_k}_2$ in terms of system properties. In particular, we have
\begin{align*}
    \norm{H_k}_2 =  \sqrt{\norm{H_k}_2^2} = \sqrt{\norm{H_k H_k^\top}_2} = \sqrt{\norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2}.
\end{align*}

Thus 
\begin{align}\label{eq:sigmabound}
    \norm{\Sigma}_2 \leq \norm{\bar{\Sigma}}_2 \norm{H_k}_2^2 + \norm{\Sigma_v}_2 \norm{L}_2^2\leq\norm{\bar{\Sigma}}_2 \norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2 + \norm{\Sigma_v}_2 \norm{L}_2^2.
\end{align}

Next we obtain a bound on $\norm{L_k}_2$. As in the proof of Theorem \ref{thm:lb}, we will assign $m: = \rank(M)$ and take the singular value decomposition of $M$ to be $U \bmat{S & 0 \\ 0 & 0} V^\top$ where $S = \diag\paren{\bmat{s_1 & \dots & s_m}}$ with $s_1 \geq s_2 \geq \dots \geq s_m \geq 0$, while $U \in\R^{n(N+1)\times n(N+1)}$ and $V \in \R^{p(N+1)\times p(N+1)}$. 
\begin{align*}
    \norm{L_k}_2 &= \norm{H_k M\paren{M^\top M + \Sigma_v}^{-1}}_2 \leq \norm{H_k}_2 \norm{M\paren{M^\top M + \sigma_{\min}\paren{\Sigma_v}}^{-1}}_2 \\
    &= \norm{H_k}_2\norm{U \bmat{S &  \\  & 0} V^\top \paren{V \paren{\bmat{S^2 & \\ & 0} + \sigma_{\min}(\Sigma_v)} V^\top}^{-1}} \\
    &= \norm{H_k}_2\norm{U \bmat{S & \\ & 0} \paren{\bmat{ S^2 & \\ & 0} + \sigma_{\min}(\Sigma_v)}^{-1}V^\top}_2 \\
        &= \norm{H_k}_2\norm{S (S^2 + \sigma_{\min}(\Sigma_v))^{-1}}_2 \\
        %\leq \norm{H_k}_2 \frac{s_m}{s_m^2 + \sigma_{\min}(Sigma_v} 
        &\leq \norm{H_k}_2\max_{1\leq k \leq m}\frac{s_k}{s_k^2 + \sigma_{\min}(\Sigma_v)}.
\end{align*}
%where the final inequality follows from the assumption that $\sigma_{\min}(\Sigma_v)$

A simple bound on the last maximization would be
\begin{align}\label{eq: s_k upper bound}
    \max_{1\leq k \leq m}\frac{s_k}{s_k^2 + \sigma_{\min}(\Sigma_v)} &\leq \max_{1\leq k \leq m}\frac{s_k}{s_k^2} \leq \frac{1}{s_m}
\end{align}
Note that $s_m \geq \lambda_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}$, %\Bruce{may need to include more justification - smallest nonzero singular value of the shift matrices will be 1}, 
so $\frac{1}{s_m} \leq \frac{1}{\lambda_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}}$. Then 
\begin{align}\label{eq:Lbound}
    \norm{L_k}_2 \leq  \frac{\norm{H_k}_2}{\lambda_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}} \leq \frac{\sqrt{\norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2}}{\lambda_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}}.
\end{align}

Then the first half of the theorem follows by combining the result of Lemma \ref{thm:general upper bound} with \eqref{eq:norm equivalence},  \eqref{eq:sigmabound} and \eqref{eq:Lbound}.

However, if $\lambda_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma} \geq \sigma_v$, then the maximum \eqref{eq: s_k upper bound} is attained at
\begin{align*}
    \max_{1\leq k \leq m}\frac{s_k}{s_k^2 + \sigma_{\min}(\Sigma_v)} &= \frac{s_m}{s_m^2 + \sigma_v} \\
    &\leq \frac{\lambda_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}}{\lambda_{\min}(W_o(N)) \sigma_{\min} \paren{\bar\Sigma}^2 + \sigma_v}.
\end{align*}
Therefore, when $\lambda_{\min}(W_o(N))^{1/2} \geq \sigma_v/\sigma_{\min} \paren{\bar\Sigma}$, we have the more precise bound
\begin{align*}
    \norm{L_k}_2 &\leq  \norm{H_k}_2 \frac{\lambda_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}}{\lambda_{\min}(W_o(N)) \sigma_{\min} \paren{\bar\Sigma}^2 + \sigma_v} \\
    &\leq \sqrt{\norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2} 
    \frac{\lambda_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}}{\lambda_{\min}(W_o(N)) \sigma_{\min} \paren{\bar\Sigma}^2 + \sigma_v},
\end{align*}
which leads to the second half of the theorem.

% \begin{align*}
%     \AR(\hat{L}_k) - \SR(\hat{L}_k) &\leq \varepsilon \paren{\sqrt{\norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2} 
%     \frac{\lambda_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}}{\lambda_{\min}(W_o(N)) \sigma_{\min} \paren{\bar\Sigma}^2 + \sigma_v} } \\
%     & \quad \times \Bigg[2 \sqrt{n}
%     %\paren{ \norm{A^k \Sigma_0 \paren{A^k}^\top + \sum_{i=1}^k A^{k-i} \Sigma_w \paren{A^{k-i}}^\top_2}_2
%         \paren{\sigma_{\vee}^2 + \sigma_v^2 \paren{
%     \frac{\lambda_{\min}(W_o(N))^{1/2} \sigma_{\min} \paren{\bar\Sigma}}{\lambda_{\min}(W_o(N)) \sigma_{\min} \paren{\bar\Sigma}^2 + \sigma_v}}^2 }^{1/2} \\ & \quad \quad   \quad
%     %\paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}} {\paren{\norm{\bar{\Sigma}}_2 \norm{H_k}_2^2 + \norm{\Sigma_v}_2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2 }^{1/2}}  \\
%     %+ \varepsilon^2 \paren{\kappa^{1/2} \sum_{i=0}^k \frac{\norm{  A^{k-i} }_2}{\sqrt{\lambda_{min}\paren{ \calO\T \calO}}}}^2.
%     +\varepsilon \paren{\frac{\sigma_{\wedge}^2 \lambda_{\min}(W_o(N))^{1/2}}{\sigma_{\wedge}^4 \lambda_{\min}(W_o(N)) + \sigma_v^2 }}\Bigg].
% \end{align*}


\hfill $\blacksquare$

% \section{Additional Experiments} \label{ss:additional experiments}
% \begin{figure*}[t]
%     \centering
%     \makebox[\textwidth][c]{
%         \subfloat[][Linear estimator]{
%         \includegraphics[width = 0.5 \textwidth]{arSrTradeoff_nonorthogonal/arSrTradeoff_smoothing_linearFilter.pdf}
%         }
%         \subfloat[][Neural Network]{
%         \includegraphics[width = 0.5 \textwidth]{arSrTradeoff_nonorthogonal/arSrTradeoff_smoothing_NNFilter.pdf}
%         }
%     }

%     \caption{Pareto boundaries of $(\SR, \AR)$ for initial state estimation for a variety of measurement matrices $C$ by both a linear state estimator and a neural network (Lower bound on $\AR$ plotted for NN). As the first entry of $C$ decreases, the tradeoff curve becomes more severe. The trade-offs are not alleviated by a nonlinear estimator.}
    
%     %Plots are not finalized. The first plot has the tradeoff curves with a linear filter, while the second plot has the tradeoff curves with  a NN. We can see that they look quite similar. Note that the NN plot uses an under approximation for AR, and thus can fall below the linear tradeoff curves WITHOUT disproving Nik's conjecture that linear is optimal. The third has tradeoff curves for filtering rather than initial state estimation. Working out optimization hyperparameters for the third plot }
%     \label{fig:NN tradeoffs}
% \end{figure*}
% In Figure~\ref{fig:NN tradeoffs}, we demonstrate that the fundamental tradeoffs are not overcome by using nonlinear state estimators. In particular, we consider the system and adversary defined by  $(A, \Sigma_0, \Sigma_w, \Sigma_v, N) = \paren{\bmat{1 & 1 \\ 0 & 1}, I, 0.1 I, 0.1, 5}$, and $C$ as in the legend of the plots. We solve for the Pareto boundary for the linear estimator as in Figure~\ref{fig:state estimation tradeoffs}. We also parametrize a two layer network with 10 neurons per layer, and perform a SGD procedure similar to that used in the linear case, with the exception being that we do not solve the exact adversary corresponding to each data point, but rather apply 100 steps of gradient ascent to find the adversarial perturbation. Once the neural network is trained, this same approach to find the adversary is used again to estimate the adversarial risk of the resultant estimator. Thus the tradeoff curves shown for the neural network use an under approximation to the true values of $\AR$ achieved. As such, it appears that the neural network is getting roughly the same tradeoff curves as the linear estimator. This may suggest that linear estimators are optimal, even for minimizing the adversarial risk. 

\begin{thebibliography}{31}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
  \providecommand{\doi}[1]{doi: #1}\else
  \providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi

\bibitem[Al~Makdah et~al.(2020)Al~Makdah, Katewa, and
  Pasqualetti]{al2020accuracy}
A.~A. Al~Makdah, V.~Katewa, and F.~Pasqualetti.
\newblock Accuracy prevents robustness in perception-based control.
\newblock In \emph{2020 American Control Conference (ACC)}, pages 3940--3946.
  IEEE, 2020.

\bibitem[Bottou et~al.(2018)Bottou, Curtis, and Nocedal]{bottou2018first}
L.~Bottou, F.~E. Curtis, and J.~Nocedal.
\newblock Optimization methods for large-scale machine learning.
\newblock \emph{SIAM Review}, 60\penalty0 (2):\penalty0 223--311, 2018.
\newblock \doi{10.1137/16M1080173}.
\newblock URL \url{https://doi.org/10.1137/16M1080173}.

\bibitem[Boyd and Vandenberghe(2004)]{boyd2004convex}
S.~P. Boyd and L.~Vandenberghe.
\newblock \emph{Convex optimization}.
\newblock Cambridge university press, 2004.

\bibitem[Candes and Tao(2005)]{candes2005decoding}
E.~J. Candes and T.~Tao.
\newblock Decoding by linear programming.
\newblock \emph{IEEE transactions on information theory}, 51\penalty0
  (12):\penalty0 4203--4215, 2005.

\bibitem[Carlini and Wagner(2016)]{carlini2016defensive}
N.~Carlini and D.~Wagner.
\newblock Defensive distillation is not robust to adversarial examples.
\newblock \emph{arXiv preprint arXiv:1607.04311}, 2016.

\bibitem[Carlini and Wagner(2017)]{carlini2017towards}
N.~Carlini and D.~Wagner.
\newblock Towards evaluating the robustness of neural networks.
\newblock In \emph{2017 IEEE symposium on security and privacy}, pages 39--57.
  IEEE, 2017.

\bibitem[Carrasco et~al.(2007)Carrasco, Florens, and
  Renault]{carrasco2007linear}
M.~Carrasco, J.-P. Florens, and E.~Renault.
\newblock Linear inverse problems in structural econometrics estimation based
  on spectral decomposition and regularization.
\newblock \emph{Handbook of econometrics}, 6:\penalty0 5633--5751, 2007.

\bibitem[Chen et~al.(2020)Chen, Min, Zhang, and Karbasi]{chen2020more}
L.~Chen, Y.~Min, M.~Zhang, and A.~Karbasi.
\newblock More data can expand the generalization gap between adversarially
  robust and standard models.
\newblock In \emph{International Conference on Machine Learning}, pages
  1670--1680. PMLR, 2020.

\bibitem[Deka et~al.(2020)Deka, Stipanovi{\'c}, and
  Tomlin]{deka2020dynamically}
S.~A. Deka, D.~M. Stipanovi{\'c}, and C.~J. Tomlin.
\newblock Dynamically computing adversarial perturbations for recurrent neural
  networks.
\newblock \emph{arXiv preprint arXiv:2009.02874}, 2020.

\bibitem[Dobriban et~al.(2020)Dobriban, Hassani, Hong, and
  Robey]{dobriban2020provable}
E.~Dobriban, H.~Hassani, D.~Hong, and A.~Robey.
\newblock Provable tradeoffs in adversarially robust classification.
\newblock \emph{arXiv preprint arXiv:2006.05161}, 2020.

\bibitem[Doyle and Stein(1979)]{doyle1979robustness}
J.~Doyle and G.~Stein.
\newblock Robustness with observers.
\newblock \emph{IEEE Transactions on Automatic Control}, 24\penalty0
  (4):\penalty0 607--611, 1979.
\newblock \doi{10.1109/TAC.1979.1102095}.

\bibitem[Engl et~al.(2009)Engl, Flamm, K{\"u}gler, Lu, M{\"u}ller, and
  Schuster]{engl2009inverse}
H.~W. Engl, C.~Flamm, P.~K{\"u}gler, J.~Lu, S.~M{\"u}ller, and P.~Schuster.
\newblock Inverse problems in systems biology.
\newblock \emph{Inverse Problems}, 25\penalty0 (12):\penalty0 123014, 2009.

\bibitem[Goodfellow et~al.(2014)Goodfellow, Shlens, and
  Szegedy]{goodfellow2014explaining}
I.~J. Goodfellow, J.~Shlens, and C.~Szegedy.
\newblock Explaining and harnessing adversarial examples.
\newblock \emph{arXiv preprint arXiv:1412.6572}, 2014.

\bibitem[Hassibi et~al.(1999)Hassibi, Kailath, and Sayed]{hassibi1999}
B.~Hassibi, T.~Kailath, and A.~H. Sayed.
\newblock \emph{Indefinite-quadratic estimation and control: a unified approach
  to {H$2$} and {H$\infty$} theories}.
\newblock SIAM studies in applied and numerical mathematics, 1999.

\bibitem[Huang et~al.(2017)Huang, Papernot, Goodfellow, Duan, and
  Abbeel]{huang2017adversarial}
S.~Huang, N.~Papernot, I.~Goodfellow, Y.~Duan, and P.~Abbeel.
\newblock Adversarial attacks on neural network policies.
\newblock \emph{arXiv preprint arXiv:1702.02284}, 2017.

\bibitem[Javanmard et~al.(2020)Javanmard, Soltanolkotabi, and
  Hassani]{javanmard2020precise}
A.~Javanmard, M.~Soltanolkotabi, and H.~Hassani.
\newblock Precise tradeoffs in adversarial training for linear regression.
\newblock In \emph{Conference on Learning Theory}, pages 2034--2078. PMLR,
  2020.

\bibitem[Khargonekar et~al.(1996)Khargonekar, Rotea, and
  Baeyens]{khargonekar1996}
P.~P. Khargonekar, M.~A. Rotea, and E.~Baeyens.
\newblock Mixed {H2/H$\infty$} filtering.
\newblock \emph{International Journal of Robust and Nonlinear Control},
  6\penalty0 (4):\penalty0 313--330, 1996.

\bibitem[Lutter et~al.(2021)Lutter, Mannor, Peters, Fox, and
  Garg]{lutter2021robust}
M.~Lutter, S.~Mannor, J.~Peters, D.~Fox, and A.~Garg.
\newblock Robust value iteration for continuous control tasks.
\newblock \emph{arXiv preprint arXiv:2105.12189}, 2021.

\bibitem[Madry et~al.(2017)Madry, Makelov, Schmidt, Tsipras, and
  Vladu]{madry2017towards}
A.~Madry, A.~Makelov, L.~Schmidt, D.~Tsipras, and A.~Vladu.
\newblock Towards deep learning models resistant to adversarial attacks.
\newblock \emph{arXiv preprint arXiv:1706.06083}, 2017.

\bibitem[Mandlekar et~al.(2017)Mandlekar, Zhu, Garg, Fei-Fei, and
  Savarese]{mandlekar2017adversarially}
A.~Mandlekar, Y.~Zhu, A.~Garg, L.~Fei-Fei, and S.~Savarese.
\newblock Adversarially robust policy learning: Active construction of
  physically-plausible perturbations.
\newblock In \emph{2017 IEEE/RSJ International Conference on Intelligent Robots
  and Systems (IROS)}, pages 3932--3939. IEEE, 2017.

\bibitem[Nakkiran(2019)]{nakkiran2019adversarial}
P.~Nakkiran.
\newblock Adversarial robustness may be at odds with simplicity.
\newblock \emph{arXiv preprint arXiv:1901.00532}, 2019.

\bibitem[Petersen et~al.(2008)Petersen, Pedersen, et~al.]{petersen2008matrix}
K.~B. Petersen, M.~S. Pedersen, et~al.
\newblock The matrix cookbook.
\newblock \emph{Technical University of Denmark}, 7\penalty0 (15):\penalty0
  510, 2008.

\bibitem[Pinto et~al.(2017)Pinto, Davidson, Sukthankar, and
  Gupta]{pinto2017robust}
L.~Pinto, J.~Davidson, R.~Sukthankar, and A.~Gupta.
\newblock Robust adversarial reinforcement learning.
\newblock In \emph{International Conference on Machine Learning}, pages
  2817--2826. PMLR, 2017.

\bibitem[Raghunathan et~al.(2019)Raghunathan, Xie, Yang, Duchi, and
  Liang]{raghunathan2019adversarial}
A.~Raghunathan, S.~M. Xie, F.~Yang, J.~C. Duchi, and P.~Liang.
\newblock Adversarial training can hurt generalization.
\newblock \emph{arXiv preprint arXiv:1906.06032}, 2019.

\bibitem[Recht et~al.(2010)Recht, Fazel, and Parrilo]{recht2010guaranteed}
B.~Recht, M.~Fazel, and P.~A. Parrilo.
\newblock Guaranteed minimum-rank solutions of linear matrix equations via
  nuclear norm minimization.
\newblock \emph{SIAM review}, 52\penalty0 (3):\penalty0 471--501, 2010.

\bibitem[Ribes and Schmitt(2008)]{ribes2008linear}
A.~Ribes and F.~Schmitt.
\newblock Linear inverse problems in imaging.
\newblock \emph{IEEE Signal Processing Magazine}, 25\penalty0 (4):\penalty0
  84--99, 2008.

\bibitem[Szegedy et~al.(2013)Szegedy, Zaremba, Sutskever, Bruna, Erhan,
  Goodfellow, and Fergus]{szegedy2013intriguing}
C.~Szegedy, W.~Zaremba, I.~Sutskever, J.~Bruna, D.~Erhan, I.~Goodfellow, and
  R.~Fergus.
\newblock Intriguing properties of neural networks.
\newblock \emph{arXiv preprint arXiv:1312.6199}, 2013.

\bibitem[Tsipras et~al.(2018)Tsipras, Santurkar, Engstrom, Turner, and
  Madry]{tsipras2018robustness}
D.~Tsipras, S.~Santurkar, L.~Engstrom, A.~Turner, and A.~Madry.
\newblock Robustness may be at odds with accuracy.
\newblock \emph{arXiv preprint arXiv:1805.12152}, 2018.

\bibitem[Xie et~al.(2020)Xie, Tan, Gong, Yuille, and Le]{xie2020smooth}
C.~Xie, M.~Tan, B.~Gong, A.~Yuille, and Q.~V. Le.
\newblock Smooth adversarial training.
\newblock \emph{arXiv preprint arXiv:2006.14536}, 2020.

\bibitem[Zhang et~al.(2019)Zhang, Yu, Jiao, Xing, Ghaoui, and
  Jordan]{zhang2019theoretically}
H.~Zhang, Y.~Yu, J.~Jiao, E.~P. Xing, L.~E. Ghaoui, and M.~I. Jordan.
\newblock Theoretically principled trade-off between robustness and accuracy.
\newblock \emph{arXiv preprint arXiv:1901.08573}, 2019.

\bibitem[Zhou and Doyle(1998)]{zhou1998robust}
K.~Zhou and J.~C. Doyle.
\newblock \emph{Essentials of Robust Control}.
\newblock Prentice-Hall, 1998.

\end{thebibliography}