Safety Evaluation of Motion Plans Using Trajectory Predictors as Forward Reachable Set Estimators
Extracted source text · arXiv 2507.22389
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%2345678901234567890123456789012345678901234567890123456789012345678901234567890
% 1 2 3 4 5 6 7 8
\documentclass[letterpaper, 10 pt, conference]{ieeeconf} % Comment this line out if you need a4paper
%\documentclass[a4paper, 10pt, conference]{ieeeconf} % Use this line for a4 paper
\IEEEoverridecommandlockouts % This command is only needed if
% you want to use the \thanks command
\overrideIEEEmargins % Needed to meet printer requirements.
%In case you encounter the following error:
%Error 1010 The PDF file may be corrupt (unable to open PDF file) OR
%Error 1000 An error occurred while parsing a contents stream. Unable to analyze the PDF file.
%This is a known problem with pdfLaTeX conversion filter. The file cannot be opened with acrobat reader
%Please use one of the alternatives below to circumvent this error by uncommenting one or the other
%\pdfobjcompresslevel=0
%\pdfminorversion=4
% See the \addtolength command later in the file to balance the column lengths
% on the last page of the document
% The following packages can be found on http:\\www.ctan.org
%\usepackage{graphics} % for pdf, bitmapped graphics files
%\usepackage{epsfig} % for postscript graphics files
%\usepackage{mathptmx} % assumes new font selection scheme installed
%\usepackage{times} % assumes new font selection scheme installed
%\usepackage{amsmath} % assumes amsmath package installed
%\usepackage{amssymb} % assumes amsmath package installed
\input{imports}
\title{\LARGE \bf
Safety Evaluation of Motion Plans Using Trajectory Predictors as Forward Reachable Set Estimators
}
\author{Kaustav Chakraborty$^{*1,2,3}$, Zeyuan Feng$^{*2}$, Sushant Veer$^{* 3}$, Apoorva Sharma$^{3}$, Wenhao Ding$^{3}$\\ Sever Topan$^{3}$, Boris Ivanovic$^{3}$, Marco Pavone$^{2, 3}$, Somil Bansal$^{2}$ % <-this % stops a space
\thanks{*equal contribution, arranged in alphabetical order of last names. Corresponding author: kaustavc@usc.edu.}% <-this % stops a space
\thanks{$^{1}$Authors are with the Department of Electrical Engineering, University of Southern California, USA.}
%
\thanks{$^{2}$Authors are with the Department of Aeronautics and Astronautics, Stanford University, USA}
%
\thanks{$^{3}$Authors are with NVIDIA Research
}
}
\begin{document}
\maketitle
\thispagestyle{empty}
\pagestyle{empty}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{abstract}
\input{abstract}
\end{abstract}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{INTRODUCTION}
\input{introduction}
\section{RELATED WORKS}
\input{related}
\section{Equivalence of Deterministic and Stochastic Notions of FRS}
\input{problem}
\section{FORCE-OPT}
\input{forceopt}
\section{Experimental Results}
\input{results}
\section{Conclusion and Future Work}
\input{conclusion}
% \addtolength{\textheight}{-10cm} % This command serves to balance the column lengths
% on the last page of the document manually. It shortens
% the textheight of the last page by a suitable amount.
% This command does not take effect until the next page
% so it should come on the page before the last. Make
% sure that you do not shorten the textheight too much.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\bibliographystyle{IEEEtran}
\bibliography{references}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\newpage
\section*{APPENDIX}
\section*{Proofs}
\begin{lemma}\label{lem:subset}
Let $A$ and $B$ be measurable sets. If every measurable subset of $A$ with non-zero measure is a subset of $B$, and if every measurable subset of $B$ with non-zero measure is a subset of $A$, then $A=B$ almost everywhere.
\end{lemma}
\begin{proof}
We will prove this by contradiction. Let's assume that the hypothesis of the lemma holds, but $\vol(A\setminus B) > 0$. Since $A\setminus B=A \cap B^{\rm c}$, it is a measurable set and $A\setminus B \subseteq A$, i.e., it is a measurable subset of $A$. By the hypothesis of the lemma, we have that $A\setminus B \subseteq B$ (as every measurable subset of $A$ with non-zero measure is a subset of $B$). However, by definition, $A\setminus B$ is disjoint from $B$, therefore $A\setminus B=\emptyset \implies \vol(A\setminus B)=0$, leading to a contradiction to our initial claim that $\vol(A\setminus B)>0$. Therefore our initial claim must be wrong, and $\vol(A\setminus B)=0$. Using the same argument $\vol(B\setminus A)=0$, implying that $A=B$ almost everywhere.
\end{proof}
\begin{proof}[\textbf{Proof of Theorem~\ref{thm:deter-prob-FRS}}]
We establish this proof by showing that any measurable subset of $F_t$ with non-zero measure is also a subset of $\omega_t^*$, and every measurable subset of $\omega_t^*$ with non-zero measure is a subset of $F_t$ and then invoking Lemma~\ref{lem:subset}.
Let $\omega\subseteq\omega_t^*$ be a measurable set such that $\vol(\omega)>0$. It follows that $\mu_t(\omega)>0$ for if it doesn't, then $\mu_t(\omega_t^*\setminus\omega)=1$, but $\vol(\omega_t^*\setminus\omega) < \vol(\omega_t^*)$ leading to a contradiction with the definition of $\omega_t^*$ in \eqref{eq:probabilistic-FRS}. Since $\mu_t(\omega)>0$, by the properties of push-forward measure, there exist subsets $\mathcal{U}_i$ and $\mathcal{W}_i$ with non-zero measure that can bring the agent's state to $\omega$. Therefore, $\omega \subseteq F_t$.
Let $F\subseteq F_t$ be a measurable set with $\vol(F)>0$.
% Let $\hat{f}(u_{t-1},\cdots,u_0,w_{t-1},\cdots,w_0):=f(t-1,$
As the composition of $f$ in \eqref{eq:deterministic-FRS} for a fixed $x_0$---for the sake of convenience, let's call it $\hat{f}$---is a continuously differentiable map, by the Luzin's N-property for multi-dimensional maps, the pre-image $\hat{f}^{-1}(F)\subseteq\mathcal{U}_{t-1}\times\cdots\mathcal{U}_0\times\mathcal{W}_{t-1}\times\cdots\times\mathcal{W}_0$ will be a set of non-zero Lebesgue measure as well. As we assume that the distribution over the space of controls and disturbances is absolutely continuous, and $\hat{f}^{-1}(F)$ lies in the support of these distributions with non-zero Lebesgue measure, it follows that the probability measure of $\hat{f}^{-1}(F)$ is also non-zero. By the property of the push-forward measure if the probability measure of $\hat{f}^{-1}(F)$ is non-zero, then the measure under the push-forward of these sets is also non-zero, i.e., $\mu_t(F)>0$. Hence, $F\subseteq\omega_t^*$, for if it were removed from the set, the feasibility constraint of the lower bound will be violated.
\end{proof}
\begin{proof}[\textbf{Proof of Theorem~\ref{thm:convex-opt}}]
The volume of an ellipse $E_i(c_i)$ in an $n$-dimensional space \cite{wilson2010volume} is given by
\begin{equation}
\vol(E_i(c_i)) = \frac{\pi^{n/2}c_i^{n/2}}{\Gamma(n/2+1)}\sqrt{\lambda_{i,1}\cdots\lambda_{i,n}}
\end{equation}
where $\Gamma$ is the standard gamma function in calculus while $\lambda_{i,j}$ are the eigen values of $\Sigma$. Setting $n=2$ in the above and using it in \eqref{eq:approx-opt} gives the objective function of \eqref{eq:convex-opt}.
In an $n$-dimensional space the function $V_i(x)$ is $\chi^2$ distributed if $x$ is drawn from a Gaussian \cite{zorin1989distribution}. For the special case of $n=2$, the cumulative distribution of $\chi^2$ gives $\mu_{t,i}(E_i(c_i)) = (1-\exp(-c_i/2))$. Using this in \eqref{eq:approx-opt} gives the constraint of \eqref{eq:convex-opt}.
Finally, we note that the objective function of \eqref{eq:convex-opt} is linear, while the constraint is convex. For the latter assertion, note that the second derivative of $p_i (1-\exp(-c_i/2))$ is $-p_i \exp(-c_i/2)/4$ which is always negative, hence $p_i (1-\exp(-c_i/2))$ is concave for all $i$. Sum of concave functions is concave, therefore $\sum_{i=1}^K p_i (1-\exp(-c_i/2))$ is concave. Since the super-level sets of concave functions are convex, we get that the constraint of \eqref{eq:convex-opt} is also convex, implying that \eqref{eq:convex-opt} is a convex optimization problem.
\end{proof}
\begin{proof}[\textbf{Proof of Corollary~\ref{cor:scale-invariance}}]
We will establish this corollary by contradiction. Let $\alpha_1,\alpha_2\in(0,\infty)$ and $\alpha_1\neq \alpha_2$. Let $\{c_{i,\alpha_1}^*\}_{i=1}^K$ and $\{c_{i,\alpha_2}^*\}_{i=1}^K$ be the solutions of \eqref{eq:convex-opt} for scaled covariances $\{\alpha_1\Sigma_i\}_{i=1}^K$ and $\{\alpha_2\Sigma_i\}_{i=1}^K$, respectively. Let's assume that there exists an $i$, such that $c_{i,\alpha_1}^*\neq c_{i,\alpha_2}^*$, and without loss of generality assume that $c_{i,\alpha_1}^*< c_{i,\alpha_2}^*$. Then,
\begin{align}\nonumber
\sum_{i=1}^K \pi \alpha_2 \sqrt{\lambda_{i,1}\lambda_{i,2}} c_{i,\alpha_1}^* < \sum_{i=1}^K \pi \alpha_2 \sqrt{\lambda_{i,1}\lambda_{i,2}} c_{i,\alpha_2}^*.
\end{align}
Further, observe that $\sum_{i=1}^K p_i (1-\exp(-c_{i,\alpha_1}^*/2))\geq \tau$ as it is an optimal (hence, feasible) solution of \eqref{eq:convex-opt} for $\{\alpha_1\Sigma_i\}_{i=1}^K$, therefore, $c_{1,\alpha_1}^*,\cdots,c_{K,\alpha_1}^*$ is also a feasible solution of \eqref{eq:convex-opt} for $\{\alpha_1\Sigma_i\}_{i=1}^K$, but attains a smaller objective function value than the optimal solution of \eqref{eq:convex-opt} for $\alpha_2$, leading to a contradiction, hence, the claim must be true.
\end{proof}
\begin{proof}[\textbf{Proof of Lemma~\ref{lem:monotonic-set-inclusion}}]
Using Corollary~\ref{cor:scale-invariance}, it follows that $\{c_i^*\}_{i=1}^K$ are the same regardless of the scaling choice. As $\alpha_1 < \alpha_2$, then it follows that $\{x~|~(x-\overline{x}_i)^{\rm T} \alpha_1^{-1}\Sigma_i^{-1} (x-\overline{x}_i) \leq c_i^*\} \subseteq \{x~|~(x-\overline{x}_i)^{\rm T} \alpha_2^{-1}\Sigma_i^{-1} (x-\overline{x}_i) \leq c_i^*\}$ as $(x-\overline{x}_i)^{\rm T} \alpha_1^{-1}\Sigma_i^{-1} (x-\overline{x}_i) > (x-\overline{x}_i)^{\rm T} \alpha_2^{-1}\Sigma_i^{-1} (x-\overline{x}_i)$. Therefore, taking the union of all these sets gives us $E^*(\{\alpha_1\Sigma_i\}_{i=1}^K) \subseteq E^*(\{\alpha_2\Sigma_i\}_{i=1}^K)$.
\end{proof}
\end{document}
% \usepackage[Symbolsmallscale]{upgreek}
\usepackage{xargs}
\usepackage{float}
\usepackage{wrapfig}
% \usepackage[normalem]{ulem}
\usepackage{amsmath,amsfonts}
\usepackage{algorithm}
\usepackage{array}
% \usepackage[caption=false,font=normalsize,labelfont=sf,textfont=sf]{subfig}
\usepackage{textcomp}
\usepackage{stfloats}
\usepackage{url}
\usepackage{verbatim}
\usepackage{graphicx}
\usepackage{cite}
\usepackage{xcolor}
\def\bibfont{\small}
\usepackage{graphicx}
\usepackage{caption}
% \captionsetup[table]{position=bottom}
\usepackage{subcaption}
\usepackage{svg}
\usepackage{algorithm, setspace}
% \usepackage{arevmath} % For math symbols
\usepackage[noend]{algpseudocode}
\usepackage[%
colorlinks=true,
pdfborder={0 0 0},
linkcolor=red
]{hyperref}
\algnewcommand\algorithmicforeach{\textbf{for each}}
\algdef{S}[FOR]{ForEach}[1]{\algorithmicforeach\ #1\ \algorithmicdo}
\usepackage{graphics} % for pdf, bitmapped graphics files
\usepackage{epsfig} % for postscript graphics files
% \usepackage{mathptmx} % assumes new font selection scheme installed
\usepackage{times} % assumes new font selection scheme installed
\usepackage{amsmath} % assumes amsmath package installed
\usepackage{amssymb} % assumes amsmath package installed
% \usepackage{gensymb} % assumes amsmath package installed
\usepackage{textcomp, gensymb}
% \usepackage{parskip} % removes the underfull issues
% \usepackage{subcaption}
\usepackage{url}
\usepackage{graphicx}
\usepackage{color}
\usepackage{xcolor}
\usepackage{soul}
\usepackage{mathtools}
\usepackage{algorithm, setspace}
\usepackage[noend]{algpseudocode}
\usepackage{ccicons}
\usepackage{comment}
\usepackage{bbm}
\usepackage{multirow}
\usepackage[compact]{titlesec}
\usepackage{booktabs}
% \usepackage{enumitem}
\renewcommand{\baselinestretch}{0.99}
\newtheorem{theorem}{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{corollary}{Corollary}
\newcommand{\vol}{\mathrm{vol}}
% \titlespacing{\section}{0mm}{1mm}{1mm}
% \titlespacing{\subsection}{0mm}{1mm}{1mm}
\usepackage[font={small}]{caption}
% \setlength{\abovecaptionskip}{1mm}
% \setlength{\belowcaptionskip}{-3mm}
% \setlength{\abovedisplayskip}{0mm}
% \setlength{\belowdisplayskip}{0mm}
% \usepackage{titlesec}
% \titlespacing{\section}{0mm}{0mm}{0mm}
% \titlespacing{\subsection}{0.5mm}{0.5mm}{0.5mm}
% \setlength{\abovecaptionskip}{1mm}
% \setlength{\belowcaptionskip}{-2mm}
% \setlength{\abovedisplayskip}{-5mm}
% \setlength{\belowdisplayskip}{-5mm}
\newcommand{\qa}[1]{{\textcolor[HTML]{6491EA}{\textbf{#1}}}}
\newcommand{\kccom}[1]{[{\textbf{\textcolor{blue}{KC: #1}}}]}
\newcommand{\sbnote}[1]{[{{\textcolor{magenta}{SB: #1}}}]}
\newcommand{\zfnote}[1]{[{{\textcolor{orange}{ZF: #1}}}]}
\newcommand{\svnote}[1]{[{{\textcolor{teal}{SV: #1}}}]}
\maxdeadcycles=500
% The advent of end-to-end autonomy stacks that lack modules with interpretable interfaces have placed an increased burden on ensuring that the final output of the stack, i.e., the motion plan, is safe in order to validate the safety of the entire stack. A monitor that assesses the safety of the motion plan should be as close as possible to being complete (triggers for all unsafe plans) while maintaining high soundness (does not trigger for safe plans). In this paper, we build a safety monitor by interpreting multi-modal learning-based trajectory predictors as neural representations for data-driven forward-reachable sets (FRS) which are extracted from the predictors by solving a convex program. This allows us to jointly generate FRS for very many agents in a scene and conditional on other contextual cues, such as lane graphs. To improve the completeness of our monitor, we leverage conformal prediction to adapt the FRS to ensure that the FRS covers the ground truth with high probability. Finally, to deal with any prediction failures, we employ a Bayesian filtering approach that switches to a more conservative FRS when it gauges deterioration of the prediction performance. Extensive experiments on nuScenes, a real-world driving dataset, demonstrates the ability of our monitor to improve upon soundness without compromising on completeness.
%
The advent of end-to-end autonomy stacks---often lacking interpretable intermediate modules---has placed an increased burden on ensuring that the final output, i.e., the motion plan, is safe in order to validate the safety of the entire stack.
This requires a safety monitor that is both complete (able to detect all unsafe plans) and sound (does not flag safe plans).
In this work, we propose a principled safety monitor that leverages modern multi-modal trajectory predictors to approximate forward reachable sets (FRS) of surrounding agents.
By formulating a convex program, we efficiently extract these data-driven FRSs directly from the predicted state distributions, conditioned on scene context such as lane topology and agent history.
To ensure completeness, we leverage conformal prediction to calibrate the FRS and guarantee coverage of ground-truth trajectories with high probability.
To preserve soundness in out-of-distribution (OOD) scenarios or under predictor failure, we introduce a Bayesian filter that dynamically adjusts the FRS conservativeness based on the predictor’s observed performance.
We then assess the safety of the ego vehicle’s motion plan by checking for intersections with these calibrated FRSs, ensuring the plan remains collision-free under plausible future behaviors of others.
Extensive experiments on the nuScenes dataset show our approach significantly improves soundness while maintaining completeness, offering a practical and reliable safety monitor for learned autonomy stacks.
% Multimodal predictors have emerged as a powerful tool for behavior estimation, capturing not only the most likely actions of a tracked agent but also potential deviations, each with an associated confidence level. Ignoring this uncertainty in planning algorithms of autonomous systems can result in biased decisions in the downstream process, while treating all possible outcomes as equally likely may lead to overly conservative behavior.
% The deployment of safety-critical systems necessitates rigorous assurances of completeness. Reachability-based methods employing the worst-case assumptions are often proposed as the solution for such problems; we seek the set of all possible states that the system can exist in, given certain assumptions on the error/control bounds. Nevertheless, such methods are quite impractical in real-world deployment due to their overly conservative approach.
% This work introduces an algorithm that systematically integrates these predictive uncertainties into the planning process. We then employ the advances in the literature of online conformal prediction to provide a smooth growth of conservativeness while accounting for the errors in the predictive capability of the underlying predictor. If at any point we encounter a situation where the predictions are catastrophically incorrect, we employ a hard switch to a traditional HJ-based conservative solution to preserve completeness in the safety of the system.
% Hence, our algorithm can provide provably safe reachable sets incorporating the confidence of the underlying predictor while correcting for any errors in the prediction if they are incorrect.
% \sbnote{I believe RA-L papers are now double blind. Make sure you are using the latest template.}
Classical planning stacks come in various shapes and forms \cite{teng2023motion,schwarting2018planning}, but, importantly, they share the characteristic of optimizing an interpretable cost function which allows reasoning about the plan's safety during synthesis.
With the ever-increasing adoption of learning-based planners and end-to-end robot stacks, safe-by-construction planning has become extremely challenging, if not outright impossible.
Therefore, safety monitors for motion plans have grown in prominence for ensuring that these often uninterpretable learning-based plans are safe.
% These monitors are either used to trigger fallback to a classical stack \citep{} or safety maneuvers \citep{} while operating online, or they can be used offline to provide a signal to developers for improving safety in subsequent design cycles.
% Like any effective monitor, we desire our safety monitor to exhibit completeness (trigger for all unsafe plans) and soundness (not trigger for safe plans).
Like any effective monitor, a safety monitor should satisfy two key properties: completeness (it must flag all unsafe plans) and soundness (it must not flag safe ones).
% Often in practice, there is a a pareto frontier that traces a tradeoff between completeness and soundness.
Our objective in this paper is to develop a method that can improve on the soundness of reachability-based safety monitors without compromising on completeness.
% We will accomplish this by leveraging trajectory predictors as generators of data-driven forward reachable sets (FRS).
\begin{figure*}[t]
\centering
\begin{subfigure}[b]{0.32\textwidth}
\includegraphics[width=\textwidth]{figures/baseline.png}
\caption{Baseline Predictor}
\label{fig:subfig1}
\end{subfigure}
\hfill
\begin{subfigure}[b]{0.32\textwidth}
\includegraphics[width=\textwidth]{figures/FRS.png}
\caption{FRS}
\label{fig:subfig2}
\end{subfigure}
\hfill
% \begin{subfigure}[b]{0.23\textwidth}
% \includegraphics[width=\textwidth]{figures/2P.png}
% \caption{Pursue-Evade Game}
% \label{fig:subfig3}
% \end{subfigure}
% \hfill
\begin{subfigure}[b]{0.32\textwidth}
\includegraphics[width=\textwidth]{figures/forceopt.png}
\caption{FORCE-OPT (ours)}
\label{fig:subfig3}
\end{subfigure}
\caption{\small{In this intersection scenario from the nuScenes dataset, we evaluate the safety of the ego (blue) vehicle's planned trajectory (black) in the vicinity of a non-ego contender agent (green). Our objective is to verify that the ego’s planned path avoids any potential collisions with the contender by ensuring it remains outside all regions that the contender could possibly occupy in the future. We compare three methods of reachable set computation for safety evaluation of ego's motion plan. \textbf{(a) Baseline Predictor:} This method produces very small, overconfident predictions (cyan ellipses) for the other agent's future positions. While it correctly avoids a false positive (FP) collision warning, it fails to cover the true future positions of the contender (the cyan ellipses do not cover the true ground truth states states shown with red dots), making it unreliable for safety---indeed, in the detailed results later in the paper (Section~\ref{sec:result}) this methods exhibits a high False Negative Rate (FNR). \textbf{(b) FRS (Forward Reachable Set):} This approach creates a large, conservative reachable set (the large cyan circle) for the other agent. It achieves excellent coverage, but its excessive size causes it to overlap with the ego vehicle's path, triggering a FP alarm. \textbf{(c) FORCE-OPT (ours): } Our method generates a more realistic and tighter reachable set (the smaller cyan area) for the other agent. It successfully achieves coverage by containing the agent's true future path (green dots) while being precise enough to avoid intersecting with the ego's trajectory, preventing a FP, correctly identifying the plan as safe. Overall, our method demonstrates a good balance of FPR and FNR as discussed in greater detail in Section~\ref{sec:result}.}}
\label{fig:all_subfigures}
\vspace{-5mm}
\end{figure*}
We achieve this by reinterpreting trajectory predictors - typically generative models trained to forecast future agent behavior conditioned on histories and scene context (e.g., lane graphs, traffic signals) - as \textit{data-driven forward reachable set (FRS) estimators}.
% Trajectory predictors often take the form of generative networks trained to predict future trajectories of agents given histories and other contextual cues, such as lane graphs or traffic signs.
% Without squinting too hard, one can see that trajectory predictors are learning to estimate where an agent can go based on the logged driving data, i.e., it learns a stochastic representation of the agent dynamics.
% We extract an FRS from stochastic dynamics by solving an optimization problem that aims to identify the set with the smallest ``volume" that has a probability measure of $1$ under the stochastic dynamics.
These predictors, often instantiated as Gaussian Mixture Models (GMMs), implicitly learn a stochastic model of agent dynamics from logged driving data. Our \textit{key insight} is that we can extract a probabilistic FRS by solving an optimization problem that identifies the smallest-volume set that captures a desired amount of probability mass under the learned distribution.
Leveraging the fact that many modern trajectory predictors output GMMs on the future states \cite{salzmann2020trajectron++,girgis2021latent}, we formulate a convex relaxation of this problem that allows us to efficiently compute tight reachable sets that are significantly less conservative than traditional worst-case reachability approaches. However, these learned distributions are subject to modeling error and may fail to capture the true dynamics, especially under distribution shift.
% Unlike prior literature that estimates FRS from trajectory predictors, this results in leveraging multi-modality of the trajectory predictors. It is also worthwhile to note that classical notions of reachability struggle to scale to a large number of agents in a scene and account for other contextual cues, such as lane graphs, which our trajectory predictor-based FRS approach can easily tackle.
% The learned distributions from trajectory predictors are indeed much tighter than a worst-case reachability analysis, but they are subject to learning errors, i.e., the output distribution on the future states might not be reflective of the actual dynamics.
To combat learning errors, we use conformal prediction (CP) \cite{angelopoulos2021gentle,shafer2008tutorial} to calibrate the FRS.
% Effectively, we use CP to identify how much we should scale up / down the covariance of our GMMs to achieve a desired error rate on the FRS covering the ground truth future trajectory.
Rather than inflating the predicted sets indiscriminately, CP allows us to scale the covariance of each GMM component just enough to ensure that the reachable set covers the ground-truth trajectory with a user-specified error rate.
Our overall algorithm to extract the FRS is called FORCE-OPT, which stands for \underline{FO}rward \underline{R}eachable sets from \underline{C}onformal \underline{E}stimation and convex \underline{OPT}imization.
Although FORCE-OPT comes with a probabilistic guarantee on covering the ground truth when operating within the calibration distribution, it
may still degrade in the presence of distribution shift.
To handle this, we introduce a Bayesian filtering mechanism that monitors the consistency between predicted and observed agent behavior, adjusting the FRS conservativeness on the fly based on our confidence in the trajectory predictor.
% To remedy this we use a Bayesian filtering approach to adapt the FRS on-the-fly according to our confidence in the correctness of the trajectory predictor.
This main contributions of the paper are: (i) We provide a rigorous formulation of the problem of estimating FRS from trajectory predictors.
(ii) We introduce FORCE-OPT, an efficient algorithm that combines convex optimization and conformal prediction to compute calibrated, data-driven FRS from GMM-based predictors.
% (ii) We introduce FORCE-OPT, an algorithm that leverages the GMM structure of the output of many learned trajectory predictors, to compute FRS. FORCE-OPT involves applying conformal prediction on a convex optimization-based non-conformity score, providing us learned FRS that come equipped with a probabilistic guarantee of covering the ground truth within the calibration distribution.
% (iii) We use a belief-based adaptation of the FRS to robustify our approach to distribution shifts.
(iii) We incorporate a belief-based Bayesian filtering mechanism that adapts the reachable set dynamically to account for predictor reliability under distribution shift.
(iv) Through testing on nuScenes \cite{nuscenes}, real-world driving dataset, we demonstrate that FORCE-OPT achieves the best balance of false positive and false negative rates, outperforming both conservative baselines and uncalibrated learned methods.
% \textcolor{red}{[Rewrite after experiments section is done]}
% \begin{itemize}
% \item We provide a rigorous formulation of the problem of estimating FRS from trajectory predictors.
% \item We introduce FORCE-OPT, an algorithm that leverages the GMM structure of the output of many learned trajectory predictors, to compute FRS. FORCE-OPT involves applying conformal prediction on a convex optimization-based non-conformity score, providing us learned FRS that come equipped with a probabilistic guarantee of covering the ground truth within the calibration distribution.
% % develop a convex program to extract data-driven FRS from multi-modal trajectory predictors that take the form of a generative network that outputs GMMs.
% \item We use a belief-based adaptation of the FRS to robustify our approach to distribution shifts.
% \item Through testing on real-world driving datasets nuScenes \cite{} and nuPlan-mini \cite{} we demonstrate that our approach achieves the best balance of false positive and false negative rates. \textcolor{red}{[Rewrite after experiments section is done]}
% \end{itemize}
% Some points to mention:
% \begin{itemize}
% \item We view trajectory predictors as a learned FRS generator (can we go as far as calling it an implicit representation of FRS?)
% \item Trajectory predictors learn to assess where agents might go while accounting for the scene context, multiple agents, road rules (implicitly / explicitly [MPF, RuleFuser]) and road geometry, unlike classical reachability methods that are limited by the number of agents one can reason about and are definitely challenged by road geometry and road rules.
% \item However, trajectory predictor standalone suffers from various shortcomings: the output distributions may not be well calibrated and OOD robustness---we shore up our method against these shortcomings via conformal estimation of the FRS and Bayesian belief based adaptation for OOD.
% \item The end result is a method that successfully captures safety-critical situations without being overly conservative.
% \end{itemize}
% [Key points: Overly conservative. Limited in their ability to ingest context and other cues. End with pointing to trajectory prediction-based safety eval which overcomes these issues.] \textcolor{red}{[Kaustav]}
\textbf{Control-Theoretic Safety Evaluators.}
% The capability to assess the safety of a robot's actions is a key requirement towards their deployment in safety-critical frameworks.
Classical approaches for safety evaluation of motion plans have relied on control-theoretic tools such as formal verification \cite{luckcuck2019formal, mitra2024formal}, which provide mathematical guarantees that a system satisfies a predefined set of safety specifications. While rigorous, these methods often fall short when applied to modern robotic systems that encounter large uncertainties and operate in high-dimensional, stochastic environments. To handle this imminent uncertainty, reachability-based techniques—including those based on zonotopes \cite{althoff2011zonotope}, Hamilton-Jacobi formulations \cite{lygeros1999controllers,mitchell2005toolbox}, or sums-of-squares \cite{majumdar2017funnel}—aim to characterize the set of all future states a system can reach under bounded disturbances. However, their worst-case assumptions often lead to overly conservative safety bounds that limit practical usability. Furthermore, these methods typically struggle to incorporate rich sensory inputs (e.g., images, LiDAR) and contextual cues that modern autonomous systems depend on. High-dimensional observations are often abstracted through models \cite{fridovich2020confidence} or simplifying assumptions are made, reducing the granularity of the safety assessment. A common alternative to handling the difficult-to-model uncertainties in the real world is through data-driven methods. One such approach, tailored towards multi-agent settings, is to leverage learning-based trajectory predictors, as we will discuss next.
% To overcome these limitations, recent efforts have focused on safety evaluation frameworks built on top of learning-based trajectory predictors \cite{salzmann20, hwang2024emma,girgis2021latent}. These models are designed to ingest multimodal context—including map data, agent interactions, and sensor inputs—and produce diverse, probabilistic forecasts of future behavior. As a result, they enable more nuanced, less conservative safety analysis that better reflects the complexities of real-world environments.
% Enabling robots with the capability to access the safety of their actions is a key requirement towards their deployment in safety critical frameworks. Historically, such assessments were either done through the lens of formal verification \cite{luckcuck2019formal} providing mathematical basis for checking if the robotic systems satisfy the required specification essential for their safe operation. However,these methods don't support robust verifications --> important because the systems often employ stochastic components which are hard to model exactly. Reachability analysis \kccom{cite some reachability literature zonotopes and hjb etc} allows us to account for their safety under certain worst-case disturbance bounds. Such methods compute the set of states that a system can reach under the specified settings. However, these methods often end up being overly conservative due to their worst case assumiptions, which results in them being impractical for actual use. Moreover, using such methods it is often difficult to account for the rich sensory inputs that these systems function with, and often times one has to make simplifying assumption. For such reasons, safety analysis with state of the art trajectory predictors \kccom{cite a few predictors} has seen much success in recent times owing to their ability to account for such rich contextual cues and sensory inputs.
\textbf{Trajectory Prediction-based Safety Evaluators.} Motivated by the limitations of worst-case control-theoretic safety evaluators, there has been growing interest in safety evaluation frameworks built on top of learning-based trajectory predictors \cite{salzmann20, hwang2024emma,girgis2021latent}.
% These models are designed to ingest multimodal context—including map data, agent interactions, and sensor inputs—and produce diverse, probabilistic forecasts of future behavior; essentially, trajectory predictors are learned, context-aware, transition dynamics.
% As a result, they enable more nuanced and less conservative safety analysis that better reflects the complexities of real-world environments.
Although trajectory predictors were devised to aid with planning \cite{karkus2023diffstack,chen2023interactive,casas2021mp3,cui2021lookout}, they have found widespread use in other applications, such as failure monitoring \cite{antonante2023task,chakraborty2024system}, mining interactive scenarios \cite{dinparastdjadid2023measuring,ding2025surprise}, and assessing planner safety \cite{nakamura2023online,lindemann2023safe,li2020prediction} which is of primary interest to us in this paper. Trajectory predictors are often used to identify reachable zones for other agents that the motion plan should stay out of, violation of which is considered to trigger a safety failure; see \cite{althoff2021set} for a survey. Methods in this category include those that use trajectory predictors to guide controllability bounds on other agents to be used in a classical reachability formulation \cite{nakamura2023online,li2020prediction,fridovich2020confidence} and those that directly extract a reachable zone for contender agents from the predictor \cite{paparusso2024zapp,lindemann2023safe,dixit2023adaptive,chen2021reactive}. In the latter category, approaches have explored fitting zonotopes to prediction outputs for estimating reachable sets \cite{paparusso2024zapp}, leveraged conformal prediction \cite{lindemann2023safe,dixit2023adaptive,chen2021reactive}, and solved a probabilistic optimization on the output distribution of the predictor \cite{driggs2017robust,devonport2020estimating}. However, many of these approaches require sampling \cite{tumu2024multi,xiang2024convex,dietrich2025data} which is computationally expensive, lack multi-modality \cite{lindemann2023safe}, or lack calibration of the predictors \cite{driggs2017robust,driggs2017integrating} resulting in learning-errors affecting the quality of the FRS. The approach presented in this paper is computationally efficient, accounts for multi-modality in predictions, calibrates the predictions to mitigate the impact of learning errors, and performs a belief-based FRS adaptation to impart some degree of OOD robustness.
% addresses many of these issues by computing FRSs efficiently using a convex program without sampling from the predictor's output by leveraging the structure of GMMs. As we calibrate the predictor using CP, the resulting FRS has a probabilistic guarantee of covering the ground truth future when within the calibration distribution. Furthermore, we perform a belief based adaptation, akin to \cite{nakamura2023online,fridovich2020confidence} to adapt the FRS in OOD scenarios.
% [Ongoing] Various approaches for estimating FRS from stochastic dynamics or trajectory predictors have appeared in prior literature, however, they either use simple prediction models \cite{driggs2017robust}, expensive sampling-based approaches \cite{lew2021sampling,tumu2024multi}, or lack multimodality .
% [Key points: Trajectory predictors provide a way to perform safety analysis without crash data while accounting for multiple agents in the scene and other cues. Existing approaches (that use CP) produce conservative sets as they do not leverage the structure of the predictor's output (i.e., GMM), additionally they also do not leverage multi-modality. Fragile to OOD scenarios.]
%
We now present a rigorous mathematical framework to bridge the probabilistic and deterministic definitions of FRS. While the former is characteristic of contemporary trajectory forecasting literature, the latter is fundamental to control-theoretic safety analysis.
% By establishing a formal mapping between them, we demonstrate that the FRS can be computed by solving a well-posed optimization problem whose parameters are obtained from a probabilistic predictor's output distribution.
Let $x_t\in \mathcal{X}\subseteq\mathbb{R}^{n_x}$ denote the state of an agent at discrete time $t\in\mathbb{Z}_+$. The agent’s control inputs are denoted by $u_t \in \mathcal{U}_t \subseteq \mathbb{R}^{n_u}$, while other disturbances arising from exogenous factors and uncertainties are captured by $w_t\in\mathcal{W}_t\subseteq\mathbb{R}^{n_w}$.
The agent's dynamics evolve according to a discrete-time function $f: \mathbb{Z}_{+} \times \mathcal{X} \times \mathcal{U} \times \mathcal{W} \rightarrow \mathcal{X}$, which is assumed to be continuously differentiable in its arguments:
%
\begin{equation}\label{eq:dynamics}
x_{t+1} = f(t,x_t, u_t, w_t).
\end{equation}
%
For notational brevity, we define $\hat{f}_{t,u_t,w_t}(x_t):=f(t,x_t, u_t, w_t)$. The \emph{forward-reachable set} (FRS) $F_t$ at time $t$, starting from an initial state $x_0\in\mathcal{X}$, is the set of all states that the agent can reach under a set of possible control actions and disturbances:
%
\begin{align}\label{eq:deterministic-FRS}
F_t := & \{\hat{f}_{t-1, u_{t-1}, w_{t-1}}\circ \cdots \circ \hat{f}_{0,u_0, w_0}(x_0) \\ \nonumber
&~|~u_i\in\mathcal{U}_i, w_i\in\mathcal{W}_i, i=0,\cdots,t-1\}
\end{align}
%
% In practice, it is usually infeasible to know the exact control set $\mathcal{U}_i$ and disturbance set $\mathcal{W}_i$ for each timestep. The sets are influenced by difficult-to-model properties such as the agent's intent, driving style, road geometry, etc., and are subject to high degree of uncertainty.
In practice, however, the precise control sets $\mathcal{U}_i$ and disturbance sets $\mathcal{W}_i$ are unknown and influenced by hard-to-model aspects such as driver intent, road geometry, and local context.
% Classical approaches make sweeping worst-case assumptions on the control and disturbance sets to generate overly-conservative FRS. Indeed, worst-case bounds on control and disturbance is too harsh of an assumption.
% For instance, it is unreasonable to expect a stopped vehicle on a red light to floor the accelerator when the traffic light is still red.
Classical reachability methods often adopt worst-case assumptions over these sets, resulting in overly conservative FRSs. Such assumptions are unnecessarily pessimistic - for instance, it is unreasonable to expect that a stopped vehicle at a red light will suddenly accelerate through the intersection while the traffic light is still red.
% Ideally, these assumptions should be aligned with what agents usually do in a given situation which can often be uncovered from data. In what follows, we will present a formulation for the FRS that facilitates the usage of trajectory predictors trained on driving data to predict what an agent might do in a given situation, thereby, lowering the conservatism of the FRS.
A more realistic alternative is to infer the agent’s behavior from data. In what follows, we introduce a probabilistic formulation of the FRS that facilitates using data-driven trajectory predictors to estimate likely future states of an agent, thereby, reducing the conservatism of the FRS.
% Alternately, we can express the FRS as an optimization problem.
% To extract an FRS that holds with high probability, but is not overly conservative, we can solve for the set with the smallest volume that meets a desired probability bound [Driggs-Campbell et al., 2017].
% We can also express the FRS by leveraging a probabilistic setup inspired by prior works \citep{}.
We now reformulate the FRS using a probabilistic lens, as inspired by prior work \cite{driggs2017robust}.
As we highlight later in this paper, this probabilistic viewpoint is readily compatible with modern multi-modal trajectory predictors.
Let $\vol$ represent the volume (Lebesgue measure) of a measurable set $\omega \in \Omega$, where $\Omega$ is the collection of all measurable subsets of $\mathcal{X}$.
% Let $\mu_t:\Omega \to [0,1]$ be a probability measure that expresses where an agent's state might transition to in $t$ timesteps.
Let $\mu_t : \Omega \to [0, 1]$ represent a probability measure describing the distribution over the agent’s state at time $t$.
% This measure arises as the push-forward measure of absolutely continuous probability distributions with support on $\mathcal{U}_i$ and $\mathcal{W}_i$, mapped through the continuously differentiable mapping that arises from the composition of $f$ in \eqref{eq:deterministic-FRS}.
This measure arises as the push-forward of absolutely continuous probability distributions over the sequences of control and disturbance inputs, with support on $\mathcal{U}_i$ and $\mathcal{W}_i$, mapped through the composed dynamics in \eqref{eq:deterministic-FRS}.
The probabilistic FRS can then be defined through the following optimization problem:
\begin{equation}\label{eq:probabilistic-FRS}
\omega_t^* := \underset{\{\omega\in\Omega: \mu_t(\omega) = 1\}}{\arg\inf} \vol(\omega).
\end{equation}
%
% Effectively, \eqref{eq:probabilistic-FRS} finds the smallest set that the agent's state can end up in under the probabilistic dynamics $\mu_t$. The sets from \eqref{eq:deterministic-FRS} and \eqref{eq:probabilistic-FRS} are, in fact, the same almost everywhere (i.e., excluding a set of measure zero) as formalized in the following theorem.
Intuitively, \eqref{eq:probabilistic-FRS} returns the smallest (i.e., minimal-volume) set that captures all the future states of the agent under the distribution $\mu_t$. Remarkably, this probabilistic formulation is equivalent to the classical deterministic definition in \eqref{eq:deterministic-FRS} almost everywhere (i.e., excluding a set of measure zero) as formalized in the following theorem.
%
\begin{theorem}[Equivalence of FRS]\label{thm:deter-prob-FRS}
Let the dynamics $f$ in \eqref{eq:dynamics} be continuously differentiable. Let $\mu_t$ be the push-forward of absolutely continuous probability distributions with support on $\mathcal{U}_i$ and $\mathcal{W}_i$, mapped through the composed dynamics $\hat{f}_{t-1, u_{t-1}, w_{t-1}}\circ \cdots \circ \hat{f}_{0,u_0, w_0}$.
% Let $F_t$ and $\omega_t^*$ be defined as in \eqref{eq:deterministic-FRS} and \eqref{eq:probabilistic-FRS}, respectively. Then, $F_t=\omega_t^*$ almost everywhere.
Then the sets $F_t$ and $\omega_t^*$ defined in \eqref{eq:deterministic-FRS} and \eqref{eq:probabilistic-FRS} are equal almost everywhere (except for sets with measure zero).
\end{theorem}
%
The proof of this result is provided in the Appendix.
The probabilistic formulation in \eqref{eq:probabilistic-FRS} offers a powerful bridge to estimating FRS using modern trajectory predictors, which are generative networks that learn distributions $\mu_t$ over future states of agents from some training driving data.
% The aforementioned difficult-to-model sources of uncertainty that determine the sets $\mathcal{U}_i$ and $\mathcal{W}_i$ are now implicitly learned by trajectory predictors from data, allowing us to extract more refined FRS instead of using the sledgehammer of worst-case reachability---worth remarking here, that worst-case reachability is still of relevance as a fallback in regimes where the learned predictor fails, either due to learning errors or the data being out-of-distribution.
These models implicitly encode the hard-to-specify uncertainties (e.g., $\mathcal{U}_i$, $\mathcal{W}_i$) and allow us to extract refined, data-driven FRSs that are less conservative than worst-case reachability. Nonetheless, it is worth noting that worst-case approaches remain valuable as fallback strategies when predictors fail due to learning errors or out-of-distribution (OOD) conditions.
% In the rest of this paper, we will use the perspective of \eqref{eq:probabilistic-FRS} to accomplish the following: (i) formulate an efficient algorithm called FORCE-OPT to extract FRS from learning-based trajectory predictors which are accompanied with a probabilistic guarantee of including the ground truth future within it, and (ii) leverage this FRS to develop a safety monitor for plans and empirically demonstrate low false-negative and low false-positive rates for within distribution and out-of-distribution operation.
%
In the remainder of this paper, we adopt the probabilistic view of the FRS to achieve two key goals:
(i) develop an efficient algorithm, FORCE-OPT, to extract FRSs from learning-based trajectory predictors with probabilistic guarantees of ground-truth coverage; and
(ii) use these calibrated FRSs to build a safety monitor that evaluates motion plans and empirically demonstrate low false-negative and false-positive rates under both in-distribution and OOD operation.
%
% In this section, we will elaborate our algorithm, FORCE-OPT, for estimating FRS using trajectory predictors and calibrating these sets with conformal prediction.
We now present FORCE-OPT, our algorithm for estimating FRS from trajectory predictors and calibrating them using conformal prediction. FORCE-OPT combines learned generative models with convex optimization to efficiently compute calibrated FRSs, and uses Bayesian filtering to hedge against out-of-distribution (OOD) deployment failures.
\subsection{Trajectory Predictors}
\label{subsec:traj-pred}
%
Without loss of generality, let the current time be $0$.
% Let $\xi \in \mathcal{X}^{n_{\rm agents}}$ represent the state of every agent in the scene. The historical trajectories of all agents be denoted by $\xi_{-H:0}\in\mathcal{X}^{H n_{\rm agents}}$ and map information and other relevant scene information is denoted by $m\in\mathcal{M}$.
Denote the state of all agents in the scene by $\xi \in \mathcal{X}^{n_{\rm agents}}$, and their historical trajectories over a horizon $H$ by $\xi_{-H:0}\in\mathcal{X}^{H n_{\rm agents}}$.
Let $m \in \mathcal{M}$ represent map and other contextual scene information.
A traffic scene is then denoted by $s:=(\xi_{-H:0}, m)\in \mathcal{X}^H\times\mathcal{M}=:\mathcal{S}$.
A trajectory predictor is a function $\phi:\mathcal{S} \to \mathcal{P}(\mathcal{X}^T)$ that maps a scene $s\in\mathcal{S}$ to a probability distribution $\hat{\mu}\in\mathcal{P}(\mathcal{X}^T)$ over future trajectories of horizon $T$.
Most modern trajectory predictors \cite{salzmann20,girgis2021latent} represent $\phi$ using generative neural networks that output a sequence of Gaussian Mixture Models (GMMs) $\{\hat{\mu}_t\}_{t=1}^T$ -- one for each future timestep.
A GMM with $K$ modes is of the form: $\hat{\mu}_t = \sum_{i=1}^K p_i \hat{\mu}_{t,i}$ where $p_i\geq 0$, $\sum_{i=1}^K p_i = 1$, and each mode $\hat{\mu}_{t,i}$ is a Gaussian distribution $\mathcal{N}(\overline{x}_i, \Sigma_i)$ with mean at $\overline{x}_i$ and a covariance $\Sigma_i$.
\subsection{Extracting FRS from Trajectory Predictors}
\label{subsec:convex-opt}
We treat $\hat{\mu}_t$ as a learned approximation of the push-forward distribution $\mu_t$ in \eqref{eq:probabilistic-FRS}.
% However, these measures, being GMMs are supported over the entire unbounded real space, so to satisfy the lower bound in \eqref{eq:probabilistic-FRS} the set would have to be the entire unbounded real space as well.
% This is, of course, a modeling issue with $\hat{\mu}$ that we will, in practice, tackle by restricting the lower-bound on probability to a scalar $\tau \in (0,1)$ giving rise to:
However, given that GMMs have an unbounded support, these measures make it infeasible to find a bounded set $\omega$ satisfying $\hat{\mu}_t(\omega) = 1$.
To make the problem tractable, we relax the constraint to require only a minimum mass $\tau \in (0, 1)$:
%
\begin{equation}\label{eq:general-opt}
\omega^* := \underset{\{\omega\in\Omega: \hat{\mu}_t(\omega) \geq \tau\}}{\arg\inf} \vol(\omega).
\end{equation}
%
Ideally, we would choose a $\tau$ that is very close to 1. However, at this level of generality, this problem is very challenging to solve.
Leveraging the fact that the distribution over each timestep is a GMM, we solve a tractable proxy for this problem by restricting the search to unions of sub-level sets of the GMM modes.
% that retains the essence of what we desire to compute, albeit at the price of sets that are larger than what the true optimal (but unsolvable) solution could offer. We achieve this by limiting the space of sets to the union of sub-level sets of GMM modes.
For a given mode $\hat{\mu}_{t,i}$, define the Mahalanobis energy function $V_i(x) := (x-\overline{x}_i)^{\rm T} \Sigma_i^{-1} (x-\overline{x}_i)$, and its sublevel set $E_i(c_i) := \{x : V_i(x) \leq c_i\}$ for $c_i \geq 0$.
We then solve:
%
\begin{equation}\label{eq:approx-opt}
\begin{aligned}
c_1^*, \cdots, c_K^* = & \underset{c_1,\cdots,c_K}{\arg\inf} \sum_{i=1}^K \vol(E_i(c_i)), \\
& \mathrm{subject~to~} \sum_{i=1}^K p_i \hat{\mu}_{t,i}(E_i) \geq \tau, \\
& \phantom{\mathrm{subject~to~}} c_i \geq 0 \quad \text{for all } i = 1, \cdots, K.
\end{aligned}
\end{equation}
%
The resulting FRS is $E^* = \bigcup_{i=1}^K E_i(c_i^*)$.
Although $E^*$ is not necessarily the smallest such set, it is a feasible solution to \eqref{eq:general-opt}, and tight in practice. When $\mathcal{X} \subseteq \mathbb{R}^2$, it takes the form of a convex optimization, as detailed in the following theorem.
% As $\bigcup_{i=1}^K E_i$ is measurable being the union of sub-level sets of continuous functions, by the countable sub-additivity of the Lebesgue measure $\vol(\bigcup_{i=1}^K E_i) \leq \sum_{i=1}^K \vol(E_i)$, and using the fact that $E_i \subseteq \bigcup_{i=1}^K E_i$ for any i, we have that $\hat{\mu}_t(\bigcup_{i=1}^K E_i) = \sum_{i=1}^K p_i \hat{\mu}_{t,i}(\bigcup_{i=1}^K E_i) \geq \sum_{i=1}^K p_i \hat{\mu}_{t,i}(E_i)$. Therefore, the optimal set $E^*$ from \eqref{eq:approx-opt} is a feasible, albeit not necessarily optimal, solution of \eqref{eq:general-opt}. For 2D state space $\mathcal{X}$, \eqref{eq:approx-opt} takes the form of a convex optimization, as detailed in the following theorem.
%
\begin{theorem}[Convex Optimization for FRS Extraction]\label{thm:convex-opt}
If $\mathcal{X}\subseteq \mathbb{R}^2$, then \eqref{eq:approx-opt} takes the form of a convex optimization:
\begin{align}\label{eq:convex-opt}
\begin{aligned}
c_1^*,\cdots,c_K^* = & \underset{c_1,\cdots,c_K}{\arg\min} \sum_{i=1}^K \pi \sqrt{\lambda_{i,1}\lambda_{i,2}} c_i ,\\
& \mathrm{subject~to~} \sum_{i=1}^K p_i (1-\exp(-c_i/2)) \geq \tau , \\
& \phantom{\mathrm{subject~to~}} c_i \geq 0 \quad \text{for all } i = 1, \cdots, K.
\end{aligned}
\end{align}
where $\lambda_{i,1}$ and $\lambda_{i,2}$ are eigen-values of $\Sigma_i$.
\end{theorem}
%
The proof of this theorem is provided in the Appendix. The optimization in \eqref{eq:convex-opt} has a few key advantages. First, it computes the size of sub-level sets in accordance with the probabilities assigned to individual modes, resulting in ``tighter" sets than if we were to simply choose the $\tau$ probability sub-level sets for each mode of the GMM.
Second, unlike prior work \cite{tumu2024multi,lew2021sampling}, this approach does not require us to draw samples from the distribution; instead, it exploits the GMM structure to solve the optimization problem and compute the FRS in microseconds, significantly lowering the FRS inference time.
Finally, the solution to \eqref{eq:convex-opt} is invariant to scaling of the covariance matrix, as shown in the following corollary of Theorem~\ref{thm:convex-opt}.
This property will be instrumental in conformal calibration of the FRS.
%
\begin{corollary}[Invariance of \eqref{eq:convex-opt} to Covariance Scaling]\label{cor:scale-invariance}
The solution of the convex optimization \eqref{eq:convex-opt} established in Theorem~\ref{thm:convex-opt} is the same for any scaling $\alpha\in(0,\infty)$ of the GMM covariances $\{\Sigma_i\}_{i=1}^K$.
\end{corollary}
\subsection{Conformalizing FRS from Trajectory Predictors}
\label{subsec:CP}
%
% We need to obtain certain guarantees for our prediction sets. Until now, the guarantees provided by the Optimization process are directly translated from the predictor. However, the confidence levels rising form from such a predictor are often uncalibrated. Additionally these confidences are often relative to predictors own belief of its outputs trained to minimize the negative log likelihood of its training data.
%
% Idea is the the 0.9 probability is a calibrated measure of the uncertainty. We can now use a held-out calibration dataset computed offline to update the sublevel sets to the desired confidence level. To that end we use the standard
%
% Probabilistic trajectory prediction models are typically trained to minimize the negative log likelihood of the data under their output distribution. This encourages the output distributions to match the true conditional distribution reflected in the training dataset, but due to limited model capacity and a finite dataset, the resulting distribution may not match perfectly. This means that computing a set which contains, say, 90\% of the probability mass w.r.t. the predicted distribution may not contain the ground truth future with probability 0.9. To remedy this, we use conformal prediction to further update the sub-level sets that we extract by solving \eqref{eq:convex-opt}.
%
The FRS obtained from \eqref{eq:convex-opt} assumes the predictor’s distribution $\hat{\mu}_t$ closely matches the true transition dynamics. In reality, due to modeling choices and limited training data, $\hat{\mu}_t$ may fail to cover the true ground-truth distribution. We address this by applying split conformal prediction to calibrate the reachable set to achieve high-probability coverage of the ground truth future.
% The FRS that we compute by solving \eqref{eq:convex-opt} holds for the transition dynamics $\mu_t$ given by the trajectory predictor. However, due to modeling choices and limited training data, the distribution learned by the trajectory predictor may not exactly match the true ground-truth distribution. To address this, we employ split conformal prediction \citep{} which calibrates the stochastic transition dynamics $\mu_t$ to ensure that solving \eqref{eq:convex-opt} results in a set that covers the ground truth future with a high probability.
We calibrate our FRS by scaling the covariance of the GMM modes.
% To do so, we define a non-conformity score function equal to the amount by which we need to scale the covariances $\Sigma_i$ to ensure that the ground truth is just within the FRS we obtain by solving \eqref{eq:convex-opt}.
To do so, we define a non-conformity score function $\psi(s,x)$ as the smallest scaling factor $\alpha > 0$ such that the ground-truth $x$ lies inside the FRS computed using covariances ${ \alpha \Sigma_i }$:
%
\begin{equation}
\psi(s,x):=\inf\{\alpha~|~x\in E^*(\{\alpha\Sigma_i\}_{i=1}^K) \}
\end{equation}
%
where $s$ is the scene information as defined in Section~\ref{subsec:traj-pred} and $E^*(\{\Sigma_i\}_{i=1}^K)$ be the FRS obtained by solving \eqref{eq:convex-opt} for GMM covariances $\{\Sigma_i\}_{i=1}^K$. Thanks to Corollary~\ref{cor:scale-invariance}, we only need to solve \eqref{eq:convex-opt} once to obtain $c_i^*$, after which we can compute $\psi(s,x)$ analytically: $\psi(s,x) = \min\{V_i(x)/c_i^*~|~i=1,\cdots,K\}$ where $V_i$ is as defined in Section~\ref{subsec:convex-opt} for $\Sigma_i$ that are outputted by the trajectory predictor.
% Thanks to Corollary~\ref{cor:scale-invariance}, we can compute $\psi(s,x)$ efficiently via bisection on $\alpha$.
% \sbnote{@Sushant, @Apoorva: Does this description seem accurate for CP?}
% \svnote{It's actually even better than bisection, we can analytically compute this $\alpha$ by finding the level $c_i$ of the gt position in an unscaled GMM mode. Then we can take $\min_{i=1}^K c_i/c_i^*$ to find the alpha. I can add this later.}
% Overloading the notation, let $E^*(\{\Sigma_i\}_{i=1}^K)$ be the solution of \eqref{eq:convex-opt} for a particular GMM with mode covariances $\{\Sigma_i\}_{i=1}^K$.
% The non-conformity score $\psi:\mathcal{S}\times\mathcal{X} \to (0,\infty)$ is then defined as $\psi(s,x):=\inf\{\alpha~|~x\in E^*(\{\alpha\Sigma_i\}_{i=1}^K) \}$ where $s$ is the scene information as defined in Section~\ref{subsec:traj-pred} while $x$ is the ground truth future state of the agent that we are predicting for.
% \textcolor{red}{[Need to mention how Corollary~\ref{cor:scale-invariance} helps with computing the non-conformity score efficiently.]}
% Note that
Given $N$ calibration examples $\mathcal{D} := \{(s_j, x_j)\}_{j=1}^N$, if we let $\eta$ be the $(1 - \gamma)$ empirical quantile of the scores ${\psi(s_j, x_j)}$. Then, from \cite[Proposition~2a]{vovk2012conditional}, with probability $1 - \delta$:
% Applying split conformal prediction, if we choose a threshold $\tau$ as the $1-\hat{\alpha}$ empirical quantile of the nonconformity score of a set of $N$ calibration data points, then with probability $1-\delta$ over the sampling of the calibration data,
% %
% \begin{align}
% \Pr_{s,x}\left( x \notin C(s,x,\tau) \right) < \alpha - \sqrt{-\log\delta / 2 N},
% \end{align}
% %
% where the conformalized prediction set $C(s,x,\tau)$ is given by
% %
% \begin{align}
% C(s,x,\tau) &:= \{ x | \psi(s,x) < \tau \} \\
% &= \{ x | \inf \{ \alpha | x \in E^*(\{\alpha \Sigma_i\}_{i=1}^K) \} < \tau \} \\
% &= \bigcup_{0 < \alpha < \tau} E^*(\{\alpha \Sigma_i\}_{i=1}^K) \\
% &= E^*(\{\tau\Sigma_i\}_{i=1}^K)
% \end{align}
% %
%
\begin{align} \label{eqn:conformal_guarantees}
\Pr_{s,x}\left( x \notin C(s,\eta) \right) < \gamma - \sqrt{-\log\delta / 2 N},
\end{align}
%
where $C(s,\eta)$ is the conformalized FRS given by:
%
\begin{align}
C(s, \eta) &:= \{ x : \psi(s,x) < \eta \} \\
&= \{ x : \inf \{ \alpha | x \in E^*(\{\alpha \Sigma_i\}_{i=1}^K) \} < \eta \} \\
&= \bigcup_{0 < \alpha < \eta} E^*(\{\alpha \Sigma_i\}_{i=1}^K) \\
&= E^*(\{\eta\Sigma_i\}_{i=1}^K)
\end{align}
%
where the last line follows due to the monotonicity of sub-level sets:
% from the following lemma relating to the monotonicity of set inclusion as we scale the mode variances:
\begin{lemma}\label{lem:monotonic-set-inclusion}
Let $\alpha_1,\alpha_2 \in (0,\infty)$ and $\alpha_1<\alpha_2$, then $E^*(\{\alpha_1\Sigma_i\}_{i=1}^K) \subseteq E^*(\{\alpha_2\Sigma_i\}_{i=1}^K)$.
\end{lemma}
%
% [Need to show that for $\alpha_1<\alpha_2$, $E^*(\{\alpha_1\Sigma_i\}_{i=1}^K) \subseteq E^*(\{\alpha_2\Sigma_i\}_{i=1}^K)$]
%
Intuitively, \eqref{eqn:conformal_guarantees} states that the probability of ground truth future state lying outside the conformalized FRS $C(s, \eta)$ is very low.
% To ensure reliable safety guarantees from our prediction sets, it is essential that the underlying confidence levels are correctly calibrated. Until now, the guarantees produced by our optimization process have been directly inherited from the predictor’s output. However, the confidence scores produced by such predictors are often uncalibrated, reflecting the internal belief of the model rather than an accurate measure of uncertainty. In particular, probabilistic trajectory predictors are typically trained to minimize the negative log-likelihood of the training data. Yet, due to finite model capacity and limited training data, the learned distributions may deviate from the true data-generating process, which is often complex and impossible to model. As a result, the set extracted by solving \eqref{eq:convex-opt} may be unable to provide the desired coverage of the ground truth.
%
% To address this mismatch, we employ split conformal prediction \citep{}, which enables us to recalibrate the sub-level sets obtained from our optimization procedure. Using a held-out calibration dataset, conformal prediction adjusts these sets to guarantee the desired coverage level in a statistically valid manner, thereby providing a properly calibrated notion of uncertainty.
%
% In a general setting, one can compute the conformal scores by determining the distance between the true ground truth and the predicted state.
% However, the conformal scores between a state and its prediction are not IID for different states along a single trajectory since each state depends on the previous states. Hence, we compute the conformal score across different trajectories, which are now IID \kccom{cite lars' paper}.
%
% Existing works, however, require the explicit availability of trajectories in order to compute the conformity scores.
% In our method we propose in directly using the sets that we have obtained from the convex optimization\footnote{We could still have sampled points from the sets and used their underlying likelihood from the original predictions to obtain the trajectories.}.
%
% \input{algorithm_conformal}
\subsection{Hedging Against OOD Failures via Bayesian Filtering} \label{subsec:bayesian}
% The conformalization process in section \ref{subsec:CP} enables FORCEOPT to provide statistical guarantees on the coverage of its output sets. However, these guarantees are valid only under the assumption that test-time data is independent and identically distributed (IID) with respect to the calibration dataset. In practice, this assumption is often violated—particularly in autonomous vehicle (AV) applications, where the online test-time distribution may significantly deviate from the offline calibration distribution. Consequently, the theoretical guarantees offered by conformal prediction no longer hold in such out-of-distribution (OOD) settings.
The conformalization process in Section~\ref{subsec:CP} allows FORCE-OPT to provide statistical guarantees on the coverage of predicted reachable sets. However, these guarantees hold only under the assumption that test-time inputs are drawn independently and identically (IID) from the same distribution as the calibration data. In practice---especially in autonomous driving---this assumption is often violated, and distribution shift can cause these guarantees to break down.
% To address this mismatch between offline calibration and online deployment, we introduce a model confidence parameter , $\beta \in [\beta_{high}, \beta_{low}] \in \mathbb{R}$ that quantifies our trust in FORCEOPT's predictions. We use $\beta$ to scale the covariance of the conformalized GMMs adaptively: when the model's confidence is low, the covariance is increased, effectively dilating the predicted reachable sets to reflect greater uncertainty.
To address this challenge, we augment FORCE-OPT with a Bayesian filtering mechanism that dynamically adjusts uncertainty in the reachable set when the trajectory predictor appears unreliable. Specifically, we introduce a model confidence parameter $\beta \in [\beta_{\text{low}}, \beta_{\text{high}}]$ that reflects our trust in the predictor. This parameter scales the covariance of the GMMs used in the FRS adaptively, effectively dilating the predicted reachable sets to reflect greater uncertainty..
% As an example, when confidence is low (i.e., $\beta = \beta_{\text{low}}$), we inflate the covariances, effectively dilating the predicted reachable sets to reflect greater uncertainty.
% We adopt the Bayes filtering-based algorithm from \cite{nakamura2023online}, to update $\beta$ online based on the observed states of the AV.
% We initialize the confidence with a uniform distribution, i.e., $bel^0(\beta_{low}) = bel^0(\beta_{high}) = 0.5$, where $bel^\tau(\beta)$ denotes the belief that we have on our confidence parameter $\beta$ at time $\tau$.
To track this confidence online, we adopt the Bayesian update scheme proposed in \cite{nakamura2023online}.
At each timestep $t$, we maintain a belief distribution over $\beta$, denoted by $\text{bel}^t(\beta)$.
The belief is initialized uniformly: $bel^0(\beta_{low}) = bel^0(\beta_{high}) = 0.5$.
The belief is then updated at every timestep using the likelihood of the \textit{observed} agent state $x_t$ under the conformalized GMM, whose covariances are additionally scaled by the inverse of $\beta$:
% The update rule for the belief is given by,
\vspace{-3mm}
\small
\begin{equation}
\text{bel}^{t+1}(\beta) =
\frac{\varphi\bigg({x}_{t}; \text{GMM}\bigg(\{\bar{x}^{t}\}_{i=1}^{K}, \frac{1}{\beta} \{\eta\Sigma^{t}_i\}_{i=1}^{K}\bigg) \bigg) \text{bel}^{t}(\beta)}
{\sum_{\tilde{\beta}} \varphi\bigg({x}_{t}; \text{GMM}\bigg(\{\bar{x}^{t}\}_{i=1}^{K}, \frac{1}{\tilde\beta} \{\eta\Sigma^{t}_i\}_{i=1}^{K}\bigg) \bigg) \text{bel}^{t}(\tilde{\beta})},
\end{equation}
\normalsize
%
where $\tilde\beta \in \{\beta_{\text{low}}, \beta_{\text{high}}\}$. Here $\varphi(x, GMM(\cdot))$ denotes the likelihood of state $x$ under the given GMM.
Note that the GMM covariances are already scaled by the conformal calibration factor $\eta$; the inverse $\beta$ scaling further adjusts the uncertainty based on current trust in the predictor.
% Finally we obtain the realization of the confidence as the expected value, $\hat\beta = \mathbb{E}[\beta]$.
Finally, we compute the effective model confidence as the expected value under the belief: $\hat\beta = \mathbb{E}[\beta]$, and use this $\hat{\beta}$ to further scale the covariances of the distribution to adjust the final FRS returned by FORCE-OPT.
% \sbnote{@Kaustav: Could you write a final expression for the FRS with everything taken into account?}
%
% An interesting distinction of our method is the tracking of belief over multiple modes instead of the most likely mode used in \cite{nakamura2023online}. Such a multimodal belief update is essential for the handling of the modern predictors where each mode can acoound tfor differnet plaussible behaour of the predicted agnet leading to accurate prediction.
% \textbf{Parameterized FRS:} The set of all states the system can reach under the specific control bounds. We obtain the control bounds as the $3\sigma$ (or 99\% confidence interval) support of the control distribution returned by the predictor. The parameter refers to the mean and covariance of the control distribution returned by the predictor.
% \textbf{Worst Case FRS}: The set of all states that the system can reach under fixed control authority irrespective of the outputs of the underlying predictor.
% In both cases, we use the standard unicycle dynamics with maximum
% In summary, we combine conformal prediction to calibrate the GMMs for in-distribution generalization and a Bayesian belief update mechanism to handle online OOD scenarios by dynamically adjusting prediction uncertainty.
% However, it is important to note that conformal prediction offers guarantees only in the average case. It does not account for worst-case errors that may lie in the tail of the distribution. To capture such rare but potentially catastrophic failures, we leverage techniques from Hamilton-Jacobi (HJ) reachability analysis. In particular, when the online model confidence $\beta$ falls below a critical threshold, it serves as a trigger indicating that the predictor may be operating outside the distribution of the calibration data—making. This can be treated as situations where FORCEOPT might be susceptible to long-tailed failure modes. In this work, we implement two distinct reachable sets as a fallback when $\beta$ is lower than the desired threshold amount: Parameterized FRS and worst-case FRS. We will go in-depth into those methods when comparing different types of predictors in the result section.
% Overall the belief updates and switching setup enables FORCE-OPT to conservatively hedge in uncertain or out-of-distribution situations while retaining its tight conformal bounds in well-calibrated, in-distribution settings.
While conformal prediction provides statistical guarantees on coverage, these guarantees hold only in the average case. It does not account for rare but high-impact failures that may lie in the tails of the distribution. To address such worst-case scenarios, we incorporate techniques from Hamilton-Jacobi (HJ) reachability analysis. Specifically, when the online model confidence $\beta$ drops below a critical threshold, it indicates that the predictor may be operating outside the distribution represented by the calibration dataset. These are precisely the conditions under which FORCE-OPT becomes vulnerable to long-tailed failure modes.
To mitigate this risk, we employ two fallback mechanisms based on reachability theory: \textbf{(i)} a Parameterized FRS that adapts to observed uncertainty levels, and \textbf{(ii)} a Worst-Case FRS that assumes bounded adversarial disturbances. These fallback strategies are activated when $\beta$ falls below the specified threshold. We explore the behavior and trade-offs of these approaches in detail in the results section, particularly in the context of different predictor types.
The belief update mechanism and switching strategy allow FORCE-OPT to maintain tight, calibrated bounds in-distribution, while conservatively hedging against uncertainty in out-of-distribution or low-confidence scenarios.
\label{sec:result}
We now present an empirical evaluation of FORCEOPT to assess its effectiveness in safety-critical motion planning. Our experiments are designed to address the following key research questions: \qa{(Q1)} How effective is FORCE-OPT at balancing completeness and soundness of safety evaluation for autonomous driving? \qa{(Q2)} Do the belief-based extensions of FORCE-OPT result in more robust monitoring in out-of-distribution operation? \qa{(Q3)} How effectively can FORCE-OPT and its extensions leverage multimodality of trajectory prediction? \qa{(Q4)} Does FORCE-OPT offer a computational advantage that might enable online deployment?
% \begin{enumerate}
% \item \textbf{Overall Performance:} How does FORCEOPT perform across various safety and conservativeness metrics compared to baseline methods? Does the use of conformal prediction (CP) improve coverage, and does it result in overly conservative sets? \svnote{Vague, needs to be more precise, do we mean within distribution performance?}
% \item \textbf{Online Adaptation:} Can FORCEOPT adapt to online distribution shifts compared to variants that do not incorporate online correction?
% \kccom{Justiifcation forBelief based updates}
% \item \textbf{Robustness to Out-of-Distribution Inputs:} How well does FORCEOPT handle long-tailed or out-of-distribution (OOD) failure modes of the underlying predictor? Is the switching mechanism effective in such scenarios? \kccom{chance to highlight the timing of the FRS methods being too high and FORCEOPT + belief is quite sufficient. No need to compute controls!}
% \item \textbf{Multimodality Benefits:} What additional advantages does our method offer over unimodal forward reachable set (FRS) approaches? Specifically, how does the number of prediction modes affect performance?
% \end{enumerate}
\subsection{Datasets}
% We employ the full scale nuScenes and nuPlan-mini dataset. nuScenes contains around 15h of driving data in Boston and Singapore with challenging scenarios labelled by human experts. nuPlan-mini contains data sourced from 4 cities Las Vegas, Singapore, Pittsburgh, Boston. The diversity of the dataset allows us to demonstrate the versatility of our method.
To answer the above questions, we evaluate FORCE-OPT's performance on nuScenes \cite{nuscenes}, a large-scale autonomous driving dataset. The nuScenes dataset includes approximately 15 hours of expert-labeled driving data in Boston and Singapore.
% nuPlan-mini complements this with data from four cities (Las Vegas, Singapore, Pittsburgh, Boston), offering significant geographical diversity.
%
% Each dataset comes with its own training and test splits.
We train the Autobots predictor \cite{girgis2021latent} on the training split of Singapore and then test it on the test splits of both cities. This setup allows us to test the performance of our approach on in distribution (ID) where the model is trained and tested on the same city as well as out-of-distribution (OOD) where the model is tested on a city different from the one it is trained on.
% We chose to train in Singapore and test in Boston because the scenarios in Boston feature a higher number of agents, resulting in a more challenging and diverse test set for safety evaluation.
% : \textbf{(i)} nuScenes validation dataset from Boston \textbf{(ii)} nuScenes validation dataset from Singapore, and \textbf{(iii)} validation split of the nuPlan-mini dataset.
% This setup allows us to expose the trained predictor to diverse testing conditions that may not be well-represented in the training distribution.
% \kccom{Need to explicitly state how did we conclude which pairs are in and out of ditribution}
\subsection{Synthetic Unsafe Data Generation}
Although nuScenes provides diverse testing conditions, all the driving data available in the dataset is inherently safe, which makes it challenging to assess the monitor's ability to detect potential safety violations.
To remedy this, we synthetically generate unsafe scenarios by modifying scenes within the nuScenes dataset.
Specifically, we identify potential intersections between the trajectories of the ego vehicle and a surrounding agent, defining the point of closest approach $p_c$ as the collision point.
% The ego vehicle is assumed to reach this point at time step $t_e$, while the other vehicle arrives at $t_o$.
Let the ego and the contender arrive at $p_c$ at times $t_e$ and $t_o$, respectively.
% To generate a new trajectory for the ego vehicle, we apply a bicycle dynamics model and optimize the trajectory using the Sequential Least Squares Programming (SLSQP) algorithm from the SciPy library.
% The optimization is subject to constraints requiring the ego vehicle to start from its initial position at time step 0 and reach $p_c$ at $t_o$.
We use a bicycle dynamics model for the ego vehicle and optimize its trajectory using Sequential Least Squares Programming (SLSQP) to force it to reach $p_c$ at $t_o$, while obeying initial conditions and physical constraints.
The resulting trajectory of the ego vehicle, along with the original trajectory of the contender, constitutes a synthesized unsafe scenario.
This yields plausible but unsafe plans, which serve as ground truth for evaluating false negative rates.
\subsection{Metrics}
The primary objective of this paper is to assess the safety of motion plans. To that end, we focus on two broad categories of metrics: \textbf{completeness} and \textbf{soundness}. These metrics evaluate on a per-frame basis how effectively and reliably a safety assessment algorithm captures true safety violations while minimizing over-conservatism.
\begin{itemize}
\item \textbf{Completeness} is quantified using two key metrics: \begin{enumerate}
\item \textit{Coverage (Cov):}
% This measures whether the ground-truth trajectory lies within the computed reachable sets.
The fraction of ground-truth future trajectories that fall within the predicted reachable set.
High coverage indicates that the predicted FRS captures the actual future behavior well.
% High coverage indicates that the predicted FRS successfully encapsulate the actual future behavior, which is desirable for any safety-critical system.
\item \textit{False Negative Rate (FNR):}
% This quantifies the proportion of true collisions\footnote{Since standard real-world driving datasets typically contain no actual collisions, we construct ground-truth collisions by manually injecting synthetic collision scenarios into the dataset. Details of this augmentation process are provided in the appendix.} that were not flagged by the algorithm. A low FNR indicates high \textit{completeness} of the safety evaluator.
The proportion of true collision cases (from synthesized unsafe data) that are not flagged by the monitor.
% A low FNR reflects completeness.
\end{enumerate}
\item \textbf{Soundness} is evaluated using the \textit{False Positive Rate (FPR)}: The fraction of safe scenarios that are incorrectly flagged as unsafe. A low FPR indicates soundness, avoiding unnecessary interventions.
% or plan rejections.
% A false positive occurs when the algorithm incorrectly identifies a safe scenario as safety-critical. A high FP rate reflects an overly conservative system. Therefore, a low FP rate is essential for ensuring the \textit{soundness} of the algorithm, avoiding unnecessary interventions or plan rejections.
\item \textbf{Balance between Completeness and Soundness} is evaluated using the \emph{Balanced Error Rate (BER)}: Arithmetic mean of FPR and FNR effectively capturing the tradeoff between them.
% This metric highlights methods with strong balance between soundness and completeness. \sbnote{@Kaustav, Zeyuan: Please feel free to remove this metric if you are not reporting F1-score.}
% {AS: \color{red}[Can we clarify how we measure coverage rate and FNR/FPR for the adaptive methods? Are the labels on our datasets for FNR and FPR at the level of specific timesteps of the a full clip]}
\end{itemize}
\input{table_train_singapore_test_singapore}
\subsection{Baselines and Variants}
\label{subsec:baselines}
% We compare the above metrics on \textit{7 different} variants of the reachable set computation methods:
We categorize all methods in three broad categories:
\begin{itemize}
\item \textbf{Uncalibrated Trajectory Predictor:} The methods that fall under this class directly use the trajectory predictor without calibrating them. \textbf{99\% CI}: The original predictor where the sets occupy 99\% of GMM probability mass. $E = \bigcup_{i=1}^K E_i(C)$ where C = value at 99th Percentile of a $\chi^2$ distribution. \textbf{Parametric Worst Case-FRS (pWC-FRS)}: We obtain the control bounds as the $3\sigma$ (or 99\% confidence interval) support of the Gaussian control distribution for each mode predicted by a trajectory predictor and then estimate a worst-case FRS for each mode and take the union of the sets. (the parameter is the velocity and the control bound of he agent) \textbf{Nakamura et al. \cite{nakamura2023online}}: Adapts FRS via belief tracking of a trajectory predictor's performance. In this case the control bounds enclose the 3\% probaility mass around the mean of the normal distribution predicted by the trajectory predictor \cite{salzmann20}.
\item \textbf{Calibrated Trajectory Predictor:} The methods within this class leverage trajectory predictors that are calibrated using CP. We use a dataset with a cardinality of 35220 for calibration and set the desired coverage probability in CP to 0.95. \textbf{Lindemann et al.} \cite{lindemann2023safe}: Provides coverage guarantees via conformalization between the highest likelihood trajectory and the ground truth. \textbf{FORCE-OPT (Ours)}: Solves the convex optimization from Sec.~\ref{subsec:convex-opt} using GMMs from learned predictors, along with the calibration schemes in Sec.\ref{subsec:CP}. \textbf{FORCE-OPT + belief}: Additionally adjusts the GMM covariances of FORCE-OPT using Bayesian filtering approach in Sec. \ref{subsec:bayesian}. For this and all the subsequent methods that use belief-based adaptation, we set $\beta_{\rm low} = 0.3$ and $\beta_{\rm high} = 1$. \textbf{FORCE-OPT + pWC-FRS}: A hybrid approach that switches from FORCE-OPT to Parametric WC-FRS when $\beta<0.75$ indicating a drop in the predictor's performance. \textbf{FORCE-OPT + WC-FRS}: This approach switches from FORCE-OPT to worst-case FRS when $\beta<0.75$.
\item \textbf{Data-Free:} This category contains only one baseline metric \textbf{Worst Case FRS (WC-FRS)} which computes worst-case FRS assuming 4D Dubins vehicle dynamics with bounded control inputs.
\end{itemize}
% We compare FORCE-OPT to the following baselines:
% \textbf{Static FRS methods:} These approaches apply the same strategy to compute an FRS for each timestep along a trajectory.
% \textbf{99\% CI}: The original predictor where the sets occupy 99\% of GMM probability mass. $E = \bigcup_{i=1}^K E_i(C)$ where C = value at 99th Percentile of a $\chi^2$ distribution.
% \textbf{Parametric FRS}: Forward Reachable Set for a system under variable control bounds. We obtain the control bounds as the $3\sigma$ (or 99\% confidence interval) support of the control distribution returned by the predictor. The parameter refers to the mean and covariance of the control distribution returned by the predictor.
% \textbf{Worst Case FRS (WC-FRS)}:
% Computes conservative FRS assuming 4D Dubins vehicle dynamics with bounded control inputs.
% \textbf{FORCE-OPT (Ours)}:
% Solves the convex optimization from Sec.~\ref{subsec:convex-opt} using GMMs from learned predictors, along with the calibration schemes in Sec.\ref{subsec:CP}. \textbf{Lindemann et al.} \cite{lindemann2023safe}: Provides coverage guarantees via conformalization between a high likelihood trajectory and the ground truth.
% % \begin{itemize}[leftmargin=*]
% % \item \textbf{99\% CI}: The original predictor where the sets occupy 99\% of GMM probability mass.
% % $E = \bigcup_{i=1}^K E_i(C)$ where C = value at 99th Percentile of a $\chi^2$ distribution.
% % % with 2 degrees of freedom.
% % \item \textbf{Parametric FRS}:
% % Forward Reachable Set for a system under variable control bounds. We obtain the control bounds as the $3\sigma$ (or 99\% confidence interval) support of the control distribution returned by the predictor. The parameter refers to the mean and covariance of the control distribution returned by the predictor.
% % % \zfnote{the control distribution from FORCEOPT+belief. It's not a static FRS, basically Ken's method in a GMM setting, where the belief is based on the CP set and inflating the CP set. In Ken's method, belief is computed based on the raw predictor and used to inflate the raw prediction.}
% % % \sbnote{@Zeyuan: don't we first compute possible control bounds for the system and then use them to compute FRS? And are we really using a constant velocity system?}
% % \item \textbf{Worst Case FRS (WC-FRS)}:
% % % Forward Reachable Set for a 4D dubins car with fixed control bounds prioritizing guaranteed safety over performance.
% % Computes conservative FRS assuming 4D Dubins vehicle dynamics with bounded control inputs.
% % \item \textbf{FORCE-OPT (Ours)}:
% % % Our proposed method which takes a different probability mass per mode.
% % Solves the convex optimization from Sec.~\ref{subsec:convex-opt} using GMMs from learned predictors, along with the calibration schemes in Sec.\ref{subsec:CP}.
% % \end{itemize}
% \textbf{Adaptive FRS methods:} We also consider methods which adapt the strategy over the course of a clip of interaction using observations of a particular agent's behavior.
% \textbf{FORCE-OPT + belief (Ours)}:
% % FORCE-OPT over a base predictor with a scale on the GMM variances are adjusted via Bayesian filtering.
% Additionally adjusts the GMM covariances of FORCE-OPT using Bayesian filtering approach in Sec. \ref{subsec:bayesian}.
% \textbf{Nakamura et al. \cite{nakamura2023online}}:
% % A prior method \cite{nakamura2023online}, which computes the FRS with fixed control bounds.
% Adapts FRS via belief tracking but uses fixed control bounds .
% \textbf{Switch to FRS}:
% A hybrid approach that switches from FORCE-OPT to Parametric FRS when predictor uncertainty is high.
% \textbf{Switch to WC-FRS}: Same as above, but switches to the most conservative worst-case FRS.
% \begin{itemize}[leftmargin=*]
% \item \textbf{FORCE-OPT + belief (Ours)}:
% % FORCE-OPT over a base predictor with a scale on the GMM variances are adjusted via Bayesian filtering.
% Additionally adjusts the GMM covariances of FORCE-OPT using Bayesian filtering approach in Sec. \ref{subsec:bayesian}.
% \item \textbf{Nakamura et al. \cite{nakamura2023online}}:
% % A prior method \cite{nakamura2023online}, which computes the FRS with fixed control bounds.
% Adapts FRS via belief tracking but uses fixed control bounds .
% \item \textbf{Switch to FRS}:
% A hybrid approach that switches from FORCE-OPT to Parametric FRS when predictor uncertainty is high.
% \item \textbf{Switch to WC-FRS}: Same as above, but switches to the most conservative worst-case FRS.
% \item PLACEHOLDER for LARS ETAL
% \end{itemize}
\subsection{Results and Discussion}
% \sbnote{I have tried to write a results subsection, but I think it requires some work. In general, we want to first highlight that our method is maintaining a good safety and conservatism tradeoff (compared to baselines), as well as a rough intuition for why that is the case. We then want to highlight the utility of CP. Finally, we want to highlight the utility of belief update.}
The results for in-distribution evaluation of all approaches discussed in Section~\ref{subsec:baselines} are summarized in Table \ref{tab:method_comparison}, while those for OOD are summarized in Table~\ref{tab:method_comparison_OOD}.
\subsubsection{Balance between Completeness and Soundness \qa{(Q1)}}
\label{subsubsec:balance}
From Table~\ref{tab:method_comparison} we observe that FORCE-OPT achieves the lowest BER, indicating the best balance between FPR and FNR. The metrics that use CP-based calibration on the trajectory predictor have a significantly lower FNR at the expense of a slightly higher FPR than the ones that do not calibrate the predictor, generally indicating that the trajectory predictor without calibration tends to be over-optimistic in safety assessment missing out safety critical events, highlighting the importance of calibrating the trajectory predictor---one exception to this observation is the parametric FRS that counters the uncalibrated predictor's over-optimism by plugging the predicted control bounds to solve for the worst-case FRS. We also note that FORCE-OPT and its belief-based extensions, in comparison to \cite{lindemann2023safe}, have a lower FPR while maintaining a similar FNR and higher coverage rates. These additional gains are the outcome of the Theorem~\ref{thm:deter-prob-FRS}-inspired convex optimization \eqref{eq:convex-opt} that more precisely models the forward reachable space while leveraging multi-modality (unlike \cite{lindemann2023safe} which uses a single mode); this is discussed in greater detail in Section~\ref{subsubsec:multimodality}. FORCE-OPT has lower FPR and slightly worse FNR than its belief-based variants, which is in alignment with our expectations as the belief-based approaches introduce greater conservatism in the event when the predictor's performance deteriorates.
% Within distribution FORCE-OPT exhibits the best balance of FPR and FNR as indicated by the lowest BER. Other methods lean more towards better FPR at the detriment of FNR or vice-versa. For instance, worst-case FRS has 0 FNR, but a very high FPR of 51.85\%. On the other hand, \cite{nakamura2023online} has a lower FPR than FORCE-OPT but a higher FNR.
%
% Our method (FORCE-OPT) achieves the lowest FNR and a low FPR.
% This indicated that FORCE-OPT achieves the best trade-off between coverage and conservatives.
% Its highest F1 score among all methods validates its ability to balance completeness and soundness.
% Worst-case methods, while achieve the highest coverage, incur high false positives, making them overly conservative and impractical for real-world deployment.
% Additionally, directly using the learning-based predictor as is, leads to poor coverage as well as high false negatives, despite using a 99\% confidence interval. This indicates the importance of conformalizing the predictor outputs to calibrate the monitor with actual data, like in FORCE-OPT.
\input{table_train_boston_test_OOD}
\subsubsection{Out-of-distribution Robustness \qa{(Q2)}}
The belief-based adaptation mechanism enables smooth adjustment to uncertain contexts without excessive conservatism, as evidenced by the fact that the performance of the belief-based variants of FORCE-OPT remains similar between Tables~\ref{tab:method_comparison} and \ref{tab:method_comparison_OOD} and the best performing metric according to BER in Table~\ref{tab:method_comparison_OOD} is FORCE-OPT + pWC-FRS. Comparing the results in Table~\ref{tab:method_comparison} (ID) and Table~\ref{tab:method_comparison_OOD} (OOD), we also observe that the metrics that use an uncalibrated trajectory predictor suffer a drop in performance, especially with the FNR which jumps from 33.33\% and 36.36\% (ID) to 51.85\% and 55.56\% (OOD) for 99\% CI and \cite{nakamura2023online}, respectively---as before, parametric FRS is an exception to this for the same reason mentioned in Section~\ref{subsubsec:balance}. On the other hand, the approaches that use the calibrated trajectory predictor exhibit stronger OOD robustness; albeit, there is a modest increase in the FNR for all these approaches. Unsurprisingly, the performance of Worst-Case FRS is unaffected by the distribution shift.
% The belief-based adaptation mechanism in FORCE-OPT + belief enables smooth adjustment to uncertain contexts without excessive conservatism.
% The switching variants also offer additional robustness in challenging prediction regimes, although their FPR is slightly higher than FORCE-OPT with belief.
\subsubsection{Impact of Multi-modality \qa{(Q3)}}
\label{subsubsec:multimodality}
One of the key strengths of FORCE-OPT is its ability to systematically integrate probabilities from multiple prediction modes. To study the impact of multi-modality, we ablate the performance of the metrics in Section~\ref{subsec:baselines} that can handle multimodality, i.e., 99\% CI, parametric FRS, FORCE-OPT and its belief-based variants, on the Singapore dataset against the number of GMM modes. As the number of modes increase from one to five, we observe that the BER drops for all methods other than pWC-FRS Fig.~\ref{fig:num_modes_ber}. The drop in BER in different methods is fueled by different reasons: for 99\% CI, the BER improves because the FNR improves with more modes Fig.~\ref{fig:num_modes_fnr}, while for FORCE-OPT and its variants the BER improvement arises from an improvement in the FPR Fig.~\ref{fig:num_modes_fpr}. At first look, it is indeed surprising that more modes result in a lower FPR for FORCE-OPT; however, this counter-intuitive outcome is the result of the fact that greater multi-modality requires smaller set inflations via CP, as evidenced by the fact that the $\alpha$'s decrease as the number of modes increase, as shown in Table~\ref{tab:alphas}. If a mode other than the most-likely one is nearer to the ground truth in the calibration set, then the amount of set inflation needed to cover that ground-truth position would have to be less than the inflation needed for the most-likely mode that is further away. Overall, our ablations suggest that greater multi-modality promotes better safety assessment by allowing us to reason about multiple plausible future outcomes which could be closer to the ground truth behavior than whatever the model deems to be the most likely.
\begin{table}[h!]
\centering
\setlength{\tabcolsep}{2pt}
\small
\begin{tabular}{c|cccccc}
\toprule
\textbf{\# Modes} & $\mathbf{t=1}$ & $\mathbf{t=2}$ & $\mathbf{t=3}$ & $\mathbf{t=4}$ & $\mathbf{t=5}$ & $\mathbf{t=6}$ \\
\midrule
1 & 3.18 & 11.73 & 21.63 & 32.84 & 43.99 & 53.44 \\
2 & 2.23 & 7.72 & 14.51 & 23.36 & 32.3 & 41.43 \\
3 & 2.11 & 7.07 & 13.21 & 21.06 & 29.23 & 37.48 \\
4 & 2.04 & 6.45 & 11.67 & 18.69 & 25.55 & 33.65 \\
5 & 1.92 & 6.10 & 11.20 & 17.43 & 24.05 & 31.72 \\
\hline
\end{tabular}
\caption{\small{Calibrated $\alpha$ for each timestep along the predicted trajectory as the number of modes vary. The table shows that as the number of modes increase the amount of set inflation $\alpha$ needed for calibration reduces.}}
\label{tab:alphas}
\end{table}
\vspace{-1em}
% FORCE-OPT maintains low FPR as the number of modes increases, while its coverage improves. By contrast, baseline methods suffer from either high FPR (due to over-conservatism) or poor scalability.
% \kccom{introduce the qualitative images where unimodal methods fail due to over-reliance of one mode. Note for the switch, The modes do not really seem to matter since when in distribution, the modes are similar to an unimodal prediction, while in online-out of ODD, the FRS takes over. COnincidentally it is during this out-of-ODD region that the predictor behaves like a multimodal but is overridden by the FRS.}
%
% \begin{figure}[h!]
% \centering
% \includegraphics[width=0.8\textwidth]{figures/num_modes.png}
% \caption{Effect of number of modes on \textbf{Coverage (Left)} and \textbf{FPR (Right)}}
% \label{fig:num_modes}
% \end{figure}
%
\begin{figure*}[t]
\centering
\begin{subfigure}[b]{0.33\textwidth}
\includegraphics[width=\textwidth]{figures/singapore_mode_ablation_BER.png}
\caption{\small{BER vs. Number of Modes}}
\label{fig:num_modes_ber}
\end{subfigure}
\hspace{-5mm}
\hfill
\begin{subfigure}[b]{0.33\textwidth}
\includegraphics[width=\textwidth]{figures/singapore_mode_ablation_fp.png}
\caption{\small{FPR vs. Number of Modes}}
\label{fig:num_modes_fpr}
\end{subfigure}
\hspace{-5mm}
\hfill
\begin{subfigure}[b]{0.33\textwidth}
\includegraphics[width=\textwidth]{figures/singapore_mode_ablation_fn.png}
\caption{\small{FNR vs. Number of Modes}}
\label{fig:num_modes_fnr}
\end{subfigure}
\caption{\small{Ablation with different number of GMM modes from the trajectory predictor. The performance of FORCE-OPT and its belief-based variants improves with more GMM modes.}}
\label{fig:num_modes}
\end{figure*}
% \textit{Effect of number of timesteps.} Figure~\ref{fig:num_times} shows performance degradation as prediction horizon increases. As expected, coverage declines and FPR rises across all methods, since predicting farther into the future introduces more uncertainty, reducing the ability to confidently cover ground-truth outcomes.
% Notably, FORCE-OPT + belief and switching strategies show strong robustness, maintaining better balance across both metrics.
% %
% \begin{figure}[h!]
% \centering
% \includegraphics[width=0.8\textwidth]{figures/num_times.png}
% \caption{Effect of number of timesteps on \textbf{Coverage (Left)} and \textbf{FPR (Right)}}
% \label{fig:num_times}
% \end{figure}
%
% \begin{itemize}
% % \item One of the key strengths of our approach is its ability to systematically integrate probabilities from multiple prediction modes. To illustrate this, we plot the coverage and false positive rate as functions of the number of predicted modes in fig. \ref{fig:num_modes}. In general, we observe that coverage increases with the number of modes, while the false positive rate either decreases or remains stable.
% % \begin{figure}[t]
% % \centering
% % \includegraphics[width=0.8\textwidth]{figures/num_modes.png}
% % \caption{Effect of number of modes on \textbf{Coverage (Left)} and \textbf{FPR (Right)}}
% % \label{fig:num_modes}
% % \end{figure}
% %
% \item
% \sbnote{This content should be distributed to either the results subsection or the ablation study subsection.} Another interesting feature of FORCEOPT was the abilty to adjust its predictions based on its prediction error via \textit{online belief computation}. Additionally if the belief was two low in the current predictor, we \textit{switch over to a more conservative predictor}. To further probe the effect of this feature we plot the coverage and the FPR as functions of the number of predicted modes in fig. \ref{fig:num_times}. We observe that for all methods, coverage decreases and FPR increases as the number of steps increases. This is expected, as predicting farther into the future introduces more uncertainty, reducing the ability to confidently cover ground-truth outcomes.
% Across methods we see that even thought the coverage of the Baseline prediction is high, it suffers from high FPR, whereas the FORCEOPT with switching mechanisms (or simply belief) maintain a nice tradeoff between FNR and coverage. The vanilla FORCEOPT without belief method enjoys low FPR but at the cost of reduced coverage.
% However \sbnote{I have tried to explain this in the number of timesteps ablation study but I am not sure how we are connecting the number of timesteps with the utility of belief updates.}
% \end{itemize}
% In general our method can be seen to obtain a nice balance between the performance and conservativeness. Note that the FNR for our method being 0 justifies its usage as a valid safety accessor.
\subsubsection{Compuational Efficiency \qa{(Q4)}}
% \kccom{refine}
In Table \ref{tab:timing}, we show the computation time for all the methods presented in Section~\ref{subsec:baselines} along with their performance on BER in Tables~\ref{tab:method_comparison} and \ref{tab:method_comparison_OOD}. FORCE-OPT demonstrates fast runtimes while achieving strong BER results in both ID and OOD settings, outperforming faster baselines such as 99\% CI and \cite{lindemann2023safe}. Notably, adding belief tracking adds negligible overhead—FORCE-OPT + belief is only 0.001 seconds slower than FORCE-OPT alone. With the exception of FORCE-OPT + pWC-FRS, all other variants of FORCE-OPT are faster than 0.1 seconds, suggesting that these algorithms are well-suited for deployment in real-time, safety-critical applications.
% This impressive performance confirms that FORCE-OPT is well-suited for deployment in real-time, safety-critical applications. The efficient implementation and transparent formulation shows the potential of FORCEOPT as an online safety monitor especially when deployed on real hardware stacks; which can be an excellent future work.
% \sbnote{We should really mention something about the computation time of these methods if possible.}
\input{table_timing}
% \begin{table}[h!]
% \centering
% % \renewcommand{\arraystretch}{0.95}
% % \setlength{\tabcolsep}{2pt}
% \small
% \resizebox{1.0\textwidth}{!}{
% \begin{tabular}{c|ccccccccc}
% \toprule
% \textbf{Metric} & 99\% & FORCE-OPT & Parametric & Worst Case & FORCE-OPT & Nakamura & Switch to & Switch to & Lindemann\\
% ~& CI & (ours) & FRS & FRS & + belief (ours) & et al \cite{nakamura2023online} & FRS (ours) & WC-FRS (ours) & et al. \cite{lindemann2023safe}\\
% \midrule
% \textbf{Cov} $(\uparrow)$ & 57.96\% & 89.74\% & 96.00\% & \textbf{99.53\%} & 93.93\% & 87.30\% & 92.38\% & 94.92\% &86.58\% \\
% \textbf{Cov Quan.} $(\uparrow)$ & 39.01\% & 80.43\% & 89.17\% & \textbf{98.48\%} & 88.63\% & 70.85\% & 84.00\% & 90.71\% &64.38\% \\
% \textbf{FPR} $(\downarrow)$ & \textbf{1.19\%} & 10.01\% & 20.41\% & 44.58\% & 19.09\% & 8.57\% & 12.26\% & 22.75\% & 29.93\% \\
% \textbf{FNR} $(\downarrow)$ & 14.29\% & \textbf{0\%} & \textbf{0\%} & \textbf{0\%} & \textbf{0\%} & 14.29\% & \textbf{0\%} & \textbf{0\%} & 4.76\% \\
% \textbf{BER} $(\downarrow)$ & 7.74\% & \textbf{5.00\%} & 10.21\% & 22.29\% & 9.55\% & 10.83\% & 6.13\% & 11.38\% & 15.25\% \\
% \bottomrule
% \end{tabular}
% }
% \caption{Performance comparison across methods (best-performing values in bold) within distribution.\svnote{Stale numbers, need changing}}
% \label{tab:method_comparison}
% \end{table}
% \begin{table}[h!]
% \centering
% % \renewcommand{\arraystretch}{0.95}
% \setlength{\tabcolsep}{2pt}
% \small
% \resizebox{1.0\textwidth}{!}{
% \begin{tabular}{c|cc|cccccc|c}
% \toprule
% \textbf{Approach} & \multicolumn{2}{c|}{\textbf{Trajectory Predictor}} & \multicolumn{6}{c|}{\textbf{Trajectory Predictor + Reachability}} & \textbf{Reachability} \\
% \midrule
% \textbf{Metric} & 99\% & Lindemann & Parametric & Nakamura & FORCE-OPT & FORCE-OPT & Switch to & Switch to & Worst Case \\
% ~& CI & et al. \cite{lindemann2023safe} & FRS & et al. \cite{nakamura2023online} & (ours) & + belief (ours) & FRS (ours) & WC-FRS (ours) & FRS\\
% \midrule
% \textbf{Cov} $(\uparrow)$ & 57.96\% & 86.58\% & 96.00\% & 87.30\% & 89.74\% & 93.93\% & 92.38\% & 94.92\% & \textbf{99.53\%} \\
% \textbf{Cov Quan.} $(\uparrow)$ & 39.01\% & 64.38\% & 89.17\% & 70.85\% & 80.43\% & 88.63\% & 84.00\% & 90.71\% & \textbf{98.48\%} \\
% \textbf{FPR} $(\downarrow)$ & \textbf{1.19\%} & 29.93\% & 20.41\% & 8.57\% & 10.01\% & 19.09\% & 12.26\% & 22.75\% & 44.58\% \\
% \textbf{FNR} $(\downarrow)$ & 14.29\% & 4.76\% & \textbf{0\%} & 14.29\% & \textbf{0\%} & \textbf{0\%} & \textbf{0\%} & \textbf{0\%} & \textbf{0\%} \\
% \textbf{BER} $(\downarrow)$ & 7.74\% & 15.25\% & 10.21\% & 10.83\% & \textbf{5.00\%} & 9.55\% & 6.13\% & 11.38\% & 22.29\% \\
% \bottomrule
% \end{tabular}
% }
% \caption{Performance comparison across methods (best-performing values in bold) within distribution.}
% \label{tab:method_comparison}
% \end{table}
% \begin{table}[h!]
% \centering
% % \renewcommand{\arraystretch}{0.95}
% \setlength{\tabcolsep}{2pt}
% \small
% \resizebox{1.0\textwidth}{!}{
% \begin{tabular}{c|ccc|ccccc|c}
% \toprule
% \textbf{Approach} & \multicolumn{3}{c|}{\textbf{Uncalibrated Trajectory Predictor}} & \multicolumn{5}{c|}{\textbf{Calibrated Trajectory Predictor with Conformal Prediction}} & \textbf{Data-free} \\
% \midrule
% \textbf{Metric} & 99\% & Parametric & Nakamura & Lindemann & FORCE-OPT & FORCE-OPT & Switch to & Switch to & Worst Case \\
% ~& CI & FRS & et al. \cite{nakamura2023online} & et al. \cite{lindemann2023safe} & (ours) & + belief (ours) & FRS (ours) & WC-FRS (ours) & FRS\\
% \midrule
% \textbf{Cov} $(\uparrow)$ & 57.96\% & 96.00\% & 87.30\% & 86.58\% & 89.74\% & 93.93\% & 92.38\% & 94.92\% & \textbf{99.53\%} \\
% % \textbf{Cov Quan.} $(\uparrow)$ & 39.01\% & 89.17\% & 70.85\% & 64.38\% & 80.43\% & 88.63\% & 84.00\% & 90.71\% & \textbf{98.48\%} \\
% \textbf{FPR} $(\downarrow)$ & \textbf{1.19\%} & 20.41\% & 8.57\% & 29.93\% & 10.01\% & 19.09\% & 12.26\% & 22.75\% & 44.58\% \\
% \textbf{FNR} $(\downarrow)$ & 14.29\% & \textbf{0\%} & 14.29\% & 4.76\% & \textbf{0\%} & \textbf{0\%} & \textbf{0\%} & \textbf{0\%} & \textbf{0\%} \\
% \textbf{BER} $(\downarrow)$ & 7.74\% & 10.21\% & 10.83\% & 15.25\% & \textbf{5.00\%} & 9.55\% & 6.13\% & 11.38\% & 22.29\% \\
% % \textbf{F2-Score} $(\uparrow)$ & 0.8802 & 0.9608 & 0.8670 & 0.9067 & \textbf{0.9804} & 0.9632 & 0.9761 & 0.9565 & 0.9181 \\
% \bottomrule
% \end{tabular}
% }
% \caption{Performance comparison across methods (best-performing values in bold) within distribution.}
% \label{tab:method_comparison}
% \end{table}
\begin{table*}[ht]
\centering
% \renewcommand{\arraystretch}{0.95}
\setlength{\tabcolsep}{2pt}
\small
\resizebox{1.0\textwidth}{!}{
\begin{tabular}{c|ccc|ccccc|c}
\toprule
\textbf{Approach} & \multicolumn{3}{c|}{\textbf{Uncalibrated Trajectory Predictor}} & \multicolumn{5}{c|}{\textbf{Calibrated Trajectory Predictor with Conformal Prediction}} & \textbf{Data-free} \\
\midrule
\textbf{Metric} & 99\% & Parametric & Nakamura & Lindemann & FORCE-OPT & FORCE-OPT+ & FORCE-OPT+ & FORCE-OPT+ & Worst Case \\
~& CI & WC-FRS & et al. \cite{nakamura2023online} & et al. \cite{lindemann2023safe} & (ours) & belief (ours) & pWC-FRS (ours) & WC-FRS (ours) & FRS\\
\midrule
\textbf{Cov} $(\uparrow)$ & 54.76\% & 97.40\% & 86.84\% & 89.50\% & 89.95\% & 93.94\% & 94.90\% & 96.01\% & \textbf{99.53\%} \\
% \textbf{Cov Quan.} $(\uparrow)$ & 39.01\% & 89.17\% & 70.85\% & 64.38\% & 80.43\% & 88.63\% & 84.00\% & 90.71\% & \textbf{98.48\%} \\
\textbf{FPR} $(\downarrow)$ & \textbf{1.19\%} & 19.68\% & 7.33\% & 31.12\% & 8.62\% & 15.57\% & 13.35\% & 24.43\% & 43.73\% \\
\textbf{FNR} $(\downarrow)$ & 33.33\% & \textbf{0\%} & 36.36\% & \textbf{0\%} & 3.03\% & \textbf{0\%} & \textbf{0\%} & \textbf{0\%} & \textbf{0\%} \\
\textbf{BER} $(\downarrow)$ & 17.26\% & 9.84\% & 20.84\% & 15.56\% & \textbf{5.83\%} & 7.78\% & 6.78\% & 12.21\% & 21.86\% \\
% \textbf{F2-Score} $(\uparrow)$ & 0.8802 & 0.9608 & 0.8670 & 0.9067 & \textbf{0.9804} & 0.9632 & 0.9761 & 0.9565 & 0.9181 \\
\bottomrule
\end{tabular}
}
\caption{\small{Performance comparison across methods (best-performing values in bold) in Singapore (ID).}}
\label{tab:method_comparison}
\end{table*}
\vspace{-1em}
% \begin{table}[h!]
% \centering
% % \renewcommand{\arraystretch}{0.95}
% \setlength{\tabcolsep}{2pt}
% \small
% \resizebox{1.0\textwidth}{!}{
% \begin{tabular}{c|ccccccccc}
% \toprule
% \textbf{Metric} & 99\% & FORCE-OPT & Parametric & Worst Case & FORCE-OPT & Nakamura & Switch to & Switch to & Lindemann\\
% ~& CI & (ours) & FRS & FRS & + belief (ours) & et al \cite{nakamura2023online} & FRS (ours) & WC-FRS (ours) & et al. \cite{lindemann2023safe}\\
% \midrule
% \textbf{Cov} $(\uparrow)$ & 62.85\% & 90.06\% & 96.59\% & \textbf{99.46\%} & 94.02\% & 88.70\% & 93.39\% & 95.11\% & 88.02\%\\
% \textbf{Cov Quan.} $(\uparrow)$ & 40.31\% & 81.44\% & 91.13\% & \textbf{98.46\%} & 88.87\% & 79.81\% & 86.20\% & 90.78\% &78.55\% \\
% \textbf{FPR}$(\downarrow)$ & \textbf{1.18\%} & 5.25\% & 16.66\% & 45.67\% & 11.84\% & 8.99\% & 8.52\% & 18.90\% & 36.38\%\\
% \textbf{FNR}$(\downarrow)$ & 42.86\% & 5.71\% & 5.71\% & \textbf{0\%} & 2.86\% & 34.30\% & 5.71\% & \textbf{0\%} & 2.86\%\\
% \textbf{BER} $(\downarrow)$ & 22.02\% & \textbf{5.48\%} & 11.19\% & 22.84\% & 7.35\% & 20.74\% & 7.12\% & 9.45\% & 19.62\%\\
% \bottomrule
% \end{tabular}
% }
% \caption{Performance comparison across methods on the OOD dataset (best-performing values in bold).\zfnote{We mine the OOD scenes from nuScenes-Singapore dataset where the predictor has a high final displacement error.}}
% \label{tab:method_comparison_OOD}
% \end{table}
% \begin{table}[h!]
% \centering
% % \renewcommand{\arraystretch}{0.95}
% \setlength{\tabcolsep}{2pt}
% \small
% \resizebox{1.0\textwidth}{!}{
% \begin{tabular}{c|cc|cccccc|c}
% \toprule
% \textbf{Approach} & \multicolumn{2}{c|}{\textbf{Trajectory Predictor}} & \multicolumn{6}{c|}{\textbf{Trajectory Predictor + Reachability}} & \textbf{Reachability} \\
% \midrule
% \textbf{Metric} & 99\% CI & Lindemann & Parametric & Nakamura & FORCE-OPT & FORCE-OPT & Switch to & Switch to & Worst Case \\
% ~& & et al. \cite{lindemann2023safe} & FRS & et al. \cite{nakamura2023online} & (ours) & + belief (ours) & FRS (ours) & WC-FRS (ours) & FRS\\
% \midrule
% \textbf{Cov} $(\uparrow)$ & 62.85\% & 88.02\% & 96.59\% & 88.70\% & 90.06\% & 94.02\% & 93.39\% & 95.11\% & \textbf{99.46\%} \\
% \textbf{Cov Quan.} $(\uparrow)$ & 40.31\% & 78.55\% & 91.13\% & 79.81\% & 81.44\% & 88.87\% & 86.20\% & 90.78\% & \textbf{98.46\%} \\
% \textbf{FPR} $(\downarrow)$ & \textbf{1.18\%} & 36.38\% & 16.66\% & 8.99\% & 5.25\% & 11.84\% & 8.52\% & 18.90\% & 45.67\% \\
% \textbf{FNR} $(\downarrow)$ & 42.86\% & 2.86\% & 5.71\% & 34.30\% & 5.71\% & 2.86\% & 5.71\% & \textbf{0\%} & \textbf{0\%} \\
% \textbf{BER} $(\downarrow)$ & 22.02\% & 19.62\% & 11.19\% & 20.74\% & \textbf{5.48\%} & 7.35\% & 7.12\% & 9.45\% & 22.84\% \\
% \bottomrule
% \end{tabular}
% }
% \caption{Performance comparison across methods on the OOD dataset (best-performing values in bold).}
% \label{tab:method_comparison_OOD}
% \end{table}
% \begin{table}[h!]
% \centering
% % \renewcommand{\arraystretch}{0.95}
% \setlength{\tabcolsep}{2pt}
% \small
% \resizebox{1.0\textwidth}{!}{
% \begin{tabular}{c|ccc|ccccc|c}
% \toprule
% \textbf{Approach} & \multicolumn{3}{c|}{\textbf{Uncalibrated Trajectory Predictor}} & \multicolumn{5}{c|}{\textbf{Calibrated Trajectory Predictor with Conformal Prediction}} & \textbf{Data-free} \\
% \midrule
% \textbf{Metric} & 99\% CI & Parametric & Nakamura & Lindemann & FORCE-OPT & FORCE-OPT & Switch to & Switch to & Worst Case \\
% ~& & FRS & et al. \cite{nakamura2023online} & et al. \cite{lindemann2023safe} & (ours) & + belief (ours) & FRS (ours) & WC-FRS (ours) & FRS\\
% \midrule
% \textbf{Cov} $(\uparrow)$ & 62.85\% & 96.59\% & 88.70\% & 88.02\% & 90.06\% & 94.02\% & 93.39\% & 95.11\% & \textbf{99.46\%} \\
% % \textbf{Cov Quan.} $(\uparrow)$ & 40.31\% & 91.13\% & 79.81\% & 78.55\% & 81.44\% & 88.87\% & 86.20\% & 90.78\% & \textbf{98.46\%} \\
% \textbf{FPR} $(\downarrow)$ & \textbf{1.18\%} & 16.66\% & 8.99\% & 36.38\% & 5.25\% & 11.84\% & 8.52\% & 18.90\% & 45.67\% \\
% \textbf{FNR} $(\downarrow)$ & 42.86\% & 5.71\% & 34.30\% & 2.86\% & 5.71\% & 2.86\% & 5.71\% & \textbf{0\%} & \textbf{0\%} \\
% \textbf{BER} $(\downarrow)$ & 22.02\% & 11.19\% & 20.74\% & 19.62\% & \textbf{5.48\%} & 7.35\% & 7.12\% & 9.45\% & 22.84\% \\
% % \textbf{F2-Score} $(\uparrow)$ & 0.1239 & 0.5988 & 0.3337 & 0.8395 & 0.5988 & 0.8395 & 0.5988 & \textbf{1.0000} & \textbf{1.0000} \\
% \bottomrule
% \end{tabular}
% }
% \caption{Performance comparison across methods on the OOD dataset (best-performing values in bold).}
% \label{tab:method_comparison_OOD}
% \end{table}
\begin{table*}[ht]
\centering
% \renewcommand{\arraystretch}{0.95}
\setlength{\tabcolsep}{2pt}
\small
\resizebox{1.0\textwidth}{!}{
\begin{tabular}{c|ccc|ccccc|c}
\toprule
\textbf{Approach} & \multicolumn{3}{c|}{\textbf{Uncalibrated Trajectory Predictor}} & \multicolumn{5}{c|}{\textbf{Calibrated Trajectory Predictor with Conformal Prediction}} & \textbf{Data-free} \\
\midrule
\textbf{Metric} & 99\% & Parametric & Nakamura & Lindemann & FORCE-OPT & FORCE-OPT+ & FORCE-OPT+ & FORCE-OPT+ & Worst Case \\
~& CI & WC-FRS & et al. \cite{nakamura2023online} & et al. \cite{lindemann2023safe} & (ours) & belief (ours) & pWC-FRS (ours) & WC-FRS (ours) & FRS\\
\midrule
\textbf{Cov} $(\uparrow)$ & 62.85\% & 97.45\% & 88.99\% & 88.02\% & 90.06\% & 93.57\% & 95.07\% & 96.06\% & \textbf{99.46\%} \\
% \textbf{Cov Quan.} $(\uparrow)$ & 40.31\% & 91.13\% & 79.81\% & 78.55\% & 81.44\% & 88.87\% & 86.20\% & 90.78\% & \textbf{98.46\%} \\
\textbf{FPR} $(\downarrow)$ & \textbf{1.18\%} &17.08\% & 7.27\% & 36.38\% & 5.22\% & 9.16\% & 9.85\% & 21.15\% & 45.42\% \\
\textbf{FNR} $(\downarrow)$ & 51.85\% & 3.70\% & 55.56\% & 9.26\% & 12.96\% & 7.41\% & 3.70\% & \textbf{0\%} & \textbf{0\%} \\
\textbf{BER} $(\downarrow)$ & 26.52\% & 10.39\% & 31.41\% & 22.79\% & 9.09\%& 8.28\% & \textbf{6.77}\% & 10.57\% & 22.71\% \\
% \textbf{F2-Score} $(\uparrow)$ & 0.1239 & 0.5988 & 0.3337 & 0.8395 & 0.5988 & 0.8395 & 0.5988 & \textbf{1.0000} & \textbf{1.0000} \\
\bottomrule
\end{tabular}
}
\caption{\small{Performance comparison across methods (best-performing values in bold) in Boston (OOD).}}
\label{tab:method_comparison_OOD}
\end{table*}
\vspace{-1em}
\begin{table}[ht]
\centering
\setlength{\tabcolsep}{2pt}
% \small
\resizebox{0.95\columnwidth}{!}{
\begin{tabular}{c c c c}
\toprule
\textbf{Method} & \textbf{Time (s)} & \textbf{ID BER} & \textbf{OOD BER} \\
\midrule
99\% CI & \textbf{0.001} & 17.26\% & 26.52\% \\
pWC-FRS & 0.187 & 9.84\% & 10.39\% \\
Nakamura et al. \cite{nakamura2023online} & 0.155 & 20.84\% & 31.41\% \\
Lindemann et al. \cite{lindemann2023safe} & 0.005 & 15.56\% & 22.79\% \\
FORCE-OPT (ours) & 0.022 & \textbf{5.83\%} & 9.09\% \\
FORCE-OPT + belief (ours) & 0.023 & 7.78\% & 8.28\% \\
FORCE-OPT + pWC-FRS (ours) & 0.210 & 6.78\% & \textbf{6.77\%} \\
FORCE-OPT + WC-FRS (ours) & 0.079 & 12.21\% & 10.57\% \\
WC-FRS & 0.056 & 21.86\% & 22.71\% \\
\bottomrule
\end{tabular}
}
\caption{Computation Time for Different FRS Methods}
% \kccom{add this to the horizontal table and normalize wrt to one method}}
\normalsize
\label{tab:timing}
\end{table}
\vspace{-1em}
This paper introduced FORCE-OPT, a principled framework for evaluating the safety of motion plans using trajectory predictors as estimators of forward reachable sets. By combining convex optimization, conformal prediction, and Bayesian filtering, our method generates calibrated uncertainty sets that balance completeness (low false negatives) with soundness (low false positives). Empirical results on nuScenes demonstrate that FORCE-OPT significantly outperforms both conservative model-based and raw learning-based baselines, while gracefully handling out-of-distribution scenarios. We believe FORCE-OPT offers a promising building block for runtime safety monitoring in learned autonomy stacks.
4This work opens up several directions for exploration: (i) While the trajectory predictor conditions on scene context, FORCE-OPT itself operates independently for each agent when computing FRS. Joint multi-agent reachability, especially in dense traffic scenarios with interdependent behaviors, remains an open direction. (ii) Trajectory predictors are trained to distributionally mimic the observed data, not to facilitate the extraction of FRS. Training a neural FRS generator that directly outputs sets is another exciting open direction.
% Generated by IEEEtran.bst, version: 1.14 (2015/08/26)
\begin{thebibliography}{10}
\providecommand{\url}[1]{#1}
\csname url@samestyle\endcsname
\providecommand{\newblock}{\relax}
\providecommand{\bibinfo}[2]{#2}
\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax}
\providecommand{\BIBentryALTinterwordstretchfactor}{4}
\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus
\BIBentryALTinterwordstretchfactor\fontdimen3\font minus \fontdimen4\font\relax}
\providecommand{\BIBforeignlanguage}[2]{{%
\expandafter\ifx\csname l@#1\endcsname\relax
\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%
\typeout{** loaded for the language `#1'. Using the pattern for}%
\typeout{** the default language instead.}%
\else
\language=\csname l@#1\endcsname
\fi
#2}}
\providecommand{\BIBdecl}{\relax}
\BIBdecl
\bibitem{teng2023motion}
S.~Teng \emph{et~al.}, ``Motion planning for autonomous driving: The state of the art and future perspectives,'' \emph{TIV}, vol.~8, no.~6, pp. 3692--3711, 2023.
\bibitem{schwarting2018planning}
W.~Schwarting \emph{et~al.}, ``Planning and decision-making for autonomous vehicles,'' \emph{Annual Review of CRAS}, vol.~1, no.~1, pp. 187--210, 2018.
\bibitem{salzmann2020trajectron++}
T.~Salzmann \emph{et~al.}, ``Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data,'' in \emph{ECCV}.\hskip 1em plus 0.5em minus 0.4em\relax Springer, 2020, pp. 683--700.
\bibitem{girgis2021latent}
R.~Girgis \emph{et~al.}, ``Latent variable sequential set transformers for joint multi-agent motion prediction,'' \emph{arXiv preprint arXiv:2104.00563}, 2021.
\bibitem{angelopoulos2021gentle}
A.~N. Angelopoulos and S.~Bates, ``A gentle introduction to conformal prediction and distribution-free uncertainty quantification,'' \emph{arXiv preprint arXiv:2107.07511}, 2021.
\bibitem{shafer2008tutorial}
G.~Shafer and V.~Vovk, ``A tutorial on conformal prediction.'' \emph{JMLR}, vol.~9, no.~3, 2008.
\bibitem{nuscenes}
H.~Caesar \emph{et~al.}, ``nuscenes: A multimodal dataset for autonomous driving,'' 2020, pp. 11\,621--11\,631.
\bibitem{luckcuck2019formal}
M.~Luckcuck \emph{et~al.}, ``Formal specification and verification of autonomous robotic systems: A survey,'' \emph{CSUR}, vol.~52, no.~5, pp. 1--41, 2019.
\bibitem{mitra2024formal}
S.~Mitra \emph{et~al.}, ``Formal verification techniques for vision-based autonomous systems--a survey,'' in \emph{Principles of Verification: Cycling the Probabilistic Landscape: Essays Dedicated to Joost-Pieter Katoen on the Occasion of His 60th Birthday, Part III}.\hskip 1em plus 0.5em minus 0.4em\relax Springer, 2024, pp. 89--108.
\bibitem{althoff2011zonotope}
M.~Althoff and J.~M. Dolan, ``Reachability analysis of nonlinear systems with uncertain parameters using conservative linearization,'' \emph{CDC}, 2011.
\bibitem{lygeros1999controllers}
J.~Lygeros \emph{et~al.}, ``Controllers for reachability specifications for hybrid systems,'' \emph{Automatica}, vol.~35, no.~3, pp. 349--370, 1999.
\bibitem{mitchell2005toolbox}
I.~M. Mitchell \emph{et~al.}, ``A toolbox of hamilton–jacobi solvers for analysis of nondeterministic continuous and hybrid systems,'' \emph{Hybrid Systems: Computation and Control}, 2005.
\bibitem{majumdar2017funnel}
A.~Majumdar and R.~Tedrake, ``Funnel libraries for real-time robust feedback motion planning,'' \emph{IJRR}, vol.~36, no.~8, pp. 947--982, 2017.
\bibitem{fridovich2020confidence}
D.~Fridovich-Keil \emph{et~al.}, ``Confidence-aware motion prediction for real-time collision avoidance1,'' \emph{IJRR}, vol.~39, no. 2-3, pp. 250--265, 2020.
\bibitem{salzmann20}
T.~Salzmann \emph{et~al.}, ``Trajectron++: Dynamically-feasible trajectory forecasting with heterogeneous data,'' in \emph{ECCV}, 2020, pp. 683--700.
\bibitem{hwang2024emma}
J.-J. Hwang \emph{et~al.}, ``Emma: End-to-end multimodal model for autonomous driving,'' \emph{arXiv preprint arXiv:2410.23262}, 2024.
\bibitem{karkus2023diffstack}
P.~Karkus \emph{et~al.}, ``Diffstack: A differentiable and modular control stack for autonomous vehicles,'' in \emph{CoRL}.\hskip 1em plus 0.5em minus 0.4em\relax PMLR, 2023, pp. 2170--2180.
\bibitem{chen2023interactive}
Y.~Chen \emph{et~al.}, ``Interactive joint planning for autonomous vehicles,'' \emph{RA-L}, 2023.
\bibitem{casas2021mp3}
S.~Casas \emph{et~al.}, ``Mp3: A unified model to map, perceive, predict and plan,'' in \emph{CVPR}, 2021, pp. 14\,403--14\,412.
\bibitem{cui2021lookout}
A.~Cui \emph{et~al.}, ``Lookout: Diverse multi-future prediction and planning for self-driving,'' in \emph{ICCV}, 2021, pp. 16\,107--16\,116.
\bibitem{antonante2023task}
P.~Antonante \emph{et~al.}, ``Task-aware risk estimation of perception failures for autonomous vehicles,'' \emph{arXiv preprint arXiv:2305.01870}, 2023.
\bibitem{chakraborty2024system}
K.~Chakraborty \emph{et~al.}, ``System-level safety monitoring and recovery for perception failures in autonomous vehicles,'' \emph{arXiv preprint arXiv:2409.17630}, 2024.
\bibitem{dinparastdjadid2023measuring}
A.~Dinparastdjadid \emph{et~al.}, ``Measuring surprise in the wild,'' \emph{arXiv preprint arXiv:2305.07733}, 2023.
\bibitem{ding2025surprise}
W.~Ding \emph{et~al.}, ``Surprise potential as a measure of interactivity in driving scenarios,'' \emph{arXiv preprint arXiv:2502.05677}, 2025.
\bibitem{nakamura2023online}
K.~Nakamura and S.~Bansal, ``Online update of safety assurances using confidence-based predictions,'' in \emph{ICRA}.\hskip 1em plus 0.5em minus 0.4em\relax IEEE, 2023, pp. 12\,765--12\,771.
\bibitem{lindemann2023safe}
L.~Lindemann \emph{et~al.}, ``Safe planning in dynamic environments using conformal prediction,'' \emph{RA-L}, 2023.
\bibitem{li2020prediction}
A.~Li \emph{et~al.}, ``Prediction-based reachability for collision avoidance in autonomous driving,'' \emph{arXiv preprint arXiv:2011.12406}, 2020.
\bibitem{althoff2021set}
M.~Althoff \emph{et~al.}, ``Set propagation techniques for reachability analysis,'' \emph{Annual Review of CRAS}, vol.~4, no.~1, pp. 369--395, 2021.
\bibitem{paparusso2024zapp}
L.~Paparusso \emph{et~al.}, ``Zapp! zonotope agreement of prediction and planning for continuous-time collision avoidance with discrete-time dynamics,'' in \emph{ICRA}.\hskip 1em plus 0.5em minus 0.4em\relax IEEE, 2024, pp. 9285--9292.
\bibitem{dixit2023adaptive}
A.~Dixit \emph{et~al.}, ``Adaptive conformal prediction for motion planning among dynamic agents,'' in \emph{L4DC}.\hskip 1em plus 0.5em minus 0.4em\relax PMLR, 2023, pp. 300--314.
\bibitem{chen2021reactive}
Y.~Chen \emph{et~al.}, ``Reactive motion planning with probabilisticsafety guarantees,'' in \emph{CoRL}.\hskip 1em plus 0.5em minus 0.4em\relax PMLR, 2021, pp. 1958--1970.
\bibitem{driggs2017robust}
K.~Driggs-Campbell \emph{et~al.}, ``Robust, informative human-in-the-loop predictions via empirical reachable sets,'' \emph{arXiv preprint arXiv:1705.00748}, 2017.
\bibitem{devonport2020estimating}
A.~Devonport and M.~Arcak, ``Estimating reachable sets with scenario optimization,'' in \emph{L4DC}.\hskip 1em plus 0.5em minus 0.4em\relax PMLR, 2020, pp. 75--84.
\bibitem{tumu2024multi}
R.~Tumu \emph{et~al.}, ``Multi-modal conformal prediction regions by optimizing convex shape templates,'' in \emph{L4DC}.\hskip 1em plus 0.5em minus 0.4em\relax PMLR, 2024, pp. 1343--1356.
\bibitem{xiang2024convex}
J.~Xiang and J.~Chen, ``Convex approximation of probabilistic reachable sets from small samples using self-supervised neural networks,'' \emph{arXiv preprint arXiv:2411.14356}, 2024.
\bibitem{dietrich2025data}
E.~Dietrich \emph{et~al.}, ``Data-driven reachability with scenario optimization and the holdout method,'' \emph{arXiv preprint arXiv:2504.06541}, 2025.
\bibitem{driggs2017integrating}
K.~Driggs-Campbell \emph{et~al.}, ``Integrating intuitive driver models in autonomous planning for interactive maneuvers,'' \emph{ITS}, vol.~18, no.~12, pp. 3461--3472, 2017.
\bibitem{lew2021sampling}
T.~Lew and M.~Pavone, ``Sampling-based reachability analysis: A random set theory approach with adversarial sampling,'' in \emph{CoRL}.\hskip 1em plus 0.5em minus 0.4em\relax PMLR, 2021, pp. 2055--2070.
\bibitem{vovk2012conditional}
V.~Vovk, ``Conditional validity of inductive conformal predictors,'' in \emph{ACML}.\hskip 1em plus 0.5em minus 0.4em\relax PMLR, 2012, pp. 475--490.
\bibitem{wilson2010volume}
A.~J. Wilson, ``Volume of n-dimensional ellipsoid,'' \emph{Sciencia Acta Xaveriana}, vol.~1, no.~1, pp. 101--6, 2010.
\bibitem{zorin1989distribution}
V.~Zorin \emph{et~al.}, ``The distribution of quadratic forms of gaussian vectors,'' \emph{Theory of Probability \& Its Applications}, vol.~33, no.~3, pp. 557--560, 1989.
\end{thebibliography}