A Novel Bayes' Theorem for Upper Probabilities
Extracted source text · arXiv 2307.06831
%\documentclass{uai2023} % for initial submission
\documentclass[accepted]{uai2023} % after acceptance, for a revised
% version; also before submission to
% see how the non-anonymous paper
% would look like
%% There is a class option to choose the math font
% \documentclass[mathfont=ptmx]{uai2023} % ptmx math instead of Computer
% Modern (has noticable issues)
% \documentclass[mathfont=newtx]{uai2023} % newtx fonts (improves upon
% ptmx; less tested, no support)
% NOTE: Only keep *one* line above as appropriate, as it will be replaced
% automatically for papers to be published. Do not make any other
% change above this note for an accepted version.
%% Choose your variant of English; be consistent
\usepackage[american]{babel}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsthm}
% \usepackage[british]{babel}
%% Some suggested packages, as needed:
\usepackage{natbib} % has a nice set of citation styles and commands
\bibliographystyle{plainnat}
\renewcommand{\bibsection}{\subsubsection*{References}}
\usepackage{mathtools} % amsmath with fixes and additions
% \usepackage{siunitx} % for proper typesetting of numbers and units
\usepackage{booktabs} % commands to create good-looking tables
\usepackage{tikz} % nice language for creating drawings and diagrams
\usepackage{mathrsfs}
\usepackage{bbm}
%% Provided macros
% \smaller: Because the class footnote size is essentially LaTeX's \small,
% redefining \footnotesize, we provide the original \footnotesize
% using this macro.
% (Use only sparingly, e.g., in drawings, as it is quite small.)
%% Self-defined macros
\newcommand{\swap}[3][-]{#3#1#2} % just an example
\newcommand{\redc}[1]{\textcolor{red}{#1}}
\newcommand{\bluec}[1]{\textcolor{blue}{#1}}
\newtheorem{example}{Example}
\newtheorem{theorem}{Theorem}
\newtheorem{corollary}{Corollary}[theorem]
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{remark}{Remark}
\newtheorem{proposition}{Proposition}
\title{A Novel Bayes' Theorem for Upper Probabilities}
% The standard author block has changed for UAI 2023 to provide
% more space for long author lists and allow for complex affiliations
%
% All author information is authomatically removed by the class for the
% anonymous submission version of your paper, so you can already add your
% information below.
%
% Add authors
\author[1]{\href{mailto:<caprio@seas.upenn.edu>?Subject=Your E-pi UAI 2023 paper}{Michele Caprio}{}}
\author[2,3]{Yusuf Sale}
\author[2,3]{Eyke Hüllermeier}
\author[1]{Insup Lee}
%\author[1]{Further~Coauthor}
%\author[3]{Further~Coauthor}
%\author[3,1]{Further~Coauthor}
% Add affiliations after the authors
\affil[1]{%
PRECISE Center\\
Department of Computer and Information Science\\
University of Pennsylvania\\
USA
}
\affil[2]{%
Institute of Informatics\\
University of Munich (LMU)\\
Germany
}
\affil[3]{%
Munich Center for Machine Learning\\
Germany
}
\begin{document}
\maketitle
\begin{abstract}
In their seminal 1990 paper, Wasserman and Kadane establish an upper bound for the Bayes' posterior probability of a measurable set $A$, when the prior lies in a class of probability measures $\mathcal{P}$ and the likelihood is precise. They also give a sufficient condition for such upper bound to hold with equality. In this paper, we introduce a generalization of their result by additionally addressing uncertainty related to the likelihood. We give an upper bound for the posterior probability when both the prior and the likelihood belong to a set of probabilities. Furthermore, we give a sufficient condition for this upper bound to become an equality. This result is interesting on its own, and has the potential of being applied to various fields of engineering (e.g. model predictive control), machine learning, and artificial intelligence.
% We introduce a novel Bayes' theorem for upper probabilities, generalizing the classical one presented by Wasserman and Kadane.
% We give a new Bayes' theorem for upper probabilities that generalizes the classical one by Wasserman and Kadane \redc{and we discuss its usefulness in a model predictive control framework.}
% {\color{blue}MC: maybe we need a longer abstract.}
%Our result allows to find an analytical solution to an MPC problem. In particular, we prove that, under stable operating conditions, the optimal parameter for the function governing the MPC structure at a given time is the arithmetic mean of the previously observed parameters.
\end{abstract}
\section{Introduction}\label{intro}
Bayes' rule (BR) is arguably the best-known mechanism to update subjective beliefs. It prescribes the agent to elicit a prior distribution that encapsulates their initial opinion, and to come up with a likelihood that describes the data generating process. Combining prior and likelihood via BR produces the posterior distribution, which captures the agent's revised opinion in light of the collected data.
But what happens if the agent is not able to specify a single prior distribution? This may occur if they face \textit{ambiguity} \citep{ellsberg,gilboa2}. In \cite[Section 1.1.4]{walley} and \cite{prob.kin}, the authors point out that missing information and bounded rationality may prevent the agent from assessing probabilities precisely in practice, even if doing so is possible in principle. This may be due to the lack of information on how likely events of interest are, lack of computational time or ability, or because it is extremely difficult to analyze a complex body of evidence. Similarly, the agent may face difficulties in gauging the data generating process, so specifying a single likelihood may become a challenging task.
The notion of ambiguity is strictly related with that of epistemic uncertainty in machine learning (ML) and artificial intelligence (AI). Let us illustrate this clearly by borrowing concepts from \cite[Section 3.2]{ibnn}. Epistemic uncertainty (EU) corresponds to reducible uncertainty caused by a lack of knowledge about the best model. Notice how, in the precise case -- that is, when the agent specifies a single distribution -- EU is absent. In many applications, a single probability measure is only able to capture the idea of irreducible uncertainty, since it represents a case in which the agent knows exactly the true data generating process, and the prior probability that perfectly encapsulates their initial beliefs. This is a well-studied property of sets of probabilities \citep[Page 458]{eyke}. Due to the increasing relevance of reliable and trustworthy ML and AI applications, effective uncertainty representation and quantification have become vital research areas \citep{kendall2017uncertainties,smith2018understanding, depeweg2018decomposition, kapoor2022uncertainty,sale2023volume,wimmer2023quantifying}. Thus, theoretic underpinnings of imprecise probability theories emerge as a valuable methodology for improving the representation and quantification of uncertainties. Adopting concepts like (convex) sets of probabilities and upper and lower probabilities foster a more sophisticated and fine-tuned articulation of uncertainty.
We remark that EU should not be confused with the concept of \textit{epistemic probability} \citep{definetti1,definetti2,walley}. In the subjective probability literature, epistemic probability can be captured by a single distribution. Its best definition can be found in \cite[Sections 1.3.2 and 2.11.2]{walley}. There, the author specifies how epistemic probabilities model logical or psychological degrees of partial belief of the agent. We remark, though, how de Finetti and Walley work with finitely additive probabilities, while in this paper we use countably additive probabilities.
The field of imprecise probabilities \citep{augustin,walley}, and in particular the classic paper by \cite{wasserman}, and successive works like \cite{cozman,larry,giaco,kali}, study the problem of an agent updating their beliefs using BR in the presence of ambiguity.
%provide an answer to the question ``How can an agent update their beliefs using BR in the presence of ambiguity on the prior and the likelihood?''
Our paper belongs to this body of work. Our main result, Theorem \ref{main1}, generalizes the theorem in \cite[Section 2]{wasserman} to the case where the agent faces ambiguity on both what prior and what likelihood to choose to model the phenomenon of interest. We find the upper posterior, that is, the ``upper bound'' to the set of posteriors, using only the upper prior and the upper likelihood. Thanks to the conjugacy property of upper probabilities, introduced in the next section, we derive the lower posterior, that is the ``lower bound'' to the set of posteriors. We also give a necessary condition for the bound to hold with equality. In addition, we hint at possible applications, in particular in the field of model predictive control, a method of process control that is used to control a process while satisfying a set of constraints \cite{hb_mpc}.
%We adopt a (robust) Bayesian approach, and we put ourselves in the worst case scenario, that is, when the agent faces ambiguity on both the prior and likelihood needed to solve the MPC problem. We show that, under stable operating conditions, the optimal parameter $u^\star$ for the MPC objective function at time $t+1$ corresponds to the arithmetic mean of the previously observed parameters. This result confirms the intuition that in a steady environment, stability conditions are preserved.
The paper is divided as follows. Section \ref{prelim} introduces the concepts that are needed to understand our result.
% , and revisits Wasserman and Kadane's infamous Bayes' Theorem for upper probabilities in Section \ref{theorem:wk}.
In Section \ref{result_fixed} we present the main theorem, and conclude our work Section \ref{concl}. We prove our results in Section \ref{proof}.
% and discuss the weak$^\star$ topology in Appendix \ref{app-top}.
%
%
\section{Bayes' Theorem for Upper Probabilities}\label{theory_portion}
%
\subsection{Preliminaries}\label{prelim}
%We plan on solving a problem similar to \eqref{goal} , and also a more general one, analytically. We will use techniques from the IP literature \cite{ibnn,walley}.
In this section, we introduce the background notions from the IP literature \citep{augustin,decooman,walley} that are needed to understand our main results.
%Instead, we adopt an approach similar to that in \cite{ibnn}.
Call $\Delta(\Omega,\mathcal{F})$ the space of (countably additive) probability measures on a generic measurable space $(\Omega,\mathcal{F})$. Pick a generic set $\mathcal{P} \subset \Delta(\Omega,\mathcal{F})$. We denote by $\overline{P}$ the \textit{upper probability} associated with $\mathcal{P}$, that is, $\overline{P}(A)=\sup_{P\in\mathcal{P}}P(A)$, for all $A\in\mathcal{F}$. Its conjugate is called \textit{lower probability}, $\underline{P}(A)=1-\overline{P}(A^c)=\inf_{P\in\mathcal{P}}P(A)$, for all $A\in\mathcal{F}$. Because of the conjugacy property, in the remainder of this document we focus on upper probabilities only.
We say that upper probability $\overline{P}$ is \textit{concave} or \textit{$2$-alternating} if $\overline{P}(A\cup B) \leq \overline{P}(A)+\overline{P}(B)-\overline{P}(A\cap B)$, for all $A,B\in\mathcal{F}$. Upper probability $\overline{P}$ is \textit{compatible} \citep{gong} with the set
\begin{align}
\text{core}(\overline{P}):=\{&P \in \Delta(\Omega,\mathcal{F}): P(A) \leq \overline{P}(A), \forall A \in \mathcal{F}\} \nonumber\\
=\{&P \in \Delta(\Omega,\mathcal{F}): \overline{P}(A) \geq P(A) \geq \underline{P}(A), \nonumber \\ &\forall A \in \mathcal{F}\} \label{core1}
\end{align}
where \eqref{core1} is a characterization \citep[Page 3389]{cerreia}. The core is the set of all (countably additive) probability measures on $\Omega$ that are set-wise dominated by $\overline{P}$. It is convex: it is immediate to see that if $P$ and $Q$ are dominated by $\overline{P}$, then $\gamma P$ and $(1-\gamma)Q$ are dominated by $\gamma\overline{P}$ and $(1-\gamma)\overline{P}$, respectively, for all $\gamma\in [0,1]$. In turn, $\gamma P+(1-\gamma)Q$ is dominated by $\overline{P}$, thus giving the desired convex property. In addition, throughout the present work, we assume that $\text{core}(\overline{P})$ is nonempty and weak$^\star$-closed.\footnote{Recall that in the weak$^\star$ topology, a net $(P_\alpha)_{\alpha \in I}$ converges to $P$ if and only if $P_\alpha(A) \rightarrow P(A)$, for all $A \in \mathcal{F}$. See also results presented in \cite[Appendix D3]{walley}} Then, as a result of \cite[Section 3.6.1]{walley}, it is also weak$^\star$-compact.
% this is a characterization. The definition, based on the notion of continuous dual of a topological space, and how to obtain to such characterization, are presented in \cite[Appendix D3]{walley}
%and weak$^\star$-compact \cite[Proposition 3]{marinacci2}.\footnote{Recall that in the weak$^\star$ topology, a net $(P_\alpha)_{\alpha \in I}$ converges to $P$ if and only if $P_\alpha(A) \rightarrow P(A)$, for all $A \in \mathcal{F}$.} By complete characterization, we mean that it is sufficient to know $\overline{P}$ to be able to completely specify core$(\overline{P})$. To emphasize this aspect, some authors say that $\overline{P}$ is \textit{compatible} with core$(\overline{P})$ \cite{gong}.
% The definition, based on the notion of continuous dual of a topological space, and how to obtain to such characterization, are presented in \cite[Appendix D3]{walley}
\begin{remark}
% Let us pause here to comment on the result by Walley.
In \cite[Section 3.6.1]{walley}, the author shows that the finitely additive core is weak$^\star$-compact. The latter is defined as the set of all \textit{finitely additive} probabilities that are set-wise dominated by $\overline{P}$. It is a superset of the countably additive core in \eqref{core1}. To see this, notice that, in general, there might well be a probability measure that is set-wise dominated by $\overline{P}$, but that is merely finitely additive. In fact, there might even be no countably additive probabilities that are set-wise dominated by $\overline{P}$. For this reason, we have to assume that the countably additive core in \eqref{core1} is nonempty. If we further require that the countably additive core in \eqref{core1} is weak$^\star$-closed, then, this implies that it is a weak$^\star$-closed subset of a weak$^\star$-compact space. By \cite[Theorem 2.35]{rudin2}, closed subsets of compact sets are compact. In turn, we have that if the countably additive core is weak$^\star$-closed, it is also weak$^\star$-compact.
% \redc{We also remark how in \cite{berger_book,berger_berliner,huber_choq,wasserman} the authors do not distinguish between weak and weak$^\star$-topologies. An attentive read reveals that they refer to the weak$^\star$ topology as introduced in the present work.}
\end{remark}
We now present a class of probabilities that (i) is well-studied and used in robust statistics \cite{huber}, and (ii) is the core of a concave upper probability.
%We introduce it here mainly to show that we do not work with the core just for mathematical convenience.
Other classes with similar properties are presented in \cite[Examples 3-7]{wasserman}.
\begin{example}[$\varepsilon$-contaminated class]\label{ex_cont}
Consider the space $\Delta(\Omega,\mathcal{F})$ of probability measures on a generic measurable space $(\Omega,\mathcal{F})$, and assume $\Omega$ is compact. Pick any $P\in\Delta(\Omega,\mathcal{F})$ and any $\varepsilon\in [0,1]$. Define
\begin{align}\label{eps_cont_ex}
\mathcal{Q}^\text{co}:=\{Q\in\Delta(\Omega,\mathcal{F}) : Q(A)=(1-\varepsilon)P(A)+ \varepsilon R(A), \nonumber \\ \forall A\in\mathcal{F} \text{, } R \in \Delta(\Omega,\mathcal{F})\}.
\end{align}
Superscript ``co'' stands for convex and core. $\mathcal{Q}^\text{co}$ is the $\varepsilon$-contaminated class induced by $P$; it was studied in \cite[Example 3]{wasserman} and references therein. We have that $\overline{Q}(A)=(1-\varepsilon)P(A)+\varepsilon$, for all $A\in\mathcal{F}\setminus\{\emptyset\}$ and $\underline{Q}(A)=(1-\varepsilon)P(A)$, for all $A\in\mathcal{F}\setminus\{\Omega\}$. In addition, $\mathcal{Q}^\text{co}=\text{core}(\overline{Q})$, and $\overline{Q}$ is concave.
\end{example}
The $\varepsilon$-contaminated class is also instrumental for a future application of Theorem \ref{main1} to model predictive control. We will discuss this at length at the end of section \ref{result_fixed}. There, we also explain why it is important to account for the ambiguity in the likelihood model in real-world safety-critical scenarios.
% \noindent
% \textcolor{red}{YS: Shouldn't it be rather
% $$\underline{Q}(A) = \max ( (1-\epsilon) P(A), 0 ), \quad \textbf{for all} \quad A \in \mathcal{F}\setminus \{\Omega \} $$
% and
% $$\overline{Q}(A) = \min ( (1-\epsilon) P(A) + \epsilon, 1 ), \quad \textbf{for all} \quad A \in \mathcal{F}\setminus \{\emptyset \} $$?}
%
\iffalse
\subsection{Wasserman and Kadane's Bayes' Theorem for Upper Probabilities}
\label{theorem:wk}
%
In this section we briefly revisit Wasserman and Kadane's eminent Bayes' theorem for upper probabilities \citep{wasserman}.
%
% Some preliminaries for WK theorem
Let $(\Theta,\mathcal{F})$ be the measurable parameter space of interest and $\Delta(\Theta,\mathcal{F})$ the space of (countably additive) probability measures on $(\Theta,\mathcal{F})$. Let $\mathcal{Y}$ be the set of all \textit{bounded}, \textit{non-negative}, $\mathcal{F}$-measurable functional on $\Theta$. Call $\mathscr{D}$ the sample space endowed with $\sigma$-algebra $\mathcal{A}$. That is, for any random variable $Y$ of interest and all $\theta\in\Theta$, $Y(\theta)=y\in\mathscr{D}$. Let the agent elicit a set of probabilities $\mathcal{L}_\theta:=\{P_\theta\in\Delta (\mathscr{D},\mathcal{A}) : \theta \in\Theta\}$ on $\mathscr{D}$, parameterized by $\theta\in\Theta$. Assume that each $P_\theta\in\mathcal{L}_\theta$ has density $L(\theta)=p(y\mid \theta)$ with respect to some $\sigma$-finite dominating measure $\nu$ on $(\mathscr{D},\mathcal{A})$; this represents the likelihood function for $\theta$ having observed data $y\in \mathscr{D}$. Now, let the agent specify a set $\mathcal{P}$ of probabilities on $(\Theta,\mathcal{F})$, i.e., $\mathcal{P} \subseteq \Delta(\Theta, \mathcal{F})$. Then, compute $\overline{P}(A)=\sup_{P\in\mathcal{P}}P(A)$, for all $A\in\mathcal{F}$, and consider $\mathcal{P}^{\text{co}}:=\text{core}(\overline{P})$. It represents the agent's initial beliefs. We assume that every $P\in\mathcal{P}^{\text{co}}$ has a density $p$ with respect to some $\sigma$-finite dominating measure $\mu$ on $(\Theta,\mathcal{F})$.
% We call $\mathcal{P} \subseteq \Delta(\Theta, \mathcal{F})$ $m$-closed, if $P \leq \Bar{P}$ implies that $P \in \mathcal{P}$.
Further, we write $\mathcal{P}_y$ for the set of posteriors obtained by applying BR to each prior $P \in \mathcal{P}$ for which we have $\int L(\theta) dP(\theta) > 0$, and $\overline{P}_y(A) = \sup_{P \in \mathcal{P}_y} P(A)$ for each $A \in \mathcal{F}$.
% Then the \textit{upper Choquet integral} of $Y \in \mathcal{Y}$ is defined by
% \begin{align*}
% \overline{C}(Y) = \int_{0}^{\infty} \overline{P}( )
% \end{align*}
% and the \textit{lower Choquet integral} by
% \begin{align*}
% \underline{C}(Y) = \int_{0}^{\infty} ,
% \end{align*}
% respectively.
% WK Bayes' theorem
\begin{theorem}[Wasserman and Kadane, 1990]
\label{theorem:wk}
Suppose $\mathcal{P}^{\text{co}}$ is nonempty and weak$^\star$-closed. Then for all $A\in\mathcal{F}$,
\begin{align}
\overline{P}_y(A) &\leq \frac{\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta L(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)}{\mathbf{c}} \label{ineq_main} \\
&\leq \frac{\int_0^\infty \overline{P}\left( \left\lbrace{\theta\in\Theta : L(\theta)\mathbbm{1}_A(\theta)>t}\right\rbrace \right) \text{d}t}{\mathbf{c}^\prime}, \label{ineq_main2} \end{align}
provided that the ratios are well defined. Here $\mathbbm{1}_A$ denotes the indicator function for $A\in\mathcal{F}$, $\mathbf{c}:=\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta L(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)+ \inf_{P\in\mathcal{P}^{\text{co}}}\int_\Theta L(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta)$, and
\begin{align*}
\mathbf{c}^\prime :=&\underbrace{\int_0^\infty \overline{P}\left( \left\lbrace{\theta\in\Theta : L(\theta)\mathbbm{1}_A(\theta)>t}\right\rbrace \right) \text{d}t}_{\text{\textit{upper Choquet integral} of } L\mathbbm{1}_A}\\ + &\underbrace{\int_0^\infty \underline{P}\left( \left\lbrace{\theta\in\Theta : L(\theta)\mathbbm{1}_{A^c}(\theta)>t}\right\rbrace \right) \text{d}t}_{\text{\textit{lower Choquet integral} of } L\mathbbm{1}_{A^c}}.
\end{align*}
In addition, if $\overline{P}$ is concave, then the inequalities in \eqref{ineq_main} and \eqref{ineq_main2} are equalities, for all $A\in\mathcal{F}$.
\end{theorem}
Thus, Theorem \ref{theorem:wk} gives an upper bound for the posterior probability of $A \in \mathcal{F}$, when the prior $P$ lies in a set of probability measures $\mathcal{P}$.
\fi
% {\color{blue} MC: Look at example 3, page 8/13, \href{https://bit.ly/41wdFli}{this paper}}
\subsection{A novel Bayes' Theorem for Upper Probabilities}\label{result_fixed}
%In this section, we present our results in the fixed state case. As we shall see, if we fix a particular state $s\in\mathcal{S}$, we are able to find the optimal parameter $\theta^\star$ analytically, that is, without needing to resort to an algorithm. We keep the notation of section \ref{muller}. To make our assumptions clear, we will highlight them in boldface.
Let $(\Theta,\mathcal{F})$ be the measurable parameter space of interest and $\Delta(\Theta,\mathcal{F})$ the space of (countably additive) probability measures on $(\Theta,\mathcal{F})$.
%\textbf{Assumption 1}: Space $(\Theta,\mathcal{F})$ is bounded.
%\textbf{Assumption 2}: Function $J$ is known. This is a safe assumption when we know the dynamics of the world, that is, when we know the world model. This happens, for example, in chess, go, or robotics when the Lyapunov function is known.
\iffalse
Let $\mathcal{S}$ be the states space, and $\mathcal{A}$ be the set of actions. Consider a parameterized policy $\pi:\Theta \times \mathcal{S}\rightarrow \mathcal{A}$, $(\theta,s)\mapsto \pi(\theta,s)\equiv\pi_\theta(s)\in\mathcal{A}$. For any $s\in\mathcal{S}$, define the objective function $J_s$ as
\begin{equation}\label{obj_funct}
J_s:\mathbb{R}^{p+1}\rightarrow\mathbb{R}, \quad \theta\mapsto J_s(\theta):=\mathbb{E}_{\pi_\theta(s)}\left( \sum_{i=0}^I r_i \right),
\end{equation}
***HERE $\pi_\theta$ SHOULD NOT DEPEND ON $s$; WE SHOULD HAVE THAT $J(\theta)=\mathbb{E}_{\pi_\theta}(r)=\sum_{s\in\mathcal{S}} \{d(s) \sum_{a\in\mathcal{A}} \pi_\theta(s,a) R_{s,a}\}$, where $R_{s,a}$ denotes the reward function for the MDP; reward after selecting action $a$ when we are in state $s$; CHECK THIS!!***
where $\mathbb{E}_{\pi_\theta(s)}$ denotes expectation under policy $\pi_\cdot(s)$, $r_i$ is the reward at time step $i$, and $I$ is the length of the episode. Fix any $s\in\mathcal{S}$, so we can drop the explicit dependence of \eqref{obj_funct} on $s$ and write $J_s \equiv J$.
\fi
%This is mainly for mathematical purpose, and it often holds in practice.
Let $\mathcal{Y}$ be the set of all \textit{bounded}, \textit{non-negative}, $\mathcal{F}$-measurable functionals on $\Theta$. Call $\mathscr{D}$ the sample space endowed with sigma-algebra $\mathcal{A}$. That is, for any random variable $Y$ of interest and all $\theta\in\Theta$, $Y(\theta)=y\in\mathscr{D}$. Let the agent elicit a set of probabilities $\mathcal{L}_\theta:=\{P_\theta\in\Delta (\mathscr{D},\mathcal{A}) : \theta \in\Theta\}$ on $\mathscr{D}$, parameterized by $\theta\in\Theta$. This captures the ambiguity faced by the agent in determining the true data generating process \citep{ellsberg,gilboa2}. We write $P_\theta \equiv P(\cdot \mid \theta)$ for notational convenience. Assume that each $P_\theta\in\mathcal{L}_\theta$ has density $L(\theta)=p(y\mid \theta)$ with respect to some sigma-finite dominating measure $\nu$ on $(\mathscr{D},\mathcal{A})$; this represents the likelihood function for $\theta$ having observed data $y\in \mathscr{D}$.
%, for all $y\in\mathscr{D}$.
\textbf{Assumption 1.} Every density $L$ corresponding to an element $P_\theta$ of $\mathcal{L}_\theta$ belongs to $\mathcal{Y}$; that is, every density is bounded and non-negative.
Assumption 1 is needed mainly for mathematical purposes; as we shall see later in this section, it can be relaxed. Let the agent specify a set $\mathcal{P}$ of probabilities on $(\Theta,\mathcal{F})$. It represents their (incomplete) prior knowledge on the elements of $\mathcal{F}$; its elements may be informed by the collected data, thus giving the analysis an (imprecise) empirical Bayes flavor \citep{casella}. Then, compute $\overline{P}(A)=\sup_{P\in\mathcal{P}}P(A)$, for all $A\in\mathcal{F}$, and consider $\mathcal{P}^{\text{co}}:=\text{core}(\overline{P})$, assumed nonempty and weak$^\star$-closed. It represents the agent's initial beliefs. We assume that every $P\in\mathcal{P}^{\text{co}}$ has density $p$ with respect to some sigma-finite dominating measure $\mu$ on $(\Theta,\mathcal{F})$, that is, $p={\text{d}P}/{\text{d}\mu}$. We require the agent's beliefs to be represented by the core for two main reasons. The first, mathematical, one is to ensure that the upper probability is compatible with the belief set. The second, philosophical, one is the following. In Bayesian statistics, the agent selects a specific prior to encapsulate their initial beliefs. \cite{berger} points out how such choice is oftentimes arbitrary, and posits the \textit{dogma of ideal precision} (DIP). It states that in any problem there is an \textit{ideal probability model} $P_T$ which is precise, but which may not be precisely known. To overcome this shortcoming, the agent should specify a finite collection $\{P_s\}_{s\in\mathcal{S}}$ of ``plausible'' prior distributions, and compute the posterior for each $P_s$. Notice how this corresponds to selecting a finite number of elements from the core of $\overline{P}_\mathcal{S}$, where $\overline{P}_\mathcal{S}(A)=\sup_{s\in\mathcal{S}}P_s(A)$, for all $A\in\mathcal{F}$. A criticism to the DIP was brought forward by Walley. In \cite[Section 2.10.4.(c)]{walley}, he claims how given an upper probability $\overline{P}$, there is no cogent reason for which the agent should choose a specific $P_T$ that is dominated by $\overline{P}$, or -- for that matter -- a collection $\{P_s\}_{s\in\mathcal{S}}$ of ``plausible'' probabilities. Because the core considers all (countably additive) probability measures that are dominated by $\overline{P}$, it is the perfect instrument to address Walley's criticism \citep{prob.kin}.
Let the agent compute $\overline{P}_\theta$, the upper probability associated with $\mathcal{L}_\theta$, and consider $\mathcal{L}^{\text{co}}_\theta:=\text{core}(\overline{P}_\theta)$, assumed nonempty and weak$^\star$-closed. It represents the set of plausible likelihoods. As \cite{grun} point out, accounting for ambiguity around the true data generating process is crucial, as Bayesian inference may suffer from inconsistency issues if carried out using a misspecified likelihood.
%We assume that every $P_\theta\in\mathcal{L}_\theta^{\text{co}}$ has density $p(\cdot \mid\theta)=L(\theta)$ with respect to some sigma-finite dominating measure $\nu$ on $(\mathscr{D},\mathcal{A})$, that is, $L=\frac{\text{d}P_\theta}{\text{d}\nu}$.
Let
\begin{equation}\label{L_scr}
\mathscr{L}:=\left\lbrace{L=\frac{\text{d}P_\theta}{\text{d}\nu} \text{, } P_\theta \in \mathcal{L}_\theta^{\text{co}}}\right\rbrace \subset \mathcal{Y}
\end{equation}
be the set of pdf/pmf associated with the elements of $\mathcal{L}^{\text{co}}_\theta$. Let also $\overline{L}(\theta):=\sup_{L\in\mathscr{L}}L(\theta)$ and $\underline{L}(\theta):=\inf_{L\in\mathscr{L}}L(\theta)$, for all $\theta\in\Theta$. Call
\begin{align*}
\mathcal{P}^{\text{co}}_y:=\Bigg\{ P_y &\in \Delta(\Theta,\mathcal{F}) : \\
\frac{\text{d}P_y}{\text{d}\mu}&=p(\theta\mid y)=\frac{L(\theta)p(\theta)}{\int_\Theta L(\theta)p(\theta) \text{d}\theta}\text{, }\\
p&=\frac{\text{d}P}{\text{d}\mu} \text{, } P\in\mathcal{P}^{\text{co}} \text{, } L=\frac{\text{d}P_\theta}{\text{d}\nu} \text{, } P_\theta\in\mathcal{L}_\theta^{\text{co}}\Bigg\}
\end{align*}
the class of posterior probabilities when the prior is in $\mathcal{P}^{\text{co}}$ and the likelihood is in $\mathcal{L}_\theta^{\text{co}}$, and let $\overline{P}_y(A)=\sup_{P_y\in\mathcal{P}^{\text{co}}_y}P_y(A)$, for all $A\in\mathcal{F}$. Then, the following is a generalization of Bayes' theorem in \cite[Section 2]{wasserman}, and is an extension of \cite[Theorem 7]{ibnn}. We prove it in Section \ref{proof}.
\begin{theorem}[Bayes' theorem for upper probabilities]\label{main1}
Suppose $\mathcal{P}^{\text{co}},\mathcal{L}_\theta^{\text{co}}$ are nonempty and weak$^\star$-closed. Then for all $A\in\mathcal{F}$,
\begin{align}
\overline{P}_y(A) &\leq \frac{\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)}{\mathbf{c}} \label{ineq_main} \\
&\leq \frac{\int_0^\infty \overline{P}\left( \left\lbrace{\theta\in\Theta : \overline{L}(\theta)\mathbbm{1}_A(\theta)>t}\right\rbrace \right) \text{d}t}{\mathbf{c}^\prime}, \label{ineq_main2} \end{align}
provided that the ratios are well defined. Here $\mathbbm{1}_A$ denotes the indicator function for $A\in\mathcal{F}$, $\mathbf{c}:=\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)+ \inf_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \underline{L}(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta)$, and
\begin{align*}
\mathbf{c}^\prime :=&\underbrace{\int_0^\infty \overline{P}\left( \left\lbrace{\theta\in\Theta : \overline{L}(\theta)\mathbbm{1}_A(\theta)>t}\right\rbrace \right) \text{d}t}_{\text{\textit{upper Choquet integral} of } \overline{L}\mathbbm{1}_A}\\ + &\underbrace{\int_0^\infty \underline{P}\left( \left\lbrace{\theta\in\Theta : \underline{L}(\theta)\mathbbm{1}_{A^c}(\theta)>t}\right\rbrace \right) \text{d}t}_{\text{\textit{lower Choquet integral} of } \underline{L}\mathbbm{1}_{A^c}}.
\end{align*}
In addition, if $\overline{P}$ is concave, then the inequalities in \eqref{ineq_main} and \eqref{ineq_main2} are equalities, for all $A\in\mathcal{F}$.
\end{theorem}
This result is particularly appealing. Under Assumption 1, if the prior upper probability (PUP) is concave and the prior and likelihood sets $\mathcal{P}^{\text{co}},\mathcal{L}_\theta^{\text{co}}$ are nonempty and weak$^\star$-closed, then the agent can perform a (generalized) Bayesian update of the PUP by carrying out only one operation. This is the case even when the agent faces ambiguity around the true data generating process so that a set of likelihoods is needed. The posterior lower probability is obtained immediately via the conjugacy property $\underline{P}_y(A)=1-\overline{P}_y(A^c)$.
\begin{corollary}\label{cor_main1}
Retain the assumptions in Theorem \ref{main1}. If $\mathcal{L}^\text{co}_\theta$ is a singleton, we retrieve Bayes' theorem in \cite[Section 2]{wasserman}.
\end{corollary}
Corollary \ref{cor_main1} tell us that when there is no ambiguity around the likelihood, Theorem \ref{main1} recovers Wasserman and Kadane's classical Bayes' theorem. Given the straightforward nature of the proof, we omit it here. We also have the following lemma, that is proved in Section \ref{proof}.
%To further illustrate the extent of our generalization, we show that Theorem \ref{main1} indeed generalizes Wasserman and Kadane's results.
%\begin{proposition}[Wasserman and Kadane, 1990]
%Suppose $\mathcal{P}^{\text{co}}$ is nonempty and weak$^{\star}$-closed, and there is no ambiguity about the likelihood. Then Theorem \ref{main1} coincides with Wasserman and Kadane's theorem for upper probabilities.
%\end{proposition}
%\bluec{Maybe write some remark with respect to $m$-closed etc.}Thus,
\begin{lemma}[Preserved concavity]\label{lemma_ccv}
Suppose $\mathcal{P}^{\text{co}},\mathcal{L}_\theta^{\text{co}}$ are nonempty and weak$^\star$-closed. Then, if $\overline{P}$ is concave, we have that $\overline{P}_y$ is concave as well.
\end{lemma}
This lemma is important because it tells us that the generalized Bayesian update of Theorem \ref{main1} preserves concavity, and so it can be applied to successive iterations. If at time $t$ the PUP is concave, then the PUP at time $t+1$ -- that is, the posterior upper probability at time $t$ -- will be concave too. Necessary and sufficient conditions for a generic upper probability to be concave are given in \cite[Section 5]{marinacci2}.
In the future, we plan to forego Assumption 1 and use the techniques developed in \cite{decooman} to generalize our results to the case in which the elements of $\mathscr{L}$ are unbounded and not necessarily non-negative. We also intend to extend our results to the case in which the elements of $\mathcal{Y}$ are $\mathbb{R}^{d}$-valued, for some $d\in\mathbb{N}$. We suspect this is a less demanding endeavor since we do not use specific properties of $\mathbb{R}$ in our proofs.
As mentioned earlier, a natural application of our results is model predictive control (MPC). MPC is a method that is used to control a process while satisfying a set of constraints \citep{hb_mpc}. Typically, when the process is stable (or at least stable for the past $k$ time steps, for some $k\geq 0$) and if the scholar decides to take the Bayesian approach, they proceed as follows. They specify a Normal likelihood (the distribution of the control inputs) centered at the objective function of the process, and a Normal prior on the parameter of such function. In this framework, if ambiguity enters the picture, then our results become relevant.
The importance of addressing prior ambiguity was discussed at length in section \ref{result_fixed}. The reasons why accounting for likelihood ambiguity is important are as follows. First, as pointed out earlier, we may run into inconsistency issues if we perform Bayesian analysis using a misspecified likelihood \citep{grun}. Second, a (precise) Normal likelihood is a good choice only if stability of the process is ensured. MPC procedures are used in the process industries in chemical plants, oil refineries, power system balancing models, and power electronics. These are all safety-critical applications where accounting for possible sudden unexpected instabilities is of paramount importance.
The scholar may specify a class of $\epsilon$-contaminated truncated Normal priors and a class of $\eta$-contaminated truncated Normal likelihoods, and use Theorem \ref{main1} to compute the upper posterior. Notice that the Normals need to be truncated in light of Assumption 1. This requirement is not too stringent, and -- as pointed out earlier in this section -- our future work will allow us do without it.
%\bluec{These results can be generalized to the case in which the elements of $\mathscr{L}$ are unbounded and not necessarily non-negative using techniques in \cite{decooman}; this allows us to forego Assumption 1.}
%\bluec{YS: Since we agreed to have just the theoretical part around the motivation, background of the theorem and the theorem itself, we can include this comment here.}
%, and to the case in which the elements of $\mathcal{Y}$ are $\mathbb{R}^{p+1}$-valued, for some $d\in\mathbb{N}$, since we never used specific properties of $\mathbb{R}$ in our proofs (e.g. the fact that $\mathbb{R}$ is a poset).
%After that, we introduce our algorithm.
\section{Conclusion}\label{concl}
In this paper, we present a new Bayes' theorem for upper probabilities that extends the one in \cite[Section 2]{wasserman}, and \cite[Theorem 7]{ibnn}. In the future, we plan to generalize Theorem \ref{main1} by letting go of Assumption 1, and to apply it to an MPC problem and to other fields of engineering, and ML and AI. For example, we intend to use it to overcome the computational bottleneck of step 2 of the algorithm that computes the posterior set in an imprecise Bayesian neural network procedure \citep{ibnn}. There, an element-wise application of Bayes' rule for all the extreme elements of the prior and likelihood sets is performed. As we can see, this is a combinatorial task that can potentially be greatly simplified in light of Theorem \ref{main1}, conveying a computationally cheaper algorithm.
\section{Proofs}\label{proof}
\begin{proof}[Proof of Theorem \ref{main1}]
Assume that $\mathcal{P}^{\text{co}},\mathcal{L}_\theta^{\text{co}}$ are nonempty and weak$^\star$-closed. Pick any $A\in\mathcal{F}$. Recall that we can rewrite the usual Bayes' updating rule as
\begin{align*}
{P}_y(A) &= \frac{\int_\Theta {L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)}{\int_\Theta {L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)+ \int_\Theta {L}(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta)}\\
&=\frac{1}{1+\frac{\int_\Theta {L}(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta)}{\int_\Theta {L}(\theta) \mathbbm{1}_{A}(\theta) P(\text{d}\theta)}},
\end{align*}
which is maximized when
$$\frac{\int_\Theta {L}(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta)}{\int_\Theta {L}(\theta) \mathbbm{1}_{A}(\theta) P(\text{d}\theta)}$$
is minimized. But
$$\frac{\int_\Theta {L}(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta)}{\int_\Theta {L}(\theta) \mathbbm{1}_{A}(\theta) P(\text{d}\theta)} \geq \frac{\inf_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \underline{L}(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta)}{\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)},$$
which proves the inequality in \eqref{ineq_main}. The inequality in \eqref{ineq_main2} is true because
\begin{align*}
\inf_{P\in\mathcal{P}^{\text{co}}}&\int_\Theta \underline{L}(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta) \\
\geq &\int_0^\infty \underline{P}\left( \left\lbrace{\theta\in\Theta : \underline{L}(\theta)\mathbbm{1}_{A^c}(\theta)>t}\right\rbrace \right) \text{d}t
\end{align*}
and
\begin{align*}
\sup_{P\in\mathcal{P}^{\text{co}}}&\int_\Theta \overline{L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)\\
\leq &\int_0^\infty \overline{P}\left( \left\lbrace{\theta\in\Theta : \overline{L}(\theta)\mathbbm{1}_{A}(\theta)>t}\right\rbrace \right) \text{d}t.
\end{align*}
Assume now that $\overline{P}$ is concave. By \cite[Lemma 1]{wasserman}, we have that there exists $\check{P}\in\mathcal{P}^{\text{co}}$ such that
\begin{align}\label{eq_lemma1}
\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta L(\theta)\mathbbm{1}_A(\theta)P(\text{d}\theta)=\int_\Theta L(\theta)\mathbbm{1}_A(\theta)\check{P}(\text{d}\theta),
\end{align}
for all $L\in\mathscr{L}$. In addition, by \cite[Lemma 4]{wasserman}, we have that for all $Y\in\mathcal{Y}$ and all $\epsilon>0$, there exists a non-negative, upper semi-continuous function $h\leq Y$ such that
\begin{align}\label{eq_pre_lemma}
\begin{split}
\left[\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta Y(\theta)P(\text{d}\theta)\right] - \epsilon &< \sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta h(\theta)P(\text{d}\theta) \\&\leq \sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta Y(\theta)P(\text{d}\theta).
\end{split}
\end{align}
Let now $Y=\overline{L}\mathbbm{1}_A$. Notice that since $\mathcal{L}^\text{co}_\theta$ is weak$^\star$-compact (as a result of \cite[Section 3.6.1]{walley}), by \eqref{L_scr} so is $\mathscr{L}$. This implies that $\underline{L},\overline{L}\in\mathscr{L}$, since a compact set always contains its boundary, so $\underline{L},\overline{L}\in\mathcal{Y}$ as well, and in turn $\underline{L}\mathbbm{1}_{A^c},\overline{L}\mathbbm{1}_A\in\mathcal{Y}$. Fix then any $L\in\mathscr{L}$ and put $h=L\mathbbm{1}_A$. It is immediate to see that $h$ is non-negative and upper semi-continuous. Then, by \eqref{eq_pre_lemma}, we have that for all $\epsilon>0$
\begin{align}\label{eq_lemma4}
\begin{split}
&\left[\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta)\mathbbm{1}_A(\theta)P(\text{d}\theta)\right] - \epsilon < \\&\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta {L}(\theta)\mathbbm{1}_A(\theta)P(\text{d}\theta) \leq \sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta)\mathbbm{1}_A(\theta)P(\text{d}\theta).
\end{split}
\end{align}
Combining \eqref{eq_lemma1} and\eqref{eq_lemma4}, we obtain
\begin{align}\label{eq_lemma_comb}
\begin{split}
&\left[\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta)\mathbbm{1}_A(\theta)P(\text{d}\theta)\right] - \epsilon\\ &< \int_\Theta {L}(\theta)\mathbbm{1}_A(\theta)\check{P}(\text{d}\theta) \leq \sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta)\mathbbm{1}_A(\theta)P(\text{d}\theta),
\end{split}
\end{align}
for all $L\in\mathscr{L}$.
Pick now any $\epsilon>0$ and put
\begin{align*}
k&:=\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)\\&+ \inf_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \underline{L}(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta)>0.
\end{align*}
Choose any $L\in\mathscr{L}$ and $\delta\in (0,\epsilon k)$. By \eqref{eq_lemma_comb} we have that
\begin{equation}\label{ineq_imm_1}
\left[\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta)\mathbbm{1}_A(\theta)P(\text{d}\theta)\right] - \delta < \int_\Theta {L}(\theta)\mathbbm{1}_A(\theta)\check{P}(\text{d}\theta)
\end{equation}
and that
\begin{equation}\label{ineq_imm_2}
\left[\inf_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \underline{L}(\theta)\mathbbm{1}_{A^c}(\theta)P(\text{d}\theta)\right] + \delta > \int_\Theta {L}(\theta)\mathbbm{1}_{A^c}(\theta)\check{P}(\text{d}\theta).
\end{equation}
The inequality in \eqref{ineq_imm_1} comes from the fact that the first inequality in \eqref{eq_lemma_comb} holds for all $\epsilon>0$, and -- given how $k$ is defined -- we have that $\delta>0$. The inequality in \eqref{ineq_imm_2} is obtained by re-deriving \eqref{eq_lemma1}, \eqref{eq_pre_lemma}, \eqref{eq_lemma4}, and \eqref{eq_lemma_comb} for the infimum of set $\mathcal{P}^\text{co}$ rather than the supremum. In that case, we simply substitute $\sup$ with $\inf$, $\overline{L}$ with $\underline{L}$, $\mathbbm{1}_A$ with $\mathbbm{1}_{A^c}$, ``$-\epsilon$'' with ``$+\epsilon$'', and we reverse the inequalities. %order of the
Recall that $\mathbf{c}:=\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)+ \inf_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \underline{L}(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta)$, and define $$\mathbf{d}:=\int_\Theta {L}(\theta) \mathbbm{1}_A(\theta) \check{P}(\text{d}\theta)+ \int_\Theta {L}(\theta) \mathbbm{1}_{A^c}(\theta) \check{P}(\text{d}\theta).$$ Then we have,
\begin{align*}
\check{P}_y(A)&=\frac{\int_\Theta {L}(\theta) \mathbbm{1}_A(\theta) \check{P}(\text{d}\theta)}{\mathbf{d}}\\
&\geq \frac{\left[\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)\right]-\delta}{\mathbf{c}+\delta-\delta}\\
&=\frac{\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)}{\mathbf{c}} -\frac{\delta}{k}\\
&>\frac{\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)}{\mathbf{c}} -\epsilon.
\end{align*}
Since this holds for all $\epsilon>0$, we have that
$$\sup_{P_y\in\mathcal{P}_y^{\text{co}}}P_y(A)=\frac{\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)}{\mathbf{c}},$$
concluding the proof of inequality \eqref{ineq_main} being an equality when $\overline{P}$ is concave. Inequality \eqref{ineq_main2} being an equality when $\overline{P}$ is concave comes immediately from \cite[Lemma 4]{wasserman}, and the fact that $\overline{L}\mathbbm{1}_A,\underline{L}\mathbbm{1}_{A^c}\in\mathcal{Y}$, as pointed out above.
\end{proof}
\begin{proof}[Proof of Lemma \ref{lemma_ccv}]
In their works \cite{walley2,wasserman}, the authors show that concave upper probabilities are closed with respect to the generalized Bayes' rule. In particular, this means that, if we let $\mathbf{b}:=\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta {L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)+ \inf_{P\in\mathcal{P}^{\text{co}}}\int_\Theta {L}(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta)$, for any fixed $A\in\mathcal{F}$, if $\overline{P}$ is concave, then for all $L\in\mathscr{L}$
\begin{equation}\label{ccv_gen}
\overline{P}_y(A)=\frac{\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta {L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)}{\mathbf{b}}
\end{equation}
is concave. But since $\mathcal{L}^\text{co}_\theta$ is weak$^\star$-compact (as a consequence of \cite[Section 3.6.1]{walley}), by \eqref{L_scr} so is $\mathscr{L}$. This implies that $\underline{L},\overline{L}\in\mathscr{L}$, since a compact set always contains its boundary. Call then $L^\prime=\overline{L}\mathbbm{1}_A+\underline{L}\mathbbm{1}_{A^c}$. It is immediate to see that $L^\prime\in\mathscr{L}$. Then, by \eqref{ccv_gen} we have that if we call $\mathbf{b}^\prime:=\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta {L^\prime}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)+ \inf_{P\in\mathcal{P}^{\text{co}}}\int_\Theta {L^\prime}(\theta) \mathbbm{1}_{A^c}(\theta) P(\text{d}\theta)$, it follows that
\begin{align*}
\overline{P}_y(A)&=\frac{\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta {L^\prime}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)}{\mathbf{b}^\prime}\\
&=\frac{\sup_{P\in\mathcal{P}^{\text{co}}}\int_\Theta \overline{L}(\theta) \mathbbm{1}_A(\theta) P(\text{d}\theta)}{\mathbf{c}}
\end{align*}
is concave, concluding the proof.
\end{proof}
\iffalse
\appendix
\section{The weak$^\star$ topology}\label{app-top}
In an effort to make this paper self-contained, in this section we introduce the concept of weak$^\star$ topology. To do so, we follow the classic presentation by \cite{rudin3}. Let $X=(X,\tau)$ be a generic topological vector space endowed with topology $\tau$. Its \textit{continuous dual} $X^\star$ is the space of all continuous linear functionals $f:X\rightarrow \mathbb{R}$ on $X$. The \textit{weak$^\star$} topology on $X^\star$ is defined to be the $X$-topology on $X^\star$. It is the coarsest topology (that is, the one with the fewest open sets) under which every element $x \in X$ corresponds to a continuous map on $X^\star$. The fundamental observation is that every element $x \in X$ induces a linear functional $f_x$ on $X^\star$. In particular, $f_x$ is of the form $f_x(\Lambda)=\Lambda(x)$, for all $\Lambda \in X^\star$. Because of this, one can view the space $X$ as a collection of linear functionals on $X^\star$ and hence can define the $X$-topology thereon.
In the present work, both parameter $\theta\in\Theta$ and observations $y\in\mathscr{D}$ are realizations of random variables. Call $\Omega$ the state space, $\Theta$ the parameter space, and $\mathscr{D}$ the sample space associated with the experiment of interest. We can see $\theta \in \Theta$ as the realization of a random variable $\vartheta:\Omega \rightarrow \Theta$, $\omega \mapsto \vartheta(\omega)=\theta$, and $y\in\mathscr{D}$ as the realization of a random variable $Y:\Theta \rightarrow \mathscr{D}$, $\theta \mapsto Y(\theta)=y$. So the weak$^\star$ topology is the coarsest one that makes the evaluation functionals $f_\omega$ and $g_\theta$ on $\Omega^\star$ and $\Theta^\star$, respectively -- defined as $f_\omega(\vartheta)=\vartheta(\omega)$, for all $\vartheta \in \Omega^\star$, and $g_\theta(Y)=Y(\theta)$, for all $Y \in \Theta^\star$ -- continuous. The weak$^\star$ topology on the dual spaces $\Omega^\star$ and $\Theta^\star$ induces a topology on the class of (countably additive) probability measures $\Delta(\Omega)$ and $\Delta(\Theta)$ by forming intersections with $\Delta(\Omega)$ and $\Delta(\Theta)$ of the open sets. The neighborhoods of, say, $P_0\in\Delta(\Omega)$ under the weak$^\star$ topology are the sets containing $\{P\in \Delta(\Omega) : |P(A_i)-P_0(A_i)| \leq 1 \text{, } i\in\{1,\ldots,n\}\}$, for some $A_i$ in the sigma-algebra endowed to $\Omega$. A similar definition holds for the neighborhoods of $P_0^\prime\in\Delta(\Theta)$.
\fi
%Then, when we ask in Theorem \ref{main1} and in Lemma \ref{lemma_ccv} that $\mathcal{P}^{\text{co}}$ and $\mathcal{L}_\theta^{\text{co}}$ are weak$^\star$-closed, we want them to be closed in the coarsest topology that makes the above mentioned functions continuous.
%\newpage
\begin{contributions}
Michele Caprio and Yusuf Sale contributed equally to this paper.
% will be removed in pdf for initial submission
% (without ‘accepted’ option in \documentclass)
% so you can already fill it to test with the
% ‘accepted’ class option
\end{contributions}
\begin{acknowledgements} % will be removed in pdf for initial submission,
% (without ‘accepted’ option in \documentclass)
% so you can already fill it to test with the
% ‘accepted’ class option
Michele Caprio would like to acknowledge partial funding by the Army Research Office (ARO MURI W911NF2010080). Yusuf Sale is supported by the DAAD programme Konrad Zuse Schools of Excellence in Artificial Intelligence, sponsored by the Federal Ministry of Education and Research.
\end{acknowledgements}
% References
\bibliography{uai2023-template}
\end{document}
\begin{thebibliography}{31}
\providecommand{\natexlab}[1]{#1}
\providecommand{\url}[1]{\texttt{#1}}
\expandafter\ifx\csname urlstyle\endcsname\relax
\providecommand{\doi}[1]{doi: #1}\else
\providecommand{\doi}{doi: \begingroup \urlstyle{rm}\Url}\fi
\bibitem[{Augustin, Thomas and Coolen, Frank P.A. and De Cooman, Gert and
Troffaes, Matthias C.M.}({2014})]{augustin}
{Augustin, Thomas and Coolen, Frank P.A. and De Cooman, Gert and Troffaes,
Matthias C.M.}, editor.
\newblock \emph{{Introduction to imprecise probabilities}}.
\newblock {Wiley Series in Probability and Statistics}. {John Wiley and Sons},
{2014}.
\bibitem[Berger(1984)]{berger}
James~O. Berger.
\newblock The robust {B}ayesian viewpoint.
\newblock In Joseph~B. Kadane, editor, \emph{Robustness of Bayesian Analyses}.
Amsterdam : North-Holland, 1984.
\bibitem[Caprio and Gong(2023)]{prob.kin}
Michele Caprio and Ruobin Gong.
\newblock Dynamic imprecise probability kinematics.
\newblock \emph{Proceedings of Machine Learning Research}, 215:\penalty0
72--83, 2023.
\bibitem[Caprio et~al.(2023)Caprio, Dutta, Ivanov, Jang, Lin, Sokolsky, and
Lee]{ibnn}
Michele Caprio, Souradeep Dutta, Radoslav Ivanov, Kuk Jang, Vivian Lin, Oleg
Sokolsky, and Insup Lee.
\newblock Imprecise {B}ayesian neural networks.
\newblock \emph{Available at
\href{https://arxiv.org/abs/2302.09656}{arXiv:2302.09656}}, 2023.
\bibitem[Casella(1985)]{casella}
George Casella.
\newblock An introduction to empirical {B}ayes data analysis.
\newblock \emph{The American Statistician}, 39\penalty0 (2):\penalty0 83--87,
1985.
\bibitem[Cerreia-Vioglio et~al.(2015)Cerreia-Vioglio, Maccheroni, and
Marinacci]{cerreia}
Simone Cerreia-Vioglio, Fabio Maccheroni, and Massimo Marinacci.
\newblock {Ergodic theorems for lower probabilities}.
\newblock \emph{Proceedings of the American Mathematical Society},
144:\penalty0 3381--3396, 2015.
\bibitem[Cozman(2000)]{cozman}
Fabio~Gagliardi Cozman.
\newblock Credal networks.
\newblock \emph{Artificial Intelligence}, 120:\penalty0 199--233, 2000.
\bibitem[de~Finetti(1974)]{definetti1}
Bruno de~Finetti.
\newblock \emph{{Theory of Probability}}, volume~1.
\newblock New York : Wiley, 1974.
\bibitem[de~Finetti(1975)]{definetti2}
Bruno de~Finetti.
\newblock \emph{{Theory of Probability}}, volume~2.
\newblock New York : Wiley, 1975.
\bibitem[Depeweg et~al.(2018)Depeweg, Hernandez-Lobato, Doshi-Velez, and
Udluft]{depeweg2018decomposition}
Stefan Depeweg, Jose-Miguel Hernandez-Lobato, Finale Doshi-Velez, and Steffen
Udluft.
\newblock Decomposition of uncertainty in bayesian deep learning for efficient
and risk-sensitive learning.
\newblock In \emph{International Conference on Machine Learning}, pages
1184--1193. PMLR, 2018.
\bibitem[Ellsberg(1961)]{ellsberg}
Daniel Ellsberg.
\newblock Risk, ambiguity, and the {S}avage axioms.
\newblock \emph{The Quarterly Journal of Economics}, 75\penalty0 (4):\penalty0
643--669, 1961.
\bibitem[Epstein and Wang(1994)]{larry}
Larry~G Epstein and Tan Wang.
\newblock {Intertemporal Asset Pricing Under Knightian Uncertainty}.
\newblock \emph{Econometrica}, 62\penalty0 (2):\penalty0 283--322, 1994.
\bibitem[Giacomini and Kitagawa(2021)]{giaco}
Raffaella Giacomini and Toru Kitagawa.
\newblock Robust {B}ayesian inference for set-identified models.
\newblock \emph{Econometrica}, 89\penalty0 (4):\penalty0 1519--1556, 2021.
\bibitem[Gilboa and Marinacci(2013)]{gilboa2}
Itzhak Gilboa and Massimo Marinacci.
\newblock Ambiguity and the {B}ayesian paradigm.
\newblock In Daron Acemoglu, Manuel Arellano, and Eddie Dekel, editors,
\emph{Advances in Economics and Econometrics, Tenth World Congress},
volume~1. Cambridge : Cambridge University Press, 2013.
\bibitem[Gong and Meng(2021)]{gong}
Ruobin Gong and Xiao-Li Meng.
\newblock Judicious judgment meets unsettling updating: dilation, sure loss,
and {S}impson's paradox.
\newblock \emph{Statistical Science}, 36\penalty0 (2):\penalty0 169--190, 2021.
\bibitem[Gr{\"u}nwald and van Ommen(2017)]{grun}
Peter Gr{\"u}nwald and Thijs van Ommen.
\newblock {Inconsistency of Bayesian Inference for Misspecified Linear Models,
and a Proposal for Repairing It}.
\newblock \emph{Bayesian Analysis}, 12\penalty0 (4):\penalty0 1069 -- 1103,
2017.
\bibitem[Huber and Ronchetti(2009)]{huber}
Peter~J. Huber and Elvezio~M. Ronchetti.
\newblock \emph{Robust statistics}.
\newblock Wiley Series in Probability and Statistics. Hoboken, New Jersey :
Wiley, 2nd edition, 2009.
\bibitem[Hüllermeier and Waegeman(2021)]{eyke}
Eyke Hüllermeier and Willem Waegeman.
\newblock Aleatoric and epistemic uncertainty in machine learning: an
introduction to concepts and methods.
\newblock \emph{Machine Learning}, 3\penalty0 (110):\penalty0 457--506, 2021.
\bibitem[Kapoor et~al.(2022)Kapoor, Maddox, Izmailov, and
Wilson]{kapoor2022uncertainty}
Sanyam Kapoor, Wesley~J Maddox, Pavel Izmailov, and Andrew~Gordon Wilson.
\newblock On uncertainty, tempering, and data augmentation in bayesian
classification.
\newblock \emph{arXiv preprint arXiv:2203.16481}, 2022.
\bibitem[Kendall and Gal(2017)]{kendall2017uncertainties}
Alex Kendall and Yarin Gal.
\newblock What uncertainties do we need in bayesian deep learning for computer
vision?
\newblock \emph{Advances in neural information processing systems}, 30, 2017.
\bibitem[Klibanoff and Hanany(2007)]{kali}
Peter Klibanoff and Eran Hanany.
\newblock Updating preferences with multiple priors.
\newblock \emph{Theoretical Economics}, 2\penalty0 (3):\penalty0 261--298,
2007.
\bibitem[Marinacci and Montrucchio(2004)]{marinacci2}
Massimo Marinacci and Luigi Montrucchio.
\newblock Introduction to the mathematics of ambiguity.
\newblock In Itzhak Gilboa, editor, \emph{Uncertainty in economic theory: a
collection of essays in honor of David Schmeidler's 65th birthday}. London :
Routledge, 2004.
\bibitem[Rakovi and Levine(2018)]{hb_mpc}
Saa~V. Rakovi and William~S. Levine.
\newblock \emph{Handbook of Model Predictive Control}.
\newblock Birkh\"{a}user Basel, 1st edition, 2018.
\bibitem[Rudin(1976)]{rudin2}
Walter Rudin.
\newblock \emph{Principles of Mathematical Analysis}.
\newblock New york : McGraw-Hill, 3rd edition, 1976.
\bibitem[Sale et~al.(2023)Sale, Caprio, and H{\"o}llermeier]{sale2023volume}
Yusuf Sale, Michele Caprio, and Eyke H{\"o}llermeier.
\newblock Is the volume of a credal set a good measure for epistemic
uncertainty?
\newblock In \emph{Uncertainty in Artificial Intelligence}, pages 1795--1804.
PMLR, 2023.
\bibitem[Smith and Gal(2018)]{smith2018understanding}
Lewis Smith and Yarin Gal.
\newblock Understanding measures of uncertainty for adversarial example
detection.
\newblock \emph{arXiv preprint arXiv:1803.08533}, 2018.
\bibitem[Troffaes and de~Cooman(2014)]{decooman}
Matthias~C.M. Troffaes and Gert de~Cooman.
\newblock \emph{Lower Previsions}.
\newblock Chichester, United Kingdom : John Wiley and Sons, 2014.
\bibitem[Walley(1981)]{walley2}
Peter Walley.
\newblock Coherent lower (and upper) probabilities.
\newblock Technical report, University of Warwick, Coventry, 1981.
\bibitem[Walley(1991)]{walley}
Peter Walley.
\newblock \emph{{Statistical Reasoning with Imprecise Probabilities}},
volume~42 of \emph{Monographs on Statistics and Applied Probability}.
\newblock London : Chapman and Hall, 1991.
\bibitem[Wasserman and Kadane(1990)]{wasserman}
Larry~A. Wasserman and Joseph~B. Kadane.
\newblock Bayes' theorem for {C}hoquet capacities.
\newblock \emph{The Annals of Statistics}, 18\penalty0 (3):\penalty0
1328--1339, 1990.
\bibitem[Wimmer et~al.(2023)Wimmer, Sale, Hofman, Bischl, and
H{\"u}llermeier]{wimmer2023quantifying}
Lisa Wimmer, Yusuf Sale, Paul Hofman, Bernd Bischl, and Eyke H{\"u}llermeier.
\newblock Quantifying aleatoric and epistemic uncertainty in machine learning:
Are conditional entropy and mutual information appropriate measures?
\newblock In \emph{Uncertainty in Artificial Intelligence}, pages 2282--2292.
PMLR, 2023.
\end{thebibliography}