On the Regret of Online Edge Service Hosting
Extracted source text · arXiv 2303.06851
\documentclass[preprint,12pt]{elsarticle}
% Authors, replace the red X's with your assigned DOI string during the rightsreview eform process.
\usepackage{mathtools,dsfont}
\usepackage{psfrag,bbm,graphicx}
%\usepackage{algorithm,algorithmic}
\usepackage{comment,balance}
\usepackage{epstopdf}
\usepackage{breakurl}
\usepackage{tikz}
\usepackage{url}
\usepackage[shortlabels]{enumitem}
%\usetikzlibrary{arrows}
\usepackage{epsfig,subfig}
\usepackage{multicol}
\usepackage{multirow}
\usepackage{amsfonts, color}
\usepackage{array}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage[linesnumbered,ruled,vlined]{algorithm2e}
\usepackage{nccmath}
%\usepackage[caption=false]{subfig}
%\usepackage{caption}
\usepackage{bm}
\usepackage[hidelinks]{hyperref}
%\usepackage{blindtext}
%\usepackage{multirow}
\allowdisplaybreaks
\newtheorem{corollary}{Corollary}
\newtheorem{lemma}{Lemma}
\newtheorem{remark}{Remark}
\newtheorem{exercise}{Exercise}
\newtheorem{example}{Example}
\newtheorem{prop}{Proposition}
\newtheorem{theorem}{Theorem}
\newtheorem{approximation}{Approximation}
\newtheorem{definition}{Definition}
%\newproof{proof}{Proof}
%\newproof{pot}{Proof of Theorem}
\newtheorem{infthm}{Informal Theorem}
\newtheorem{assumption}{Assumption}
\newtheorem{model}{Model}
\newcommand{\remove}[1]{{}}
\newcommand\numberthis{\addtocounter{equation}{1}\tag{\theequation}}
\def\E{{\mathbb{E}}}
\def\Prob{{\mathbb{P}}}
\def\BB{{\mathcal{B}}}
\def\Indc{{\mathbbm{1}}}
\def\setR{\mathbb{R}}
\def\setZ{\mathbb{Z}}
\def\setC{\mathbb{C}}
\def\setN{\mathbb{N}}
\def\bZ{\mathbf{Z}}
\def\bz{\mathbf{z}}
\def\FF{\mathcal{F}}
\newcommand{\fall}{\,\forall\,}
\def\vecV{\underbar{v}}
\def \OO {\mathrm{O}}
\def \oo {\mathrm{o}}
\def \idle {k}
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
%\makeatletter
%\def\endthebibliography{%
% \def\@noitemerr{\@latex@warning{Empty `thebibliography' environment}}%
% \endlist
%}
%\makeatother
%\fancyhead{}
\begin{document}
%\maketitle
\begin{frontmatter}
\title{On the Regret of Online Edge Service Hosting \tnoteref{t1}}
%\tnotetext[t1]{This work is supported by a SERB grant on Leveraging Edge Resources for Service Hosting and a SERB grant on Online Learning with Constraints.}
\author{R Sri Prakash}
\ead{prakash.14191@gmail.com;sriprakash@ee.iitb.ac.in}
\author{Nikhil Karamchandani}
\ead{nikhilk@ee.iitb.ac.in}
\author{Sharayu Moharir}
\ead{sharayum@ee.iitb.ac.in}
\affiliation{organization={Indian Institute of Technology Bombay},
addressline={Powai},
postcode={400076},
city={Mumbai},
country={India}}
\begin{abstract}
We consider the problem of service hosting where a service provider can dynamically rent edge resources via short term contracts to ensure better quality of service to its customers. The service can also be partially hosted at the edge, in which case, customers' requests can be partially served at the edge.
The total cost incurred by the system is modeled as a combination of the rent cost, the service cost incurred due to latency in serving customers, and the fetch cost incurred as a result of the bandwidth used to fetch the code/databases of the service from the cloud servers to host the service at the edge. In this paper, we compare multiple hosting policies with regret as a metric, defined as the difference in the cost incurred by the policy and the optimal policy over some time horizon $T$. In particular we consider the Retro Renting (RR) and Follow The Perturbed Leader (FTPL) policies proposed in the literature and provide performance guarantees on the regret of these policies. We show that under i.i.d stochastic arrivals, RR policy has linear regret while FTPL policy has constant regret. Next, we propose a variant of FTPL, namely Wait then FTPL (W-FTPL), which also has constant regret while demonstrating much better dependence on the fetch cost. We also show that under adversarial arrivals, RR policy has linear regret while both FTPL and W-FTPL have regret \(\mathrm{O}(\sqrt{T})\) which is order-optimal.
\end{abstract}
\end{frontmatter}
\section{Introduction}
\label{sec:intro}
\input{introduction-N.tex}
%\section{Related Work}
%\label{sec:related_work}
%\input{relatedWork.tex}
\section{System Setup}
\label{sec:setting}
\input{setting.tex}
\section{Policies}
\label{sec:policies}
\input{policies.tex}
\section{Main Results and Discussion}
\label{sec:mainResults}
\input{mainResults.tex}
\section{Simulations}
\label{sec:simulations}
\input{simulations.tex}
\section{Proof of theorems}
\label{sec:proofs}
\input{proofs.tex}
%\section*{References}
\bibliography{ref_PC}
\bibliographystyle{elsarticle-num}
\section{Appendix}
\label{sec:appendix}
\input{appendix.tex}
\end{document}
%\color{magenta}
Software as a Service (SaaS) instances like online navigation platforms, Video-on-Demand services, etc., have stringent latency constraints in order to provide a good quality of experience to their customers. While most SaaSs use cloud resources, low latency necessitates the use of storage/computational resources at the edge, i.e., close to the end-user. A service is said to be hosted at the edge if the code and databases needed to serve user queries are stored on the edge servers and requests can be served at the edge. In this work, the service can also be partially hosted at the edge, in which case, customers' requests can be partially served at the edge \cite{prakash2020partial, LN_partial_hosting}. For instance, consider a SaaS application which provides access to news articles on demand. A typical news article includes some text, some images and possibly some embedded videos. One way to partially host such a service is to store the text corresponding to the news articles at the edge and store the images/videos in the cloud. Each user request will then be served partially by the edge.
%For example consider the news application where there are text, image and video requests, in partial service hosting text request can be served by edge and images, videos can be loaded from cloud.
%\color{black}\footnote{This work is supported by a SERB grant on Leveraging Edge Resources for Service Hosting and a SERB grant on Online Learning with Constraints.}
We consider the setting where third-party resources can be rented via short-term contracts to host the service, and the edge hosting status of the SaaS can be dynamically changed over time. If the service is not hosted at the edge, it can be fetched from the cloud servers by incurring a fetch cost. The performance of a hosting policy is a function of the rent cost, the fetch cost, and the quality of experience of the users. We refer to the algorithmic challenge of determining what fraction of the service to host at the edge over time as the \emph{service hosting problem}. Note that we study the service hosting problem from the perspective of a specific SaaS provider which can rent third-party edge resources to improve their customers' quality of experience by reducing the latency of their service.
%The presence of other SaaS providers in the system is captured through the cost of renting resources
%we are considering the problem in application provider perspective and the edge resources are rented by the third-party. So hosting other applications at edge is not a constraint for the service provider but for the third party resource provider. Therefore we consider that resources are always available for service provider and rent cost is known.
\color{black}
%\begin{comment}
Novel online service hosting policies with provable performance guarantees have been proposed in \cite{narayana2021renting, zhao2018red}.
%Online hosting policies make hosting decisions without the knowledge of the future arrival sequence. A commonly used benchmark to compare the performance of such online policies is the offline optimal policy which knows the entire request arrival sequence apriori and uses this information to make hosting decisions.
The metric of interest in \cite{narayana2021renting,zhao2018red} is the \emph{competitive ratio}, defined as the ratio of the cost incurred by an online policy to the cost incurred by an optimal offline policy for the same request arrival sequence. Since the competitive ratio is multiplicative by definition, the absolute value of the difference in the cost incurred by an online policy and the optimal policy can be large even though the competitive ratio of the online policy is close to one. This motivates studying the performance of candidate policies in terms of \emph{regret}, defined as the difference in the cost incurred by the policy and the optimal policy. Regret is a widely used metric in online learning \cite{lai1985asymptotically}, including recently for the caching problem \cite{paschos2019learning,bhattacharjee2020fundamental}, which is closely related to the service hosting problem. In this work, one of our goals is to design hosting policies with provable guarantees in terms of the regret. Another key dimension in the performance guarantees of online policies is the assumption made on the request arrival process. Commonly studied settings include the \emph{stochastic setting} and the \emph{adversarial setting}, and
%In the stochastic setting, the arrival process is modeled as a stochastic process whose distribution is not known to the online algorithm. In the adversarial setting, no structural assumptions are made on the arrival process and worst-case guarantees are provided.
we provide performance guarantees for both settings.
%\end{comment}
\subsection{Our Contributions}
We propose a variant of the FTPL policy called Wait then Follow the Perturbed Leader (W-FTPL). W-FTPL is a randomized policy that {takes into account the fetch cost in its decision-making}. More specifically, W-FTPL does not fetch the service for an initial wait period which depends on the request arrivals and is an increasing function of the fetch cost. Following the wait period, W-FTPL mimics the FTPL policy.
For i.i.d. stochastic request arrivals and the regret metric, we show that RR is sub-optimal while FTPL and W-FTPL are both order-optimal with respect to the time horizon. While the regret of FTPL can increase linearly with the fetch cost, the regret of W-FTPL increases at most polylogarithmically. The improved performance of W-FTPL over FTPL is a consequence of the fact that W-FTPL avoids most of the fetches made by FTPL in the initial time-slots and by the end of the wait period, its estimate of the arrival rate is accurate enough to avoid multiple fetches.
Further, we characterize the competitive ratio of FTPL and W-FTPL for the setting where a finite number of partial hosting levels are allowed. We compare these results with the competitive ratio of RR when a single partial hosting level is allowed \cite{LN_partial_hosting} to conclude that FTPL and W-FTPL outperform RR with respect to the competitive ratio.
For the adversarial setting, we first characterize a fundamental lower bound on the regret of any online hosting policy. In terms of regret, we then show that RR is strictly suboptimal, while FTPL and W-FTPL have order-optimal performance with respect to time. We show that the competitive ratios of FTPL and W-FTPL are upper bounded by a constant (with respect to time) for any finite number of partial hosting levels. This is a more general setting than the one considered in \cite{LN_partial_hosting}, where the competitive ratio of a variant of RR is characterized when only one partial hosting level is permitted.
%The competitive ratio of FTPL and W-FTPL can increase with the fetch cost while that of RR decreases with fetch cost\cite{LN_partial_hosting}.
\subsection{Related work}
%
Several emerging applications such as Augmented / Virtual Reality (AR/VR), online gaming, the Internet of Things (IoT), and autonomous driving have very stringent latency requirements. In addition, many of these applications are also compute-and-storage heavy. This necessitates the migration of some of the storage and computation requirements of these services from remote cloud servers to the edge. To enable such offloading, several edge computing architectures have been proposed in the literature and their performance has been analyzed extensively, see for example \cite{Puliafito:2019, mao2017survey, mach2017mobile} and references therein.
The service hosting problem has received a lot of attention recently, and various settings have been studied. One approach has been to pose the design problem of where to host which service as a one-shot large-scale optimization problem, see for example \cite{pasteris2019service, bi2019joint, 9326369}. Our work differs from this line of work in that we consider an online setting where we adaptively decide when (and what fraction of) service to host at any time, depending on the varying number of requests seen thus far.
There have been recent works \cite{zhao2018red, tan2021asymptotically} which consider the online service hosting problem, and design adaptive schemes whose performance is characterized in terms of the competitive ratio with respect to an oracle which known the entire request sequence apriori. Our work differs from these in a couple of ways. Firstly, the above mentioned works study the problem from the perspective of an edge resource provider who has to decide which amongst a collection of services to host at each time on a storage-constrained edge server. On the other hand, similar to some other works \cite{xia2020online, narayana2021renting}, we study the problem from the perspective of a service provider which needs to decide when to host its application at the edge so as to minimize the overall cost. Secondly, most of the works mentioned above do not consider partial hosting of the service, which is allowed in our framework. While \cite{LN_partial_hosting} does study partial hosting and proposed the Retro-Renting (RR) policy which is shown to have a constant competitive ratio, it only allows one level of partial hosting of the service. On the other hand, we consider a much more general setting where multiple partial service hosting levels are allowed and are able to show that both the FTPL and its variant W-FTPL are able to achieve a constant competitive ratio.
While the works mentioned above study competitive ratio as a performance metric, another popular measure of performance for online algorithms is the regret which considers the difference in the cost incurred by an online policy and the optimal (static) policy. In particular, for the closely related caching problem, several policies inspired by the recent advances in online convex optimization have been proposed, including Online Gradient Ascent \cite{paschos2019learning}, Online Mirror Descent \cite{salem2021no}, and Follow the Perturbed Leader (FTPL) \cite{mukhopadhyay2021online,bhattacharjee2020fundamental}. In particular, FTPL has been shown to have order-optimal regret for the caching problem in the adversarial setting \cite{mukhopadhyay2021online,bhattacharjee2020fundamental}. In this work, we study regret for the RR and FTPL policies for the service hosting problem under both the stochastic and adversarial settings.
Since RR is a deterministic policy, its regret performance in the adversarial setting is poor. While FTPL does achieve order-optimal regret (with respect to time), one limitation is that it makes hosting decisions {agnostic of the fetch cost}. As a result, in some cases, FTPL is prone to fetching and evicting the service multiple times in the initial time-slots when its estimate of the request arrival rate is noisy, thus leading to poor performance. This motivated our design and regret analysis of W-FTPL, a variant of FTPL which takes into account the fetch cost. One recent work which also considers regret as a performance metric is \cite{fan2022online}, which studied the problem of joint online service hosting and routing and proposed a two time-scale algorithm based on online gradient descent with provably order-optimal regret under adversarial requests.
Finally, there are several other approaches which have been used to address the online service hosting problem, including Markov Decision Processes \cite{wang2015dynamic, xiong2101learning,prakash2020partial}, Reinforcement Learning \cite{zeng2019resource}, and Multi-Armed Bandits \cite{chen2019budget, he2020bandit}.
%
%For the adversarial setting, the competitive ratio of FTPL and W-FTPL can increase with the fetch cost while that of RR decreases with fetch cost.
%In recent times there is increase in computational demand for mobile users because of many applications using complex learning algorithms to enhance the user experience. One way to reduce the computational demand from mobile user is to use cloud computing and provide Software as a service (SaaS). In this the user requests a query and computations are done at the back-end servers to provide the result back to the user. The issue with the cloud computing is the latency (few ms). There are many applications which are not tolerant with this latency and need much faster response to run the application smoothly. Edge computing is considered as one of the solution to deal with latency. In the edge computing apart from back-end server (cloud server) which has huge computational power it also has a front-end server (edge server) close to the user but has limited computational power compared to back-end server. Any request from the user first comes to the edge server and if the edge server was unable to serve the request due to the load on it or for any other reason then the request will be forwarded to the cloud where the computations are done and the result is forwarded to the user. Since the edge server is closer to users the latency incurred is much less than in cloud computing.
%We consider a system where the edge servers are rented from resource providers like Amazon, Google, etc., to host the service. There is a cost for forwarding the request to back-end server if it was not served at the edge. Apart from that we consider the cost of fetching a service from back-end to edge server called as \emph{fetch cost}. Since the edge servers are rented for a fixed duration and can change the renting status only after that duration, we consider a time slotted system. We can either host the service or not host the service in each time slot which is decided by the policy we consider. The request arrivals at the edge may or may not follow any distribution, so we need an on-line policy which performs well for any type of request sequence. Another way to look at it is that the policy should perform well for both stochastic and adversarial arrivals.
%To compare different policies and to decide which policy performs better we need some metrics of comparison. The metrics we considered are expected regret and competitive ratio which are widely used in literature to evaluate on-line policies. The regret is defined as the difference between the cost of the policy considered and the optimal static policy (see section \ref{sec:setting} for more details about regret). Smaller the regret better the policy. Competitive ratio is defined as the ratio between the expected cost of the policy and the cost of optimal policy (see section \ref{sec:setting} for more details about competitive ratio). Smaller the competitive ratio better the policy.
%Retro Renting (RR) policy in \cite{narayana2021renting} is one of the on-line policies which fits exactly to the model we considered. It is a deterministic policy which considers both fetch cost and rent cost along with past arrivals till the previous fetch or evict has happened for making decisions. This policy is analyzed by considering competitive ratio as a metric in \cite{narayana2021renting} and shown that as fetch cost increases competitive ratio decreases. Since it is a deterministic policy this policy inccurs high regret in case of adversarial arrivals.
%Follow The Perturbed Leader (FTPL) is another on-line policy which is there in file caching literature and we mapped it to our setting. It is a randomized policy and considers rent cost along with all the requests seen till that point for making decisions and ignores the fetch cost. Since FTPL ignores fetch cost the regret of FTPL policy can be high for large fetch cost due to multiple fetches. The FTPL policy performs well in case of stochastic and adversarial arrivals for small fetch cost. As the fetch cost increases the performance degrades. So we need a policy which performs good in terms of regret for large fetch cost and for stochastic, adversarial arrival processes.
\color{black}
We consider a {system} consisting of a back-end server and an edge-server to serve customers of a service. The back-end server always hosts the service and can serve any requests that are routed to it.
% and if the service is not hosted at the edge, the requests are served by the back-end server.
In this work, we allow the service to be partially hosted at the edge-server. When the service is partially hosted at the edge, requests are partially served at the edge and partially at the back-end server.
%Formally, if $\alpha$ fraction of service is hosted at edge, then $g(\alpha)$ fraction of requests is served by the back-end server and $1-g(\alpha)$ fraction of requests is served at the edge, where $g(\alpha)$ is a decreasing function of $\alpha$ and $g(\alpha)\in [0,1]$.
Further, the fraction of service hosted at the edge can be changed over time.
% In addition, any fraction of the service can be hosted at the edge-server. If a service is hosted at the edge, requests can be served locally, i.e., at the edge. The hosting status at the edge can be changed over time. If we host fraction of service at the edge then part of the request is served locally and part of the request is served by the cloud. If $\alpha$ fraction of service is hosted at edge, then $g(\alpha)$ fraction of request is served by cloud and $1-g(\alpha)$ fraction of service is served locally. Where $g(\alpha)$ is a decreasing function of $\alpha$ and $g(\alpha)\in [0,1]$.
If the service is not hosted or partially hosted at the edge, parts of the service can be fetched from the back-end server to host at the edge.
\emph{Sequence of events in each time-slot}: We consider a time-slotted system. In each time-slot, we first make the service hosting decision for that time-slot. Following this, requests may arrive and are served at the edge and/or the back-end server.
\emph{Request arrivals}: We consider two types of arrival processes in this paper.
\begin{itemize}
\item[-] \emph{Stochastic arrivals}: In this case, request arrivals are i.i.d. across time-slots with mean $\mu$.
\item[-] \emph{Adversarial arrivals}: In this case, the request arrival sequence is generated by an oblivious adversary\footnote{ The entire request sequence is assumed to be fixed apriori.}.
\end{itemize}
Let $r_t$ denote the number of request arrivals in time-slot $t$. We consider bounded requests, specifically, $0\le r_t\le \kappa$, and let $R_t=\sum_{i=1}^{t}r_i$ denote the cumulative requests observed by the end of time-slot $t$. In both cases, we assume that there are at most $\kappa$ arrivals per slot.
\emph{Costs}: We model three types of costs.
\begin{enumerate}
\item[-] \emph{Rent cost} ($\mathcal{C}^{\mathcal{P}}_{R,t}$): The system incurs a cost of $c$ units per time-slot to host the entire service at the edge. For hosting $f$ fraction of service, the system incurs a cost of $cf$ units per time-slot.
\item[-] \emph{Service cost} ($\mathcal{C}^{\mathcal{P}}_{S,t}$): This is the cost incurred to use the back-end server to serve (parts of) requests.
%The system incurs a cost of 1 unit for each request that is served by the back-end server.
If $f$ fraction of service is hosted at the edge, the system incurs a service cost of $g(f)$ units per request, where $g(f)$ is a decreasing function of $f$. We fix $g(0)=1$, i.e., service cost is one unit when the service is not hosted at the edge.
\item[-] \emph{Fetch cost} ($\mathcal{C}^{\mathcal{P}}_{F,t}$): The system incurs a cost of $M> 1$ units for each fetch of the entire service from the back-end server to host on the edge-server. For fetching $f$ fraction of service, the system incurs a cost of $Mf$ units.
\end{enumerate}
Let $\rho_t^{\mathcal{P}}$ denote the edge hosting status of the service in time slot $t$ under policy $\mathcal{P}$. We consider the setting where $\rho_{t}^\mathcal{P} \in \{\alpha_1,\alpha_2, \cdots , \alpha_{K-1},\alpha_{K}\}$ and $\rho_{t}^\mathcal{P}=\alpha_{i}$ implies that $\alpha_{i}$ fraction of the service is hosted at the edge-server in time-slot $t$. Note that $\alpha_{i}\in [0,1]$ and in particular we consider $\alpha_1=0$, $\alpha_K=1$ and $\alpha_{i}<\alpha_{j}$ for $i<j$. It follows that $\rho_t^\mathcal{P}=1$ implies that the entire service is hosted at the edge in time-slot $t$, and $\rho_t^\mathcal{P}=0$ implies that the service is not hosted at the edge in time-slot $t$.
We make some assumptions about the various system parameters.
\begin{assumption} \label{assump:k<c}
$c\le \kappa$.
\end{assumption}
This assumption is motivated by the fact that if $\kappa < c$, it is strictly sub-optimal to host the entire service at the edge, irrespective of the number of arrivals in a time-slot.
\begin{assumption}\label{assump:g(f)_convex}
For a fraction of service $\alpha_{i}$, $ 1\le i\le K$, $\alpha_{i}+g(\alpha_{i})\le 1 $.
\end{assumption}
In \cite{prakash2020partial, LN_partial_hosting}, it is shown that the benefits of partial hosting are limited to the setting where $\alpha_{i}+g(\alpha_{i})\le 1 $. Motivated by this, we consider the case where $\alpha_{i}+g(\alpha_{i})\le 1 $ for all candidate values of $\alpha_{i}$.
\begin{assumption} \label{assump:cost<k}
For any fraction of service $\alpha_{i}$, $1\le i\le K$,
$c\alpha_{i}+g(\alpha_{i})r_t\le \kappa$
\end{assumption}
Assumption \ref{assump:cost<k} follows from Assumptions \ref{assump:k<c} and \ref{assump:g(f)_convex}
\begin{assumption}
For stochastic arrivals, $\mu>0$ and for adversarial arrivals, $R_T\ge 1$.
\end{assumption}
This assumption eliminates degenerate cases where there are no request arrivals in the time-horizon of interest.
%We make Assumption \ref{assump:g(f)_convex} because for $\alpha_{i}+g(\alpha_{i})>1$, $\alpha_{i}$ can not be the optimal fraction of service to host \cite{prakash2020partial, LN_partial_hosting}. Assumption \ref{assump:k<c} is motivated by the fact that if $\kappa < c$, it is strictly sub-optimal to host the entire service at the edge irrespective of the number of arrivals in a time-slot. Assumption \ref{assump:cost<k} follows from Assumption \ref{assump:g(f)_convex}, \ref{assump:k<c}.
The total cost incurred in time-slot $t$ by policy $\mathcal{P}$, denoted by $\mathcal{C}^{\mathcal{P}}_t(r_t)$, is the sum of the rent, service, and fetch costs. It follows that,
\begin{align*}
\mathcal{C}^{\mathcal{P}}_t(r_t)%&=\mathcal{C}^{\mathcal{P}}_{R,t} + \mathcal{C}^{\mathcal{P}}_{S,t} + +\mathcal{C}^{\mathcal{P}}_{F,t}\\
&=c\rho_{t}^{\mathcal{P}}+ g(\rho_{t}^{\mathcal{P}}) r_t+ M (\rho_{t}^{\mathcal{P}}-\rho_{t-1}^{\mathcal{P}})^+. \numberthis \label{eq:cost_slot}
\end{align*}
Let $r=\{r_t\}_{t\ge1}$ denote the request arrival sequence and {$\mathcal{C}^{\mathcal{P}}(T,r)$} denote the cumulative cost incurred by policy $\mathcal{P}$ in time-slots 1 to $T$. It follows that,
$$
\mathcal{C}^{\mathcal{P}}(T,r) = \sum_{t=1}^T \mathcal{C}^{\mathcal{P}}_t(r_t).
$$
\emph{Performance metrics}: We consider two performance metrics, namely regret and competitive ratio.
For i.i.d. stochastic arrivals, the regret of a policy $\mathcal{P}$, denoted by $\mathcal{R}^{\mathcal{P}}_S(T)$, %\E_{\mathcal{P},r}[\text{Regret}^{\mathcal{P}}_T]$,
is defined as the difference in the total expected cost incurred by the policy and the oracle static hosting policy. The oracle static hosting policy makes a hosting decision at $t=1$ using the knowledge of the statistics of the request arrival process but not the entire sample-path. For ease of notation, we define $\mu=\E[r_t]$, $\mu_i=c\alpha_i+g(\alpha_i)\mu$. It follows that the expected cost incurred by the optimal static hosting policy in time-slots 1 to $T$ is $\min_i\{c\alpha_iT+g(\alpha_i)\mu T+M\alpha_i\}$, and therefore,
\begin{align*}
\mathcal{R}^{\mathcal{P}}_S(T)&= \E_{\mathcal{P},r}[\mathcal{C}^\mathcal{P}(T,r)] -\min_i\{c\alpha_iT+g(\alpha_i)\mu T+M\alpha_i\}\\
&=\E_{\mathcal{P},r}[\mathcal{C}^\mathcal{P}(T,r)] -\min_i\{\mu_i T+M\alpha_i\},\\
& \le \E_{\mathcal{P},r}[\mathcal{C}^\mathcal{P}(T,r)] -\min_i\{\mu_i T\}, \numberthis \label{eq:Regret_stch}
\end{align*}
where $\E_{\mathcal{P},r}$ represents the expectation over the randomization in policy $\mathcal{P}$ and the request arrival sequences $r$.
For i.i.d. stochastic arrivals, the competitive ratio of a policy $\mathcal{P}$, denoted by $\sigma^{\mathcal{P}}_S(T)$, %\E_{\mathcal{P},r}[\text{Regret}^{\mathcal{P}}_T]$,
is defined as the ratio of the expected total cost incurred by the policy and expected total cost incurred by the optimal static hosting policy as discussed above. It follows that,
\begin{align*}
\sigma_{S}^{\mathcal{P}}(T)= \frac{\E_{\mathcal{P},r}[\mathcal{C}^{\mathcal{P}}(T,r)]}{\min_i\{c\alpha_iT+g(\alpha_i)\mu T+M\alpha_i\}}.
\end{align*}
For adversarial arrivals, the regret of a policy $\mathcal{P}$, denoted by $\mathcal{R}^{\mathcal{P}}_A(T)$, is defined as the worst case difference in the total expected cost incurred by the policy and the optimal static hosting policy. The expectation is taken over the randomization in policy $\mathcal{P}$. It follows that for any request sequence $r$, the total cost incurred by the optimal static hosting policy in time-slots 1 to $T$ is $\min_i\{c\alpha_iT+g(\alpha_i)R_T+M\alpha_i\}$, and therefore,
\begin{align*}
% \mathcal{C}^{\text{OPT-S}}_{T}(r)&=\min\{cT+M, \sum_{t=1}^T r_t\},\\
% %\E_{\mathcal{P}}[\text{Regret}^\mathcal{P}_T(r)]&=\E_\mathcal{P}[\mathcal{C}^\mathcal{P}_T(r)] -\mathcal{C}^{\text{OPT-S}}_{T}(r)\\
%\text{and therefore, }
\mathcal{R}^{\mathcal{P}}_A(T,r)&=\E_\mathcal{P}[\mathcal{C}^\mathcal{P}(T,r)] - \min_i\{c\alpha_iT+g(\alpha_i)R_T+M\alpha_i\},\\
\mathcal{R}^{\mathcal{P}}_A(T)&=\sup_{r} \left(\mathcal{R}^{\mathcal{P}}_A(T,r) \right). %\numberthis \label{eq:Regret_any_policy}
\end{align*}
We have the following relation between the regret induced by a policy under i.i.d stochastic arrivals and adversarial arrivals.
\begin{lemma}\label{lem:adv_stch_reg}
For any online policy $\mathcal{P}$ and stochastic arrivals we have,
$\mathcal{R}_S^\mathcal{P}(T)\le \mathcal{R}_A^\mathcal{P}(T)$.
\end{lemma}
The proof can be found in Section \ref{sec:appendix}.
\color{black}
For adversarial arrivals, the competitive ratio of a policy $\mathcal{P}$, denoted by $\sigma^{\mathcal{P}}_A(T)$, is defined as the worst case ratio of the expected total cost incurred by the policy and the cost incurred by the offline optimal policy. The optimal offline policy can change the hosting status over time using the knowledge of the entire request process a priori. Let the cost incurred by the optimal offline hosting policy in time-slots 1 to $T$ be denoted by $\mathcal{C}^{\text{OFF-OPT}}(T,r)$. It follows that,
\begin{align*}
% \mathcal{C}^{\text{OPT-S}}_{T}(r)&=\min\{cT+M, \sum_{t=1}^T r_t\},\\
% %\E_{\mathcal{P}}[\text{Regret}^\mathcal{P}_T(r)]&=\E_\mathcal{P}[\mathcal{C}^\mathcal{P}_T(r)] -\mathcal{C}^{\text{OPT-S}}_{T}(r)\\
%\text{and therefore, }
\sigma^{\mathcal{P}}_A(T)=\sup_{r} \frac{\E_{\mathcal{P}}[\mathcal{C}^{\mathcal{P}}(T,r)]}{\mathcal{C}^{\text{OFF-OPT}}(T,r)}.% \numberthis \label{eq:comp_any_policy}
\end{align*}
\emph{Goal}: The goal is to design online hosting policies with provable performance guarantees with respect to regret and competitive ratio.
\emph{Notations}: We define some notations which will be used in the rest of this paper. Let $\bm{s}=[0 , \alpha_2, \alpha_3, \cdots, 1]'$, $\bm{f}=[1,g(\alpha_2), g(\alpha_3), \cdots, 0]'$, and $\bm{\rho}_t^{\mathcal{P}}\in \mathcal{X}$, where $\mathcal{X}$ is the collection of all possible one hot vectors in $\{0,1\}^K$ and thus $|\mathcal{X}|=K$. The position of 1 in the one hot vector $\bm{\rho}_t^{\mathcal{P}}$ represents the level of service hosted at the edge in slot $t$, that is $\rho_t^{\mathcal{P}}=\langle \bm{\rho}_t^{\mathcal{P}},\mathbf{s} \rangle$.
%$$ \mathbf{s}=\begin{bmatrix} 0 \\ 1 \end{bmatrix}, \ \mathbf{f}=\begin{bmatrix} 1 \\ 0 \end{bmatrix}.$$
The total cost in slot $t$ as given in \eqref{eq:cost_slot} can be rewritten using the above notations as follows,
\begin{align*}
\mathcal{C}_{t}^{\mathcal{P}}(r_t) &= c\langle \bm{\rho}_t^{\mathcal{P}},\bm{s} \rangle + r_t \langle \bm{\rho}_t^{\mathcal{P}},\bm{f} \rangle + M (\rho_t^{\mathcal{P}}-\rho_{t-1}^{\mathcal{P}})^+,\\
%&=\langle \bm{\rho}_t,c\mathbf{s}+r_t\mathbf{f} \rangle + M (\rho_t-\rho_{t-1})^+, \\
&=\langle \bm{\rho}_t^{\mathcal{P}},\bm{\theta}_{t} \rangle + M (\rho_t^{\mathcal{P}}-\rho_{t-1}^{\mathcal{P}})^+,
\end{align*}
where $\bm{\theta}_t= c\bm{s}+r_t\bm{f}$.
%We define $\bm{\Theta}_{t+1}=\sum_{l=1}^t\bm{\theta}_l$, $\Theta_{t+1,i}=c\alpha_{i}t+g(\alpha_{i})R_t$.
For ease of notation, for stochastic arrivals we define $\mu = \E[r_t]$, $\mu_i=c\alpha_{i}+g(\alpha_{i})\mu$, $\Delta_{ij}=\mu_i-\mu_j$, $i^\star=\argmin_i \mu_i$ , $\Delta_i=\mu_i-\mu_{i^\star}$, $\Delta_{\text{min}}=\min_i \Delta_{i}$, $\Delta_{\text{max}}=\max_i \Delta_{i}$.
We summarize important notations used in this paper in \tablename~\ref{tab:notations}.
\color{black}
\begin{table}
\begin{tabular}{|c|l|}
\hline
Symbol & Description \\
\hline \hline
$c$ & Rent cost for hosting complete service in a slot \\
\hline
$M$ & Fetch cost \\
\hline
$K$ & Number of hosting levels allowed\\
\hline
$t$ & Time index\\
\hline
$\rho_{t}^\mathcal{P}$ & Fraction of service hosted in time slot $t$ under policy $\mathcal{P}$\\
\hline
$\kappa$ & Maximum number of arrivals in a slot\\
\hline
$\alpha_i$ & Fraction of service \\
\hline
$g(f)$ & Fraction of request forwarded if $f$ fraction is hosted \\
\hline
$r_t$ & Number of requests in time slot $t$\\
\hline
$R_t$ & Cumulative number of requests up to time slot $t$\\
\hline
$\mu$ & $\E[r_t]$\\
\hline
$T$ & Time horizon\\
\hline
$\mathcal{C}^\mathcal{P}(T,r)$ & Cost incurred till $T$ under policy $\mathcal{P}$ for request sequence $r$ \\
\hline
$\mathcal{R}_S^\mathcal{P}(T)$ & Regret under stochastic arrival setting\\
\hline
$\sigma_{S}^\mathcal{P}(T)$ & Competitive ratio under stochastic arrival setting\\
\hline
$\mathcal{R}^\mathcal{P}_A(T)$ & Regret under adversarial arrival setting\\
\hline
$\sigma_{A}^\mathcal{P}(T)$ & Competitive ratio under adversarial arrival setting\\
\hline
$\mu_i$ & $c\alpha_{i}+g(\alpha_{i})\mu$\\
\hline
$\Delta_{ij}$ & $\mu_i-\mu_j$\\
\hline
$i^\star$ & $\argmin_i \mu_i$ \\
\hline
$\Delta_{i}$ & $\mu_i-\mu_{i^\star}$ \\
\hline
$\Delta_{\text{min}}$ & $\min_i \Delta_i$\\
\hline
$\Delta_{\text{max}}$ & $\max_i \Delta_i$\\
\hline
$\bm{s}$ & $[0 , \alpha_2, \alpha_3, \cdots, 1]'$\\
\hline
$\bm{f}$ & $[1,g(\alpha_2), g(\alpha_3), \cdots, 0]'$\\
\hline
$\bm{\theta}_{t}$ & $c\bm{s}+r_t\bm{f}$\\
\hline
$\bm{\Theta}_{t}$ & $\sum_{i=1}^{t-1}\bm{\theta}_i$\\
\hline
$\eta_{t}$ & Learning rate\\
\hline
$\alpha$ & Hyper parameter in FTPL, W-FTPL policies\\
\hline
$\beta$, $\delta$ & Hyperparameters in W-FTPL policy\\
\hline
$\bm{\gamma}$ & IID Gaussian vector\\
\hline
$\mathcal{X}$ & Collection of one hot vectors in $\{0,1\}^K$\\
\hline
$T_s$ & Waiting time for W-FTPL policy\\
\hline
\end{tabular}
\caption{Notations}
\label{tab:notations}
\end{table}
%The network we considered consists of a back-end server which hosts the service and an edge server which caches the service close to the end-user by paying some rent cost. Any query request from end-user comes to edge server. If the request is not served at the edge, then it will be forwarded to the back-end server where it will execute the query and the result is sent back to user. If the query is served at the edge, then we call the request is served locally.
%\subsection{Arrival process}
%In this paper we consider two types of arrivals one is adversarial and other is stochastic. We consider adversarial arrivals are generated by an oblivious adversary, which means the request sequence is fixed and chosen at beginning of slot $t=1$ and cannot be changed. Where as in stochastic arrivals the requests are generated in Independent Identical Distribution (IID) manner with mean $\mu$.
%Let $r_t$ be the number of requests observed at edge from end users in time slot $t\ (\ge1)$. Let $R_t=\sum_{i=1}^{t-1} r_i$ which represents the cumulative requests till time $t$. We also consider that at most one request can be served in each time slot, i.e., $r_t\le 1$.
%\subsection{Sequence of events}
%In each time slot first we decide whether to host or not to host the service at edge and then the requests in the slot arrive. The cost is calculated at the end of each slot before taking decision of hosting for next slot.
%\subsection{Metrics}
%Let $c$ be the cost of hosting the service at edge in a slot. Let $M$ be the cost of fetching the service. The cost of forwarding a request to cloud is 1 unit and cost of serving locally at edge is 0 units. Let $\rho_t$ represents the status of service hosted at edge in time slot $t$, $\rho_t\in \{0,1\}$. $\rho_t=0$ mean service is not hosted at edge, $\rho_{t}=1$ mean service is hosted at the edge. We only consider $c<1$ because if $c>1$ it is optimal to forward in each slot, similarly we consider $M>1$ since cost of forwarding is 1 unit.
%\subsubsection{Cost}
% The cost incurred in slot $t$ for hosting status $\rho_{t}$ is the sum of rent cost ($\mathcal{C}_{R,t}$) for hosting the service at edge, forwarding cost($\mathcal{C}_{S,t}$) for forwarding the requests that re not served locally and fetch cost ($\mathcal{C}_{F,t}$) if the service is fetched from back-end.
%\begin{align}
%\mathcal{C}_t(\rho_{t},r_t)&= \mathcal{C}_{R,t}+\mathcal{C}_{S,t}+\mathcal{C}_{F,t}\\
%&=c\rho_{t}+ (1-\rho_{t}) r_t+ M (\rho_{t}-\rho_{t-1})^+.\label{eq:cost_in_slot}
%\end{align}
%The overall cost incurred is sum of the costs in each slot. The cost incurred in a slot depends on the policy considered which decides the sequence $\{\rho_t\}_{t\ge1}$. Cost also depends on the request arrival sequence $r=\{r_t\}_{t\ge1}$. In general the overall cost at the end of time slot $T$ under policy $\mathcal{P}$ is
%$$\mathcal{C}^\mathcal{P}_T(r)=\sum_{t=1}^{T}\mathcal{C}_t(\rho_t,r_t).$$
%Let $\bm{\rho}_t\in \mathcal{X}$, where $\mathcal{X}= \{ [1, 0]', [0 , 1]' \}$, represents the one hot vector representing status of service hosted near edge. We define locally served vector $\mathbf{s}=[0 ,1]'$, request forward vector $\mathbf{f}=[1,0]'$.
%$$ \mathbf{s}=\begin{bmatrix} 0 \\ 1 \end{bmatrix}, \ \mathbf{f}=\begin{bmatrix} 1 \\ 0 \end{bmatrix}.$$
%Note that $\rho_t=\langle \bm{\rho}_t,\mathbf{s} \rangle$. Now the cost in slot $t$ given in \eqref{eq:cost_in_slot} can also be represented as
%\begin{align*}
%\mathcal{C}_{t}(r_t) &= c\langle \bm{\rho}_t,\bm{s} \rangle + r_t \langle \bm{\rho}_t,\bm{f} \rangle + M (\rho_t-\rho_{t-1})^+,\\
%&=\langle \bm{\rho}_t,c\mathbf{s}+r_t\mathbf{f} \rangle + M (\rho_t-\rho_{t-1})^+, \\
%&=\langle \bm{\rho}_t,\bm{\theta}_{t} \rangle + M (\rho_t-\rho_{t-1})^+,
%\end{align*}
%where $\bm{\theta}_t= c\bm{s}+r_t\bm{f}=[r_t, c]'$.
%\subsubsection{Regret}
%We define regret of a policy $\mathcal{P}$ as the difference in cost incurred by the policy $\mathcal{P}$ and the cost under optimal static policy. Let the cost of optimal static policy at the end of time $T$ for adversarial request arrival sequence $r$ be $\mathcal{C}^{\text{OPT-S}}_{T}(r)$ where,
%\begin{align*}
%\mathcal{C}^{\text{OPT-S}}_{T}(r)&=\min\{cT+M, \sum_{t=1}^T r_t\}, \\
%\E_{\mathcal{P}}[\text{Regret}^\mathcal{P}_T(r)]&=\E_\mathcal{P}[\mathcal{C}^\mathcal{P}_T(r)] -\mathcal{C}^{\text{OPT-S}}_{T}(r)\\
%\E_{\mathcal{P}}[\text{Regret}^\mathcal{P}_T]&=\sup_{r\in \mathcal{R}}\E_{\mathcal{P}}[\text{Regret}^\mathcal{P}_T(r)], \numberthis \label{eq:Regret_any_policy}
%\end{align*}
%where $\mathcal{R}$ represents all possible request arrival sequences, $\E_\mathcal{P}$ represents the expectation with respect to policy $\mathcal{P}$.
%In case of stochastic arrivals since arrivals in a slot follow a distribution with mean $\mu=\E[r_t]$ we consider the expected cost of static optimal policy with respect to arrival sequence $r$ at the end of time slot $T$ denoted by $\E_r[\mathcal{C}_T^{\text{OPT-S}}]= \min\{cT+M,\mu T\}$. Therefore the expected regret is given by,
%\begin{align*}
%\E_{\mathcal{P},r}[\text{Regret}^\mathcal{P}_T]&= \E_{\mathcal{P},r}[\mathcal{C}^\mathcal{P}_T(r)] -\E_r[\mathcal{C}^{\text{OPT-S}}_{T}]. \numberthis \label{eq:Regret_stch}
%\end{align*}
%\subsubsection{Competitive ratio}
%We consider two types of competitive ratio based on arrivals. Firstly for the adversarial arrivals, we define competitive ratio of any policy as the supremum of ratio between the expected cost incurred by the policy and the optimal off-line policy.
%\begin{align*}
%\sigma_{d,T}^{\mathcal{P}}=\sup_{r\in \mathcal{R}} \frac{\E_{\mathcal{P}}[\mathcal{C}_T^{\mathcal{P}}(r)]}{\mathcal{C}_T^{\text{OPT-D}}(r)}.
%\end{align*}
%Note that optimal off-line policy is not a static policy. It can change the decision in each time slot based on the arrivals seen in past. For stochastic arrivals we define competitive ratio of any policy as the ratio of expected cost incurred by the policy and the expected cost of optimal static policy.
%\begin{align*}
%\sigma_{s,T}^{\mathcal{P}}= \frac{\E_{\mathcal{P},r}[\mathcal{C}_T^{\mathcal{P}}(r)]}{\E_r[\mathcal{C}_T^{\text{OPT-S}}]}.
%\end{align*}
%We also denote $\lim\limits_{T\to \infty}\sigma^{\mathcal{P}}_{s,T}$ as $\sigma^{\mathcal{P}}_s$, and $\lim\limits_{T\to \infty}\sigma^{\mathcal{P}}_{d,T}$ as $\sigma^{\mathcal{P}}_d$.
%\subsection{Goal}
%Our goal is to compare an analyze regret, competitive ratio of various policies mentioned in Section \ref{sec:policies}.
%\remove{We consider a time slotted system. In each slot we will observe some requests from end users to edge. We serve the request at edge if the service is hosted at edge else we will forward the request to cloud. Our goal is to find when the service be hosted at edge, when it is not.
%
%Let $c$ be the cost of hosting the entire service at edge in a slot. Let $M$ be the cost of fetching the entire service. The cost of forwarding a request to cloud is 1 unit. Let $r_t$ be the number of requests observed at edge from end users in time slot $t\ (\ge1)$. Let $R_t=\sum_{i=1}^{t-1} r_i$ which represents the cumulative requests till time $t$. We also consider that at most one request can be served in each time slot, i.e., $r_t\le 1$.
%
%In each time slot first we decide whether to host or not to host the service at edge and then the requests in the slot arrive. The cost is calculated at the end of each slot before taking decision of hosting for next slot. Let $\rho_t$ represents the status of service hosted at edge in time slot $t$, $\rho_t\in \{0,1\}$. $\rho_t=0$ mean service is not hosted at edge, $\rho_{t}=1$ mean service is hosted at the edge. The cost incurred in slot $t$ for hosting status $\rho_{t}$ is the sum of rent cost ($\mathcal{C}_{R,t}$), forwarding cost($\mathcal{C}_{S,t}$) and fetch cost ($\mathcal{C}_{F,t}$), given by.
%
%\begin{align}
%\mathcal{C}_t(\rho_{t},r_t)&= \mathcal{C}_{R,t}+\mathcal{C}_{S,t}+\mathcal{C}_{F,t}\\
%&=c\rho_{t}+ (1-\rho_{t}) r_t+ M (\rho_{t}-\rho_{t-1})^+.\label{eq:cost_in_slot}
%\end{align}
%
%The overall cost incurred is sum of the costs in each slot. The cost incurred in a slot depends on the policy considered which decides the sequence $\{\rho_t\}_{t\ge1}$. Cost also depends on the request arrival sequence $r=\{r_t\}_{t\ge1}$. In general the overall cost at the end of time slot $T$ under policy $\mathcal{P}$ is
%$$\mathcal{C}^\mathcal{P}_T(r)=\sum_{t=1}^{T}\mathcal{C}_t(\rho_t,r_t).$$
%
%Let $\bm{\rho}_t\in \mathcal{X}$, where $\mathcal{X}= \left\{\begin{bmatrix} 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \end{bmatrix}\right \}$, represents the one hot vector representing status of service hosted near edge. We define locally served vector $\mathbf{s}=[0 ,1]'$, request forward vector $\mathbf{f}=[1,0]'$.
%%$$ \mathbf{s}=\begin{bmatrix} 0 \\ 1 \end{bmatrix}, \ \mathbf{f}=\begin{bmatrix} 1 \\ 0 \end{bmatrix}.$$
% Observe that $\rho_t=\langle \bm{\rho}_t,\mathbf{s} \rangle$. Now the cost in slot $t$ given in \eqref{eq:cost_in_slot} can also be represented as
%
%\begin{align*}
%\mathcal{C}_{t}(r_t) &= c\langle \bm{\rho}_t,\bm{s} \rangle + r_t \langle \bm{\rho}_t,\bm{f} \rangle + M (\rho_t-\rho_{t-1})^+,\\
%%&=\langle \bm{\rho}_t,c\mathbf{s}+r_t\mathbf{f} \rangle + M (\rho_t-\rho_{t-1})^+, \\
%&=\langle \bm{\rho}_t,\bm{\theta}_{t} \rangle + M (\rho_t-\rho_{t-1})^+,
%\end{align*}
%where $\bm{\theta}_t= c\bm{s}+r_t\bm{f}=\begin{bmatrix} r_t\\c \end{bmatrix}$.
%
%In this paper we consider two types of arrivals one is adversarial and other is stochastic. We consider oblivious adversary, which means the request sequence is fixed and chosen at beginning of slot $t=$ and cannot be changed. Where as in stochastic arrivals the requests are generated in Independent Identical Distribution (IID) manner with mean $\mu$ i.e., $\E[r_t]=\mu$.
%
%%So we consider $\E[\mathcal{C}^\text{OPT}_{s,T}(r)] =\min\{cT,\mu T\}$.
%
%We define regret of a policy $\mathcal{P}$ as the difference in cost incurred by the policy $\mathcal{P}$ and the cost under optimal static policy. Let the cost of optimal static policy at the end of time $T$ for adversarial request arrival sequence $r$ be $\mathcal{C}^{\text{OPT-S}}_{T}(r)$ where,
%\begin{align*}
%\mathcal{C}^{\text{OPT-S}}_{T}(r)&=\min\{cT, \sum_{t=1}^T r_t\}, \\
%\E_{\mathcal{P}}[\text{Regret}^\mathcal{P}_T(r)]&=\E_\mathcal{P}[\mathcal{C}^\mathcal{P}_T(r)] -\mathcal{C}^{\text{OPT-S}}_{T}(r)\\
%\E_{\mathcal{P}}[\text{Regret}^\mathcal{P}_T]&=\sup_{r\in \mathcal{R}}\E_{\mathcal{P}}[\text{Regret}^\mathcal{P}_T(r)], \numberthis \label{eq:Regret_any_policy}
%\end{align*}
%where $\mathcal{R}$ represents all possible request arrival sequences, $\E_\mathcal{P}$ represents the expectation with respect to policy $\mathcal{P}$.
%
%In case of stochastic arrivals since arrivals in a slot follow some distribution with mean $\mu$ we consider the expected cost of static optimal policy with respect to arrival sequence $r$ at the end of time slot $T$ denoted by $\E_r[\mathcal{C}_T^{\text{OPT-S}}]= \min\{cT,\mu T\}$. Therefore the expected regret is given by,
%\begin{align*}
%\E_{\mathcal{P},r}[\text{Regret}^\mathcal{P}_T]&= \E_{\mathcal{P},r}[\mathcal{C}^\mathcal{P}_T(r)] -\E_r[\mathcal{C}^{\text{OPT-S}}_{T}]. \numberthis \label{eq:Regret_stch}
%\end{align*}
%
%We consider two types of competitive ratio based on arrivals. Firstly for the adversarial arrivals, we define competitive ratio of any policy as the supremum of ratio between the expected cost incurred by the policy and the optimal off-line policy.
%\begin{align*}
%\sigma_{d,T}^{\mathcal{P}}=\sup_{r\in \mathcal{R}} \frac{\E_{\mathcal{P}}[\mathcal{C}_T^{\mathcal{P}}(r)]}{\mathcal{C}_T^{\text{OPT-D}}(r)}
%\end{align*}
%Note that optimal off-line policy is not a static policy. It can change the decision in each time slot based on the arrivals seen in past. For stochastic arrivals we define competitive ratio of any policy as the ratio of expected cost incurred by the policy and the expected cost of optimal static policy.
%\begin{align*}
%\sigma_{s,T}^{\mathcal{P}}= \frac{\E_{\mathcal{P},r}[\mathcal{C}_T^{\mathcal{P}}(r)]}{\E_r[\mathcal{C}_T^{\text{OPT-S}}]}.
%\end{align*}
% We also denote $\lim\limits_{T\to \infty}\sigma^{\mathcal{P}}_{s,T}$ as $\sigma^{\mathcal{P}}_s$, and $\lim\limits_{T\to \infty}\sigma^{\mathcal{P}}_{d,T}$ as $\sigma^{\mathcal{P}}_d$.
%}
%
%\remove{The competitive ratio of a policy $\mathcal{P}$ is defined as the ratio between the expected cost under policy $\mathcal{P}$ and the optimal policy.
%We consider two types of optimal policies one is static optimal and another is off-line optimal. The optimal policy considered gets clear by the notation we use. Off-line optimal policy knows all the request sequence $r$ and chooses the best action possible in time slot $t$. The cost of the off-line optimal policy is denoted by $\mathcal{C}_{d,T}^{\text{OPT}}(r)$. Note that static optimal policy either hosts or does not host the service throughout. Where as off-line optimal policy can change the hosting decisions in each time slot. Therefore $\mathcal{C}_{d,T}^{\text{OPT}}(r) \le \mathcal{C}_{s,T}^{\text{OPT}}(r)$.
%
%We define competitive ratio of any policy $\mathcal{P}$ with respect to off-line optimal policy as
%$$\sigma^\mathcal{P}_{d,T}=\sup_{r\in \mathcal{R}}\frac{\E[\mathcal{C}^\mathcal{P}_T (r)]}{\mathcal{C}^{\text{OPT}}_{d,T}(r)},$$
%where $\mathcal{R}$ represents all possible request arrival sequences. Similarly we define competitive ratio of any policy $\mathcal{P}$ with respect to static optimal policy as
%$$\sigma^\mathcal{P}_{s,T}=\sup_{r\in \mathcal{R}}\frac{\E[\mathcal{C}^\mathcal{P}_T(r)]}{\E[\mathcal{C}^{\text{OPT}}_{s,T}(r)]},\\
%$$
%where $\mathcal{R}$ represents all possible request arrival sequences. Since cost of off-line optimal policy is less than or equal to the cost of static optimal policy we have $\sigma^{\mathcal{P}}_{s,T} \le \sigma^{\mathcal{P}}_{d,T}$. We also denote $\lim\limits_{T\to \infty}\sigma^{\mathcal{P}}_{s,T}$ as $\sigma^{\mathcal{P}}_s$, and $\lim\limits_{T\to \infty}\sigma^{\mathcal{P}}_{d,T}$ as $\sigma^{\mathcal{P}}_d$.
%}
\emph{Our Policy}: Our policy called Wait then Follow the Perturbed Leader (W-FTPL), is a variant of the popular FTPL policy. A formal definition of FTPL is given in Algorithm \ref{alg:FTPL}. FTPL is a randomized policy and is known to perform well for the traditional caching problem, in fact achieving order-wise optimal regret for adversarial arrivals \cite{bhattacharjee2020fundamental}. Under FTPL, in any time slot $t$, for each $i$, FTPL considers the cumulative cost that would have been incurred had the system used $\alpha_i$ hosting fraction in the entire duration $[1, t -1]$, and then perturbs this by an appropriately scaled version of a sample of a standard Gaussian random variable. FTPL then hosts the fraction of service for which the perturbed cost is minimum in that slot.
%Under FTPL, in each time-slot, the cumulative request arrival count is perturbed by an appropriately scaled version of a sample of a standard Gaussian random variable. FTPL hosts the service in a time-slot, if and only if, for this perturbed cumulative arrival count, the total cost incurred if the service was hosted from time-slots 1 to $t$ is less than the total cost incurred if the service was not hosted through this duration.
%\color{violet} Another way of considering FTPL is to add
%\color{black}\right)
\begin{algorithm}
\caption{Follow The Perturbed Leader (FTPL)}
\label{alg:FTPL}
\KwIn{$c$, $\{\eta_t\}_{t\ge 1}$, $\{r_t\}_{t\ge 1}$}
Set $\bm{\Theta}_{1} \leftarrow \bm{0}$\\
Sample $\bm{\gamma} \sim \mathcal{N}(\bm{0}, \bm{I})$\\
\For{$t=1$ to $T$}{
host $\bm{\rho}_t\in \argmin_{\bm{\rho} \in \mathcal{X}} \langle \bm{\rho}, \bm{\Theta}_t +\eta_t\bm{\gamma} \rangle $\\
% Cost incurred is given by \eqref{eq:cost_in_slot}. \\
$\bm{\theta}_{t}=c\bm{s}+r_t\bm{f}$\\
Update $\bm{\Theta}_{t+1}=\bm{\Theta}_t+ \bm{\theta}_{t}$}
\end{algorithm}
We propose a policy called Wait then Follow The Perturbed Leader (W-FTPL). The key idea behind the W-FTPL policy is to not host the service for an initial wait period. This is to reduce the number of fetches made initially when the estimate of the arrivals is noisy. The duration of this wait period is not fixed apriori and is a function of the arrival pattern seen till that time. Following the wait period, W-FTPL mimics the FTPL policy. Refer to Algorithm \ref{alg:W-FTPL} for a formal definition of W-FTPL. Let $T_s$ be the time slot, after which W-FTPL starts acting as FTPL. Formally we define $T_s=\min\{t:t<\frac{(\max_{i\ne j}(\Theta_{t+1,i}-\Theta_{t+1,j}))^2}{\kappa^2\beta (\log M)^{1+\delta}}\}$, where $\beta>1$, $\delta>0$ are constants.
% that is when wait=0 occurs in Algorithm \ref{alg:W-FTPL}
\begin{algorithm}
\caption{Wait then Follow The Perturbed Leader (W-FTPL)}
\label{alg:W-FTPL}
\KwIn{$c$, $M$, $\{\eta_t\}_{t\ge 1}$, $\{r_t\}_{t\ge1}$, $\beta$, $\delta$, $\kappa$}
Set $\bm{\Theta_{1}} \leftarrow \bm{0}$, wait=1\\
Sample $\bm{\gamma} \sim \mathcal{N}(\bm{0}, \bm{I})$\\
\For{$t=1$ to $T$}
{ \eIf{$\text{wait}=1$}{$\rho_t=0$}{
host $ \bm{\rho}_t\in \argmin_{\bm{\rho} \in \mathcal{X}} \langle \bm{\rho}, \bm{\Theta}_t +\eta_t\bm{\gamma} \rangle $}
$\bm{\theta}_{t}=c\bm{s}+r_t\bm{f}$\\
Update $\bm{\Theta}_{t+1} = \bm{\Theta}_{t} + \bm{\theta}_t$\\
$\text{wait} = \min\{\text{wait},\mathbbm{1}_{t\ge \frac{(\max_{i\ne j}(\Theta_{t+1,i}-\Theta_{t+1,j}))^2}{\kappa^2\beta (\log M)^{1+\delta}}}\}$
% \eIf{$\text{wait} = 1$}{
% $\rho_t = 0$
% }{
%\color{red} $\rho_t = \mathbbm{1}_{R_t + \eta_t \gamma_1 > ct+\eta_{t}\gamma_2}$
%}
% \While{(1)}{
% \eIf{$t>\frac{\beta \log M}{(c-R_{t}/t)^2}$}{
% $T_s=t$\\
% break;
% }{$\rho_{t}=0$}
% Update $\bm{\Theta_{t+1}}=\bm{\Theta_{t}}+ \bm{\theta_{t}$}, $R_{t+1}=R_{t}+r_t$, $t = t+1$
% }
% Sample $\bm{\gamma} \sim \mathcal{N}(\bm{0}, \bm{I})$.\\
% \For{$t=T_s$ to $T$}{
% $\bm{\rho}_t\in \argmin_{\bm{\rho} \in \mathcal{X}} \langle \bm{\rho}, \bm{\Theta_t} +\eta_t\bm{\gamma} \rangle $.\\
% Cost incurred is given by \eqref{eq:cost_in_slot}.\\
% Update $\bm{\Theta_{t+1}}=\bm{\Theta_t}+ \bm{\theta_{t}$}}
}
\end{algorithm}
\emph{Retro-Renting (RR)} \cite{narayana2021renting}: The RR policy is a deterministic hosting policy, and it either hosts the complete service or hosts nothing in a slot. The key idea behind this policy is to use recent arrival patterns to make hosting decisions. We refer the reader to Algorithm 1 in \cite{narayana2021renting} for a formal definition of the RR policy. The performance of RR with respect to the competitive ratio was analyzed in \cite{narayana2021renting}.
$\alpha-$RR \cite{LN_partial_hosting} is a variant of RR where one partial level of hosting the service is allowed. The formal definition of $\alpha-$RR, borrowed from \cite[Algorithm 1]{LN_partial_hosting} is given in Algorithm \ref{alg:alpha_RR}. Similar to RR, the key idea behind $\alpha-$RR policy is to check whether the current hosting status is optimal in the hindsight using recent arrival patterns to make hosting decisions. $\alpha-$RR checks the optimal offline policy cost for the recent request arrivals and chooses the fraction of service with the minimum cost. For $\rho_{t}^\mathcal{P}\in\{0,1\}$, $\alpha-$RR behaves as RR \cite{LN_partial_hosting}. Therefore we refer $\alpha-$RR as RR in this paper.
\begin{algorithm}
\caption{$\alpha$-RetroRenting ($\alpha$-RR)}\label{alg:alpha_RR}
\SetAlgoLined
\SetKwFunction{FtotalCost}{totalCost}
\SetKwProg{Fn}{Function}{:}{end}
Input: Fetch cost $M$, partial hosting level $\alpha_2$, latency cost under partial hosting $g(\alpha_2)$, rent cost $c$, request arrival sequence $\{x_l\}_{l\geq0}^t$\\
Output: service hosting strategy $r_{t+1}$, $t > 0$\\
Initialize: $r_1= t_{\text{recent}} = 0$\\
\For {\textbf{each} time-slot $t$}{
$I_t=(M$, $g(\alpha_2)$, $t$, $t_{\text{recent}}$, $c$, $\{x_l\}_{l\geq t_{\text{recent}}}^t )$\\
% \color{red}$t_{\text{last}}=\max\{t_{\text{evict}},t_{\text{fetch}}\}$\color{black}\\
$R_0^{(\tau_0)} = [\underbrace{r_t,r_t,\ldots,r_t}_{\tau_0-t_{\text{recent}}},\underbrace{0,0,\ldots, 0}_{t-\tau_0}]$ \\
$R_\alpha^{(\tau_{\alpha})}= [\underbrace{r_t,r_t,\ldots,r_t}_{\tau_{\alpha}-t_{\text{recent}}},\underbrace{\alpha_2,\alpha_2,\ldots, \alpha_2}_{t-\tau_{\alpha}}]$ \\
$R_1^{(\tau_1)} = [\underbrace{r_t,r_t,\ldots,r_t}_{\tau_1-t_{\text{recent}}},\underbrace{1,1,\ldots, 1}_{t-\tau_1}]$ \\
$\text{minCost}(0)=\displaystyle\min_{\tau_0\in (t_{\text{recent}},t)} \FtotalCost(R_0^{(\tau_0)},I_t)$\\
$\text{minCost}(\alpha_2)=\displaystyle\min_{\tau_{\alpha}\in (t_{\text{recent}},t)} \FtotalCost(R_\alpha^{(\tau_{\alpha})},I_t)$\\
$\text{minCost}(1)=\displaystyle\min_{\tau_1\in (t_{\text{recent}},t)} \FtotalCost(R_1^{(\tau_1)},I_t)$\\
$r_{t+1} = \displaystyle \argmin_{i \in \{0, \alpha_2, 1\}} \text{minCost}(i)$ \\
\If{$r_{t+1} \neq r_t$}
{$t_{\text{recent}} = t$
}
}
\Fn{\FtotalCost{$R, I_t$}}{
$g(0)=1$, $g(1)=0$\;
cost = $R(1)\times c+x_1 \times g(R(1))$\;
\For{$j\gets 2$ \KwTo $t-t_{\text{recent}}$ }{
cost = cost $+ R(j)\times c+x_j \times g(R(j))$ \\
\hspace{.67in}$+ M \times \left|R(j)-R(j-1)\right|$\;
}
\KwRet cost;
}
\end{algorithm}
\begin{comment}
In this section we consider different polices we want to analyze with the metrics defined in Section \ref{sec:setting}.
\subsection{Retro Renting (RR)}
This policy was proposed in \cite{narayana2021renting}. It is a deterministic policy and tries to mimic the optimal off-line policy and it is considered as optimal off-line policy with some delay. The Algorithm \ref{alg:RR} represents the RR policy. The competitive ratio analysis for RR is also there in \cite{narayana2021renting}. We analyze the regret of RR in this paper.
\subsection{Follow The Perturbed Leader (FTPL)}
This policy already exists in the content caching, on-line learning literature \cite{cohen2015following}, \cite{bhattacharjee2020fundamental} where as we adapted it to the service caching setting and analyzed its regret. The Algorithm \ref{alg:W-FTPL} with $\beta=0$ represents the FTPL policy. Note that it is a randomized algorithm and does not consider $M$ while making decisions and leads to large fetch cost if $M$ is large.
\subsection{Our policy (W-FTPL)}
In our policy there will be an initial wait period in which service is not fetched or hosted at edge and all the requests are forwarded to edge. After the wait period we follow the FTPL policy. So we named our policy as Wait then FTPL (W-FTPL) which is given in Algorithm \ref{alg:W-FTPL}. Note that the initial wait period depends on $M$, which helps in reducing the fetch cost when $M$ is large compared to FTPL.
%In this paper we analyze the algorithms Retro Renting(RR) (Algorithm \ref{alg:RR})defined in \cite{narayana2021renting}, Follow The Perturbed Leader (FTPL) (Algorithm \ref{alg:W-FTPL} with $\beta=0$) which exist in the literature, our policy Wait then Follow The Perturbed Leader (W-FTPL) (Algorithm \ref{alg:W-FTPL}). We only consider $c<1$ because if $c>1$ it is optimal to forward in each slot. RR policy is a deterministic policy and it try to mimic the optimal offline policy which considers fetch cost $M$ while taking the decisions. RR is considered as some delayed version of optimal offline policy. The FTPL policy is a randomized policy which try to achieve optimal static policy as time progresses and does not consider the fetch cost $M$ while taking decisions. \color{blue} In order to consider $M$ while taking decisions we proposed Wait then FTPL (W-FTPL) policy. In this policy initially some requests are forwarded to cloud and after that FTPL policy will be implemented hence the name Wait then FTPL. The initial wait will depend on $M$.
\color{black}
\begin{algorithm}
\caption{Retro Renting (RR)}
\label{alg:RR}
\KwIn{$c$, $M$,initial storage $\rho_1$, request sequece $\{r_l\}_{l=1}^t$}
Set $t_{f} = t_{e} = 0$\\
\For{$t=1$ to $T$}{
$\rho_{t+1}=\rho_{t}$\\
\If{$\rho_{t}=0$}{
\For{$t_e<\tau<t$}{
\If{$\sum_{l=\tau}^{t}r_t\ge M+\sum_{l=\tau}^{t}c$}{
$\rho_{t+1}=1$, $t_f=t$\\
break
}
}
}
\If{$\rho_{t}=1$}{
\For{$t_f<\tau<t$}{
\If{$\sum_{l=\tau}^{t}r_t +M \le \sum_{l=\tau}^{t}c$}{
$\rho_{t+1}=0$, $t_e=t$\\
break
}
}
}
}
\end{algorithm}
\begin{algorithm}
\color{blue}
\caption{Wait then Follow The Perturbed Leader (W-FTPL)}
\label{alg:W-FTPL}
\KwIn{$c$, $M$, $\{\eta_t\}_{t\ge 1}$, $\beta$, request sequece$\{r_l\}_{l=1}^t$}
Set $\bm{\Theta_{1}} \leftarrow \bm{0}$, $R_1=0$, $t=1$.\\
\While{(1)}{
\eIf{$t>\frac{\beta log M}{(c-R_{t}/t)^2}$}{
$T_s=t$\\
break;
}{$\rho_{t}=0$}
Update $\bm{\Theta_{t+1}}=\bm{\Theta_{t}}+ \bm{\theta_{t}$}, $R_{t+1}=R_{t}+r_t$\\
$t++$
}
Sample $\bm{\gamma} \sim \mathcal{N}(\bm{0}, \bm{I})$.\\
\For{$t=T_s$ to $T$}{
host $\bm{\rho}_t\in \argmin_{\bm{\rho} \in \mathcal{X}} \langle \bm{\rho}, \bm{\Theta_t} +\eta_t\bm{\gamma} \rangle $.\\
Cost incurred is given by \eqref{eq:cost_in_slot}.\\
Update $\bm{\Theta_{t+1}}=\bm{\Theta_t}+ \bm{\theta_{t}$}}
\end{algorithm}
content...
\end{comment}
In this section, we state and discuss our key results.
\subsection{Regret Analysis}
Our first result characterizes the regret performance of the policies discussed in Section \ref{sec:policies} and the fundamental limit on the performance of any online policy for adversarial arrivals.
\begin{theorem}[Adversarial Arrivals] \label{thm:reg_adv}
Let the arrivals be generated by an oblivious adversary under the constraint that at most $\kappa$ requests arrive in each time-slot. Then,
\begin{enumerate}[(a)]
\item $
\mathcal{R}^{\mathcal{P}}_{A}(T) =\Omega(\sqrt{T})$ for any online policy $\mathcal{P}$.
\item $\mathcal{R}^{\text{RR}}_A(T) =\Omega({T}).$
\item For $\eta_{t}=\alpha\sqrt{t}$, $\alpha>0$,
\begin{align*}
\mathcal{R}^{\text{FTPL}}_A(T)\le& \sqrt{2T\log K}\left( \alpha + \frac{4\kappa^2}{\alpha}\right)
+\frac{K^2M(c+2\kappa)}{2\alpha\sqrt{\pi}} \sqrt{T+1}.
\end{align*}
\item For $\eta_{t}=\alpha\sqrt{t}$, $\alpha>0$,
$$\mathcal{R}_A^{\text{FTPL}}(T) \ge M\alpha_{2}/K.$$ \color{black}
% where $b=\Prob \left(\frac{2c-\kappa}{2\alpha}\le Z\le \frac{c}{\alpha\sqrt{2}} \right)$, $Z\sim \mathcal{N}(0,1)$.
\item For $\eta_{t}=\alpha\sqrt{t}$, $\alpha>0$, $\beta>1$, $\delta>0$,
\begin{align*}
\mathcal{R}^{\text{W-FTPL}}_A(T)\le& \sqrt{\beta T (\log M)^{1+\delta}}
+\sqrt{2T\log K} \left(\alpha + \frac{4\kappa^2}{\alpha} \right)\\
&+\frac{K^2M(c+2\kappa)}{2\alpha\sqrt{\pi}}\sqrt{T+1}.
\end{align*}
\end{enumerate}
\end{theorem}
The key take-away from Theorem \ref{thm:reg_adv} is that RR has linear regret and FTPL, W-FTPL have order-optimal regret, i.e., $\OO(\sqrt{T})$ with respect to the time horizon under adversarial arrivals. The proof of Theorem \ref{thm:reg_adv} is in Section \ref{sec:proofs}.
Our second result characterizes the regret performance of the policies discussed in Section \ref{sec:policies} for i.i.d. stochastic arrivals.
\begin{theorem}[Stochastic Arrivals]\label{thm:reg_stch}
Let the arrivals in each time-slot be i.i.d. stochastic with mean $\mu$ and $\Delta_{\text{min}}\ne 0$. Then we have
\begin{enumerate}[(a)]
\item $\mathcal{R}^{\text{RR}}_S(T) =\Omega({T}).$
\item For $\eta_{t}=\alpha\sqrt{t-1}$, $\alpha>0$,
\begin{align*}
\mathcal{R}^{\text{FTPL}}_S(T)
\le& \left( \sqrt{2\log K}+\frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}} \right) \left( \alpha + \frac{4\kappa^2}{\alpha}\right)
+ \frac{16\alpha^2+4\kappa^2}{\Delta_{\text{min}}}\\
&+(16\alpha^2+3\kappa^2) \frac{M}{\Delta_{\text{min}}^2},
\end{align*}
where $h_1= 4\max\{8\alpha^2,\kappa^2\}$.
\item For $\eta_{t}=\alpha\sqrt{t-1}$, $\alpha>0$, $\beta>1$, $\delta>0$, and $M$ large enough we have,
\begin{align*}
\mathcal{R}^{\text{W-FTPL}}_S(T)
\le 1&+\beta \kappa^2(\log M)^{1+\delta}\left(\frac{4}{\Delta_{\text{min}}^2}+\frac{16\alpha^2+3\kappa^2}{\Delta_{\text{min}}^2 \Delta_{\text{max}}^2}\right)\\
&+(16\alpha^2+4\kappa^2)\left(\frac{1}{\Delta_{\text{min}}}+\frac{1}{\Delta_{\text{min}}^2}\right)\\
&+\left( \sqrt{2\log K}+\frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}} \right) \left( \alpha + \frac{4\kappa^2}{\alpha}\right),
\end{align*}
where $h_1= 4\max\{8\alpha^2,\kappa^2\}$.
% \begin{align*}
% &\mathcal{R}^{\text{W-FTPL}}_S(T) \\
% \le & 1+\frac{4\beta \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{min}}^2} + \frac{1}{M^{2\beta(\log M)^\delta}}\frac{2\kappa^2 }{\Delta_{\text{min}}^2}\\
% &+(16\alpha^2+3\kappa^2)\sum_{i\ne i^\star}\frac{1}{\Delta_i}\\
% &+M\frac{(\sqrt{\beta}-1)^2\kappa^2 (\log M)^{1+\delta}}{\Delta_{\text{max}}^2 M^{2(\log M)^\delta}}\frac{16\alpha^2+3\kappa^2}{\Delta_{\text{min}}^2}\\
% &+\left( \frac{16\alpha^2 M}{M^{\frac{(\sqrt{\beta}-1)^2\kappa^2\Delta_{\text{min}}^2(\log M)^\delta}{16\alpha^2\Delta_{\text{max}}^2}}}+ \frac{2M\kappa^2}{M^{\frac{(\sqrt{\beta}-1)^2\Delta_{\text{min}}^2(\log M)^\delta}{2\Delta_{\text{max}}^2}}} \right) \\
% &\times\frac{1}{\Delta_{\text{min}}^2}.
% \end{align*}
\end{enumerate}
\end{theorem}
\begin{corollary}[Stochastic Arrivals]
Let the arrivals in each time-slot be i.i.d. stochastic with mean $\mu$ and $\Delta_{\text{min}}\ne 0$. For the W-FTPL policy, if $\rho_{t}^\mathcal{P}\in\{0,1\}$, $\beta=\max\{(1+4\alpha/\kappa)^2, (1+\sqrt{2}/\kappa)^2\}$, then the regret scales at most logarithmically with $M$.
\end{corollary}
\begin{remark} While the result in \ref{thm:reg_stch} $(c)$ is stated for $\delta>0$, it can e shown that for $\delta=0$ and $\beta$ chosen suitably large, the regret of W-FTPL scales logarithmically with $M$.
% For the W-FTPL policy, if $\delta=0$ then for large value of $\beta$, regret scales at most logarithmically with $M$. Where as for $\delta>0$ for any value of $\beta>1$ our upper bound on regret scales as $(\log M)^{1+\delta}$.
\end{remark}
\color{black}
%\begin{remark} For the W-FTPL policy,
%\begin{enumerate}[a)]
% \item If $\delta=0$ then for large value of $\beta$ we get regret upper bound scale logarithmically with $M$.
%
% \item If $\rho_{t}^\mathcal{P}\in\{0,1\}$, $\beta=\max\{(1+4\alpha/\kappa)^2, (1+\sqrt{2}/\kappa)^2\}$ we get regret upper bound scale logarithmically with $M$.
%
% \item For $\delta>0$ and $M$ large enough, the regret bound can be simplified as follows
% \begin{align*}
% &\mathcal{R}^{\text{W-FTPL}}_S(T) \\
% \le& (16\alpha^2+4\kappa^2)\left(\frac{1}{\Delta_{\text{min}}^2}+ \frac{\beta \kappa^2}{\Delta_{\text{min}}^2\Delta_{\text{max}}^2}+\sum_{i\ne i^\star}\frac{1}{\Delta_{i}}\right) \\
% &+\frac{4\beta \kappa^2 (\log M)^{1+\delta}}{\Delta_{\text{min}}^2}.
% \end{align*}
%\end{enumerate}
%\end{remark}
The key take-aways from Theorem \ref{thm:reg_stch} are that RR has linear regret with respect to time, and FTPL/W-FTPL has regret that does not scale with time. Also, the upper bound on the regret of W-FTPL scales as $(\log M)^{1+\delta}$ for $\delta\ge 0$ where as for FTPL, the upper bound on the regret scales linearly with $M$. W-FTPL thus outperforms FTPL w.r.t. to the dependence of regret on the fetch cost $M$. We validate these results via simulations in Section \ref{sec:simulations}.
%\begin{table}
% \color{blue}
% \centering
% \renewcommand{\arraystretch}{1.5}
% \begin{tabular}{ |c|c|c|c|c| }
% \hline
% Arrivals & Metric & RR & FTPL & W-FTPL \\ [0.5ex]
% \hline
% Adversarial& $\E_\mathcal{P}[\text{Regret}^{\mathcal{P}}_T]$ & $\Omega(T)$ & $\OO(\sqrt{T}) $ & $\OO(\sqrt{T}) $\\
% \hline
% Stochastic & $\E_{\mathcal{P},r}[\text{Regret}^{\mathcal{P}}_T]$&$\Omega(T)$ & $\OO(M) $ & $\OO((\log M)^{1+\delta}) $\\
% \hline
% \end{tabular}
% \caption{Regret of candidate policies}
% \label{tab:regret}
%\end{table}
\color{black}
\subsection{Competitive Ratio Analysis}
Our next result characterizes the competitive ratio of the policies discussed in Section \ref{sec:policies}. The competitive ratio of RR is characterized in \cite{narayana2021renting, LN_partial_hosting}, and it is shown that RR has a constant competitive ratio and the competitive ratio decreases with an increase in fetch cost $M$. We present the results for FTPL and W-FTPL policies below.
\begin{theorem}[Adversarial Arrivals]\label{thm:comp_adv}
Let the arrivals be generated by an oblivious adversary under the constraint that at most $\kappa$ requests arrive in each time-slot. Then we have
\begin{enumerate}[(a)]
\item For $\eta_{t}=\alpha\sqrt{t-1}$, $\alpha>0$,
\begin{align*}
\sigma^{\text{FTPL}}_{A}(T)\le& \frac{\kappa^2(3+2M/c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\max_{\alpha_{i}\ne 0}\frac{1-g(\alpha_{i})}{\alpha_{i}} \\&+\frac{\kappa^2(M+c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\sum_{i=2}^K\frac{16\alpha^2}{c^2\alpha_{i}^2}.
\end{align*}
\item For $\eta_{t}=\alpha\sqrt{t-1}$, $\alpha>0$, $\beta>1$, $\delta\ge0$,
\begin{align*}
\sigma^{\text{W-FTPL}}_{A}(T)\le& \frac{\kappa^2(3+2M/c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\max_{\alpha_{i}\ne 0}\frac{1-g(\alpha_{i})}{\alpha_{i}}\\
&+\frac{\kappa^2(M+c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\sum_{i=2}^K\frac{16\alpha^2}{c^2\alpha_{i}^2}\\
&+\frac{\kappa^2}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}.
\end{align*}
\end{enumerate}
\end{theorem}
The key take-away from Theorem \ref{thm:comp_adv} is that the competitive ratios of both FTPL and W-FTPL are bounded by a constant. The competitive ratio of FTPL and W-FTPL deteriorates with $M$, while the competitive ratio of RR improves with $M$ when only one partial hosting level is permitted \cite{LN_partial_hosting}. Note that the competitive ratio of FTPL and W-FTPL is bounded by a constant for the setting where any finite number of partial hosting levels are allowed. Compared to this, for RR in \cite{LN_partial_hosting}, the competitive ratio result holds only for one partial hosting level.
\begin{theorem}[Stochastic Arrivals] \label{thm:comp_stch}
Let the arrivals in each time-slot be i.i.d. stochastic with mean $\mu$. Then we have
\begin{enumerate}[(a)]
\item For $T$ large enough, $\sigma^{RR}_S (T)> 1$.
\item For $\eta_{t}=\alpha \sqrt{t-1}$, $\sigma^{\text{FTPL}}_S(T) \le 1+\OO(1/T) $.
\item For $\eta_{t}=\alpha\sqrt{t-1}$, $\alpha>0$, $\beta>1$, $\delta>0$, $\sigma^{\text{W-FTPL}}_S(T) \le 1+\OO(1/T) $.
\end{enumerate}
\end{theorem}
The key take-away from Theorem \ref{thm:comp_stch} is that for $T$ large enough, FTPL and W-FTPL outperform RR in terms of competitive ratio for stochastic arrivals.
%\tablename ~\ref{tab:comp_ratio} summarizes the results for competitive ratio for different polices considered in this paper.
%\begin{theorem}
% The competitive ratio of FTPL policy with learning rate $\eta_{t}=\alpha\sqrt{t-1}$, $\alpha>0$ as $T\to \infty$ is lower bounded as follows
%
% \label{thm:FTPL_comp_stch_ub}
%\end{theorem}
%
%\begin{theorem}
% The competitive ratio of W-FTPL policy with learning rate $\eta_{t}=\alpha\sqrt{t-1}$, $\alpha>0$, $\beta=\max\{(1+{\frac{4\alpha\Delta_{\text{max}}}{\Delta_{\text{min}}}})^2,(1+\frac{\sqrt{2}\Delta_{\text{max}}}{\Delta_{\text{min}}})^2 \}$ as $T\to \infty$ is lower bounded as follows
% $$\sigma^{\text{W-FTPL}}_S \le 1 $$
% \label{thm:WFTPL_comp_stch_ub}
%\end{theorem}
\begin{comment}
\begin{table}[h]
\centering
\renewcommand{\arraystretch}{1.5}
\begin{tabular}{ |c|c|c|c|c| }
\hline
Arrivals & Metric & RR & FTPL & W-FTPL \\ [0.5ex]
\hline
Stochastic & $\displaystyle \lim_{T \rightarrow \infty} \sigma_S^{\mathcal{P}}(T)$& $\ge1$ & $\le1$ & $\le1$\\
\hline
Adversarial &$\displaystyle \lim_{T \rightarrow \infty} \sigma_A^{\mathcal{P}}(T)$& $\OO(1/M)$ & $\OO(M)$& $\OO(M)$\\
\hline
\end{tabular}
\caption{Competitive ratio of candidate policies}
\label{tab:comp_ratio}
\end{table}
\end{comment}
In Section \ref{sec:simulations} we supplement the results stated in this section through simulations.
In this section, we compare RR, FTPL and W-FTPL policies via simulations using synthetic request sequences, as well as real trace data from \cite{iosup2008grid}. In all our simulations, we consider $\eta_t=0.1\sqrt{t}$ for FTPL and W-FTPL, and $\beta=6$ for W-FTPL.
\subsection{Synthetic arrivals}
In \figurename ~\ref{fig:RR_worst}, we plot the regret of different policies as a function of time horizon $T$. The parameters considered are $c=0.1$, $M=50$, $\kappa=5$. The request arrival considered is similar to the one used in the proof of Theorem \ref{thm:comp_adv}(b), i.e., we divide time into frames, and each frame starts with $\lceil \frac{M}{\kappa-c}\rceil$ slots of $\kappa$ requests each followed by $\lceil \frac{M}{c}\rceil$ slots of zero requests. We consider 100 such frames for this experiment. The regret is averaged over 50 experiments. We observe that RR has linear regret in this case which agrees with Theorem \ref{thm:reg_adv}.
\begin{figure}[ht]
\centering
\includegraphics[width=\linewidth]{RR_worst.pdf}
\caption{Regret as a function of Time horizon ($T$)}
\label{fig:RR_worst}
\end{figure}
\subsection{Stochastic arrivals}
For the stochastic case, we consider arrivals in each slot to be Bernoulli with parameter $\mu$. We will restrict attention to the case where there is only one partial level of hosting, i.e., $\rho_{t}^\mathcal{P}\in \{0,\alpha_{2},1\}$. We consider $\alpha_{2}=0.5$ and $g(\alpha_{2})=0.45$. In \figurename \ref{fig:reg_T} and \figurename \ref{fig:reg_M}, we consider $\mu=0.4$, $c=0.45$. All the results plotted are averaged over 50 independent experiments. In \figurename \ref{fig:reg_T}, we fix $M=5$ and compare the regret performance of the policies as a function of time. We observe that W-FTPL performs better than other policies, and RR has linear regret, which agrees with Theorem \ref{thm:reg_stch}. In \figurename \ref{fig:reg_M}, we compare the performance of the policies with respect to $M$ for $T=10^4$. We observe that W-FTPL performs better than other policies, and the main difference between the cost of policies is due to fetch cost. Also note that W-FTPL has better dependence on $M$ as compared to FTPL, which agrees with Theorem \ref{thm:reg_stch}. From Theorem \ref{thm:reg_stch} we observe that for large values of $M$ the regret of FTPL is proportional to $M/\Delta_{\text{min}}^2$ where as W-FTPL is proportional to $\log M /\Delta_{\text{min}}^2$. Therefore in \figurename \ref{fig:reg_c}, we compare the performance of the policies by varying rent cost $c$ for $M=500$, $\mu = 0.4$, and $T=10^4$. Again we note that W-FTPL outperforms RR and FTPL. As $c$ gets closer to $\mu$, the FTPL policy does multiple fetches, and as a result, the regret increases when $c$ is close to $\mu$.
\begin{figure}[ht]
\centering
\includegraphics[width=\linewidth]{reg_stch_T}
\caption{Regret as a function of time horizon ($T$)}
\label{fig:reg_T}
\end{figure}
\begin{figure}[ht]
\centering
\includegraphics[width=\linewidth]{reg_stch_M}
\caption{Cost, Regret as a function of fetch cost ($M$)}
\label{fig:reg_M}
\end{figure}
\begin{figure}[ht]
\centering
\includegraphics[width=\linewidth]{reg_stch_alpha_c.eps}
\caption{Cost, Regret as a function of rent cost ($c$)}
\label{fig:reg_c}
\end{figure}
\subsection{Trace driven arrivals}
For trace-driven simulations, we use the data collected by Grid Workloads Archive \cite{iosup2008grid}, which consists of requests arriving at computational servers. Among the traces mentioned in \cite{iosup2008grid} we use DAS-2 traces, which were provided by the Advanced School for Computing and Imaging (ASCI). We consider slot duration as 1 hour and plot a snapshot of the trace in \figurename~\ref{fig:trace_arrvl}. We consider $\kappa=300$ for all the experiments in this subsection. All the results plotted for trace driven experiments are averaged over 50 independent experiments.
We consider partial service hosting with one partial level for our simulations. This dataset provides information on the number of processors required to serve each request, and this number lies between 1 and 128 for all requests. For this experiment, we consider hosting the entire service as equivalent to having 128 processors at the edge and hosting the partial service as having 16 processors at the edge. It follows that the hosting status $\rho_t^{\mathcal{P}} \in \{0, 16/125(=0.125), 1\}$. To compute $g(0.125)$, we use the dataset to compute the fraction of requests that require more than 16 processors for service and obtain that $g(0.125)=0.0328$.
%Hosting the entire service is having 128 processors near edge for computation. Hosting partial service mean we will have 16 processors at edge. Therefore we have $\rho_{t}^\mathcal{P}\in \{0,\alpha,1\}$ where $\alpha=16/128=0.125$ and $g(0.125)=0.0328$ which is calculated using the trace.
%We observe that $\alpha$-RR \color{red}(a variant of RR which can partially host the service proposed in \cite{LN_partial_hosting}) \color{black} performs better than other policies.
\begin{figure}[ht]
\centering
\includegraphics[width=\linewidth]{trace_data_plot}
\caption{Arrivals in a slot for trace data considered }
\label{fig:trace_arrvl}
\end{figure}
\begin{figure}[ht]
\centering
\includegraphics[width=\linewidth]{reg_adv_T}
\caption{Regret as a function of time horizon}
\label{fig:reg_alpha_T}
\end{figure}
\begin{figure}[ht]
\centering
\includegraphics[width=\linewidth]{reg_adv_M}
\caption{Cost, Regret as a function of fetch cost ($M$)}
\label{fig:reg_M_adv}
\end{figure}
\begin{figure}[ht]
\centering
\includegraphics[width=\linewidth]{reg_adv_alpha_c}
\caption{Cost, Regret as a function of rent cost ($c$)}
\label{fig:reg_c_adv}
\end{figure}
In \figurename \ref{fig:reg_alpha_T}, we consider $M=500$, $c=100$, $\kappa=300$ and compare the regret performance of policies as a function of time. We observe that RR performs better than other policies for most values of time horizon. This is because RR attempts to mimic the optimal offline policy, which is not a static policy for this arrival sequence. Note that FTPL and W-FTPL attempt to mimic the optimal static policy and perform worse than RR.
In \figurename \ref{fig:reg_M_adv}, we compare the performance of different policies with respect to $M$ and fix $c=100$, $\kappa=300$. We observe that RR performs better than the other policies for some values of $M$ because of the same reason mentioned before. In \figurename \ref{fig:reg_c_adv}, we compare the performance of different policies by varying rent cost $c$ and fix $M=500$, $\kappa=300$. Again we note that RR outperforms W-FTPL and FTPL in some cases.
%\begin{figure}[ht]
% \centering
% \includegraphics[width=\linewidth]{plots/reg_alpa_t}
% \caption{Regret as a function of time horizon}
% \label{fig:reg_alpha_T}
%\end{figure}
Our simulation results thus indicate that FTPL and W-FTPL outperform RR for i.i.d. stochastic arrivals. In the case where the arrival process is such that the offline optimal policy is not static, RR can outperform FTPL and W-FTPL. Additionally, RR can have linear regret with respect to time, while FTPL and W-FTPL have sub-linear regret with respect to time for all arrival processes considered.
%For \figurename \ref{fig:reg_T1}, \ref{fig:reg_T2} we fix $M=5$, $\mu=0.3$ and consider $c=0.25$, $c=0.35$ respectively and plot regret as a function of time. As expected from Theorem \ref{thm:RR_reg_LB_stch}, \ref{thm:FTPL_reg_ub_stch} we observe that RR has linear regret and FTPL is bounded by constant regret.
%\begin{figure}[h]
% \centering
% \includegraphics[width=\linewidth]{plots/all_stch_c1.jpg}
% \caption{plots varying $c$ for $M=1$, $\mu=0.3$}
% \label{fig:reg_c1}
%\end{figure}
%\begin{figure}[h]
% \centering
% \includegraphics[width=\linewidth]{plots/all_stch_c3.jpg}
% \caption{plots varying $c$ for $M=100$, $\mu=0.3$}
% \label{fig:reg_c2}
%\end{figure}
%Next we plot cost under RR policy, FTPL policy, regret and competitive ratio by varying $c$. We fix $M=1$, $M=100$ for \ref{fig:reg_c1}, \ref{fig:reg_c2} respectively. Clearly as we can see the fetch cost $M$ plays a key role in the Regret and cost of FTPL, RR policies. If $M$ is small then RR policy fetch cost is more compared to FTPL and for high value of $M$ FTPL policy has larger fetch cost. This is because FTPL does not consider $M$ while taking decisions. As we can observe the competitive ratio is close to 1 when $M$ is small.
%\begin{figure}
% \centering
% \includegraphics[width=\linewidth]{plots/all_stch_M1.jpg}
% \caption{plots varying $M$ for $c=, \mu=0.3$}
% \label{fig:reg_M1}
%\end{figure}
%
%\begin{figure}
% \centering
% \includegraphics[width=\linewidth]{plots/all_stch_M7.jpg}
% \caption{plots varying $M$ for $c=, \mu=0.3$}
% \label{fig:reg_M2}
%\end{figure}
%Next we plot cost under RR policy, FTPL policy, regret and competitive ratio by varying $M$. We fix $c=0.25$, $\mu=0.3$ for \figurename \ref{fig:reg_M1}, \ref{fig:reg_M2}. The regret and competitive ratio of FTPL increases with $M$ as expected from Theorem \ref{thm:FTPL_comp_adv_lb}. Where as for RR as $M$ increases the regret decreases.
%\begin{theorem}
% \label{thm:reg_LB}
% Regret for any online policy $\mathcal{P}$ under adversarial setting follows
% \begin{align*}
% \mathcal{R}^{\mathcal{P}}_{A}(T) =\Omega(\sqrt{T})
% \end{align*}
%\end{theorem}
\subsection{Proof of Theorem \ref{thm:reg_adv}(a)}
\begin{proof}[Proof of Theorem \ref{thm:reg_adv}(a)]
Recall that regret of any policy is given by
\begin{align*}
\mathcal{R}^{\mathcal{P}}_{A}(T,r)=&\sum_{t=1}^Tc\rho_t+ g(\rho_t) r_t+ M (\rho_t-\rho_{t-1})^+\\
&- \min_i\left\{c\alpha_iT+g(\alpha_i)R_T+M\alpha_{i}\right\}.
\end{align*}
Note that $\sup_{r} \mathcal{R}^{\mathcal{P}}_A(T,r) \ge \E_r[\mathcal{R}^{\mathcal{P}}_{A}(T,r)]$ where the expectation is w.r.t some specified distribution over request sequences. We lower bound $\E_r[\mathcal{R}^{\mathcal{P}}_{A}(T,r)]$ to get a lower bound on $\mathcal{R}^{\mathcal{P}}_A(T)$.
\begin{align*}
\E_r[\mathcal{R}^{\mathcal{P}}_{A}(T,r)]=&\sum_{t=1}^T c\rho_t+ g(\rho_t)\E_r[r_t] + M (\rho_t-\rho_{t-1})^+\\
& - \E_r [\min_i\left\{c\alpha_iT+g(\alpha_i)R_T+M\alpha_{i}\right\}]\\
\ge &\sum_{t=1}^T c\rho_t+ g(\rho_t)\E_r[r_t] \\
& - \E_r [\min_i\left\{c\alpha_iT+g(\alpha_i)R_T\right\}]-M.
\end{align*}
Let $\ell=\argmin_{i\ne 1} \frac{\alpha_{i}}{1-g(\alpha_{i})}$, $X\sim \text{Ber}(\frac{c\alpha_{\ell}}{\kappa(1-g(\alpha_{\ell}))})$, and $r_t =\kappa X$ and i.i.d overtime, therefore $\E_r[r_t]=\frac{c\alpha_{\ell}}{1-g(\alpha_{\ell})}$. From Assumption \ref{assump:g(f)_convex} we have $\alpha_i+g(\alpha_{i})r_t\le \kappa$ for all $1\le i\le K$ and $0\le r_t\le \kappa$ therefore $\frac{c\alpha_{\ell}}{\kappa(1-g(\alpha_{\ell}))}\le1$ and $X$ is a valid Bernoulli random variable.
\begin{align*}
\E_r[\mathcal{R}^{\mathcal{P}}_{A}(T,r)]
\ge& \sum_{t=1}^T c\rho_t+ g(\rho_t)\frac{c\alpha_{\ell}}{1-g(\alpha_{\ell})}\\
& - \E_r [\min_i\left\{c\alpha_iT+g(\alpha_i)R_T\right\}]-M\\
= & \sum_{t=1}^T (1-g(\rho_t))\frac{c\rho_{t}}{(1-g(\rho_t))}\mathds{1}_{\rho_{t}\ne 0}+ g(\rho_t) \frac{c\alpha_{\ell}}{1-g(\alpha_{\ell})}\\
& - \E_r [\min_i\left\{c\alpha_iT+g(\alpha_i)R_T\right\}]-M\\
\overset{(a)}{\ge} & \sum_{t=1}^T (1-g(\rho_t))\frac{c\alpha_{\ell}}{(1-g(\alpha_\ell))}+ g(\rho_t) \frac{c\alpha_{\ell}}{1-g(\alpha_{\ell})}\\
& - \E_r [\min_i\left\{c\alpha_iT+g(\alpha_i)R_T\right\}]-M\\
\overset{(b)}{\ge} & \frac{cT\alpha_{\ell}}{1-g(\alpha_{\ell})}- \E_r[\min\{R_T,cT\alpha_{\ell}+g(\alpha_{\ell})R_T\}]-M\\
\overset{(c)}{=} & \frac{1}{2}\E_r[|R_T-cT\alpha_{\ell}-g(\alpha_{\ell})R_T|]-M\\
=& \frac{\kappa(1-g(\alpha_\ell))}{2}\E_r\left[\left|\frac{R_T}{\kappa} - \frac{Tc\alpha_{\ell}}{\kappa(1-g(\alpha_\ell))}\right|\right] - M\\
\overset{(d)}{=}&\Omega(\sqrt{T}),
\end{align*}
where $(a)$ follows from definition of $\ell$, $(b)$ follows because of $\min_{1\le k\le K} a_k\le \min\{a_i,a_j\}$ for $1\le i,j\le K$, $(c)$ follows from Lemma \ref{lem:min_E} in Section \ref{sec:appendix}, and $(d)$ follows from Lemma \ref{lem:binomial_t} in Section \ref{sec:appendix}.
\end{proof}
\begin{comment}
%\begin{proof}
% The proof is similar to the proof of Theorem 1 in \cite{bhattacharjee2020fundamental}.
% We have
% \begin{align*}
% \mathcal{R}^{\mathcal{P}}_{A}(T,r)=&\sum_{t=1}^Tc\rho_t+ (1-\rho_t) r_t+ M (\rho_t-\rho_{t-1})^+\\
% &- \min\left\{cT, \sum_{t=1}^Tr_t\right\}.
% \end{align*}
% Since $\sup_{r\in \mathcal{R}} \mathcal{R}^{\mathcal{P}}_\mathcal{A}(T,r) \ge \E_r[\mathcal{R}^{\mathcal{P}}_{\mathcal{A}}(T,r)]$. We lower bound $\E_r[\mathcal{R}^{\mathcal{P}}_{A}(T,r)]$ to get the required result.
% \begin{align*}
% \E_r[\mathcal{R}^{\mathcal{P}}_{A}(T,r)]=&\sum_{t=1}^T c\rho_t+ (1-\rho_t)\E_r[r_t] + M (\rho_t-\rho_{t-1})^+\\
% & - \E_r [\min \{ cT, \sum_{t=1}^T r_t\}]
% \end{align*}
% To remove the dependence on policy $\mathcal{P}$ we consider $\E_r[r_t]=c$ to get
% \begin{align*}
% \mathcal{R}^{\mathcal{P}}_{A}(T)\ge & cT- \E_r[\min\{cT, \sum_{t=1}^Tr_t\}]\\
% =& \frac{1}{2}\E_r\left[|cT-R_T|\right] \\
% \ge & c^{\lfloor cT \rfloor}(1-c)^{T-\lfloor cT\rfloor} (\lfloor cT \rfloor+1) {T \choose \lfloor cT+1\rfloor}\\
% \ge& \sqrt{\frac{Tc(1-c)}{2\pi}} \left[ 1-\frac{1}{12\lfloor cT \rfloor}- \frac{1}{12(T-\lfloor cT \rfloor-1)}\right]\\
% \ge& \sqrt{\frac{Tc(1-c)}{2\pi}} \left[ 1-\frac{1}{12(cT-1)(1-c)}\right]
% \end{align*}
%\end{proof}
\end{comment}
%Now we show bounds on regret of FTPL and RR policy under adversarial and stochastic arrivals.
%\subsubsection{Adversarial arrivals}
%\begin{theorem} \label{thm:RR_reg_adv}
% Regret of RR policy under adversarial arrivals.
% \begin{align*}
% \mathcal{R}^{\text{RR}}_{A}(T)=\Omega(T)
% %&\ge M\left(\frac{T}{\frac{M}{\kappa-c\alpha_{i^\prime}-g(\alpha_{i^\prime})\kappa}+ \frac{M}{c} +2}-1\right),
% \end{align*}
% where $\alpha_{{i}^\prime}=\argmin_{\alpha_i\ne 0}c\alpha_i+g(\alpha_i)\kappa$.
%\end{theorem}
\subsection{Proof of Theorem \ref{thm:reg_adv}(b)}
Note that the RR algorithm in \cite{narayana2021renting} can have only one intermediate level of partial hosting i.e., $\rho_{t}^{\text{RR}}\in \{0,\alpha_{2},1\}$. So throughout the is subsection we consider $\rho_{t}^{\text{RR}}\in \{0,\alpha_{2},1\}$. We use the following lemmas to prove Theorem \ref{thm:reg_adv}(b).
\begin{lemma}\label{lem:RR_min_evict}
RR does not fetch any non zero fraction of service for at least $\left\lceil \frac{M\alpha_{i'}}{\kappa-c\alpha_{i^\prime} - g(\alpha_{i^\prime})\kappa}\right \rceil$ slots after eviction of complete service, where \\$\alpha_{i'} = \argmin_{\alpha_i\ne 0}\frac{M\alpha_{i}}{\kappa-c\alpha_{i}-\kappa g(\alpha_i)}$.
\end{lemma}
\begin{proof}
Without loss of generality we consider $t=0$ is when RR last evicted the service. If RR policy fetches $\alpha_{i}\ne0$ fraction of service in slot $t+1$, then $c\alpha_{i} t+g(\alpha_{i})R_{t}+M\alpha_{i} < R_{t}$.
We now prove the lemma by contradiction. Let us assume that RR fetches $\alpha_{i}$ fraction of the service in time slot $t_f+1$ and $t_f< \left\lceil \frac{M\alpha_{i'}}{\kappa-c\alpha_{i'} - g(\alpha_{i'})\kappa}\right\rceil.$ Then by using the definition of $\alpha_{i'}$, we get $t_f<\frac{M\alpha_{i}}{\kappa-c\alpha_{i} - g(\alpha_{i})\kappa}$ and $c\alpha_{i} t_f+g(\alpha_{i})\kappa{t_f}+M\alpha_{i}>\kappa t_f\ge R_{t_f}$ which is a contradiction.
\end{proof}
%
\begin{lemma}\label{lem:RR_min_fetch}
Once RR fetches any non zero fraction of service, it hosts (some positive fraction of the service) for at least $\lceil\frac{M}{c}\rceil$ slots before it evicts the completely.
\end{lemma}
\begin{proof}
Without loss of generality we consider $t=0$ is when RR last fetched the service and say it hosted $\alpha_i\ne 0 $ fraction of the service. For the RR policy to evict the service in slot $t+1$, we need $ c\alpha_{i}t + g(\alpha_{i})R_{t}> M\alpha_{i}+R_{t}$, or equivalently $R_{t}< \frac{c\alpha_i t-M\alpha_i}{1-g(\alpha_{i})}$.
We now prove the lemma by contradiction. Let us assume that RR evicts the service in time slot $t_e+1$ and $t_e<\lceil \frac{M}{c}\rceil$. Then from the assumption above, we must have $R_{t_e}<\frac{c\alpha_i t_e-M\alpha_i}{1-g(\alpha_i)}<0$ which is a contradiction.
\end{proof}
\begin{proof}[Proof of Theorem \ref{thm:reg_adv}(b)]
%\color{violet}
We split the entire time duration $T$ into frames of size $t_f+t_e$ and consider the arrival sequence in each frame as $r_t=\kappa$ for $1 \le t \le t_f$ and $r_t=0$ for $t_f+1\le t \le t_f+t_e$, where $\alpha_{i'} = \argmin_{\alpha_i\ne 0}\frac{M\alpha_{i}}{\kappa-c\alpha_{i}-\kappa g(\alpha_i)}$, $t_f=\left\lceil \frac{M\alpha_{i'}}{\kappa-c\alpha_{i'} - g(\alpha_{i'})\kappa} \right\rceil$, and $t_e=\lceil\frac{M}{c}\rceil$.
From Lemma \ref{lem:RR_min_evict} we know that RR does not fetch any fraction of service till time $t_f$. At $t=t_f+1$ RR fetches some positive fraction of the service because $c\alpha_{i'}t_f + g(\alpha_{i'})R_{t_f} + M\alpha_{i'} < R_{t_f}$ holds by definition of $t_f$.
From Lemma \ref{lem:RR_min_fetch} we know that RR does not evict the service for $t\in[t_f+1, t_f+t_e]$. At $t=t_f+t_e+1$ RR evicts the service because for $\alpha_i\ne 0$, $c\alpha_{i}t_e+g(\alpha_{i})\sum_{t=t_f+1}^{t_f+t_e}r_t<M\alpha_{i}+\sum_{t=t_f+1}^{t_f+t_e}r_t$ holds by definition of $t_e$.
%
So amongst the first $t_f + t_e$ time-slots, the RR policy does not host for the first $t_f$ slots and then hosts some positive fraction of service for the following $t_e$ slots. Let $i^*=\argmin_i c\alpha_iT+g(\alpha_{i}R_T+M\alpha_i)$ be the fraction of service to be hosted by optimal static policy. If the optimal static policy is to host $\alpha_{i^*}\ne 0$ fraction of service then we have a regret of at least $(\kappa -c\alpha_{i^*} -g(\alpha_{i^*})\kappa) t_f-M\alpha_{i^*}$ for initial frame and $(\kappa -c\alpha_{i^*} -g(\alpha_{i^*})\kappa) t_f=(\kappa -c\alpha_{i^*} -g(\alpha_{i^*})\kappa) \lceil \frac{M\alpha_{i'}}{\kappa-c\alpha_{i'} - g(\alpha_{i'})\kappa} \rceil>0$ form second frame onwards because of forwarding, which is a constant and does not depend on $T$. If the optimal static policy is not to host the service, then we have a regret of $c\alpha_{i}t_e>0$ till $t_f+t_e$ for some $\alpha_{i}\ne 0$, which is a constant and does not depend on $T$. So in either case, a regret larger than some positive constant(say $d$) is occurred in a frame of size $t_f+t_e$. Since we split the entire time duration $T$ into frames of size $t_f+t_e=\lceil \frac{M\alpha_{i'}}{\kappa-c\alpha_{i^\prime}-g(\alpha_{i^\prime})\kappa}\rceil + \lceil\frac{M}{c} \rceil$ and the request sequence is repeated in each frame, we get linear regret.
\begin{align*}
\mathcal{R}^{\text{RR}}_A(T)&\ge \sum_{f=1}^{\lfloor T/(t_f+t_e) \rfloor} d - M\alpha_{i^*}\\
&\ge \lfloor T/(t_f+t_e) \rfloor d- M\alpha_{i^*}\\
&\ge d\left(\frac{T}{\frac{M\alpha_{i'}}{\kappa-c\alpha_{i^\prime}-g(\alpha_{i^\prime})\kappa}+ \frac{M}{c} +2}-1\right)- M\alpha_{i^*} \\
&=\Omega(T).
\end{align*}
\end{proof}
\subsection{Proof of Theorem \ref{thm:reg_adv}(c)}
The optimal static policy only fetches the optimal fraction of service to be hosted once. Therefore,
\begin{align}
\mathcal{R}^{\text{FTPL}}_{A}(T,r)=&\E[\mathcal{C}^\text{FTPL}(T,r)] \nonumber\\
&- \min_i\{c\alpha_iT+g(\alpha_i)R_T+M\alpha_{i}\} \nonumber\\
\le& \E[\mathcal{C}^\text{FTPL}(T,r)] - \min_i\{c\alpha_iT+g(\alpha_i)R_T\}.
\label{eq:FTPL_reg}
\end{align}
Since FTPL policy does not consider fetch cost $M$ while making decisions we can decouple the fetch cost and the non fetch cost incurred by the FTPL policy. Therefore we can also decouple the expected regret incurred by FTPL into expected regret without fetch cost ($M=0$) and the fetch cost incurred by FTPL to bound \eqref{eq:FTPL_reg}.
We first bound the expected regret of the FTPL policy with fetch cost $M=0$ and then add it's expected fetch cost to get the final regret bound.
\begin{lemma}
The regret for FTPL policy with non decreasing learning rate $\{\eta_t\}_{t=1}^T$ and $M=0$ is given by
\begin{align*}
\mathcal{R}^{\text{FTPL}}_{A}(T)&\le \sqrt{2\log K}\left( \eta_T + \kappa^2 \sum_{t=1}^{T}\frac{1}{\eta_t}\right) .
\end{align*}
\label{lem:FTPL_reg_adv}
\end{lemma}
\begin{proof}
The proof of this theorem follows along the same lines as Theorem 1 of \cite{cohen2015following} and, Proposition 4.1 of \cite{mukhopadhyay2021online}. Recall that $\rho_{t}^{\mathcal{P}}$ represents the fraction of service hosted in slot $t$ by policy $\mathcal{P}$ and $\bm{\rho}_t^\mathcal{P}$ is a one hot vector where position of one represents the level of service hosted at the edge.
In slot $t$ FTPL hosts fraction of service $\rho_{t}^{\text{FTPL}}=\langle\bm{\rho}_{t}^{\text{FTPL}},\bm{s} \rangle$, where $\bm{\rho}_t^{\text{FTPL}}= \argmin_{\bm{\rho}\in \mathcal{X}} \langle \bm{\rho},\bm{\Theta}_t+\eta_t \bm{\gamma}\rangle$.
For ease of notation we suppress the policy in super script and denote $\bm{\rho}_t^{\mathcal{P}}$ as $\bm{\rho}_t$ and $\rho_{t}^{\mathcal{P}}$ as $\rho_{t}$ in further discussion. Define a time varying potential function
\begin{align}
\bm{\Phi}_t(\bm{\theta})= \E_{\bm{\gamma}} \left[ \min_{\bm{\rho}\in \mathcal{X}} \langle \bm{\rho},\bm{\theta} + \eta_t \bm{\gamma} \rangle \right],
\end{align}
and observe that the gradient of $\bm{\Phi}_t$ at $\bm{\Theta}_t$ is $\E_{\bm{\gamma}}[\bm{\rho}_t]$. So $\langle \nabla\bm{\Phi}_t(\bm{\Theta}_t), \bm{\theta}_t \rangle =\E_{\bm{\gamma}}[\langle \bm{\rho}_t, \bm{\theta}_t \rangle]$. Consequently
\begin{align*}
\E_{\bm{\gamma}}[\langle \bm{\rho}_t, \bm{\theta}_t \rangle] &= \langle \nabla\bm{\Phi}_t(\bm{\Theta}_t),\bm{\Theta}_{t+1}- \bm{\Theta}_t \rangle, \\
&= \bm{\Phi}_t(\bm{\Theta}_{t+1})- \bm{\Phi}_t(\bm{\Theta}_t)-\frac{1}{2} \langle \bm{\theta}_t, \nabla^2\Phi_t(\tilde{\bm{\theta}}_t) \bm{\theta}_t) \rangle,
\end{align*}
where $\tilde{\bm{\theta}}_t$ is the line segment joining $\bm{\Theta_{t+1}}$ and $\bm{\Theta_t}$ which follows from Taylor's expansion of $\bm{\Phi}_t(\bm{\Theta}_{t+1})$.
\begin{align*}
&\sum_{t=1}^{T}\E_{\bm{\gamma}}\left[ \langle \bm{\rho}_t, \bm{\theta}_t \rangle \right]\\
&=\sum_{t=1}^T \bm{\Phi}_t(\bm{\Theta}_{t+1})- \bm{\Phi}_t(\bm{\Theta}_t)-\frac{1}{2} \langle \bm{\theta}_t, \nabla^2\Phi_t(\tilde{\bm{\theta}}_t) \bm{\theta}_t) \rangle,\\
&=\bm{\Phi}_{T}(\bm{\Theta}_{T+1}) - \bm{\Phi}_{1}(\bm{\Theta}_{1}) +\sum_{t=2}^T \bm{\Phi}_{t-1}(\bm{\Theta}_{t})- \bm{\Phi}_t(\bm{\Theta}_t)\\
&-\sum_{t=1}^T \frac{1}{2} \langle \bm{\theta}_t, \nabla^2\Phi_t(\tilde{\bm{\theta}}_t) \bm{\theta}_t \rangle. \numberthis \label{eq:reg_adv_thetas}
\end{align*}
By Jensen’s inequality
\begin{align*}
\bm{\Phi}_{T}(\bm{\Theta}_{T+1}) & = \E \left[ \min_{\bm{\rho} \in \mathcal{X}}\langle \bm{\rho}, \bm{\Theta}_{T+1}+\eta_t \bm{\gamma} \rangle \right]\\
&\le \min_{\bm{\rho} \in \mathcal{X}} \E \left[ \langle \bm{\rho}, \bm{\Theta}_{T+1}+\eta_t \bm{\gamma} \rangle \right]\\
&\overset{(a)}{=} \min_{\bm{\rho} \in \mathcal{X}}\langle \bm{\rho}, \bm{\Theta}_{T+1} \rangle,
\end{align*}
where $(a)$ follows because $\gamma_i$'s are standard Gaussian random variables. Recall that $\min_{\bm{\rho} \in \mathcal{X}}\langle \bm{\rho}, \bm{\Theta}_{T+1} \rangle$ is the loss incurred by the offline static policy. Therefore by rearranging \eqref{eq:reg_adv_thetas} and using Jensen’s inequality we get
\begin{align*}
\mathcal{R}^{\text{FTPL}}_A(T) \le& -\bm{\Phi}_{1}(\bm{\Theta_{1}}) + \sum_{t=2}^T \bm{\Phi}_{t-1}(\bm{\Theta}_{t})- \bm{\Phi}_t(\bm{\Theta}_t)\\
&-\sum_{t=1}^T \frac{1}{2} \langle \bm{\theta}_t, \nabla^2\Phi_t(\tilde{\bm{\theta}}_t) \bm{\theta}_t \rangle. \numberthis \label{eq:Reg_ftpl_adv}
\end{align*}
We bound each term in the RHS separately to get the final result.
The first term is $-\bm{\Phi}_{1}(\bm{\Theta}_{1})=-\bm{\Phi}_{1}(0)=-\eta_1\E_{\bm{\gamma}}[\min_{\bm{\rho}} \langle \bm{\rho}, \bm{\gamma} \rangle]$\\$=\eta_1\E_{\bm{\gamma}}[\max_{\bm{\rho}\in \mathcal{X}} \langle \bm{\rho}, \bm{\gamma} \rangle] $ since Gaussian random variables are symmetric. By using the result of Lemma 9 in \cite{cohen2015following}, we get
\begin{equation}
-\bm{\Phi}_{1}(0)\le \eta_1 \sqrt{2 \log K}. \label{eq:FTPL_reg_term1}
\end{equation}
The second term can be analyzed by considering
\begin{align*}
&\bm{\Phi}_{t-1}(\bm{\Theta}_{t})- \bm{\Phi}_t(\bm{\Theta}_t) \nonumber\\
&=\E_{\bm{\gamma}} [ \min_{\bm{\rho}\in \mathcal{X}} \langle \bm{\rho},\bm{\theta}_{t} + \eta_{t-1} \bm{\gamma} \rangle- \min_{\bm{\rho}\in \mathcal{X}} \langle \bm{\rho},\bm{\theta_t} + \eta_t \bm{\gamma} \rangle ]\\
&\overset{(a)}{\le} \E_{\bm{\gamma}} [ \max_{\bm{\rho}\in \mathcal{X}} \langle \bm{\rho}, (\eta_{t-1}-\eta_{t}) \bm{\gamma} \rangle ] \\
&\le |\eta_t-\eta_{t-1}|\sqrt{2\log K}, \numberthis \label{eq:FTPL_reg_term2}
\end{align*}
where $(a)$ follows because $\max_{\bm{y}\in \mathcal{Y}}(f(\bm{y})-g(\bm{y}) )\ge \max_{\bm{y}\in \mathcal{Y}}f(\bm{y})-\max_{\bm{y}\in \mathcal{Y}}g(\bm{y})$ for any arbitrary functions $f$, $g$. We obtain \eqref{eq:FTPL_reg_term2} by using Lemma 9 in \cite{cohen2015following}.
Now we bound the third term in \eqref{eq:Reg_ftpl_adv}. Let $H=\nabla^2\Phi_t(\tilde{\bm{\theta}}_t)$. By using Lemma 7 of \cite{abernethy2014online} we have,
\begin{align*}
H_{i,j}=\frac{1}{\eta_t} \E[\hat{x}(\tilde{\bm{\theta}_t}+\eta_t \bm{\gamma} )_i \bm{\gamma}_j].
\end{align*}
If we replace $\eta_{t}$ with $\eta_{t}/\kappa$ and $\theta_{t}$ with $\theta_{t}/\kappa$ and apply Lemma 2 of \cite{cohen2015following} we get
\begin{align*}
-\bigg\langle \frac{\bm{\theta}_t}{\kappa}, \kappa H \frac{\bm{\theta}_t}{\kappa} \bigg\rangle \le \frac{2\kappa}{\eta_{t}} \sqrt{2\log K} \\
\implies -\langle \bm{\theta}_t, \nabla^2\Phi_t(\tilde{\bm{\theta}}_t) \bm{\theta}_t \rangle \le \frac{2\kappa^2}{\eta_{t}} \sqrt{2\log K}. \numberthis \label{eq:FTPL_reg_term3}
\end{align*}
% The third term in RHS of \eqref{eq:Reg_ftpl_adv} bounded as follows
% \begin{align*}
% -\langle \bm{\theta}_t, \nabla^2\Phi_t(\tilde{\bm{\theta}}_t) \bm{\theta}_t \rangle \le& (||\bm{\theta}_t||_\infty)^2\sum_{i,j} |H_{i,j}|,\\
% \le& \kappa^2 \sum_{i,j} |H_{i,j}|,
% \end{align*}
% where $H=\nabla^2\Phi_t(\tilde{\bm{\theta}}_t)$. By using Lemma 7 of \cite{abernethy2014online} we have,
% \begin{align*}
% H_{i,j}=\frac{1}{\eta_t} \E[\hat{x}(\tilde{\bm{\theta}_t}+\eta_t \bm{\gamma} )_i \bm{\gamma}_j],
% \end{align*}
% By using Lemma 2 of \cite{cohen2015following} we have $\sum_{i,j}|H_{i,j}|\le \sqrt{2\logi^\prime}$, therefore
% \begin{align}\color{red}
% -\langle \bm{\theta}_t, \nabla^2\Phi_t(\tilde{\bm{\theta}}_t) \bm{\theta}_t \rangle \le \frac{\kappa^2}{\eta_{t}} \sqrt{2\logi^\prime} \label{eq:FTPL_reg_term3}
% \end{align}
% where $\hat{x}(z) \in \argmin_{\bm{\rho} \in \mathcal{X}} \langle\bm{\rho}, \bm{z} \rangle$. By using Jensen's inequality
% \begin{align*}
% |H_{i,j}|&\le \frac{1}{\eta_t}\E\left[ |\hat{x}(\tilde{\bm{\theta}_t}+\eta_t \bm{\gamma} )_i |\ |\bm{\gamma}_j|\right] \\
% &\le \frac{1}{\eta_t}\E[|\gamma_j|]\\
% &\le \frac{1}{\eta_t}\sqrt{\frac{2}{\pi}}. \numberthis
% \end{align*}
% Therefore,
% \begin{align}
% -\langle \bm{\theta}_t, \nabla^2\Phi_t(\tilde{\bm{\theta}}_t) \bm{\theta}_t \rangle \le (||\bm{\theta}_t||_\infty)^2 \frac{2}{\eta_t} \sqrt{\frac{2}{\pi}}. \label{eq:FTPL_reg_term3}
% \end{align}
By using \eqref{eq:FTPL_reg_term1}, \eqref{eq:FTPL_reg_term2} and \eqref{eq:FTPL_reg_term3}, we bound \eqref{eq:Reg_ftpl_adv},
\begin{align*}
\mathcal{R}^{\text{FTPL}}_A(T)\le & \eta_1 \sqrt{2\log K}+\sum_{t=2}^{T} |\eta_t-\eta_{t-1}|\sqrt{2\log K}\\
&+\sum_{t=1}^{T} \frac{2\kappa^2}{\eta_t} \sqrt{2\log K}\\
\le& \eta_T \sqrt{2\log K} + \sum_{t=1}^{T} \frac{2\kappa^2}{\eta_t} \sqrt{2\log K}\\
\le&\eta_T \sqrt{2\log K} + 2\kappa^2 \sqrt{2\log K} \sum_{t=1}^{T}\frac{1}{\eta_t}.
\end{align*}
% \begin{align*}
% \mathcal{R}^{\text{FTPL}}_A(T)&\le \eta_T \sqrt{2\log K} + \sum_{t=1}^{T} (||\bm{\theta}_t||_\infty)^2 \frac{K^2}{\eta_t} \sqrt{\frac{2}{\pi}}\\
% &\le\eta_T \sqrt{2\log K} + \kappa^2 K^2 \sqrt{\frac{2}{\pi}} \sum_{t=1}^{T}\frac{1}{\eta_t}.
% \end{align*}
\end{proof}
Now we bound the expected fetch cost incurred by FTPL policy.
\begin{lemma}\label{lem:FTPL_swcost_adv}
The expected fetch cost under FTPL policy upto time $T$ denoted by $\E[\mathcal{C}^{\text{FTPL}}_{F}(T)]$ with learning rate $\eta_t=\alpha \sqrt{t}$ is bounded as follows
$$\E[\mathcal{C}^{\text{FTPL}}_{F}(T)] \le \frac{MK^2 (c+2\kappa)}{2\alpha\sqrt{\pi}}{\sqrt{T+1}}.$$
\end{lemma}
\begin{proof} The proof of this theorem follows similar a line of arguments as in \cite{mukhopadhyay2021online} for the case with $N=2$ and cache size 1 case.
The fetch cost is incurred only when the service is fetched. The overall cost of fetching is $\sum_{t=1}^{T}M (\rho_{t+1}-\rho_{t})^+$. So, the expected fetch cost is given by
\begin{align*}
\E[\mathcal{C}^{\text{FTPL}}_{F}(T)]&= \E\left[ \sum_{t=1}^{T}M (\rho_{t+1}-\rho_{t})\mathds{1}_{\{\rho_{t+1}>\rho_{t}\}} \right]\\
&\le M\sum_{t=1}^T \Prob(\rho_{t+1}>\rho_{t})\\
&\le M\sum_{t=1}^T \sum_{j>i} \Prob(\rho_{t+1}=\alpha_{j},\rho_{t}=\alpha_{i}).\\
\end{align*}
%If we bound $\Prob(\rho_{t+1}=\alpha_{j},\rho_{t}=\alpha_{i})$, then we bound the expected fetch cost. Note that $\rho_t<\rho_{t+1}$ happens if $\alpha_i<\alpha_j$ and $c\alpha_i (t-1)+g(\alpha_i)R_{t-1}+\eta_t \gamma_i< c\alpha_j (t-1)+g(\alpha_j)R_{t-1}+\eta_t \gamma_j$ and $c\alpha_i t+g(\alpha_i)R_{t}+\eta_{t+1} \gamma_i> c\alpha_j t+g(\alpha_j)R_{t}+\eta_{t+1} \gamma_j$.
Let $\mathcal{E}_{i,j}^{(t)}$ be the event that $\rho_{t}=\alpha_{i}$, $\rho_{t+1}=\alpha_{j}$. $\mathcal{E}_{i,j}^{(t)}$ occurs only if $\Theta_{t,i}+\eta_t \gamma_i< \Theta_{t,j}+\eta_t \gamma_j$, $\Theta_{t+1,i}+\eta_{t+1} \gamma_i> \Theta_{t+1,j}+\eta_{t+1} \gamma_j$. Let $\delta_\alpha=\alpha_j-\alpha_i$, $\delta_g=g(\alpha_i)-g(\alpha_j)$, note that $0<\delta_\alpha\le1$, $0<\delta_g\le 1$.
\begin{align*}
&\Prob(\mathcal{E}_{i,j}^{(t)})\\
&\le\Prob\left(\Theta_{t,i}+\eta_{t} \gamma_i< \Theta_{t,j}+\eta_{t} \gamma_j,\right.\\
&\quad\qquad \left.\Theta_{t+1,i}+\eta_{t+1} \gamma_i> \Theta_{t+1,j}+\eta_{t+1} \gamma_j\right)\\
&= \Prob\left( \frac{\Theta_{t+1,j}-\Theta_{t+1,i}}{\sqrt{2}\eta_{t+1}} < \frac{(\gamma_i-\gamma_j)}{\sqrt{2}} <\frac{\Theta_{t,j}-\Theta_{t,i}}{\sqrt{2}\eta_{t}} \right)\\ %\numberthis \label{eq:FTPL_sw_prob}\\
&\overset{(a)}{\le} \frac{\Theta_{t,j}-\Theta_{t,i}}{2\sqrt{\pi}\eta_{t}}- \frac{\Theta_{t+1,j}-\Theta_{t+1,i}}{2\sqrt{\pi}\eta_{t+1}}\\ %\numberthis \label{eq:FTPL_sw_prob}\\
&=\frac{1}{2\sqrt{\pi}}\left( (c(t-1)\delta_\alpha- \delta_gR_{t-1})\left(\frac{1}{\eta_{t}} -\frac{1}{\eta_{t+1}}\right) \right)\\
&\quad- \frac{1}{2\sqrt{\pi}} \left(\frac{c\delta_\alpha-\delta_g r_{t}}{\eta_{t+1}}\right)\\ %\numberthis \label{eq:gaus_prob}\\
&\le \frac{1}{2\sqrt{\pi}} \left(c t\left(\frac{1}{\eta_{t}}-\frac{1}{\eta_{t+1}}\right)+\frac{\delta_g r_{t}-c\delta_\alpha}{\eta_{t+1}}\right)\\
&\overset{(b)}{\le} \frac{1}{2\sqrt{\pi}} \left(c t \left(\frac{1}{\eta_{t}} - \frac{1}{\eta_{t+1}}\right) + \frac{\kappa}{\eta_{t+1}}\right) \numberthis \label{eq:FTPL_sw_eta}\\
% \end{align*}
% \begin{align*}
&=\frac{1}{2 \alpha\sqrt{\pi}} \left(c t\left( \frac{1}{\sqrt{t}}-\frac{1}{\sqrt{t+1}}\right)+\frac{\kappa}{\sqrt{t+1}}\right)\\
&=\frac{1}{2 \alpha\sqrt{\pi}} \left( c\sqrt{\frac{t}{t+1}} \frac{1}{\sqrt{t}+\sqrt{t+1}} +\frac{\kappa}{\sqrt{t+1}}\right)\\
&\overset{(c)}{\le}\frac{(c+2\kappa)}{4 \alpha\sqrt{\pi(t+1)}}.
\end{align*}
Here $(a)$ follows from the fact that $\Prob(a<Z<b)\le \frac{b-a}{\sqrt{2\pi}}$, where $Z\sim\mathcal{N}(0,1)$, $(b)$ follows because $\delta_g\le 1$ and $r_t\le \kappa$ and, $(c)$ follows because $\frac{\sqrt{t}}{\sqrt{t}+\sqrt{t+1}}\le \frac{1}{2}$.
Therefore,
\begin{align*}
\E[\mathcal{C}_f^{\text{FTPL}}(T)] &\le \frac{M(c+2\kappa)}{4\alpha\sqrt{\pi}} \sum_{t=1}^T \sum_{j>i} \frac{1}{\sqrt{t+1}}\\
& \le \frac{M K^2(c+2\kappa)}{2\alpha\sqrt{\pi}}{\sqrt{T+1}}.
\end{align*}
\end{proof}
%\begin{theorem}
% The expected regret of FTPL policy with fetch cost $M\ne0$, $\eta_{t}=\alpha \sqrt{t}$, $\alpha>0$ for oblivious adversary is bounded as follows
% \begin{align*}
% \mathcal{R}^{\text{FTPL}}_A(T)\le& \sqrt{2T\log K}\left( \alpha + \frac{4\kappa^2}{\alpha}\right) +\frac{M(c+2\kappa)}{2\alpha\sqrt{\pi}} \sqrt{T}.
% \end{align*}
% \label{thm:FTPL_reg_adv}
%\end{theorem}
\begin{proof}[Proof of Theorem \ref{thm:reg_adv}(c)]
By using Lemma \ref{lem:FTPL_reg_adv} and \ref{lem:FTPL_swcost_adv} we get,
\begin{align*}
\mathcal{R}^{\text{FTPL}}_A(T)\le&\sqrt{2\log K}\left( \alpha\sqrt{T} + 2\kappa^2 \sum_{t=1}^{T}\frac{1}{\alpha\sqrt{t}}\right) \\
&+\frac{K^2M(c+2\kappa)}{2\alpha\sqrt{\pi}} \sqrt{T+1}\\
\le& \sqrt{2T\log K}\left( \alpha + \frac{4\kappa^2}{\alpha}\right)\\ &+\frac{K^2M(c+2\kappa)}{2\alpha\sqrt{\pi}} \sqrt{T+1}.
\end{align*}
\end{proof}
\subsection{Proof of Theorem \ref{thm:reg_adv}(d)}
%\color{violet}
%\begin{lemma}
% For $\eta_{t}=\alpha\sqrt{t}$, $\alpha>0$ without partial hosting, we have
% $$\mathcal{R}_A^{\text{FTPL}}(T) \ge bM,$$
% where $b=\Prob \left(\frac{2c-\kappa}{2\alpha}\le Z\le \frac{c}{\alpha\sqrt{2}} \right)$, $Z\sim \mathcal{N}(0,1)$.
% \label{thm:FTPL_comp_adv_lb}
%\end{lemma}
\begin{proof}[Proof of Theorem \ref{thm:reg_adv}(d)]
We characterize the fetch cost under FTPL to analyze the impact of $M$ on the total cost incurred under FTPL. Fetch cost is incurred if $\rho_{t-1}=0$, $\rho_{t}=1$. Consider a request sequence $r_1=1$ and $r_t=0$ for all $t\ge2$. The optimal static policy is to not host any fraction of service i.e., to forward all the requests to back-end server.
% The probability of incurring a fetch cost in slot $t=1$ with $\eta_{t}=\alpha \sqrt{t}$, $\alpha>0$ is equal to probability of hosting any non zero fraction of service in slot $t=1$.
The probability of hosting $\alpha_{i}$, $1\le i\le K$ fraction of service in first slot is $1/K$. Hosting any non zero fraction of service will incur at least $M\alpha_{2}$ fetch cost. Therefore the expected fetch cost of FTPL policy will be at least $M\alpha_{2}/K$ which is a lower bound on the expected cost of FTPL policy for this request sequence. For FTPL we can decouple the fetch cost and the non fetch cost and FTPL does not consider $M$ while making decisions. Therefore $M\alpha_{2}/K$ is also a lower bound for regret of FTPL. This completes the proof.
% We characterize the fetch cost under FTPL to analyze the impact of $M$ on the total cost incurred under FTPL. Fetch cost is incurred if $\rho_{t-1}=0$, $\rho_{t}=1$. Consider a request sequence $r_t=0$ for all $t\ne2$ and $r_2=\kappa$. The probability of \color{cyan} incurring a fetch cost in slot $t=3$ with $\eta_{t}=\alpha \sqrt{t}$ is equal to $\Prob \left(\frac{2c-\kappa}{2\alpha}\le \frac{\gamma_1-\gamma_2}{\sqrt{2}} \le \frac{c}{\alpha\sqrt{2}} \right) $. For $c\le\kappa$, under FTPL, the probability of \color{cyan} incurring a fetch cost in slot $t=3$ is a constant $b > 0$ (independent of $M$). Therefore expected fetching cost of FTPL policy is lower bounded by $bM$ which is also a lower bound on the expected cost of FTPL policy for this request sequence. \color{cyan} For FTPL we can decouple the fetch cost and the non fetch cost and FTPL does not consider $M$ while making decisions. Therefore $bM$ is also a lower bound for regret of FTPL. This completes the proof.
\end{proof}
\subsection{Proof of Theorem \ref{thm:reg_adv}(e)}
%\begin{theorem} \label{thm:WFTPL_adv_Reg_UB}
% The expected regret under W-FTPL with $M>1$, $\eta_t=\alpha \sqrt{t}$, $\alpha>0$, $\beta>0$, $\delta>0$ is bounded as follows
% \begin{align*}
% \mathcal{R}^{\text{W-FTPL}}_A(T)\le& \sqrt{2T\log K} \left(\alpha + \frac{4\kappa^2}{\alpha} \right)+\frac{M(c+2\kappa)}{2\alpha\sqrt{\pi}} \sqrt{T}\\
% &+\sqrt{\beta T (\log M)^{1+\delta}}
% \end{align*}
%\end{theorem}
\begin{proof}[Proof of Theorem \ref{thm:reg_adv}(e)]
Recall that $T_s=\min\{t:t<\frac{(\max_{i\ne j}(\Theta_{t+1,i}-\Theta_{t+1,j}))^2}{\kappa^2\beta (\log M)^{1+\delta}}\}$. For arrival sequences with requests arriving in time-slots 1 to $T$, we have two possible cases, namely, $T_s\ge T$ and $T_s<T$. We consider each case to bound the regret of W-FTPL policy.
%First let us consider the case when $T_S>T$ and bound the regret and later considered the case $T_s<T$ and bound the regret. The maximum among both cases will be considered to get the final regret upper bound.\\
\textit{Case I ($T_s\ge T$)}: Let $r^{(1)}$ be a request sequence chosen by adversary such that $T_s\ge T$, then
\begin{align*}
% T<\frac{\beta (\log M)^{1+\delta}}{(\max_{i\ne j}c(\alpha_j-\alpha_i)-(g(\alpha_i)-g(\alpha_j))R_T/T)^2}\\
|\max_{i\ne j}(\Theta_{T+1,j}-\Theta_{T+1,i})|&<\kappa\sqrt{\beta T (\log M)^{1+\delta}}\\
\implies |\max_{i}(\Theta_{T+1,1}-\Theta_{T+1,i})|&<\kappa\sqrt{\beta T (\log M)^{1+\delta}}\\
\overset{(a)}{\implies}\mathcal{R}^{\text{W-FTPL}}_A(T) &<\kappa\sqrt{\beta T (\log M)^{1+\delta}},
% \implies \mathcal{R}^{\text{W-FTPL}}_{A}(T_s)&<\kappa\sqrt{\beta T (\log M)^{1+\delta}}
\end{align*}
$(a)$ follows because $\rho_{t}=0$ for $t<T_s$ for W-FTPL.
\textit{Case II ($T_s<T$)}: Let $r^{(2)}$ be a request sequence chosen by adversary such that $T_s<T$, then by using case I we can bound
\begin{align*}
% &T_s -1<\frac{\beta (\log M)^{1+\delta}}{(\max_{i\ne j}(\Theta_{T_s,i}-\Theta_{T_s,j})/(T_{s}-1))^2}\\
|\max_{i\ne j}\Theta_{T_s,i}-\Theta_{T_s,j}|&<\kappa\sqrt{\beta (T_s-1) (\log M)^{1+\delta}}\\
\implies \mathcal{R}^{\text{W-FTPL}}_{A}(T_s-1)&<\kappa\sqrt{\beta T (\log M)^{1+\delta}}.
\end{align*}
Thus combining both cases we get upper bound on regret in wait phase as
\begin{align}
\mathcal{R}^{\text{W-FTPL}}_{A}(T_s-1)&<\kappa\sqrt{\beta T (\log M)^{1+\delta}}. \label{eq:WFTPL_wait}
\end{align}
Note that W-FTPL policy follows FTPL policy after its waiting time i.e., from time $T_s$. Therefore W-FTPL and FTPL take same decisions and have same cost from time $T_s$ to $T$. Thus the regret in a slot is also same for W-FTPL and FTPL from time $T_s$ to $T$. If $T_s\ge T$ then only wait phase will be there and regret in FTPL phase is considered to be zero. We denote regret from time $t_1$ to $t_2$ as $\mathcal{R}_A^{\mathcal{P}}(t_1:t_2)$.
\begin{align*}
\mathcal{R}_A^{\text{W-FTPL}}(T)&=\mathcal{R}_A^{\text{W-FTPL}}(T_s-1) + \mathcal{R}_A^{\text{W-FTPL}}(T_s:T)\\
&=\mathcal{R}_A^{\text{W-FTPL}}(T_s-1)+\mathcal{R}_A^{\text{FTPL}}(T_s:T)\\
&\le \mathcal{R}_A^{\text{W-FTPL}}(T_s-1)+\mathcal{R}_A^{\text{FTPL}}(1:T). \numberthis \label{eq:ftpl_wftpl_adv_reg}
\end{align*}
By \eqref{eq:WFTPL_wait}, \eqref{eq:ftpl_wftpl_adv_reg} and Theorem \ref{thm:reg_adv}(c) we get
\begin{align*}
\mathcal{R}^{\text{W-FTPL}}_A(T)\le& \kappa\sqrt{\beta T (\log M)^{1+\delta}}\\
&+\sqrt{2T\log K} \left( \alpha+ \frac{4\kappa^2}{\alpha^2}\right)\\ &+\frac{K^2M(c+2\kappa)}{2\alpha\sqrt{\pi}} \sqrt{T+1}.
\end{align*}
\end{proof}
\subsection{Proof of Theorem \ref{thm:reg_stch}(a)}
%\begin{theorem} \label{thm:RR_reg_LB_stch}
% The regret of RR policy is bounded as follows
% $$\mathcal{R}^{\text{RR}}_{S}(T) =\Omega(T)$$
% %\ge M\left(\frac{T}{\frac{M}{\kappa-c\alpha_{i'}-g(\alpha_{i'})\kappa}+ \frac{M}{c} +2}-1\right) p,$$
%% where $p>0$, $\alpha_{i'}=\argmin_{\alpha_i\ne 0}c\alpha_i+g(\alpha_i)\kappa$.
%\end{theorem}
\begin{proof}[Proof of Theorem \ref{thm:reg_stch}(a)]
From Lemma \ref{lem:RR_min_fetch} we know that once RR fetches any non-zero fraction of service, it hosts (some positive fraction of the service) for at least $\left \lceil \frac{M}{c}\right \rceil$ slots before it evicts completely. From Lemma \ref{lem:RR_min_evict}, we know that, once RR evicts the complete service from the edge, then it does not fetch any non zero fraction of the service for at least $\left \lceil \frac{M\alpha_{i'}}{\kappa-c\alpha_{i'}-g(\alpha_{i'})\kappa}\right \rceil$ slots.
We divide the entire time $T$ into frames of size $f_s=\left \lceil \frac{M}{c}\right \rceil + \left \lceil \frac{M\alpha_{i'}}{\kappa-c\alpha_{i'}-g(\alpha_{i'})\kappa}\right \rceil$ slots. %Since requests are generated i.i.d across time, the probability of observing $\kappa$ requests or zero requests in a slot are independent of time horizon $T$.
Let us define an event $\mathcal{F}$ as the frame that starts with $\left \lceil \frac{M\alpha_{i'}}{\kappa-c\alpha_{i'}-g(\alpha_{i'})\kappa}\right\rceil$ slots with $\kappa$ arrivals in each slot and followed by $\left\lceil \frac{M}{c}\right \rceil$ slots of zeros arrivals. Let the probability of event $\mathcal{F}$ occurring be $\Prob(\mathcal{F})$ (independent of $T$). Conditioned on event $\mathcal{F}$, if optimal static policy is to host non zero fraction of service then we have regret at least $(\kappa -c\alpha_i +g(\alpha_i)\kappa)\left \lceil \frac{M\alpha_{i'}}{\kappa-c\alpha_{i'}-g(\alpha_{i'})\kappa}\right \rceil>0$ in a frame where $\alpha_i\ne 0$. If optimal is not to host any fraction of service then we have regret of $c\alpha_{i}\left \lceil\frac{M}{c}\right \rceil>0$ in a frame where $\alpha_{i}\ne 0$. Therefore conditioned on event $\mathcal{F}$ RR always has a finite nonzero regret say $d(>0)$ which is independent of $T$. Therefore,
\begin{align*}
\mathcal{R}^{\text{RR}}_S(T)&= \E_{r} \left[\sum_{f=1}^{\lfloor T/f_s \rfloor} d \mathds{1}(\text{event $\mathcal{F}$ occured})\right]\\
&\ge \left(\frac{T}{f_s}-1\right) \Prob(\mathcal{F})d \\
&\ge \left(\frac{T}{\frac{M}{\kappa-c\alpha_{i'}-g(\alpha_{i'})\kappa}+ \frac{M}{c} +2}-1\right) \Prob(\mathcal{F})d\\
&=\Omega(T).
\end{align*}
So even in the stochastic case RR observes linear regret.
\end{proof}
\subsection{Proof of Theorem \ref{thm:reg_stch}(b)}
\begin{lemma} \label{lem:FTPL_subOpt_prob}
Probability of hosting a sub-optimal fraction of service $\alpha_j\ne \alpha_{i^\star}$ under the FTPL policy in time slot $t+1$ is bounded as follows
% The regret for FTPL policy with learning rate $\eta_t=\alpha \sqrt{t-1}$ and $M=0$ is given by
% \begin{align*}
% &\mathcal{R}^{\text{FTPL}}_{S}(T)\le \sum_{i\ne i^\star}\frac{16\alpha^2+3\kappa^2}{\Delta_i}
% %\le \sum_{j\ne i^\star} \sum_{t=1}^T \Delta_j \exp\left(-\frac{t^2\Delta_j^2}{16\eta_{t}^2}\right) + \exp\left(-2\Delta_j^2t\right)
% \end{align*}
\begin{align*}
\Prob(\rho_{t+1}=\alpha_j)\le \exp\left(-\frac{t^2\Delta_j^2}{16\eta_{t+1}^2}\right) + \exp\left(\frac{-\Delta_j^2t}{2\kappa^2}\right).
\end{align*}
\end{lemma}
\begin{proof}
FTPL hosts $\alpha_j$ fraction of service if $c\alpha_it+g(\alpha_i)R_t+\eta_{t+1}\gamma_i > c\alpha_jt+g(\alpha_j)R_t+\eta_{t+1}\gamma_j$ in time slot $t+1$, for all $i\ne j$. Let $p_{t,j}=\Prob(\rho_{t+1}=\alpha_{j})$. %We first bound the probability of hosting the sub optimal arm in a time $t+1$ i.e., probability of hosting $\alpha_j$ when $\mu_j>\mu_{i^\star}$.
\begin{align*}
p_{t,j}\le\Prob(c\alpha_jt&+g(\alpha_j)R_t+\eta_{t+1}\gamma_j \\
& < c\alpha_{i^\star}t +g(\alpha_{i^\star})R_t +\eta_{t+1}\gamma_{i^\star})\\
=\Prob(\eta_{t+1}&(\gamma_{i^\star}-\gamma_j) \\
&\ge c(\alpha_j-\alpha_{i^\star})t+ (g(\alpha_j)-g(\alpha_{i^\star}))R_t).
\end{align*}
There can be two possibilities one is $\alpha_j>\alpha_{i^\star}$ or $\alpha_j<\alpha_{i^\star}$ we bound the probability $p_{t,j}$ by considering each case separately.\\
\textit{Case 1 ($\alpha_j>\alpha_{i^\star}$):} Since $g(.)$ is a decreasing function $g(\alpha_j)<g(\alpha_{i^\star})$. Let $\mathcal{E}_j$ be the event that $R_t<\mu t + t\Delta_j/2$.
\begin{align*}
p_{t,j}\le& \Prob(\eta_{t+1}(\gamma_{i^\star}-\gamma_j)\ge c(\alpha_j-\alpha_{i^\star})t \\
&\qquad\qquad + (g(\alpha_j)-g(\alpha_{i^\star}))R_t, \mathcal{E}_j)
+\Prob(\mathcal{E}_j^c)\\
\le&\Prob(\eta_{t+1}(\gamma_{i^\star}-\gamma_j)\ge t\Delta_j-(g(\alpha_{i^\star})-g(\alpha_j))t\Delta_j/2)\\
&+ \Prob(\mathcal{E}_j^c)\\
\le&\Prob(\eta_{t+1}(\gamma_{i^\star}-\gamma_j)\ge t\Delta_j/2 )+ \Prob(\mathcal{E}_j^c)\\
\overset{(a)}{\le}&\exp\left(-\frac{t^2\Delta_j^2}{16\eta_{t+1}^2}\right) + \exp\left(\frac{-\Delta_j^2t}{2\kappa^2}\right),
\end{align*}
where $(a)$ is obtained by using the fact that complementary CDF of standard Gaussian $Q(x)\le e^{-x^2/2}$ for $x>0$ and Hoeffding's inequality \cite{hoeffding1994probability}.
\textit{Case 2 ($\alpha_j<\alpha_{i^\star}$):} Let $\mathcal{E}_j$ be an event $R_t>\mu t - t\Delta_j/2$.
\begin{align*}
p_{t,j}\le& \Prob(\eta_{t+1}(\gamma_{i^\star}-\gamma_j)\ge c(\alpha_j-\alpha_{i^\star})t \\
&\qquad\qquad+ (g(\alpha_j)-g(\alpha_{i^\star}))R_t, \mathcal{E}_j)+\Prob(\mathcal{E}_j^c)\\
\le&\Prob(\eta_{t+1}(\gamma_{i^\star}-\gamma_j)\ge t\Delta_j-(g(\alpha_{j})-g(\alpha_{i^\star}))t\Delta_j/2)\\
&+ \Prob(\mathcal{E}_j^c)\\
\le&\Prob(\eta_{t+1}(\gamma_{i^\star}-\gamma_j)\ge t\Delta_j/2 )+ \Prob(\mathcal{E}_j^c)\\
\overset{(a)}{\le}&\exp\left(-\frac{t^2\Delta_j^2}{16\eta_{t+1}^2}\right) + \exp\left(\frac{-\Delta_j^2t}{2\kappa^2}\right),
\end{align*}
where $(a)$ is obtained by using the fact that $Q(x)\le e^{-x^2/2}$ for $x>0$ and Hoeffding's inequality \cite{hoeffding1994probability}.
Therefore by combining both cases we get
\begin{equation}\label{eq:sub_arm_prob}
\Prob(\rho_{t+1}=\alpha_i)\le \exp\left(-\frac{t^2\Delta_j^2}{16\eta_{t+1}^2}\right) + \exp\left(\frac{-\Delta_j^2t}{2\kappa^2}\right).
\end{equation}
\end{proof}
\begin{comment}
\begin{lemma}\label{lem:FTPL_navie_bound}
For $M=0$ we bound the Regret of FTPL policy as follows
\begin{align*}
\mathcal{R}_{S}^{\text{FTPL}}(T)\le \sum_{i\ne i^\star} \frac{16\alpha^2+3\kappa^2}{\Delta_i}.
\end{align*}
\end{lemma}
\begin{proof}
We use the bound in Lemma \ref{lem:FTPL_subOpt_prob} on probability of not hosting the optimal fraction of service to get the regret bound.
\begin{align*}
\mathcal{R}_{S}^{\text{FTPL}}(T)=&\left( \sum_{t=1}^T c\rho_{t}+g(\rho_t)\mu-\mu_{i^\star}\right) -M\alpha_{i^\star}\\
\le&\sum_{t=1}^T \left( \sum_{i} \mu_i \Prob(\rho_{t}=\alpha_{i})\right) -\mu_{i^\star}\\
=& \sum_{t=1}^T \sum_{i\ne i^\star}\Delta_i \Prob(\rho_{t}=\alpha_{i})\\
=& \sum_{i\ne i^\star}\Delta_i \left( 1+\sum_{t=2}^T\Prob(\rho_{t}=\alpha_{i})\right) \\
\le&\sum_{i\ne i^\star}\Delta_{i}+ \sum_{i\ne i^\star} \sum_{t=2}^T \Delta_i \left[ \exp\left(-\frac{(t-1)^2\Delta_i^2}{16\eta_{t}^2}\right) \right.\\
&+\left. \exp\left(\frac{-\Delta_i^2(t-1)}{2\kappa^2}\right)\right] \\
\le& \sum_{i\ne i^\star} \Delta_{i} + \sum_{i\ne i^\star} \sum_{t=1}^{T-1} \Delta_i \left[ \exp\left(-\frac{t\Delta_i^2}{16\alpha^2}\right) \right.\\
&\left.+ \exp\left(\frac{-\Delta_i^2t}{2\kappa^2}\right)\right] \\
\le& \sum_{i\ne i^\star}\frac{\kappa^2}{\Delta_{i}}+\frac{16\alpha^2+2\kappa^2}{\Delta_i}\\
= &\sum_{i\ne i^\star} \frac{16\alpha^2+3\kappa^2}{\Delta_i}.
\end{align*}
\end{proof}
We improve the bound on regret of FTPL in Lemma \ref{lem:FTPL_navie_bound} to get better dependence on number of storage levels we can have.
\end{comment}
\begin{lemma} \label{lem:FTPL_reg_stch}
For $M=0$,
\begin{align*}
\mathcal{R}_{S}^{\text{FTPL}}(T)\le& \left( \sqrt{2\log K}+\frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}} \right) \left( \alpha + \frac{4\kappa^2}{\alpha}\right)\\
&+ \frac{16\alpha^2+4\kappa^2}{\Delta_{\text{min}}},
\end{align*}
where $h_1= 2\max\{16\alpha^2,2\kappa^2\}$.
\end{lemma}
\begin{proof} Let $t_0=\lceil h_1\frac{\log K}{\Delta_{\text{min}}^2}\rceil$ where $h_1=2\max\{16\alpha^2,2\kappa^2\}$. From \eqref{eq:Regret_stch} we have
\begin{align*}
\mathcal{R}_{S}^{\text{FTPL}}(T)\le &\left( \sum_{t=1}^T c\rho_{t}+g(\rho_t)\mu-\mu_{i^\star}\right)\\
=&\sum_{t=1}^T \left( \sum_{i} \mu_i \Prob(\rho_{t}=\alpha_{i})\right) -\mu_{i^\star}\\
=& \sum_{t=1}^T \sum_{i\ne i^\star}\Delta_i \Prob(\rho_{t}=\alpha_{i})\\
=& \sum_{t=1}^{t_0} \sum_{i\ne i^\star}\Delta_i \Prob(\rho_{t}=\alpha_{i})\\
&+\sum_{t=t_0+1}^T \sum_{i\ne i^\star}\Delta_i \Prob(\rho_{t}=\alpha_{i})\\
\overset{(a)}{\le}& \sqrt{2t_0\log K}\left( \alpha+\frac{4\kappa^2}{\alpha}\right)\\
&+ \sum_{t=t_0+1}^T \sum_{i\ne i^\star}\Delta_i \Prob(\rho_{t}=\alpha_{i}),
\end{align*}
where $(a)$ is obtained by using adversarial regret bound when $M=0$. Now we consider each term in RHS separately and bound them.
\begin{align*}
\sqrt{2t_0\log K}\le& \sqrt{2\log K}\left( 1+\frac{\sqrt{h_1\log K}}{\Delta_{\text{min}}}\right).
% \le& \frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}}.
\end{align*}
By using Lemma \ref{lem:FTPL_subOpt_prob} we have
\begin{align*}
\Delta_{j} \Prob(\rho_{t+1}&=\alpha_{j})\\
\le& \Delta_j\left[ \exp\left(-\frac{t^2\Delta_j^2}{16\eta_{t+1}^2}\right) + \exp\left(\frac{-\Delta_j^2t}{2\kappa^2}\right)\right] \\
= & \Delta_j\left[ \exp\left(-\frac{t\Delta_j^2}{16\alpha^2}\right) + \exp\left(\frac{-\Delta_j^2t}{2\kappa^2}\right)\right]\\
\overset{(b)}{\le}& \Delta_{\text{min}}\left[ \exp\left(-\frac{t\Delta_{\text{min}}^2}{16\alpha^2}\right) + \exp\left(\frac{-\Delta_{\text{min}}^2t}{2\kappa^2}\right)\right].
\end{align*}
Here $(b)$ is obtained using the fact that $f(u)=ue^{-u^2}$ is a decreasing function over $u \in [1,\infty)$, and for $t>t_0$, $\frac{t\Delta_{j}^2}{16\alpha^2} > \frac{h_1\log K\Delta_{j}^2}{16\alpha^2\Delta_{\text{min}}^2} \ge 1$ and $\frac{t\Delta_{j}^2}{2\kappa^2} > \frac{h_1\log K\Delta_{j}^2}{2\kappa^2\Delta_{\text{min}}^2} \ge 1$. Therefore,
\begin{align*}
\mathcal{R}_{S}^{\text{FTPL}}&(T)\\
\le&\left( \sqrt{2\log K}+\frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}}\right) \left( \alpha + \frac{4\kappa^2}{\alpha}\right)\\
&+\sum_{t=t_0}^{T} K\Delta_{\text{min}}\left[ \exp\left(-\frac{t\Delta_{\text{min}}^2}{16\alpha^2}\right) + \exp\left(\frac{-\Delta_{\text{min}}^2t}{2\kappa^2}\right)\right]\\
\overset{(c)}{\le}& \left( \sqrt{2\log K}+\frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}}\right) \left( \alpha + \frac{4\kappa^2}{\alpha}\right)\\
& + \frac{16\alpha^2+2\kappa^2}{\Delta_{\text{min}}}+2\Delta_{\text{min}}\\
\le& \left( \sqrt{2\log K}+\frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}} \right) \left( \alpha + \frac{4\kappa^2}{\alpha}\right)\\
&+ \frac{16\alpha^2+4\kappa^2}{\Delta_{\text{min}}}.
\end{align*}
Here $(c)$ follows because for $t>t_0$ we have $K\exp(-t\Delta_{\text{min}}^2/16\alpha^2)<1$, $K\exp(-t\Delta_{\text{min}}^2/2\kappa^2)<1$ and $\sum_{t=1}^{\infty}e^{-at}\le 1/a$ for $a>0$.
\end{proof}
\begin{comment}
Let $\mathcal{E}_1$ be the event that $R_t<\mu t+|\mu-c|t/2$ and $\mathcal{E}_2$ be the event that $R_t \ge \mu t-|\mu-c|t/2 $.
\begin{align*}
p_t=&\Prob(ct+\eta_t\gamma_2 < R_t+\eta_{t}\gamma_1)\\
=&\Prob(ct+\eta_t\gamma_2 < R_t+\eta_{t}\gamma_1, \mathcal{E}_1)\\
&+\Prob(ct+\eta_t\gamma_2 < R_t+\eta_{t}\gamma_1| \mathcal{E}_1^c)\Prob(\mathcal{E}_1^c)\\
\le& \Prob(ct+\eta_t\gamma_2 < \mu t+|\mu-c|t/2+\eta_{t}\gamma_1)+\Prob(\mathcal{E}_1^c)\\
\overset{(a)}{\le}& \Prob \left( \frac{\gamma_1-\gamma_2} {\sqrt{2}} > \frac{(c-\mu)t-|\mu-c|t/2}{\sqrt{2}\eta_{t}} \right)+ e^{-2|\mu-c|^2t }.
\end{align*}
We get $(a)$ by using Hoeffding's inequality \cite{hoeffding1994probability} on $\Prob(\mathcal{E}_1^c)$. Let us define $q_t=1-p_t$
\begin{align*}
q_t=&\Prob(ct+\eta_t\gamma_2 \ge R_t+\eta_{t}\gamma_1)\\
=&\Prob(ct+\eta_t\gamma_2 \ge R_t+\eta_{t}\gamma_1, \mathcal{E}_2)\\
&+\Prob(ct+\eta_t\gamma_2 \ge R_t+\eta_{t}\gamma_1| \mathcal{E}_2^c)\Prob(\mathcal{E}_2^c)\\
\le& \Prob(ct+\eta_t\gamma_2 \ge \mu t-|\mu-c|t/2+\eta_{t}\gamma_1)+\Prob(\mathcal{E}_2^c)\\
\overset{(b)}{\le}& \Prob \left( \frac{\gamma_2-\gamma_1} {\sqrt{2}} > \frac{(\mu-c)t-|\mu-c|t/2}{\sqrt{2}\eta_{t}} \right)+e^{-2|\mu-c|^2t}.
\end{align*}
We get $(b)$ by using Hoeffding's inequality \cite{hoeffding1994probability} on $\Prob(\mathcal{E}_2^c)$. Now we consider the cases where $c<\mu$ and $c>\mu$ and analyze them separately.
\textit{Case1} - ($c<\mu$): In this case static optimal policy will always host the service and the cost of policy at time $T$ is $cT$. So the expected regret in this case is given as
\begin{align*}
&\E[\mathcal{R}^\text{FTPL}_T]\\
&= \left( \sum_{t=1}^{T}c p_t+ \mu(1-p_t) \right) -cT\\
&= \sum_{t=1}^T (\mu-c)(1-p_t)\\
&\le(\mu-c)\sum_{t=1}^T \Prob\left(\frac{\gamma_2-\gamma_1}{\sqrt{2}} \ge \frac{(\mu-c)t-|\mu-c|t/2}{\sqrt{2}\eta_{t}} \right)+e^{-2|\mu-c|^2t}\\
&\le (\mu-c) \sum_{t=1}^T \exp\left( -\frac{((\mu-c)t-|\mu-c|t/2)^2}{4\eta_{t}^2}\right) +e^{-2|\mu-c|^2t} \\
&\le (\mu-c) \sum_{t=1}^T \exp\left( -\frac{(\mu-c)^2t^2}{16\eta_{t}^2}\right) +e^{-2|\mu-c|^2t}. \numberthis \label{eq:case1}
\end{align*}
\textit{Case2} - $(c>\mu)$: In this case the optimal static policy is to forward everything to cloud and the expected cost incurred is $\mu T$. The expected regret in this case is given as
\begin{align*}
&\E[\mathcal{R}^\text{FTPL}_T]\\
&= \left( \sum_{t=1}^{T}c p_t+ \mu(1-p_t) \right) -\mu T\\
&= \sum_{t=1}^T (c-\mu)p_t\\
&\le(c-\mu)\sum_{t=1}^T \Prob \left(\frac{\gamma_1-\gamma_2}{\sqrt{2}} > \frac{(c-\mu)t-|\mu-c|t/2}{\sqrt{2}\eta_{t}} \right)+e^{-2|\mu-c|^2t}\\
&\le (c-\mu) \sum_{t=1}^T \exp\left( -\frac{((c-\mu)t-|\mu-c|t/2)^2}{4\eta_{t}^2}\right) +e^{-2|\mu-c|^2t} \\
&\le (c-\mu) \sum_{t=1}^T \exp\left( -\frac{(c-\mu)^2t^2}{16\eta_{t}^2}\right) +e^{-2|\mu-c|^2t}.\numberthis \label{eq:case2}
\end{align*}
Using equations \eqref{eq:case1} and \eqref{eq:case2} we get
\begin{align*}
&\E[\mathcal{R}^{FTPL}_T]\\
&\le |c-\mu|\sum_{t=1}^T\left( \exp\left( -\frac{|\mu-c|^2t^2}{16\eta_{t}^2}\right)
+e^{-2|\mu-c|^2t}\right).
\end{align*}
\end{proof}
\end{comment}
\begin{lemma} \label{lem:FTPL_swcost_stc}
Fetch cost under FTPL policy with learning rate $\eta_t=\alpha\sqrt{t-1}$ is bounded as follows
$$\E[\mathcal{C}_f^{\text{FTPL}}(T)]\le M \frac{16\alpha^2+2\kappa^2}{\Delta_{\text{min}}^2}.$$
\end{lemma}
\begin{proof}
The fetch cost is incurred when we fetch extra fraction of service. Similar to the proof of Lemma \ref{lem:FTPL_swcost_adv} we get
\begin{align*}
\E[\mathcal{C}^{\text{FTPL}}_{f}(T)]&\le M\sum_{t=1}^T \Prob(\rho_t<\rho_{t+1})\\
&= M\sum_{t=1}^T \sum_{j>i}\Prob(\rho_t=\alpha_{i}, \rho_{t+1}=\alpha_{j}).
\end{align*}
Let $\mathcal{E}_{i,j}^{(t)}$ be the event that $\rho_{t}=\alpha_{i}$, $\rho_{t+1}=\alpha_{j}$. $\mathcal{E}_{i,j}^{(t)}$ occurs if $\Theta_{t,i}+\eta_t \gamma_i< \Theta_{t,j}+\eta_t \gamma_j$, $\Theta_{t+1,i}+\eta_{t+1} \gamma_i> \Theta_{t+1,j}+\eta_{t+1} \gamma_j$.
\begin{align*}
\Prob(\mathcal{E}_{i,j}^{(t)})
&=\Prob\left(\frac{\Theta_{t+1,j}-\Theta_{t+1,i}}{\sqrt{2}\eta_{t+1}} \le \frac{\gamma_i-\gamma_j}{\sqrt{2}} \le \frac{\Theta_{t,j}-\Theta_{t,i}}{\sqrt{2}\eta_{t}}\right).
\end{align*}
We consider two cases $\mu_i>\mu_j$ and $\mu_i<\mu_j$ and bound the probability.\\
\textit{Case 1 ($\mu_i>\mu_j$):} Let $\mathcal{E}_1$ be the event that $|R_{t-1}-\mu (t-1)| \le |\Delta_{ij}| (t-1)/2$,
\begin{align*}
\Prob(&\mathcal{E}_{i,j}^{(t)})\\
\le&\Prob\left(\frac{\gamma_i-\gamma_j}{\sqrt{2}}\le \frac{\Theta_{t,j}-\Theta_{t,i}}{\sqrt{2}\eta_{t}}\right)\\
\le&\Prob\left(\frac{\gamma_i-\gamma_j}{\sqrt{2}}\le \frac{\Theta_{t,j}-\Theta_{t,i}}{\sqrt{2}\eta_{t}},\mathcal{E}_1\right)+\Prob(\mathcal{E}_1^c)\\
% \le&\Prob\left(\frac{\gamma_i-\gamma_j}{\sqrt{2}} \le \frac{c(\alpha_{j}-\alpha_i)(t-1)-(g(\alpha_i)-g(\alpha_j))R_{t-1}}{\sqrt{2}\eta_{t}}\right)\\
=&\Prob\left(\frac{\gamma_i-\gamma_j}{\sqrt{2}}\le \frac{c(\alpha_{j}-\alpha_i)(t-1)}{\sqrt{2}\eta_{t}}\right.\\
&\hspace{2.7cm} \left. -\frac{(g(\alpha_i)-g(\alpha_j))R_{t-1}}{\sqrt{2}\eta_{t}} ,\mathcal{E}_1\right)+\Prob(\mathcal{E}_1^c)\\
\le&\Prob\left(\frac{\gamma_i-\gamma_j}{\sqrt{2}}\le \frac{-|\Delta_{ij}|(t-1)}{2\sqrt{2}\eta_{t}}\right)+\Prob(\mathcal{E}_1^c)\\
\overset{(a)}{\le}& \exp\left(-\frac{(t-1)^2\Delta_{ij}^2}{16\eta_{t}^2}\right)+ \exp\left(\frac{-\Delta_{ij}^2(t-1)}{2\kappa^2}\right)\\
\le& \exp \left(-\frac{(t-1)^2 \Delta_{\text{min}}^2} {16\eta_{t}^2}\right)+ \exp\left(\frac{-\Delta_{\text{min}}^2(t-1)}{2\kappa^2}\right)\\
\le& \exp \left(-\frac{(t-1) \Delta_{\text{min}}^2} {16\alpha^2}\right)+ \exp\left(\frac{-\Delta_{\text{min}}^2(t-1)}{2\kappa^2}\right),
\end{align*}
where $(a)$ is obtained by using the fact that $Q(x)\le e^{-x^2/2}$ for $x>0$ and Hoeffding's inequality \cite{hoeffding1994probability}.
\textit{Case 2 ($\mu_i<\mu_j$):} Let $\mathcal{E}_2$ be the event that $|R_{t}-\mu t| \le |\Delta_{ij}| t/2$,
\begin{align*}
\Prob&(\mathcal{E}_{i,j}^{(t)})\\
\le&\Prob\left(\frac{\Theta_{t+1,j}-\Theta_{t+1,i}}{\sqrt{2}\eta_{t+1}} \le \frac{\gamma_i-\gamma_j}{\sqrt{2}}\right)\\
\le&\Prob\left(\frac{c(\alpha_{j}-\alpha_i)t-(g(\alpha_i)-g(\alpha_j))R_{t}}{\sqrt{2}\eta_{t}} \le \frac{\gamma_i-\gamma_j}{\sqrt{2}}\right)\\
\le&\Prob\left(\frac{\gamma_i-\gamma_j}{\sqrt{2}}\ge \frac{c(\alpha_{j}-\alpha_i)t-(g(\alpha_i)-g(\alpha_j))R_{t}}{\sqrt{2}\eta_{t}} ,\mathcal{E}_2\right)\\
&+\Prob(\mathcal{E}_2^c)\\
\le&\Prob\left(\frac{\gamma_i-\gamma_j}{\sqrt{2}}\ge \frac{|\Delta_{ij}|t}{2\sqrt{2}\eta_{t}}\right)+\Prob(\mathcal{E}_2^c)\\
\overset{(b)}{\le}& \exp\left(-\frac{t^2\Delta_{ij}^2}{16\eta_{t}^2}\right)+ \exp\left(\frac{-\Delta_{ij}^2t}{2\kappa^2}\right)\\
\le& \exp \left(-\frac{t^2 \Delta_{\text{min}}^2} {16\eta_{t}^2}\right)+ \exp\left(\frac{-\Delta_{\text{min}}^2t}{2\kappa^2}\right)\\
\le& \exp \left(-\frac{t \Delta_{\text{min}}^2} {16\alpha^2}\right)+ \exp\left(\frac{-\Delta_{\text{min}}^2t}{2\kappa^2}\right)\\
\le& \exp \left(-\frac{(t-1) \Delta_{\text{min}}^2} {16\alpha^2}\right)+ \exp\left(\frac{-\Delta_{\text{min}}^2(t-1)}{2\kappa^2}\right),
\end{align*}
where $(b)$ is obtained by using the fact that $Q(x)\le e^{-x^2/2}$ for $x>0$ and Hoeffding's inequality \cite{hoeffding1994probability}.
From combining both cases we get
\begin{align}\label{eq:swFTPL_inq}
\Prob(\mathcal{E}_{i,j}^{(t)})\le \exp \left(-\frac{(t-1) \Delta_{\text{min}}^2} {16\alpha^2}\right)+ \exp\left(\frac{-\Delta_{\text{min}}^2(t-1)}{2\kappa^2}\right) .
\end{align}
Therefore,
\begin{align*}
\E[\mathcal{C}_F^{\text{FTPL}}]\le& M \sum_{t=1}^{T}\sum_{j>i}\Prob(\mathcal{E}_{i,j}^{(t)})\\
\le &MK^2\sum_{t=1}^{T-1} \exp \left(-\frac{t \Delta_{\text{min}}^2}{16\alpha^2}\right)+ \exp\left(\frac{-\Delta_{\text{min}}^2t}{2\kappa^2}\right)\\
&+MK^2\\
\le& MK^2\frac{16\alpha^2+3\kappa^2}{\Delta_{\text{min}}^2}.
\end{align*}
\end{proof}
\begin{comment}
\begin{align*}
&\Prob(\rho_t<\rho_{t+1})\\
=&\Prob\left(\frac{R_{t}t-ct}{\sqrt{2}\eta_{t}}\le \frac{\gamma_2-\gamma_1}{\sqrt{2}}\le \frac{R_{t+1}-c(t+1)}{\sqrt{2}\eta_{t+1}}\right) \\
\le& \Prob\left(\frac{R_{t}-ct}{\sqrt{2}\eta_{t}}\le \frac{\gamma_2-\gamma_1}{\sqrt{2}}\le \frac{R_{t+1}-c(t+1)}{\sqrt{2}\eta_{t+1}}\bigg\vert \mathcal{E}\right)\Prob(\mathcal{E})\\
&\quad+\Prob(\mathcal{E}^c)\numberthis \label{eq:fetch_cost_interm1}
\end{align*}
We consider two cases $c<\mu$ and $c>\mu$ and bound \eqref{eq:fetch_cost_interm1} in each case.\\
\textit{Case1 ($c<\mu$):}
\begin{align*}
&\Prob(\rho_t<\rho_{t+1})\\
&\le \Prob\left(\frac{\gamma_2-\gamma_1}{\sqrt{2}} \ge \frac{R_t-ct}{\sqrt{2}\eta_{t}},\mathcal{E} \right)+e^{-2|\mu-c|^2t}\\
&\le \Prob\left(\frac{\gamma_2-\gamma_1}{\sqrt{2}} \ge \frac{(\mu-c)t-|\mu-c|t/2}{\sqrt{2}\eta_{t}} \right)+e^{-2|\mu-c|^2t}\\
&\le \exp\left( -\frac{((\mu-c)t-|\mu-c|t/2)^2}{4\eta_{t}^2}\right)+e^{-2|\mu-c|^2t}\\
&\le\exp\left( -\frac{|\mu-c|^2t^2}{16\eta_{t}^2}\right)+e^{-2|\mu-c|^2t}.
\end{align*}
\textit{Case2 ($c>\mu$):}
\begin{align*}
&\Prob(\rho_t<\rho_{t+1})\\
\le &\Prob\left(\frac{\gamma_2-\gamma_1}{\sqrt{2}} \le \frac{R_{t+1}-ct-c}{\sqrt{2}\eta_{t}},\mathcal{E} \right)+e^{-2|\mu-c|^2t}\\
\le& \Prob\left(\frac{\gamma_1-\gamma_2}{\sqrt{2}} \ge \frac{(c-\mu)(t+1)-|\mu-c|(t+1)/2}{\sqrt{2}\eta_{t}} \right)\\
&+e^{-2|\mu-c|^2t}\\
\le &\exp\left( -\frac{((c-\mu)-|\mu-c|/2)^2(t+1)^2}{4\eta_{t}^2}\right)+e^{-2|\mu-c|^2t}\\
=&\exp\left( -\frac{(c-\mu)^2(t+1)^2}{16\eta_{t}^2}\right)+e^{-2|\mu-c|^2t}\\
\le&\exp\left( -\frac{|\mu-c|^2t^2}{16\eta_{t}^2}\right)+e^{-2|\mu-c|^2t}.
\end{align*}
Therefore, by combining both cases we get
\begin{align*}
\E[\mathcal{C}_F^{\text{FTPL}}]\le M \sum_{t=1}^T \left( \exp\left( -\frac{|\mu-c|^2t^2}{16\eta_{t}^2}\right)+e^{-2|\mu-c|^2t}\right).
\end{align*}
\end{proof}
\end{comment}
%\begin{theorem}
% The expected regret of FTPL policy with fetch cost $M$, $\eta_{t}=\alpha \sqrt{t-1}$, $\alpha>0$ for stochastic setting is bounded as follows
% \begin{align*}
% \mathcal{R}^{\text{FTPL}}_S(T)\le (16\alpha^2+3\kappa^2)\left( \frac{M}{\Delta_{\text{min}}^2}+\sum_{i\ne i^\star}\frac{1}{\Delta_i}\right) .
% \end{align*}
% \label{thm:FTPL_reg_ub_stch}
%\end{theorem}
\begin{proof}[Proof of Theorem \ref{thm:reg_stch}(b)]
Note that FTPL policy does not consider fetch cost $M$ while taking the decisions. By using Lemma \ref{lem:FTPL_reg_stch}, \ref{lem:FTPL_swcost_stc} we get the result stated.
\begin{comment}
\begin{align*}
\E[\mathcal{R}^{\text{FTPL}}_T]\le& |c-\mu|\sum_{t=1}^T \exp\left( -\frac{|c-\mu|^2 t^2}{16\eta_{t}^2}\right)+ e^{-2|\mu-c|^2t}\\
&+ M\sum_{t=1}^T \exp\left( -\frac{|c-\mu|^2t^2}{16\eta_{t}^2}\right)+e^{-2|\mu-c|^2t},
\end{align*}
by substituting $\eta_{t}=\alpha\sqrt{t}$ we get
\begin{align*}
\E[\mathcal{R}^{\text{FTPL}}_T]\le& |c-\mu|\sum_{t=1}^T \exp\left( -\frac{|c-\mu|^2 t}{16\alpha^2}\right)+ e^{-2|\mu-c|^2t}\\
&+ M\sum_{t=1}^T \exp\left(- \frac{|c-\mu|^2t}{16\alpha^2}\right)+e^{-2|\mu-c|^2t}\\
\le&(M+|c-\mu|) \left( \frac{16\alpha^2+1}{|c-\mu|^2}\right).
\end{align*}
\end{comment}
\end{proof}
\subsection{Proof of Theorem \ref{thm:reg_stch}(c)}
\begin{lemma}\label{lem:T_s,mu_hat}
Under the W-FTPL policy, $T_s>t'$ if and only if there exist $t<t'$ such that the following condition holds:
\begin{align*}
(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t &> -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})+\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}},\\ \text{ or}\\
(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t &< -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})-\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}},
\end{align*}
where $(\tilde{i}_t,\tilde{j}_t)=\argmax_{i\ne j}\Theta_{t+1,i}-\Theta_{t+1,j}$, $\hat{\mu}_t=R_t/t$.
\end{lemma}
\begin{proof}
Let $(\tilde{i}_t,\tilde{j}_t)=\argmax_{i\ne j}\Theta_{t+1,i}-\Theta_{t+1,j}$, $\hat{\mu}_t=R_t/t$. $T_s>t'$ if and only if there exist $t<t'$ such that
\begin{align*}
% &t>\frac{\beta \kappa^2(\log M)^{1+\delta}}{(\max_{i\ne j}(c(\alpha_{i}-\alpha_{j}) + (g(\alpha_{i})-g(\alpha_{j}))R_t/t))^2}\\
\frac{|\max_{i\ne j}(\Theta_{t+1,i}-\Theta_{t+1,j})|}{t}&< \kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}\\
\iff \frac{|\Theta_{t+1,\tilde{j}_t}-\Theta_{t+1,\tilde{i}_t|}}{t} &> \kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}\\
&\Updownarrow\\
|c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t}) + (g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t| &> \kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}\\
&\Updownarrow\\
(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t > -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})&+\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}\\
&\text{or } \\
(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t < -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})&-\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}.
\end{align*}
This completes the proof.
\end{proof}
\begin{lemma}\label{lem:Ts>t0}
Under the W-FTPL policy, $T_s>\frac{(\sqrt{\beta}-1)^2 (\log M)^{1+\delta}}{\Delta_{\text{max}}^2}$ with probability at least $1-\frac{(\sqrt{\beta}-1)^2 (\log M)^{1+\delta}}{M^{2(\log M)^\delta}\Delta_{\text{max}}^2}$.
\end{lemma}
%\begin{proof}
% Let $\mathcal{E}_t$ be the event that $|\mu-\hat{\mu}_t|\le \kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}}$, where $\hat{\mu}_t=R_t/t$. By using Hoeffding's inequality \cite{hoeffding1994probability} we get $\Prob(\mathcal{E}_t)\ge 1-\frac{1}{M^{2(\log M)^\delta}}$. Let $\mathcal{E}=\cap_{t=1}^{T_0}\mathcal{E}_t$, where $T_0=\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{max}}^2}$. Using the union bound, we get: $\Prob(\mathcal{E})\ge 1-\frac{T_0}{M^{2(\log M)^\delta}}$. Let $\epsilon_t=\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}$.
% We prove the lemma by contradiction. Let us consider $T_s<\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{max}}^2}$ and $\mathcal{E}$ holds, then by Lemma \ref{lem:T_s,mu_hat}, $\exists t<\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{max}}^2}$ such that $(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t > -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})+\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}$ or $(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t < -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})-\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}$. There are two possibilities $\alpha_{\tilde{j}_t}<\alpha_{\tilde{i}_t}$, $\alpha_{\tilde{j}_t}>\alpha_{\tilde{i}_t}$ we consider them separately to prove the final result.\\
\begin{proof}
Let $\mathcal{E}_t$ be the event that $$|\mu-\hat{\mu}_t|\le \kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}},$$
where $\hat{\mu}_t=R_t/t$. By using Hoeffding's inequality \cite{hoeffding1994probability} we get $$\Prob(\mathcal{E}_t)\ge 1-\frac{1}{M^{2(\log M)^\delta}}.$$ Let $\mathcal{E}=\cap_{t=1}^{T_0}\mathcal{E}_t$, where $T_0=\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{max}}^2}$. Using the union bound, we get:
$$\Prob(\mathcal{E})\ge 1-\frac{T_0}{M^{2(\log M)^\delta}}.$$
Let $\epsilon_t=\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}.$ We prove the lemma by contradiction. Let us consider $$T_s<\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{max}}^2},$$
and $\mathcal{E}$ holds, then by Lemma \ref{lem:T_s,mu_hat}, $\exists t<\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{max}}^2}$ such that \begin{align*}
(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t &> -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})+\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}},\\ &\text{ or}\\ (g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t &< -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})-\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}},
\end{align*}
holds. There are two possibilities $\alpha_{\tilde{j}_t}<\alpha_{\tilde{i}_t}$, $\alpha_{\tilde{j}_t}>\alpha_{\tilde{i}_t}$ we consider them separately to prove the final result.\\
\textit{Case 1 ($\alpha_{\tilde{j}_t}<\alpha_{\tilde{i}_t}$):} It follows that
\begin{align*}
&(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t > -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})+\epsilon_t\\
&\implies \hat{\mu}_t-\mu >\frac{c(\alpha_{\tilde{i}_t}-\alpha_{\tilde{j}_t})+\epsilon_t} {g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t})}-\mu\\
&\implies \kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}} > \frac{-\Delta_{\text{max}}+\epsilon_t} {g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t})}\\
&\implies (\sqrt{\beta}-g(\alpha_{\tilde{j}_t})+g(\alpha_{\tilde{i}_t}))\kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}}< \Delta_{\text{max}}\\
&\implies (\sqrt{\beta}-1))\kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}}< \Delta_{\text{max}}\\
&\implies t>\frac{(\sqrt{\beta}-1)^2 \kappa^2 (\log M)^{1+\delta}}{\Delta_{\text{max}}^2},
\end{align*}
which is a contradiction. Alternatively,
\begin{align*}
&(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t < -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})-\epsilon_t\\
&\implies \hat{\mu}_t-\mu <\frac{c(\alpha_{\tilde{i}_t}-\alpha_{\tilde{j}_t})-\epsilon_t} {g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t})}-\mu\\
&\implies -\kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}} < \frac{\Delta_{\text{max}}-\epsilon_t} {g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t})}\\
&\implies (\sqrt{\beta}-g(\alpha_{\tilde{j}_t})+g(\alpha_{\tilde{i}_t}))\kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}}< \Delta_{\text{max}}\\
&\implies (\sqrt{\beta}-1))\kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}}< \Delta_{\text{max}}\\
&\implies t>\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{max}}^2},
\end{align*}
which is also a contradiction. \\
\textit{Case 2 }($\alpha_{\tilde{j}_t}>\alpha_{\tilde{i}_t}$): It follows that:
\begin{align*}
& (g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t > -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})+\epsilon_t\\
&\implies \hat{\mu}_t-\mu <\frac{c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})-\epsilon_t} {g(\alpha_{\tilde{i}_t})-g(\alpha_{\tilde{j}_t})}-\mu\\
&\implies -\kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}} < \frac{\Delta_{\text{max}}-\epsilon_t} {g(\alpha_{\tilde{i}_t})-g(\alpha_{\tilde{j}_t})}\\
% \implies (\sqrt{\beta}-g(\alpha_{\tilde{i}_t})+g(\alpha_{\tilde{j}_t}))\sqrt{\frac{(\log M)^{1+\delta}}{t}}&< \Delta_{\text{max}}\\
% \implies (\sqrt{\beta}-1))\sqrt{\frac{(\log M)^{1+\delta}}{t}}&< \Delta_{\text{max}}\\
&\implies t>\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{max}}^2},
\end{align*}
which is a contradiction. Alternatively,
\begin{align*}
&(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t < -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})-\epsilon_t\\
&\implies \hat{\mu}_t-\mu >\frac{c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})+\epsilon_t} {g(\alpha_{\tilde{i}_t})-g(\alpha_{\tilde{j}_t})}-\mu\\
&\implies \kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}} > \frac{-\Delta_{\text{max}}+\epsilon_t}{g(\alpha_{\tilde{i}_t})-g(\alpha_{\tilde{j}_t})}\\
% \implies (\sqrt{\beta}-g(\alpha_{\tilde{i}_t})+g(\alpha_{\tilde{j}_t}))\kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}}&< \Delta_{\text{max}}\\
% \implies (\sqrt{\beta}-1))\kappa\sqrt{\frac{(\log M)^{1+\delta}}{t}}&< \Delta_{\text{max}}\\
&\implies t>\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{max}}^2},
\end{align*}
which is also a contradiction.
Therefore $T_s>\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{max}}^2}$, if $\mathcal{E}$ occurs, which happens with probability at least $1-\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{M^{2(\log M)^\delta}\Delta_{\text{max}}^2}$.
\end{proof}
\begin{lemma}
Under the W-FTPL policy, $\Prob(T_s>t)\le \exp(-\frac{\Delta_{\text{min}}^2 t}{2\kappa^2})$ for $t>\frac{4\beta \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{min}}^2}$. \label{lem:Ts>T_0}
\end{lemma}
\begin{proof} For $t>\frac{4\beta \kappa^2 (\log M)^{1+\delta}}{\Delta_{\text{min}}^2}$, we have $\frac{\Delta_{\text{min}}}{2}>\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}$. For $T_s>t$, we have
\begin{align*}
(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t &< -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})+\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}},\\
\text{ and}\\
(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))\hat{\mu}_t &> -c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})-\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}.
\end{align*}
Let us consider the following cases \\
\textit{Case 1} ($\alpha_{\tilde{j}_t}<\alpha_{\tilde{i}_t}$, $\Delta_{\tilde{i}_t\tilde{j}_t}>0$):\\
\begin{align*}
\Prob(T_s>t)&\le\Prob\left(\hat{\mu}_t > \frac{-c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})- \kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}}{(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))}\right)\\
&\le \Prob\left( \hat{\mu}_t-\mu>\frac{\Delta_{\tilde{i}_t\tilde{j}_t}-\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}}{g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t})}\right) \\
&\le \Prob\left( \hat{\mu}_t-\mu>\frac{\Delta_{\text{min}}-\Delta_{\text{min}}/2} {g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t})}\right) \\
&\le \Prob\left( \hat{\mu}_t-\mu>\frac{\Delta_{\text{min}}}{2}\right) \\
&\overset{(a)}{\le} \exp\left(\frac{-\Delta_{\text{min}}^2 t}{2\kappa^2}\right),
\end{align*}
where $(a)$ is obtained by using Hoeffding's inequality \cite{hoeffding1994probability}.\\
\textit{Case 2} ($\alpha_{\tilde{j}_t}<\alpha_{\tilde{i}_t}$, $\Delta_{\tilde{i}_t\tilde{j}_t}<0$):
\begin{align*}
\Prob(T_s>t)&\le\Prob\left(\hat{\mu}_t < \frac{-c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})+ \kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}}{(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))}\right)\\
&\le \Prob\left( \hat{\mu}_t-\mu<\frac{\Delta_{\tilde{i}_t\tilde{j}_t}+\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}}{g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t})}\right) \\
&\le \Prob\left( \hat{\mu}_t-\mu<\frac{-\Delta_{\text{min}}+\Delta_{\text{min}}/2} {g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t})}\right) \\
&\le \Prob\left( \hat{\mu}_t-\mu<\frac{-\Delta_{\text{min}}}{2}\right) \\
&\overset{(a)}{\le} \exp\left(\frac{-\Delta_{\text{min}}^2 t}{2\kappa^2}\right).
\end{align*}
where $(a)$ is obtained by using Hoeffding's inequality \cite{hoeffding1994probability}.\\
\textit{Case 3} ($\alpha_{\tilde{j}_t}>\alpha_{\tilde{i}_t}$, $\Delta_{\tilde{i}_t\tilde{j}_t}>0$):\\
\begin{align*}
\Prob(T_s>t)&\le\Prob\left(\hat{\mu}_t < \frac{-c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})- \kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}}{(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))}\right)\\
&\le \Prob\left( \hat{\mu}_t-\mu<\frac{-\Delta_{\tilde{i}_t\tilde{j}_t}+\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}}{g(\alpha_{\tilde{i}_t})-g(\alpha_{\tilde{j}_t})}\right) \\
&\le \Prob\left( \hat{\mu}_t-\mu<\frac{-\Delta_{\text{min}}+\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}}{g(\alpha_{\tilde{i}_t})-g(\alpha_{\tilde{j}_t})}\right) \\
&\le \Prob\left( \hat{\mu}_t-\mu<\frac{-\Delta_{\text{min}}/2} {g(\alpha_{\tilde{i}_t})-g(\alpha_{\tilde{j}_t})}\right) \\
&\le \Prob\left( \hat{\mu}_t-\mu<-\Delta_{\text{min}}/2\right) \\
&\overset{(a)}{\le} \exp\left(\frac{-\Delta_{\text{min}}^2 t}{2\kappa^2}\right).
\end{align*}
where $(a)$ is obtained by using Hoeffding's inequality \cite{hoeffding1994probability}.\\
\textit{Case 4} ($\alpha_{\tilde{j}_t}>\alpha_{\tilde{i}_t}$, $\Delta_{\tilde{i}_t\tilde{j}_t}<0$):\\
\begin{align*}
\Prob(T_s>t)&\le\Prob\left(\hat{\mu}_t > \frac{-c(\alpha_{\tilde{j}_t}-\alpha_{\tilde{i}_t})+ \kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}}{(g(\alpha_{\tilde{j}_t})-g(\alpha_{\tilde{i}_t}))}\right)\\
&\le \Prob\left( \hat{\mu}_t-\mu>\frac{-\Delta_{\tilde{i}_t\tilde{j}_t}-\kappa\sqrt{\frac{\beta (\log M)^{1+\delta}}{t}}}{g(\alpha_{\tilde{i}_t})-g(\alpha_{\tilde{j}_t})}\right) \\
&\le \Prob\left( \hat{\mu}_t-\mu>\frac{\Delta_{\text{min}}-\Delta_{\text{min}}/2} {g(\alpha_{\tilde{i}_t})-g(\alpha_{\tilde{j}_t})}\right) \\
&\le \Prob\left( \hat{\mu}_t-\mu>\frac{\Delta_{\text{min}}}{2}\right) \\
&\overset{(a)}{\le} \exp\left(\frac{-\Delta_{\text{min}}^2 t}{2\kappa^2}\right),
\end{align*}
where $(a)$ is obtained by using Hoeffding's inequality \cite{hoeffding1994probability}.
The result follows by combining these cases.
\end{proof}
\begin{lemma}
Under the W-FTPL policy
$$\E[T_s]\le 1+\frac{4\beta \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{min}}^2} + \frac{1}{M^{2\beta}}\frac{2\kappa^2}{\Delta_{\text{min}}^2}.$$
\label{lem:E[T_s]}
\end{lemma}
\begin{proof}Let $T_1=\lceil \frac{4\beta \kappa^2 (\log M)^{1+\delta}}{\Delta_{\text{min}}^2}\rceil$. It follows that
\begin{align*}
\E[T_s]&=\sum_{t=1}^{T}\Prob(T_s>t)\\
&=\sum_{t=1}^{T_1}\Prob(T_s>t)+\sum_{t=T_1+1}^{T}\Prob(T_s>t)\\
&\overset{(a)}{\le} T_1+\sum_{t=T_1}^{T}\exp\left(-\frac{\Delta_{\text{min}}^2 t}{2\kappa^2}\right)\\
&\le 1+\frac{4\beta \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{min}}^2} + \frac{1}{M^{2\beta(\log M)^\delta}}\frac{2\kappa^2}{\Delta_{\text{min}}^2},
\end{align*}
where $(a)$ is obtained by using Lemma \ref{lem:Ts>T_0}.
\end{proof}
%\begin{theorem}
% The regret under W-FTPL with learning rate $\eta_{t}=\alpha\sqrt{t-1}$, $\alpha>0$, $\beta>1$, $\delta\ge 0$ is given by
% \begin{align*}
% \mathcal{R}^{\text{W-FTPL}}_S(T)\le& \color{red}1+\sum_{i\ne i^\star}\frac{\beta\kappa^2}{\Delta_i}+\frac{\beta\kappa^2 (1+4(\log M)^{1+\delta})}{\Delta_{\text{min}}^2}+\frac{\beta^2\kappa^4} {\Delta_{\text{max}}^2\Delta_{\text{min}}^2}.
% \end{align*}
% \label{thm:WFTPL_reg_ub_stch}
%\end{theorem}
\begin{proof}[Proof of Theorem \ref{thm:reg_stch}(c)]
Under W-FTPL,
\begin{align*}
\E_r[\mathcal{C}^\text{W-FTPL}(T,r)]=& \E_r[T_s]+ \E_r\left[ \sum_{t=T_s+1}^{T}(c\rho_t+ g(\rho_t)r_t)\right] \\
&+M\E_r\left[ \sum_{t=T_s+1}^T \Prob(\rho_t>\rho_{t-1})\right] .
\end{align*}
By the using definition of regret we get,
\begin{align*}
\mathcal{R}^{\text{W-FTPL}}_S(T)\le& \E[T_s]+\sum_{t=1}^{T}(c\rho_t+\mu g(\rho_t))-\mu_{i^\star}\\
&+M\E_r\left[ \sum_{t=T_s+1}^T \Prob(\rho_t>\rho_{t-1})\right] .
\end{align*}
By Lemma \ref{lem:FTPL_reg_stch}, we get
\begin{align*}
\mathcal{R}^{\text{W-FTPL}}_S(T)\le& \E[T_s]+ \frac{16\alpha^2+4\kappa^2}{\Delta_{\text{min}}}\\
&+\left( \sqrt{2\log K}+\frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}} \right) \left( \alpha + \frac{4\kappa^2}{\alpha}\right)\\
&+M\E_r\left[ \sum_{t=T_s+1}^T \Prob(\rho_t>\rho_{t-1})\right] .
\end{align*}
Let $\E_r[\mathcal{C}_f]=M\E_r\left[ \sum_{t=T_s+1}^T \Prob(\rho_t>\rho_{t-1})\right] $, then
\begin{align*}
\E_r[\mathcal{C}_f]=&\E_r[\mathcal{C}_f|T_s\le T_0]\Prob(T_s\le T_0)\\
&+\E_r[\mathcal{C}_f|T_s> T_0]\Prob(T_s>T_0)\\
\le &\E[\mathcal{C}_f|T_s=1]\Prob(T_s\le T_0)+\E[\mathcal{C}_f|T_s= \lceil T_0 \rceil]\\
\overset{(a)}{\le} &M \Prob(T_s\le T_0)\E[\mathcal{C}_F^{\text{FTPL}}]\\
&+M \sum_{t=\lceil T_0\rceil}^T \left( \exp\left( -\frac{\Delta_{\text{min}}^2t}{16\alpha^2}\right)+e^{-\Delta_{\text{min}}^2t/2\kappa^2}\right)\\
\le & M \Prob(T_s\le T_0) \frac{16\alpha^2+3\kappa^2}{\Delta_{\text{min}}^2}\\
&+ \frac{16\alpha^2}{\Delta_{\text{min}}^2}\exp\left( -\frac{\Delta_{\text{min}}^2T_0}{16\alpha^2}\right)\\
&+\frac{2\kappa^2}{\Delta_{\text{min}}^2}\exp\left(-\frac{\Delta_{\text{min}}^2T_0}{2\kappa^2}\right).
\end{align*}
Here, $(a)$ is obtained by using inequality \eqref{eq:swFTPL_inq}.
By considering $T_0=\frac{(\sqrt{\beta}-1)^2 \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{max}}^2}$ and using Lemma \ref{lem:Ts>t0}, \ref{lem:E[T_s]} we get,
\begin{align*}
\mathcal{R}^{\text{W-FTPL}}_S&(T) \\
\le& 1+\frac{4\beta \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{min}}^2} + \frac{1}{M^{2\beta(\log M)^\delta}}\frac{2\kappa^2 }{\Delta_{\text{min}}^2}\\
&+\frac{(16\alpha^2+4\kappa^2)}{\Delta_{\text{min}}}\\
&+\left( \sqrt{2\log K}+\frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}} \right) \left( \alpha + \frac{4\kappa^2}{\alpha}\right)\\
&+M\frac{(\sqrt{\beta}-1)^2\kappa^2 (\log M)^{1+\delta}}{\Delta_{\text{max}}^2 M^{2(\log M)^\delta}}\frac{16\alpha^2+3\kappa^2}{\Delta_{\text{min}}^2}\\
&+\left( \frac{16\alpha^2 M}{M^{\frac{(\sqrt{\beta}-1)^2\kappa^2\Delta_{\text{min}}^2(\log M)^\delta}{16\alpha^2\Delta_{\text{max}}^2}}}+ \frac{2M\kappa^2}{M^{\frac{(\sqrt{\beta}-1)^2\Delta_{\text{min}}^2(\log M)^\delta}{2\Delta_{\text{max}}^2}}} \right)\\ &\times \frac{1}{\Delta_{\text{min}}^2}.
% \mathcal{R}^{\text{W-FTPL}}_S(T)\le& \color{red} 1+\sum_{i\ne i^\star}\frac{\beta\kappa^2}{\Delta_i}+\frac{\beta\kappa^2 (1+4(\log M)^{1+\delta})}{\Delta_{\text{min}}^2}+\frac{\beta^2 \kappa^4}{\Delta_{\text{max}}^2\Delta_{\text{min}}^2}.
\end{align*}
For large values of $M$ we have
\begin{align*}
\mathcal{R}^{\text{W-FTPL}}_S&(T) \\
\le& 1+\frac{4\beta \kappa^2(\log M)^{1+\delta}}{\Delta_{\text{min}}^2} +\frac{2\kappa^2 }{\Delta_{\text{min}}^2}\\
&+\frac{(16\alpha^2+4\kappa^2)}{\Delta_{\text{min}}}\\
&+\left( \sqrt{2\log K}+\frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}} \right) \left( \alpha + \frac{4\kappa^2}{\alpha}\right)\\
&+\frac{(\sqrt{\beta}-1)^2\kappa^2 (\log M)^{1+\delta}}{\Delta_{\text{max}}^2 }\frac{16\alpha^2+3\kappa^2}{\Delta_{\text{min}}^2}\\
&+\frac{16\alpha^2 + 2\kappa^2}{\Delta_{\text{min}}^2}\\
\le&1+\beta \kappa^2(\log M)^{1+\delta}\left(\frac{4}{\Delta_{\text{min}^2}}+\frac{16\alpha^2+3\kappa^2}{\Delta_{\text{min}^2}\Delta_{\text{max}}^2}\right)\\
&+(16\alpha^2+4\kappa^2)\left(\frac{1}{\Delta_{\text{min}}}+\frac{1}{\Delta_{\text{min}^2}}\right)\\
&+\left( \sqrt{2\log K}+\frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}} \right) \left( \alpha + \frac{4\kappa^2}{\alpha}\right).
% &+\frac{\beta\kappa^2 (\log M)^{1+\delta}}{\Delta_{\text{max}}^2 }\frac{16\alpha^2+3\kappa^2}{\Delta_{\text{min}}^2}\\
% \le&1+\frac{12\beta h_1\kappa^2(\log M)^{1+\delta}}{\Delta_{\text{min}}^2}\\
% &+(16\alpha^2+4\kappa^2)\left(\frac{1}{\Delta_{\text{min}}}+\frac{1}{\Delta_{\text{min}^2}}\right)\\
% &+\left( \sqrt{2\log K}+\frac{2\sqrt{2h_1}\log K}{\Delta_{\text{min}}} \right) \left( \alpha + \frac{4\kappa^2}{\alpha}\right)\\
% &+\frac{\beta\kappa^2 (\log M)^{1+\delta}}{\Delta_{\text{max}}^2 }\frac{16\alpha^2+3\kappa^2}{\Delta_{\text{min}}^2}.
\end{align*}
% Note that $M>1$, $\beta=\max\{(1+{4\alpha\Delta_{\text{max}}/\Delta_{\text{min}}\kappa})^2,(1+{\sqrt{2}}\Delta_{\text{max}}/\Delta_{\text{min}})^2 \}$, which implies ${(\sqrt{\beta}-1)^2}\kappa^2\Delta_{\text{min}}^2/16\alpha^2\Delta_{\text{max}}^2\ge1$ and ${(\sqrt{\beta}-1)^2}\Delta_{\text{min}}^2/2\Delta_{\text{max}}^2\ge1$, $M>\log M$, therefore the result follows.
\end{proof}
\subsection{Proof of Theorem \ref{thm:comp_adv}(a)}
\begin{lemma}
The cost of the offline optimal policy is lower bounded as follows
$$\mathcal{C}^\text{OPT-OFF}(T,r)\ge \frac{R_T}{\kappa^2}(\min_i{c\alpha_{i}+g(\alpha_{i})\kappa}).$$
\label{lem:opt_offline_cost_LB}
\end{lemma}
\begin{proof}
The cost incurred by the optimal offline policy with $M>0$ is lower bounded by the cost incurred by the optimal offline policy when $M=0$. For $M>0$, there is an additional fetching cost. Therefore,
\begin{align*}
\mathcal{C}^\text{OPT-OFF}(T,r)&\ge \sum_{t=1}^T \min_i (c\alpha_{i}+g(\alpha_{i})r_t)\\
&\ge \sum_{t=1}^T \min_i (c\alpha_{i}/\kappa+g(\alpha_{i})r_t)\\
&\ge \sum_{t=1}^T \min_i (c\alpha_{i}/\kappa+g(\alpha_{i}))\mathds{1}_{\{r_t\ge 1\}}\\
&\ge \sum_{t=1}^T \min_i (c\alpha_{i}/\kappa+g(\alpha_{i})) r_t/\kappa\\
&\ge \frac{R_T}{\kappa^2}(\min_i{c\alpha_{i}+g(\alpha_{i})\kappa}).
\end{align*}
\end{proof}
\begin{lemma}
The expected cost of FTPL policy for any request sequence $r$ for $\eta_{t}=\alpha\sqrt{t-1}$, $\alpha>0$, can be upper bounded as follows
\begin{align*}
\E[\mathcal{C}^{\text{FTPL}}(T,r)]\le \left((3+2M/c)\max_{\alpha_{i}\ne 0}\frac{1-g(\alpha_{i})}{\alpha_{i}}\right) R_T\\+(c+M)\sum_{i=2}^K\frac{16\alpha^2}{c^2\alpha_{i}^2}.
\end{align*}
\label{lem:FTPL_comp_adv}
\end{lemma}
\begin{proof}
FTPL does not host any fraction the service for time $t+1$ if $c\alpha_{i}t+g(\alpha_{i})R_t+\eta_{t+1}\gamma_i> R_t+\eta_{t+1} \gamma_1$, for all $i\ne 1$. The probability of hosting any non zero fraction of service ($p_{h,t}$) in time slot $t+1$ is given as
\begin{align*}
p_{h,t}&\le\sum_{i=2}^K\Prob(c\alpha_{i}t+g(\alpha_{i})R_t+\eta_{t+1}\gamma_i< R_t+\eta_{t+1} \gamma_1)\\
&=\sum_{i=2}^K\Prob\left( \frac{\gamma_1-\gamma_i}{\sqrt{2}}> \frac{c\alpha_{i}t+g(\alpha_{i})R_t-R_t}{\sqrt{2}\eta_{t+1}}\right)\\
&=\sum_{i=2}^K\Prob\left( \frac{\gamma_1-\gamma_i}{\sqrt{2}}> \frac{c\alpha_{i}t-(1-g(\alpha_{i}))R_t}{\sqrt{2}\eta_{t+1}}\right).
\end{align*}
Let $T'=\frac{2R_T}{c}\max_{\alpha_{i}\ne 0}\frac{1-g(\alpha_{i})}{\alpha_{i}}$, if $t\ge T'$ then $R_t\le R_T\le \frac{ct}{2}\min_{\alpha_{i}\ne0}\frac{\alpha_{i}}{1-g(\alpha_{i})}$. Therefore for $t\ge T'$, $R_t\le \frac{c\alpha_{i}t}{2(1-g(\alpha_{i}))}$ for all $\alpha_{i}\ne 0$ and we get
\begin{align*}
p_{h,t}&\le \sum_{i=2}^K\Prob\left( \frac{\gamma_1-\gamma_i}{\sqrt{2}}> \frac{c\alpha_{i}t-c\alpha_{i}t/2}{\sqrt{2}\eta_{t+1}}\right)\\
&\le \sum_{i=2}^K\exp\left(-\frac{c^2\alpha_{i}^2t^2}{16\eta_{t+1}^2}\right)\\
&=\sum_{i=2}^K \exp\left(-\frac{c^2\alpha_{i}^2 t}{16\alpha^2}\right). \numberthis \label{eq:prob_ftpl_comp}
\end{align*}
Therefore, the expected cost under FTPL can be bounded as
\begin{align*}
\E[\mathcal{C}^{\text{FTPL}}&(T,r)] \\
\le& \sum_{t=1}^{T}(c+M)p_{h,t}+r_t\\
=& R_T+\sum_{t=1}^{T}(c+M)p_{h,t}\\
\le& R_T+ (c+M) T'+(c+M)\sum_{t=\lceil T'\rceil}^T p_{h,t} \\
\overset{(a)}{\le}& R_T+ (c+M) T'\\
&+(c+M)\sum_{t=\lceil T'\rceil}^T \sum_{i=2}^K \exp\left(-\frac{c^2\alpha_{i}^2t}{16\alpha^2}\right) \\
\le& \left((3+2M/c)\max_{\alpha_{i}\ne 0}\frac{1-g(\alpha_{i})}{\alpha_{i}}\right) R_T\\ &+(c+M)\sum_{i=2}^K\frac{16\alpha^2}{c^2\alpha_{i}^2}.
\end{align*}
Here $(a)$ follows from \eqref{eq:prob_ftpl_comp}. If $T<T'$, then the bound will only have the first term and even in that case the above bound holds. Therefore the above bound holds for any $T$.
\end{proof}
\begin{proof}[Proof of Theorem \ref{thm:comp_adv}(a)]
By using Lemma \ref{lem:opt_offline_cost_LB}, we have
\begin{align*}
\sigma_{A}^{\text{FTPL}}\le& \sup_{r} \frac{\E[\mathcal{C}^{\text{FTPL}}(T,r)]}{(\min_i{c\alpha_{i}+g(\alpha_{i})\kappa})R_T/\kappa^2}\\
\overset{(a)}{\le}& \frac{\kappa^2(3+2M/c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\max_{\alpha_{i}\ne 0}\frac{1-g(\alpha_{i})}{\alpha_{i}} \\
&+\sup_r \frac{\kappa^2(M+c)}{R_T\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\sum_{i=2}^K\frac{16\alpha^2}{c^2\alpha_{i}^2}\\
\le &\frac{\kappa^2(3+2M/c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\max_{\alpha_{i}\ne 0}\frac{1-g(\alpha_{i})}{\alpha_{i}} \\
&+ \frac{\kappa^2(M+c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\sum_{i=2}^K\frac{16\alpha^2}{c^2\alpha_{i}^2}.
\end{align*}
Here $(a)$ is obtained by using Lemma \ref{lem:FTPL_comp_adv}.
\end{proof}
\begin{comment}
\color{blue} By Theorem \ref{thm:reg_adv}(c), we know that for $\eta_{t}=\alpha\sqrt{t}$ we get $\OO(\sqrt{T})$ regret. If we replace $\eta_{t}$ as $\eta_{t}=\alpha\sqrt{t-1}$ then also we will get $\OO(\sqrt{T})$ regret. We can see this with help of Lemma \ref{lem:FTPL_reg_adv} and \eqref{eq:FTPL_sw_eta}. Therefore
\begin{align*}
\sigma^{\text{FTPL}}_{A}&(T) \\
&=\color{blue} \sup_{r \in\mathcal{R}} \frac{\mathcal{R}^{\text{FTPL}}_A(T,r)+\min_i\{c\alpha_iT+g(\alpha_{i})R_T+M\alpha_{i}\}}{\mathcal{C}^{\text{OPT-OFF}}(T,r)}\\
&\le \frac{O(\sqrt{T})+\min_i\{c\alpha_iT+g(\alpha_{i})R_T+M\alpha_{i}\}} {(\min_i{c\alpha_{i}+g(\alpha_{i})\kappa})R_T/\kappa^2 }.
\end{align*}
If $R_T>\min_i T(c\alpha_{i}+g(\alpha_{i})\kappa)$, \color{blue} then the static optimal policy is to host \color{blue} some non zero fraction of the service, therefore
\begin{align*}
\sigma^{\text{FTPL}}_{A}(T) \le \frac{\kappa^2}{T}\frac{O(\sqrt{T})+\min_i\{c\alpha_iT+g(\alpha_{i})\kappa T+M\alpha_{i}\}}{(\min_i\{c\alpha_i+g(\alpha_{i})\kappa\})^2}.
\end{align*}
% Therefore when optimal policy is to host the service then we get $\sigma^{\text{FTPL}}_A \le \frac{\kappa}{\min_i\{c\alpha_iT+g(\alpha_{i})\kappa\}}$ as $T\to \infty$ .
The other case is when $R_T\le T \min_{\alpha_{i}\ne0}(c\alpha_i+g(\alpha_{i})\kappa)$ and
by using Lemma \ref{lem:FTPL_comp_adv} we get
\begin{align*}
\sigma^{\text{FTPL}}_A(T) \le& \frac{\kappa^2(3+2M/c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\max_{\alpha_{i}\ne 0}\frac{1-g(\alpha_{i})}{\alpha_{i}}\\ &+\frac{\kappa^2(M+c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\sum_{i=2}^K\frac{16\alpha^2}{c^2\alpha_{i}^2}.
\end{align*}
Therefore the competitive ratio combining both the cases is bounded as
\begin{align*}
\sigma^{\text{FTPL}}_A(T)\le& \frac{\kappa^2(3+2M/c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\max_{\alpha_{i}\ne 0}\frac{1-g(\alpha_{i})}{\alpha_{i}} \\&+\frac{\kappa^2(M+c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\sum_{i=2}^K\frac{16\alpha^2}{c^2\alpha_{i}^2}+\OO(1/{\sqrt{T}}).
\end{align*}
\end{comment}
\subsection{Proof of Theorem \ref{thm:comp_adv}(b)}
\begin{proof}[Proof of Theorem \ref{thm:comp_adv}(b)]
By using Lemma \ref{lem:opt_offline_cost_LB}, we have
\begin{align*}
\sigma^{\text{W-FTPL}}_A \le& \sup_{r} \frac{R_{T_s}+\E[\mathcal{C}_{A}^{\text{FTPL}}(T,r)]}{R_T(\min_i\{c\alpha_i+g(\alpha_{i})\kappa\})/\kappa^2}\\
\overset{(a)}{\le} & \frac{\kappa^2(3+2M/c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\max_{\alpha_{i}\ne 0}\frac{1-g(\alpha_{i})}{\alpha_{i}}\\
&+\frac{\kappa^2(M+c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\sum_{i=2}^K\frac{16\alpha^2}{c^2\alpha_{i}^2}\\
&+\frac{\kappa^2}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}.
\end{align*}
Here $(a)$ is obtained by using Theorem \ref{thm:comp_adv}(a) and the fact that $R_{T_s}\le R_T$.
\end{proof}
\begin{comment}
By using Theorem \ref{thm:reg_adv}, and $\eta_{t}=\alpha\sqrt{t}$ we get
\begin{align*}
\sigma^{\text{W-FTPL}}_{A}(T) &\le \frac{O(\sqrt{T})+\min_i\{c\alpha_iT+g(\alpha_{i})R_T+M\alpha_{i}\}} {(\min_i{c\alpha_{i}+g(\alpha_{i})\kappa})R_T/\kappa^2 }.
\end{align*}
If $R_T>\min_{\alpha_i\ne 0} T(c\alpha_{i}+g(\alpha_{i})\kappa)$ static optimal policy is to host \color{blue} some non zero fraction of the service, therefore
\begin{align*}
\sigma^{\text{W-FTPL}}_{A}&(T)\\
&\le \frac{\kappa^2}{T}\frac{O(\sqrt{T})+\min_i\{c\alpha_iT+g(\alpha_{i})\kappa T+M\alpha_{i}\}}{(\min_i\{c\alpha_i+g(\alpha_{i})\kappa\})^2}.
\end{align*}
% Therefore when optimal policy is to host the service then we get $\sigma^{\text{W-FTPL}}_A \le \frac{\kappa}{\min_i\{c\alpha_iT+g(\alpha_{i})\kappa\}}$ as $T\to \infty$ .
The other case is when $R_T\le T \min_{\alpha_{i}\ne0}(c\alpha_i+g(\alpha_{i})\kappa)$
\begin{align*}
\sigma^{\text{W-FTPL}}_A \le& \frac{R_{T_s}+\E[\mathcal{C}_{A}^{\text{FTPL}}(T)]}{R_T(\min_i\{c\alpha_i+g(\alpha_{i})\kappa\})/\kappa^2}.
\end{align*}
Therefor the competitive ratio combining all the cases is bounded as
\begin{align*}
\sigma^{\text{W-FTPL}}_A\le& \frac{\kappa^2(3+2M/c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\max_{\alpha_{i}\ne 0}\frac{1-g(\alpha_{i})}{\alpha_{i}}\\
&+\frac{\kappa^2(M+c)}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}\sum_{i=2}^K\frac{16\alpha^2}{c^2\alpha_{i}^2}\\
&+\frac{\kappa^2}{\min_i(c\alpha_{i}+g(\alpha_{i}) \kappa)}+\OO(1/\sqrt{T}).
\end{align*}
\end{comment}
%\begin{theorem}
% The competitive ratio of FTPL policy with $\eta_{t}=\alpha\sqrt{t}$, $\alpha>0$ as $T\to \infty$ is lower bounded as follows
% $$\sigma^{FTPL}_d \ge bM,$$
% where $b=\Prob \left(\frac{2c-1}{2\alpha}\le \frac{\gamma_1-\gamma_2}{\sqrt{2}} \le \frac{c}{\alpha\sqrt{2}} \right).$
% \label{thm:FTPL_comp_adv_lb}
%\end{theorem}
%\begin{proof}
% We are interested in only switching cost to analyze the impact of $M$ on FTPL policy. Since we can start either with storing the service or not storing the service and switching cost only occurs if $\rho_{t-1}=0$, $\rho_{t}=1$. We will calculate the probability of $\rho_{t-1}=0$, $\rho_{t}=1$.
%
% Let us consider a request sequence $r_t=0$ for all $t\ne2$ and $r_2=1$. By using \eqref{eq:FTPL_sw_prob} the probability of switching at time $t=2$ for $\eta_{t}=\alpha \sqrt{t}$ is $\Prob \left(\frac{2c-1}{2\alpha}\le \frac{\gamma_1-\gamma_2}{\sqrt{2}} \le \frac{c}{\alpha\sqrt{2}} \right) $. It is a finite quantity(say $b$). Therefore expected switching cost of FTPL policy is lower bounded by $bM$ which is also a lower bound on expected cost of FTPL policy for this request sequence. Since competitive ratio is supremum over the request sequences we get the result stated.
%\end{proof}
%\subsubsection{Stochastic arrivals}
%For any policy $\mathcal{P}$, the competitive ratio with respect to static optimal policy is
%\begin{align*}
%\sigma^{\mathcal{P}}_{S}(T)&\le \frac{\mathcal{R}^{\mathcal{P}}_S(T)+ \E[\mathcal{C}^{OPT}(T,r)]}{\E[\mathcal{C}^{OPT}(T,r)]} \\
%&=1+\frac{\mathcal{R}^{\mathcal{P}}_{S}(T)}{T\mu_{i^\star}}
%. \numberthis \label{eq:comp_gen_stat}
%\end{align*}
\subsection{Proof of Theorem \ref{thm:comp_stch}(a)}
\begin{proof}[Proof of Theorem \ref{thm:comp_stch}(a)]
\begin{align*}
\sigma^{\text{RR}}_{S}(T)&=\frac{\E[\mathcal{C}^{\text{RR}}(T,r)]}{\min_i\{c\alpha_iT+g(\alpha_i)\mu T+M\alpha_i\}}\\
&= \frac{\mathcal{R}^{\text{RR}}_S(T)+\min_i\{c\alpha_iT+g(\alpha_i)\mu T+M\alpha_i\}}{\min_i\{c\alpha_iT+g(\alpha_i)\mu T+M\alpha_i\}}\\
& \le 1+\frac{\mathcal{R^{\text{RR}}}_S(T)}{\mu_{i^\star}T}.
\end{align*}
By using Theorem \ref{thm:reg_stch} we have,
\begin{align*}
\sigma^{\text{RR}}_{S}(T)\ge 1+ \frac{M}{\mu_{i^\star}}\left(\frac{1}{\frac{M}{\kappa-c\alpha_{i'}-g(\alpha_{i'})\kappa}+ \frac{M}{c} +2}-\frac{1}{T}\right) pd.
\end{align*}
For large $T$ we get,
\begin{align*}
\sigma^{\text{RR}}_{S}(T)> 1.
\end{align*}
\end{proof}
\subsection{Proof of Theorem \ref{thm:comp_stch}(b)}
\begin{proof}[Proof of Theorem \ref{thm:comp_stch}(b)]
By using Theorem \ref{thm:reg_stch} we have
\begin{align*}
\sigma^{\text{FTPL}}_{S}(T) &\le 1+\frac{1}{\mu_{i^\star}T}(16\alpha^2+1)\left( \frac{M}{\Delta_{\text{min}}^2}+\sum_{i\ne i^\star}\frac{1}{\Delta_i}\right)\\
&=1+\OO(1/T).
\end{align*}
% By considering $T\to \infty$ we get,
% $$\sigma^{\text{FTPL}}_S(T) \le 1 .$$
\end{proof}
\subsection{Proof of Theorem \ref{thm:comp_stch}(c)}
\begin{proof}[Proof of Theorem \ref{thm:comp_stch}(c)]
By using Theorem \ref{thm:reg_stch} we have $\mathcal{R}^{\text{W-FTPL}}_S(T)$ as a constant and
\begin{align*}
\sigma^{\text{W-FTPL}}_{S}(T) \le 1+\frac{\mathcal{R}^{\text{W-FTPL}}_S(T)}{T\mu_{i^\star}}.
\end{align*}
Therefore we get,
$$\sigma^{\text{W-FTPL}}_S(T) \le 1+\OO(1/T). $$
\end{proof}
\begin{proof}[Proof of Lemma \ref{lem:adv_stch_reg}] For i.i.d. stochastic arrivals, we have
\begin{align*}
\mathcal{R}^{\mathcal{P}}_S&(T)\\
&=\E_{\mathcal{P},r}[\mathcal{C}^\mathcal{P}(T,r)] -\min_i\{c\alpha_iT+g(\alpha_i)\mu T+M\alpha_i\}\\
&=\E_{\mathcal{P},r}[\mathcal{C}^\mathcal{P}(T,r)] -\min_i\{\E_r[c\alpha_iT+g(\alpha_i)R_T+M\alpha_i]\}\\
&\overset{(a)}{\le} \E_{\mathcal{P},r}[\mathcal{C}^\mathcal{P}(T,r)] -\E_r[\min_i\{c\alpha_iT+g(\alpha_i)R_T +M\alpha_i\}]\\
&=\E_r[\mathcal{R}_A(T,r)] \le \sup_{r} \left(\mathcal{R}^{\mathcal{P}}_A(T,r) \right) = \mathcal{R}_A^\mathcal{P}(T),
\end{align*}
where $(a)$ is obtained by Jensen's inequality.
%By Jensen's inequality $\mathcal{R}_S^\mathcal{P}(T) \le \E_r[\mathcal{R}_A^\mathcal{P}(T,r)]$. Since $\E_{r\in \mathcal{R}} \left(\mathcal{R}^{\mathcal{P}}_A(T,r) \right) \le \sup_{r\in \mathcal{R}} \left(\mathcal{R}^{\mathcal{P}}_A(T,r) \right)$ we have $\mathcal{R}_S^\mathcal{P}(T)\le \mathcal{R}_A^\mathcal{P}(T)$.
\end{proof}
%In this section we prove the lemmas used in the proof Theorem \ref{thm:reg_adv} (a).
\begin{lemma}
\label{lem:min_E}
If $X$ is a random variable and $\E[f_1(X)]=\E[f_2(X)]=m$, then
$$m-\E[\min\{f_1(X),f_2(X)\}]=\frac{1}{2}\E[|f_1(X)-f_2(X)|].$$
\end{lemma}
\begin{proof}
\begin{align*}
\min\{X,Y\}=\frac{1}{2}(X+Y)-\frac{1}{2}(|X-Y|)\\
\implies m-\min\{f_1(X),f_2(X)\}=m-\frac{1}{2}(f_1(X)+f_2(X))\\
+\frac{1}{2}(|f_1(X)-f_2(X)|)\\
\implies m-\E[\min\{f_1(X),f_2(X)\}]=\frac{1}{2}\E[|f_1(X)-f_2(X)|].
\end{align*}
\end{proof}
\begin{lemma}
\label{lem:binomial_t}
If $X\sim \text{Bin}(T,p)$ then
\begin{align*}
\frac{1}{2}\E[|X-Tp|]\ge &\frac{\sqrt{Tp(1-p)}}{e\sqrt{2\pi}} \left[1+\frac{1}{12T+1}-\frac{1}{12\lfloor Tp\rfloor} \right. \\
&\left.- \frac{1}{12(T-\lfloor Tp\rfloor-1)} \right].
\end{align*}
\end{lemma}
\begin{proof}
By using equation (1) from \cite{berend2013sharp} we get
\begin{align*}
\E[|X&-Tp|]\\
=&2(1-p)^{T-\lfloor Tp\rfloor} p^{\lfloor Tp\rfloor+1}(\lfloor Tp\rfloor+1) {T\choose \lfloor Tp+1\rfloor}\\
=&2(1-p)^{T-\lfloor Tp\rfloor} p^{\lfloor Tp\rfloor+1} \frac{T!}{\lfloor Tp\rfloor! (T-\lfloor Tp\rfloor-1)! }\\
\overset{(a)}{\ge}&2(1-p)^{T-\lfloor Tp\rfloor} p^{\lfloor Tp\rfloor+1} \frac{1}{e\sqrt{2\pi}}\frac{T^{T+\frac{1}{2}}e^{\frac{1}{12T+1}}} {\lfloor Tp\rfloor^{\lfloor Tp\rfloor+\frac{1}{2}}e^{\frac{1}{12\lfloor Tp\rfloor}}}\\
&\times \frac{1}{(T-\lfloor Tp\rfloor-1)^{T-\lfloor Tp\rfloor-1+\frac{1}{2}}e^{\frac{1}{12(T-\lfloor Tp\rfloor-1)}}}\\
\overset{(b)}{\ge}& \frac{2}{e\sqrt{2\pi}}(1-p)^{T-\lfloor Tp\rfloor} p^{\lfloor Tp\rfloor+1} \frac{T^{T+\frac{1}{2}}e^{\frac{1}{12T+1}}} {(Tp)^{\lfloor Tp\rfloor+\frac{1}{2}}e^{\frac{1}{12\lfloor Tp\rfloor}}}\\
&\times \frac{1}{(T-Tp)^{T-\lfloor Tp\rfloor-\frac{1}{2}}e^{\frac{1}{12(T-\lfloor Tp\rfloor-1)}}}\\
= & \frac{2\sqrt{Tp(1-p)}}{e\sqrt{2\pi}} e^{\frac{1}{12T+1}- \frac{1}{12\lfloor Tp\rfloor}- \frac{1}{12(T-\lfloor Tp\rfloor-1)}}\\
\overset{(c)}{\ge} &\frac{2\sqrt{Tp(1-p)}}{e\sqrt{2\pi}} \left[1+\frac{1}{12T+1}-\frac{1}{12\lfloor Tp\rfloor} \right. \\
&\left.- \frac{1}{12(T-\lfloor Tp\rfloor-1)}\right],
\end{align*}
where $(a)$ follows by using the result in \cite{robbins1955remark} which is $\sqrt{2\pi}n^{n+\frac{1}{2}}e^{-n}e^{\frac{1}{12n+1}} < n! <\sqrt{2\pi}n^{n+\frac{1}{2}}e^{-n}e^{\frac{1}{12n}}$. $(b)$ follows by using the fact that $x-1\le \lfloor x \rfloor\le x$. $(c)$ follows from the fact $e^x\ge 1+x$.
\end{proof}
\begin{thebibliography}{10}
\expandafter\ifx\csname url\endcsname\relax
\def\url#1{\texttt{#1}}\fi
\expandafter\ifx\csname urlprefix\endcsname\relax\def\urlprefix{URL }\fi
\expandafter\ifx\csname href\endcsname\relax
\def\href#1#2{#2} \def\path#1{#1}\fi
\bibitem{prakash2020partial}
R.~S. Prakash, N.~Karamchandani, V.~Kavitha, S.~Moharir, Partial service
caching at the edge, in: 2020 18th International Symposium on Modeling and
Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOPT), IEEE, 2020,
pp. 1--8.
\bibitem{LN_partial_hosting}
V.~S. C.~L. Narayana, M.~Agarwala, R.~S. Prakash, N.~Karamchandani, S.~Moharir,
\href{https://arxiv.org/abs/2103.00555}{Online partial service hosting at the
edge}, CoRR abs/2103.00555 (2021).
\newblock \href {http://arxiv.org/abs/2103.00555} {\path{arXiv:2103.00555}}.
\newline\urlprefix\url{https://arxiv.org/abs/2103.00555}
\bibitem{narayana2021renting}
V.~C.~L. Narayana, S.~Moharir, N.~Karamchandani, On renting edge resources for
service hosting, ACM Transactions on Modeling and Performance Evaluation of
Computing Systems 6~(2) (2021) 1--30.
\bibitem{zhao2018red}
T.~Zhao, I.-H. Hou, S.~Wang, K.~Chan, Red/led: An asymptotically optimal and
scalable online algorithm for service caching at the edge, IEEE Journal on
Selected Areas in Communications 36~(8) (2018) 1857--1870.
\bibitem{lai1985asymptotically}
T.~L. Lai, H.~Robbins, Asymptotically efficient adaptive allocation rules,
Advances in applied mathematics 6~(1) (1985) 4--22.
\bibitem{paschos2019learning}
G.~S. Paschos, A.~Destounis, L.~Vigneri, G.~Iosifidis, Learning to cache with
no regrets, in: IEEE INFOCOM 2019-IEEE Conference on Computer Communications,
IEEE, 2019, pp. 235--243.
\bibitem{bhattacharjee2020fundamental}
R.~Bhattacharjee, S.~Banerjee, A.~Sinha, Fundamental limits on the regret of
online network-caching, Proceedings of the ACM on Measurement and Analysis of
Computing Systems 4~(2) (2020) 1--31.
\bibitem{Puliafito:2019}
C.~Puliafito, E.~Mingozzi, F.~Longo, A.~Puliafito, O.~Rana,
\href{http://doi.acm.org/10.1145/3301443}{Fog computing for the internet of
things: A survey}, ACM Trans. Internet Technol. 19~(2) (2019) 18:1--18:41.
\newblock \href {https://doi.org/10.1145/3301443} {\path{doi:10.1145/3301443}}.
\newline\urlprefix\url{http://doi.acm.org/10.1145/3301443}
\bibitem{mao2017survey}
Y.~Mao, C.~You, J.~Zhang, K.~Huang, K.~B. Letaief, A survey on mobile edge
computing: The communication perspective, IEEE Communications Surveys \&
Tutorials 19~(4) (2017) 2322--2358.
\bibitem{mach2017mobile}
P.~Mach, Z.~Becvar, Mobile edge computing: A survey on architecture and
computation offloading, IEEE Communications Surveys \& Tutorials 19~(3)
(2017) 1628--1656.
\bibitem{pasteris2019service}
S.~Pasteris, S.~Wang, M.~Herbster, T.~He, Service placement with provable
guarantees in heterogeneous edge computing systems, in: IEEE INFOCOM
2019-IEEE Conference on Computer Communications, IEEE, 2019, pp. 514--522.
\bibitem{bi2019joint}
S.~{Bi}, L.~{Huang}, Y.~A. {Zhang}, Joint optimization of service caching
placement and computation offloading in mobile edge computing systems, IEEE
Transactions on Wireless Communications 19~(7) (2020) 4947--4963.
\newblock \href {https://doi.org/10.1109/TWC.2020.2988386}
{\path{doi:10.1109/TWC.2020.2988386}}.
\bibitem{9326369}
O.~Ascigil, A.~Tasiopoulos, T.~K. Phan, V.~Sourlas, I.~Psaras, G.~Pavlou,
Resource provisioning and allocation in function-as-a-service edge-clouds,
IEEE Transactions on Services Computing (2021) 1--1\href
{https://doi.org/10.1109/TSC.2021.3052139}
{\path{doi:10.1109/TSC.2021.3052139}}.
\bibitem{tan2021asymptotically}
H.~Tan, S.~H.-C. Jiang, Z.~Han, M.~Li, Asymptotically optimal online caching on
multiple caches with relaying and bypassing, IEEE/ACM Transactions on
Networking (2021).
\bibitem{xia2020online}
X.~Xia, F.~Chen, Q.~He, J.~Grundy, M.~Abdelrazek, H.~Jin, Online collaborative
data caching in edge computing, IEEE Transactions on Parallel and Distributed
Systems 32~(2) (2020) 281--294.
\bibitem{salem2021no}
T.~S. Salem, G.~Neglia, S.~Ioannidis, No-regret caching via online mirror
descent, in: ICC 2021-IEEE International Conference on Communications, IEEE,
2021, pp. 1--6.
\bibitem{mukhopadhyay2021online}
S.~Mukhopadhyay, A.~Sinha, Online caching with optimal switching regret, in:
2021 IEEE International Symposium on Information Theory (ISIT), IEEE, 2021,
pp. 1546--1551.
\bibitem{fan2022online}
S.~Fan, I.-H. Hou, L.~B. Van Sy~Mai, Online service caching and routing at the
edge with unknown arrivals (2022).
\bibitem{wang2015dynamic}
S.~Wang, R.~Urgaonkar, M.~Zafer, T.~He, K.~Chan, K.~K. Leung, Dynamic service
migration in mobile edge-clouds, in: 2015 IFIP Networking Conference (IFIP
Networking), IEEE, 2015, pp. 1--9.
\bibitem{xiong2101learning}
G.~Xiong, R.~Singh, J.~Li, Learning augmented index policy for optimal service
placement at the network edge, arXiv preprint arXiv:2101.03641.
\bibitem{zeng2019resource}
D.~Zeng, L.~Gu, S.~Pan, J.~Cai, S.~Guo, Resource management at the network
edge: A deep reinforcement learning approach, IEEE Network 33~(3) (2019)
26--33.
\bibitem{chen2019budget}
L.~Chen, J.~Xu, Budget-constrained edge service provisioning with demand
estimation via bandit learning, arXiv preprint arXiv:1903.09080 (2019).
\bibitem{he2020bandit}
W.~He, D.~He, Y.~Huang, Y.~Zhang, Y.~Xu, G.~Yun-feng, W.~Zhang, Bandit
learning-based service placement and resource allocation for mobile edge
computing, in: 2020 IEEE 31st Annual International Symposium on Personal,
Indoor and Mobile Radio Communications, IEEE, pp. 1--6.
\bibitem{iosup2008grid}
A.~Iosup, H.~Li, M.~Jan, S.~Anoep, C.~Dumitrescu, L.~Wolters, D.~H. Epema, The
grid workloads archive, Future Generation Computer Systems 24~(7) (2008)
672--686.
\bibitem{cohen2015following}
A.~Cohen, T.~Hazan, Following the perturbed leader for online structured
learning, in: International Conference on Machine Learning, PMLR, 2015, pp.
1034--1042.
\bibitem{abernethy2014online}
J.~Abernethy, C.~Lee, A.~Sinha, A.~Tewari, Online linear optimization via
smoothing, in: Conference on Learning Theory, PMLR, 2014, pp. 807--823.
\bibitem{hoeffding1994probability}
W.~Hoeffding, Probability inequalities for sums of bounded random variables,
in: The collected works of Wassily Hoeffding, Springer, 1994, pp. 409--426.
\bibitem{berend2013sharp}
D.~Berend, A.~Kontorovich, A sharp estimate of the binomial mean absolute
deviation with applications, Statistics \& Probability Letters 83~(4) (2013)
1254--1259.
\bibitem{robbins1955remark}
H.~Robbins, A remark on stirling's formula, The American mathematical monthly
62~(1) (1955) 26--29.
\end{thebibliography}