Deep Reinforcement Learning based Triggering Function for Early Classifiers of Time Series
Extracted source text · arXiv 2502.06584
\documentclass[sigconf, nonacm, table]{acmart}
\AtBeginDocument{%
\providecommand\BibTeX{{%
Bib\TeX}}}
\usepackage{bbm}
\usepackage{ulem}
\usepackage{amsmath}
\usepackage{multirow}
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\usepackage{pifont}%
\newcommand{\cmark}{\ding{51}}%
\newcommand{\xmark}{\ding{55}}%
\usepackage{subfig}
\setcopyright{acmlicensed}
\copyrightyear{2018}
\acmYear{2018}
\acmDOI{XXXXXXX.XXXXXXX}
\acmConference[Conference acronym 'XX]{Make sure to enter the correct
conference title from your rights confirmation emai}{June 03--05,
2018}{Woodstock, NY}
\acmISBN{978-1-4503-XXXX-X/18/06}
\begin{document}
\title{Deep Reinforcement Learning based Triggering Function \newline
for Early Classifiers of Time Series}
\author{Aurélien Renault}
\affiliation{%
\institution{Orange Innovation \& AgroParisTech}
\city{Paris}
\country{France}
}
\email{aurelien.renault@orange.com}
\author{Alexis Bondu}
\affiliation{%
\institution{Orange Innovation}
\city{Paris}
\country{France}}
\email{alexis.bondu@orange.com}
\author{Antoine Cornuéjols}
\affiliation{%
\institution{AgroParisTech UMR MIA-Paris}
\city{Palaiseau}
\country{France}}
\email{antoine.cornuejols@agroparistech.fr}
\author{Vincent Lemaire}
\affiliation{%
\institution{Orange Innovation}
\city{Paris}
\country{France}}
\email{vincent.lemaire@orange.com}
\renewcommand{\shortauthors}{Renault et al.}
\begin{abstract}
Early Classification of Time Series (ECTS) has been recognized as an important problem in many areas where decisions have to be taken as soon as possible, before the full data availability, while time pressure increases. Numerous ECTS approaches have been proposed, based on different triggering functions, each taking into account various pieces of information related to the incoming time series and/or the output of a classifier. Although their performances have been empirically compared in the literature, no studies have been carried out on the optimality of these triggering functions that involve ``man-tailored'' decision rules. Based on the same information, could there be better triggering functions?
This paper presents one way to investigate this question by showing first how to translate ECTS problems into Reinforcement Learning (RL) ones, where the very same information is used in the state space. A thorough comparison of the performance obtained by ``handmade'' approaches and their ``RL-based'' counterparts has been carried out.
A second question investigated in this paper is whether a different combination of information, defining the state space in RL systems, can achieve even better performance. Experiments show that the system we describe, called \textsc{Alert}, significantly outperforms its state-of-the-art competitors on a large number of datasets.
\end{abstract}
\begin{CCSXML}
<ccs2012>
<concept>
<concept_id>10010147.10010257.10010258.10010261</concept_id>
<concept_desc>Computing methodologies~Reinforcement learning</concept_desc>
<concept_significance>300</concept_significance>
</concept>
<concept>
<concept_id>10010147.10010178.10010187.10010193</concept_id>
<concept_desc>Computing methodologies~Temporal reasoning</concept_desc>
<concept_significance>500</concept_significance>
</concept>
<concept>
<concept_id>10010147.10010257.10010258.10010259.10010266</concept_id>
<concept_desc>Computing methodologies~Cost-sensitive learning</concept_desc>
<concept_significance>300</concept_significance>
</concept>
</ccs2012>
\end{CCSXML}
\ccsdesc[500]{Computing methodologies~Temporal reasoning}
\ccsdesc[300]{Computing methodologies~Cost-sensitive learning}
\ccsdesc[300]{Computing methodologies~Reinforcement learning}
\keywords{Time Series, Early Classification, Reinforcement Learning}
\maketitle
\section{Introduction}
\label{sec_Introduction}
In many real-world applications, early decisions must be made without \textit{complete knowledge} of the situation.
For instance, in Machine Learning, particularly in \textit{time-sensitive} applications such as anomaly detection \cite{Ruff2021}, predictive maintenance \cite{ran2019survey}, and autonomous driving \cite{ma2020}, a trade-off exists between making timely decisions and ensuring their reliability.
Therefore, it is crucial to find a balance between the \textit{earliness} (i.e. delay cost) and \textit{accuracy} (i.e. misclassification cost) of decisions, as they tend to evolve in opposite directions as new measurements become available.
\begin{figure}[]
\centering
\includegraphics[width=0.9\linewidth]
{plots/no_expe/fig-ECTS-archi-man-made-sep-v4.pdf}
\end{figure}
\begin{figure}[]
\centering
\includegraphics[width=0.9\linewidth]
{plots/no_expe/fig-ECTS-archi-man-made-non-sep-v3.pdf}
\end{figure}
\begin{figure}[]
\centering
\includegraphics[width=0.9\linewidth]
{plots/no_expe/fig-ECTS-archi-RL-based.pdf}
\caption{Different architectures for the ECTS problem. The top ones, separable and non separable, involve a man-tailored decision rule, whereas the bottom one does not rely on it. \\
}
\label{fig_archi_ECTS}
\end{figure}
This \textit{earliness} vs. \textit{accuracy} dilemma has been especially studied in the context of Early Classification of Time Series (ECTS) \cite{bondu2022open,renault2024early}.
In its most general form, an ECTS system can be defined as a function $d({\mathbf x}_t)$, such that:
\begin{equation}
d({\mathbf x}_t) =
\left\{
\begin{array}{ll}
\text{wait} & \mbox{if extra measures are queried;}\\
\hat{y} & \mbox{if prediction is triggered, or when $t=T$};
\end{array}
\right.
\label{eq:ects_model}
\end{equation}
\noindent
where, ${\mathbf x}_t$ represents the incoming time series, $T$ is its maximum length, and $\hat{y}$ is a predicted class value.
It is commonly recognized that two functions are involved in ECTS systems: (\textit{i}) a \textit{classifier} $h$ which computes the class $\hat{y}$ of the incoming time series and may provide additional information as a set of features $\mathcal{F}$ characterizing both the time series and the prediction itself, such as current time and confidence levels, and (\textit{ii}) a \textit{triggering function} $g$ that
decides, on the basis of $\mathcal{F}$,
\textit{when} to make a prediction and produces $\hat{y}$ if the time is deemed correct.
Most of the proposed approaches implement separately the two functions, often with the triggering function only using information provided by the classifier. This type of approach is called \textit{separable} \cite{renault2024early}. Other systems do learn the two functions in an \textit{end-to-end} way, as a single combined function denoted by $(g \& h)$.
However, the difference between separable and end-to-end approaches does not tell the whole story (see Fig. \ref{fig_archi_ECTS}).
It is interesting to decompose the triggering function as $g = r \circ c$, where $c$ is a \textit{trigger criterion} that produces an intermediate representation $\mathcal{F}'$ containing all the features used to trigger decisions, and $r$ a \textit{decision rule} that accounts for the making of the final decision, based on $\mathcal{F}'$.
Then, separable approaches can be described as $d(\mathbf{x}_t) = r \circ c \circ h(\mathbf{x}_t)$.
In the same way, end-to-end approaches also involve a trigger rule such as $d(\mathbf{x}_t)=r \circ (c \& h)(\mathbf{x}_t)$.
Indeed, most of the time when looking closely, there is a final decision rule $r(f')$, with $f' \in \mathcal{F}'$, that determines triggering moments, such as a comparison with a threshold \cite{mori2019early}, or a rule like: \textit{if the expected cost of the decision is lower now than what is expected for any future time, predict now} \cite{achenchabe2021early}.
When this final decision rule $r$ has been put by hand in the algorithm, we say that the decision rule is \textit{man-tailored}. To the best of our knowledge, this is the case for the vast majority of approaches, except for ECTS systems learned using reinforcement learning. In the latter, the system makes decisions
without any reference to the ECTS problem. This case corresponds to what we call \textit{RL-based} triggering function.
The choice of the decision rule $r$ and of the feature sets $\mathcal{F}$ and $\mathcal{F}'$ to take into account are crucial parts of an ECTS approach. It drives the decisions and makes the difference, given that classifiers for time series are well-developed and readily available \cite{bagnall2017great, ruiz2021great}.
Accordingly, a number of separable approaches have been proposed in recent years, representing most of the literature, whose decision rules $r$ are man-tailored and that exploit carefully designed feature sets $\mathcal{F}$ and $\mathcal{F}'$.
For example, the SR \cite{mori2017early} and ECEC \cite{lv2019effective} systems rely on the classifier's confidence at time $t$ in their predictions (see Section \ref{sec:related_work}).
Although these approaches offer state-of-the-art performance (see \cite{renault2024early} for rigorous, in-depth comparisons), the question of the decision rules' and feature sets' \textit{optimality} still remains. A first question is thus \textbf{using the same features set $\mathcal{F}$, are there better triggering functions and decision rules?}
A second question is: \textbf{can we find better features than the ones used in existing ECTS systems?}
One way to answer these questions is to use Reinforcement Learning (RL), which is suitable for online decision-making and is feature-agnostic.
It becomes possible to train separable ECTS systems that use the same input feature set $\mathcal{F}$, and then compare their performance with the corresponding literature approaches whose decision rules are man-tailored. One may also choose other sets of features $\mathcal{F}$ and let the systems learn to use them for decision making, resulting in new ECTS algorithms.
This is what we have done in the study presented in this paper.
\begin{figure}[!htb]
\centering
\subfloat[Stopping Rule \cite{mori2017early}]{\includegraphics[width=0.22\textwidth]{plots/decision_contour/tex/trigger_heat_stopping_rule_top.pdf}\label{r_sr}}
\subfloat[Reinforcement Learning]{\includegraphics[width=0.22\textwidth]{plots/decision_contour/tex/trigger_heat_dqn_sr_top.pdf}\label{r_dqn_sr}}
\caption{Heatmap representing decision rule $r$ on the ChilledWaterPredictor dataset, learned by \textsc{Stopping Rule} (\protect\ref{r_sr}) and using RL (\protect\ref{r_dqn_sr}), based on (\textit{i}) the maximum probability estimated by $h$, in $y$-axis and (\textit{ii}) the proportion seen of the time series, in $x$-axis (see Section \ref{sec:expe}). Red lines delimit areas where the probability of triggering, estimated by a sigmoid function, is above 0.5. %
}
\label{fig:motivation_example}
\end{figure}
For instance, Figure \ref{r_dqn_sr} represents a decision rule $r$ learned by a RL agent, using the same features $\mathcal{F}$ and the same classifier $h$ as in the SR approach \cite{mori2017early}.
It is noticeable that SR can only learn a linear decision rule, whereas reinforcement learning produces a more complex non-linear one (lines in red in Figure \ref{r_dqn_sr}).
This example shows that RL can be used to learn new types of triggering functions, and the question then arises about their performance.
\smallskip
This paper presents two main contributions:
\begin{enumerate}
\item First, we present a methodology to translate ECTS problems into Reinforcement Learning problems, in the case of separable approaches. The same feature sets $\mathcal{F}$ as the literature approaches are used to define the state space. It is then possible to compare the performance obtained by competing approaches based on ``man-tailored'' decision rules and their ``RL-based'' counterparts, all other things being equal. Extensive experiments have been carried out on a large number of public data sets. %
\item Second, based on this methodology, we present a new ECTS system, called \textsc{Alert} (\textit{A reinforcement Learning based Early classifieR's Trigger function}) that takes into account a combination of the features used in several methods from the literature and automatically learns to make timely decisions. We empirically compare its performance with state-of-the-art competitors for a range of weighted misclassification and delay costs and on the same set of public datasets.
\end{enumerate}
The document is organized as follows.
Section \ref{sec:problem_statement} presents the ECTS problem.
Section \ref{sec_information_input_ECTS} focuses on information that ECTS systems take into account.
Section \ref{sec:related_work} presents a perspective on the literature.
Section \ref{sec:proposed_approach} describes the approach and methodology proposed to answer the questions raised above.
Experimental results are presented in Section \ref{sec:expe}.
In the concluding section, we highlight the importance of our results and the impact they may have on future work.
\subsection*{Notations}
\smallskip
\begin{tabular}{lcl}
$d({\mathbf x}_t)$&:& is an ECTS system such that $d = r \circ c \circ h$.\\
$h({\mathbf x}_t)$&:& a classification function that returns $f \in \mathcal{F}$, a\\
&& set of features describing both the class prediction\\
&& and possibly the incoming time series.\\
$g(f)$&:& a triggering function composed by $g = r \circ c$, where:\\
$c(f)$&:& is a triggering criterion producing $f' \in \mathcal{F}'$, a set of\\
&& features on which the decision is made; \\
$r(f')$&:& is a decision rule triggering predictions $\hat{y}$ at time $\hat{t}$.\\
\end{tabular}
\section{Problem statement}
\label{sec:problem_statement}
In the ECTS problem, measurements of an input time series are observed over time. At time $t$, the incomplete time series ${\mathbf x}_t \, = \, \langle {x_1}, \ldots, {x_t} \rangle$ is available where ${x_i}_{(1 \leq i \leq t)}$ denotes the time indexed measurements. These measurements can be single or multi-valued. It is assumed that each input time series belongs to an unknown class $y \in {\mathcal{Y}}$. The task is to make a prediction $\hat{y} \in {\mathcal{Y}}$ about the class of the incoming time series, at a time $\hat{t} \in [1,T]$
which optimizes a trade-off between two costs:
\begin{itemize}
\item The \textit{misclassification cost} of predicting $\hat{y}$ when the true class is $y$: $\mathrm{C}_m(\hat{y}|y): {\mathcal{Y}} \times {\mathcal{Y}} \rightarrow \mathbb{R}$.
\item The \textit{delay cost}: $\mathrm{C}_d(t) : \mathbb{R}^+ \rightarrow \mathbb{R}$, which is usually a non-decreasing function over time.
\end{itemize}
Given a classifier $h(\mathbf{x}_t)$, which predicts the class of an input time series $\mathbf{x}_t$ for any $t \in [1, T]$: $\hat{y}=h(\mathbf{x}_t)$, the cost incurred when a prediction has been triggered at time $t$ is given by a loss function\footnote{In the literature, this additive form of the costs is widely used for didactic purposes. More generally, the delay cost may depend on the true class $y$ and the predicted one $\hat{y}$, and a single cost function $\mathrm{C}(\hat{y}|y, t)$ integrating misclassification and delay costs should then be used.} that sums the two costs: $\mathcal{L}(\hat{y}, y, \hat{t}) = \mathrm{C}_m(\hat{y}|y) + \mathrm{C}_d(\hat{t})$. The trade-off comes from the fact that the misclassification cost is generally a decreasing function of time as new measurements allow for better predictions, whereas the delay cost increases over time.
The crucial part is to decide \textit{when to make a prediction}, given that the incoming time series is incomplete before $T$. %
From a machine learning point of view, answering this question amounts to find a function $d \in {\mathcal{D}}$, whose general form is given by Equation \ref{eq:ects_model}, that best optimizes the loss function $\mathcal{L}$, minimizing the true risk over all time series distributed according to the distribution\footnote{
Notice that the notation $\mathcal{X}$ is an abuse that we use use to simplify our purpose. In all mathematical rigor, the measurements observed successively constitute a family of time-indexed random variables $\mathbf{x} = (\mathbf{x}_t)_{t \in [1,T]}$. This stochastic process $\mathbf{x}$ is not generated as commonly by a distribution, but by a filtration $\mathbb{F} = (\mathcal{F}_t)_{t \in [1,T]}$ which is defined as a collection of nested $\sigma$-algebras \cite{klenke2013} allowing to consider time dependencies. Therefore, the distribution $\mathcal{X}$ should also be re-written as a filtration.
}
$\mathbb{P}_{\mathcal{X}}$ that governs the time series in the application:
\begin{equation}
\small
\normalsize
\argmin_{d \in \mathcal{D}} \mathbb{E}_{\mathbf{x} \sim \mathbb{P}_{\mathcal{X}}} \mathcal{L}(\hat{y}, y, \hat{t})
\label{eq:cost1}
\end{equation}
$\mathbb{P}_{\mathcal X}$ being unknown, instead of using Equation \ref{eq:cost1},
the purpose is to minimize the empirical risk, also called \textit{average cost} in the ECTS literature, for a training set of $M$ time series:
\vspace{-5mm}
\begin{align}
AvgCost \; = \frac{1}{M} \sum_{i=1}^{M} \mathcal{L}(\hat{y}_i, y_i, \hat{t}) \; = \; \frac{1}{M} \sum_{i=1}^{M} \mathrm{C}_m(\hat{y}_i|y_i) + \mathrm{C}_d(\hat{t}_i)
\label{eq:avgcost_train}
\end{align}
\vspace{-1mm}
Finally, $AvgCost$ is an essential metric for guiding both the training of the $d$ function and its evaluation, since it measures the compromise achieved between the two conflicting objectives of \textit{earliness} and decision \textit{accuracy} (we note $AvgCost^\star$ the best achievable cost).
\section{Information that ECTS systems take into account
\label{sec_information_input_ECTS}}
\noindent
An important question is about which \textit{information} is taken into account when deciding when to stop observing the incoming time series and make a prediction about its class.
Several possibilities exist:
\begin{enumerate}
\item Using only the \textit{time information} $t$. In this case, $c$ (or $c \& h$ in the non separable approaches) simply transmits $f' = \{\hat{y}_t, t\}$ and the decision function $r$ triggers a prediction when $t$ meets some condition, such as: $t=1$ (as soon as possible), or when $t=T$ (at last as possible), or for any other \textit{a priori} determined instant \cite{xing2009early}.
\item Using the \textit{representation of the incoming time series} $\mathbf{x}_t$. Here, $h$ transmits $\hat{y}_t$ and the representation of $\mathbf{x}_t$ as $\mathcal{F}$ in addition to other information.
\item Using the \textit{confidence levels} of the predictions of the classifier. There, $c$ (or $c \& h$) transmits $\hat{y}_t$ and the confidence levels computed by $h$ for all classes \cite{mori2017early, lv2019effective, schafer2020teaser, bilski2023calimera}, and $r$ triggers the prediction $\hat{y}_t$ as soon the highest confidence level, $\max_{y \in \mathcal{Y}} p(y|\mathbf{x}_t))$, is above some predefined threshold. Or it may decide when the difference between the highest confidence level and the second one is above a certain value.
\item It must be noted that the above kinds of triggering criteria do not take into account \textit{the costs} involved in the trade-off to be optimized. It would be natural to take these explicitly into account. For instance, the ECTS systems \cite{dachraoui2015early, tavenard2016cost, achenchabe2021early, zafar2021early, bilski2023calimera} aim to estimate the total cost expectation for future time steps, and are referred to as \textit{non-myopic}.
\end{enumerate}
Within the possible architectures identified in Figure \ref{fig_archi_ECTS}, there is thus a whole range of possible realizations for ECTS systems. Apart from the classifier used, the difference between the ECTS systems rests mainly on the design of the feature sets $\mathcal{F}$ and $\mathcal{F}$',
and the decision function $r$. A study of the state-of-the-art reveals the variety of possibilities explored so far.
\section{A perspective on the state of the art in ECTS}
\label{sec:related_work}
This section highlights the distinction between ``man-tailored'' and ``RL-based'' approaches as introduced in Section \ref{sec_Introduction}.
The following section identifies for each state-of-the-art approach the important components, such as $\mathcal{F}$, $\mathcal{F}'$, $c$, and $r$.
\subsection{ECTS using man-tailored decision rules}\label{man_tailored}
Algorithms presented in \cite{xing2009early, xing2011extracting, xing2012early} use the raw representation of time series \cite{xing2009early}, or shapelet-based representation \cite{xing2011extracting, xing2012early}. The function $c$ transmits the one nearest neighbor of $\mathbf{x}_t$, in the chosen representation space as $\hat{y}_t$, plus the time $t$, and $r$ triggers the prediction as soon as $t = \nu$ where the time $\nu$ is when predictions based on the one nearest neighbor do not vary anymore for all time series in the training set, or in other words, when the accuracy of classification most likely is close to the accuracy on the full time series.
Other algorithms also use either raw representations or dictionary-based ones of $\mathbf{x}_t$, are separable, and use triggering criteria $c$ based on confidence levels estimated by the classifier. This corresponds to the case (3) above in Section \ref{sec_information_input_ECTS}.
For instance, SR \cite{mori2017early} computes the highest confidence level and the second one for the possible classes in addition to $\frac{t}{T}$: this is $\mathcal{F}$. It then transmits a linear combination $\mathcal{F}'$ of them to $r$, which simply checks whether the expression is positive.
In the ECEC system \cite{lv2019effective}, $h$ computes at time $t$ the set of features $\mathcal{F}$ as the history of past classifications $(h(\mathbf{x}_i))_{1\leq i \leq t}$ and transmits it to $c$. In turn, $c$ implements a criterion that evaluates, in essence, the stability and hence the confidence of the predictions of the classifier, and $r$ decides to trigger a prediction when this estimated confidence is above some threshold.
The TEASER method \cite{schafer2020teaser} uses a classifier $h$ that computes the class probabilities for the incoming $\mathbf{x}_t$ and transmits them to $c$. In turn $c$, is a classifier in its own right that classifies the prediction $\hat{y}_t$ as reliable or not. Finally, $r$ takes the last reliable predictions and their associated times and decides to trigger the prediction $\hat{y}$ only if the same prediction was also given for a number of successive time steps (i.e. a hyperparameter to be tuned).
ECONOMY and its variant \cite{dachraoui2015early, tavenard2016cost, achenchabe2021early, zafar2021early} are non-myopic approaches, where the function $c$ computes the expected costs, a combination of the misclassification cost and the delay cost, for the current time step $t$ and all future ones and transmits them in $\mathcal{F}$'. The function $r$ then
triggers a decision
as soon as the expected cost for the current time step $t$ is the lowest among all expected costs for future times.
CALIMERA \cite{bilski2023calimera} is another non-myopic approach that exploits a collection of regressor models, learned in a backward induction fashion as a triggering function. The minimum cost occurring in the future time period $[t,T]$ is denoted as $minFutureCost_t$. For a particular time step $t$, the corresponding regressor aims to predict the difference $\Delta = minFutureCost_t - minFuturCost_{t+1}$. Then, the function $r$ triggers a decision when $\Delta > 0$, i.e. the optimum trigger time is about to be exceeded.
Another range of methods is based on Sequential Probability Ratio Test \cite{wald1948optimum}, which, for a given error rate, offers theoretical optimality under i.i.d. assumption between measurements of the time series, as well as an infinite sampling horizon. Recent papers \cite{ebihara2020sequential, ebihara2023toward, ringelearly} try to relax those constraints to make this kind of methods more easily applicable in practice. Those methods fall into the separable realm, where log-likelihood ratios are calculated first and then compared to thresholds.
Full deep-learning architectures have been recently developed. These can be either separable like the SOCN \cite{lv2023second} algorithm, the $\mathcal{F}$ set consists here in the predicted class probabilities sequence from a deep-learning based time series classifier $h$. It is then passed to a transformer-based network $c$, which outputs a confidence scalar as $\mathcal{F}^{'}$, which is itself compared to a threshold by $r$ in order to trigger. Full deep-learning architectures also can be end-to-end as exemplified with ELECTS \cite{russwurm2023end}, where $c\&h$ outputs both the predicted class value and a probability distribution of triggering the decision. Then, the function $r$ samples a value using this distribution to trigger (or not) the decision.
\subsection{ECTS using Reinforcement Learning}
While the works cited above use man-tailored decision rules involving \textit{ad-hoc} parameters (e.g. thresholds), other works have explored the use of RL, without the need to define the $r$ function beforehand.
\cite{martinez2018deep, martinez2020adaptive} show in a didactic way how to express the ECTS problem in terms of state space, action space and rewards in order to solve it using a value-based reinforcement learning technique. As an end-to-end RL method, both $h$ and $g$ functions are learned simultaneously.
EarlyStop-RL \cite{wang2024deep} is aimed at the early detection of lung cancer. It implements an end-to-end approach as well and highlights the possibility of handling any cost function.
Among the separable approaches, EARLIEST and its variants \cite{hartvigsen2019adaptive, hartvigsen2020recurrent, hartvigsen2022stop} train three modules jointly: an encoder, a triggering agent, and a discriminator, the latter making the classification decision. Here, $\mathcal{F}$ represents the time series embedding, produced by the RNN-based encoder; $g$ is the triggering agent, and $h$ thus includes both the encoder and discriminator. The use of a shared loss function encourages collaboration between the different modules.
The SNP algorithm \cite{huang2022snippet1, huang2022snippet}, is a separable framework that learns a triggering agent $g$ using RL, optimized with evolutionary algorithms, given pre-trained encoder and classification modules.
These pioneering works show that RL is one possible solution to learn ECTS systems. What remained to be done is a thorough comparison between the RL-based triggering functions and the ones devised by experts. Does RL find innovative triggering criteria and decision rules with better performance? In order to answer this question, the comparison must bear only on the triggering part, all other things being equal. Therefore, the same classification function should be used, which implies the use of separable approaches.
\section{Proposed approach}
\label{sec:proposed_approach}
In this section, we show how to formulate the ECTS problem such that it can be solved using Reinforcement Learning, in view of being able to compare existing triggering functions, involving man-tailored decision rules, and RL-based ones.
We further present \textsc{Alert}, a deep RL triggering function, that can be used in any kind of separable ECTS architecture. In particular, the state space is versatile and can easily be customized by the user.
\subsection{Reinforcement learning}
\label{sec_RL_in_general}
Reinforcement learning \cite{sutton2018reinforcement} aims at learning a function, called a policy $\pi$, from states to actions: $\pi: {\mathcal{S}} \rightarrow {\mathcal{A}}$. Rewards can be associated with transitions from states $s_t \in {\mathcal{S}}$ to states $s_{t+1} \in {\mathcal{S}}$ under an action $a \in {\mathcal{A}}$: $reward(s_t, a, s_{t+1}) \in \mathbb{R}$.
Given a state $s_t$ and an action $a_t$, the sequence of rewards received after time step $t$ gives rise to a \textit{gain} which classically is a discounted sum of the rewards: $G_t=\sum_{k=0}^\infty \gamma^k R_{t+k+1}$ with $\gamma \in [0, 1]$ the discount factor and $R_{t} = reward(s_t, a_t, s_{t+1})$.
In all generality, the result of an action $a$ in state $s_t$ may be non-deterministic. An optimal policy $\pi^\star$ maximizes the expected gain from any state $s_t \in {\mathcal{S}}$.
One approach to learn a policy is to use a state-action value function, which maps for each pair $(s, a)$ the expected gain starting from $s$, taking action $a$ and following $\pi$ afterwards:
\begin{equation}
\begin{split}
q_\pi(s_t, a_t) \; &\doteq \; \mathbb{E}_\pi[G_t \, | \, s_t, a_t] \\
\end{split}
\label{eq_rl_v_function}
\end{equation}
where $\mathbb{E}_\pi[\cdot]$ denotes the expected value of a random variable given that the agent follows the policy $\pi$.
Optimal policies share the same optimal action-value functions:
\begin{equation}
q^\star(s_t, a_t) = \operatornamewithlimits{max}_\pi \, q_\pi(s_t,a_t)
\end{equation}
Given the optimal state-action value function, one can easily derive an optimal policy as : $\pi^\star(s) = \argmax_{a} q^\star(s,a)$. \textit{Q-learning} \cite{dayan1992q} is a popular way of directly approximating $q^{\star}$ with updates defined by :
\begin{equation}
Q(s_t, a_t) \xleftarrow{}Q(s_t,a_t) + \alpha \bigl[ R_{t+1}' + \gamma \max_a Q(s_{t+1}, a) - Q(s_t, a_t) \bigr]
\label{eq_Q_learning}
\end{equation}
where $Q$ is the estimated $q$ and $R_{t+1}'$ is the measured return from state $s_t$ when choosing action $a_t$. $R_{t+1}'$ is typically either the immediate reward $R_t$ or a cumulated gain from the current state to the episode's end, if episodes are defined.
\subsection{RL formulation of the ECTS problem}
\label{sec_RL_for_ECTS}
In this section, we reformulate the ECTS problem as a Reinforcement Learning one for separable approaches, where the classifier $h$ is provided.
The agent must learn which \textit{action} to take (i.e. decision ``wait'' or ``trigger'' prediction $\hat{y}$) given its current \textit{state}, i.e. any information from the classifier $h$.
An \textit{episode} is defined as the sequence of states $s_t$ and actions $a_t$ starting from $t=1$ until a prediction is triggered. For each training time series, therefore, the agent observes a sequence of states that describe information provided by $h$, and receives rewards according to its choice of actions.
The \textit{rewards} function should be defined using the previously defined costs function $C_m$ and $C_d$. The simplest way would consist in only providing the inverse of the full paid cost once the agent decides to trigger. However, it has been shown that for ECTS, providing intermediate rewards facilitates the agent's learning \cite{martinez2020adaptive}. Thus the following reward function is defined as:
\begin{equation}
reward(s_t, a_t) = \left\{
\begin{array}{ll}
-\Delta C_d(t) & \mbox{if } a_t=\text{``wait''} \\
-C_m(y|\hat{y}) - \Delta C_d(t) & \mbox{if } a_t=\text{``trigger''}
\end{array}
\right.
\label{eq:grad_reward}
\end{equation}
where
\begin{equation}
\Delta C_d(t) = \left\{
\begin{array}{ll}
C_d(1) & \mbox{if } t=1 \\
C_d(t) - C_d(t-1) & \mbox{if } t>1
\end{array}
\right.
\end{equation}
This function does not depend on $s_{t+1}$ anymore, due to the deterministic cost functions and classifier used in separable ECTS approaches. This definition assumes that the cost functions $C_d$ and $C_m$ are given by the environment and that they can be decomposed additively\footnote{The proposed approach can be extended to non-decomposable reward functions, as shown by complementary experiments in Appendix \ref{app_end}, where reward is delayed at the end of episodes.}.
The \textit{state} space $\mathcal{S}$ is of arbitrary form and can encode both a representation of the time series and any information provided by the classifier $h$. We consider $\mathcal{S}$ as a vector space of arbitrary dimension, playing the same role as the set of features $\mathcal{F}$ described above. In the case of a continuous state space, Q-values can be estimated by using a parameterized function $Q_\theta(s, a)$ typically implemented by a neural network.
\subsection{State space}
\label{state_space}
As pointed out in Section \ref{sec_information_input_ECTS}, the information taken into account by ECTS systems is a key determinant of their performance.
For designing performing RL-based triggering functions, the state space $\mathcal{S}$ may be of limited dimension and, at the same time, it needs to include as much relevant information as possible.
\textsc{Alert} is a generic approach taking into account any vector space $\mathcal{S}$, which will vary during experiments. Based on the most performing state-of-the-art approaches \cite{renault2024early}, we identify a set of features from which experiments will be carried out: the \textit{predicted class label} \cite{schafer2020teaser}: $\argmax_{k \in \mathcal{Y}} p(y=k|x_t)$, the \textit{maximum posterior} \cite{mori2017early, bilski2023calimera}: $\max_{k \in \mathcal{Y}} p(y = k | x_t)$, the \textit{margin} \cite{mori2017early, schafer2020teaser, bilski2023calimera} which is the difference between the two largest posterior probabilities, estimation of the \textit{level of confidence} \cite{achenchabe2021early, zafar2021early} in the prediction(s), which can be represented by the bin index within an equal-frequency discretization of maximum posteriors and the \textit{current time} $t$ \cite{mori2017early}, which, in the case of finite time series of length $T$, provides the proportion of the time series observed so far.
Specifically, \textsc{Alert$^{\star}$} refers in the following to a variant that uses all of these features within the state space $\mathcal{S}$.
\subsection{Training methodology}
In our implementation, called \textsc{Alert} (A reinforcement Learning based Early classifierR's Trigger function), we chose to use the popular Double Deep Q-Network (\textit{DDQN}) algorithm \cite{mnih2015human, van2016deep} with some adaptations to handle ECTS problems:
\begin{itemize}
\item \textit{The state space} components have been normalized when needed, to keep them all in the $[0,1]$ range \cite{tarasov2024revisiting, fujimoto2021minimalist}. The predicted class label has been one-hot encoded and the indexes of confidence levels have been MinMax scaled.
\item \textit{Offline RL} \cite{levine2020offline, prudencio2023survey}, also called \textit{Batch RL}, allows one to learn a policy without interacting directly with the environment, but rather from a static train set of previously collected interactions. The \textsc{Alert} approach exploits a particular case of Offline RL where interactions $(s_t, a_t, s_{t+1}, r_t)$ are exhaustively extracted from training time series. Indeed, ECTS is a simple problem where actions ($\mathcal{A} = \{wait,\:trigger \}$) do not modify the observed data, i.e. triggered predictions are final and measurements after triggering are not observed. %
\item \textit{Layer Normalization} \cite{lei2016layer} in the Q-network has been used, as it has been found to mitigate over-estimation biases, bounding the outputted Q-values \cite{ball2023efficient, tarasov2024revisiting}.
\item %
\textit{Regularization} is ensured by a model selection strategy that limits the number of epochs and thus effectively combats overfitting.
First, several policies are trained over different train/validation splits, as it has been shown to greatly improve offline off-policy evaluation \cite{nie2022data}. Then, for each split, the policy under training is evaluated over the validation set at a given epoch frequency in an \textit{online} fashion, i.e.
with time-series measurements being observed progressively, as at the testing time. The \textit{AvgCost} is used as a metric at each validation stage, and the corresponding model is saved. Finally, at the end of training, the epoch index for which the validation metric is lowest on average across all splits is selected.
Among the models trained for this number of epochs, the best-performing one over all splits is then selected to be the final model.
\end{itemize}
\section{Experiments}
\label{sec:expe}
The first part of the experiments is dedicated to \textbf{question \#1}: \textit{do RL-based triggering functions outperform their state-of-the-art counterparts, using man-tailored decision rules}, i.e. when using the same input information? %
The second part aims at examining \textbf{question \#2}: \textit{whether a different combination of information within $\mathcal{S}$
can improve the performance}.
Finally, we examine the sensitivity in state space definition.
\subsection{Datasets}
\label{sec:expe_data_and_costs}
Extensive experiments have been carried out on 31 datasets: 20 from the UCR archive \cite{dau2019ucr} and 11\footnote{And not 15 as in \cite{renault2024early}, as in the anomaly detection setting, some of the problems become too hard for the considered classifiers to operate, i.e. there is no performance gain when increasing the number of observations in the time series.} from the Monash time series extrinsic regression archive \cite{tan2021time}, transformed into binary classification task. We have selected datasets that are not z-normalized, so as to avoid possibilities of information leakage \cite{wu2021early}.
\subsection{Evaluation and cost setting}
To suit numerous applications, for instance anomaly detection or in hospital emergency services, we chose to use imbalanced misclassification costs and exponential delay costs, as in \cite{renault2024early}.
For our experiments, we used the definition of the costs described:
\begin{align}
&C_d(t) = \exp( \frac{t}{T} \times \log100) \\
&C_m(\hat{y}|y) = \left\{
\begin{array}{ll}
100 \times \mathbbm{1}(\hat{y} \neq y) & \mbox{if $y=$ minority class} \\
\mathbbm{1}(\hat{y} \neq y) & \mbox{otherwise}
\end{array}
\right.
\end{align}
Evaluation is conducted using the \textit{AvgCost} metric. Furthermore, in order to assess how the methods adapt to various balances between the misclassification and the delay costs,
the methods are evaluated using a weighted \textit{AvgCost}, as defined in Equation (\ref{eq:avgcost_weigth}), for values of $\alpha$ varying from $0$ to $1$, with a $0.1$ step:
\begin{align}
AvgCost_{\alpha} = \frac{1}{N} \sum_{i=0}^{N} \alpha \times C_m(\hat{y}_i|y_i) + (1 - \alpha) \times C_d(\hat{t}_i)
\label{eq:avgcost_weigth}
\end{align}
\subsection{Competing Approaches}
Four competing separable approaches have been selected from the top performers benchmarked in \cite{renault2024early}. The end-to-end approach \textsc{Earliest}, although not directly comparable since it does not use the same classifier, is still considered the main RL-based competitor.
\begin{itemize}
\item \textsc{Alert} variants %
consist in varying the components in the state space to match the one of competitors,
taking exactly the same information as input, e.g. \textsc{Alert\_SR} is the RL counterpart of \textsc{Stopping Rule}.
\item \textsc{Alert}$^\star$ is the variant that takes as input all the features described in Section \ref{state_space}.
\item \textsc{Calimera} \cite{bilski2023calimera} triggers a decision when the value predicted buy regressor models becomes positive (see Section \ref{man_tailored}).
\item \textsc{Economy-$\gamma$-Max} \cite{achenchabe2021early} triggers a decision if the predicted cost expectation is the lowest at time $t$ when compared with the expected cost for all future time steps. %
\item \textsc{Stopping Rule} \cite{mori2017early} uses a linear combination of two confidence levels and a delay measure.
\item \textsc{Proba Threshold} triggers a prediction if the maximum posterior exceeds some threshold, found by grid search.
\item \textsc{Earliest} \cite{hartvigsen2019adaptive} is the only end-to-end deep RL method that has been adapted to different cost setting by cross-validating the $\lambda$ hyperparameter, measuring the earliness importance.
\end{itemize}
\subsection{Implementation specifications}
For all datasets, we use $70\%$ training, $30\%$ testing. Within the training set, $50\%$ is used for classifiers' and $50\%$ for training the trigger model. The classifier module is a collection of \textsc{MiniROCKET} \cite{dempster2021minirocket} estimators, learned over every $5\%$ of the time series. A calibration step is added as in \cite{bilski2023calimera, renault2024early}. For \textsc{Alert}, $30\%$ of the training data is used for validation and model selection. We use a single-layer Neural Network with a hidden dimension of 32 to be our Q-estimator. The model is optimized using Adam \cite{kingma2015adam} with a learning rate of $1e^{-4}$. The target network uses soft updates based on parameter $\tau$ \cite{lillicrap2015continuous} equal to $3e^{-3}$. The code to run the experiments is available on \url{https://anonymous.4open.science/r/ALERT}. It is based on \texttt{PyTorch} \cite{paszke2019pytorch} for automatic differentiation and on \texttt{ml\_edm} \cite{renault2024ml_edm} for general ECTS evaluation functions and interface.
\subsection{Results}
\subsubsection{SOTA methods vs. their RL counterparts {\bf (question \#1)}}
\begin{figure}[!htb]
\centering
\subfloat[\textsc{Economy}: $\mathcal{S} = \{ \textit{level of confidence} \}$]{\includegraphics[width=0.9\linewidth]{plots/pairwise_sota/tex/alert_eco.pdf}\label{pair_eco}} \\
\subfloat[\textsc{Stopping Rule}: $\mathcal{S} = \{\textit{max posterior}, \textit{margin}, \textit{time} \}$]{\includegraphics[width=0.9\linewidth]{plots/pairwise_sota/tex/alert_sr.pdf}\label{pair_sr}} \\
\subfloat[\textsc{Calimera}: $\mathcal{S} = \{\textit{all posteriors}, \textit{max posterior}, \textit{margin} \}$]{\includegraphics[width=0.9\linewidth]{plots/pairwise_sota/tex/alert_cal.pdf}\label{pair_cal}} \\
\caption{Pairwise comparison of SOTA methods versus their RL counterpart using same information as input. The \textcolor{blue}{blue} curve, ranging from 0 to 1, represents the win rate of the man-tailored method over full benchmark. The \textcolor{orange}{orange} curve, ranging from -1 to 1, represents the difference of \textit{AvgCost} between base and RL counterparts, normalized by $\textit{AvgCost}^{\star}$, occurring at the best triggering time. In both cases, points above the horizontal line indicates that the man-tailored method is better than its RL-based counterpart. %
}
\label{fig:pairwise}
\end{figure}
For almost all values of the parameter $\alpha$, \textsc{Economy} is better than its RL counterpart, and significantly better for $\alpha \geq 0.5$ (see Appendix \ref{wilco_pair}). For instance, in Figure \ref{pair_eco}, when $\alpha = 0.8$, \textsc{Economy} wins over almost $85\%$ of the datasets, and is $10\%$ closer to the $AvgCost^{\star}$.
\textsc{Stopping Rule} on its side, shows no significant differences from its RL counterpart, except for $\alpha = 0.5$. (see Appendix \ref{wilco_pair}).
The verdict is different for \textsc{Calimera}, for which the RL version tends to be better as $\alpha$ grows.
What could explain these differences in the comparison of state-of-the-art methods with RL counterparts? It is noticeable that the state spaces $\mathcal{S}$ of (\textit{i}) \textsc{Economy}
, (\textit{ii}) \textsc{Stopping Rule }
and (\textit{iii}) \textsc{Calimera}
are in increasing order of size.
RL thus seems to take better advantage of a larger state space. The question then arises as to the extent to which a larger state space could further improve the performance of \textsc{Alert}.
\subsubsection{$\textsc{Alert}^{\star}$ vs. SOTA methods {\bf (question \#2)}}
One advantage of RL is that adding features to the state space has little impact on computation time and implementation. Given the observation that increased state space allows better performance of the \textsc{Alert} method, we thus consider \textsc{Alert$^{\star}$}, where $\mathcal{S_{\textsc{Alert}^{\star}}} = \{\textit{max posterior},\: \textit{margin},$ $\textit{pred class}$,\: $\textit{level of confidence},\: \textit{time} \}$.
Figure \ref{bump_sota} shows that, on average, \textsc{Alert$^{\star}$} dominates all state-of-the-art methods for the whole range of $\alpha$ values. For $\alpha > 0.5$, in Figure \ref{bump_sota}, the difference in terms of mean ranks is significant (see Appendix \ref{wilco_alert}). When $\alpha = 0.8$ for example, Figure \ref{cd_sota} confirms the statistical significance between \textsc{Alert$^{\star}$} and competitors. For $\alpha \leq 0.5$, differences between approaches are not significant: %
with the cost of delay increasing exponentially, the optimal strategy is to trigger as soon as possible, making it difficult to distinguish a dominant approach.
Even though not strictly comparable, the performance of \textsc{Earliest}, an end-to-end method, has been reported in Figure \ref{bump_sota}. It performs as well as \textsc{Proba Threshold}, which is a strong baseline. This is remarkable since \textsc{Earliest} only takes the cost into account through a single hyperparameter rather than directly in the reward signal and does not benefit from the high-quality predictions of the specialized classifier \textsc{MiniROCKET}. Given these results, end-to-end RL-based approaches deserve to be further investigated.
\begin{figure}[!htb]
\centering
\subfloat[Evolution of the mean ranks, for every $\alpha$, based on the \textit{AvgCost} metric. \centering Shaded areas correspond to 90\% bootstrap confidence intervals.]{\includegraphics[width=0.95\linewidth]{plots/complete_sota/ranks/tex/bump_sota.pdf}\label{bump_sota}} \\
\subfloat[Wilcoxon signed-rank test labeled with mean \textit{AvgCost}, $\alpha = 0.8$.]{\includegraphics[width=1\linewidth]{plots/complete_sota/wilcoxon/tex/cd_diag_0.8.pdf}\label{cd_sota}}
\caption{The ranking plot \protect\subref{bump_sota} shows that, across all $\alpha$, \textsc{Alert$^{\star}$} dominates all competitors. This result is significant as supported by statistical tests as shown in plot \protect\subref{cd_sota} for $\alpha = 0.8$.}
\label{fig:ranks_sota}
\end{figure}
Considering the minimization of misclassification and delay costs as two conflicting objectives, one can draw the Pareto front of each method: the set of points for which no other point dominates with respect to both objectives. Figure \ref{fig:pareto_sota} shows the result for $\alpha \in \{0, 0.1, 0.2, \ldots, 1.0\}$. What stands out first is the clear domination of \textsc{Alert}$^\star$. A closer examination reveals that \textsc{Alert}$^\star$ generally makes its decision later than its competitors, at the cost of higher delay costs, but that this extra cost is more than offset by lower misclassification costs. This is confirmed in Figure \ref{fig:scatter_rmse} with the marginals over the trigger moments.
\begin{figure}[!htb]
\centering
\includegraphics[width=0.9\linewidth]{plots/complete_sota/pareto/tex/pareto.pdf}
\caption{Pareto front, displaying for each $\alpha$, the normalized version of the \textit{AvgCost}, decomposed over delay and misclassification cost on $x$-axis and $y$-axis respectively. Best approaches are located on the top left corner. High $\alpha$ values are located on the right, low ones on the left. Due to the exponential shape of the delay cost, the $x$-axis is on log scale.}
\label{fig:pareto_sota}
\end{figure}
\begin{figure*}[!htb]
\centering
\subfloat[\textsc{Alert$^{\star}$}]{\includegraphics[width=0.3\linewidth]{plots/qualitative/scatter_rmse/tex/scatter_rmse_alert.pdf}\label{fig:scatter_alert}}
\subfloat[\textsc{Economy}]{\includegraphics[width=0.3\linewidth]{plots/qualitative/scatter_rmse/tex/scatter_rmse_economy.pdf}\label{fig:scatter_eco}}
\subfloat[\textsc{Calimera}]{\includegraphics[width=0.3\linewidth]{plots/qualitative/scatter_rmse/tex/scatter_rmse_calimera_innerplot.pdf}}\label{fig:scatter_cal}
\caption{The $x$-axis reports how far is the triggering time from the best a posteriori one: left is better. The $y$-axis reports the difference between the \textit{AvgCost} incurred by the algorithm compared to the best a posteriori one, \textit{AvgCost}$^{\star}$ : lower is better. The black crosses report the average performance for each value of $\alpha$ (greater values correspond to higher relative importance of the delay cost). Ellipses display $2 \times$ the standard deviation over both axis, computed for each $\alpha$ value.}
\label{fig:scatter_rmse}
\end{figure*}
Indeed, Figure \ref{fig:scatter_rmse} allows a finer examination of the behavior of ECTS algorithms.
A general conclusion is that the more difficult problems are associated with medium values of $\alpha$. When $\alpha \approx 0$, it is better to decide early without considering the misclassification cost, while for $\alpha \approx 1$, the decision time is entirely controlled by the estimation of the misclassification cost by the algorithm.
The comparison of the three graphs in Figure \ref{fig:scatter_rmse} reveals that: (\textit{i}) \textsc{Alert}$^\star$ brings lower values of \textit{AvgCost}, especially for intermediate values of $\alpha$, (\textit{ii}) \textsc{Alert}$^{\star}$ is more robust with respect to the variety of data sets, displaying smaller ellipses than its competitors and (\textit{iii}) an important lesson for ECTS systems is that it can be profitable not to take a decision at what is the a posteriori best triggering time! The latter may seem surprising, but due to data noise and general uncertainties, the best a posteriori policy may not be easy to learn, and a more conservative one may be more appropriate. \textsc{Alert}$^{\star}$ seems to be the method that best handles this.
\subsubsection{State space sensitivity {\bf (question \#2)}}
We compare \textsc{Alert$^{\star}$} to: (\textit{i}) \textsc{Alert}Raw, a simple state space that only includes class predictions from the classifier $h$, (\textit{ii}) \textsc{Alert}$^\star$\&series, the \textsc{Alert$^{\star}$} state space, enriched by the raw time series, subsampled so that the length of the resulting time series is 20 points, and (\textit{iii}) \textsc{Alert}$^\star$\&ran\-dom, the \textsc{Alert$^{\star}$} state space with 20 random points, drawn from a uniform distribution $\mathcal{U}(0, 1)$.
Figure \ref{fig:bump_ablation} shows that the original version of \textsc{Alert$^{\star}$} is still the best performing algorithm, even if not significantly better than \textsc{Alert}$^\star$\&series, except for $\alpha = 0.9$ (see Appendix \ref{wilco_state}). This demonstrates that the state space of \textsc{Alert$^{\star}$} is well chosen and that adding more unprocessed information is not useful for the method. Moreover, \textsc{Alert$^{\star}$} clearly outperforms \textsc{Alert}Raw. It is thus important to carefully craft the features to include in the state space. Finally, adding pure noise proves to be the worst of the tested \textsc{Alert$^{\star}$}'s variants, indicating that it is not the inclusion of more features that explains increases in performance, but that performance degrades when useless information is added.
\begin{figure}[!htb]
\centering
\includegraphics[width=0.95\linewidth]{plots/complete_sota/ranks/tex/bump_state.pdf}
\caption{Evolution of the mean ranks, for every $\alpha$, based on the \textit{AvgCost} metric. Sensitivity study over the \textsc{Alert$^{\star}$} state space.}
\label{fig:bump_ablation}
\end{figure}
\section{Conclusion}
\label{sec:conclusion}
ECTS has been recognized as an important problem with significant applications in many fields where decisions have to be made ``on the fly'' before all measurements are available. As a result, numerous ECTS methods have been proposed, based on different triggering functions, each taking into account various features related to the incoming time series and/or the response of the classifier. Although their performance have been empirically compared in several publications, no studies have been carried out on the optimality of these criteria. On the basis of the same features, could there be better criteria?
This paper presents a way to evaluate this by showing how to translate ECTS problems into RL ones using exactly the same features in the state space. It is then possible to compare the performance obtained by the ``man-tailored'' decision rules and their ``RL-based'' counterparts, all other things being equal. Using this methodology, it was found that the man-tailored rules performed well overall, especially when input space remains small.
Based on these findings and our methodology, we investigated whether, by taking into account a combination of the features used in several state-of-the-art systems involving man-made decision rules, RL could learn good triggering functions.
Experiments showed that the resulting system, called \textsc{Alert}$^\star$, significantly outperformed its state-of-the-art competitors for all weighted combinations of misclassification and delay costs, evaluated on $31$ public datasets.
This paper opens up a new avenue for tackling the ECTS problem by showing how to invent new triggering functions: by defining \textit{a priori} the features deemed important of the time series and the classifier, and using the generic RL method presented here to derive optimized criteria.
Compared to man-tailored triggering functions, the proposed RL-based approach improves performance at the expense of interpretability of the triggering decisions (see Figure \ref{fig:motivation_example}). Another line of research would therefore be to study the interpretability of the RL-based trigger function.
\bibliographystyle{ACM-Reference-Format}
\bibliography{biblio}
\appendix
\section*{Appendix}
\section{Additional experimental results}\label{add_expe}
\subsection{Pairwise comparison: other competitor}\label{app_pairwise}
\begin{figure}[!htp]
\centering
\subfloat[\textsc{Proba threshold}: $\mathcal{S} = \{\textit{max posterior} \}$]{\includegraphics[width=0.9\linewidth]{plots/pairwise_sota/tex/alert_pt.pdf}\label{pair_pt}}
\caption{Pairwise comparison of \textsc{Proba Threshold} vs. RL counterpart using same information as input. Points above the horizontal line indicates that the man-tailored method is better than its RL-based counterpart}
\label{fig:enter-label}
\end{figure}
\subsection{Pairwise comparison: statistical tests}\label{wilco_pair}
\begin{table}[!htb]
\centering
\caption{Wilcoxon tests p-values, comparing pairwise man-tailored and RL-based algorithms. \textbf{Bold}, resp. \textit{Italic} values indicate a value below the significance level equal to 0.05, in favor of man-tailored method, resp. RL-based method.}
\resizebox{\linewidth}{!}{
\begin{tabular}{cc|c|c|c|c|c|c|c|c|c|c|c|}
\cline{3-13}
& $\alpha$ $\rightarrow$ & \multirow{2}{*}{\textbf{0 }} & \multirow{2}{*}{\textbf{0.1}} & \multirow{2}{*}{\textbf{0.2}} & \multirow{2}{*}{\textbf{0.3}} & \multirow{2}{*}{\textbf{0.4}} & \multirow{2}{*}{\textbf{0.5}} & \multirow{2}{*}{\textbf{0.6}} & \multirow{2}{*}{\textbf{0.7}} & \multirow{2}{*}{\textbf{0.8}} & \multirow{2}{*}{\textbf{0.9}} & \multirow{2}{*}{\textbf{1}} \\
\textbf{method} $\downarrow$ & & & & & & & & & & & & \\
\hline
\multicolumn{2}{|c|}{\textsc{Economy}} & NaN & 0.062 & 0.091 & 0.082 & \cellcolor{lightgray}\textbf{0.003} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{0.001} & 0.421 \\
\multicolumn{2}{|c|}{\textsc{Stopping Rule}} & \textit{0.046} & 0.672 & 0.701 & 0.635 & 0.15 & \cellcolor{lightgray}\textbf{0.01} & 0.134 & 0.931 & 0.111 & 0.141 & 0.673 \\
\multicolumn{2}{|c|}{\textsc{Calimera}} & 0.317 & 0.464 & 0.522 & 0.644 & 0.799 & 0.961 & 0.604 & 0.474 & 0.161 & 0.069 & 0.086 \\
\multicolumn{2}{|c|}{\textsc{Proba threshold}} & 0.317 & 0.803 & 0.066 & 0.097 & \cellcolor{lightgray}\textbf{0.005} & 0.074 & 0.112 & \cellcolor{lightgray}\textbf{0.036} & 0.953 & \cellcolor{lightgray}\textbf{0.018} & 0.386 \\
\hline
\end{tabular}}
\label{tab:wilco_pair}
\end{table}
\newpage
\subsection{\textsc{Alert$^{\star}$} vs. SOTA: statistical tests}\label{wilco_alert}
\begin{table}[!htb]
\centering
\caption{Wilcoxon tests p-values, comparing \textsc{Alert$^{\star}$} to state-of-the-art algorithms. \textbf{Bold} values indicate a value below the significance level equal to 0.05, in favor of \textsc{Alert$^{\star}$}. \underline{Underline} values indicate p-values below original significance level, but not below the Holm's corrected value, that depends on the number of tested hypothesis.}
\resizebox{\linewidth}{!}{
\begin{tabular}{cc|c|c|c|c|c|c|c|c|c|c|c|}
\cline{3-13}
& $\alpha$ $\rightarrow$ & \multirow{2}{*}{\textbf{0 }} & \multirow{2}{*}{\textbf{0.1}} & \multirow{2}{*}{\textbf{0.2}} & \multirow{2}{*}{\textbf{0.3}} & \multirow{2}{*}{\textbf{0.4}} & \multirow{2}{*}{\textbf{0.5}} & \multirow{2}{*}{\textbf{0.6}} & \multirow{2}{*}{\textbf{0.7}} & \multirow{2}{*}{\textbf{0.8}} & \multirow{2}{*}{\textbf{0.9}} & \multirow{2}{*}{\textbf{1}} \\
\textbf{\textsc{Alert$^{\star}$} vs.} $\downarrow$ & & & & & & & & & & & & \\
\hline
\multicolumn{2}{|c|}{\textsc{Economy}} & 1.0 & 0.759 & \underline{0.034} & 0.083 & \underline{0.024} & \underline{0.049} & \underline{0.016} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{0.002} \\
\multicolumn{2}{|c|}{\textsc{Calimera}} & 1.0 & 0.58 & \underline{0.041} & 0.342 & \underline{0.006} & \underline{0.029} & \underline{0.009} & \cellcolor{lightgray}\textbf{0.004} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{0.002} & \cellcolor{lightgray}\textbf{0.001} \\
\multicolumn{2}{|c|}{\textsc{Stopping Rule}} & 0.208 & 0.129 & 0.086 & 0.41 & \cellcolor{lightgray}\textbf{0.015} & 0.05 & \cellcolor{lightgray}\textbf{0.002} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} \\
\multicolumn{2}{|c|}{\textsc{Proba threshold}} & 1.0 & 0.223 & 0.078 & \cellcolor{lightgray}\textbf{0.037} & \cellcolor{lightgray}\textbf{0.002} & \cellcolor{lightgray}\textbf{0.002} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} \\
\multicolumn{2}{|c|}{\textsc{Earliest}} & \cellcolor{lightgray}\textbf{<1e-3} & 0.347 & 0.176 & 0.327 & \cellcolor{lightgray}\textbf{0.014} & \cellcolor{lightgray}\textbf{0.006} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} \\
\hline
\end{tabular}}
\label{tab:wilco_alert}
\end{table}
\subsection{Scatter plots: other competitors}\label{app_scatter}
\begin{figure}[!htp]
\centering
\subfloat[\textsc{Stopping Rule}]{\includegraphics[width=0.5\linewidth]{plots/qualitative/scatter_rmse/tex/scatter_rmse_stopping_rule.pdf}}
\subfloat[\textsc{Proba threshold}]{\includegraphics[width=0.5\linewidth]{plots/qualitative/scatter_rmse/tex/scatter_rmse_proba_threshold.pdf}}
\caption{The $x$-axis reports how far is the triggering time from the best a posteriori one: left is better. The $y$-axis reports the difference between the \textit{AvgCost} incurred by the algorithm compared to the best a posteriori one, \textit{AvgCost}$^{\star}$ : lower is better. The black crosses report the average performance for each value of $\alpha$ (greater values correspond to higher relative importance of the delay cost). Ellipses display $2 \times$ the standard deviation over both axis, computed for each $\alpha$ value.}
\label{fig:enter-label}
\end{figure}
\subsection{Delayed rewards}\label{app_end}
The delayed reward function is tested here, i.e. giving fully paid cost once the trigger action has been chosen. Figure \ref{fig:bump_end} shows that even without having intermediate time rewards, \textsc{Alert$^{\star}$} still manages to outperform state-of-the-art algorithms. Thus, knowing how to decompose both misclassification and delay cost is not a strong requirement for \textsc{Alert$^{\star}$} to perform.
\begin{figure}[!htb]
\centering
\includegraphics[width=0.95\linewidth]{plots/complete_sota/ranks/tex/bump_reward_end.pdf}
\caption{Evolution of the mean ranks, for every $\alpha$, based on the \textit{AvgCost} metric.}
\label{fig:bump_end}
\end{figure}
\newpage
\subsection{State space study: statistical tests}\label{wilco_state}
\begin{table}[!htb]
\centering
\caption{Wilcoxon tests p-values, comparing \textsc{Alert$^{\star}$} to variants. \textbf{Bold} values indicate a value below the significance level equal to 0.05, in favor of base \textsc{Alert$^{\star}$}. \underline{Underline} values indicate p-values below original significance level, but not below the Holm's corrected value, that depends on the number of tested hypothesis.}
\resizebox{\linewidth}{!}{
\begin{tabular}{cc|c|c|c|c|c|c|c|c|c|c|c|}
\cline{3-13}
& $\alpha$ $\rightarrow$ & \multirow{2}{*}{\textbf{0 }} & \multirow{2}{*}{\textbf{0.1}} & \multirow{2}{*}{\textbf{0.2}} & \multirow{2}{*}{\textbf{0.3}} & \multirow{2}{*}{\textbf{0.4}} & \multirow{2}{*}{\textbf{0.5}} & \multirow{2}{*}{\textbf{0.6}} & \multirow{2}{*}{\textbf{0.7}} & \multirow{2}{*}{\textbf{0.8}} & \multirow{2}{*}{\textbf{0.9}} & \multirow{2}{*}{\textbf{1}} \\
\textbf{\textsc{Alert$^{\star}$} vs.} $\downarrow$ & & & & & & & & & & & & \\
\hline
\multicolumn{2}{|c|}{\textsc{Alert}Raw} & 1.0 & 0.330 & \cellcolor{lightgray}\textbf{0.005} & \cellcolor{lightgray}\textbf{0.018} & \cellcolor{lightgray}\textbf{0.003} & \cellcolor{lightgray}\textbf{0.008} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{0.002} \\
\multicolumn{2}{|c|}{\textsc{\textsc{Alert$^{\star}$}}\&series} & 0.505 & 0.419 & 0.922 & 0.286 & 0.883 & 0.977 & 0.524 & 0.124 & 0.098 & \cellcolor{lightgray}\textbf{0.015} & 0.079 \\
\multicolumn{2}{|c|}{\textsc{Alert$^{\star}$}\&random} & 1.0 & \underline{0.026} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} & \cellcolor{lightgray}\textbf{<1e-3} \\
\hline
\end{tabular}}
\label{tab:wilco_alert}
\end{table}
\end{document}
%%% -*-BibTeX-*-
%%% Do NOT edit. File created by BibTeX with style
%%% ACM-Reference-Format-Journals [18-Jan-2012].
\begin{thebibliography}{53}
%%% ====================================================================
%%% NOTE TO THE USER: you can override these defaults by providing
%%% customized versions of any of these macros before the \bibliography
%%% command. Each of them MUST provide its own final punctuation,
%%% except for \shownote{}, \showDOI{}, and \showURL{}. The latter two
%%% do not use final punctuation, in order to avoid confusing it with
%%% the Web address.
%%%
%%% To suppress output of a particular field, define its macro to expand
%%% to an empty string, or better, \unskip, like this:
%%%
%%% \newcommand{\showDOI}[1]{\unskip} % LaTeX syntax
%%%
%%% \def \showDOI #1{\unskip} % plain TeX syntax
%%%
%%% ====================================================================
\ifx \showCODEN \undefined \def \showCODEN #1{\unskip} \fi
\ifx \showDOI \undefined \def \showDOI #1{#1}\fi
\ifx \showISBNx \undefined \def \showISBNx #1{\unskip} \fi
\ifx \showISBNxiii \undefined \def \showISBNxiii #1{\unskip} \fi
\ifx \showISSN \undefined \def \showISSN #1{\unskip} \fi
\ifx \showLCCN \undefined \def \showLCCN #1{\unskip} \fi
\ifx \shownote \undefined \def \shownote #1{#1} \fi
\ifx \showarticletitle \undefined \def \showarticletitle #1{#1} \fi
\ifx \showURL \undefined \def \showURL {\relax} \fi
% The following commands are used for tagged output and should be
% invisible to TeX
\providecommand\bibfield[2]{#2}
\providecommand\bibinfo[2]{#2}
\providecommand\natexlab[1]{#1}
\providecommand\showeprint[2][]{arXiv:#2}
\bibitem[Achenchabe et~al\mbox{.}(2021)]%
{achenchabe2021early}
\bibfield{author}{\bibinfo{person}{Youssef Achenchabe}, \bibinfo{person}{Alexis Bondu}, \bibinfo{person}{Antoine Cornu{\'e}jols}, {and} \bibinfo{person}{Asma Dachraoui}.} \bibinfo{year}{2021}\natexlab{}.
\newblock \showarticletitle{Early classification of time series: Cost-based optimization criterion and algorithms}.
\newblock \bibinfo{journal}{\emph{Machine Learning}} \bibinfo{volume}{110}, \bibinfo{number}{6} (\bibinfo{year}{2021}), \bibinfo{pages}{1481--1504}.
\newblock
\bibitem[Bagnall et~al\mbox{.}(2017)]%
{bagnall2017great}
\bibfield{author}{\bibinfo{person}{Anthony Bagnall}, \bibinfo{person}{Jason Lines}, \bibinfo{person}{Aaron Bostrom}, \bibinfo{person}{James Large}, {and} \bibinfo{person}{Eamonn Keogh}.} \bibinfo{year}{2017}\natexlab{}.
\newblock \showarticletitle{The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances}.
\newblock \bibinfo{journal}{\emph{Data mining and knowledge discovery}} \bibinfo{volume}{31} (\bibinfo{year}{2017}), \bibinfo{pages}{606--660}.
\newblock
\bibitem[Ball et~al\mbox{.}(2023)]%
{ball2023efficient}
\bibfield{author}{\bibinfo{person}{Philip~J Ball}, \bibinfo{person}{Laura Smith}, \bibinfo{person}{Ilya Kostrikov}, {and} \bibinfo{person}{Sergey Levine}.} \bibinfo{year}{2023}\natexlab{}.
\newblock \showarticletitle{Efficient online reinforcement learning with offline data}. In \bibinfo{booktitle}{\emph{International Conference on Machine Learning}}. PMLR, \bibinfo{pages}{1577--1594}.
\newblock
\bibitem[Bilski and Jastrzebska(2023)]%
{bilski2023calimera}
\bibfield{author}{\bibinfo{person}{Jakub~Michal Bilski} {and} \bibinfo{person}{Agnieszka Jastrzebska}.} \bibinfo{year}{2023}\natexlab{}.
\newblock \showarticletitle{CALIMERA: A new early time series classification method}.
\newblock \bibinfo{journal}{\emph{Information Processing \& Management}} \bibinfo{volume}{60}, \bibinfo{number}{5} (\bibinfo{year}{2023}), \bibinfo{pages}{103465}.
\newblock
\bibitem[Bondu et~al\mbox{.}(2022)]%
{bondu2022open}
\bibfield{author}{\bibinfo{person}{Alexis Bondu}, \bibinfo{person}{Youssef Achenchabe}, \bibinfo{person}{Albert Bifet}, \bibinfo{person}{Fabrice Cl{\'e}rot}, \bibinfo{person}{Antoine Cornu{\'e}jols}, \bibinfo{person}{Joao Gama}, \bibinfo{person}{Georges H{\'e}brail}, \bibinfo{person}{Vincent Lemaire}, {and} \bibinfo{person}{Pierre-Fran{\c{c}}ois Marteau}.} \bibinfo{year}{2022}\natexlab{}.
\newblock \showarticletitle{Open challenges for machine learning based early decision-making research}.
\newblock \bibinfo{journal}{\emph{ACM SIGKDD Explorations Newsletter}} \bibinfo{volume}{24}, \bibinfo{number}{2} (\bibinfo{year}{2022}), \bibinfo{pages}{12--31}.
\newblock
\bibitem[Dachraoui et~al\mbox{.}(2015)]%
{dachraoui2015early}
\bibfield{author}{\bibinfo{person}{Asma Dachraoui}, \bibinfo{person}{Alexis Bondu}, {and} \bibinfo{person}{Antoine Cornu{\'e}jols}.} \bibinfo{year}{2015}\natexlab{}.
\newblock \showarticletitle{Early classification of time series as a non myopic sequential decision making problem}. In \bibinfo{booktitle}{\emph{Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2015, Porto, Portugal, September 7-11, 2015, Proceedings, Part I 15}}. Springer, \bibinfo{pages}{433--447}.
\newblock
\bibitem[Dau et~al\mbox{.}(2019)]%
{dau2019ucr}
\bibfield{author}{\bibinfo{person}{Hoang~Anh Dau}, \bibinfo{person}{Anthony Bagnall}, \bibinfo{person}{Kaveh Kamgar}, \bibinfo{person}{Chin-Chia~Michael Yeh}, \bibinfo{person}{Yan Zhu}, \bibinfo{person}{Shaghayegh Gharghabi}, \bibinfo{person}{Chotirat~Ann Ratanamahatana}, {and} \bibinfo{person}{Eamonn Keogh}.} \bibinfo{year}{2019}\natexlab{}.
\newblock \showarticletitle{The UCR time series archive}.
\newblock \bibinfo{journal}{\emph{IEEE/CAA Journal of Automatica Sinica}} \bibinfo{volume}{6}, \bibinfo{number}{6} (\bibinfo{year}{2019}), \bibinfo{pages}{1293--1305}.
\newblock
\bibitem[Dayan and Watkins(1992)]%
{dayan1992q}
\bibfield{author}{\bibinfo{person}{Peter Dayan} {and} \bibinfo{person}{CJCH Watkins}.} \bibinfo{year}{1992}\natexlab{}.
\newblock \showarticletitle{Q-learning}.
\newblock \bibinfo{journal}{\emph{Machine learning}} \bibinfo{volume}{8}, \bibinfo{number}{3} (\bibinfo{year}{1992}), \bibinfo{pages}{279--292}.
\newblock
\bibitem[Dempster et~al\mbox{.}(2021)]%
{dempster2021minirocket}
\bibfield{author}{\bibinfo{person}{Angus Dempster}, \bibinfo{person}{Daniel~F Schmidt}, {and} \bibinfo{person}{Geoffrey~I Webb}.} \bibinfo{year}{2021}\natexlab{}.
\newblock \showarticletitle{Minirocket: A very fast (almost) deterministic transform for time series classification}. In \bibinfo{booktitle}{\emph{Proceedings of the 27th ACM SIGKDD conference on knowledge discovery \& data mining}}. \bibinfo{pages}{248--257}.
\newblock
\bibitem[Ebihara et~al\mbox{.}(2020)]%
{ebihara2020sequential}
\bibfield{author}{\bibinfo{person}{Akinori~F Ebihara}, \bibinfo{person}{Taiki Miyagawa}, \bibinfo{person}{Kazuyuki Sakurai}, {and} \bibinfo{person}{Hitoshi Imaoka}.} \bibinfo{year}{2020}\natexlab{}.
\newblock \showarticletitle{Sequential density ratio estimation for simultaneous optimization of speed and accuracy}.
\newblock \bibinfo{journal}{\emph{arXiv preprint arXiv:2006.05587}} (\bibinfo{year}{2020}).
\newblock
\bibitem[Ebihara et~al\mbox{.}(2023)]%
{ebihara2023toward}
\bibfield{author}{\bibinfo{person}{Akinori~F Ebihara}, \bibinfo{person}{Taiki Miyagawa}, \bibinfo{person}{Kazuyuki Sakurai}, {and} \bibinfo{person}{Hitoshi Imaoka}.} \bibinfo{year}{2023}\natexlab{}.
\newblock \showarticletitle{Toward Asymptotic Optimality: Sequential Unsupervised Regression of Density Ratio for Early Classification}. In \bibinfo{booktitle}{\emph{ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}}. IEEE, \bibinfo{pages}{1--5}.
\newblock
\bibitem[Fujimoto and Gu(2021)]%
{fujimoto2021minimalist}
\bibfield{author}{\bibinfo{person}{Scott Fujimoto} {and} \bibinfo{person}{Shixiang~Shane Gu}.} \bibinfo{year}{2021}\natexlab{}.
\newblock \showarticletitle{A minimalist approach to offline reinforcement learning}.
\newblock \bibinfo{journal}{\emph{Advances in neural information processing systems}} \bibinfo{volume}{34} (\bibinfo{year}{2021}), \bibinfo{pages}{20132--20145}.
\newblock
\bibitem[Hartvigsen et~al\mbox{.}(2022)]%
{hartvigsen2022stop}
\bibfield{author}{\bibinfo{person}{Thomas Hartvigsen}, \bibinfo{person}{Walter Gerych}, \bibinfo{person}{Jidapa Thadajarassiri}, \bibinfo{person}{Xiangnan Kong}, {and} \bibinfo{person}{Elke Rundensteiner}.} \bibinfo{year}{2022}\natexlab{}.
\newblock \showarticletitle{Stop\&hop: Early classification of irregular time series}. In \bibinfo{booktitle}{\emph{Proceedings of the 31st ACM International Conference on Information \& Knowledge Management}}. \bibinfo{pages}{696--705}.
\newblock
\bibitem[Hartvigsen et~al\mbox{.}(2019)]%
{hartvigsen2019adaptive}
\bibfield{author}{\bibinfo{person}{Thomas Hartvigsen}, \bibinfo{person}{Cansu Sen}, \bibinfo{person}{Xiangnan Kong}, {and} \bibinfo{person}{Elke Rundensteiner}.} \bibinfo{year}{2019}\natexlab{}.
\newblock \showarticletitle{Adaptive-halting policy network for early classification}. In \bibinfo{booktitle}{\emph{Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining}}. \bibinfo{pages}{101--110}.
\newblock
\bibitem[Hartvigsen et~al\mbox{.}(2020)]%
{hartvigsen2020recurrent}
\bibfield{author}{\bibinfo{person}{Thomas Hartvigsen}, \bibinfo{person}{Cansu Sen}, \bibinfo{person}{Xiangnan Kong}, {and} \bibinfo{person}{Elke Rundensteiner}.} \bibinfo{year}{2020}\natexlab{}.
\newblock \showarticletitle{Recurrent halting chain for early multi-label classification}. In \bibinfo{booktitle}{\emph{Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining}}. \bibinfo{pages}{1382--1392}.
\newblock
\bibitem[Huang et~al\mbox{.}(2022a)]%
{huang2022snippet1}
\bibfield{author}{\bibinfo{person}{Yu Huang}, \bibinfo{person}{Gary~G Yen}, {and} \bibinfo{person}{Vincent~S Tseng}.} \bibinfo{year}{2022}\natexlab{a}.
\newblock \showarticletitle{Snippet policy network for multi-class varied-length ECG early classification}.
\newblock \bibinfo{journal}{\emph{IEEE Transactions on Knowledge and Data Engineering}} \bibinfo{volume}{35}, \bibinfo{number}{6} (\bibinfo{year}{2022}), \bibinfo{pages}{6349--6361}.
\newblock
\bibitem[Huang et~al\mbox{.}(2022b)]%
{huang2022snippet}
\bibfield{author}{\bibinfo{person}{Yu Huang}, \bibinfo{person}{Gary~G Yen}, {and} \bibinfo{person}{Vincent~S Tseng}.} \bibinfo{year}{2022}\natexlab{b}.
\newblock \showarticletitle{Snippet policy network v2: Knee-guided neuroevolution for multi-lead ecg early classification}.
\newblock \bibinfo{journal}{\emph{IEEE Transactions on Neural Networks and Learning Systems}} \bibinfo{volume}{35}, \bibinfo{number}{2} (\bibinfo{year}{2022}), \bibinfo{pages}{2167--2181}.
\newblock
\bibitem[Kingma and Ba(2015)]%
{kingma2015adam}
\bibfield{author}{\bibinfo{person}{Diederik~P Kingma} {and} \bibinfo{person}{Jimmy~Lei Ba}.} \bibinfo{year}{2015}\natexlab{}.
\newblock \showarticletitle{Adam: A method for stochastic gradient descent}. In \bibinfo{booktitle}{\emph{ICLR: international conference on learning representations}}. ICLR US., \bibinfo{pages}{1--15}.
\newblock
\bibitem[Klenke(2013)]%
{klenke2013}
\bibfield{author}{\bibinfo{person}{Achim Klenke}.} \bibinfo{year}{2013}\natexlab{}.
\newblock \bibinfo{booktitle}{\emph{Probability theory: a comprehensive course}}.
\newblock \bibinfo{publisher}{Springer Science \& Business Media}.
\newblock
\bibitem[Lei~Ba et~al\mbox{.}(2016)]%
{lei2016layer}
\bibfield{author}{\bibinfo{person}{Jimmy Lei~Ba}, \bibinfo{person}{Jamie~Ryan Kiros}, {and} \bibinfo{person}{Geoffrey~E Hinton}.} \bibinfo{year}{2016}\natexlab{}.
\newblock \showarticletitle{Layer normalization}.
\newblock \bibinfo{journal}{\emph{ArXiv e-prints}} (\bibinfo{year}{2016}), \bibinfo{pages}{arXiv--1607}.
\newblock
\bibitem[Levine et~al\mbox{.}(2020)]%
{levine2020offline}
\bibfield{author}{\bibinfo{person}{Sergey Levine}, \bibinfo{person}{Aviral Kumar}, \bibinfo{person}{George Tucker}, {and} \bibinfo{person}{Justin Fu}.} \bibinfo{year}{2020}\natexlab{}.
\newblock \showarticletitle{Offline reinforcement learning: Tutorial, review, and perspectives on open problems}.
\newblock \bibinfo{journal}{\emph{arXiv preprint arXiv:2005.01643}} (\bibinfo{year}{2020}).
\newblock
\bibitem[Lillicrap(2015)]%
{lillicrap2015continuous}
\bibfield{author}{\bibinfo{person}{TP Lillicrap}.} \bibinfo{year}{2015}\natexlab{}.
\newblock \showarticletitle{Continuous control with deep reinforcement learning}.
\newblock \bibinfo{journal}{\emph{arXiv preprint arXiv:1509.02971}} (\bibinfo{year}{2015}).
\newblock
\bibitem[Lv et~al\mbox{.}(2023)]%
{lv2023second}
\bibfield{author}{\bibinfo{person}{Junwei Lv}, \bibinfo{person}{Yuqi Chu}, \bibinfo{person}{Jun Hu}, \bibinfo{person}{Peipei Li}, {and} \bibinfo{person}{Xuegang Hu}.} \bibinfo{year}{2023}\natexlab{}.
\newblock \showarticletitle{Second-order Confidence Network for Early Classification of Time Series}.
\newblock \bibinfo{journal}{\emph{ACM Transactions on Intelligent Systems and Technology}} (\bibinfo{year}{2023}).
\newblock
\bibitem[Lv et~al\mbox{.}(2019)]%
{lv2019effective}
\bibfield{author}{\bibinfo{person}{Junwei Lv}, \bibinfo{person}{Xuegang Hu}, \bibinfo{person}{Lei Li}, {and} \bibinfo{person}{Peipei Li}.} \bibinfo{year}{2019}\natexlab{}.
\newblock \showarticletitle{An effective confidence-based early classification of time series}.
\newblock \bibinfo{journal}{\emph{IEEE Access}} \bibinfo{volume}{7} (\bibinfo{year}{2019}), \bibinfo{pages}{96113--96124}.
\newblock
\bibitem[Ma et~al\mbox{.}(2020)]%
{ma2020}
\bibfield{author}{\bibinfo{person}{Yifang Ma}, \bibinfo{person}{Zhenyu Wang}, \bibinfo{person}{Hong Yang}, {and} \bibinfo{person}{Lin Yang}.} \bibinfo{year}{2020}\natexlab{}.
\newblock \showarticletitle{Artificial intelligence applications in the development of autonomous vehicles: a survey}.
\newblock \bibinfo{journal}{\emph{IEEE/CAA Journal of Automatica Sinica}} \bibinfo{volume}{7}, \bibinfo{number}{2} (\bibinfo{year}{2020}), \bibinfo{pages}{315--329}.
\newblock
\bibitem[Martinez et~al\mbox{.}(2018)]%
{martinez2018deep}
\bibfield{author}{\bibinfo{person}{Coralie Martinez}, \bibinfo{person}{Guillaume Perrin}, \bibinfo{person}{Emmanuel Ramasso}, {and} \bibinfo{person}{Mich{\`e}le Rombaut}.} \bibinfo{year}{2018}\natexlab{}.
\newblock \showarticletitle{A deep reinforcement learning approach for early classification of time series}. In \bibinfo{booktitle}{\emph{2018 26th European Signal Processing Conference (EUSIPCO)}}. IEEE, \bibinfo{pages}{2030--2034}.
\newblock
\bibitem[Martinez et~al\mbox{.}(2020)]%
{martinez2020adaptive}
\bibfield{author}{\bibinfo{person}{Coralie Martinez}, \bibinfo{person}{Emmanuel Ramasso}, \bibinfo{person}{Guillaume Perrin}, {and} \bibinfo{person}{Mich{\`e}le Rombaut}.} \bibinfo{year}{2020}\natexlab{}.
\newblock \showarticletitle{Adaptive early classification of temporal sequences using deep reinforcement learning}.
\newblock \bibinfo{journal}{\emph{Knowledge-Based Systems}} \bibinfo{volume}{190} (\bibinfo{year}{2020}), \bibinfo{pages}{105290}.
\newblock
\bibitem[Mnih et~al\mbox{.}(2015)]%
{mnih2015human}
\bibfield{author}{\bibinfo{person}{Volodymyr Mnih}, \bibinfo{person}{Koray Kavukcuoglu}, \bibinfo{person}{David Silver}, \bibinfo{person}{Andrei~A Rusu}, \bibinfo{person}{Joel Veness}, \bibinfo{person}{Marc~G Bellemare}, \bibinfo{person}{Alex Graves}, \bibinfo{person}{Martin Riedmiller}, \bibinfo{person}{Andreas~K Fidjeland}, \bibinfo{person}{Georg Ostrovski}, {et~al\mbox{.}}} \bibinfo{year}{2015}\natexlab{}.
\newblock \showarticletitle{Human-level control through deep reinforcement learning}.
\newblock \bibinfo{journal}{\emph{nature}} \bibinfo{volume}{518}, \bibinfo{number}{7540} (\bibinfo{year}{2015}), \bibinfo{pages}{529--533}.
\newblock
\bibitem[Mori et~al\mbox{.}(2017)]%
{mori2017early}
\bibfield{author}{\bibinfo{person}{Usue Mori}, \bibinfo{person}{Alexander Mendiburu}, \bibinfo{person}{Sanjoy Dasgupta}, {and} \bibinfo{person}{Jose~A Lozano}.} \bibinfo{year}{2017}\natexlab{}.
\newblock \showarticletitle{Early classification of time series by simultaneously optimizing the accuracy and earliness}.
\newblock \bibinfo{journal}{\emph{IEEE transactions on neural networks and learning systems}} \bibinfo{volume}{29}, \bibinfo{number}{10} (\bibinfo{year}{2017}), \bibinfo{pages}{4569--4578}.
\newblock
\bibitem[Mori et~al\mbox{.}(2019)]%
{mori2019early}
\bibfield{author}{\bibinfo{person}{Usue Mori}, \bibinfo{person}{Alexander Mendiburu}, \bibinfo{person}{Isabel~Marta Miranda}, {and} \bibinfo{person}{Jos{\'e}~Antonio Lozano}.} \bibinfo{year}{2019}\natexlab{}.
\newblock \showarticletitle{Early classification of time series using multi-objective optimization techniques}.
\newblock \bibinfo{journal}{\emph{Information Sciences}} \bibinfo{volume}{492} (\bibinfo{year}{2019}), \bibinfo{pages}{204--218}.
\newblock
\bibitem[Nie et~al\mbox{.}(2022)]%
{nie2022data}
\bibfield{author}{\bibinfo{person}{Allen Nie}, \bibinfo{person}{Yannis Flet-Berliac}, \bibinfo{person}{Deon Jordan}, \bibinfo{person}{William Steenbergen}, {and} \bibinfo{person}{Emma Brunskill}.} \bibinfo{year}{2022}\natexlab{}.
\newblock \showarticletitle{Data-efficient pipeline for offline reinforcement learning with limited data}.
\newblock \bibinfo{journal}{\emph{Advances in Neural Information Processing Systems}} \bibinfo{volume}{35} (\bibinfo{year}{2022}), \bibinfo{pages}{14810--14823}.
\newblock
\bibitem[Paszke et~al\mbox{.}(2019)]%
{paszke2019pytorch}
\bibfield{author}{\bibinfo{person}{Adam Paszke}, \bibinfo{person}{Sam Gross}, \bibinfo{person}{Francisco Massa}, \bibinfo{person}{Adam Lerer}, \bibinfo{person}{James Bradbury}, \bibinfo{person}{Gregory Chanan}, \bibinfo{person}{Trevor Killeen}, \bibinfo{person}{Zeming Lin}, \bibinfo{person}{Natalia Gimelshein}, \bibinfo{person}{Luca Antiga}, {et~al\mbox{.}}} \bibinfo{year}{2019}\natexlab{}.
\newblock \showarticletitle{Pytorch: An imperative style, high-performance deep learning library}.
\newblock \bibinfo{journal}{\emph{Advances in neural information processing systems}} \bibinfo{volume}{32} (\bibinfo{year}{2019}).
\newblock
\bibitem[Prudencio et~al\mbox{.}(2023)]%
{prudencio2023survey}
\bibfield{author}{\bibinfo{person}{Rafael~Figueiredo Prudencio}, \bibinfo{person}{Marcos~ROA Maximo}, {and} \bibinfo{person}{Esther~Luna Colombini}.} \bibinfo{year}{2023}\natexlab{}.
\newblock \showarticletitle{A survey on offline reinforcement learning: Taxonomy, review, and open problems}.
\newblock \bibinfo{journal}{\emph{IEEE Transactions on Neural Networks and Learning Systems}} (\bibinfo{year}{2023}).
\newblock
\bibitem[Ran et~al\mbox{.}(2019)]%
{ran2019survey}
\bibfield{author}{\bibinfo{person}{Yongyi Ran}, \bibinfo{person}{Xin Zhou}, \bibinfo{person}{Pengfeng Lin}, \bibinfo{person}{Yonggang Wen}, {and} \bibinfo{person}{Ruilong Deng}.} \bibinfo{year}{2019}\natexlab{}.
\newblock \showarticletitle{A survey of predictive maintenance: Systems, purposes and approaches}.
\newblock \bibinfo{journal}{\emph{arXiv preprint arXiv:1912.07383}} (\bibinfo{year}{2019}).
\newblock
\bibitem[Renault et~al\mbox{.}(2024a)]%
{renault2024ml_edm}
\bibfield{author}{\bibinfo{person}{Aur{\'e}lien Renault}, \bibinfo{person}{Youssef Achenchabe}, \bibinfo{person}{{\'E}douard Bertrand}, \bibinfo{person}{Alexis Bondu}, \bibinfo{person}{Antoine Cornu{\'e}jols}, \bibinfo{person}{Vincent Lemaire}, {and} \bibinfo{person}{Asma Dachraoui}.} \bibinfo{year}{2024}\natexlab{a}.
\newblock \showarticletitle{ml\_edm package: a Python toolkit for Machine Learning based Early Decision Making}.
\newblock \bibinfo{journal}{\emph{arXiv e-prints}} (\bibinfo{year}{2024}), \bibinfo{pages}{arXiv--2408}.
\newblock
\bibitem[Renault et~al\mbox{.}(2024b)]%
{renault2024early}
\bibfield{author}{\bibinfo{person}{Aur{\'e}lien Renault}, \bibinfo{person}{Alexis Bondu}, \bibinfo{person}{Antoine Cornu{\'e}jols}, {and} \bibinfo{person}{Vincent Lemaire}.} \bibinfo{year}{2024}\natexlab{b}.
\newblock \showarticletitle{Early Classification of Time Series: Taxonomy and Benchmark}.
\newblock \bibinfo{journal}{\emph{arXiv preprint arXiv:2406.18332}} (\bibinfo{year}{2024}).
\newblock
\bibitem[Ringel et~al\mbox{.}(2024)]%
{ringelearly}
\bibfield{author}{\bibinfo{person}{Liran Ringel}, \bibinfo{person}{Regev Cohen}, \bibinfo{person}{Daniel Freedman}, \bibinfo{person}{Michael Elad}, {and} \bibinfo{person}{Yaniv Romano}.} \bibinfo{year}{2024}\natexlab{}.
\newblock \showarticletitle{Early Time Classification with Accumulated Accuracy Gap Control}. In \bibinfo{booktitle}{\emph{Forty-first International Conference on Machine Learning}}.
\newblock
\bibitem[Ruff et~al\mbox{.}(2021)]%
{Ruff2021}
\bibfield{author}{\bibinfo{person}{Lukas Ruff}, \bibinfo{person}{Jacob~R. Kauffmann}, \bibinfo{person}{Robert~A. Vandermeulen}, \bibinfo{person}{Gr{\'{e}}goire Montavon}, \bibinfo{person}{Wojciech Samek}, \bibinfo{person}{Marius Kloft}, \bibinfo{person}{Thomas~G. Dietterich}, {and} \bibinfo{person}{Klaus{-}Robert M{\"{u}}ller}.} \bibinfo{year}{2021}\natexlab{}.
\newblock \showarticletitle{A Unifying Review of Deep and Shallow Anomaly Detection}.
\newblock \bibinfo{journal}{\emph{Proc. {IEEE}}} \bibinfo{volume}{109}, \bibinfo{number}{5} (\bibinfo{year}{2021}), \bibinfo{pages}{756--795}.
\newblock
\urldef\tempurl%
\url{https://doi.org/10.1109/JPROC.2021.3052449}
\showDOI{\tempurl}
\bibitem[Ruiz et~al\mbox{.}(2021)]%
{ruiz2021great}
\bibfield{author}{\bibinfo{person}{Alejandro~Pasos Ruiz}, \bibinfo{person}{Michael Flynn}, \bibinfo{person}{James Large}, \bibinfo{person}{Matthew Middlehurst}, {and} \bibinfo{person}{Anthony Bagnall}.} \bibinfo{year}{2021}\natexlab{}.
\newblock \showarticletitle{The great multivariate time series classification bake off: a review and experimental evaluation of recent algorithmic advances}.
\newblock \bibinfo{journal}{\emph{Data Mining and Knowledge Discovery}} \bibinfo{volume}{35}, \bibinfo{number}{2} (\bibinfo{year}{2021}), \bibinfo{pages}{401--449}.
\newblock
\bibitem[Ru{\ss}wurm et~al\mbox{.}(2023)]%
{russwurm2023end}
\bibfield{author}{\bibinfo{person}{Marc Ru{\ss}wurm}, \bibinfo{person}{Nicolas Courty}, \bibinfo{person}{R{\'e}mi Emonet}, \bibinfo{person}{S{\'e}bastien Lef{\`e}vre}, \bibinfo{person}{Devis Tuia}, {and} \bibinfo{person}{Romain Tavenard}.} \bibinfo{year}{2023}\natexlab{}.
\newblock \showarticletitle{End-to-end learned early classification of time series for in-season crop type mapping}.
\newblock \bibinfo{journal}{\emph{ISPRS Journal of Photogrammetry and Remote Sensing}} \bibinfo{volume}{196} (\bibinfo{year}{2023}), \bibinfo{pages}{445--456}.
\newblock
\bibitem[Sch{\"a}fer and Leser(2020)]%
{schafer2020teaser}
\bibfield{author}{\bibinfo{person}{Patrick Sch{\"a}fer} {and} \bibinfo{person}{Ulf Leser}.} \bibinfo{year}{2020}\natexlab{}.
\newblock \showarticletitle{TEASER: early and accurate time series classification}.
\newblock \bibinfo{journal}{\emph{Data mining and knowledge discovery}} \bibinfo{volume}{34}, \bibinfo{number}{5} (\bibinfo{year}{2020}), \bibinfo{pages}{1336--1362}.
\newblock
\bibitem[Sutton and Barto(2018)]%
{sutton2018reinforcement}
\bibfield{author}{\bibinfo{person}{Richard~S Sutton} {and} \bibinfo{person}{Andrew~G Barto}.} \bibinfo{year}{2018}\natexlab{}.
\newblock \bibinfo{booktitle}{\emph{Reinforcement learning: An introduction}}.
\newblock \bibinfo{publisher}{MIT press}.
\newblock
\bibitem[Tan et~al\mbox{.}(2021)]%
{tan2021time}
\bibfield{author}{\bibinfo{person}{Chang~Wei Tan}, \bibinfo{person}{Christoph Bergmeir}, \bibinfo{person}{Fran{\c{c}}ois Petitjean}, {and} \bibinfo{person}{Geoffrey~I Webb}.} \bibinfo{year}{2021}\natexlab{}.
\newblock \showarticletitle{Time series extrinsic regression: Predicting numeric values from time series data}.
\newblock \bibinfo{journal}{\emph{Data Mining and Knowledge Discovery}} \bibinfo{volume}{35}, \bibinfo{number}{3} (\bibinfo{year}{2021}), \bibinfo{pages}{1032--1060}.
\newblock
\bibitem[Tarasov et~al\mbox{.}(2024)]%
{tarasov2024revisiting}
\bibfield{author}{\bibinfo{person}{Denis Tarasov}, \bibinfo{person}{Vladislav Kurenkov}, \bibinfo{person}{Alexander Nikulin}, {and} \bibinfo{person}{Sergey Kolesnikov}.} \bibinfo{year}{2024}\natexlab{}.
\newblock \showarticletitle{Revisiting the minimalist approach to offline reinforcement learning}.
\newblock \bibinfo{journal}{\emph{Advances in Neural Information Processing Systems}} \bibinfo{volume}{36} (\bibinfo{year}{2024}).
\newblock
\bibitem[Tavenard and Malinowski(2016)]%
{tavenard2016cost}
\bibfield{author}{\bibinfo{person}{Romain Tavenard} {and} \bibinfo{person}{Simon Malinowski}.} \bibinfo{year}{2016}\natexlab{}.
\newblock \showarticletitle{Cost-aware early classification of time series}. In \bibinfo{booktitle}{\emph{Machine Learning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2016, Riva del Garda, Italy, September 19-23, 2016, Proceedings, Part I 16}}. Springer, \bibinfo{pages}{632--647}.
\newblock
\bibitem[Van~Hasselt et~al\mbox{.}(2016)]%
{van2016deep}
\bibfield{author}{\bibinfo{person}{Hado Van~Hasselt}, \bibinfo{person}{Arthur Guez}, {and} \bibinfo{person}{David Silver}.} \bibinfo{year}{2016}\natexlab{}.
\newblock \showarticletitle{Deep reinforcement learning with double q-learning}. In \bibinfo{booktitle}{\emph{Proceedings of the AAAI conference on artificial intelligence}}, Vol.~\bibinfo{volume}{30}.
\newblock
\bibitem[Wald and Wolfowitz(1948)]%
{wald1948optimum}
\bibfield{author}{\bibinfo{person}{Abraham Wald} {and} \bibinfo{person}{Jacob Wolfowitz}.} \bibinfo{year}{1948}\natexlab{}.
\newblock \showarticletitle{Optimum character of the sequential probability ratio test}.
\newblock \bibinfo{journal}{\emph{The Annals of Mathematical Statistics}} (\bibinfo{year}{1948}), \bibinfo{pages}{326--339}.
\newblock
\bibitem[Wang et~al\mbox{.}(2024)]%
{wang2024deep}
\bibfield{author}{\bibinfo{person}{Yifan Wang}, \bibinfo{person}{Qining Zhang}, \bibinfo{person}{Lei Ying}, {and} \bibinfo{person}{Chuan Zhou}.} \bibinfo{year}{2024}\natexlab{}.
\newblock \showarticletitle{Deep Reinforcement Learning for Early Diagnosis of Lung Cancer}. In \bibinfo{booktitle}{\emph{Proceedings of the AAAI Conference on Artificial Intelligence}}, Vol.~\bibinfo{volume}{38}. \bibinfo{pages}{22410--22419}.
\newblock
\bibitem[Wu et~al\mbox{.}(2021)]%
{wu2021early}
\bibfield{author}{\bibinfo{person}{Renjie Wu}, \bibinfo{person}{Audrey Der}, {and} \bibinfo{person}{Eamonn Keogh}.} \bibinfo{year}{2021}\natexlab{}.
\newblock \showarticletitle{When is early classification of time series meaningful}.
\newblock \bibinfo{journal}{\emph{IEEE Transactions on Knowledge and Data Engineering}} (\bibinfo{year}{2021}).
\newblock
\bibitem[Xing et~al\mbox{.}(2009)]%
{xing2009early}
\bibfield{author}{\bibinfo{person}{Zhengzheng Xing}, \bibinfo{person}{Jian Pei}, {and} \bibinfo{person}{S~Yu Philip}.} \bibinfo{year}{2009}\natexlab{}.
\newblock \showarticletitle{Early Prediction on Time Series: A Nearest Neighbor Approach.}. In \bibinfo{booktitle}{\emph{IJCAI}}. Citeseer, \bibinfo{pages}{1297--1302}.
\newblock
\bibitem[Xing et~al\mbox{.}(2012)]%
{xing2012early}
\bibfield{author}{\bibinfo{person}{Zhengzheng Xing}, \bibinfo{person}{Jian Pei}, {and} \bibinfo{person}{Philip~S Yu}.} \bibinfo{year}{2012}\natexlab{}.
\newblock \showarticletitle{Early classification on time series}.
\newblock \bibinfo{journal}{\emph{Knowledge and information systems}} \bibinfo{volume}{31} (\bibinfo{year}{2012}), \bibinfo{pages}{105--127}.
\newblock
\bibitem[Xing et~al\mbox{.}(2011)]%
{xing2011extracting}
\bibfield{author}{\bibinfo{person}{Zhengzheng Xing}, \bibinfo{person}{Jian Pei}, \bibinfo{person}{Philip~S Yu}, {and} \bibinfo{person}{Ke Wang}.} \bibinfo{year}{2011}\natexlab{}.
\newblock \showarticletitle{Extracting interpretable features for early classification on time series}. In \bibinfo{booktitle}{\emph{Proceedings of the 2011 SIAM international conference on data mining}}. SIAM, \bibinfo{pages}{247--258}.
\newblock
\bibitem[Zafar et~al\mbox{.}(2021)]%
{zafar2021early}
\bibfield{author}{\bibinfo{person}{Paul-Emile Zafar}, \bibinfo{person}{Youssef Achenchabe}, \bibinfo{person}{Alexis Bondu}, \bibinfo{person}{Antoine Cornu{\'e}jols}, {and} \bibinfo{person}{Vincent Lemaire}.} \bibinfo{year}{2021}\natexlab{}.
\newblock \showarticletitle{Early classification of time series: Cost-based multiclass algorithms}. In \bibinfo{booktitle}{\emph{2021 IEEE 8th International Conference on Data Science and Advanced Analytics (DSAA)}}. IEEE, \bibinfo{pages}{1--10}.
\newblock
\end{thebibliography}