Large Neural Network Based Detection of Apnea, Bradycardia and Desaturation Events
Extracted source text · arXiv 1711.06484
\documentclass{article}
% if you need to pass options to natbib, use, e.g.:
% \PassOptionsToPackage{numbers, compress}{natbib}
% before loading nips_2017
%
% to avoid loading the natbib package, add option nonatbib:
% \usepackage[nonatbib]{nips_2017}
\usepackage[final,nonatbib]{nips_2017}
% to compile a camera-ready version, add the [final] option, e.g.:
% \usepackage[final]{nips_2017}
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc} % use 8-bit T1 fonts
\usepackage{hyperref} % hyperlinks
\usepackage{url} % simple URL typesetting
\usepackage{booktabs} % professional-quality tables
\usepackage{amsfonts} % blackboard math symbols
\usepackage{nicefrac} % compact symbols for 1/2, etc.
\usepackage{microtype} % microtypography
%ADDITIONAL PACKAGE
\usepackage{amsmath,amssymb,mathtools}
\usepackage{color}
\usepackage{enumitem}
\usepackage{subcaption}
%\def\thetitle{Detection of Apnea, Bradycardia and Desaturation Events with Unbalanced Data}
\def\thetitle{Large Neural Network Based Detection of \\ Apnea, Bradycardia and Desaturation Events}
\title{\thetitle}
% The \author macro works with any number of authors. There are two
% commands used to separate the names and addresses of multiple
% authors: \And and \AND.
%
% Using \And between authors leaves it to LaTeX to determine where to
% break the lines. Using \AND forces a line break at that point. So,
% if LaTeX puts 3 of 4 authors names on the first line, and the last
% on the second line, try using \AND instead of \And before the third
% author name.
% \author{\name Antoine Honore, Saikat Chatterjee \email \{honore, sach\}@kth.se \\
% \addr Department of Information Science and Engineering\\
% KTH, Royal Institute of Technology, Stockholm, Sweden
%\AND
%\name Eric Herlenius \email eric.herlenius@ki.se \\
%\addr Department of Women's and Children's Health \\
% Karolinska Institutet, Stockholm, Sweden}
%\author{
% Antoine Honoré\thanks{Dept. of Information Science and Engineering, Royal Institute of Technology, Stockholm, Sweden}\\
% \texttt{honore@kth.se}
% \And
% Veronica Siljehav\thanks{Dept. of Women's and Children's Health, Karolinska Institutet, Stockholm, Sweden}\\
% \texttt{veronica.siljehav@ki.se}
% \And
% Saikat Chatterjee\footnotemark[1]\\
% \texttt{sach@kth.se}
% \And
% Eric Herlenius\footnotemark[2]\\
% \texttt{eric.herlenius@ki.se}
%}
%Dept. of Women's and Children's Health, Karolinska Institutet, Stockholm, Sweden
\def\ki{Dept. Women’s and Children’s Health, Karolinska Institutet, and Pediatric Emergency, Karolinska University Hospital, SE-171 76 Solna, Sweden.
}
\def\kth{Dept. Information Science and Engineering, KTH Royal Institute of Technology, Stockholm, Sweden.}
\author{
Antoine Honoré\thanks{\kth}~~\thanks{\ki}%\footnotemark[2],
\hspace {1cm} Veronica Siljehav\footnotemark[2] \hspace {1cm}
Saikat Chatterjee\footnotemark[1] \hspace {1cm}
Eric Herlenius\footnotemark[2]
\\
\texttt{honore@kth.se} \hspace {0.2cm} \texttt{veronica.siljehav@ki.se} \, \hspace {0.2cm} \texttt{sach@kth.se} \, \hspace {0.2cm} \texttt{eric.herlenius@ki.se}
}
%\author{
% David S.~Hippocampus\thanks{Use footnote for providing further
% information about author (webpage, alternative
% address)---\emph{not} for acknowledging funding agencies.} \\
% Department of Computer Science\\
% Cranberry-Lemon University\\
% Pittsburgh, PA 15213 \\
% \texttt{hippo@cs.cranberry-lemon.edu} \\
%% examples of more authors
%% \And
%% Coauthor \\
%% Affiliation \\
%% Address \\
%% \texttt{email} \\
%% \AND
%% Coauthor \\
%% Affiliation \\
%% Address \\
%% \texttt{email} \\
%% \And
%% Coauthor \\
%% Affiliation \\
%% Address \\
%% \texttt{email} \\
%% \And
%% Coauthor \\
%% Affiliation \\
%% Address \\
%% \texttt{email} \\
\begin{document}
% \nipsfinalcopy is no longer used
\maketitle
\vspace{-0.5cm}
\begin{abstract}
Apnea, bradycardia and desaturation (ABD) events often precede life-threatening events including sepsis in newborn babies. Here, we explore machine learning for detection of ABD events as a binary classification problem. We investigate the use of a large neural network to achieve a good detection performance. To be user friendly, the chosen neural network does not require a high level of parameter tuning. Furthermore, a limited amount of training data is available and the training dataset is unbalanced. Comparing with two widely used state-of-the-art machine learning algorithms, the large neural network is found to be efficient. Even with a limited and unbalanced training data, the large neural network provides a detection performance level that is feasible to use in clinical care.
\end{abstract}
\vspace{-0.5cm}
\section{Introduction}
\vspace{-0.1cm}
Cardio-respiratory dysfunction and sepsis are major causes of morbidity and mortality in the neonatal population \cite{Liu2012}. Heart rate characteristic (HRC) monitoring system helps to reduce mortality of preterm babies \cite{Fairchild2013} and such systems have some success in detecting irregularities before they are clinically prominent \cite{moorman2011mortality,Fairchild_2010,lake2014complex,Sullivan2015,Moss2016}.
Apnea – breathing pauses and hypoventilation; Bradycardia – heart frequency reduction and Desaturation (ABD) events occur in daily life especially in preterm infants \cite{Hofstetter2008}. However, increasing ABD events also precede and may predict infection and life threatening events \cite{Siljehav2015}. Clinical symptoms of ABD events are: (a) apnea – breathing pauses and irregular breathing amplitudes; (b) bradycardia – heart frequency goes lower than usual levels; (c) desaturation – blood oxygen saturation is below normal levels; please see \cite{Lee2012} for more details. Depending on patient health and age, ABD events may occur in varying frequency. A high false negative rate in detecting ABD events is detrimental to patient care and a high false positive rate results in exhausting medical resources. Accurate detection of ABD events is a difficult task. Naturally we are interested to use advanced machine learning for the detection task. Use of advanced machine learning methods by medical practitioners is often challenging. Many machine learning methods, like deep neural networks \cite{Hinton_DeepLearning_NaturePaper_2015,Hinton_Deng_2012} typically require a large amount of training data. However, even with a high level of collaboration with hospitals, we so far are unable to access a large amount of training data for ABD event detection task. Further, advanced machine learning methods often require technical knowledge on tuning their parameters for achieving a good performance. Medical practitioners typically prefer smart engineering tools without high involvement in technical parts.
%Tuning parameters of neural network may not be expected from them.
Recently a large neural network is proposed in \cite{Chatterjee2017}. This is called progressive learning network (PLN) where the network structure is determined in an autonomous way. The PLN learns its architecture in a data driven manner. At the time of training, the PLN automatically learns the required number of layers in its architecture and the number of nodes in its architecture. A learned PLN is expected to be large in the sense of its wideness and deepness. Training of the PLN is user friendly as it does not require pain-stalking involvements for tuning its parameters. Software codes are also available online. In \cite{Chatterjee2017} it was experimentally shown that PLN can provide good pattern recognition accuracy for several object recognition tasks, mainly image based objects. Here, we explore PLN for ABD event detection. An important point is that PLN was not shown to be tested on a limited training data that we explore in the present study. Further, the training dataset we have access to is unbalanced. Here unbalanced means that the training dataset has a very limited amount of labeled data with true ABD events compared to the amount of labeled data with no ABD events. PLN was not shown to be tested on unbalanced dataset. Hence, from a machine learning perspective, it is interesting to see how the PLN performs for limited as well as unbalanced dataset. We expect that PLN will provide a good performance for such data as the PLN learning process is highly regularized. At this point, we found in the literature one ABD event detection algorithm \cite{Lee2012} based on classical signal processing methods (filtering and thresholding). In practice, a thresholding based technique requires a high level of tuning, even at individual patient level. We are more interested in smart machine learning methods in this study, and refrain to pursue the existing thresholding based method.
In this article, along with PLN we perform a comparative study with other machine learning algorithms. For this study, we investigate support-vector-machine (SVM) \cite{SVM_1999} and cross-entropy minimization based artificial neural network (ANN) \cite{bishop2006}. The choice of the two algorithms -- SVM and ANN -- is motivated by their popularity and success in various machine learning applications. Our study was performed in accordance with governmental ethical guidelines. The regional ethics committees of the hospital and municipality approved the research study.
%\vspace{cm}%\footnote{We do not mention the hospital name as the manuscript goes through a double blind review process.}
\section{Cohort}
\vspace{-0.2cm}
The clinical records of 13 eligible pre-mature infants were harvested at the neonatal intensive care units (ICU), Karolinska hospital, Stockholm, Sweden. The age of the babies at discharge was $ 28.57 \pm 23.20$ days and the total amount of acquired signals was 371.35 days.
The database is called Clinisoft\textsuperscript{\textregistered} and contains patient data collected at the ICU including diagnosis codes and notes, typically manually entered by clinicians. The signals we had access to are the heart frequency signal, taken over a minute, and the oxygen saturation level in the blood, averaged from a one minute time frame of a 100 Hz signal. Therefore both are sampled at the frequency of $\frac{1}{60}Hz$. For clarity and mathematical development later in the article, we denote the heart frequency signal by $x_1$ and the oxygen saturation level signal by $x_2$. Like many other natural signals -- such as speech, audio, image, electro-cardiograms, etc. -- we assume that the $x_1$ and $x_2$ signals are non-stationary. That means statistical property of a signal varies with respect to time. To handle the non-stationary aspect, we use frame-by-frame approach which is standard in speech, audio and image processing. We assume that an $l$-sample frame (or window) is statistically stationary, where the value of $l$ is not large. To collect feature vectors, we use a window shift by one sample. That means an $l$-sample window is moved across each of signals $x_1$ and $x_2$ by one sample in time-synchronization. This one-sample-shift helps to generate a large number of feature vectors from the available amount of signals. For a windowed portion of the signals, we collect all samples of the two signals $x_1$ and $x_2$, and construct a feature vector. Let us denote a feature vector by $\mathbf{x}$. The feature vector $\mathbf{x}$ comprises of $l$ consecutive samples of $x_1$ and $x_2$. That means $\mathbf{x}$ is of $2l$ dimension. We formally write $\mathbf{x} \in \mathbb{R}^D$ where $D=2l$. We use $l=15$. That means, we use 30-dimensional feature vector $\mathbf{x}$ in our study. In total we have $672 + 533866 = 534538$ feature vector instances, with following class labels.
\begin{enumerate}[noitemsep,nolistsep]
\item An ABD event corresponds to the class $\mathcal{C}_1$; we have 672 labeled instances.
\item No ABD event corresponds to the class $\mathcal{C}_2$; we have 533866 labeled instances.
\end{enumerate}
The target corresponding to a feature vector $\mathbf{x} \in \mathbb{R}^{30}$ is denoted by a vector $\mathbf{t}$. For the class $\mathcal{C}_1$, we use the target $\mathbf{t} = [1 \,\, 0]^{\top}$ and, for class $\mathcal{C}_2$, we use $\mathbf{t} = [0 \,\, 1]^{\top}$. The ABD-events have been manually labeled on the signals in cooperation with medical specialists. Figure \ref{fig:blackbox} shows $x_1$ and $x_2$ signals for a randomly chosen patient, and occurrence of ABD events manually labeled. A frame is labeled as the class $C_1$ if the middle of the frame is a green line. All other frames correspond to class $C_2$.
\def\scale{.1}
\begin{figure}[!ht]
%\begin{subfigure}[t]{\textwidth}
\begin{minipage}[l]{0.6\textwidth}
\center
\includegraphics[width=\textwidth]{manualdetect_final}
\end{minipage}
\begin{minipage}[r]{0.37\textwidth}
\caption{~\\250 minutes of signal, showing four manually labeled ABD events region in a black box. The green line corresponds to the manually labeled point where an ABD event occurred. }
\label{fig:blackbox}
\end{minipage}
\end{figure}
%We show variability of $x_1$ and $x_2$ signals across patients. We first perform normalization of signals using an affine transform -- bringing the signal in the range [0,1]. Note that $x_1$ and $x_2$ signals individually can be viewed as one-dimensional signals. We perform this normalization for each patient individually. Figure \ref{RangeLimitation} shows histograms of signals patient wise. An important point to mention is that histogram of oxygen saturation level shows high level of non-Gaussian distribution pattern. By observing the histograms, it is clear that a conventional threshold-based detector or an energy-based detector (that uses generalized log-likelihood-ratio-test (GLRT) or Neyman-Pearson criteria) will suffer in high patient variability. For a handful of patients, this may work well; but for most of patients this will work poorly. Hence advanced detection or classification algorithms are required.
%\begin{figure}[!ht]
% \def\scale{0.1}
% \center
% \includegraphics[scale=\scale]{HistogramHFrange01.png} \includegraphics[scale=\scale]{HistogramSpO2range01.png}
% \caption{Histograms of the heart frequency signal and the oxygen saturation level signal of patients. There are 11 patients and histograms of patients are shown individually.}
% \label{RangeLimitation}
%\end{figure}
% \begin{figure}[!ht]
% \def\scale{.09}
%\includegraphics[scale=\scale]{HistogramSpO2.png} \includegraphics[scale=\scale]{HistogramSpO2range01.png}
%\caption{Range limitation of the $SpO_2$ signal of each patient individually.}
% \label{HistogramSpO2}
% \end{figure}
%\subsection{Feature extraction and pre-processing}
%Figure \ref{fig:blackbox} shows $x_1$ and $x_2$ signals for a randomly chosen patient. The time duration for this case study is four hours ten minutes. Note that we use terms `frame' and `window' interchangeably in this article. Immediately, a natural question is: what is the optimal value of $l$ for which we get best detection performance? That means, what is the optimal frame length or time window? This is difficult to answer through an analytic approach, but we can use an experimental approach. An important issue in deciding the length of window is a chosen time resolution that medical specialists find useful. Naturally a large duration may lead to a higher fatigue to medical specialists. {\color{red}5. "Naturally a large duration may lead to a higher fatigue to medical specialists" - Why is that?} {\color{blue}We should have said that large frames could contain several events.} On the other hand, a small duration may cause loss of information from signals, and lead to erroneous detection. To show the non-stationary property, we use discrete Fourier transform (DFT) based spectrograms, shown in Figure \ref{fig:Spectogram}. For the signals, we observe that the low frequency content is highly time-varying. Note that the low-frequency spectrum part is important, and not the high frequency spectrum part. To generate the spectograms, we used $l=15$, that means 15 samples window. As the window length is small (due to low frequency data collected at $\frac{1}{60} Hz$), spectral analysis suffers in high spectral leakage and hence we observe high-frequency spectrum which are not adequately time-varying.
% From the patient data, we have 336948 instances of $\mathbf{x}$. In pre-processing stage, we remove mean of $\mathbf{x}$ from the feature vectors to have a zero mean probability distribution.
%We used sample mean for the purpose.
%\subsection{Labeling of datasets and unbalance aspect of training dataset}
%\label{subsec:Labeling_datasets}
%In this subsection, we describe labeling of data, mainly creating training and test datasets for machine learning, and then comment on the unbalance aspect of training dataset. According to medical knowledge \cite{patel2016clinical}, an ABD event occurs when both the heart frequency signal $x_1$ and the oxygen saturation level signal $x_2$ drop together. Using visual inspection, we first mark some time spans that may contain instances of ABD events. Such time spans are shown by black rectangles in Figure \ref{fig:blackbox}. The black rectangle markings were verified by medical specialists to reduce error in manual marking. Then we develop a signal processing algorithm to label an exact time point in a marked time span where an ABD event occurred. The signal processing algorithm is as follows. For a time span, we find the time point where $x_1$ had the minimum value and the time point where $x_2$ had the minimum value. Then we label the occurrence of ABD event at the time point which is in the middle of the two time points where $x_1$ and $x_2$ have minimum values. In Figure \ref{fig:blackbox}, green color vertical lines are used to show time labeling of ABD events. Now considering the binary classification task, we can use a single target variable $t$ such that $t=1$ denotes class $\mathcal{C}_1$ and $t=0$ denotes class $\mathcal{C}_2$.
%For designing a machine learning algorithm, a challenge is that the dataset is unbalanced.
%When the dataset is unbalanced it is a challenge to train a machine learning algorithm. Here unbalanced means that the dataset has a very limited amount of labeled data with true ABD events compared to the labeled data with no ABD events. In Table~\ref{table:Data_Stat}, we show the number of time points (or feature vectors) where ABD events occurred and did not occur. For total amount of data, we have $\frac{1190}{336948} \times 100= 0.35\%$ ABD events and the rest are no ABD events. Such an unbalanced data will typically bias a machine learning algorithm in the sense of over-fit to no ABD events. To offset the effect of unbalance aspect, we used data duplication strategies that we will describe later in section~\ref{subsec:unbalanced_data}.
%
%Finally we inform about training dataset and testing dataset. We have data from limited number of patients, only 11 patients in this case. Among the patients, their length of stay at ICU varies significantly -- from 80 days to 3 days. Further some patients had less occurrence rate of ABD events compared to other patients. Hence we formed testing dataset from one patient who stayed in hospital a significantly longer time (around 80 days) and suffered a moderate rate of ABD events. The test patient is marked by patient identity 11, shown in Table~\ref{table:Data_Stat}. Data from the other 10 patients formed the training dataset. In terms of feature vectors, the training dataset comprises of 236476 feature vectors and the testing dataset comprises of 100472 feature vectors, totaling 336948 feature vectors as mentioned earlier. For the training dataset, among 236476 feature vectors, we have 235614 and 862 feature vectors for ABD events and no ABD events, respectively.
%
%\renewcommand{\baselinestretch}{1}
%\begin{table}[t]
%\centering
%\caption{Statistics of feature vectors labeled with ABD events and no ABD events.}
%% \vspace{0.5mm}
%\scriptsize
%\setlength{\tabcolsep}{2.5pt}
% \begin{tabular}{|c|c|c|c|c|c|c|c|c|c|c|c|c|}
% \hline
%
% \multicolumn{1}{|c|}{Type} &
% \multicolumn{11}{c|}{Patient identity} &
% \multicolumn{1}{|c|}{} \\ \hline
%
% & 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 & 11 & \\ \hline
%
% \multicolumn{1}{|c|}{} &
% \multicolumn{10}{c|}{Training dataset} &
% \multicolumn{1}{|c|}{Testing dataset} &
% \multicolumn{1}{|c|}{Total data} \\ \hline
%
% No ABD& 32311 & 23548 & 13554 & 8050 & 37634 & 11140 & 61585 & 6131& 4131& 37530& 100144 &335758\\ \hline
%ABD & 25 & 118 & 11 & 11 & 129 & 34 & 326 & 10& 17& 181& 328& 1190\\ \hline
%Total & 32336 & 23666& 13565& 8061 & 37763 & 11174 & 61911 & 6141& 4148& 37711& 100472 &336948\\ \hline
%
% \end{tabular}
% \label{table:Data_Stat}
% %\vspace{-3.5mm}
%\end{table}
%\renewcommand{\baselinestretch}{1}
%\normalsize
%\vspace{-0.6cm}
\section{Methods}
\vspace{-0.2cm}
\subsection{Machine learning algorithms}
\vspace{-0.1cm}
\subsubsection{Progressive learning network (PLN)}
\vspace{-0.1cm}
In \cite{Chatterjee2017}, the PLN is shown to address pertinent technical questions on designing large size neural networks. The questions are: (a) How to determine the number of layers and the number of hidden neuron nodes in a neural network? (b) How to avoid overfitting to training data using appropriate regularization? (3) How to reduce effort in tuning of parameters in the network? The success of the PLN to address the technical questions makes it a user friendly machine learning tool. The user does not require to do pain-stalking tuning of many parameters in the PLN. Further the user can hope for a machine learning solution where chance of over-fitting is diminished. These advantages of PLN can be exploited for machine learning in medical data analytics, where medical practitioners might not have thorough technical knowledge to deal with existing neural networks. Using a mathematical property of rectifier-linear-unit (ReLU) activation function -- called progression property in \cite{Chatterjee2017} --, the training algorithm generates a self-organizing PLN. In the training algorithm, the learning is performed in layer-wise fashion. A new layer is added on an existing optimized network to grow the network further, makes it wide and deep. For learning each layer, PLN uses a convex optimization problem with appropriate regularization. The regularization helps to address the problem of overfitting. In \cite{Chatterjee2017}, the PLN was shown to provide high quality performances for many standard pattern classification and regression tasks where it was also compared with least-squares based learning methods and extreme learning machines \cite{Huang_Extreme_learning_machine_2006}. For the ABD event detection, we are interested to see how the PLN performs when the training data is unbalanced and limited in amount.
\vspace{-0.2cm}
\subsubsection{SVM and ANN}
\vspace{-0.2cm}
Support vector machine (SVM) \cite{bishop2006} is based on a kernel approach that uses a high dimensional non-linear projection of feature vectors and then uses a linear classifier in the transformed feature space. For a binary classification task, SVM provides an optimal solution (of a suitable cost function) that provides a maximum margin between two classes that are linearly separable in a high dimensional feature space. SVM uses kernel trick and it is a maximum margin classifier. It also uses training data sparsely and is expected to be less prone to unbalanced dataset. We have used Gaussian kernels for realizing SVM in our experiments. Then we used a cross-entropy minimization based artificial neural network (ANN) \cite{bishop2006}. A feature vector $\mathbf{x}$ is an input to the ANN and then the input passes through several linear and non-linear transforms to produce the estimated class of the input feature vector. The weights of ANN are learned via a back-propagation algorithm (a gradient descent on the cross-entropy loss function). We used a single layer feed forward ANN as we had access to a limited amount of training data.
%Finally we used the extreme learning machine (ELM) \cite{Huang_Extreme_learning_machine_2006} that has a single layer feed forward architecture with many hidden nodes in its layer. In the ELM ,the input of the non-linear layer is created by multiplying a random matrix instance to the input feature vector $\mathbf{x}$ and then only the output regression matrix is optimized based on least squares based cost function minimization.
%\subsubsection{Committee machine}
%Performance of machine learning algorithms vary due to statistical variations of data, noise types, noise levels and many other aspects. For a particular application where research is limited, it is often a difficult task to decide which one is the best performing algorithm. A committee machine (CM) approach \cite{CommiteeMachine_2000}, \cite{bishop2006} is a powerful strategy to employ in such scenarios. We propose to use a voting based CM approach that combines decisions from the afore-mentioned four machine learning algorithms and achieves a good result. Note that the participating algorithms in the CM are based on generative and discriminative models, and CM can accommodate them for a joint decision. The CM approach is a data and decision fusion principle, and successfully used in many other applications \cite{Ambat_Chatterjee_Hari_FACS_2013}, \cite{Ambat_Chatterjee_Hari_2013_CoMACS}.
%%The CM approach is shown in Figure~\ref{fig:votingcomity}.
%To combine decisions of machine learning algorithms, we use a voting based classification principle. The CM decides occurrence of an ABD event if at-least two among four participating algorithms decide the same; otherwise it decides as no ABD event. We also consider the case when CM detects an ABD event if at-least three among four participating algorithms detect the same.
%\begin{figure}[ht!]
%\centering
%\includegraphics[scale=.3]{votingcomity}
%\caption{Committee machine that combines decisions from four machine learning algorithms and then take a final decision.}
%\label{fig:votingcomity}
%\end{figure}
%\begin{figure}
% \begin{minipage}[c]{0.5\textwidth}
% \includegraphics[width=\textwidth]{votingcomity}
% \end{minipage}\hfill
% \begin{minipage}[c]{0.5\textwidth}
% \caption{Committee machine that combines decisions from four machine learning algorithms and then take a final decision.} \label{fig:votingcomity}
% \end{minipage}
%\end{figure}
\vspace{-0.2cm}
\subsection{Unbalanced data and performance measure}\label{subsec:unbalanced_data}
\vspace{-0.2cm}
The size of training dataset for class $\mathcal{C}_1$ is $0.13\%$ of the size of the training dataset for class $\mathcal{C}_2$. Direct use of such an unbalanced training dataset leads to overfitting. To reduce the effect of overfitting, we endeavor to balance the training dataset by sampling feature vectors from class $\mathcal{C}_1$ and duplicating them to create a larger dataset for class $\mathcal{C}_1$.
%We used direct duplication and duplication with (noisy) perturbation to create the sampled data. For a feature vector, perturbation is performed by adding a small noise vector generated from an isotropic Gaussian vector source. We experienced that the perturbation helps to reduce overfitting.
The training dataset is thus an artificially balanced dataset. For the testing dataset, the classification performance is assessed using the geometric mean between precision and recall, also called Fowlkes-Mallows index: $g(\mathbf{t},\hat{\mathbf{t}})=\sqrt{\frac{TP}{TP+FP} \frac{TP}{TP+FN}}$ \cite{Fowlkes1983}, where TP is the number of true positive $(\mathbf{t},\hat{\mathbf{t}})=([1 \,\, 0]^{\top},[1 \,\, 0]^{\top})$, FP is the number of false positives $(\mathbf{t},\hat{\mathbf{t}})=([0 \,\, 1]^{\top},[1 \,\, 0]^{\top})$, FN is the number of false negatives $(\mathbf{t},\hat{\mathbf{t}})=([1 \,\, 0]^{\top},[0 \,\, 1]^{\top})$; here $\hat{\mathbf{t}}$ is the output of an algorithm. For an algorithm, if the average Fowlkes-Mallows index(mean index) is high then the algorithm is good. Also if the standard deviation of the Fowlkes-Mallows index is low then the algorithm is good. Therefore, a good trade-off between high average and low standard deviation of Fowlkes-Mallows index is required for a preferred algorithm.
%In section~\ref{sec:Experimental_results}, we will report experimental results where algorithms are trained using datasets with no perturbation and perturbation.
%\vspace{-0.5cm}
%\vspace{-0.3cm}
\section{Experimental results}
\vspace{-0.3cm}
\label{sec:Experimental_results}
For a statistically relevant study with limited number of patients, we use a leave-one-out strategy allowing cross-validation for the entire data. We segregate one patient and use the data for that patient as testing dataset and the remaining 12 patients as the training dataset. Each patient is used once for testing and we report the average and standard deviation of the Fowlkes-Mallows index over the 13 trials. Table \ref{table:Algo_Performance2} shows the generalization performance of the three algorithms that we compared. In the table, we also mentioned some relevant parameters of the algorithm that we set so that the results are reproducible. Note that ANN provides poorest performance. We used a single layer feed forward network with 400 hidden neurons and sigmoid activation function. The results of ANN confirm that the detection that we are dealing with in non-trivial. The SVM provides good average performance, but shows high standard deviation. This indicates that SVM works well for several patients, but the detection fails for few patients. At the last, PLN provides a good trade-off between average value and standard deviation of the Fowlkes-Mallows index. This result is encouraging as it shows that PLN works across all 13 patients in a similar fashion.
Next, we plot average precision and average recall results in Figure \ref{fig:2dplan}.
%Note that SVM has a high precision and low recall. For the PLN, we have a low precision, but high recall.
Note that SVM has a high precision and low recall whereas the PLN has a low precision, but high recall.
A high recall means that PLN provides a good detection quality for the ABD events (high true positives and low false negatives). At the same time, the low precision result of PLN means that it provides a large amount of false positives. This happened due to the duplication of ABD events in training data, that biased the PLN towards detecting ABD event even though there is no ABD event. This result shows that the duplication in training data to avoid overfitting is a critical task. As SVM uses training data sparsely, the effect of unbalanced data is less pronounced in SVM in the average sense, providing quasi similar values of precision and recall.
%high precision and low recall.
SVM triggers less false alarms (higher precision) but missed to detect ABD events more frequently than PLN (lower recall).
%A low recall means existence of high false negatives. SVM . From clinical perspective, failure in detecting ABD events has a high repercussion in patient care.
Overall, we find that the PLN can be viewed as the best performing algorithm among the three algorithms.
%Figure \ref{fig:2dplan} illustrates the average of the precision and recall values leading to the average geometric mean presented in table \ref{table:Algo_Performance2} under the form ($\mu\pm\sigma$). Experiments have shown that additive Gaussian noise does not improve the performance compared to the simple replication of features. From these results, it is clear that a single hidden layer cross-entropy minimization - based ANN is not the adequate model for tackling such a difficult problem. SVM has the best performance on average and we see on figure \ref{fig:2dplan} that it has the best precision performance but at the cost of a low sensitivity. Further, the variability between patients is high from SVM which, from a clinical point of view means that, for some arbitrary patients, SVM fails at detecting any ABD events. The progressive learning algorithm is therefore the best performing algorithm as it combines low variance of performance between patients, best sensitivity and better precision than ANN.
%and we present the average and standard deviation of the results .
% for four machine learning algorithms and using two types of dataset described above (section \ref{subsec:unbalanced_data}): duplicated data without perturbation and duplicated data with perturbation\footnote{The results with the initial unbalanced dataset is not presented.}.
%In Table~\ref{table:Algo_Performance}, we use following notations.
%\begin{itemize}[noitemsep,nolistsep]
%\item Target 1 represents the true target value with ABD event class $\mathcal{C}_1$ ($t=1$). Target 0 represents the true target value with no ABD event class $\mathcal{C}_2$ ($t=0$).
%\item Output 1 represents that an algorithm decided ABD event class $\mathcal{C}_1$. Output 0 represents that an algorithm decided no ABD event class $\mathcal{C}_2$.
%\end{itemize}
\begin{figure}[ht!]
\caption{Experimental results for ANN, SVM and PLN}
\begin{subfigure}{.49\textwidth}
\caption{Testing performance (average Fowlkes-Mallows index and standard deviation)}
%\begin{subfigure}{.5\textwidth}
\label{table:Algo_Performance2}
\def\nsys{4}
\def\nres{3}
\renewcommand{\baselinestretch}{1}
\centering
% \vspace{0.5mm}
\scriptsize
\setlength{\tabcolsep}{3.5pt}
\begin{tabular}{|c|c|c|c|}
\hline
%\multicolumn{\nsys}{|c|}{} \\ \hline
Model & ANN & SVM & PLN\\ \hline
Parameters & 1 layer, 400 hidden nodes&Gaussian kernel&$(\lambda,\mu)=(0.1,1e4)$\\ \hline
& \multicolumn{\nres}{|c|}{Testing performance} \\ \hline
Feature replication & $0.03 \pm 0.03$& $0.28 \pm 0.18$&$0.22 \pm 0.06$\\ \hline
\end{tabular}
\end{subfigure}\hfill
\begin{subfigure}{.32\textwidth}
%\begin{figure}[ht!]
\centering
\caption{Recall - Precision plane}
\includegraphics[width=\textwidth]{res_2dplan_textbox_svm_ann_pln}
% \end{subfigure}
% \begin{subfigure}{.5\textwidth}
\label{fig:2dplan}
%\end{figure}
\end{subfigure}
%\end{subfigure}
\end{figure}
%\vspace{-0.7cm}
\section{Conclusions}
\vspace{-0.2cm}
For detection of ABD events, we have investigated three algorithms in this article. Two of them are widely used and one of them is recently proposed. The new progressive learning network (PLN) algorithm is found to be the best. The result for a standard artificial neural network (ANN) based on cross-entropy loss minimization is pessimistic. The support vector machine is promising, but suffers in high variability across patients. Naturally, for further improvement a direction is to exploit the technical advantages and structures of support vector machine in the framework of progressive learning network. Another important line of study is how to handle the limited as well as unbalanced data. For medical data analytics, it is natural that we will continue to deal with limited data in many situations as well as unbalanced data - more data labeled with no clinical symptoms and less data with clinical symptoms. By its nature, the machine learning system will be improved by including more patients. A better design of machine learning algorithm is also expected to contribute to a substantial improvement in detection performance. This should lead to better care of patients.
%With more patients included into the database and an improved machine learning algorithm, a substantial improvement in detection performance is expected. This should lead to better care of patients.
%Working in parallel at including more patients into the database and designing a more elaborated machine learning algorithm, should lead to a substantial detection performance improvement and, \textit{in fine}, to better clinical care.
%In our work, events are detected with a too low precision/recall performance to be directly deployed in the ICUs as a support for medical practitioner.
%From an algorithmic point of view, even for the difficult binary classification problem tackled in this paper, it is still possible to take advantage of the progressive learning strategy to improve our results. We have demonstrated the ability of a PLN to learn structures from data better than any other widely used machine learning algorithm.
%ERIC:\\
%NOT so negative Using only a small number of preterm infant recordings, it was possible to achive better results than any of the classical machine learning methods employed. A an AUC (Area under rthe curve above 0.85 in specificity –True positive values that are useful and feasible to use in the detection of ABD events and thus possible oprevention of life-threatening events.
% It is nonetheless interesting to observe that it is possible up to a certain point to learn different structures from the data.
%
%An unbalanced dataset is usually a big obstacle in the design of supervised learning algorithm but we have shown that standard oversampling methods are able to partially work around this issue. The progressive learning algorithm introduced and tested on artificial benchmark datasets in \cite{Chatterjee2017}, is the best performing algorithm for our problem.
%
%\begin{table}[ht!]
%\caption{Results for SVM, ANN and ELM. No normalization of the database is applied.}
%\label{table:Algo_Performance}
%
%%\begin{subfigure}{\textwidth}
%%\caption{ANN: Very bad generalization performance and very large variance during training}
%%\def\nsys{8}
%%\def\nres{7}
%%\renewcommand{\baselinestretch}{1}
%%\centering
%%\scriptsize
%%\setlength{\tabcolsep}{3.5pt}
%% \begin{tabular}{|c|c|c|c|c|c|c|c|c|}
%% \hline
%% \multicolumn{\nsys}{|c|}{\textcolor{red}{ANN}} \\ \hline
%% Node number& 10 & 30 & 40 & 100 & 200 & 300 & 400 \\ \hline
%% & \multicolumn{\nres}{|c|}{Training performance} \\ \hline
%% Replication & $0.40 \pm 0.39$ & $0.38 \pm 0.37$ & $0.40\pm 0.39$ &$ 0.49 \pm 0.34$ & $0.49 \pm 0.34$ & $0.38\pm 0.37$ & $0.40 \pm 0.35$\\ \hline
%% Additive Noise & $0.45 \pm 0.38$ & $0.54 \pm 0.31$ & $0.49 \pm 0.34$ & $0.38 \pm 0.37 $& $0.49 \pm 0.34$ &$0.27 \pm 0.36 $& $0.55 \pm 0.31$\\ \hline
%% & \multicolumn{\nres}{|c|}{Testing performance} \\ \hline
%% Replication & $0.03 \pm 0.04$ & $0.02\pm 0.02$ & $0.03\pm 0.06 $&$ 0.02 \pm 0.02 $& $0.02 \pm 0.02$ & $0.02 \pm 0.02$ & $0.03 \pm 0.03$\\ \hline
%% Additive Noise & $0.04 \pm 0.09$&$0.03 \pm 0.02$&$0.02 \pm 0.02$&$0.02 \pm 0.02$&$0.02 \pm 0.02$&$0.01 \pm 0.02$&$0.03 \pm 0.04$\\ \hline
%% \end{tabular}
%%
%%
%%\end{subfigure}\\
%\begin{subfigure}{.5\textwidth}
%\caption{Sensitivity - Specificity plan}
%
%\includegraphics[width=\textwidth]{res_2dplan_textbox}
%\label{fig:2dplan}
%\end{subfigure}
%\begin{subfigure}{.5\textwidth}
%\begin{subfigure}{\textwidth}
%\caption{ELM: Good generalization}
%\label{table:Algo_Performance_elm}
%\def\nsys{3}
%\def\nres{2}
%\renewcommand{\baselinestretch}{1}
%\centering
%% \vspace{0.5mm}
%\scriptsize
%\setlength{\tabcolsep}{3.5pt}
% \begin{tabular}{|c|c|c|}
% \hline
% \multicolumn{\nsys}{|c|}{\textcolor{red}{ELM}} \\ \hline
% Node number & 500 & 1000 \\ \hline
% & \multicolumn{\nres}{|c|}{Training performance} \\ \hline
% Replication &$ 0.98 \pm 0.002 $& $0.98 \pm 0.001$\\ \hline
% Additive Noise &$0.95 \pm 0.003$ & $0.96 \pm 0.004$\\ \hline
% & \multicolumn{\nres}{|c|}{Testing performance} \\ \hline
% Replication & $0.20 \pm 0.09 $&$ 0.20 \pm 0.07$\\ \hline
% Additive Noise & $0.20 \pm 0.07$ & $0.19 \pm 0.07$\\ \hline
%
% \end{tabular}
%
%\end{subfigure}\\
%\begin{subfigure}{\textwidth}
%\caption{SVM: Very good generalization performance but very high variance}
%\label{table:Algo_Performance_svm}
%\def\nsys{2}
%\def\nres{1}
%\renewcommand{\baselinestretch}{1}
%\centering
%\scriptsize
%\setlength{\tabcolsep}{3.5pt}
% \begin{tabular}{|c|c|c|}
% \hline
% \multicolumn{\nsys}{|c|}{\textcolor{red}{SVM}} \\ \hline
% Kernel type & Gaussian\\ \hline
% & \multicolumn{\nres}{|c|}{Training performance} \\ \hline
% Replication & $0.9994 \pm 0.0003$ \\ \hline
% Additive Noise & $0.9976 \pm 0.0002$\\ \hline
% & \multicolumn{\nres}{|c|}{Testing performance} \\ \hline
% Replication & $0.276 \pm 0.182$\\ \hline
% Additive Noise & $0.266 \pm 0.118 $\\ \hline
% \end{tabular}
%
% %\vspace{-3.5mm}
%%\end{table}
%\end{subfigure}
%\end{subfigure}\\
%%\begin{subfigure}{.5\textwidth}
%%\caption{TGMM: }
%%\def\nsys{2}
%%\def\nres{1}
%%\renewcommand{\baselinestretch}{1}
%%\centering
%%\scriptsize
%%\setlength{\tabcolsep}{3.5pt}
%% \begin{tabular}{|c|c|c|}
%% \hline
%% \multicolumn{\nsys}{|c|}{\textcolor{red}{TGMM}} \\ \hline
%% Number of gaussians & $(n_0, n_1)=(20, 20)$\\ \hline
%% & \multicolumn{\nres}{|c|}{Training performance} \\ \hline
%% Replication & $0.518491 \pm 0.023283$ \\ \hline
%% Additive Noise & $0.924282 \pm 0.007507$\\ \hline
%% & \multicolumn{\nres}{|c|}{Testing performance} \\ \hline
%% Replication & $0.133110 \pm 0.056006$\\ \hline
%% Additive Noise & $0.135645 \pm 0.056570$\\ \hline
%%
%% \end{tabular}
%%
%% %\vspace{-3.5mm}
%%%\end{table}
%%\end{subfigure}\\
%
%\begin{subfigure}{.47\textwidth}
%\caption{PLN: Good generalization and small variance}
%\label{table:Algo_Performance_pln}
%\def\nsys{5}
%\def\nres{4}
%\renewcommand{\baselinestretch}{1}
%\centering
%\scriptsize
%\setlength{\tabcolsep}{3.5pt}
% \begin{tabular}{|c|c|c|c|c|c|c|c|c|}
% \hline
% \multicolumn{\nsys}{|c|}{\textcolor{red}{PLN}} \\ \hline
% $\lambda$ & $10^5$ & $10^6$ & $10^7$ & $10^8$ \\ \hline
% & \multicolumn{\nres}{|c|}{Training performance} \\ \hline
% Replication&$0.964 \pm 0.002$&$0.965 \pm 0.002$&$0.967 \pm 0.001$&$0.965 \pm 0.001$\\ \hline
% %Replication & $0.976918 \pm 0.002577$&$0.976340 \pm 0.002331$&$0.977169 \pm 0.001666$&$0.981072 \pm 0.002989$\\ \hline
% %Additive Noise & $0.962033 \pm 0.003265$&$0.961400 \pm 0.003206$&$0.962506 \pm 0.003397$&$0.963983 \pm 0.002096$\\ \hline
% & \multicolumn{\nres}{|c|}{Testing performance} \\ \hline
% Replication &$0.19 \pm 0.07$&$0.19 \pm 0.08$&$0.19 \pm 0.07$&$0.20 \pm 0.08$\\ \hline
% %Replication & $0.201851 \pm 0.075473$&$0.204783 \pm 0.069134$&$0.207332 \pm 0.067128$&$0.210665 \pm 0.078696$\\ \hline
% %Additive Noise & $0.181429 \pm 0.066574$&$0.179800 \pm 0.063047$&$0.177864 \pm 0.058314$&$0.181806 \pm 0.066273$\\ \hline
% \end{tabular}
%\end{subfigure}\hfill
%\begin{subfigure}{.32\textwidth}
%\caption{Parameter optimization}
%\includegraphics[width=\textwidth]{testing_perf_pln_lambda_mu}
%\label{fig:parameter_optimization}
%\end{subfigure}
%\end{table}
% OLD TABLE WITH VARYING ALPHA PARAMETER
%\begin{subfigure}{\textwidth}
%\caption{PLN: Good generalization and small variance}
%\def\nsys{5}
%\def\nres{4}
%\renewcommand{\baselinestretch}{1}
%\centering
%\scriptsize
%\setlength{\tabcolsep}{3.5pt}
% \begin{tabular}{|c|c|c|c|c|c|c|c|c|}
% \hline
% \multicolumn{\nsys}{|c|}{\textcolor{red}{PLN}} \\ \hline
% $\lambda$ & $10^5$ & $10^6$ & $10^7$ & $10^8$ \\ \hline
% & \multicolumn{\nres}{|c|}{Training performance} \\ \hline
% Replication & $0.976918 \pm 0.002577$&$0.976340 \pm 0.002331$&$0.977169 \pm 0.001666$&$0.981072 \pm 0.002989$\\ \hline
% Additive Noise & $0.962033 \pm 0.003265$&$0.961400 \pm 0.003206$&$0.962506 \pm 0.003397$&$0.963983 \pm 0.002096$\\ \hline
% & \multicolumn{\nres}{|c|}{Testing performance} \\ \hline
% Replication & $0.201851 \pm 0.075473$&$0.204783 \pm 0.069134$&$0.207332 \pm 0.067128$&$0.210665 \pm 0.078696$\\ \hline
% Additive Noise & $0.181429 \pm 0.066574$&$0.179800 \pm 0.063047$&$0.177864 \pm 0.058314$&$0.181806 \pm 0.066273$\\ \hline
% \end{tabular}
%\end{subfigure}
%\end{table}
%\begin{table}
% \caption{Compare algorithms}
%
%
%\renewcommand{\baselinestretch}{1}
%
%\centering
%\begin{tabular}{|c|c|c|}
% \hline
% \multicolumn{2}{|c|}{\textcolor{red}{Algorithm comparison}} \\ \hline
% Algorithm & Training Performance & Testing Performance\\ \hline
% ANN & & \\ \hline
% TGMM & & \\ \hline
% SVM & & \\ \hline
% ELM & & \\ \hline
% PLN & & \\ \hline
%
% \end{tabular}
%\end{table}
%\renewcommand{\baselinestretch}{1}
%\begin{table}[t]
%\centering
%\caption{Classification (or detection) performance of machine learning algorithms}
%% \vspace{0.5mm}
%\scriptsize
%\setlength{\tabcolsep}{2.5pt}
% \begin{tabular}{|c|c|c|c|c|c|c|c|c|c|}
% \hline
%
% \multicolumn{10}{|c|}{\textcolor{red}{Lowest false positive is marked by red font.} \textcolor{blue}{Lowest false negative is marked by blue font.}} \\ \hline
%
% \multicolumn{1}{|c|}{Target} &
% \multicolumn{1}{|c|}{True occurrence} &
% \multicolumn{8}{|c|}{Methods} \\ \cline{3-10}
%
% \multicolumn{1}{|c|}{} &
% \multicolumn{1}{|c|}{} &
% \multicolumn{2}{|c|}{GMM} &
% \multicolumn{2}{|c|}{SVM} &
% \multicolumn{2}{|c|}{ANN} &
% \multicolumn{2}{|c|}{ELM} \\ \cline{3-10}
% & & Output 0 & Output 1 & Output 0 & Output 1 & Output 0 & Output 1 & Output 0 & Output 1 \\ \hline \hline
%
% \multicolumn{10}{|c|}{Direct use of feature vectors} \\ \hline \hline
%
%Target 0 & 100144 & 96060 & 4084 & 100060 & 84 & 100121 & \textcolor{red}{23} & 100121 & \textcolor{red}{23} \\
%Target 1 & 328 & \textcolor{blue}{161} & 167 & 249 & 79 & 294 & 34 & 322 & 6 \\ \hline
%
% \multicolumn{10}{|c|}{Replication of feature vectors} \\ \hline \hline
%
%Target 0 & 100144 & 92724 & 7420 & 99369 & \textcolor{red}{775} & 98841 & 1303 & 93840 & 6304 \\
%Target 1 & 328 & \textcolor{blue}{134} & 194 & 166 & 162 & 149 & 179 & 106 & 222 \\ \hline
%
% \multicolumn{10}{|c|}{Perturbation of feature vectors} \\ \hline \hline
%
%Target 0 & 100144 & 98173 & 1971 & 96569 & 3575 & 98399 & \textcolor{red}{1745} & 93276 & 6868 \\
%Target 1 & 328 & 195 & 133 & \textcolor{blue}{82} & 246 & 144 & 184 & 108 & 220 \\ \hline
%
% \end{tabular}
% \label{table:Algo_Performance}
% %\vspace{-3.5mm}
%\end{table}
%Next, we show the performance of committee machine (CM) approach in Table~\ref{table:CommitteeMachine_Performance}. For the CM, we used output decisions of afore-mentioned four machine learning algorithms and then made a decision. For example, if at-least two algorithms decide an ABD event then CM decides the same. To reduce false positives, we also use another system choice where CM decides an ABD event if at-least three algorithms decide the same. In our opinion, CM provides good performance when at-least three algorithms vote for ABD event and feature vectors are perturbed to train the algorithms. The CM approach provides a good trade-off between false negatives and false positives, highlighted by blue font in Table~\ref{table:CommitteeMachine_Performance}.
%
%\renewcommand{\baselinestretch}{1}
%\begin{table}[t]
%\centering
%\caption{Classification (or detection) performance of committee machine (CM) algorithm}
%% \vspace{0.5mm}
%\scriptsize
%\setlength{\tabcolsep}{2.5pt}
% \begin{tabular}{|c|c|c|c|c|c|}
% \hline
%
% \multicolumn{1}{|c|}{Target} &
% \multicolumn{1}{|c|}{True occurrence} &
% \multicolumn{4}{|c|}{Voting to detect ABD event} \\ \cline{3-6}
%
% \multicolumn{1}{|c|}{} &
% \multicolumn{1}{|c|}{} &
% \multicolumn{2}{|c|}{At least 2 voting} &
% \multicolumn{2}{|c|}{At least 3 voting} \\ \cline{3-6}
%% \multicolumn{2}{|c|}{At least 3} \\ %\cline{3-10}
%% \multicolumn{2}{|c|}{At least 4}%
% & & Output 0 & Output 1 & Output 0 & Output 1 \\ \hline \hline
%
% \multicolumn{6}{|c|}{Direct use of feature vectors} \\ \hline \hline
% %\input{competitive_table_raw}
%Target 0 & 100144 & 96767 & 3377 & 100067 & 77 \\
%Target 1 & 328 & 151 & 177 & 260 & 68 \\
% \hline
%
% \multicolumn{6}{|c|}{Replication of feature vectors} \\ \hline \hline
% % \input{competitive_table_rep}
%Target 0 & 100144 & 87644 & 12500 & 98002 & 2142 \\
%Target 1 & 328 & 59 & 270 & 120 & 208 \\
% \hline
%
%
% \multicolumn{6}{|c|}{Perturbation of feature vectors} \\ \hline \hline
% %\input{competitive_table_pert}
%Target 0 & 100144 & 91021 & 9123 & 98209 & \textcolor{blue}{1935} \\
%Target 1 & 328 & 53 & 275 & \textcolor{blue}{111} & 217 \\
% \hline
%
%
% \end{tabular}
% \label{table:CommitteeMachine_Performance}
% %\vspace{-3.5mm}
%\end{table}
%We experimentally show that machine learning algorithms can help to detect ABD events from relevant signals of patients. Signals are non-stationary and traditional threshold-based detectors are bound to provide poor performance. Our experiments with popular machine learning algorithms show that there exists enough room for further performance improvement. Significant machine learning research with scope of developing new algorithms can be performed in future for this detection application with unbalanced data and the technical challenge of achieving low false positives as well as low false negatives. To handle unbalanced data, a future path is to explore more advanced sampling techniques, such as Monte Carlo Markov Chain (MCMC). Then, to achieve better performance, a path is to explore deep learning algorithms.
\newpage
% This is where you give us some technical understanding about why your
% awesomeness is awesome, and maybe even some times when it isn't so we
% know when we should be using it. Discuss both technical and clinical
% implications, as appropriate.
%
% Make sure you also put your awesomeness in the context of related
% work. Who else has worked on this problem, and how did they approach
% it? What makes your direction interesting or distinct?
% ACKNOWLEDGEMENTS ONLY GO IN THE CAMERA-READY, NOT THE SUBMISSION
% \acks{Many thanks to all collaborators and funders!}
%\bibliography{library,General,biblio_saikat_CS1,biblio_saikat_ML,biblio_saikat_BioAndMed,biblio_saikat_Pub,references_short}
\bibliography{library}
% \bibliography{bib}
% \clearpage
%\begin{small}
%\setlength{\itemsep}{-6in}
\bibliographystyle{IEEEtran}
%\bibliographystyle{plainnat}
%%\bibliography{biblio_saikat_CS,biblio_saikat_CS1,biblio_saikat_Pub,General}
%\bibliography{General,biblio_saikat_CS1,biblio_saikat_ML,biblio_saikat_BioAndMed,biblio_saikat_Pub,references_short}
%\setlength{\itemsep}{-6in}
%\end{small}
% \appendix
% \section*{Appendix A.}
%
%We will provide data and software codes for reproducing the experimental results obtained in this article later at appropriate time (after double blind review process).
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
%
%
%
%\begin{abstract}
% The abstract paragraph should be indented \nicefrac{1}{2}~inch
% (3~picas) on both the left- and right-hand margins. Use 10~point
% type, with a vertical spacing (leading) of 11~points. The word
% \textbf{Abstract} must be centered, bold, and in point size 12. Two
% line spaces precede the abstract. The abstract must be limited to
% one paragraph.
%\end{abstract}
%
%\section{Submission of papers to NIPS 2017}
%
%NIPS requires electronic submissions. The electronic submission site
%is
%\begin{center}
% \url{https://cmt.research.microsoft.com/NIPS2017/}
%\end{center}
%
%Please read carefully the instructions below and follow them
%faithfully.
%
%\subsection{Style}
%
%Papers to be submitted to NIPS 2017 must be prepared according to the
%instructions presented here. Papers may only be up to eight pages
%long, including figures. This does not include acknowledgements and
%cited references which are allowed on subsequent pages.
%Papers that exceed these limits will not be reviewed, or in any
%other way considered for presentation at the conference.
%
%The margins in 2017 are the same as since 2007, which allow for
%$\sim$15\% more words in the paper compared to earlier years.
%
%Authors are required to use the NIPS \LaTeX{} style files obtainable
%at the NIPS website as indicated below. Please make sure you use the
%current files and not previous versions. Tweaking the style files may
%be grounds for rejection.
%
%\subsection{Retrieval of style files}
%
%The style files for NIPS and other conference information are
%available on the World Wide Web at
%\begin{center}
% \url{http://www.nips.cc/}
%\end{center}
%The file \verb+nips_2017.pdf+ contains these instructions and
%illustrates the various formatting requirements your NIPS paper must
%satisfy.
%
%The only supported style file for NIPS 2017 is \verb+nips_2017.sty+,
%rewritten for \LaTeXe{}. \textbf{Previous style files for \LaTeX{}
% 2.09, Microsoft Word, and RTF are no longer supported!}
%
%The new \LaTeX{} style file contains two optional arguments:
%\verb+final+, which creates a camera-ready copy, and \verb+nonatbib+,
%which will not load the \verb+natbib+ package for you in case of
%package clash.
%
%At submission time, please omit the \verb+final+ option. This will
%anonymize your submission and add line numbers to aid review. Please
%do \emph{not} refer to these line numbers in your paper as they will
%be removed during generation of camera-ready copies.
%
%The file \verb+nips_2017.tex+ may be used as a ``shell'' for writing
%your paper. All you have to do is replace the author, title, abstract,
%and text of the paper with your own.
%
%The formatting instructions contained in these style files are
%summarized in Sections \ref{gen_inst}, \ref{headings}, and
%\ref{others} below.
%
%\section{General formatting instructions}
%\label{gen_inst}
%
%The text must be confined within a rectangle 5.5~inches (33~picas)
%wide and 9~inches (54~picas) long. The left margin is 1.5~inch
%(9~picas). Use 10~point type with a vertical spacing (leading) of
%11~points. Times New Roman is the preferred typeface throughout, and
%will be selected for you by default. Paragraphs are separated by
%\nicefrac{1}{2}~line space (5.5 points), with no indentation.
%
%The paper title should be 17~point, initial caps/lower case, bold,
%centered between two horizontal rules. The top rule should be 4~points
%thick and the bottom rule should be 1~point thick. Allow
%\nicefrac{1}{4}~inch space above and below the title to rules. All
%pages should start at 1~inch (6~picas) from the top of the page.
%
%For the final version, authors' names are set in boldface, and each
%name is centered above the corresponding address. The lead author's
%name is to be listed first (left-most), and the co-authors' names (if
%different address) are set to follow. If there is only one co-author,
%list both author and co-author side by side.
%
%Please pay special attention to the instructions in Section \ref{others}
%regarding figures, tables, acknowledgments, and references.
%
%\section{Headings: first level}
%\label{headings}
%
%All headings should be lower case (except for first word and proper
%nouns), flush left, and bold.
%
%First-level headings should be in 12-point type.
%
%\subsection{Headings: second level}
%
%Second-level headings should be in 10-point type.
%
%\subsubsection{Headings: third level}
%
%Third-level headings should be in 10-point type.
%
%\paragraph{Paragraphs}
%
%There is also a \verb+\paragraph+ command available, which sets the
%heading in bold, flush left, and inline with the text, with the
%heading followed by 1\,em of space.
%
%\section{Citations, figures, tables, references}
%\label{others}
%
%These instructions apply to everyone.
%
%\subsection{Citations within the text}
%
%The \verb+natbib+ package will be loaded for you by default.
%Citations may be author/year or numeric, as long as you maintain
%internal consistency. As to the format of the references themselves,
%any style is acceptable as long as it is used consistently.
%
%The documentation for \verb+natbib+ may be found at
%\begin{center}
% \url{http://mirrors.ctan.org/macros/latex/contrib/natbib/natnotes.pdf}
%\end{center}
%Of note is the command \verb+\citet+, which produces citations
%appropriate for use in inline text. For example,
%\begin{verbatim}
% \citet{hasselmo} investigated\dots
%\end{verbatim}
%produces
%\begin{quote}
% Hasselmo, et al.\ (1995) investigated\dots
%\end{quote}
%
%If you wish to load the \verb+natbib+ package with options, you may
%add the following before loading the \verb+nips_2017+ package:
%\begin{verbatim}
% \PassOptionsToPackage{options}{natbib}
%\end{verbatim}
%
%If \verb+natbib+ clashes with another package you load, you can add
%the optional argument \verb+nonatbib+ when loading the style file:
%\begin{verbatim}
% \usepackage[nonatbib]{nips_2017}
%\end{verbatim}
%
%As submission is double blind, refer to your own published work in the
%third person. That is, use ``In the previous work of Jones et
%al.\ [4],'' not ``In our previous work [4].'' If you cite your other
%papers that are not widely available (e.g., a journal paper under
%review), use anonymous author names in the citation, e.g., an author
%of the form ``A.\ Anonymous.''
%
%\subsection{Footnotes}
%
%Footnotes should be used sparingly. If you do require a footnote,
%indicate footnotes with a number\footnote{Sample of the first
% footnote.} in the text. Place the footnotes at the bottom of the
%page on which they appear. Precede the footnote with a horizontal
%rule of 2~inches (12~picas).
%
%Note that footnotes are properly typeset \emph{after} punctuation
%marks.\footnote{As in this example.}
%
%\subsection{Figures}
%
%All artwork must be neat, clean, and legible. Lines should be dark
%enough for purposes of reproduction. The figure number and caption
%always appear after the figure. Place one line space before the figure
%caption and one line space after the figure. The figure caption should
%be lower case (except for first word and proper nouns); figures are
%numbered consecutively.
%
%You may use color figures. However, it is best for the figure
%captions and the paper body to be legible if the paper is printed in
%either black/white or in color.
%\begin{figure}[h]
% \centering
% \fbox{\rule[-.5cm]{0cm}{4cm} \rule[-.5cm]{4cm}{0cm}}
% \caption{Sample figure caption.}
%\end{figure}
%
%\subsection{Tables}
%
%All tables must be centered, neat, clean and legible. The table
%number and title always appear before the table. See
%Table~\ref{sample-table}.
%
%Place one line space before the table title, one line space after the
%table title, and one line space after the table. The table title must
%be lower case (except for first word and proper nouns); tables are
%numbered consecutively.
%
%Note that publication-quality tables \emph{do not contain vertical
% rules.} We strongly suggest the use of the \verb+booktabs+ package,
%which allows for typesetting high-quality, professional tables:
%\begin{center}
% \url{https://www.ctan.org/pkg/booktabs}
%\end{center}
%This package was used to typeset Table~\ref{sample-table}.
%
%\begin{table}[t]
% \caption{Sample table title}
% \label{sample-table}
% \centering
% \begin{tabular}{lll}
% \toprule
% \multicolumn{2}{c}{Part} \\
% \cmidrule{1-2}
% Name & Description & Size ($\mu$m) \\
% \midrule
% Dendrite & Input terminal & $\sim$100 \\
% Axon & Output terminal & $\sim$10 \\
% Soma & Cell body & up to $10^6$ \\
% \bottomrule
% \end{tabular}
%\end{table}
%
%\section{Final instructions}
%
%Do not change any aspects of the formatting parameters in the style
%files. In particular, do not modify the width or length of the
%rectangle the text should fit into, and do not change font sizes
%(except perhaps in the \textbf{References} section; see below). Please
%note that pages should be numbered.
%
%\section{Preparing PDF files}
%
%Please prepare submission files with paper size ``US Letter,'' and
%not, for example, ``A4.''
%
%Fonts were the main cause of problems in the past years. Your PDF file
%must only contain Type 1 or Embedded TrueType fonts. Here are a few
%instructions to achieve this.
%
%\begin{itemize}
%
%\item You should directly generate PDF files using \verb+pdflatex+.
%
%\item You can check which fonts a PDF files uses. In Acrobat Reader,
% select the menu Files$>$Document Properties$>$Fonts and select Show
% All Fonts. You can also use the program \verb+pdffonts+ which comes
% with \verb+xpdf+ and is available out-of-the-box on most Linux
% machines.
%
%\item The IEEE has recommendations for generating PDF files whose
% fonts are also acceptable for NIPS. Please see
% \url{http://www.emfield.org/icuwb2010/downloads/IEEE-PDF-SpecV32.pdf}
%
%\item \verb+xfig+ "patterned" shapes are implemented with bitmap
% fonts. Use "solid" shapes instead.
%
%\item The \verb+\bbold+ package almost always uses bitmap fonts. You
% should use the equivalent AMS Fonts:
%\begin{verbatim}
% \usepackage{amsfonts}
%\end{verbatim}
%followed by, e.g., \verb+\mathbb{R}+, \verb+\mathbb{N}+, or
%\verb+\mathbb{C}+ for $\mathbb{R}$, $\mathbb{N}$ or $\mathbb{C}$. You
%can also use the following workaround for reals, natural and complex:
%\begin{verbatim}
% \newcommand{\RR}{I\!\!R} %real numbers
% \newcommand{\Nat}{I\!\!N} %natural numbers
% \newcommand{\CC}{I\!\!\!\!C} %complex numbers
%\end{verbatim}
%Note that \verb+amsfonts+ is automatically loaded by the
%\verb+amssymb+ package.
%
%\end{itemize}
%
%If your file contains type 3 fonts or non embedded TrueType fonts, we
%will ask you to fix it.
%
%\subsection{Margins in \LaTeX{}}
%
%Most of the margin problems come from figures positioned by hand using
%\verb+\special+ or other commands. We suggest using the command
%\verb+\includegraphics+ from the \verb+graphicx+ package. Always
%specify the figure width as a multiple of the line width as in the
%example below:
%\begin{verbatim}
% \usepackage[pdftex]{graphicx} ...
% \includegraphics[width=0.8\linewidth]{myfile.pdf}
%\end{verbatim}
%See Section 4.4 in the graphics bundle documentation
%(\url{http://mirrors.ctan.org/macros/latex/required/graphics/grfguide.pdf})
%
%A number of width problems arise when \LaTeX{} cannot properly
%hyphenate a line. Please give LaTeX hyphenation hints using the
%\verb+\-+ command when necessary.
%
%\subsubsection*{Acknowledgments}
%
%Use unnumbered third level headings for the acknowledgments. All
%acknowledgments go at the end of the paper. Do not include
%acknowledgments in the anonymized submission, only in the final paper.
%
%\section*{References}
%
%References follow the acknowledgments. Use unnumbered first-level
%heading for the references. Any choice of citation style is acceptable
%as long as you are consistent. It is permissible to reduce the font
%size to \verb+small+ (9 point) when listing the references. {\bf
% Remember that you can go over 8 pages as long as the subsequent ones contain
% \emph{only} cited references.}
%\medskip
%
%\small
%
%[1] Alexander, J.A.\ \& Mozer, M.C.\ (1995) Template-based algorithms
%for connectionist rule extraction. In G.\ Tesauro, D.S.\ Touretzky and
%T.K.\ Leen (eds.), {\it Advances in Neural Information Processing
% Systems 7}, pp.\ 609--616. Cambridge, MA: MIT Press.
%
%[2] Bower, J.M.\ \& Beeman, D.\ (1995) {\it The Book of GENESIS:
% Exploring Realistic Neural Models with the GEneral NEural SImulation
% System.} New York: TELOS/Springer--Verlag.
%
%[3] Hasselmo, M.E., Schnell, E.\ \& Barkai, E.\ (1995) Dynamics of
%learning and recall at excitatory recurrent synapses and cholinergic
%modulation in rat hippocampal region CA3. {\it Journal of
% Neuroscience} {\bf 15}(7):5249-5262.
\end{document}
% Generated by IEEEtran.bst, version: 1.14 (2015/08/26)
\begin{thebibliography}{10}
\providecommand{\url}[1]{#1}
\csname url@samestyle\endcsname
\providecommand{\newblock}{\relax}
\providecommand{\bibinfo}[2]{#2}
\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax}
\providecommand{\BIBentryALTinterwordstretchfactor}{4}
\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus
\BIBentryALTinterwordstretchfactor\fontdimen3\font minus
\fontdimen4\font\relax}
\providecommand{\BIBforeignlanguage}[2]{{%
\expandafter\ifx\csname l@#1\endcsname\relax
\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%
\typeout{** loaded for the language `#1'. Using the pattern for}%
\typeout{** the default language instead.}%
\else
\language=\csname l@#1\endcsname
\fi
#2}}
\providecommand{\BIBdecl}{\relax}
\BIBdecl
\bibitem{Liu2012}
\BIBentryALTinterwordspacing
L.~Liu, H.~L. Johnson, S.~Cousens, J.~Perin, S.~Scott, J.~E. Lawn, I.~Rudan,
H.~Campbell, R.~Cibulskis, M.~Li, C.~Mathers, and R.~E. Black, ``{Global,
regional, and national causes of child mortality: An updated systematic
analysis for 2010 with time trends since 2000},'' \emph{The Lancet}, vol.
379, no. 9832, pp. 2151--2161, 2012. [Online]. Available:
\url{http://dx.doi.org/10.1016/S0140-6736(12)60560-1}
\BIBentrySTDinterwordspacing
\bibitem{Fairchild2013}
K.~D. Fairchild, R.~L. Schelonka, D.~A. Kaufman, W.~A. Carlo, J.~Kattwinkel,
P.~J. Porcelli, C.~T. Navarrete, E.~Bancalari, J.~L. Aschner, M.~W. Walker,
J.~A. Perez, C.~Palmer, D.~E. Lake, T.~M. O'Shea, and J.~R. Moorman,
``{Septicemia mortality reduction in neonates in a heart rate characteristics
monitoring trial.}'' \emph{Pediatric research}, vol.~74, no.~5, pp. 570--5,
2013.
\bibitem{moorman2011mortality}
J.~R. Moorman, W.~A. Carlo, J.~Kattwinkel, R.~L. Schelonka, P.~J. Porcelli,
C.~T. Navarrete, E.~Bancalari, J.~L. Aschner, M.~W. Walker, J.~A. Perez, and
Others, ``{Mortality reduction by heart rate characteristic monitoring in
very low birth weight neonates: a randomized trial},'' \emph{The Journal of
pediatrics}, vol. 159, no.~6, pp. 900--906, 2011.
\bibitem{Fairchild_2010}
K.~D. Fairchild and T.~M. O'Shea, ``{Heart Rate Characteristics: Physiomarkers
for Detection of Late-Onset Neonatal Sepsis},'' \emph{Clinics in
perinatology}, vol.~37, no.~37, pp. 581--598, 2010.
\bibitem{lake2014complex}
D.~E. Lake, K.~D. Fairchild, and J.~R. Moorman, ``{Complex signals
bioinformatics: evaluation of heart rate characteristics monitoring as a
novel risk marker for neonatal sepsis},'' \emph{Journal of clinical
monitoring and computing}, vol.~28, no.~4, pp. 329--339, 2014.
\bibitem{Sullivan2015}
\BIBentryALTinterwordspacing
B.~A. Sullivan and K.~D. Fairchild, ``{Predictive monitoring for sepsis and
necrotizing enterocolitis to prevent shock},'' \emph{Seminars in Fetal and
Neonatal Medicine}, vol.~20, no.~4, pp. 255--261, 2015. [Online]. Available:
\url{http://dx.doi.org/10.1016/j.siny.2015.03.006}
\BIBentrySTDinterwordspacing
\bibitem{Moss2016}
\BIBentryALTinterwordspacing
T.~J. Moss, D.~E. Lake, J.~F. Calland, K.~B. Enfield, J.~B. Delos, K.~D.
Fairchild, and J.~R. Moorman, ``{Signatures of Subacute Potentially
Catastrophic Illness in the ICU},'' \emph{Critical Care Medicine}, p.~1,
2016. [Online]. Available:
\url{http://content.wkhealth.com/linkback/openurl?sid=WKPTLP:landingpage{\&}an=00003246-900000000-96916}
\BIBentrySTDinterwordspacing
\bibitem{Hofstetter2008}
A.~O. Hofstetter, L.~Legnevall, E.~Herlenius, and M.~Katz-Salamon,
``{Cardiorespiratory development in extremely preterm infants: Vulnerability
to infection and persistence of events beyond term-equivalent age},''
\emph{Acta Paediatrica, International Journal of Paediatrics}, vol.~97,
no.~3, pp. 285--292, 2008.
\bibitem{Siljehav2015}
V.~Siljehav, A.~M. Hofstetter, K.~Leifsdottir, and E.~Herlenius,
``{Prostaglandin E2 Mediates Cardiorespiratory Disturbances during Infection
in Neonates},'' \emph{Journal of Pediatrics}, vol. 167, no.~6, pp.
1207--1213, 2015.
\bibitem{Lee2012}
\BIBentryALTinterwordspacing
H.~Lee, C.~G. Rusin, D.~E. Lake, M.~T. Clark, L.~Guin, T.~J. Smoot, A.~O.
Paget-Brown, B.~D. Vergales, J.~Kattwinkel, J.~R. Moorman, and J.~B. Delos,
``{A new algorithm for detecting central apnea in neonates},''
\emph{Physiological Measurement}, vol.~33, no.~1, pp. 1--17, 2012. [Online].
Available:
\url{http://stacks.iop.org/0967-3334/33/i=1/a=1?key=crossref.3b861d8be3f8122435c3a8b44a9072da}
\BIBentrySTDinterwordspacing
\bibitem{Hinton_DeepLearning_NaturePaper_2015}
Y.~LeCun, Y.~Bengio, and G.~Hinton, ``{Deep learning},'' \emph{Nature}, no.
521, pp. 436--444, 2015.
\bibitem{Hinton_Deng_2012}
G.~Hinton, L.~Deng, D.~Yu, G.~E. Dahl, A.~r.~Mohamed, N.~Jaitly, A.~Senior,
V.~Vanhoucke, P.~Nguyen, T.~N. Sainath, and B.~Kingsbury, ``{Deep Neural
Networks for Acoustic Modeling in Speech Recognition: The Shared Views of
Four Research Groups},'' \emph{IEEE Signal Processing Magazine}, vol.~29,
no.~6, pp. 82--97, nov 2012.
\bibitem{Chatterjee2017}
S.~{Chatterjee}, A.~M. {Javid}, M.~{Sadeghi}, P.~P. {Mitra}, and M.~{Skoglund},
``{Progressive Learning for Systematic Design of Large Neural Networks},''
\emph{ArXiv e-prints}, Oct. 2017.
\bibitem{SVM_1999}
J.~A.~K. Suykens and J.~Vandewalle, ``{Least squares support vector machine
classifiers},'' \emph{Neural Processing Letters}, vol.~9, no.~3, pp.
293--300, jun 1999.
\bibitem{bishop2006}
\BIBentryALTinterwordspacing
C.~M. Bishop, \emph{{Pattern recognition and machine learning}}.\hskip 1em plus
0.5em minus 0.4em\relax Springer, 2006, vol.~4. [Online]. Available:
\url{http://www.bibsonomy.org/bibtex/2da4cb0f072a2671bf03b617d5001376c/aude.hofleitner}
\BIBentrySTDinterwordspacing
\bibitem{Huang_Extreme_learning_machine_2006}
G.~B. Huang, Q.~Y. Zhu, and C.~K. Siew, ``{Extreme learning machine: Theory and
applications},'' \emph{Neurocomputing}, vol.~70, pp. 489--501, 2006.
\bibitem{Fowlkes1983}
E.~B. Fowlkes and C.~L. Mallows, ``{A method for comparing two hierarchical
clusterings},'' \emph{Journal of the American Statistical Association},
vol.~78, no. 383, pp. 553--569, 1983.
\end{thebibliography}