An End-to-End and Accurate PPG-based Respiratory Rate Estimation Approach Using Cycle Generative Adversarial Networks
Extracted source text · arXiv 2105.00594
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%2345678901234567890123456789012345678901234567890123456789012345678901234567890
% 1 2 3 4 5 6 7 8
\documentclass[letterpaper, 10 pt, conference]{ieeeconf} % Comment this line out if you need a4paper
%\documentclass[a4paper, 10pt, conference]{ieeeconf} % Use this line for a4 paper
\IEEEoverridecommandlockouts % This command is only needed if
% you want to use the \thanks command
\overrideIEEEmargins % Needed to meet printer requirements.
%In case you encounter the following error:
%Error 1010 The PDF file may be corrupt (unable to open PDF file) OR
%Error 1000 An error occurred while parsing a contents stream. Unable to analyze the PDF file.
%This is a known problem with pdfLaTeX conversion filter. The file cannot be opened with acrobat reader
%Please use one of the alternatives below to circumvent this error by uncommenting one or the other
%\pdfobjcompresslevel=0
%\pdfminorversion=4
% See the \addtolength command later in the file to balance the column lengths
% on the last page of the document
% The following packages can be found on http:\\www.ctan.org
%\usepackage{graphics} % for pdf, bitmapped graphics files
%\usepackage{epsfig} % for postscript graphics files
%\usepackage{mathptmx} % assumes new font selection scheme installed
%\usepackage{times} % assumes new font selection scheme installed
%\usepackage{amsmath} % assumes amsmath package installed
%\usepackage{amssymb} % assumes amsmath package installed
\title{\LARGE \bf
An End-to-End and Accurate PPG-based Respiratory Rate Estimation Approach Using Cycle Generative Adversarial Networks
}
\author{Seyed Amir Hossein Aqajari$^{*,1}$, Rui Cao$^{1}$, Amir Hosein Afandizadeh Zargari$^{1}$, and Amir M. Rahmani$^{1,2,3}$% <-this % stops a space
%\thanks{*This work was not supported by any organization}% <-this % stops a space
\thanks{ $^{1}$ Department of Electrical Engineering and Computer Science, University of California Irvine, CA 92697, USA
{\tt\small (*correspondence e-mail: saqajari@uci.edu)}}%
%\thanks{$^{2}$Bernard D. Researcheris with the Department of Electrical Engineering, Wright State University, Dayton, OH 45435, USA{\tt\small b.d.researcher@ieee.org}}%
\thanks{ $^{2}$ Department of Computer Science, University of California, Irvine, CA 92697, USA
}
\thanks{ $^{3}$ Institute for Future Health, University of California, Irvine, CA 92697, USA
}
}
\usepackage{graphicx}
\usepackage{amsmath}
\usepackage{subcaption}
\usepackage{siunitx}
\begin{document}
\maketitle
\thispagestyle{empty}
\pagestyle{empty}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{abstract}
%Respiratory rate (RR) is a clinical sign representing ventilation (air movement in and out of the lungs). An abnormal change in RR is often the first sign of health deterioration as the body attempts to maintain oxygen delivery to the tissues. However, this measurement is often neglected by health professionals due to the lack of reliable equipment in the hospitals and clinical environment. PPG signals have been a great source of RR extraction due to existing respiration-induced modulations inside them. Furthermore, these signals can easily be collected using widely available convenient wearable devices. The existing RR estimation methods from raw PPG signals mostly rely on hand-crafted rules and manual parameters tuning. An end-to-end automatic deep learning approach was recently proposed which heavily relies on data augmentation in order to achieve a reasonable performance. We present an end-to-end pipeline architecture for RR estimation in which cycle generative adversarial networks are employed in order to reconstruct respiratory signals from raw PPG signals. According to our result, we are able to estimate RR from raw PPG signal with mean absolute error of 2.0$\pm$0.3 using five fold cross validation in a publicly available dataset. The performance of our proposed pipeline is about 2 times better than the existing end-to-end deep learning method. Our result suggest that cycle generative adversarial networks could be a valuable source of RR estimation from raw PPG signal.
Respiratory rate (RR) is a clinical sign representing ventilation. An abnormal change in RR is often the first sign of health deterioration as the body attempts to maintain oxygen delivery to its tissues. There has been a growing interest in remotely monitoring of RR in everyday settings which has made photoplethysmography (PPG) monitoring wearable devices an attractive choice.
PPG signals are useful sources for RR extraction due to the presence of respiration-induced modulations in them. The existing PPG-based RR estimation methods mainly rely on hand-crafted rules and manual parameters tuning. An end-to-end deep learning approach was recently proposed, however, despite its automatic nature, the performance of this method is not ideal using the real world data. In this paper, we present an end-to-end and accurate pipeline for RR estimation using Cycle Generative Adversarial Networks (CycleGAN) to reconstruct respiratory signals from raw PPG signals. Our results demonstrate a higher RR estimation accuracy of up to 2$\times$ (mean absolute error of 1.9$\pm$0.3 using five fold cross validation) compared to the state-of-th-art using a identical publicly available dataset. Our results suggest that CycleGAN can be a valuable method for RR estimation from raw PPG signals.
%\index \textit{Index Terms}— Photoplethysmography, Respiratory Rate,
%Generative Adversarial Networks, Cycle Generative Adversarial Network.
\end{abstract}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{INTRODUCTION}
Respiratory rate (RR), often referred to as breathing rate, is the number of breaths a person takes per minute. A normal resting RR for adults ranges from 12 to 20 \cite{clevelandclinic}. Abnormal changes in respiratory rate are an accurate indicator of physiological conditions such as anxiety, hypoxia, hypercapnia, metabolic and respiratory acidosis \cite{rolfe2019importance}. A diverse body of research studies has indicated the significance of respiration rate for forecasting events such as cardiac arrest, patient deterioration, and care escalation \cite{cooper2014respiratory,dougherty2015royal,cretikos2008respiratory}.
%Respiratory rate (RR), often referred to as your breathing rate, is the number of breaths a person take per minute. For most people, average breaths per minute typically range from 12 to 20 while in a state of rest. Abnormal changes in respiratory rate are extremely good indicator of physiological conditions such as anxiety, hypoxia, hypercapnia, metabolic and respiratory acidosis [cite magonline library paper]. There have been a lot of research studies indicating the importance of respiration rate in terms of patient clinical outcomes and monitoring. Cretikos et al. demonstrated how the respiratory rate can help to forecast events such as cardiac arrest and unexpected care admissions [cite]. Dougherty and Lister discussed about the relation between respiratory rate and patient deterioration. They further discussed that respiratory rate can help if a patient requires care escalation [cite].
%Despite the importance of respiratory rate as one of the first indicators of health deterioration, health professionals do not perceive the measurement of respiratory rate as important as recording other vital signs [cite]. A possible reason for poor monitoring of respiratory rate is the lack of reliable measurement equipment in hospitals [cite]. The current clinical practice usually measures RR by chest wall expansion which is inconvenient, inaccurate, and highly subjective. Therefore, there is a substantial value in calculating RR using more accurate and convenient devices. The blood perfusion dynamics are known to carry information on breathing, as respiration-induced modulations in the PPG signal. These days PPG signal can easily be collected continuously and remotely using inexpensive, convenient, and portable wearable devices. Recently, these devices also have been increasingly adopted into healthcare domain and clinical settings. Aforementioned reasons makes PPG signals a suitable source of respiratory rate extraction.
%The importance of respiratory rate as one of the first indicators of health deterioration has attracted significant attention in the daily monitoring of RRs [cite]. However, reliable measurement devices of RR are bulky and cumbersome, and therefore, their usage is mainly for hospitals and intensive care. The blood perfusion dynamics are known to carry information on breathing, as respiration-induced modulations in photoplethysmography (PPG) signals [cite]. Nowadays, PPG signals can easily be collected continuously and remotely using inexpensive, convenient, and portable wearable devices. With the rapid development of wearable technologies, wearable devices are gradually entering people's daily life.
%, offering PPG signals as a suitable source of respiratory rate extraction. %hence PPG signals are acknowledged as a suitable source of respiratory rate extraction in daily life setting.
%The aforementioned reasons make PPG signals a suitable source of respiratory rate extraction in order to forecast unexpected care admissions in everyday setting.
The importance of respiratory rate as one of the first indicators of health deterioration has attracted significant attention in RR's daily monitoring \cite{nicolo2020importance}. However, RR's reliable measurement devices are bulky and cumbersome, and are mainly used for inpatients. With the rapid development of wearable technologies, a change in an individual's physiological systems' functional state can be tracked and monitored in an everyday setting, for instance, by using photoplethysmography (PPG) \cite{allen2007photoplethysmography}. PPG signals can easily be collected continuously and remotely using a wide range of inexpensive, convenient, and portable wearable devices (.e.g., smart watches, rings, etc.). The blood perfusion dynamics are known to carry information on breathing, as respiration-induced modulations in PPG signals \cite{pirhonen2018acquiring}. Hence, PPG signals are considered as a suitable source for respiratory rate extraction to forecast unexpected care admissions in a daily life setting .
The RR estimation from PPG signals has received remarkable attention in the literature \cite{charlton2017breathing}. Traditional RR estimation methods require several steps, including digital filtering, time/frequency domain analysis, extraction of signal components from composite signals, deriving respiratory surrogate waveforms and features using the fiducial points, signal quality estimations, and sensor fusion \cite{charlton2016assessment}. These techniques rely heavily on manual parameter tuning, optimization, and hand-crafted rules designed for specific target patient population. In the past few years, machine learning techniques and neural networks have been widely used in health monitoring domains \cite{zargari2020newertrack, aqajari2021pyeda, mehrabadi2020detection, aqajari2021pain, zargari2021accurate}. Bian et al. \cite{bian2020respiratory} recently proposed an end-to-end deep learning approach in order to automatically and accurately estimate RR from raw PPG signals. Despite the automatic nature of their proposed model, the performance of this method is not ideal (mean absolute error (MAE) of 3.8 $\pm$ 0.5 bpm, which is about \%25 inaccuracy considering 16 bpm as an average RR per minute).
%The RR estimation from PPG signals has received remarkable attention by research studies in the past few years [cite]. Most of the proposed methods in these research works rely on manual parameter tuning, optimization techniques, and hand-crafted rules designed for specific target population. Bian et al. recently proposed an end-to-end deep learning automatic approach in order to estimate RR from raw PPG signals. However, one of the limitations in their work is that it requires a relatively large amount of annotated data in order to achieve a clinically reasonable performance.
This work proposes an automatic end-to-end generative deep learning approach using cycle generative adversarial networks (CGAN) \cite{CycleGAN2017} to reconstruct respiratory signals from raw PPG signals and estimate RR with a high accuracy. CGAN is a novel and powerful approach in the field of unsupervised learning, which targets learning the structure of two given data domains to translate an individual input from one domain to a desired output from the second domain. We also propose a novel loss function to be integrated in our CGAN model that takes into account the key attribute (i.e., RR) of the generated respiratory signals. %A publicly available toolbox is used for processing, filtering, and calculating RR from derived respiratory signals.
Our results demonstrate that the proposed GAN-based approach estimates RR from raw PPG signals with 2$\times$ higher accuracy compared with the state-of-the-art approach \cite{bian2020respiratory} using real-world data. Furthermore, our method outperforms the classical RR estimations methods, despite utilizing the complete automatic end-to-end design.
In summary, this work makes the following key contributions:
\begin{itemize}
\item Proposing an end-to-end automatic approach based on CGAN which outperforms the performance of the classical RR estimation methods (using an identical setting and dataset).
\item Proposing a novel loss function for our CGAN model that takes into account the RR of the generated respiratory signals.
\item Demonstrating the performance of our approach using the real-world data and comparing it against the state-of-the-art (using an identical setting and dataset).
\end{itemize}
%In this work, we propose an automatic end-to-end pipeline architecture that estimates RR from raw PPG signals with about 2 times better performance than the Bian et al. model (using real-word data). Furthermore, our proposed method outperforms the classical RR estimations methods, despite utilizing the complete automatic end-to-end design. A cycle generative adversarial network is employed in our pipeline to reconstruct the respiratory signals from raw PPG signals. A CGAN is a new strong technique in the field of unsupervised learning, which was first introduced by Jun-Yan Zhu et al. This network targets to learn the structure of two given data domains to translate an individual input from one domain to a desired output from the second domain. A publicly available toolbox is used for processing, filtering, and calculating RR from derived respiratory signals.
The rest of this paper is organized as follows. Section \ref{methods} introduces the employed dataset and our proposed pipeline architecture. In Section \ref{results} we summarize the result obtained by our proposed method. Section \ref{discussion} compares our result with the state-of-the-art in RR estimation from raw PPG signal. Finally, Section \ref{conclusion} concludes the paper.
\section{MATERIAL AND METHODS}
\label{methods}
\subsection{Dataset}
We employed BIDMC PPG and Respiration Dataset \cite{pimentel2016toward} to evaluate our RR estimator method. This is a publicly available dataset which contains signals and numerics extracted from the much larger MIMIC II matched waveform Database, along with manual breath annotations made from two annotators, using the impedance respiratory signal.
In this dataset, PPG and impedence respiratory signals are collected from 53 adult patients for about 8-minute duration at sampling rate of 125 Hz. This dataset is widely used to evaluate the performance of different algorithms for estimating respiratory rate from PPG signals \cite{charlton2017breathing, birrenkott2017robust, jarchi2018accelerometry, bian2020respiratory}.
\subsection{RR Estimation Pipeline}
Figure \ref{fig:pipeline} shows our proposed pipeline for estimating respiratory rate from PPG signals. There are three different main stages in this pipeline: (1) Data Preparation, (2) PPG to Respiration Translator (PRT), and (3) RR Estimator. In the following subsections, we discuss each part in detail.
\subsubsection{Data Preparation}
The primary purpose of this stage is to prepare the data and pre-process it for the PPG-to-Respiration Translator (PRT) module. PPG signals are sampled at much higher frequency than required in the BIDMC dataset. Therefore, downsampling is done to save memory, and reduce processing time and computational complexity of our model while preserving the signals integrity. In this stage, first, raw PPG data are normalized to 0-1. Then, the signals are down-sampled to 30 Hz. Finally, 30-second windows of data are extracted from the signals to be used in the PRT module for translation. %Finally these 1 dimensional signals are reshaped to 2D images in order to be used in our PRT module for image-to-image translation.
\begin{figure}[h]
\centering
\includegraphics[width=0.8\linewidth]{fig/RR_estimator_pipeline.pdf}
\caption{RR Estimation Pipeline}
\label{fig:pipeline}
\end{figure}
\subsubsection{PPG to Respiration Translator (PRT)}
In this module, the Cycle Generative Adversarial Networks (The Cycle GAN) are employed to reconstruct respiratory signals from raw PPG signals. The Generative Adversarial Networks belong to the field of unsupervised learning targeting to learn the structure of a given data in order to generate new unseen data. The GANs are composed of two models: a generator network and a discriminator network. The generator network starts at a point from a latent space as an input and aims to generate new data similar to the expected domain. The discriminator network on the other hand attempts to recognize if an input data is real (belongs to the original dataset) or fake (generated by the generator network).
The Cycle GAN is an extension of the Generative Adversarial Networks which was first proposed by Jun-Yan Zhu et al. \cite{CycleGAN2017}. The idea behind the Cycle GAN is to take an input from the first domain and generate an output of the second domain. In our case, the goal of Cycle GAN is to learn the mapping between PPG signals (domain \textit{X}) and respiratory signals (domain \textit{Y}). Each domain contains set of training samples $\{{x_i}\}_{i=1}^N \in X$ and $\{{y_i}\}_{i=1}^N \in Y$ used directly from BIDMC dataset. The model includes two generators with mapping functions as $G : X \rightarrow Y$ and $F : Y \rightarrow X$ and two discriminators $D_X$ and $D_Y$. In the discriminator networks, $D_X$ aims to distinguish between real PPG signals ($x_i$) and synthetic PPG signals ($F(y)$) while $D_Y$ aims to discriminate between real respiratory signals ($y_i$) and synthetic respiratory signals ($G(x)$).
We indicate the distributions of our data as $x \sim p_{data}(x)$ and $y \sim p_{data}(y)$. Our objective loss function contains three terms: (1) adversarial losses \cite{goodfellow2014generative}, (2) cycle consistency losses, and (3) RR Loss.
Adversarial losses are employed for matching the distribution of generated synthetic signals to the data distribution of original signals. We apply adversarial loss function on both of our mapping functions. The objective function applied to the mapping function \textit{G} is expressed as below:
\begin{align}
\begin{split}
L_{GAN}(G, D_Y, X, Y) = E_{y\sim p_{data}(y)}[log D_{Y}(y)]\\ + E_{x\sim p_{data}(x)}[log(1- D_{Y}(G(x)))]
\end{split}
\end{align}
where $G$ tries to generate respiratory signals $(G(x))$ that look similar to original respiratory signals collected from BIDMC dataset (domain Y), while $D_Y$ aims to discriminate between synthetic respiratory signals $(G(x))$ and real samples $(y)$. In a same way, adversarial loss for the mapping function \textit{F} is expressed as $L_{GAN}(F, D_X, Y, X)$.
A mapping function trained only by adversarial loss as an objective function can map the same set of signals from the first domain to any random permutation of signals in the second domain. Therefore, cycle consistency losses are added to guarantee the mapping from an individual input ($x_i$) to a desired output $(y_i)$ by considering learned mapping functions to be cycle consistent. This means that for each PPG signal $x$ from domain X we must have $x \rightarrow G(x) \rightarrow F(G(x)) \approx x$ while for each respiratory signal $y$ we have $y \rightarrow F(y) \rightarrow G(F(y)) \approx y$. This behaviour is indicated as:
\begin{align}
\begin{split}
L_{cyc}(G, F) = E_{x\sim p_{data}(x)}[||F(G(x))-x||_1]\\ + E_{y\sim p_{data}(y)}[||G(F(y))-y||_1]
\end{split}
\end{align}
In order to force the synthesized respiratory signals to only keep their main features, we define RR loss function which attempts to take into account the RR of the generated respiratory signals. The BioSPPy \cite{biosppy} public python library is used to calculate the respiration rate of the synthetic and original respiratory signals. This additional loss function can be expressed as:
\begin{align}
L_{RR}(G) = E_{y\sim p_{data}(y)}[||G(F(y))_{RR}-y_{RR}||_1]
\end{align}
Therefore, the final objective is the weighted sum of the above loss functions:
\begin{align}
\begin{split}
L(G, F, D_X, D_Y) = L_{GAN}(G, D_Y, X, Y) \\ + L_{GAN}(F, D_X, Y, X) \\ + \lambda_1 L_{cyc}(G, F) \\ + \lambda_2 L_{RR}(G)
\end{split}
\end{align}
where $\lambda_1$ and $\lambda_2$ are the weights of cycle consistency loss and RR loss respectively (both are empirically selected as 10 in our work).
$G$ and $F$ attempt to minimize this objective against adversaries $D_X$ and $D_Y$ that try to maximize it. Hence, we aim to solve:
\begin{align}
G^*,F^* = \arg \min_{G, F} \max_{D_X, D_Y}L(G, F, D_X, D_Y)
\end{align}
We use the CGAN architecture proposed by \cite{CycleGAN2017}. The architecture of generative networks is adopted from Johnson et al. \cite{johnson2016perceptual}. This network contains two stride-2 convolutions, several residual blocks \cite{he2016deep}, and two fractionally-strided convolutions with stride 0.5. For the discriminator networks we use 70$\times$70 PathGANs \cite{isola2017image, li2016precomputed, ledig2017photo} which aims to classify whether the signals are fake or real.
%To understand our PPG to Respiration Translator module, in the following paragraphs we describe in detail about what exactly CGANs are and how these models can benefit us in reconstructing the respiratory signals from the raw PPG signals.
%The CGAN architecture was first described by Jun-Yan Zhu et al. []. The main idea behind the CGAN is to take an input from the first domain and generate an output of the second domain, and the reverse. Therefore, the CGAN is consist of two generator and two discriminator networks which are simultaneously being trained. One generator is for generating data for the first domain and the second generator for generating data for the second domain. The discriminators take real input data from their target domain and generated data from their corresponding generator and are being trained to predict whether they are real or fake. The generators are updated accordingly. To be able to translate an individual input from one domain to a desired output from the second domain, the CGAN employs a cycle consistency loss as an additional extension to the model. This loss measures the difference between the generated output data of the first generator and the original data, and the reverse. This extra extension leads the CGAN toward data translation from one domain to another (and the reverse).
%Our proposed PPG to Respiration Translator (PRT) module aims to translate the PPG signals (domain A) to the respiration signals (domain B) using the Cycle Generative Adversarial Networks. In this work, we use the CGAN architecture proposed by [cite]. They adopted the architecture of their generative networks from Johnson et al. [cite]. This network contains four convolutions, several residual blocks [cite], and two fractionally-strided convolutions with stride 0.5. For the discriminator networks they use 70x70 PathGANs [cite].
%The PRT module trained by respiratory and PPG data collected from BIDMC dataset is used in our proposed pipeline to translate new raw PPG signals to their corresponding respiratory signals.
\subsubsection{RR Estimator}
This module uses the BreathMetrics \cite{noto2018automated} for pre-processing, filtering, and calculating the respiratory rate of the signals. The BreathMetrics is a Matlab toolbox for analyzing respiratory recordings. Breathmetrics was designed to make analyzing respiratory recordings easier by automatically de-noising the data and extracting the many features embedded within respiratory recordings including respiratory rate. The methods used in this tool (de-noising and features extraction) have been validated, peer-reviewed, and published in Chemical Senses, a scientific journal.
%\begin{figure*}
%\begin{subfigure}{.5\textwidth}
% \centering
% \includegraphics[width=1.08\linewidth, height=0.65\linewidth]{fig/fake.pdf}
% \caption{Generated respiratory signal before and after preprocessing}
% \label{fig:sfig1}
%\end{subfigure}%
%\begin{subfigure}{.5\textwidth}
% \centering
% \includegraphics[width=1.08\linewidth, height=0.65\linewidth]{fig/real.pdf}
% \caption{Real respiratory signal before and after preprocessing}
% \label{fig:sfig2}
%\end{subfigure}
%\caption{Generated respiratory signal by our proposed pipeline alongside with corresponding real respiratory signal.}
%\label{fig:signals}
%\end{figure*}
\begin{figure}[h]
\centering
\includegraphics[width=0.5\linewidth,height=0.7\linewidth]{fig/sample.png}
\caption{Example of the synthetic respiratory signal generated by our PRT module, from top to bottom: reference respiratory signal, synthetic respiratory signal, and processed synthetic respiratory signal using the BreathMetrics module.}
\label{fig:sample}
\end{figure}
\subsection{Performance Metric}
%We calculate the mean absolute error (MAE) and the root mean square error (RMSE) as two performance metrics in our study in order to evaluate our RR estimation method. MAE is calculated by averaging the absolute differences between the values estimated by a model and the values observed. RMSE is a frequently used measure of the differences between values (sample or population values) predicted by a model or an estimator and the values observed.
We calculate the mean absolute error (MAE) as the performance metric in our study in order to evaluate our RR estimation method. MAE is calculated by averaging the absolute differences between the values estimated by a model and the values observed.
MAE is defined as:
%\[
%MAE = \frac{1}{N}\sum_{i=1}^{N} %|RR_{e}^i-RR_{r}^i| \label{MAE} %\tag{1}
%\]
\begin{align}
\begin{split}
MAE = \frac{1}{N}\sum_{i=1}^{N} |RR_{e}^i-RR_{r}^i| \\ %RMSE = \sqrt{\frac{1}{N}\sum_{i=1}^{N} (RR_{e}^i-RR_{r}^i)^2}
\end{split}
\end{align}
%\[
%RMSE = \sqrt{\frac{1}{N}\sum_{i=1}^{N%} (RR_{e}^i-RR_{r}^i)^2} \label{RMSE} %\tag{2}
%\]
Where $N$ is total number of respiratory segments. $RR_{e}^i$ and $RR_{r}^i$ are estimated and reference respiratory rate for each 1 minute data respectively.
%Since our window sizes are 30 seconds, we are multiplying the MAE by 2 to calculate the number of errors per minute.
To evaluate the performance of our algorithm, we use 5-fold cross validation. We split the BIDMC dataset into five folds while making sure each subject's data is appeared in one fold only. We perform the training 5 times and each time 4 folds are for training and 1 fold is for testing. The average MAE of 5 runs is presented as the final performance results for our proposed model. We trained our models on Nvidia Quadro RTX 5000 GPU with 125 GB of RAM. For each training experiment we performed 100 epochs in which early stopping technique were employed to reduce over-fitting.
\section{RESULTS}
\label{results}
Figure \ref{fig:sample} shows an example of a reconstructed respiration signal using our method alongside with its reference respiration signal for the same timing window of the same test subject. The blue signal represents the reference respiration signal from the BIDMC dataset. The red signal is the synthetic respiratory signal which is an output of our PRT module. The green signal shows the final respiratory signal after de-noising and pre-processing of the signal using the BreathMetrics tool. By comparing the reference respiratory signal and the processed synthetic respiratory signal it can be seen that our pipeline is capable of reconstructing the respiratory signal itself from the PPG signal with high precision.
Table \ref{tbl:features} shows the summary of the MAE performance (average$\pm$std) of our method compared with the state of the arts. The same publicly available dataset (BIDMC) and the same performance metric (MAE using five fold cross validation) are used to fairly evaluate and compare the performance of our automatic approach.
%\begin{table}[!h]
%\centering
%\caption{Summary of the MAE Performances in compare with state of the arts}
%\label{tbl:features}
%\resizebox{\columnwidth}{!}{%
%\begin{tabular}{|c|c|c|c|}
%\hline
% RR Estimation Models & MAE & RMSE\\
%\hline
%Bian et al. model & 3.8$\pm$0.5 & -\\
%Bian et al. enhanced model & Real world + synthetic data & 2.5$\pm$0.6 & -\\
%Karlen et al. model & - & 3.0\\
%Our proposed pipeline & 1.9$\pm$0.3 & 2.7\\
%\hline
%\end{tabular}
%}
%\end{table}
\begin{table}[!h]
\centering
\caption{The summary of MAE Performances in compare with state of the arts}
\label{tbl:features}
\resizebox{.8\columnwidth}{!}{%
\begin{tabular}{|c|c|c|}
\hline
\textbf{RR Estimation Models} & \textbf{MAE}\\
\hline
Bian et al. \cite{bian2020respiratory} DL method & 3.8$\pm$0.5\\
Bian et al. \cite{bian2020respiratory} SmartQualityFusion method & 2.6$\pm$0.4\\
Our proposed CycleGAN-based method & 1.9$\pm$0.3\\
\hline
\end{tabular}
}
\end{table}
As can be seen from the table, our CGAN-based method significantly (up to 2$\times$) outperforms the state-of-the-art RR estimation methods from a raw PPG signals. In the next section, we discuss the results in more details.
\section{DISCUSSION}
\label{discussion}
%Respiratory rate (RR) is a clinical sign representing the ventilation (the movement of air in and out of the lungs). Abnormal change in RR is often the first sign of health deterioration as the body attempts to maintain oxygen delivery to the tissues [cite: www.nursingtimes.net/].
Existing PPG-based RR estimation methods heavily rely on features derived from various hand-crafted algorithms and tuning parameters for specific settings. In this work, we presented a novel fully end-to-end automatic approach for estimating RR from raw PPG signals.
A classical RR estimation method has been implemented in \cite{bian2020respiratory} which is referred as SmartQualityFusion, as a combination of Smart fusion \cite{karlen2013multiparameter} and Quality fusion \cite{chan2013ambulatory}. The SmartQualityFusion algorithm achieved the MAE of 2.6$\pm$0.4 breaths/min (Table \ref{tbl:features}). According to Table \ref{tbl:features}, our proposed RR estimation pipeline demonstrate a significant improvement in accurately estimating RR compared with their algorithm (the MAE of 1.9 breaths/min), despite the complete end-to-end automatic environment of our method.
%The state-of-the-art RR estimation algorithm based on hand-crafted rules [cite] presented the Smart Fusion algorithm. They tested their algorithm on data obtained from 29 children and 13 adults. Their Smart Fusion algorithm achieved the RMSE of 3.0 breaths/min (Table \ref{tbl:features}). According to Table \ref{tbl:features}, our proposed RR estimation pipeline demonstrate an improvement in accurately estimating RR in compare with the Smart Fusion algorithm (the RMSE of 2.7 breaths/min), despite the complete end-to-end automatic environment of our method.
Bian et al. \cite{bian2020respiratory}, also proposed an end-to-end learning approach based on deep learning to estimate RR from PPG. Their results demonstrate a clinically reasonable performance; however, the performance of their proposed automatic approach heavily depends to the availability of real world data. They used the mean absolute error (MAE) as a performance metric in their work. According to their results, their deep learning approach trained with real data could achieve the MAE of 3.8$\pm$0.5 breath/minutes which is substantially higher than the MAE error achieved by their classical algorithm (SmartQualityFusion).
%, despite the fact that [MAE] $\le$ [RMSE].
According to Table \ref{tbl:features}, our proposed pipeline architecture %is capable of estimating RR from raw PPG signal with MAE of 1.9$\pm$0.3.
%using the same real existing dataset as their work (BIDMC).
%Therefore, our proposed algorithm
provides about 2 times better performance of estimating respiratory rate compared with their proposed deep learning approach. %In [], they increased the performance of their proposed method, by augmenting the real datasets using the generated synthetic PPG data. This enhanced the MAE of their model from 3.8 brpm to 2.5 brpm. Thus, the data augmentation approach significantly enhanced the performance of their model. However, our proposed model could still achieve a better performance (2.0 brpm), despite being trained only on a real data.
%Augmenting the data using synthetic data is more challenging in our work.
%In order to augment the data we need to generate both synthetic PPG data alongside with its corresponding synthetic respiratory data. Therefore, the data augmentation is a more challenging work in our proposed method in compare with Bian et al.
One existing limitation of our work, which is also present in the other state-of-the-art methods, is the lack of noisy PPG data in the real datasets used for training our models. Therefore, the performance of our proposed method might not be as expected on a noisy PPG data collected during daily life and physical activities. This limitation is also present in the other state-of-the-arts methods since the employed datasets are mostly consist of stationary data. However, as also mentioned in \cite{bian2020respiratory}, this problem can easily be diminished by retraining the model using the new existing noisy PPG and respiratory signals.
In \cite{bian2020respiratory}, the authors increased the performance of their proposed method, by augmenting the real datasets using the generated synthetic PPG data. This enhanced the MAE of their model from 3.8 brpm to 2.5 brpm. Thus, the data augmentation approach significantly enhanced the performance of their model. However, our proposed model still achieves a better performance (1.9 brpm), despite being trained only on a real data. As a future work, we intend to further increase the performance of our proposed method by augmenting our dataset using synthetic data.
%Furthermore, we plan to enhance the preprocessing units in our pipeline to train the models using the more informative PPG and respiratory signals with higher qualities.
\section{Conclusion}
\label{conclusion}
In summary, in this work we presented a novel pipeline architecture in order to estimate respiratory rate using PPG signals. We are the first one to use cycle adversarial networks in our model to reconstruct the respiratory signals from PPG signals. According to our results, our proposed pipeline architecture is able to estimate RR with the MAE of 1.9 which has the performance of 2.0x better than the state-of-the-art automatic RR estimation method.
% \section*{ACKNOWLEDGMENT}
% To Do.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\begin{thebibliography}{99}
%\bibitem{c1} G. O. Young, ÒSynthetic structure of industrial plastics (Book style with paper title and editor),Ó in Plastics, 2nd ed. vol. 3, J. Peters, Ed. New York: McGraw-Hill, 1964, pp. 15Ð64.
%\bibitem{c2} W.-K. Chen, Linear Networks and Systems (Book style). Belmont, CA: Wadsworth, 1993, pp. 123Ð135.
%\end{thebibliography}
\bibliographystyle{IEEEtran}
\bibliography{IEEEabrv,ref}
\end{document}
% Generated by IEEEtran.bst, version: 1.14 (2015/08/26)
\begin{thebibliography}{10}
\providecommand{\url}[1]{#1}
\csname url@samestyle\endcsname
\providecommand{\newblock}{\relax}
\providecommand{\bibinfo}[2]{#2}
\providecommand{\BIBentrySTDinterwordspacing}{\spaceskip=0pt\relax}
\providecommand{\BIBentryALTinterwordstretchfactor}{4}
\providecommand{\BIBentryALTinterwordspacing}{\spaceskip=\fontdimen2\font plus
\BIBentryALTinterwordstretchfactor\fontdimen3\font minus
\fontdimen4\font\relax}
\providecommand{\BIBforeignlanguage}[2]{{%
\expandafter\ifx\csname l@#1\endcsname\relax
\typeout{** WARNING: IEEEtran.bst: No hyphenation pattern has been}%
\typeout{** loaded for the language `#1'. Using the pattern for}%
\typeout{** the default language instead.}%
\else
\language=\csname l@#1\endcsname
\fi
#2}}
\providecommand{\BIBdecl}{\relax}
\BIBdecl
\bibitem{clevelandclinic}
``{my.clevelandclinic.org},''
\url{https://my.clevelandclinic.org/health/articles/}, [Online; accessed
Mar-2021].
\bibitem{rolfe2019importance}
S.~Rolfe, ``The importance of respiratory rate monitoring,'' \emph{BJN}, 2019.
\bibitem{cooper2014respiratory}
S.~J.~R. Cooper \emph{et~al.}, ``Respiratory rate records: the repeated rate?''
\emph{JCN}, 2014.
\bibitem{dougherty2015royal}
L.~Dougherty \emph{et~al.}, \emph{The Royal Marsden manual of clinical nursing
procedures}.\hskip 1em plus 0.5em minus 0.4em\relax John Wiley \& Sons, 2015.
\bibitem{cretikos2008respiratory}
M.~A. Cretikos \emph{et~al.}, ``Respiratory rate: the neglected vital sign,''
\emph{MJA}, 2008.
\bibitem{nicolo2020importance}
A.~Nicol{\`o} \emph{et~al.}, ``The importance of respiratory rate monitoring:
From healthcare to sport and exercise,'' \emph{Sensors}, 2020.
\bibitem{allen2007photoplethysmography}
J.~Allen, ``Photoplethysmography and its application in clinical physiological
measurement,'' \emph{Physiological measurement}, 2007.
\bibitem{pirhonen2018acquiring}
M.~Pirhonen \emph{et~al.}, ``Acquiring respiration rate from
photoplethysmographic signal by recursive bayesian tracking of intrinsic
modes in time-frequency spectra,'' \emph{Sensors}, 2018.
\bibitem{charlton2017breathing}
P.~H. Charlton \emph{et~al.}, ``Breathing rate estimation from the
electrocardiogram and photoplethysmogram: A review,'' \emph{IEEE RBME}, 2017.
\bibitem{charlton2016assessment}
------, ``An assessment of algorithms to estimate respiratory rate from the
electrocardiogram and photoplethysmogram,'' \emph{Physiological measurement},
2016.
\bibitem{zargari2020newertrack}
A.~H.~A. Zargari \emph{et~al.}, ``Newertrack: Ml-based accurate tracking of
in-mouth nutrient sensors position using spectrum-wide information,''
\emph{IEEE Transactions on Computer-Aided Design of Integrated Circuits and
Systems}, vol.~39, no.~11, pp. 3833--3841, 2020.
\bibitem{aqajari2021pyeda}
S.~A.~H. Aqajari \emph{et~al.}, ``pyeda: An open-source python toolkit for
pre-processing and feature extraction of electrodermal activity,''
\emph{Procedia Computer Science}, vol. 184, pp. 99--106, 2021.
\bibitem{mehrabadi2020detection}
M.~A. Mehrabadi \emph{et~al.}, ``Detection of covid-19 using heart rate and
blood pressure: Lessons learned from patients with ards,'' \emph{arXiv
preprint arXiv:2011.10470}, 2020.
\bibitem{aqajari2021pain}
S.~A.~H. Aqajari \emph{et~al.}, ``Pain assessment tool with electrodermal
activity for postoperative patients: Method validation study,'' \emph{JMIR
mHealth and uHealth}, vol.~9, no.~5, p. e25258, 2021.
\bibitem{zargari2021accurate}
A.~H.~A. Zargari \emph{et~al.}, ``An accurate non-accelerometer-based ppg
motion artifact removal technique using cyclegan,'' \emph{arXiv preprint
arXiv:2106.11512}, 2021.
\bibitem{bian2020respiratory}
D.~Bian \emph{et~al.}, ``Respiratory rate estimation using ppg: A deep learning
approach,'' in \emph{2020 42nd Annual International Conference of the IEEE
EMBC}, 2020.
\bibitem{CycleGAN2017}
J.-Y. Zhu \emph{et~al.}, ``Unpaired image-to-image translation using
cycle-consistent adversarial networkss,'' in \emph{2017 IEEE ICCV}, 2017.
\bibitem{pimentel2016toward}
M.~A. Pimentel \emph{et~al.}, ``Toward a robust estimation of respiratory rate
from pulse oximeters,'' \emph{IEEE TBME}, 2016.
\bibitem{birrenkott2017robust}
D.~A. Birrenkott \emph{et~al.}, ``A robust fusion model for estimating
respiratory rate from photoplethysmography and electrocardiography,''
\emph{IEEE TBME}, 2017.
\bibitem{jarchi2018accelerometry}
D.~Jarchi \emph{et~al.}, ``Accelerometry-based estimation of respiratory rate
for post-intensive care patient monitoring,'' \emph{IEEE Sensors Journal},
2018.
\bibitem{goodfellow2014generative}
I.~J. Goodfellow \emph{et~al.}, ``Generative adversarial networks,''
\emph{arXiv preprint arXiv:1406.2661}, 2014.
\bibitem{biosppy}
\BIBentryALTinterwordspacing
C.~Carreiras, ``{BioSPPy}: Biosignal processing in {Python},'' 2015--.
[Online]. Available: \url{https://github.com/PIA-Group/BioSPPy/}
\BIBentrySTDinterwordspacing
\bibitem{johnson2016perceptual}
J.~Johnson \emph{et~al.}, ``Perceptual losses for real-time style transfer and
super-resolution,'' in \emph{ECCV}.\hskip 1em plus 0.5em minus 0.4em\relax
Springer, 2016.
\bibitem{he2016deep}
K.~He \emph{et~al.}, ``Deep residual learning for image recognition,'' in
\emph{Proceedings of the IEEE CVPR}, 2016.
\bibitem{isola2017image}
P.~Isola \emph{et~al.}, ``Image-to-image translation with conditional
adversarial networks,'' in \emph{Proceedings of the IEEE conference on CVPR},
2017.
\bibitem{li2016precomputed}
C.~Li \emph{et~al.}, ``Precomputed real-time texture synthesis with markovian
generative adversarial networks,'' in \emph{ECCV}.\hskip 1em plus 0.5em minus
0.4em\relax Springer, 2016.
\bibitem{ledig2017photo}
C.~Ledig \emph{et~al.}, ``Photo-realistic single image super-resolution using a
generative adversarial network,'' in \emph{Proceedings of the IEEE CVPR},
2017.
\bibitem{noto2018automated}
T.~Noto \emph{et~al.}, ``Automated analysis of breathing waveforms using
breathmetrics: a respiratory signal processing toolbox,'' \emph{Chemical
senses}, 2018.
\bibitem{karlen2013multiparameter}
W.~Karlen \emph{et~al.}, ``Multiparameter respiratory rate estimation from the
photoplethysmogram,'' \emph{IEEE TBME}, 2013.
\bibitem{chan2013ambulatory}
A.~M. Chan \emph{et~al.}, ``Ambulatory respiratory rate detection using ecg and
a triaxial accelerometer,'' in \emph{2013 35th Annual International
Conference of the IEEE EMBC}.
\end{thebibliography}