Heuristics for Efficient Sparse Blind Source Separation
Extracted source text ยท arXiv 1812.06737
\documentclass[a4paper,10pt,twocolumn]{article}
%-------------------------------------------------------------%
%-------- Please do not change the following commands --------%
\usepackage[english]{babel}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
% Defining Times Fonts (more compact)
\renewcommand{\sfdefault}{phv}
\renewcommand{\rmdefault}{ptm}
\renewcommand{\ttdefault}{pcr}
\newcommand{\norm}[2]{\left\Vert {#1} \right\Vert _{#2}}
%% The page style must be empty (we will add page numbers in the final
%% proceedings)
\renewcommand{\thepage}{}
% Remark: Command \pagestyle{empty} does not delete the first page
% number
%% Please don't touch this size commands
%% Set a margin of 15mm from each edge of the paper
\usepackage[top=1.5cm, left=1.5cm, right=1.5cm, bottom=1.5cm]{geometry}
%% Redefining abstract default style in the article template
\renewenvironment{abstract}{\bf\small {\em\ Abstract---}}{}
%% Included packages, if you need others, please insert
% them in the section provided below
\usepackage{amsfonts,amssymb,amsmath,amsthm}
\usepackage{subfigure}
\usepackage{graphicx}
\usepackage[footnotesize]{caption}
%-------- Please do not change the previous commands ---------%
%-------------------------------------------------------------%
%% If you want to include more packages, please do so here
% \usepackage{mypackages}
\usepackage{amsfonts,amssymb,amsmath,amsthm}
%\usepackage[caption=false,font=normalsize,labelfont=sf,textfont=sf]{subfig}
%% Insert your title here
\title{Heuristics for Efficient Sparse Blind Source Separation}
%% Order author name and affiliation simply like this
\author{Christophe Kervazo$^1$, J\'{e}r\^{o}me Bobin$^1$ and C\'{e}cile Chenot$^2$.\\
\footnotesize $^1$CEA Saclay, Gif-sur-Yvette, 91191 cedex, France \ $^2$Institute for Digital Communications, University of Edinburgh, UK\
} \date{\empty} % no need for a date
\begin{document}
\maketitle
\begin{abstract}
Sparse Blind Source Separation (sparse BSS) is a key method to analyze multichannel data in fields ranging from medical imaging to astrophysics. However, since it relies on seeking the solution of a {\it non-convex} penalized matrix factorization problem, its performances largely depend on the optimization strategy.
In this context, Proximal Alternating Linearized Minimization (PALM) has become a standard algorithm which, despite its theoretical grounding, generally provides poor practical separation results. In this work, we propose a novel strategy that combines a heuristic approach with PALM. We show its relevance on realistic astrophysical data.
\end{abstract}
\section{Introduction}
\subsection{Blind Source Separation problem}
In the BSS \cite{comon2010handbook} framework, the data are composed of $m$ observations, each of which has $t$ samples. These observations are supposed to be some linear combinations of $n$ sources. The objective of BSS is to retrieve the sources as well as the mixing coefficients. In matrix form, the goal is therefore to find two matrices $\mathbf{S}$ (of size $n\times t$) and $\mathbf{A}$ (of size $m \times n$), called respectively the source and the mixing matrices, such that: $\mathbf{X = AS + N}$, where $\mathbf{X}$ (of size $m\times t$) is the observation matrix that is corrupted with some unkwown noise $\mathbf{N}$. Since it requires tackling an ill-posed unsupervised matrix factorization problem, further assumptions are needed, including the statistical independance of the sources (ICA - \cite{comon2010handbook}), the non-negativity of $\mathbf{A}$ and $\mathbf{S}$ \cite{gillis2012accelerated}. In this work, we will focus on the sparsity of the sources \cite{bobin20085,zibulevsky2001blind,bronstein2005sparse,li2006underdetermined}. In this framework, sparse BSS will aim at finding a (local) minimum of:
\begin{equation}
\small
\min_{\mathbf{A,S}} \frac{1}{2} \norm{\mathbf{X-AS}}{F}^{2}
+ \iota_{\{\forall i; \|{\bf A}^i \|_2^2 \leq 1\}}(\bf A)
+ \norm{R_\mathbf{S}\odot(\mathbf{S\Phi_S^T})}{1}
\label{funcGMCA}
\end{equation}
The first data fidelity term promotes a faithful reconstruction of the data. The use of the Froebenius norm $\norm{.}{F}$ stems from the assumption of a Gaussian noise $\mathbf{N}$. The second term ensures that the columns of $\mathbf{A}$ are all in the $\ell_2$ ball. This avoids degenerated $\mathbf{A}$ and $\mathbf{S}$ ($\norm{\mathbf{A}}{ } \rightarrow \infty$ and $\norm{\mathbf{S}}{ } \rightarrow 0$). The last term involving the Hadamard product $\odot$ enforces a $\ell_1$ sparsity constraint in a transformed domain $\mathbf{\Phi_S}$ (here, $\mathbf{\Phi_S}$ will be the starlet transform \cite{starck2010sparse}). $\mathbf{R_\mathbf{S}}$ controls the trade-off between the data fidelity and the sparsity terms. It can be decomposed into $\mathbf{R_\mathbf{S}} = \mathbf{\mathbf{\Lambda_S}} \mathbf{W}$ where $\mathbf{\mathbf{\Lambda_S}}$ is a diagonal matrix of the regularization parameters $\lambda_1, \lambda_2,...,\lambda_n$ and $\mathbf{W}$ is a matrix used to introduce individual penalization coefficients in the context of reweighted $\ell_1$ \cite{candes2008enhancing}.
\subsection{Sparse BSS in practice}
\label{thrdStrat}
Since sparse BSS requires solving a penalized matrix factorization problem, the separation quality strongly depends on the optimization strategy. Different algorithmic frameworks have been used so far: projected Alternate Least-Square (ALS - \cite{gillis2012accelerated}), PALM \cite{bolte2014proximal} and Block-Coordinate Descent (BCD - \cite{tseng2001convergence}, which is not studied here due to a high computational burden):
\begin{itemize}
\item \emph{PALM algorithm}: PALM is an iterative algorithm, which alternates at each iteration between a proximal gradient step on $\mathbf{A}$ and $\mathbf{S}$. PALM is proved to converge to a local minimum of (\ref{funcGMCA}) under mild conditions \cite{bolte2014proximal}.
\item \emph{GMCA algorithm}: the Generalized Morphological Component Analysis (GMCA - \cite{bobin20085}) algorithm is based on projected ALS. At each iteration (k), the gradient step appearing in PALM is replaced by a multiplication by a pseudo-inverse. Compared to PALM, GMCA cannot be proved to converge. Furthermore, even when stabilizing the output of GMCA is not really minimizing (\ref{funcGMCA}).
\end{itemize}
In practice, the solution of sparse BSS methods is highly sensitive to the initial point and the values of the regularization parameters, which are generally tricky to tune without any first guess of the solution.
Thanks to heuristics, GMCA benefits from an automatic thresholding strategy (see Sec.\ref{sec:DiffParamPALMGMCA}) which has been empirically shown to improve its robustness with respect to the initialization and give good estimations of $\mathbf{A^*}$ and $\mathbf{S^*}$. Such heuristics do not exist in PALM.
\subsection{Contributions}
While PALM is theoretically well rooted and yields rather fast minimization schemes (in contrast to BCD), it generally provides poor separation results. We show how PALM-based implementations can benefit from the information provided by heuristic approaches which are in practice more robust. The robustness of the proposed combined strategy is demonstrated on realistic astrophysical data.
%\section{Optimization strategy: PALM and GMCA}
%\label{sec:SensInitec:OptimStrat}
%As emphasized before, the optimization strategy is decisive to avoid stationary points. In the following, we will discuss PALM \cite{bolte2014proximal} and projected ALS such as Generalized Morphological Component Analysis (GMCA - \cite{bobin20085}).
%BCD will not be studied since it requires an exact minimization at each iteration, leading in general to a high computational cost.
%\subsection{PALM algorithm}
%In our case, it can be shown that the proximal operator corresponding to the $\ell_1$ sparsity term in
%(\ref{funcGMCA}) is the soft-thresholding operator $\mathcal{S}_{\Lambda_\mathbf{S}}(.)$ applied in the transform domain. Furthermore, the operator corresponding to the oblique constraint is the projection of each column of $\mathbf{A}$ on the $\ell_2$ unit ball, which we shall denote as $\Pi_{\norm{.}{2}}(.)$. Each iteration (k) can then be read as:\\ \\
%\hspace{-0.8cm}
%{
% \fbox{
% \parbox{\linewidth}{\vspace{-0.2cm}
% \begin{itemize}
% \item[1 -] Update of $\mathbf{S}$ using a fixed $\mathbf{A}$:
% \vspace{-0.3cm}
% \begin{equation}
% \small
% \hspace{-0.8cm}
% \mathbf{\hat{S}} = \mathbf{{S}}^{(k-1)} - \frac{\gamma}{\norm{\mathbf{A}^{(k-1)T} \mathbf{A}^{(k-1)}}{2}}\mathbf{A}^{(k-1)T} (\mathbf{A}^{(k-1)}\mathbf{S}^{(k-1)}-\mathbf{X})
% \label{upS_PALM}
% \end{equation}
% \vspace{-0.4cm}
% \begin{equation}
% \small
% \mathbf{{S}}^{(k)} = \mathcal{S}_{\frac{\gamma \mathbf{R}_\mathbf{S}}{\norm{\mathbf{A}^{(k-1)T} \mathbf{A}^{(k-1)}}{2}}}\left(\mathbf{\hat{S}}{\bf \Phi_S^T}\right){\bf \Phi_S}
% \end{equation}
% \vspace{-0.5cm}
% \item[2 -] Update of
% $\mathbf{A}$ using a fixed $\mathbf{S}$:
% \vspace{-0.5cm}
% \begin{equation}
% \small
% \mathbf{\hat{A}} =
% \mathbf{A}^{(k-1)} - \frac{\delta}{\norm{\mathbf{S}^{(k)}\mathbf{S}^{(k)T}}{2}}(\mathbf{A}^{(k-1)}\mathbf{S}^{(k)}-\mathbf{X})\mathbf{S}^{(k)T}
% \label{upA_PALM}
% \end{equation}
% \vspace{-0.4cm}
% \begin{equation}
% \small
% \mathbf{A}^{(k)} = \Pi_{\norm{.}{2}}
% \left(\mathbf{\hat{A}}\right)
% \end{equation}
% \end{itemize}
% \vspace{-0.2cm}
% }
% }
%}
%\subsection{GMCA algorithm}
%GMCA algorithm is based on projected ALS. At each iteration (k), the gradient step appearing in PALM is replaced by a multiplication by a pseudo-inverse.
%%: $\mathbf{\hat{S}} = \mathbf{A}^{(k-1)\dagger}\mathbf{X}$ and $\mathbf{\hat{A}} = \mathbf{X}\mathbf{S}^{(k)\dagger}$.\\
%
%Compared to PALM, GMCA cannot be proved to converge. Furthermore, even when stabilizing the output of GMCA is not really minimizing (\ref{funcGMCA}). However, thanks to heuristics, it benefits from an automatic thresholding strategy (see Sec.\ref{sec:DiffParamPALMGMCA}) which has been empirically shown to improve its robustness with respect to the initialization and give good estimations of $\mathbf{A^*}$ and $\mathbf{S^*}$.
%\section{PALM-based sparse BSS in practice}
%\label{part2Thrd}
%The objective of this section is to empirically shed light on the lack of robustness of PALM to motivate the need of finding a relevant strategy to enable its use on practical problems.
%\subsection{Experimental protocol}
%\label{ExpProt2thrd}
%To bring out PALM sensibility and the mechanisms at stake, our experiments focus in this subsection on simple data coming from $n = 2$ equilibrated (with equal $\ell_2$ norm) sources. The sources are assumed to be exactly sparse in the direct domain, a proportion $p = 0.1$ of the $t = 500$ samples being non-zeros and drawn according to a standard normal distribution. The mixing is performed through a $\mathbf{A}$ matrix drawn randomly following a standard normal distribution and modified to have unit columns. Its condition number is $C_d = 10$. There is $m = n$ observations. To complete the creation of the $\mathbf{X}$ data, a Gaussian additive noise is added to the mixing, such that the Signal to Noise Ratio is SNR = 60 dB.\\
%Once the algorithm launched, it stops based on a convergence criterion computed through the angular distance of the columns of $\mathbf{A}$ between two successive iterations. The separation quality is then mesured using a global criterion on $\mathbf{A}$ \cite{bobin2015sparsity}: $C_A = -10\log_{10}(\mathtt{median}(\mathbf{\rvert PA^\dagger A^*\rvert - I_d}))$, with $\mathbf{A}^*$ the true mixing matrix and $\mathbf{PA}^\dagger$ the pseudo-inverse of the solution found by the algorithm corrected through $\mathbf{P}$ for the scale and permutation indeterminacies. The higher $C_A$, the better the separation.\\
%To study PALM robustness, we perform an exhaustive search for the parameters. This would be intractable on practical cases with a higher number of sources and unknown true $\mathbf{A}^*$ and $\mathbf{S}^*$. In this section, the parameters are held fixed during the whole algorithm, the goal being to show the sensibility to the parameter choice. The results might however be sub-optimal due to the fact that varying parameters could define a better trajectory avoiding spurious critical points.
%
%\subsection{Sensibility to the thresholding strategy}
%\begin{figure}[!t]
% \centering
% \includegraphics[width=2.5in]{fig1.png}\hspace{.2in}
% \includegraphics[width=2.5in]{fig2.png}
% \caption{\emph{Left:} Mean of $C_A$ as a 2D function of the thresholds for 5 different initializations of PALM algorithm. \emph{Right:} Standard deviation of $C_A$ as a 2D function of the thresholds for 5 different initializations of PALM algorithm.}
% \label{fig_2thrdMeanES}
% \label{fig_2thrdStdES}
%\end{figure}
%Following our experimental protocol, we obtain Figure \ref{fig_2thrdMeanES}, on which the mean of $C_A$ is displayed as a function of the threshold of each of the 2 sources. The lack of robustness and sensibility to the parameter choice is underlined by the following remarks:
%\begin{itemize}
% \item High quality separation happens for values close to the diagonal. A slight deviation from it can cause a 30 dB drop for a $3\times10^{-4}$ modification of one parameter and lead to a very bad separation. One could argue that this problem could be circumvented by imposing $\lambda_1 = \lambda_2$. First, it has to be emphasized that the maximum is \emph{not} on the diagonal but slightly outside, for $\lambda_1 = 3.81\times 10^{-3}$ and $\lambda_2 = 3.88\times 10^{-3}$. Merely setting $\lambda_1 = \lambda_2 = 3.81\times 10^{-3}$ yields a loss of more than 10 dB. Second, in this simple toy example, the sources are equilibrated. It would not necessarily be the case with real-life sources, which could lead to a change in the diagonal structure of the figure;
% \item Inside of the diagonal itself, a $2\times 10^{-3}$ change of $\lambda_1 = \lambda_2$ can lead to a loss of about 7 dB.
%\end{itemize}
%
%\subsection{Sensibility to the initialization}
%\label{sec:SensInit}
%As described in section \ref{thrdStrat}, the initialization impacts the results due to the non-convex nature of problem (\ref{funcGMCA}) because the algorithm can stay trapped in spurious critical points. The standard deviation of $C_A$ over different initializations of $\mathbf{A}$ is plotted in Fig. \ref{fig_2thrdStdES} to quantify this impact. It can reach up to 5 dB for some regularization parameters choices. The best parameter setting corresponds to a quite high standard deviation and can shift depending on the initialization. It makes the practical choice of an initialization a critical point which directly impact the quality of the local minimum found by the algorithm.\\
%It further highlights the sensitivity to the values of the regularization parameters. A key question is thus {\it how to tune PALM in practice and if it can be done in an automatic way without prior knowledge about the solution.}
\section{Complexity of introducing heuristics in PALM}
\label{HeurInPalm}
Building on the automatic thresholding strategy of GMCA, the goal of this part is to try to derive one for PALM.
\subsection{Automatic parameter choice in GMCA}\label{sec:DiffParamPALMGMCA}
In GMCA, the threshold choice is performed computing the Median Absolute Deviation (MAD) for each currently estimated source $\mathbf{\hat{S}}$. The corresponding threshold (which changes during the iterations) is set to a multiple $k$ of this value:
\begin{equation}
\small
\left( \lambda_1,\lambda_2,...,\lambda_n \right)^T
= k\times \mathtt{MAD}(\mathbf{\hat{S}})
\label{eq:kmad}
\end{equation}
In this equation and in the remaining of this section, ${\bf \Phi_S^T}$ was supposed to be the identity matrix without loss of generality. $k$ is a positive number and the $\mathtt{MAD}$ operator is computed row-by-row with $\mathtt{MAD}(z) = \mathtt{median}(|z - \mathtt{median}(z)|)$ for $z\in \mathbb{R}^t$.
%If we suppose that GMCA has found the true $\mathbf{A}$ matrix denoted by $\mathbf{A}^*$, it can be shown that:
%\begin{equation}
%\small
%k\times \mathtt{MAD}(\mathbf{\hat{S}})
%\simeq k\times \mathtt{MAD} \left(\mathbf{A}^{*\dagger}\mathbf{N} \right)
%\end{equation}
%The true thresholds should therefore remove a dense signal coming from the back-projection of the data noise $\mathbf{N}$. This interpretation of the thresholding is however limited to the case when the estimation of $\mathbf{A}$ is close to $\mathbf{A}^*$. In the opposite case, the unperfect demixing results in an increase of the thresholds due to an extra noise coming from interferences between the sources.
Using this strategy, the threshold choice can be interpreted as a dense noise removal.
\subsection{Introducing heuristics in PALM}
\label{MotivHeur}
Let us assume that the thresholds in PALM are computed the same way as in GMCA through Eq.~(\ref{eq:kmad}) and that during the iterations the algorithm finds estimates that are close to \emph{both} the true matrices $\mathbf{A}^*$ and $\mathbf{S}^*$. Then, due to the assumption of $\mathbf{S}^*$ sparsity:
\begin{equation}
\small
\left( \lambda_1,\lambda_2,...,\lambda_n\right)^T
\simeq k\times \frac{\gamma}{\norm{\mathbf{A}^{*T} \mathbf{A}^{*}}{2}}\mathtt{MAD}(\mathbf{A}^{*T} \mathbf{N})
\end{equation}
Therefore, using the MAD enables a thresholding of a projection of the noise $\mathbf{N}$, which yields a similar interpretation as in GMCA. \emph{However, this interpretation requires that the algorithm finds good estimates of $\mathbf{A^*}$ and $\mathbf{S^*}$ during the iterations} (contrary to GMCA, which does not require to find a good estimate of $\mathbf{S^*}$ for the interpretation to be verified). In practice, using directly this strategy within a PALM algorithm therefore does not yield good estimates because when initialized with a random initialization the algorithm never becomes close to such good estimates.
%This observation must however be tempered:
%\begin{itemize}
% \item It only holds when and if PALM has converged towards good $\mathbf{A}$ \emph{and} $\mathbf{S}$;
% \item The projection is not performed in the same space as GMCA, for which the noise is back-projected in the \emph{source} space. Both spaces however merge when $\mathbf{A}$ is orthogonal.
%\end{itemize}
%Finally, it has to be noted that the thresholds become fixed after a given number of iterations to ensure the convergence.
%\subsection{Experimental protocol}
%\label{sec:ExpProtKmadPALM}
%The mixing and the metric on the separation quality are the same as in sec. \ref{ExpProt2thrd}. The automatic parameter setting described in the previous subsection is used inside of a PALM algorithm for different $k$ values.
%\subsection{Empirical performances of the heuristic}
%The results are displayed in Fig \ref{fig_kmadHybES}. The low values of $C_A$, reaching at most 6dB, are to be compared with the 39 dB obtained during the exhaustive parameter search in \ref{ExpProt2thrd}. While efficient in GMCA, the MAD heuristic leads to a bad separation quality due to the different space in which the thresholding is performed (cf. \ref{MotivHeur}), yielding a different role to the thresholds. To better understand this role, lets assume that the algorithm is initialized with the true mixing matrix $\mathbf{A}^*$ \emph{only} (and not with $\mathbf{S}^*$, contrary to what is described in \ref{MotivHeur}). In PALM, the use of the transpose $\mathbf{A}^{*T}$ in eq. (\ref{upS_PALM}) leads towards a re-mixing of the sources at each iteration whose goal is to gradually decrease the data fidelity term. This re-mixing however increases the sparsity term and should be impeded by the thresholds, giving to them a demixing role.\\
%On the opposite, when using GMCA, such a re-mixing between the sources does not exist due to the use of the pseudo-inverse $\mathbf{A}^{*\dagger}$ instead of $\mathbf{A}^{*T}$, which alleviate somehow the issue of finding relevant thresholds.
%Therefore, using the MAD heuristic in PALM does not results in good separations in practice, because the interpretation in terms of noise removal is less relevant than in GMCA.
\section{Combining GMCA and PALM: a hybrid strategy}
In this part, we propose to combine the best of GMCA and PALM in a two step approach. The algorithm comprehends a warm-up stage, in which GMCA is performed, followed by a refinement stage during which PALM is performed retaining as much information as possible coming from the warm-up stage.
\subsection{Motivation and full description of the algorithm}
Our approach is motivated by several remarks: i) \emph{PALM theoretical background:} in particular, the 2-step algorithm is thus proved to converge and, once the weights $\mathbf{W}$ estimated by the warm-up stage, it truly looks for a critical point of Eq.~(\ref{funcGMCA}); ii) \emph{GMCA robustness with regards to initialization}, which will lead to a robust 2-step algorithm; iii) \emph{Benefit from GMCA solution:}
\begin{itemize}
\item Since \emph{both} $\mathbf{A_{GMCA}}$ and $\mathbf{S_{GMCA}}$ are close to $\mathbf{A}^*$ and $\mathbf{S}^*$, they can be used to derive the thresholds using the MAD according to the previous section.
\item $\mathbf{S_{GMCA}}$ should already give a good approximation of the most prominent peaks. This can be exploited in the refinement stage through the introduction of reweighted L1 \cite{candes2008enhancing} using the reweighting matrix $\mathbf{W}$ in problem (\ref{funcGMCA}): $\mathbf{W}_i^j = \frac{\epsilon}{\epsilon + \frac{|\mathbf{\hat{S}}^j_i|}{\norm{\mathbf{\hat{S}}_i}{\infty}}}$,
with $\epsilon$ a small constant, $\mathbf{W}_i^j$ the coefficient of $\mathbf{W}$ corresponding to the {\it i}\textsuperscript{th} line and {\it j}\textsuperscript{th} column and $\mathbf{\hat{S}}_i$ the {\it i}\textsuperscript{th} line of $\mathbf{\hat{S}}$.
\end{itemize}
These remarks lead to the following 2-step algorithm:
{\hspace{-0.8cm}
\fbox{
\parbox{\linewidth}{
Input : $\mathbf{X}$ (data matrix)
\begin{itemize}
\item Random initialization $\mathbf{A_0}$ and $\mathbf{S_0}$
\item \emph{Warm-up stage}: \\$\mathbf{A_{GMCA}}$, $\mathbf{S_{GMCA}}$ = GMCA($\mathbf{X}$,$\mathbf{A_0}$,$\mathbf{S_0}$)
\item \emph{Refinement stage}: \\$\mathbf{A_{PALM}}$, $\mathbf{S_{PALM}}$ = PALM($\mathbf{X}$,$\mathbf{A_{GMCA}}$,$\mathbf{S_{GMCA}}$)\\ The initialization, thresholding strategy and reweighting information come from the warm-up stage.
\end{itemize}
}
}
}
%\subsection{Empirical results}
%\begin{figure}
% \begin{minipage}[b]{.46\linewidth}
% \includegraphics[width=3in]{fig4.png}
% \vspace{-0.5cm}
% \caption{Mean and quartiles of $C_A$ for 10 different initializations.}
% \label{fig_kmadHybES}
% \end{minipage}\hspace{.2in}
% \begin{minipage}[b]{.46\linewidth}
% \includegraphics[width=2.5in]{fig56.png}
% \caption{\emph{Up:} true sources. \emph{Down:} sources estimated by the two step algorithm.}
% \label{fig_SourcesChandra}
% \end{minipage}
%\end{figure}
%
%The experimental protocol is the same as described in sec. \ref{sec:ExpProtKmadPALM} except that the 2-step algorithm is used instead of PALM. The results are plotted in Fig.~\ref{fig_kmadHybES}. With values of $C_A$ higher than 33 dB, the demixing is very accurate. Compared to PALM only, the variance of results over different initializations is also much decreased, becoming almost zero, which shows the robustness of the algorithm with regards to the initialization.
\section{Experiment on realistic data}
\subsection{Data description and experimental protocol}
The goal of this part is to apply our algorithm on realistic data to show its efficiency.
The $n = 2$ sources come from simulations obtained from real data of Cassiopeia A supernova remnant.
%As displayed in Fig. \ref{fig_SourcesChandra},
They each consists in a 2D image of resolution $t = $ 128 $\times$ 128 pixels, supposed to be approximately sparse in the starlet domain. The mixing is performed through a $\mathbf{A}$ matrix drawn randomly following a standard normal distribution and modified to have unit columns. Its condition number is $C_d = 10$. There is $m = n$ observations. To increase the realism of the data and further test the algorithm, we tried three relatively low SNR values: 10, 15 and 20 dB. $k$ is set to 3, which corresponds to a classical hypothesis in terms of Gaussian noise removal.
\subsection{Empirical results}
\begin{table}[!t]
\caption{Average $C_A$ (10 different initializations) for 3 SNR values and 5 algorithms.}
\label{fig_kmadHybChandra}
\centering
\begin{tabular}{llll}
& 10 dB & 15dB & 20 dB\\
2 step & {\bf 11.57} & {\bf 16.92} & {\bf 22.09}\\
PALM & 9.38 & 10.94 & 11.01\\
GMCA & 8.87 & 12.15 & 15.87\\
EFICA & -6.92 & 5.11 & 9.41\\
RNA & -6.68 & -5.49 & 6.27\\
\end{tabular}
\end{table}
The mean of of the mixing matrix criterion $C_A$ \cite{bobin2015sparsity} is displayed in Table~\ref{fig_kmadHybChandra}. The 2-step approach always achieve better results than the two classical BSS algorithms with which we performed the comparison, namely Relative Newton Algorithm (RNA) and Eficient FastICA (EFICA). It also outerpeforms both GMCA and a PALM using directly the MAD heuristic, being always better by at least 2 dB than the best of them.\\
In addition to the results displayed in Fig.~\ref{fig_kmadHybChandra}, the standard deviation of $C_A$ over different initializations is almost 0, which shows the robustness of the algorithm.
\section*{Conclusion}
In this work, we introduce a 2-step strategy combining PALM with robust heuristic methods such as GMCA. Beyond improving the robustness of PALM-based implementations with respect to initialization, the regularization parameters can be automatically set in the proposed approach. Numerical experiments on realistic data demonstrate a high separation quality and good robustness on mixings with low SNR.
% use section* for acknowledgment
\section*{Acknowledgment}
This work is supported by the European Community through the grant LENA (ERC StG - contract no. 678282).
%% You can make the bibliography smaller
\begin{thebibliography}{10}
\bibitem{comon2010handbook}
Pierre Comon and Christian Jutten.
\newblock {\em Handbook of Blind Source Separation: Independent component
analysis and applications}.
\newblock Academic {P}ress, 2010.
\bibitem{gillis2012accelerated}
Nicolas Gillis and Fran{\c{c}}ois Glineur.
\newblock Accelerated multiplicative updates and hierarchical {ALS} algorithms
for nonnegative matrix factorization.
\newblock {\em Neural {C}omputation}, 24(4):1085--1105, 2012.
\bibitem{bobin20085}
J{\'e}r{\^o}me Bobin, Jean-Luc Starck, Yassir Moudden, and Mohamed~Jalal
Fadili.
\newblock Blind source separation: The sparsity revolution.
\newblock {\em {A}dvances in {I}maging and {E}lectron {P}hysics},
152(1):221--302, 2008.
\bibitem{zibulevsky2001blind}
Michael Zibulevsky and Barak~A Pearlmutter.
\newblock Blind source separation by sparse decomposition in a signal
dictionary.
\newblock {\em Neural {C}omputation}, 13(4):863--882, 2001.
\bibitem{bronstein2005sparse}
Alexander~M Bronstein, Michael~M Bronstein, Michael Zibulevsky, and Yehoshua~Y
Zeevi.
\newblock Sparse {ICA} for blind separation of transmitted and reflected
images.
\newblock {\em International {J}ournal of {I}maging {S}ystems and
{T}echnology}, 15(1):84--91, 2005.
\bibitem{li2006underdetermined}
Yuanqing Li, Shun-Ichi Amari, Andrzej Cichocki, Daniel~WC Ho, and Shengli Xie.
\newblock Underdetermined blind source separation based on sparse
representation.
\newblock {\em {IEEE} Transactions on {S}ignal {P}rocessing}, 54(2):423--437,
2006.
\bibitem{starck2010sparse}
Jean-Luc Starck, Fionn Murtagh, and Jalal~M Fadili.
\newblock {\em Sparse {I}mage and {S}ignal {P}rocessing: {W}avelets,
{C}urvelets, {M}orphological {D}iversity}.
\newblock {Cambridge University Press}, 2010.
\bibitem{candes2008enhancing}
Emmanuel~J Candes, Michael~B Wakin, and Stephen~P Boyd.
\newblock Enhancing sparsity by reweighted $\ell_1$ minimization.
\newblock {\em Journal of Fourier analysis and applications}, 14(5-6):877--905,
2008.
\bibitem{tseng2001convergence}
Paul Tseng.
\newblock Convergence of a block coordinate descent method for
nondifferentiable minimization.
\newblock {\em Journal of {O}ptimization {T}heory and {A}pplications},
109(3):475--494, 2001.
\bibitem{bolte2014proximal}
J{\'e}r{\^o}me Bolte, Shoham Sabach, and Marc Teboulle.
\newblock Proximal alternating linearized minimization for nonconvex and
nonsmooth problems.
\newblock {\em Mathematical {P}rogramming}, 146(1-2):459--494, 2014.
\bibitem{bobin2015sparsity}
Jerome Bobin, Jeremy Rapin, Anthony Larue, and Jean-Luc Starck.
\newblock Sparsity and adaptivity for the blind separation of partially
correlated sources.
\newblock {\em {IEEE} {T}ransanctions on {S}ignal {P}rocessing},
63(5):1199--1213, 2015.
\end{thebibliography}
\end{document}