Nonnegative Matrix Factorization in the Component-Wise L1 Norm for Sparse Data

Extracted source text · arXiv 2603.29715

\let\ORIlabel\label
\let\ORIrefstepcounter\refstepcounter
\AddToHook{package/hyperref/before}{
   \let\label\ORIlabel 
   \let\refstepcounter\ORIrefstepcounter}
   
\documentclass{siamart190516}
\usepackage[utf8]{inputenc}

\setlength{\textwidth}{170mm}
\setlength{\textheight}{220mm}
\setlength{\topmargin}{-5mm}
\setlength{\oddsidemargin}{-5mm}
\setlength{\evensidemargin}{-5mm}

\usepackage{amsmath,amssymb,amsfonts,color,multirow,comment,pgfplots,pgfplotstable,tikz}
%\usepackage{natbib}

\usepackage{longtable}
\usepackage{booktabs}
\usepackage{graphicx}
\usetikzlibrary{plotmarks}
\usetikzlibrary{positioning}
\usetikzlibrary{arrows.meta}
\usepgfplotslibrary{patchplots}
\usepgfplotslibrary{groupplots}
\usepackage{grffile}
\usepackage{url}
\usepackage{hyperref}
\usepackage{cleveref}
\usepackage{algorithm,algpseudocode}
\usepackage{color}
\usepackage{mdframed}
\newcommand{\comp}[1]{$\mathcal{O}(#1)$}
%\newtheorem{theorem}{Theorem}
%\newtheorem{example}{Example}

\pgfplotsset{compat=1.17}

\definecolor{greeng}{rgb}{0.0, 0.5, 0.4}
\newcommand{\gs}[1]{{\color{greeng} (\textbf{GS:} #1)}}
\newcommand{\name}{sCD}

\definecolor{brightpink}{rgb}{1.0, 0.0, 0.5}
\newcommand{\ngc}[1]{{\color{brightpink} (\textbf{NG:} #1)}}
\newcommand{\ngi}[1]{{{\color{brightpink} #1}}}

% *** Arnaud
\definecolor{forestgreen(web)}{rgb}{0.13, 0.55, 0.13}
\newcommand{\av}[1]{{\color{forestgreen(web)} (\textbf{AV:} #1)}}

\DeclareMathOperator{\supp}{supp}
\DeclareMathOperator{\nnz}{nnz}
\DeclareMathOperator{\rank}{rank}
\DeclareMathOperator{\argmin}{argmin}
\DeclareMathOperator{\argmax}{argmax}
\DeclareMathOperator{\var}{var}

\title{Nonnegative Matrix Factorization in the \\  Component-Wise L1  Norm for Sparse Data\thanks{GS and NG acknowledge the support by the European Union (ERC consolidator, eLinoR, no 101085607). The work of GS\ was partially supported by INdAM-GNCS through Progetti di Ricerca. Kévin Dubrulle is a FRIA grantee of the Fonds de la Recherche Scientifique - FNRS.}}
\author{Giovanni Seraghiti\thanks{Corresponding author. University of Mons, Rue de Houdain 9, 7000 Mons, Belgium, and 
 Dipartimento di Ingegneria Industriale, Universit\`a degli Studi di Firenze, Viale Morgagni 40/44, 50134 Firenze, Italia. Member of the INdAM Research Group GNCS.}   %\and{Kevin Dubrulle, Arnaud Vandaele, Nicolas Gillis} \thanks{University of Mons, Rue de Houdain 9, 7000 Mons}\\ 
 \and
Kévin Dubrulle\thanks{University of Mons, Rue de Houdain 9, 7000 Mons, Belgium.} 
\and
Arnaud Vandaele\footnotemark[3]
\and
Nicolas Gillis\footnotemark[3]
}
\date{March 2021}
\begin{document}
\maketitle

\begin{abstract}
   Nonnegative matrix factorization (NMF) approximates a nonnegative matrix, $X$, by the product of two nonnegative factors, $WH$, where $W$ has $r$ columns and $H$ has $r$ rows. In this paper, we consider NMF using the component-wise L1 norm as the error measure (L1-NMF), which is suited for data corrupted by heavy-tailed noise, such as Laplace noise or salt and pepper noise, or in the presence of outliers. Our first contribution is an NP-hardness proof for L1-NMF, even when $r=1$, in contrast to the standard NMF that uses least squares. Our second contribution is to show that  L1-NMF strongly enforces sparsity in the factors for sparse input matrices, thereby favoring  interpretability. However, if the data is affected by false zeros, too sparse solutions might degrade the model. Our third contribution is a new, more general, L1-NMF model for sparse data, dubbed weighted L1-NMF (wL1-NMF), where the sparsity of the factorization is controlled by adding a penalization parameter to the entries of $WH$ associated with zeros in the data. The fourth contribution is a new coordinate descent (CD) approach for wL1-NMF, denoted as sparse CD (\name), where each subproblem is solved by a weighted median algorithm. To the best of our knowledge, \name\ is the first algorithm for L1-NMF whose complexity scales with the number of nonzero entries in the data, making it efficient in handling large-scale, sparse data. We perform extensive numerical experiments on synthetic and real-world data to show the effectiveness of our new proposed model (wL1-NMF) and algorithm (\name).
\end{abstract}

\begin{keywords}
    low-rank matrix approximations, nonnegative matrix factorization, robustness, least absolute deviation, sparsity, missing data. 
\end{keywords}

%\ngc{Can you clean the mess of the files? Put every .tex and data files in a separate folder :-)?}

\section{Introduction}
Given a nonnegative matrix $X \in \mathbb{R}_+^{m \times n}$ and a factorization rank $r$, 
the goal of nonnegative matrix factorization (NMF) is to find $W \in \mathbb{R}_+^{m \times r}$ and $H \in \mathbb{R}_+^{r\times n}$ such that $X \approx WH$. 
NMF can be formulated as follows
\begin{equation}
    \min_{W \geq 0, \ H\geq 0} \; \; d(X, WH),
    \label{eq:gen_NMF}
\end{equation}
where $d: \mathbb{R}_+^{m \times n} \times \mathbb{R}_+^{m \times n} \to \mathbb{R}_+$ is an error measure. 
Two popular examples~\cite{lee1999learning} are (1)~Frobenius NMF (FroNMF), which is the maximum likelihood estimator (MLE) for additive Gaussian noise,  that minimizes the least squares error, $d(X,WH) = \lVert X - WH \rVert_F^2 = \sum_{i=1}^m \sum_{j=1}^n(X-WH)_{i,j}^2$,  
and (2)~Kullback-Leibler NMF (KL-NMF), which is the MLE when $X$ follows a Poisson distribution of parameters $WH$, 
and the error measure is the Kullback-Leibler divergence, $d(X,WH)=\sum_{i=1}^m \sum_{j=1}^nX_{i,j}\log \left( \frac{X_{i,j}}{(WH)_{i,j}}\right)-X_{i,j}+(WH)_{i,j}$. 
Although FroNMF, KL-NMF, and their extensions have been widely studied, see, e.g.,~\cite{CZA09, gillis2020nonnegative}, 
it is well known that they are sensitive to outliers. Moreover, FroNMF and KL-NMF exhibit poor performance when data are affected by long-tailed noise such as Laplacian noise or salt and pepper noise. To overcome these limitations, more robust NMF models have been proposed. A popular choice for robust NMF is the L21 norm~\cite{kong2011robust}, that is, 
%\gs{Let's put L21-NMF in line as well? Such that we leave as equations only the model that we investigate?}
$d(X, WH)=\lVert X- WH \rVert_{2,1}=\sum_{j=1}^n \sqrt{ \sum_{i=1}^m (X-WH)^2_{i,j}}$. 
Among robust NMF models, much less attention has been devoted to L1-NMF, which models additive Laplace noise, where the error measure is the component-wise L1 norm:
\begin{equation} 
d(X, WH)= ||X-WH||_1 =\sum_{i=1}^m \sum_{j=1}^n |X-WH|_{i,j}. 
\label{eq:l1NMF}
\end{equation}
This is due to the fact that solving L1-NMF requires more involved optimization strategies because its objective is not differentiable.  
%than those used for solving L21-NMF. 
In fact, if we remove the nonnegativity constraints in~\eqref{eq:l1NMF}, the low-rank matrix approximation (LRMA) problem in the L1 norm is known to be NP-hard, even  for $r=1$~\cite{gillis2018complexity}. 
%To the best of our knowledge, this paper is the first to prove that L1-NMF is NP-hard also under the nonnegativity constraint on the factors. 


\paragraph{Outline and Contributions} 

In Section~\ref{sec:motiv}, we further motivate L1-NMF, and briefly review the most relevant works on the topic. 
In Section~\ref{sec:complexity}, we prove the NP-hardness of rank-one L1-NMF, that is, \eqref{eq:l1NMF} when $r=1$. 
In Section~\ref{sec:sparsity_inducing}, we analyze the sparsity-inducing property of L1-NMF on its factors  when the original data is sparse: we show that solutions of L1-NMF are expected to be significantly sparser than that of L2-NMF. 
This motivates the introduction of the wL1-NMF model in Section~\ref{sec:w_ell_1_nmf}, a generalization of L1-NMF, in cases where the input data is too sparse due, for example, to the presence of false zeros. 
In Section~\ref{sec:l1_for_sparse}, we present our new algorithm, \name, for wL1-NMF.   
We provide extensive numerical experiments in Section~\ref{sec:num_res}, comparing our new model wL1-NMF with other NMF models and our new algorithm, \name, with the state of the art algorithms for L1-NMF.

%\newpage 

Let us summarize our four main contributions: 

\begin{enumerate}
    \item We show that L1-NMF is NP-hard when $r=1$ (Theorem~\ref{lem:complexityyesyes}).

    \item %Although computing L1-NMF is not a trivial task, 
    We show that  L1-NMF has the fundamental property that the sparsity of $X$ is strongly reflected in the factors $W$ and $H$; see Section~\ref{sec:sparsity_inducing}.

    \item  Even though sparsity might favor the interpretability of the factorization, excessively sparse solutions might be inappropriate,  
which may happen for zero-inflated data, such as word-document count matrices. 
For this reason, we propose a generalization of L1-NMF, referred to as weighted L1-NMF (wL1-NMF), where a parameter $\lambda$ is introduced to downweight the importance of zeros in the input data. 
%regulate the number of zeros in the data that are approximated by zeros in the product $WH$. 
%This model is particularly suitable for applications where data are affected by false zeros.  
wL1-NMF uses the following error measure
\begin{equation}
    %d(X,WH,\lambda)=
    \ell(X,W,H,\lambda) := \sum_{(i,j) \in \kappa^+} \lvert X -WH \rvert_{i,j} + \lambda \sum_{(i,j) \in \kappa^0} \lvert WH \rvert_{i,j},
    \label{eq:weight_l1}
\end{equation}
where $\lambda \in [0,1]$, 
    $\kappa^+=\left\{ (i,j) \ | \ X_{i,j}>0 \right\}$ 
    and 
    %\quad \text{and } \quad 
    $\kappa^0=\left\{ (i,j) \ | \ X_{i,j}=0 \right\}$.
    %\label{eq:def_set_kappa}
%\end{equation}
wL1-NMF covers L1-NMF as a special case when $\lambda=1$, while the case $\lambda=0$ corresponds to a nonnegative low-rank matrix completion problem where zero entries correspond to missing entries. 
%no prior knowledge is assumed about the missing values (zeros). 

    \item wL1-NMF can be computed using a coordinate descent algorithm that solves one scalar L1 regression subproblem, also known as least absolute deviation (LAD), for each entry of $W$ and $H$ in an alternating fashion, with a computational cost per iteration of $\mathcal{O}(mnr\log(mn))$. We propose a new variant, dubbed \name, that exploits the nonnegativity and the sparsity of the original matrix to reduce the computational cost to $\mathcal{O}(r\nnz(X)\log(\nnz(X)))$, where $\nnz(X)$ denotes the number of nonzero entries of $X$.  
\end{enumerate}

 



%As a first contribution, we prove that L1-NMF is NP-hard. Secondly, we establish a formal relation between the sparsity in the data and the sparsity in the solutions of the L2 and the L1 one dimensional regression problems, showing that the L1 norm strongly enforces sparsity in the solution. As a consequence, if the original data are sparse,  the L1-NMF approximation computed by the CD method is sparser than the Fro-NMF approximation. The third contribution is a new weighted L1-NMF (wL1-NMF) model for sparse data that covers the L1-NMF and the L1 nonnegative matrix completion models as particular cases. The fourth contribution is the \name\ algorithm, which is a new block coordinate descent (CD) approach for wL1-NMF (and L1-NMF) that can efficiently handle large, sparse data. The novelty of our approach is that \name\ has complexity that scales linearly with the number of non-zero entries in the original data, reducing the running time per iteration of the original CD approach from $\mathcal{O}(mnr\log(mn))$ to $\mathcal{O}(r\nnz(X)\log(\nnz(X)))$, where $m$ and $n$ are the dimensions of the problem and $\nnz(X)$ is the number of non-zero in the original data matrix $X$. 
%Our approach can be easily generalized to other NMF models and we regard it as future work.



%Moreover, the \name\ algorithm naturally extends to Binary Matrix Factorization (BMF) without explicitly including the binary constraint; thus solving L1-BMF at the same cost of L1-NMF. 

% \subsection{Outline} This paper is structured as follows: in Section~\ref{sec:motiv} we motivate the L1-NMF and wL1-NMF models; in Section~\ref{sec:l1_first} we briefly review the most relevant works on LAD and on L1-NMF; in Section~\ref{sec:complexity} we prove NP-hardness of L1-NMF; in Section~\ref{sec:sparsity_inducing}, we analyze the sparsity-inducing property of the CD approach on the factors of L1-NMF when the original data is sparse; in Section~\ref{sec:w_ell_1_nmf} we derive the wL1-NMF model, by the maximum a posteriori estimation in the presence of Laplace noise; in Section~\ref{sec:l1_for_sparse}, we present our new algorithm \name; in Section~\ref{sec:num_res}, we provide extensive numerical experiments comparing our new model wL1-NMF with other NMF models and our new approach, \name, with the state of the art algorithms for L1-NMF.


\section{L1-NMF: motivations and previous works} 
\label{sec:motiv} 

NMF finds applications in a variety of fields, such as feature extraction~\cite{lee1999learning,guillamet2002non}, topic modeling and document classification~\cite{shahnaz2006document,ding2008equivalence}, hyperspectral unmixing~\cite{bioucas2012hyperspectral,ma2013signal}; see~\cite{CZA09, gillis2020nonnegative} for books on the topic. 
An appropriate choice of the error measure in~\eqref{eq:gen_NMF} is crucial, and it depends on the type of data and the task at hand. Let us illustrate this with a simple example, showing the different low-rank approximations obtained by FroNMF, KL-NMF, L21-NMF and L1-NMF of the same matrix $X$. We constructed a biadjacency matrix representing the relationships between two groups of objects. 
The $(i,j)$th element of $X$ is equal to 1 if there is a connection between the object represented by the $j$th column and the object represented by the $i$th row. A typical application is topic modeling, where $X$ is a word-by-document matrix with $X_{i,j}=1$ if word $i$ is contained in document $j$, and $X_{i,j}=0$ otherwise. A standard problem in topic modeling is detecting groups of documents sharing the same topic (that is, same subset of words), which can be addressed by computing the NMF of the word-by-document matrix~\cite{lee1999learning}. 
\begin{comment}
Some of the algorithms for L1-LRMA can be extended to L1-NMF. This is the case for the alternate optimization strategy by  Q. Ke and T. Kanade  and the augmented Lagrangian approach by Zheng. Specifically, the alternate optimization solves both the subproblem for $W$ and $H$ either componentwise by the weighted median algorithm or using convex programming~\cite{ke2003robust,ke2005robust,guan2012mahnmf}. However, this approach is not efficient for large-scale problems, and it does not exploit the sparsity of the data. N. Guan et al. introduced also Nesterov's smoothing method for L1-NMF, that combines Nesterov's smoothing technique and Nesterov's accelerated gradient. This algorithm has been shown to achieve good results on relatively large dimension datasets but still does not benefit from the data being possibly sparse. Recently, M. Rahimi et al. proposed a convergent projected subgradient method for paraconvex functions with different choices of step sizes that have been successfully tested on L1-NMF. Despite its theoretical guarantees, the subgradient approach might be slow in practical applications and does not exploit the sparsity of the data. 
\end{comment}  
We consider a collection of 6 documents, from a dictionary of 6 words, divided into two groups of three elements each, containing documents that share exactly three words.
We also add a few common words that are in documents from both classes. The word-by-document matrix is 
\begin{equation*}
    X=\begin{pmatrix}
    1  &  1  &  1  &  0 &   0   & 1 \\
    1  &  1  &  1  &  0  &  0  &  0 \\
    1  &  1  &  1  &  0  &  1  &  0 \\
    0  &  1  &  0  &  1 &  1  &  1 \\
     0  &  0  &  0  &  1 &  1  &  1 \\
   1  &  0  &  0  &  1 &  1  & 1 
\end{pmatrix}.
\end{equation*} 
We compute rank-2 NMFs using the hierarchical alternate least squares algorithm (HALS, a coordinate descent method, similar to \name\ but for least squares) 
 for FroNMF~\cite{cichocki2007hierarchical,li2009fastnmf,gillis2020nonnegative}, the multiplicative update (MU) approach for KL-NMF~\cite{lee1999learning} and L21-NMF~\cite{kong2011robust}, and our new algorithm \name\ for L1-NMF (see Section~\ref{sec:l1_for_sparse}). Each algorithm runs for 30 iterations and is initialized with 3 iterations of HALS on random factors. 
The results are as follows: \vspace{0.2cm}

\begin{minipage}{0.45\textwidth}
\begin{center}
FroNMF \vspace{-0.2cm} 
\begin{equation*}
    %\text{FroNMF}=
    \begin{pmatrix}
    1.03  & 1.03 &  0.98 & 0.20 & 0.40 & 0.40\\
    0.98 & 0.98 & 0.96 & 0     &  0.20 & 0.20 \\
    1.03 &  1.03 &  0.98 & 0.20 & 0.40 & 0.40 \\
    0.40 & 0.40 & 0.20 & 0.98 & 1.03 &  1.03\\
     0.20 &  0.20 &  0   &    0.96 &  0.98 &  0.98 \\
    0.40 & 0.40 & 0.20 & 0.98 &  1.03  & 1.03
\end{pmatrix},
\end{equation*}
\end{center}
\end{minipage}
\begin{minipage}{0.45\textwidth}
\begin{center}
KL-NMF \vspace{-0.2cm}  
\begin{equation*}
    %\text{KL-NMF}
    \begin{pmatrix}
    0.86 &    1.04 &   0.78  &  0.31  &  0.59  &  0.41\\
    0.78 &    1.09 &    0.82 &    0 &    0.30 &    0\\
    1.04 &    1.45 &    1.09 &    0 &    0.41 &    0 \\
    0.59 &    0.41 &   0.30 &   0.78  &    0.86 &    1.04 \\
    0.31 &    0 &    0 &    0.82 &    0.78 &    1.09 \\
    0.41 &  0 &    0 &  1.09 &    1.04 &    1.45
\end{pmatrix},
\end{equation*}
\end{center}
\end{minipage}

\vspace{0.5cm}
\begin{minipage}{0.45\textwidth}
\begin{center}
L21-NMF \vspace{-0.2cm} 
\begin{equation*}
   %\text{L21-NMF}=
   \begin{pmatrix}
    1  &  1  &  1  &  0.08 &   0.35   & 0.08\\
    0.98  &  0.98  &  1  &  0  &  0.27  &  0 \\
    1.01  &  1.01  &  1  &  0.10  &  0.37  &  0.10 \\
    0.34  &  0.34  &  0  &  1 &  1.03  &  1.03 \\
     0.33  &  0.33  &  0  &  0.97 &  0.98  &  0.99 \\
   0.34  &  0.34  &  0  &  1 &  1.02  &  1.03
\end{pmatrix}, 
\end{equation*}
\end{center}
\end{minipage}
\begin{minipage}{0.45\textwidth}
\begin{center}
L1-NMF \vspace{-0.2cm} 
\begin{equation*}
    %\text{L1-NMF}=
    \begin{pmatrix}
    1  &  1  &  1  &  0 &   0   & 0\\
    1  &  1  &  1  &  0  &  0  &  0 \\
    1  &  1  &  1  &  0  &  0  &  0 \\
    0  &  0  &  0  &  1 &  1  &  1 \\
     0  &  0  &  0  &  1 &  1  &  1 \\
   0  &  0  &  0  &  1 &  1  &  1
\end{pmatrix}.
\end{equation*}
\end{center}
\end{minipage}
\vspace{0.2cm}

\noindent While FroNMF, KL-NMF and L21-NMF spread the extra-cluster information among the non-diagonal blocks, L1-NMF places more weight on the intra-cluster information and interprets the few intersections between documents from different clusters as outliers that can be neglected. 
L1-NMF provides the sparsest decomposition, favoring the interpretability of the model. 
See Section~\ref{sub:mnist_NMF_gen} for a similar example on images.  

%Moreover, contrary to the other NMF models, L1-NMF has the fundamental property that the sparsity in the data is preserved in the factors
 
%In the related context of regression problems with outliers, alternatives to the L2 norm have been proposed to favor robustness, starting from weighted L2 formulations~\cite{black1996unification} to the more robust L1 norm. Several works demonstrate that L1 is much more robust to outliers than L2 regression, but it is more difficult to solve~\cite{portnoy1997gaussian,chen2008analysis,ke2005robust}. The L1 regression problem or LAD was introduced by Boscovich and studied by Laplace. However, due to its nondifferentiability and contrary to L2 regression, it lacks a closed-form solution. Edgeworth, in 1887, with his weighted median algorithm, was the first to propose an iterative method for L1 regression~\cite{edgeworth1888xxii}. Much less attention has been devoted to regression problems in the sparse setting. It is not clear, in fact, how sparse data affects the sparsity of the L2 or L1 regression estimates. %Later, simplex-like methods arise as a solution strategy, exploiting the connection of L1 regression with linear programming~\cite{harris1950regression,charnes1955optimal,barrodale1973improved,armstrong1979revised}. Other significant contributions are: the iterative reweighted least squares (IRLS) algorithm,~\cite{schlossmacher1973iterative,dempster1977maximum,phillips2002least}, an active set method by Bloomfield and Steiger~\cite{bloomfield1980least}, and an interior point method by Zhang~\cite{zhang1993primal}. More recent contributions introduced randomization and approximation techniques to significantly speed up algorithms, resulting in methods achieving relative error approximation with high probability~\cite{clarkson2005subgradient,meng2013low,clarkson2014sketching}.

It is well known that the L1 norm is much more robust to outliers than the L2 norm in the context of regression problems~\cite{portnoy1997gaussian,chen2008analysis,ke2005robust}, or in low-rank matrix approximation (LRMA), where L1-LRMA and robust PCA are much more robust than  PCA~\cite{croux1998robust,ke2005robust,kwak2008principal,markopoulos2013some,markopoulos2014optimal}. However, the L1 norm yields optimization problems that are more difficult to solve than those involving the L2 norm. Therefore, LRMA using the component-wise L1 norm (L1-LRMA) is a relatively unexplored area. Let us mention important works addressing L1-LRMA: the alternating  optimization approach by Ke and Kanade~\cite{ke2003robust,ke2005robust}, 
the Wiberg algorithm by Eriksson and Van Den Hengel~\cite{eriksson2010efficient}, and the augmented Lagrangian approaches by Zheng et al.~\cite{zheng2012practical}. Another relevant contribution is the work of Song et al.~\cite{song2017low} who introduced an algorithm for L1-LRMA with provable approximation guarantees that scales with the number of nonzero entries in the data using sketching (that is, they use random projections of the data). 

Similarly, few NMF models consider the L1 norm directly, and robust NMF usually includes other error measures that are easier to optimize, such as the L21 norm~\cite{kong2011robust,huang2014robust}, the more general L2$p$ norm ($p>0$)~\cite{li2017robust}, or functions that smoothly interpolate between the Frobenius norm and the L1-norm, for example, the Huber function~\cite{du2012robust,guo2021modified}. Another strategy is to approximate $X$ by $WH-S$, where $S$ is a sparse matrix intended to capture the sparse additive noise in the data~\cite{zhang2011robust,shen2014robust}. 

Among the notable contributions on L1-NMF, Rahimi et al.~\cite{rahimi2024projected} showed that L1-NMF, despite being neither smooth nor convex, is paraconvex, which is a generalization of the class of weakly convex functions. Moreover, they proposed a convergent projected subgradient method for paraconvex functions, optimizing both $W$ and $H$ simultaneously, which was successfully tested on L1-NMF. Despite having theoretical guarantees, the subgradient approach might be slow in practical applications and does not exploit the property of L1-NMF to be convex with respect to each variable individually. 
Thus, block coordinate descent (BCD) is a valid alternative to compute an approximate minimizer of~\eqref{eq:l1NMF} by sequentially optimizing over $W$ and $H$ and exploiting the convexity of the subproblems, as done in~\cite{ke2003robust,ke2005robust} in the unconstrained case. 
Moreover, the subproblem for $H$ (or $W$) is separable by columns (or rows), and equivalent to $n$ (or $m$) multivariate L1 regression problems, a.k.a.\ LAD subproblems. %one for each column $X_{:,j} \in \mathbb{R}^r$ of $H$, for $j=1,\dots,n$.
%\begin{equation}
%    H_{:,j}^{k+1}=\argmin_{H_{:,j}\geq0} \lVert X_{:,j} - W^{k+1} H_{:,j} \rVert_1, \quad \text{ for } j=1,\dots,n,
%    \label{eq:vec_sub_h}
%\end{equation}
%where $X_{:,j} \in \mathbb{R}^m$ denotes the $j$th column of $X$.
Ke and Kanade~\cite{ke2003robust} showed that convex programming can be used to solve each LAD subproblem, but it yields algorithms that are not scalable for large dimensions~\cite{ke2005robust}. To address this limitation, Guan et al.~\cite{cao2023manhattan} applied Nesterov smoothing to the nonsmooth LAD subproblem. This strategy consists of solving a smooth approximation of the dual problem and projecting the solution back to the primal space. The obtained solution is an approximate stationary point of the primal function~\cite{nesterov2005smooth}.
Another approach is to split each multivariate LAD problem into $r$ smaller one-dimensional LAD subproblems, one for each component $H_{i,j}$ (or $W_{j,i}$) of $H_{:,j}$ (or $W_{j,:})$, with $i=1,\dots,r$ as suggested in~\cite{ke2003robust,guan2012mahnmf,gillis2011dimensionality}. This is a coordinate descent (CD) method for L1-NMF. 

Let us provide more details about the one-dimensional subproblems in L1-NMF, as this will be key in our theoretical analysis and algorithmic design. For simplicity, we focus on the update of the factor $H$.
Each one-variable subproblem consists of minimizing a piece-wise linear function of the form
\begin{equation}
    \argmin_{\alpha\geq 0} f(\alpha)=  \argmin_{\alpha\geq 0} \|x-\alpha y\|_1=\sum_{s=1}^m \lvert x_s -\alpha y_s \rvert=\sum_{s=1}^m \lvert y_s \rvert \left\lvert \frac{x_s}{y_s} -\alpha \right\rvert. 
     \label{eq:gen_scalar_prob}
\end{equation}
We show an example in Figure~\ref{fig:ex_piecewise}. The function in~\eqref{eq:gen_scalar_prob} is a convex, nondifferentiable function, where at least one global minimum coincides with one of the breakpoints $x_s/y_s$. 

\begin{figure}[h]
\centering
\begin{tikzpicture}[scale=0.85]

% Axes
\draw[->] (-6.5,0) -- (8.5,0) node[below right] {$\alpha$};
\draw[->] (0,0) -- (0,5.5) node[left] {$f(\alpha)$};

% Piecewise linear convex function (minimum at alpha*=2)
\draw[thick, blue]
(-4,5) -- (-2,3) -- (0,1.8) -- (2,1) -- (4,1.6) -- (6,2.4) -- (8,4);

% x-axis breakpoints
\foreach \x/\lab in {
-4/{$\frac{x_1}{y_1}$},
-2/{$\frac{x_2}{y_2}$},
 0/{$\frac{x_3}{y_3}$},
 2/{$\frac{x_4}{y_4}$},
 4/{$\frac{x_5}{y_5}$},
 6/{$\frac{x_6}{y_6}$},
 8/{$\frac{x_7}{y_7}$}
}{
\draw[fill=blue] (\x,0) circle (0.06);
\node[below] at (\x,0) {\lab};
}

% function values
\foreach \x/\y in {
-4/5,
-2/3,
 0/1.8,
 2/1,
 4/1.6,
 6/2.4,
 8/4
}{
\draw[fill=blue] (\x,\y) circle (0.06);
}


\end{tikzpicture}

\caption{Example of a one-dimensional nonnegative least absolute deviation (LAD) problem in~\eqref{eq:gen_scalar_prob}.}
\label{fig:ex_piecewise}
\end{figure}

A naive strategy for finding a global minimum of~\eqref{eq:gen_scalar_prob} is to evaluate it at all the breakpoints and take the one that corresponds to the lowest value, or zero if the optimal breakpoint is negative. This approach has a complexity of $\mathcal{O}(m^2)$ and is not used in practice. A faster approach, running in $\mathcal{O}(m\log(m))$ exists, consists of identifying an optimal solution at the breakpoint after which the sign of the slope of the function changes. 
%if the optimal breakpoint is negative, the solution to the scalar problem is zero. % too much details ;-) 
The slopes of the function at every breakpoint are obtained from the vector $y$. The algorithm implementing this strategy is known as the constrained weighted median algorithm~\cite{edgeworth1888xxii}; see Algorithm~\ref{alg:constr_w_m}. 
We also mention that there exists a more involved algorithm for computing a solution to the weighted median problem running in linear time $\mathcal{O}(m)$~\cite{gurwitz1990weighted}.
%\gs{I referred to it so many times in the following sections that it was difficult for me to place the algorithm somewhere else, I left it here for now}
 \begin{algorithm}
	\caption{\quad \texttt{$\alpha$ = constrained\_weighted\_median(x,y)}}\label{algo_cons_weightedmedian}
	\begin{algorithmic}[1]
		\item INPUT: $x\in \mathbb{R}^{m}$, $y\in \mathbb{R}^{m}$
		\item OUTPUT: $\arg\min_{\alpha\geq 0}\sum_s |x_s-\alpha y_s|$
		\State $S\gets \{\frac{x_s}{y_s}  \ | \ y_s \neq 0\}$
		\State $[S,\text{indices}]\gets \texttt{sort}(S)$ \label{linesort}
		\State $y \gets y(\text{indices})$
		\State $k \gets$ smallest index such that $\sum_{s=1}^k y_s\geq \frac{\sum_{s=1}^m y_s}{2}$ \label{lineindex}
		\State $\alpha\gets\max(0,S_k)$
	\end{algorithmic}
    \label{alg:constr_w_m}
\end{algorithm}



\section{Computational complexity of L1-NMF} 
\label{sec:complexity}

Fro-NMF is NP-hard when $r$ is part of the input~\cite{vavasis2010complexity}, but it can be solved in polynomial time when $r=1$, via the SVD, by combining Eckart-Young and Perron-Frobenius theorems.
Unfortunately, L1-NMF is significantly more complex in this case: we show in this section that L1-NMF is NP-hard even when $r=1$. %even for a binary input matrix.  


The computational complexity of the best rank-one  approximation in the L1 norm was investigated in \cite{gillis2018complexity}, where it was proved that the unconstrained rank-one problem,  
\begin{equation}\label{eq:l1lrageneral}
\min_{u\in\mathbb{R}^s,v\in\mathbb{R}^t}\ \|M-uv^\top\|_1, 
\end{equation} 
is NP-hard. 
The proof essentially follows a two-step approach. 
It first shows that for any matrix $M\in\{\pm1\}^{s\times t}$, the problem~\eqref{eq:l1lrageneral} admits an optimal solution $(u^*,v^*)$ such that $u^*\in\{\pm1\}^s$ and $v^*\in\{\pm1\}^t$.  
This property is then used to encode an NP-hard problem, namely \textsc{Max-Cut}. 

Unfortunately, the first argument does not directly extend to the nonnegative setting, where the input matrix is nonnegative and the factors are constrained to be nonnegative. %as incorrectly stated in~\cite[Theorem~6.13]{gillis2020nonnegative}.
%As a preliminary step, we show that the reduction strategy of \cite{gillis2018complexity} cannot be applied directly in the nonnegative case by exhibiting a counterexample.
%\subsection{Counterexample}
% In fact, consider the matrix
% \[
% M=\begin{pmatrix}
%  1& 1&-1& 1&-1\\
% -1& 1&-1& 1& 1\\
% -1& 1& 1& 1&-1\\
%  1&-1& 1& 1& 1
% \end{pmatrix}\in\{\pm 1\}^{4\times 5}.
% \]
% By enumeration, an optimal solution of~\eqref{eq:l1lrageneral} is given by
% \[
% u=(-1,-1,-1,1)^\top,\quad  v=(1,-1, 1,-1, 1)^\top,\quad  \text{ with }
% \| M-uv^\top \|_1=8.
% \]
In fact, consider the binary matrix $X\in\{0,1\}^{4\times 5}$ given by  %obtained from $M$ by mapping $-1$ to $0$, that is,
\[
X= %\frac{M+\mathbf{1}\mathbf{1}^\top}{2}=
\begin{pmatrix}
1&1&0&1&0\\
0&1&0&1&1\\
0&1&1&1&0\\
1&0&1&1&1
\end{pmatrix}, 
\]
and the corresponding rank-one L1-NMF problem
\[
\min_{w\geq 0,h\geq 0}\ \|X-wh^\top\|_1.
\]
An optimal binary solution can be found by enumeration, 
\[
w^* = (1,1,1,1)^\top,\quad h^*=(0,1,0,1,0)^\top, \quad \text{ with }\|X-w^* {h^*}^\top\|_1 = 7.
\]
However, there exists a nonnegative solution with a smaller error:
$$
w'=\left(1,1,1,\sqrt{\tfrac32}\right)^\top,
h'=\left(\sqrt{\tfrac23},1,\sqrt{\tfrac23},1,\sqrt{\tfrac23}\right)^\top,
 \text{ with }
\|X-w'h'^\top\|_1 \approx 6.899<7.
$$ 
Therefore, in contrast to~\eqref{eq:l1lrageneral}, we cannot assume, without loss of generality, that an optimal solution of rank-one L1-NMF with a binary input matrix is itself binary. 
This counterexample shows that the NP-hardness claim of L1-NMF as a consequence of the NP-hardness of the unconstrained case in~\cite[Theorem 6.13]{gillis2020nonnegative} was made incorrectly. 

%\subsection{Complexity of L1-NMF}
%In this subsection, we prove that 

Let us now provide a proof that rank-one L1-NMF is NP-hard, even when the input matrix is restricted to be binary, that is, $X\in\{0,1\}^{m\times n}$. 
Our proof proceeds by a polynomial-time reduction from the decision version of the problem~\eqref{eq:l1lrageneral}. 
We begin by formally stating the decision versions of both problems. 
\smallskip

\begin{mdframed}
\noindent
\emph{Problem} L1-LRMA

\noindent
Input: a natural number $D>0$ and a matrix $M\in\{\pm1\}^{s\times t}$. 

\noindent
Question: does there exist $u\in\mathbb{R}^{s}$ and $v\in\mathbb{R}^{t}$ such that $\|M-uv^\top \|_1\leq D$.
\end{mdframed}

\smallskip


\begin{mdframed}
\noindent
\emph{Problem} L1-NMF

\noindent
Input: a natural number $T>0$ and a matrix $X\in\{0,1\}^{m\times n}$. 

\noindent
Question: does there exist $w\in\mathbb{R}_+^{m}$ and $h\in\mathbb{R}_+^{n}$ such that $\|X-wh^\top \|_1\leq T$.
\end{mdframed}

\smallskip

As mentioned above, L1-LRMA is NP-hard by a polynomial-time reduction from \textsc{Max-Cut}~\cite{gillis2018complexity}.
Let us now reduce L1-LRMA to L1-NMF in polynomial-time. 
Given an instance $(M,D)$ of L1-LRMA, we construct an instance $(X,T)$ of L1-NMF as follows: 
\begin{itemize}
    \item $T=st+D$,
    \item $X\in\{0,1\}^{2s\times 2t}$ is defined blockwise: for each $(i,j)\in\{1,\dots,s\}\times\{1,\dots,t\}$,
    \[
    X(2i-1\!:\!2i,\;2j-1\!:\!2j)=
    \begin{cases}
    \begin{pmatrix}
        1 & 0\\
        0 & 1
    \end{pmatrix}, & \text{if } M_{ij}=1,\\[3mm]
    \begin{pmatrix}
        0 & 1\\
        1 & 0
    \end{pmatrix}, & \text{if } M_{ij}=-1.
    \end{cases}
    \]
\end{itemize}
Each entry of $M$ is encoded by a $2\times2$ binary block in $X$. 
The reduction mimics the two possible signs, $\{\pm1\}$, using the two 2-by-2 permutation matrices. 
This construction of $X$ allows us to ``restore'' the property that a binary optimal solution for rank-one L1-NMF exists, which is not true in general (see the example above). 
%is designed so that the two coordinates in each row-pair and column-pair  


\begin{theorem}\label{lem:complexityyesyes}
The instance $(X,T)$ is a yes-instance of L1-NMF if and only if $(M,D)$ is a yes-instance of L1-LRMA. 
Hence rank-one L1-NMF is NP-hard. 
\end{theorem} 
\begin{proof}
\noindent
\textbf{The \textit{if} part.}
Suppose first that $(M,D)$ is a yes-instance of L1-LRMA. Then there exist $u\in\mathbb{R}^s$ and $v\in\mathbb{R}^t$ such that $\|M-uv^\top \|_1\leq D$.
By~\cite{gillis2018complexity}, we may assume without loss of generality that $u\in\{\pm1\}^s$ and $v\in\{\pm1\}^t$. We define $w\in\mathbb{R}_+^{2s}$ and $h\in\mathbb{R}_+^{2t}$ blockwise as follows:
\[
w_{2i-1:2i}=
\begin{cases}
(1,0)^\top, & \text{if } u_i=1,\\
(0,1)^\top, & \text{if } u_i=-1,
\end{cases}
\qquad
h_{2j-1:2j}=
\begin{cases}
(1,0)^\top, & \text{if } v_j=1,\\
(0,1)^\top, & \text{if } v_j=-1.
\end{cases}
\]
Clearly, $w\geq 0$ and $h\geq 0$. For each $(i,j)$, let $E_{ij}$ denote the error of the $(i,j)$th block in $X$, that is,
\[
E_{ij}=\left\lVert X(2i-1\!:\!2i,\;2j-1\!:\!2j)-w_{2i-1:2i}h_{2j-1:2j}^\top\right\rVert_1. 
\]
By definition, the rank-one block $w_{2i-1:2i}h_{2j-1:2j}^\top$ contains exactly one entry equal to $1$ and three equal to $0$.
There are two possibilities:
\begin{itemize}
    \item if $M_{ij}=u_i v_j$, this entry equal to $1$ is located at an entry where the block
    $X(2i-1:2i,2j-1:2j)$ also has an entry equal to $1$, hence $E_{ij}=1$,
    \item if $M_{ij}\neq u_i v_j$, this entry equal to $1$ is located at an entry where the block
    $X(2i-1:2i,2j-1:2j)$ has an entry equal to $0$, hence $E_{ij}=3$. 
\end{itemize}
Since $M_{ij},u_iv_j\in\{\pm1\}$, we have
\[
|M_{ij}-u_iv_j|=
\begin{cases}
0, & \text{if } M_{ij}=u_iv_j,\\
2, & \text{if } M_{ij}\neq u_iv_j.
\end{cases}
\]
Therefore,
\[
E_{ij}=1+|M_{ij}-u_iv_j| \qquad \text{for all } (i,j).
\]
Summing over all blocks gives
\[
\|X-wh^\top \|_1 = \sum_{i=1}^s\sum_{j=1}^t E_{ij}
= st + \|M-uv^\top \|_1
\leq st + D = T.
\]
Hence, $(X,T)$ is a yes-instance of L1-NMF.

\medskip
\noindent
\textbf{The \textit{only if} part.} Suppose now that $(X,T)$ is a yes-instance of L1-NMF, that is,
there exist $w\in\mathbb{R}_+^{2s}$ and $h\in\mathbb{R}_+^{2t}$ such that $\|X-wh^\top \|_1\le T$.
With $u\in\mathbb{R}^{s}$ and $v\in\mathbb{R}^{t}$ defined by
\[
u_i = w_{2i-1}-w_{2i},\quad \text{for }i=1,\ldots,s, 
\quad \text{ and }\quad
v_j = h_{2j-1}-h_{2j},\quad \text{for }j=1,\ldots,t,
\]
we show that
\[
\lVert M-uv^\top\rVert_1\leq D.
\]


\begin{itemize}
    \item When $M_{ij}=1$, the error of the $(i,j)$th block satisfies
\begin{eqnarray*}
E_{ij} & = & |1-w_{2i-1}h_{2j-1}| + w_{2i-1}h_{2j} + w_{2i}h_{2j-1} + |1-w_{2i}h_{2j}| \\
& \geq & |1-w_{2i-1}h_{2j-1}+w_{2i-1}h_{2j}| + |1-w_{2i}h_{2j}+w_{2i}h_{2j-1}| \\
& = & |1-w_{2i-1}v_j| + |1+w_{2i}v_j|.
\end{eqnarray*}

If $v_j\geq 0$, then $w_{2i}v_j\geq 0$ and we obtain
\[
E_{ij} \geq |1-w_{2i-1}v_j| + 1+w_{2i}v_j \geq  1 + |1-w_{2i-1}v_j+w_{2i}v_j| = 1 + |1-u_iv_j|.
\]
If $v_j\leq 0$, then $-w_{2i-1}v_j\geq 0$ and we obtain
\[
E_{ij} \geq 1-w_{2i-1}v_j + |1+w_{2i}v_j| \geq  1 + |1+w_{2i}v_j-w_{2i-1}v_j| = 1 + |1-u_iv_j|.
\]
Hence, when $M_{ij}=1$, we have $E_{ij} \geq 1 + |M_{ij}-u_i v_j|$.
\smallskip
\item When $M_{ij}=-1$, the error of the $(i,j)$th block satisfies
\begin{eqnarray*}
E_{ij} & = & w_{2i-1}h_{2j-1} + |1-w_{2i-1}h_{2j}| + |1-w_{2i}h_{2j-1}| + w_{2i}h_{2j} \\
& \geq & |1-w_{2i-1}h_{2j}+w_{2i-1}h_{2j-1}| + |1-w_{2i}h_{2j-1}+w_{2i}h_{2j}| \\
& = & |1+w_{2i-1}v_j| + |1-w_{2i}v_j|.
\end{eqnarray*}

Using the same discussion on the sign of $v_j$, we have that
$$E_{ij} \geq 1+|1+u_iv_j| = 1 + |M_{ij}-u_i v_j|.$$
\end{itemize}
Summing over all the blocks leads to
\[
\|X-wh^\top \|_1
= \sum_{i=1}^s\sum_{j=1}^t E_{ij}
\geq  st + \|M-uv^\top \|_1.
\]
Since $\|X-wh^\top \|_1\leq T=st+D$, it follows that $\|M-uv^\top \|_1\leq D$.
Hence, $(M,D)$ is a yes-instance of L1-LRMA.
\end{proof}

% An immediate consequence of Theorem~\ref{lem:complexityyesyes} is that, although a binary matrix $X$ does not necessarily admit an optimal binary solution to rank-one L1-NMF, the matrices $X$ produced by the above reduction do.
% Indeed, given any feasible pair $(w,h)\geq 0$, define $u_i=w_{2i-1}-w_{2i}$ and $v_j=h_{2j-1}-h_{2j}$. The proof of Theorem~\ref{lem:complexityyesyes} shows that the error contributed by each block is at least $1+|M_{ij}-u_iv_j|$, and therefore $\|X-wh^\top\|_1 \geq st+\|M-uv^\top\|_1$. Let now $(u^*,v^*)$ be an optimal solution of L1-LRMA for $M$. By~\cite{gillis2018complexity}, we may assume without loss of generality that $u^*$ and $v^*$ have entries in $\{\pm1\}$.
% Constructing $(w^*,h^*)\in\{0,1\}^{2s}\times\{0,1\}^{2t}$ from $(u^*,v^*)$ exactly as in the \textit{if part}, each block then contributes exactly to $1+|M_{ij}-u_i^*v_j^*|$, so that $\|X-w^*{h^*}^\top\|_1 = st+\|M-u^*{v^*}^\top\|_1$.
% It follows that $(w^*,h^*)$ is optimal for rank-one L1-NMF for the matrix $X$.
% Hence, while nonbinary factors may be strictly better for a general binary matrix, they cannot be for the binary instances produced by our reduction. 

%\ngc{Question for Arnaud: if I understood well, we can say that for a matrix $X$ constructed from a $\pm 1$ matrix $M$ as above, L1-NMF has a binary optimal solution, right? We could mention this :-) 'Not true in general but holds for matrices constructed blabla.'} 


%As an immediate consequence of Lemma~\ref{lem:complexityyesyes}, rank-one L1-NMF is NP-hard, even when the input matrix is binary.


















\section{Sparsity-inducing property of L1-NMF for sparse data} 
\label{sec:sparsity_inducing}

Any critical point of L1-NMF has to be a coordinate-wise minimizer, that is, every entry of $W$ and $H$ has to be an optimal solution of the corresponding LAD subproblem of the form~\eqref{eq:gen_scalar_prob}.  
Let us then investigate the probability that the one-dimensional nonnegative LAD regression problem~\eqref{eq:gen_scalar_prob} has a zero solution, depending on the sparsity of the data. 
As we will see, the sparser the data, the larger the probability that the LAD problem has a zero solution. This implies that if we compute the L1-NMF of some matrix $X$ using CD (or any other method that computes a coordinate-wise minimizer), the sparser the data, the larger the probability that L1-NMF has sparse factors. 

The LAD problem~\eqref{eq:gen_scalar_prob} has an optimal zero solution if the slope of the function changes sign after a nonpositive breakpoint. Let $x \in \mathbb{R}^m$ and $y \in \mathbb{R}_+^m$ be as in~\eqref{eq:gen_scalar_prob}. 
Since $y\geq0$, a negative breakpoint $x_s / y_s$ corresponds to a negative numerator $x_s$, and we define $\mathcal{N}_{x}=\{s \ | \ x_s\leq0 \} \subseteq \{ 1,\dots,m \}$ as the set of indices corresponding to the negative and zero breakpoints. 
The corresponding LAD problem has a zero solution if
\begin{equation}
    \sum_{s \in \mathcal{N}_{x}} y_s \quad > \sum_{s \in \{1,\dots,m\} \backslash \mathcal{N}_{x}} y_s . 
    \label{eq:sparse_cond}
\end{equation}
%where $\overline{\mathcal{N}_{x}} = \{1,\dots,m\} \backslash \mathcal{N}_{x}$. 
If the data vector $x$ is sparse, the set $\mathcal{N}_{x}$ is larger than its complement $\{1,\dots,m\} \backslash \mathcal{N}_{x}$, and the condition in~\eqref{eq:sparse_cond} is more likely to be satisfied. 

We proceed in our analysis by adding some probabilistic structure to $x$ and $y$, assuming they are binary vectors in $\{0,1\}^m$, sampled from a Bernoulli distribution with parameter $p$. Therefore, $x$ and $y$ have a probability $p$ of being positive and $(1-p)$ of being zero. Moreover, we are restricting $x$ to be nonnegative, thus ensuring that all the breakpoints are nonnegative. This is a situation that occurs in rank-1 L1-NMF, where $x$ represents a column of the matrix $X$. Note that when the factorization rank is larger than 1, $x$ is the residual vector, and it might contain negative entries, resulting in an even larger number of nonpositive breakpoints. 
We aim to understand how the probability of having an optimal non-zero solution to the LAD problem and to the least squares problem %where the L1 norm is substituted by the L2 norm, 
changes with respect to the parameter $p$ that regulates the sparsity in the data. That is, we want to estimate the probability that the optimal solutions 
\begin{equation} \label{eq:l1_l2_comp}
    \alpha_2=\arg\min_{\alpha\geq 0} \|x-\alpha y\|_2, \quad \text{and} \quad \alpha_1=\arg\min_{\alpha\geq 0} \|x-\alpha y\|_1
\end{equation}
have a non-zero value as a function of $p$. Note that $\alpha_1$ and $\alpha_2$ are not necessarily unique; see the proof of Lemma~\ref{lem:explicit_prob}  below for the details. 
This allows us to have a better understanding of the probability of one entry of the L1-NMF and L2-NMF factors to be set to zero. 
The following lemma gives a simple characterization of the probabilities $\mathbb{P}(\alpha_2>0)$ and $\mathbb{P}(\alpha_1>0)$. 
\begin{lemma}
    Let $x, y\in\{0,1\}^m$ be generated from a Bernoulli distribution which takes value 1 with probability $p \in [0,1]$ and value zero with probability $1-p$ 
    as inputs for the problems~\eqref{eq:l1_l2_comp}.  Then  
\begin{equation}
\begin{aligned}
    \mathbb{P}(\alpha_1>0) &  = (1-p)^{m} + \sum_{k=1}^mp^k (1-p)^{m-k}\binom{m}{k}\left(\sum_{i=0}^{\lfloor \frac{k}{2} \rfloor}(1-p)^ip^{k-i}\binom{k}{i}\right).
    %\\ & \ngi{= \sum_{k=0}^mp^k (1-p)^{m-k}\binom{m}{k}\left(\sum_{i=0}^{\lfloor \frac{k}{2} \rfloor}(1-p)^ip^{k-i}\binom{k}{i}\right).} \ngc{\text{simplify}}
    \\
    \mathbb{P}(\alpha_2>0) 
    %& \ngi{= (1-p)^{m} + \sum_{k=1}^mp^k (1-p)^{m-k}\binom{m}{k}\left(\sum_{i=0}^{k-1}(1-p)^ip^{k-i}\binom{k}{i}\right).} \ngc{\text{right?}}\\ 
     & = (1-p)^{m} + \sum_{k=1}^mp^k (1-p)^{m-k}\binom{m}{k} \left( 1 - (1-p)^k \right).
\end{aligned}
\label{eq:explicit_prob}
\end{equation}
\label{lem:explicit_prob}
\end{lemma}
%\ngc{You can simplify the second formula because for $k=0$, the second term would give $(1-p)^m$. But maybe better to keep it as for L2?} \gs{Yes, I kept it because I thought it was easier to follow the proof but we can chose.} 
\begin{proof} 
If $y=0$, then any nonnegative $\alpha_i$ for $i=1,2$ is an optimal solution, so we consider  $\mathbb{P}(\alpha_i>0) = 1$. 
Hence
\[
\mathbb{P}(\alpha_i>0) = 
\mathbb{P}(y=0) 
+ \mathbb{P}( \alpha_i>0 \ | \ y \neq 0). 
\]
Now, let us consider the two cases separately. 
For the L1 norm ($i=1$), if $y$ is not the zero vector, looking at Algorithm~\ref{algo_cons_weightedmedian}, one can check that $\alpha_1>0$ if $\sum_{j \in \supp(y)} x_j \geq \frac{|\supp(y)|}{2}$, that is, $x$ has at least as many 1's as 0's on the support of $y$, defined as the set of indices corresponding to non-zero entries in $y$ and denoted $\supp(y)$. 
Note that when $k =|\supp(y)|$ is even and $x(\supp(y))$ contains exactly $\frac{k}{2}$ zeros, any value $\alpha_1 \in [0,1]$ is optimal, hence we consider it positive with probability 1.
%if there are fewer or equal $0$'s than $1$'s among the entries of $x$ corresponding to the entries of $y$ equal to one. 
%\ngc{Why is it strictly larger, and not larger or equal?} \gs{I changed stricly fewer with fewer or equal, was that what you meant?} \ngc{Yes.}
%One can verify it rewriting line 6 of the constrained weighted median algorithm for binary data. 
%Since $x$ and $y$ are binary, this is equivalent to requiring that the sum of the $x_j$ for $j$ in the support of $y$ is more than half the length of the support of $y$. Formally, let $\supp(y)$ denote the support of $y \in \mathbb{R}^m$, then
Finally  
\begin{equation}
\begin{aligned}
    \mathbb{P}(\alpha_1>0)=&\mathbb{P}(y=0)+ \mathbb{P}\left(\sum_{j \in \supp(y)} x_j \geq  \frac{|\supp(y)|}{2} \right)\\
    =&\mathbb{P}(y=0) + \sum_{k=1}^m \mathbb{P}\left( \sum_{j \in \supp(y)} x_j \geq \frac{k}{2} \ \middle|  \ |\supp(y)|=k\right) \mathbb{P}\left(|\supp(y)|=k\right).
\end{aligned} 
\label{eq:step1_lem_prob}
\end{equation}
Since $|\supp(y)|$ and $\sum_{j \in \mathcal K} x_j$ for any subset $\mathcal K$ of $\{1,2,\dots,m\}$  %\gs{given $|\supp(y)|=k$, right?} 
follow a binomial distribution, we obtain~\eqref{eq:explicit_prob} for the L1 norm. 
% We have that 
% \begin{equation} 
% \mathbb{P}\left(|\supp(y)|=k\right)=\mathbb{P}\left(\sum_{j=1}^m y_j=k\right)=p^k (1-p)^{m-k}\binom{m}{k}.
%     \label{eq:sec_prob}
% \end{equation}
%Moreover, the conditional probability in~\eqref{eq:step1_lem_prob} is equivalent to the probability of the components of $x$ with indices in $\supp(y)$ to contain fewer $0$'s than $\lfloor \frac{k}{2} \rfloor$.  

% Therefore, we have
% \begin{equation}
%     \sum_{k=1}^m \mathbb{P}\left( \sum_{j \in \supp(y)} x_j > \frac{|\supp(y)|}{2} \ \middle| \ |\supp(y)|=k\right)=\sum_{i=0}^{\lfloor\frac{k}{2} \rfloor}(1-p)^ip^{k-i}\binom{k}{i} . 
%     \label{eq:step2_lem_prob}
% \end{equation}
% Combining line~\eqref{eq:sec_prob} and line~\eqref{eq:step2_lem_prob} 

For the L2 norm ($i=2$), when $y \neq 0$, the optimal solution has a unique closed form expression, $\alpha_2 = \frac{x^\top y}{\|y\|_2^2}>0$. 
Hence $\alpha_2 > 0$  as long as $x(\supp(y)) \neq 0$. We get  
\begin{equation}
\begin{aligned}
    \mathbb{P}(\alpha_2>0)=&\mathbb{P}(|\supp(y)|=0)+ \mathbb{P}\left(\sum_{j \in \supp(y)} x_j \geq 1 \right). 
\end{aligned} 
\label{eq:l2_prob}
\end{equation}
Following the same steps as above, and using the binomial distribution,  we obtain~\eqref{eq:explicit_prob} for the L2 norm.
% Equation~\eqref{eq:l2_prob} share the same structure of~\eqref{eq:step1_lem_prob} for the L1 norm, where  $\alpha_1 > 0$  as long as $\sum_{j \in \supp(y)} x_j \geq |\supp(y)|/2$. 
% Note that this condition is significantly more difficult to achieve than for the L2 norm above, which only requires the sum of the entries of $x$ on the support of $y$ to have a single non-zero entry. Using similar derivations, we get the formula for the L2 norm in the second line of the lemma, that is
% \begin{equation*}
% \begin{aligned}
%  \mathbb{P}(\alpha_2>0) 
%     & = (1-p)^{m} + \sum_{k=1}^mp^k (1-p)^{m-k}\binom{m}{k}\left(\sum_{i=0}^{k-1}(1-p)^ip^{k-i}\binom{k}{i}\right).\\ 
%      & = (1-p)^{m} + \sum_{k=1}^mp^k (1-p)^{m-k}\binom{m}{k} \left( 1 - (1-p)^k \right).
% \end{aligned}
% \end{equation*}
% \gs{I put the formulas here and reported just the final result on the lemma, is it fine? Shall I just say that with similar derivation we get to this?}
\end{proof}

%\ngc{You cannot add the $(1-p)^m$, this case is already included in $(1-p^2)^m$. These are not independent. For $m=1$ and $p=1/2$ your formula gives $1-1/4+1/2\dots$... You will have to remake the plot for L1 although the change will be minor.} 
%\gs{Yes, I made a mistake in simplifing the formula you wrote that is exactly what I used to compute the plots. In fact the plot are actually correct, the probability for $m=1$ and $p=1/2$ is correctly 0.75 for the L2 case. You said I should redo L1 but nothing has changed in L1, right? Was it just a typo?}
% \ngc{If you want to do this correctly, you have to separate the two cases: $y=0$ and $y \neq 0$: 
% \[
% \mathbb{P}(\alpha_2>0) = 
% \mathbb{P}(y=0) 
% + \mathbb{P}(x^\top y = 0 \ | \ y \neq 0)  
% \]
% This is more involved. Otherwise, you could first present the proof for the L1 case, and use a similar formula for the L2 case
% \begin{equation*}
% \begin{aligned}
%     \mathbb{P}(\alpha_1>0)=&\mathbb{P}(|\supp(y)|=0)+ \mathbb{P}\left(\sum_{j \in \supp(y)} x_j \geq 1 \right)
% \end{aligned} 
% \end{equation*}
% In fact, this is nice because we see better the big difference, and we could make the presentation simpler/nicer. Something along the lines: 

% If $y=0$, $\alpha_i$ for $i=1,2$ can take any nonnegative value, so $\mathbb{P}(\alpha_i>0) = 1$. 
% Hence 
% \[
% \mathbb{P}(\alpha_i>0) = 
% \mathbb{P}(y=0) 
% + \mathbb{P}( \alpha_i>0 \ | \ y \neq 0) . 
% \]
% For the L2 norm ($i=2$), $\alpha_2 = \frac{x^\top y}{\|y\|_2^2}>0$ hence $\alpha_2 > 0$  as long as $x_j = 1$ for some $j$ such that $y_j = 1$. We get  
% \begin{equation*}
% \begin{aligned}
%     \mathbb{P}(\alpha_1>0)=&\mathbb{P}(|\supp(y)|=0)+ \mathbb{P}\left(\sum_{j \in \supp(y)} x_j \geq 1 \right)
% \end{aligned} 
% \end{equation*}
% expend the formula, get the result.  

% For the L1 norm, $\alpha_2 > 0$  as long as $\sum_{j \in \supp(y)} x_j \geq |\supp(y)|/2$. 
% Note that this condition is significantly more difficult to achieve than for the L2 norm which only requires the sum of the entries of $x$ on the support of $y$ to have a single non-zero entry. 
% Using similar derivations, we get FORMULA.
% } 
%Lemma~\ref{lem:explicit_prob} provides an explicit formula for the probability of the optimal solution of the nonnegative LAD in~\eqref{eq:l1_l2_comp} to be nonnegative when the components of the inputs are sampled from a Bernoulli distribution. 
%in $\{0,1\}$ with probability $p$. 

%Starting from Lemma~\ref{lem:explicit_prob}, 
It is interesting to analyze the behavior of $\mathcal{P}(\alpha_i >0)$ for $i=1,2$ as the dimension $m$ increases. 
Figure~\ref{fig:prob_alpha} shows $\mathbb{P}(\alpha_1 >0) $ and $\mathbb{P}(\alpha_2 >0) $  as functions of the dimension $m$ for different choices of the  Bernoulli parameter~$p$. 
For $i=1$, we observe that, as the dimension $m$ increases,  
 the probability of the LAD solution being positive (i) goes to zero for  $0\leq p<\frac{1}{2}$, (ii) goes to 1/2 for $p=1/2$, and (iii) goes to one for $p > 1/2$. 
%For , the probability of the LAD solution of being positive decreases with the dimension $m$.  When $p=\frac{1}{2}$, the probability goes to $\frac{1}{2}$ of being positive while, for $\frac{1}{2}<p \leq 1$,  $\mathbb{P}(\alpha_1 >0) $ converges to 1. 
For the least squares problem ($i=2$), 
$\mathbb{P}(\alpha_2 >0)$ goes to 1 as $m$ increases for any $p > 0$.  
In fact, one can prove that, for $p > 0$, since both $|\supp(y)|$ and $\sum_{j\in \supp(y)} x_j$ for a fixed support $\supp(y)$ are binomials, %it holds for $m \to \infty$:
\begin{equation}
    \lim_{m \to \infty} \mathbb{P}(\alpha_2>0) = 1 \quad \text{and} \quad \lim_{m \to \infty} \mathbb{P}(\alpha_1>0) = \begin{cases} 1 & \text{ if  } p > \frac{1}{2}\\
\frac{1}{2} & \text{ if  } p = \frac{1}{2}\\
0 & \text{ if  } p < \frac{1}{2}
\end{cases} . 
\label{eq:bin_lem}
\end{equation}

Therefore, for large datasets, FroNMF is likely to generate dense solutions, even for sparse data. Of course, the basis vectors are not generated randomly and will align with the data. However, our result shows that for the randomly generated data matrix $X$ and basis vector $W$, the corresponding FroNMF optimal factor $H$ will be dense with high probability, as long as $m$ is sufficiently large. 
This is somewhat odd: randomly generating basis vectors $W$  will provide dense activations $H$. 
%The reason is that any basis vector that has a non-zero inner product with a data point will imply  
On the contrary, L1-NMF will generate sparse solutions, even when the dimension increases. This will be confirmed in the numerical experiments in Section~\ref{sec:num_res}. 

%Following from Lemma~\ref{lem:explicit_prob}, Corollary~\ref{cor:bin} characterizes the behavior of $\mathbb{P}(\alpha_1 >0) $ and of $\mathbb{P}(\alpha_2 >0)$ for $m \to \infty$. 
%\begin{corollary}
%Given $x, y\in\{0,1\}^m$, generated from a Bernoulli distribution which takes value 1 with probability $p \in (0,1)$ and value zero with probability $1-p$ as inputs for the problems~\eqref{eq:l1_l2_comp}, we have 
%\begin{equation}
%    \lim_{m \to \infty} \mathbb{P}(\alpha_2>0) = 1 \quad \text{and} \quad \lim_{m \to \infty} \mathbb{P}(\alpha_1>0) = c(p)= \begin{cases} 1 & \text{ if  } p > \frac{1}{2}\\
%\frac{1}{2} & \text{ if  } p = \frac{1}{2}\\
%0 & \text{ if  } p < \frac{1}{2}
%\end{cases} . 
%\label{eq:bin_lem}
%\end{equation}
%\label{cor:bin}
%\end{corollary} 
%\begin{proof}
%The corollary is obtained by taking the limit for $m \to \infty$ of line~\eqref{eq:step1_lem_prob} and~\eqref{eq:l2_prob}. 
%For the L1 norm, r
%Recall that both $|\supp(y)|$ and $\sum_{j\in \supp(y)} x_j$ for a fixed support $\supp(y)$ are binomials. 
%Consider the second line in equation~\eqref{eq:step1_lem_prob}, we have  $|\supp(y)| \to mp \to \infty$ a.s.\ thus, in the limit, 
%$$\lim_{m \to \infty} \mathbb{P}(\alpha_1>0)= \lim_{m \to \infty} \mathbb{P}\left( \sum_{j=1}^m x_j > \frac{m}{2} \right). $$
%Similarly
%$$\lim_{m \to \infty} \mathbb{P}(\alpha_2>0)= \lim_{m \to \infty} \mathbb{P}\left( \sum_{j=1}^m x_j > 1 \right).$$
%The $x_j$'s are i.i.d random variables, thus for the law of large numbers we have that $\frac{1}{m}\sum_{j=1}^m x_j \to p$ almost surely, for $m \to \infty$. Therefore, we have the statement of the corollary as a consequence.
%\end{proof} 


\begin{comment}

\begin{equation*}
\begin{aligned}
    \mathbb{P}(\alpha_1>0)=&\mathbb{P}\left( \sum_{j \in \supp(y)} x_j > \frac{|\supp(y)|}{2} \right)=\mathbb{P}\left( \sum_{j \in \supp(y)} x_j > \frac{1}{2} \sum_{j=1}^m y_j \right)\\
    =&\mathbb{P}\left( \sum_{j=1}^m y_j x_j > \frac{1}{2} \sum_{j=1}^m y_j \right)=\mathbb{P}\left( \sum_{j=1}^m y_j \left( x_j-\frac{1}{2} \right) > 0 \right)
\end{aligned} 
\end{equation*}
Define $z_j=y_j \left( x_j-\frac{1}{2} \right)$, that is the product between two Bernoulli random variables, thus it remains a discrete random variable with  values in $\{- \frac{1}{2},0, \frac{1}{2}\}$, defined by
\begin{equation}
    \mathbb{P}\left(z_j= \frac{1}{2}\right) = p^2, \quad \mathbb{P}\left(z_j= 0\right) = 1-p, \quad \mathbb{P}\left(z_j= -\frac{1}{2}\right) = p(1-p)
\end{equation}
Simple computations show that $\mu(p)=\mathbb{E}[z_j]=p^2-\frac{p}{2}$ and $b^2(p)=\var(z_j)=\frac{1}{4}p-\left( p^2 - \frac{p}{2} \right)^2$. Let us now define $S_m(p)=\sum_{j=1}^m z_j$, since the $z_j$'s are i.i.d, by the central limit theorem we have that $$S_m(p) \xrightarrow{m \to \infty} S(p) \sim \mathcal{N}_{m \mu(p), m b^2(p)},$$
where $\mathcal{N}_{\mu,b^2}$ denotes a normal distribution of mean $\mu$ and variance $b^2$. One can verify that for $p=\frac{1}{2}$, $S(\frac{1}{2}) \sim \mathcal{N}_{0, \frac{m}{8}}$, thus $\lim_{m \to \infty} \mathbb{P}(S_m(\frac{1}{2}) >0)=\frac{1}{2}$. Finally, noticing that $\mu(p)>0$ for $\frac{1}{2}<p<1$ and $\mu(p)<0$ for $0<p<\frac{1}{2}$, and that the mean of $S(p)$ explicitly depends on $m$, we have that $\lim_{m \to \infty} \mathbb{P}(S_m(p) >0)$ is either zero or one depending on the sign of $\mu(p)$.
\end{comment}


\begin{figure}
\begin{minipage}[h]{.49\linewidth}
    \centering
       %\resizebox{\linewidth}{!}{\input{alpha2.tex}}
       \resizebox{\linewidth}{!}{\input{Tikz/alpha2_p0}}
\end{minipage}
\begin{minipage}[h]{.49\linewidth}
    \centering
    %\resizebox{\linewidth}{!}{ \input{alpha1.tex}}
    \resizebox{\linewidth}{!}{ \input{Tikz/alpha1_p0}}
\end{minipage}
 \caption{Probability of the least squares solution (on the left) and LAD solution (on the right) in~\eqref{eq:l1_l2_comp} to be greater than zero when each component of the inputs is randomly sampled from a Bernoulli distribution in $\{0,1\}$ with probability $p$ to be 1. } 
    \label{fig:prob_alpha}
\end{figure}



\section{New L1-NMF model: wL1-NMF}
\label{sec:w_ell_1_nmf}
Despite the sparsity-inducing property of L1-NMF being meaningful in several contexts, it might lead to biased factorizations when the zeros in the data contain \textit{false zeros}~\cite{loeys2012analysis, blasco2019does, lopez2023zero}. False zeros appear, for example,  in imaging mass spectrometry where the amount of data is so large that the instruments used for the measurements report only values above some threshold (\textit{peak-picking}), in order to achieve sparse representations~\cite{moens2025preserving,gonzalez2023nectar}. In such applications, a good model that aims to recover the missing information from the observed data should approximate false zeros by non-zero values. At the same time, zeroing out the probability of having a zero value in the low-rank approximation does not take into account that zeros represent small values, thereby losing an important piece of information. 
In word-by-document count data, a zero entry does not necessarily mean that the word is not associated with the topics discussed by the corresponding document. This is especially critical when the documents are short (e.g., tweets). 
Another example is in recommender systems in which   missing values are not random and correspond to a mixture of negative and unknown preference~\cite{wang2018modeling}. Since preferences are typically nonnegative, 
it makes sense to assign zeros to missing entries, meaning a negative opinion on the corresponding items. These entries carry some useful information, namely a potential negative preference: someone did not consume an item because it was not likely that he/she would enjoy it. 

 %n; e.g., in a movie-by-user matrix (e.g., the Netflix prize), a missing entry might mean that a user does not know anything about the movie or that the user knew he would not like it. 






Assuming that zeros cannot be trusted as much as the positive entries but still carry some meaningful information, wL1-NMF, introduced in~\eqref{eq:weight_l1}, 
%leverages the penalization effect of the L1 norm on the entries of $WH$ corresponding to zeros in $X$, without canceling their contribution, and by using a penalty parameter $\lambda$. 
downweights the importance of the zero entries  using a penalty parameter $\lambda$. Let us recall the formulation: 
\begin{equation}
    %d(X,WH,\lambda)=
    \ell(X,W,H,\lambda) := \sum_{(i,j) \in \kappa^+} \lvert X -WH \rvert_{i,j} + \lambda \sum_{(i,j) \in \kappa^0} \lvert WH \rvert_{i,j},
    \label{eq:weight_l1_2}
\end{equation}
where $\lambda \in [0,1]$, $\kappa^+=\left\{ (i,j) \ | \ X_{i,j}>0 \right\}$,  
    and 
    %\quad \text{and } \quad 
    $\kappa^0=\left\{ (i,j) \ | \ X_{i,j}=0 \right\}$.
This corresponds to a low-rank estimator that follows two Laplace distributions with different variances: the first for the positive values and the second for the zeros; see below for more details.  
 
We now give a simple rank-one example, $X \approx wh$, in which fixing $w$ leads to an optimal solution $h=0$ when $\lambda = 1$ (that is, for L1-NMF), while smaller values of $\lambda$ produce a nonzero optimum: let 
%\paragraph{Example 2} 
%Consider the following $4 \times 4$ matrix 
\begin{equation*}
    X=\begin{pmatrix}
        1 & 1 & 0 & 0\\
        0 & 0 & 0 & 1 \\
        1 & 0 & 1 & 0\\
        0 & 0 & 1 & 0
    \end{pmatrix},
\end{equation*}
and $w=(1,5/2,1,2)^T$, we have 
\begin{equation}
    \argmin_{h\geq0} \ell(X,w,h,1)= \argmin_{h\geq0} \lVert X - w h \rVert_1 =  (0,0,0,0).
\end{equation}
%and using~\eqref{eq:sparse_cond} for each column of $X$, it is straightforward to verify that the optimal update for any entry of $h$ is zero. 
If we consider smaller values of $\lambda$, we obtain, for example, 
\begin{equation*}
    \argmin_{h \geq 0} \ell(X,w,h,0.85)=(0,0,0.5,0)^\top, \quad \text{ and } \quad
        \argmin_{h \geq 0} \ell(X,w,h,0.4)=(1,0,0.5,0)^\top .
\end{equation*} 


\paragraph{Statistical interpretation of wL1-NMF}

The minimizer of the L1-NMF model in~\eqref{eq:l1NMF} is the maximum likelihood estimator when the $(i,j)$th entry of $X$ is the realization of a random variable $\tilde{X}_{i,j}$ that depends on the parameter $(\Hat{W}\Hat{H})_{i,j}$ and is corrupted by additive Laplacian noise, that is, 
\begin{equation}
    \tilde{X}_{i,j} =(\Hat{W}\Hat{H})_{i,j}+\tilde{N}_{i,j}, \quad \text{where } \quad \Hat{W}\geq0, \ \Hat{H}\geq0, \ \text{and} \ \tilde{N}_{i,j} \sim \mathcal{L}(0,\sigma),
    \label{eq:prob_struct}
\end{equation}
where $\tilde{N}_{i,j}$ follows a zero mean Laplace distribution with the diversity $\sigma$.  %with a probability density function. 
Equivalently, 
\begin{equation}
    p_1(\tilde{X}_{i,j};(\Hat{W}\Hat{H})_{i,j},\sigma)=\frac{1}{2 \sigma} e^{-\frac{1}{\sigma}\lvert \tilde{X}_{i,j} - (\Hat{W}\Hat{H})_{i,j} \rvert } \quad \text{ for } i=1,\dots,m \ \text{ and } \ j=1,\dots,n, 
    \label{eq:lab_noise}
\end{equation} 
from which we obtain the likelihood
\begin{equation}
    \ell_1(\Tilde{X};\Hat{W}\Hat{H},\sigma)=  \prod_{i,j } \frac{1}{2 \sigma} e^{-\frac{1}{\sigma}\lvert \Tilde{X}_{i,j} - (\Hat{W}\Hat{H})_{i,j} \rvert }.
    \label{eq:likel}
\end{equation}
Minimizing the negative logarithm of~\eqref{eq:likel} with respect to $W$ and $H$, we obtain estimates $W^*$ and $H^*$ for the unknown parameters $\Hat{W}$ and $\Hat{H}$ by solving the L1-NMF model. 

Being %the noise in~\eqref{eq:lab_noise} sampled from 
 continuous, the Laplace distribution is not designed for situations where the realization of $\tilde{X}$ contains many zero entries. When $X_{ij}$ is zero, we can deduce from~\eqref{eq:prob_struct} that the corresponding entries in the estimate $(W^* H^*)_{ij}$ must be close to the distribution of the noise $\Tilde{N}_{ij}$. Since the Laplace distribution has a sharp peak at zero, this results in a penalization of the magnitude of $(W^* H^*)_{ij}$ as the likelihood in~\eqref{eq:likel} shows. We propose to downweight the penalization effect by rescaling the noise affecting the components in $\kappa^0$, corresponding to zeros in $\Tilde{X}$. The resulting likelihood is 
%denoted $\hat X$, 
% Moreover, when $\Hat{X}_{i,j}=0$, we are implicitly assuming that $(\Hat{W}\Hat{H})_{i,j}$ and the noise $\Hat{N}_{i,j}$ follow the same distribution, which might be far from true. \ngc{I am not sure what you mean here, I removed it} 
%To overcome this limitation, we show in what follows that wL1-NMF in~\eqref{eq:weight_l1} can be derived by a MAP estimation assuming some relaxed prior information on the zero entries in the realization of $\tilde X$. Assuming prior information on the factors is a common practice in NMF to derive regularized models and MAP estimation has already been used for robust NMF, see for example~\cite{wang2012probabilistic,bayar2014probabilistic}. Let us consider the following probabilistic model:
%\begin{equation}
%    \begin{cases}
%          \tilde{X}_{i,j} =(\Hat{W}\Hat{H})_{i,j}+\tilde{N}_{i,j}, \quad & \text{for } (i,j) \in \kappa^+\\
%         (\Hat{W}\Hat{H})_{i,j} \sim \text{Exp}(\frac{\sigma}{\lambda})& \text{for}  (i,j) \in \kappa^0,
%    \end{cases}
%\end{equation}
%where $\tilde{N}_{i,j} \sim \mathcal{L}(0,\sigma)$.
%In particular, we assume the positive entries of $X$ are corrupted by a zero mean Laplace noise $\Tilde{N}$ with diversity $\sigma$, that is,~\eqref{eq:prob_struct} holds only for  $(i,j) \in \kappa^+$. 
%$$\tilde X_{i,j} =  (\Hat{W}\Hat{H})_{i,j}+\tilde{N}_{i,j}, \quad \text{where } \quad \Hat{N}_{i,j} \sim \mathcal{L}(0,\sigma) \quad \text{for} \quad (i,j) \in \kappa^+.$$ 

%Then, we consider the zeros of $X$ as missing values, but we assume to have the prior information that $(\Hat{W}\Hat{H})_{i,j} \geq 0$ is distributed according to a zero mean half-Laplace distribution with diversity $\sigma/\lambda$ in the unobserved entries, corresponding to the zeros of $X$, whose probability density function is
%$$p_2((\Hat{W}\Hat{H})_{i,j},\sigma,\lambda)=\frac{\lambda}{2 \sigma} e^{-\frac{\lambda}{\sigma}  (\Hat{W}\Hat{H})_{i,j}}\quad \text{for} \quad (i,j) \in \kappa^0.$$ 
\begin{equation}
     \ell_1(\Tilde{X};\Hat{W}\Hat{H},\sigma)=\prod_{(i,j)\in \kappa^+ } \frac{1}{2 \sigma} e^{-\frac{1}{\sigma}\lvert \Tilde{X}_{i,j} - (\Hat{W}\Hat{H})_{i,j} \rvert } \prod_{(i,j)\in \kappa^0} \frac{\lambda}{2 \sigma} e^{-\frac{\lambda}{\sigma} (\Hat{W}\Hat{H})_{i,j} }, 
    \label{eq:posteriori}
\end{equation}
where $\lambda$ is a parameter smaller than 1. Taking the negative logarithm and removing the constant terms, we obtain the wL1-NMF objective function~\eqref{eq:weight_l1_2}. Note that a larger diversity means that the Laplace distribution has a less sharp peak at zero. Equivalently, larger diversities, forced by smaller values of $\lambda$, reduce the probability of $(\Hat{W} \Hat{H})_{i,j}$ for $(i,j) \in \kappa^0$ to be zero. If the data at hand are heterogeneous, one might consider using different $\lambda$ parameters for each column (or row) of $\Tilde{X}$.
% \begin{equation}
%     \ell(X,WH,\lambda)=\sum_{(i,j) \in \kappa^+} \lvert X -WH \rvert_{i,j} + \lambda \sum_{(i,j) \in \kappa^0}  (WH)_{i,j},
%     \label{eq:weight_l1_2}
% \end{equation}
% with $W\geq 0$ and $H \geq 0$. 

Moreover, if we cannot trust the zeros or we want to obtain denser solutions, smaller values of $\lambda$ should be used since they correspond to increasing the variability in the prior Laplace distribution of the entries in $\kappa^0$. This theoretical construction can be extended to the case where the entries in $\kappa^+$ and $\kappa^0$ are modeled with different distributions or when prior information is assumed on $\Hat{W}$ and $\Hat{H}$. However, minimizing the corresponding likelihood might become harder. This alternative has already been explored for robust NMF in several contributions; see, for example, \cite{wang2012probabilistic,bayar2014probabilistic}. 

 

\section{Efficient CD algorithm for L1-NMF with sparse data} 
\label{sec:l1_for_sparse}


%\ngc{Say it is a HALS-like algorithm but for $\ell_1$}

CD for FroNMF, also known as HALS, is among the most efficient methods for solving NMF problems~\cite{cichocki2007hierarchical, gillis2012accelerated}. In this section, we introduce \name, a novel CD scheme for wL1-NMF~\eqref{eq:weight_l1_2}, that scales linearly with the number of non-zero entries of the original matrix. 
Every iteration of our method is equivalent to an iteration of the standard CD algorithm, but the computational cost is substantially reduced by exploiting the sparsity and nonnegativity in the data. 
As with CD, \name sequentially optimizes over the entries of $W$ and $H$. 
%two variables $W$ and $H$, by solving:
% \begin{equation}
% \begin{aligned}
%     &W^{k+1}=\argmin_{W\geq0}  \ell(X,WH^k,\lambda)\\
%     &H^{k+1}=\argmin_{H\geq0}  \ell(X,W^{k+1}H,\lambda),
%     \label{eq:subprob_l1}
% \end{aligned}
% \end{equation}
% where $(W^k,H^k)$ are the approximated factors at step $k$ of the iterative scheme. Each of the subproblems in~\eqref{eq:subprob_l1} is separable by rows or columns. More precisely, the first is separable by rows of $W$, leading to $m$ multivariate, nonnegative LAD problems in $r$ variables, with data vector of dimension $n$, while the subproblem for $H$ is separable by columns and it is equivalent to $n$ multivariate, nonnegative LAD problems of $r$ variables and data vector of dimension $m$. 
Let us focus on the subproblems for the entries of $H$, the subproblems for $W$ are the same, by symmetry.  
Let 
\begin{equation}
    \kappa^+_j=\left\{ i \ | \ (i,j) \in \kappa^+ \right\} \quad \text{and } \quad \kappa^0_j=\left\{ i \ | \ (i,j) \in \kappa^0 \right\},
    \label{eq:def_set_kappa_q_j}
\end{equation}
be the indices of the positive and zero entries for each column of the matrix $X$, where $\kappa^+$ and $\kappa^0$ are defined after~\eqref{eq:weight_l1_2}. The subproblem for the $j$th column $H_{:,j}$ of $H$ becomes 
\begin{equation}
    \argmin_{H_{:,j}\geq 0} \sum_{s \in \kappa^+_j} \lvert X -WH \rvert_{s,j} + \lambda \sum_{s \in \kappa^0_j} \lvert WH \rvert_{s,j}, \quad \text{for } j=1,\dots,n.
    \label{eq:sub_H_weigh}
\end{equation}
Each subproblem in one column of $H$ is independent; thus, they can be solved in parallel. We further split the problem into $r$ one-dimensional LAD problems, one for each entry $H_{i,j}$ of $H_{:,j}$, for $i=1,\dots,r$, and we obtain %a piece-wise linear function of the form 
\begin{equation}
   \argmin_{H_{i,j}\geq 0} \sum_{s \in \kappa^+_j} \lvert X_{s,j}-\sum_{t=1, \ t\neq i}^r (W_{s,t}H_{t,j})-W_{s,i}H_{i,j} \rvert + \lambda H_{i,j}  \sum_{s \in \kappa^0_j} W_{s,i},
\label{eq:sparse_l1_H_explicit}
\end{equation}
that can be simplified to 
\begin{equation}
    \argmin_{H_{i,j}\geq 0} \sum_{s \in \kappa^+_j}\lvert x_s - y_s H_{i,j} \rvert +  \lvert 0 - c H_{i,j} \rvert,
    \label{eq:fin_subprob}
\end{equation}
where 
\begin{equation}
    x_s= X_{s,j}-\sum_{t=1, \ t\neq i}^r (W_{s,t}H_{t,j}), \quad y_s=W_{s,i}, \quad c=\lambda \sum_{s \in \kappa^0_j} W_{s,i}.  
    \label{eq:a_b_c_def}
\end{equation}
Exploiting the nonnegativity of the factors $W$ and $H$ in Equation~\eqref{eq:sparse_l1_H_explicit}, we group the contributions of all the components in $\kappa^0_j$. Therefore, the one-dimensional subproblem in $H_{i,j}$ can be written as the sum of $|\kappa^+_j|+1$ terms, and the constrained weighted median algorithm can be employed to obtain one global minimum in $\mathcal{O}(|\kappa^+_j|\log(|\kappa^+_j|))$ rather than $\mathcal{O}(m\log(m))$ as in the standard CD approach. If $|\kappa^+_j| \ll m$, then this modification consistently reduces the computational cost of each one-variable subproblem while producing the same sequence of iterates as the original CD scheme. 
It is interesting and important to note that it would not be possible to take advantage of the sparsity in the unconstrained case, as the signs of the entries of $WH$ are unknown; hence, one needs to compute all entries of $WH$, leading to a computational cost of $\mathcal O(mnr)$; see also Section~\ref{sec:compcost}.  
In other words, to the best of our knowledge, L1-LRMA cannot be solved with CD with complexity linear in the number of non-zero entries in $X$. 




We have seen in Section~\ref{sec:sparsity_inducing} how the sparsity of the data affects the sparsity of the L1-NMF solution. %computed by the CD approach. 
Specifically, if the condition in~\eqref{eq:sparse_cond} is satisfied, the nonnegative LAD problem in~\eqref{eq:gen_scalar_prob} has an optimal zero solution. Using the same notation as in~\eqref{eq:a_b_c_def}, and defining $\mathcal{N}_{x}^-=\{s=1,\dots,m \ | \ x_s < 0 \}$, we rewrite the condition in~\eqref{eq:sparse_cond} to have an optimal solution $H_{i,j}=0$ for the subproblem in~\eqref{eq:fin_subprob}, that is, 
\begin{equation*}
    \sum_{s \in {\mathcal{N}}_{x}^-} W_{s,i} + \lambda \sum_{s \in \kappa_{j}^0} W_{s,i} > \frac{1}{2} \sum_{s \in \kappa_{j}^+} W_{s,i} +  \frac{\lambda}{2} \sum_{s \in \kappa_{j}^0} W_{s,i}.
    \label{eq:pre_sparse_cond_weight_pre}
\end{equation*}
Equivalently, we have
\begin{equation}
    \sum_{s \in {\mathcal{N}}_{x}^-} W_{s,i} + \frac{\lambda}{2} \sum_{s \in \kappa_{j}^0} W_{s,i} > \frac{1}{2} \sum_{s \in \kappa_{j}^+} W_{s,i}.
    \label{eq:pre_sparse_cond_weight}
\end{equation}
The smaller the parameter $\lambda$, 
the smaller the number of indices satisfying the condition in~\eqref{eq:pre_sparse_cond_weight}. Consequently, \name\ with $\lambda<1$ produces denser factorizations as $\lambda$ becomes smaller. This will be illustrated in Section~\ref{sec:num_res}. 


\subsection{Implementation details} Let us provide some implementation details for \name. 
First, the scalars $x_s$ for  $s=1,\dots,|\kappa^+_j|$ should not be computed using the explicit formula in~\eqref{eq:a_b_c_def}. In fact, let $(W^k,H^k)$ be the $k$th iterate. Defining $v_{s,j} = W^k_{s,:}H^k_{:,j} $ for every $s$ in $\kappa^+_j$ and for $j=1,\dots,n$, 
$x_s$ can be obtained with minimal cost from 
\begin{equation*}
    x_s = X_{s,j} - v_j + W^k_{s,i}H^k_{i,j}, \quad \text{for every } i=1,\dots,r.  
\end{equation*}
Similarly, after computing the new $H_{i,j}^{k+1}$,  $v_j$ is cheaply updated by
\begin{equation*}
    v_{s,j}\gets v_{s,j} + W^k_{s,i}(H_{i,j}^{k+1}-H^k_{i,j}), \quad \text{for every } s \text{ in } \kappa^+_j. 
\end{equation*}
We now give the full scheme for the update of $H$ in Algorithm~\ref{alg:H_update_full}. We use the simplified notation $\kappa$ to denote $\kappa^+_j$, where we omit the lower script $j$ and the super script $+$ for simplicity. 
\begin{algorithm}
	\caption{\quad \texttt{H = updateH($X$,$W^k$,$H^k$,$\lambda$)}}
	\begin{algorithmic}[1]
		\item INPUT: $X\in \mathbb{R}^{m\times n}$, $W^k\in \mathbb{R}^{m\times r}_+$, $H^k\in \mathbb{R}^{r\times n}_+$, $\lambda$
		\item OUTPUT: $H^{k+1}\in \mathbb{R}^{r\times n}_+$
        
		\State $s_i = \sum_{s=1}^{m} W^k_{s,i}$ \quad $\forall i = 1,\dots,r$ \label{line4}
        
         \State err = 0 
         
		%\While{stopping criterion not satisfied}
		\For{$j=1:n$}
		\State $\kappa = \{s \ |\ X_{s,j}>0\}$
		\State $v_j = W^k_{\kappa,:}H^k_{:,j}$ \label{line8}
		\For{$i=1:r$}
		\State $b \gets W^k_{\kappa,i}$ \label{line10}

%\State $a' \gets X_{\kappa,j} - v_j$

% \State err $\gets$ err - \|a'\|_1 + $e^\top v_j$

%\State $a \gets a' + W^k_{\kappa,i}H^k_{i,j}$
        
		\State $a \gets X_{\kappa,j} - v_j + W^k_{\kappa,i}H^k_{i,j}$
        
        
		\State $c \gets \lambda \left(s_i - \sum_{s\in \kappa} W^k_{s,i} \right)$
		\State $H_{i,j}^{k+1} \gets \texttt{constrained\_weighted\_median}([a\ 0],[b\ c])$
        
		\State $v_j\gets v_j + W^k_{\kappa,i}(H_{i,j}^{k+1}-H^k_{i,j})$ \label{line14}

		\EndFor
                \State err $\gets$ err + $\| X_{\kappa,j} - v_j\|_1 - e^\top v_j$ 

		\EndFor
        \State err $\gets$ err + $(e^\top W)(He)$
		%\EndWhile
	\end{algorithmic}
    \label{alg:H_update_full}
\end{algorithm}

The same algorithm applies to the update of $W^k$ observing that $\lVert X -WH \rVert=\lVert X^\top  -W^\top H^\top  \rVert$, thus Algorithm~\ref{alg:H_update_full} with inputs $X^\top $, $(H^k)^\top $ is employed to get the new updated $(W^{k+1})^\top $. The overall procedure to compute the L1-NMF is given in Algorithm~\ref{alg:sparse_l1_nmf}. 
\begin{algorithm}
	\caption{\quad \texttt{[W,H] = \name(X,W$^0$,H$^0$,$\lambda$,maxiter)}}
	\begin{algorithmic}[1]
		\item INPUT: 
        nonnegative matrix $X\in \mathbb{R}^{m\times n}_+$, 
        the  $W^0 \in \mathbb{R}^{m\times r}$, $H^0 \in \mathbb{R}^{r \times n}$, 
        regularization parameter $\lambda$, 
        maxiter. 
        
		\item OUTPUT: $W\in \mathbb{R}^{m\times r}_+$ and $H\in \mathbb{R}^{r\times n}_+$
		\For{$k=1, \dots,$ maxiter}
        \vspace{0.1cm}
		\State $(W^{k+1})^\top =\texttt{updateH}(X^\top ,(W^k)^\top ,(H^k)^\top,\lambda )$
        \vspace{0.1cm}
		\State $H^{k+1}=\texttt{updateH}(X,W^{k+1},H^k,\lambda)$
		\EndFor
	\end{algorithmic}
    \label{alg:sparse_l1_nmf}
\end{algorithm}




\subsection{Computational cost of \name} 
\label{sec:compcost}

%We start by analyzing the computational complexity of the \name algorithm. We can deduce from Algorithm~\ref{alg:H_update_full} that 

The most expensive operation of \name\ for updating one entry of $H$ in Algorithm~\ref{alg:H_update_full} is the constrained weighted median algorithm, which has complexity $\mathcal{O}(|\kappa^+_j|\log(|\kappa^+_j|))$, where $\kappa^+_j$ is the number of nonzero entries in the $j$th column of $X$. Thus, the update of all entries of $H$ has a total computational cost of $\mathcal{O}(r \sum_{j=1,\dots,n}|\kappa^+_j|\log(|\kappa^+_j|))$, which can be upper bounded by $\mathcal{O}(r \nnz(X) \log(\nnz(X)))$. 
The same reasoning applies for updating $W$. Hence, one iteration of \name\ costs $\mathcal{O}(r \nnz(X) \log(\nnz(X)))$ operations. 
If the original matrix $X$ is sparse, that is,  $\nnz(X) \ll mn$, then \name\ consistently reduces the computational cost of the  naive CD algorithm, which has a complexity of $\mathcal{O}(r mn \log(mn))$ operations. This difference will be illustrated in Section~\ref{sec:num_res}. 
To the best of our knowledge, \name\ is the first L1-NMF algorithm that scales linearly with $\nnz(X) \log(\nnz(X))$. %\ngc{is this correct?}

\paragraph{Computing the L1-NMF objective} 
It is interesting to note that the L1-NMF objective (and similarly that of wL1-NMF) can be computed in $\mathcal O(\nnz(X)r)$ operations, explaining why it is possible to design algorithms running in a number of operations proportional to $\nnz(X)$. 
In fact, 
\begin{align*} 
\| X - WH \|_1 
= 
\sum_{(i,j) \in \kappa^0} \lvert WH \rvert_{i,j} + 
\sum_{(i,j) \in \kappa^+} \lvert X -WH \rvert_{i,j} 
= 
\|WH\|_1 + \sum_{(i,j) \in \kappa^+} \big( \lvert X -WH \rvert_{i,j} - (WH)_{i,j}   \big), 
\end{align*}
where $\|WH\|_1 = (e^\top W) (H e)$ can be computed in $\mathcal O(r(m+n))$ operations, while the first term needs to evaluate $\nnz(X)$ terms with cost $\mathcal O (r)$ for each term. 
Note that this would not be possible if $W$ and $H$ were unconstrained. 

\subsection{Convergence of \name} 

\name\ decreases the objective function at every iteration, and therefore the sequence of the function values will converge since it is bounded below. 
However, to guarantee the convergence of the sequence of the CD iterates to a coordinatewise stationary point of the nondifferentiable problem in~\eqref{eq:l1NMF}, each of the subproblems in $W$ and $H$ must have a unique global minimizer~\cite{tseng2001convergence}. This is not the case for wL1-NMF since the scalar subproblem for the update of $H_{i,j}^k$ in~\eqref{eq:gen_scalar_prob} might have multiple global minima. Specifically, when two or more breakpoints achieve the lowest function value, any point in the interval between the breakpoints is a global minimum. Therefore, \name\ might converge to a point that is not a coordinate-wise stationary point of wL1-NMF. Moreover, the absence of global convergence guarantees suggests that the initialization of the algorithm plays a fundamental role, as our experience confirms. 
%However, for real data, the situations in which CD does not converge to a coordinatewise minimizer is unlikely. \ngc{We have checked this in practice?} 
%Therefore, even though we cannot ensure the convergence of the sequence generated by \name, Section~\eqref{sec:sparsity_inducing}  shows that, when this happens, the coordinatewise minimizer of wL1-NMF has high probability of being sparse in relation to the sparsity of the original data. 


\subsection{\name\ for binary factorizations}  

When $X$ is binary, one may wish to constrain $W$ and $H$ to be binary as well~\cite{zhang2007binary}, this is referred to as binary matrix factorization (BMF).  
Interestingly, if \name\ is initialized with binary factors, $W^0 \in \{0,1\}^{m \times r}$ and $H^0 \in \{0,1\}^{r \times n}$, and the input is binary, $X \in \{0,1\}^{m \times n}$, 
one can show that the iterates of \name\ will remain binary without enforcing it explicitly. The reason is that the break points in the LAD subproblems are 0, 1 or negative. Hence \name\ can be used as a BMF algorithm using the component-wise $\ell_1$ norm as the error measure. Comparing \name\ to existing BMF algorithms is out of the scope of this paper, and left for future works. 
%In that case, we can show that the updates of \name\ are simplified, and that 
%It is interesting to note that if we want to impose $X$
%\end{remark}

\begin{comment}
    

\subsection{Sparsity inducing property} We now show under which condition the constrained weighted median algorithm yields an optimal $H_{i,j}^k=0$ solution for some fixed $(i,j) \in \{1,\dots,r\} \times \{1,\dots,n\}$, promoting sparsity in the factor $H$ of the L1-NMF. A zero solution means that the constrained weighted median algorithm identifies as optimal a negative or zero breakpoint. Let $x_s$, $y_s$, and $c$ as in~\eqref{eq:a_b_c_def}, and $(W^k,H^k)$ the $k$th iterate of the scheme in~\eqref{eq:subprob_l1}. We observe that, since $y_s>0$, a negative breakpoint $x_s / y_s$ corresponds to a negative numerator $x_s$; thus we define $\mathcal{N}_{a}=\{s \in \kappa^+_j \ | \ x_s<0 \}$ as the set of indices corresponding to negative breakpoints. The first step of the constrained weighted median algorithm consists of sorting the breakpoints. Secondly, it checks after which breakpoint the slope of the function changes sign, by looking at the condition at step 6 of Algorithm~\ref{alg:constr_w_m}. Since the breakpoints are sorted, the condition will be checked first on negative and zero breakpoints and then on the positive ones. 
Therefore, according to Algorithm~\ref{alg:constr_w_m}, the optimal breakpoint is either negative or zero if the condition is first satisfied by a negative or zero one, that is
\begin{equation*}
    \sum_{s \in {N}_{a}} y_s + c > \frac{1}{2} \sum_{s=1}^m y_s,
\end{equation*}
that means
\begin{equation}
    \sum_{s \in {N}_{a}} W^k_{s,i} + \sum_{s \in k_{0j}} W^k_{s,i} > \frac{1}{2} \sum_{s=1}^m W^k_{s,i},
    \label{eq:pre_sparse_cond_1}
\end{equation}
that is also equivalent to 
\begin{equation}
    \sum_{s \in {N}_{a}} W^k_{s,i} + \frac{1}{2} \sum_{s \in k_{0j}} W^k_{s,i} > \frac{1}{2} \sum_{s \in k_{+j}}^m W^k_{s,i}.
    \label{eq:pre_sparse_cond}
\end{equation}
If the data matrix $X$ is sparse, it is likely that $|k_{+j}|\ll|k_{0j}|$ for many indices $(i,j) \in \{1,\dots,r\} \times \{1,\dots,n\}$, thus the inequality in~\eqref{eq:pre_sparse_cond} might be easily satisfied. This means that $H_{i,j}^k=0$ for many indices $(i,j) \in \{1,\dots,r\} \times \{1,\dots,n\}$, thus the resulting factor $H$ is very likely to be sparse.
\end{comment}



%\gs{To cut or reduce and mention it somewhere} \ngc{I would just put it as a remark. Is this OK like this?} 



\begin{comment}
    \subsection{Extension to L1-BMF}
\label{sec:ext_nmf}
In this section, we show that \name\ naturally extends to a specific instance of NMF problem that is the binary matrix factorization problem in the L1-norm. Given a binary matrix $X$, BMF aims at finding two binary factors $W$ and $H$ such that $X \approx WH$. BMF is a particular instance of NMF where the factors are constrained to be binary in $\{0,1\}$. The idea behind BMF is to represent binary data in a lower-dimensional space by attributes having the same nature as the original data~\cite{zhang2007binary}. Similarly to NMF, the most commonly used error measure is the Frobenius norm. A drawback of BMF in the Frobenius norm is that dealing with the binary constraint in the factors is usually computationally consuming and it requires different tools than those used in NMF. We show that substituting the Frobenius norm with the L1-norm in the BMF formulation, we can address the derived L1-BMF by the \name\, which naturally preserves binary values in the factors without explicitly including any binary constraint. 

Specifically, let $X \in \{0,1\}^{m \times n}$ be a binary matrix, and $r \ll \min(m,n)$, L1-BMF is the following problem 
\begin{equation*}
    \argmin_{W,H}\lVert X - WH \rVert_1 \ \text{ such that } W \in \{0,1\}^{m \times r}, H \in \{0,1\}^{r \times n}.
\end{equation*}
Provided that a binary initialization is given, the factors remain binary throughout all the iterations of Algorithm~\ref{alg:sparse_l1_nmf}. In fact, let $X$, $W$, and $H$ be binary matrices; then the subproblem for $H_{i,j}$, solved by the constrained weighted median algorithm, has a binary optimal solution. In fact, let $x_s$ and $y_s$, defined in~\eqref{eq:a_b_c_def}, then the breakpoints $x_s/y_s$  have values in $[-r,0]\cup \{1 \}$ which means that all the breakpoints are negative, 0 or 1. The optimal solution of the constrained weighted median algorithm is one of the positive breakpoints or zero; thus, the nonnegativity constraint forces the optimal solution of the constrained weighted median algorithm to be either 0 or 1. 

Therefore, \name\ algorithm can be used to solve L1-BMF at the same computational cost as L1-NMF ignoring the binary constraint, as long as a binary initialization is provided. The main advantage of the \name\ algorithm is that the binary constraint does not need to be directly imposed since it is automatically satisfied by construction. This is not the case for other BMF algorithms that minimize the Frobenius error measure instead.
\end{comment}



\section{Numerical experiments}
\label{sec:num_res}

%This section is devoted to numerical experiments. 
Our goal in this section is fourfold: 
\begin{enumerate}

\item[(i)] Show how \name\ allows a significant acceleration of CD for sparse data. 

    \item [(ii)] Show that the wL1-NMF is preferable to other NMF models (namely, FroNMF, KL-NMF, and L21-NMF) in situations where the data is affected by heavy tailed noise or outliers. 

    \item[(iii)] Compare \name\ with the state of the art algorithms for L1-NMF, in particular, 
    the  standard CD~\cite{ke2003robust,ke2005robust,gillis2011dimensionality}, 
    the BCD approach with Nesterov Smoothing (NS) by Guan et al.~\cite{guan2012mahnmf}, 
    and the projected subgradient (SUB) approach by  Rahimi et al.~\cite{rahimi2024projected}, showing the significant  computational advantage brought by \name\ when dealing with sparse data. 
    
\item[(iv)] Show that suitable choices of the penalization parameter $\lambda$ in wL1-NMF lead to significantly better results than L1-NMF ($\lambda = 1$).   
\end{enumerate}

We present numerical experiments on both synthetic and real datasets. In particular, we test our model and algorithm in noisy image compression, topic modeling, and matrix completion in the presence of false zeros.  

\paragraph{Setup} We use HALS to compute FroNMF, MU for KL-NMF following the implementation in~\cite{gillis2020nonnegative}, and MU for L21-NMF from~\cite{kong2011robust}. 
The smoothing parameter in the BCD approach from~\cite{guan2012mahnmf} with Nesterov smoothing is initialized at 0.5, and the number of inner iterations to compute each update of the factor is set to 30; the step size in the subgradient method is selected according to a linearly decaying rule as considered in~\cite{rahimi2024projected}, starting from an initial value of 0.01. 

 
As explained in Section~\ref{sec:sparsity_inducing}, when dealing with sparse data, some initializations of the factors may yield overly sparse L1-NMF. According to our experience, a few iterations of HALS provide a reasonable starting point for computing L1-NMF. Therefore, if not otherwise specified, we initialize the algorithms solving L1-NMF or wL1-NMF  with 10 iterations of HALS. 
 HALS is itself initialized randomly. 
 %and the starting point is normalized such that the norm of the low-rank product matches the norm of the data and the norms of the factors are equal.  
 All the results are averaged over 10 random initializations. Let $X$ be the data matrix and $(W^k,H^k)$ the approximation at step $k$ of any algorithm. If not otherwise specified, we stop each algorithm when the difference in the relative error, $\lVert X - W^k H^k \rVert_1 / \lVert X \rVert_1$, after two consecutive iterations, is below a tolerance of $10^{-6}$, or when a fixed time limit is reached. 
 %The norm of the residual changes according to the NMF model at hand and we divide the residual by the L1 norm of $X$ to have a uniform normalization. \gs{I thought this was the most natural thing to do so I used this normalization, if you think it is better to change it by dividing each residual for its proper norm of X let me know.} 
 The time limit is set according to the application being considered. Since the projected subgradient method is non-monotonic in decreasing the objective function, the only stopping criterion considered is the time limit. 

The algorithms are implemented in Julia version 1.11.4 (2025-03-10) on a 64-bit Samsung/Galaxy with 11th Gen Intel(R) Core(TM) i5 1135G7 @ 2.40GHz and 8 GB of RAM, under Windows 11 version 23H2. The code is available online from \url{https://github.com/giovanniseraghiti/wL1-NMF}. 

 
%\subsection{Synthetic data} The first experiment aims to replicate the example shown in the introduction in a more realistic scenario. We first consider binary block diagonal matrices, representing well-disjoint clusters; the matrix $X$ contains $r$ square diagonal blocks of 1, with a dimension of $d$ and zero elsewhere. Thus, the rank of $X$ is equal to $r$. We add sparse noise by randomly changing pixels to their opposite binary value with a probability of $\gamma$. We fix $d=20$ and we consider different examples with $r=\dots$ and $\gamma=\dots$; Then, we compare the capabilities of different NMF models to recover the original clusters. 


\subsection{Synthetic data} %\ngc{This does not follow your three points in the intro --confusing}\gs{Maybe we can switch point ii and i in the points in the intro? Would it be more straightforward? Any other ideas?} 
We first show the computational advantage of using \name\ with respect to the original version of CD for L1-NMF on synthetic data. 
We generate different sparse $m\times n$ matrices with dimensions $(m,n)=(100,200),$ $(300, 400),$ $(500, 600), $ $(800, 1000)$ whose entries follow a uniform distribution in $[0,1]$. 
Then, we sparsify each matrix by randomly setting some of its entries to zero. 
We fix the rank to $r=20$ for all cases and run both algorithms for 30 iterations, ignoring any other stopping criteria. We specify that, in this first experiment, we are not interested in the quality of the approximations.  We focus on comparing the computational efficiency of the two versions of CD, which perform the same updates but in a different way. 
%For this reason, we do not report the relative error which has no reason to be small since the matrices considered are high rank.  
Table~\ref{tab:synt_comp} reports the average CPU time of \name\ and standard CD. 
It also reports the theoretical factor of gain of using \name\ over standard CD, which is the ratio between the asymptotic complexities of the two algorithms, that is, \begin{equation}
     \sigma=\frac{mn\log(mn)}{\nnz(X)\log(\nnz(X))}.
     \label{eq:comp_fact}
\end{equation}
%The quantity $\sigma$ represents the ratio among the asymptotic complexities of the original CD and the \name\ scheme; thus, 
The quantity $\sigma$ is the theoretical number of iterations that \name\ can perform in the time required for one standard CD iteration. 
We compare the theoretical factor of gain, $\sigma$, with the practical gain, which is the ratio between the average CPU times of CD and \name \ (last column of Table~\ref{tab:synt_comp}). 
\begin{table}[h!]
\centering
\begin{tabular}{ c || c | c | c | c | c  } 
 \toprule % \hline
$m \times n$ & sparsity  & \name\  &  CD & $\sigma$ &  gain \\
  \midrule% \hline
     $100 \times 200$  \multirow{5}{*}{}& $25 \%$  & 0.31  & 0.40 & 1.37 & 1.29 \\
 &   $50 \%$  & 0.20 & 0.40 & 2.15 & 2.0 \\
  &  $80 \%$   & 0.08 &  0.39 & 5.97 & 4.88\\
  
   \hline
 $300 \times 400$  \multirow{5}{*}{}& $25 \%$  & 2.20   & 2.94 &1.37 & 1.33 \\
 &   $50 \%$  & 1.59 & 3.05 & 2.13 &  1.92\\
  &  $80 \%$   & 0.61 &  2.67 &5.80 & 4.38\\
  
   \hline
 $500 \times 600$  \multirow{5}{*}{}& $25 \%$  &  6.40  &  8.56 & 1.36 & 1.34 \\
 &   $50 \%$  & 4.34 &  8.85 & 2.12 & 2.04 \\
  &  $80 \%$   &  1.70 &  8.16 & 5.73 & 4.8 \\

   \hline
 $800 \times 1000$  \multirow{5}{*}{}& $25 \%$  &  20.53  &  27.68 & 1.36 & 1.35 \\
 &   $50 \%$  & 14.08 &  30.02 & 2.11 & 2.13 \\
  &  $80 \%$   & 4.75 &  27.65 & 5.67 & 5.82\\
 
 \bottomrule 
\end{tabular}
\caption{Comparison between \name\ and the original CD for L1-NMF on randomly generated data for different dimensions ($m \times n$) and sparsity of the input matrix. 
The third and fourth columns report the average CPU time per iteration for \name\ and CD, respectively, in seconds.  
The fifth and sixth column report the theoretical gain, $\sigma$ in~\eqref{eq:comp_fact}, and the practical  gain, resp. }   
\label{tab:synt_comp}
\end{table}

Table~\ref{tab:synt_comp} illustrates that \name\ follows closely the theoretically expected gain compared to the standard CD. %clearly shows the ability of \name\ to scale with the sparsity in the data.  
In fact, when the sparsity increases from $25\%$ to $80\%$, the computational time decreases  approximately by a factor of four. 
On the contrary, the standard CD algorithm has a complexity that does not depend on the sparsity of the data, resulting in similar CPU times per iteration for each level of sparsity.


\subsection{MNIST dataset} The MNIST dataset consists of a collection of images of 10 handwritten digits; each sample is a $28 \times 28$ gray and sparse  image. We consider $300$ randomly selected digit images, resulting in a matrix $X$ of dimensions $784 \times 300$, where each column of the matrix contains the vectorized image of one digit. %We normalize the dataset in the interval $[0,1]$. % We don't really care, does not influence the results 
Then, we add sparse noise to the data by randomly flipping each white pixel to black and each gray pixel to white with a probability $p$, thereby obtaining a noisy version of the dataset $\Bar{X}$.
For this instance, we fix the regularization parameter $\lambda$ in the wL1-NMF to 1, that is, L1-NMF. We compare different NMF models in Section~\ref{sub:mnist_NMF_gen}, and we focus on the performance of the algorithms solving L1-NMF in Section~\ref{sub:mnist_l1NMF}.  We stop each algorithm after 90 seconds or if the difference in the relative error between two consecutive iterations is smaller than $10^{-6}$. %Six levels of noise corrupting the data are considered. 
The rank of the factorization is fixed to $r=50$.


\subsubsection{Comparison of NMF models}
\label{sub:mnist_NMF_gen}
%At first, we test the robustness of the different NMF models. 
Given $(W^*,H^*)$, an NMF computed by an algorithm, we measure the quality of the factorization using the relative error in Frobenius norm with the ground truth data, that is, $\lVert X-W^*H^* \rVert_F/\lVert X \rVert_F$. We specify that it is beyond the purposes of this work to compare our method to standard denoising techniques, which, in general, do not require the reconstruction to be low-rank.
Figure~\ref{fig:comp_NMF_models_MNIST} displays the relative error for various NMF models, 
%the PSNR achieved by the L1-NMF, solved by the \name, against the other NMFs, 
for different values of the noise probability $p$. It shows that when the noise level is small ($p \leq 0.04$), KL-NMF, L2-NMF, and L21-NMF provide lower relative errors than L1-NMF, although comparable. 
However, as the noise level increases ($p \geq 0.08$), L1-NMF achieves the best performance. 
\begin{figure}
%\begin{minipage}[ht!]{.49\linewidth}
 %   \centering
       %\resizebox{\linewidth}{!}{\input{alpha2.tex}}
 %      \resizebox{\linewidth}{!}{\input{Tikz/PSNR_MNIST}}
%\end{minipage}
%\begin{minipage}[h]{.49\linewidth}
    \centering
    %\resizebox{\linewidth}{!}{ \input{alpha1.tex}}
    \resizebox{0.75\linewidth}{!}{ \input{Tikz/Err_MNIST}}
%\end{minipage}
 \caption{Comparison of the robustness of various NMF models on the MNIST dataset depending on the noise level. Relative error w.r.t.\ ground truth. Noisy data reports $\lVert X- \bar X \rVert_F/\lVert X \rVert_F$. %with the groudtruth image on the right for different values of the probability of the noise on the $x$-axis. 
 }
    \label{fig:comp_NMF_models_MNIST}
\end{figure}


This result is confirmed by the visual representations in Figure~\ref{fig:visual_rec}, which show examples of reconstructed digits, which are the columns of the NMFs $W^*H^*$ of $\Bar{X}$, reshaped as $28\times 28$ images.  
 L1-NMF is the only model able to recover the white background of the digits and provides much more accurate approximations, especially for higher noise levels.  
\begin{figure}[ht!]
\begin{center}
 \begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/noisy_004_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/noisy_008_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/noisy_012_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\vspace{0.5cm}

 \begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/l1_004_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/l1_008_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/l1_012_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\vspace{0.5cm}

  \begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/l2_004_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/l2_008_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/l2_012_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\vspace{0.5cm}

 \begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/l21_004_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/l21_008_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/l21_012_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\vspace{0.5cm}

 \begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/kl_004_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/kl_008_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\begin{minipage}[h]{.329\linewidth}
    \includegraphics[height=2.4cm]{Figures/kl_012_300im_90s_r50_mnist-eps-converted-to.pdf}
\end{minipage}
\end{center}
\caption{Examples of the low-rank representations of digits using different NMF models on the MNIST dataset.}
\label{fig:visual_rec}
\end{figure}



\subsubsection{Comparison of L1-NMF algorithms}
\label{sub:mnist_l1NMF}
%\gs{I changed error with residual, I think it is more correct, I also put the formula, give it a look.}
We now investigate the performance of different algorithms on the same MNIST dataset to solve L1-NMF in terms of final relative error, $\lVert \Bar{X} - W^* H^* \rVert_1 / \lVert \Bar{X} \rVert_1$. 
Table~\ref{tab:L1_MNIST_comp} reports the relative errors and the number of iterations performed by the different algorithms.  
 \begin{table}[h!]
\centering
\begin{tabular}{ c || c | c | c | c | c || c | c | c } 
 \toprule % \hline
Noise &  & NS \cite{cao2023manhattan} & SUB \cite{rahimi2024projected} &  \name\ (Alg.~\ref{alg:sparse_l1_nmf})  &  CD \cite{ke2003robust}   & sparsity & $\sigma$ & gain  \\
  \midrule% \hline
     $0\%$   \multirow{5}{*}{}& res &   0.453 & 0.448 & \textbf{0.428}  & 0.437   & 81$\%$ & 6.2 & 5.8\\
  &  iter   & 183 & $7.0 \cdot 10^3$ & 29 &  5 & & &\\
   \hline
  $4\%$  \multirow{5}{*}{}& res  & 0.60 & 0.593 & \textbf{0.572} & 0.581 & 79$\%$ & 5.4 & 4.8\\
  
   &   iter   & 177 & $6.8 \cdot 10^3$ & 24 &  5 & & & \\
   \hline
  $8\%$   \multirow{5}{*}{}& res  & 0.710& 0.703 &  \textbf{0.675}  & 0.685 & 76$\%$ & 4.8 & 4.4\\

   & iter   & 178 & $7.0 \cdot 10^3$ & 22 &  5  & & & \\

  \hline
 $12\%$  \multirow{5}{*}{}& res  & 0.789& 0.783 &  \textbf{0.750}  & 0.760 & 74$\%$ & 4.3 & 4.0 \\
 
  &   iter   & 166 & $6.8 \cdot 10^3$ & 20 &  5 & & &\\

   \hline
  $16\%$  \multirow{5}{*}{}& res  & 0.840& 0.837 &  \textbf{0.804}  & 0.813 & 71$\%$ & 3.9 & 3.8\\
 
  &   iter   & 176 & $6.7 \cdot 10^3$ & 19 &  5 & &  &\\ 
 \bottomrule 
\end{tabular}
\caption{Comparison of L1-NMF algorithms on the MNIST data with different levels of noise. We report the sparsity of the noisy data (sparsity), the theoretical factor of compression of \name\ over CD ($\sigma$ defined in~\eqref{eq:comp_fact}), the practical gain (gain) of \name\ over CD, the final relative residual (res), and number of iterations performed (iter).}  
\label{tab:L1_MNIST_comp}
\end{table} 
 \name\ achieves the lowest residual for every level of noise, followed by the original CD algorithm, which is approximately 3 to 5 times slower (depending on the sparsity of the matrix $\bar X$, which depends on $p$). %For example, in the absence of noise, the sparsity of $X$ is $81 \%$, in which case the theoretical acceleration gain, $\sigma$ defined in~\eqref{eq:comp_fact}, is equal to $6.2$, while the practical acceleration factor is $28/6 = 4.67$. \ngc{Maybe just mention the case with no Noise? Please fill in the blanks} \gs{Shall I leave it on the table?} \ngc{Yes, and add the practical acceleration factor in another column?} % as the theoretical factor of compression $\sigma$ (defined in~\eqref{eq:comp_fact}) confirms.  \ngc{Mention sparsity of the data set and the theoretical $\sigma$ value.} 
% \gs{This actually changes with the level of noise that flips zeros into black (in fact the number of iterations performed by \name\ decreases if the noise increases), shall I add two columns in the table with sparsity of $\Bar{X}$ and $\sigma$ or just mention it here? For now I added it on the table, let me know what you think}
  NS and SUB have a consistently smaller computational cost per iteration; however, they cannot reach the same accuracy as the CD-based  approaches.


\subsection{Matrix completion with false zeros}

Let us now consider a nonnegative low-rank matrix completion problem where the missing entries correspond to zeros in the observed matrix. We are interested in a setting where some missing values are due to anomalies or errors in the measurements, e.g., as in imaging mass spectrometry~\cite{moens2025preserving,gonzalez2023nectar}, see Section~\ref{sec:w_ell_1_nmf}. 
We generate synthetic data where the missing values are of two types: missing completely at random or missing because they are below a fixed threshold.   
This represents a setting where some of the zeros are \textit{false zeros}, corresponding to low-intensity values. 
We aim to show that, in this application,  wL1-NMF performs better than the standard L1-NMF and low-rank nonnegative matrix completion, corresponding to the choices $\lambda=1$ and $\lambda=0$, respectively. %In fact, adding prior assumptions about the distribution of the ground truth solution in the unobserved values, rather than considering them as completely unknown, might be beneficial by incorporating the information carried by the false zeros. 

Specifically, we generate two matrices $\hat{W} \in \mathbb{R}_+^{m \times r} $ and $\hat{H} \in \mathbb{R}_+^{r \times n}$ with $m=100$, $n=50$, and $r=20$ from a uniform distribution in $(0,1)$, and we form the ground truth low-rank matrix $\hat{W} \hat{H}$, which is dense. %Since the wL1-NMF model is the MLE estimator in the presence of additive Laplace noise, 
We then add a noise matrix $N$ to the ground truth data, whose entries follow a zero mean Laplace distribution with diversity $\sigma=0.1$, to obtain the noisy matrix ${X}=\Hat{W} \Hat{H} + N$. %where the diversity $\sigma$ is chosen small enough to ensure that ${X}$ remains nonnegative with high probability. 
Finally, we replace a proportion $q_1 \in [0,1)$ of the entries of $X$ with zero.
We consider two types of zeros: a fraction $q_2$ of type I and a fraction $1-q_2$ of type II, with $q_2 \in [0,1]$: 
\begin{enumerate}
    \item type I sets the smallest entries of $X$ to zero, for a total of $q_1 q_2 mn$ entries. These entries can be seen as small entries thresholded to zero. We call them false zeros. 

    \item type II sets randomly selected entries of $X$ to zero, for a total of $q_1 (1-q_2) mn$ entries. We call them missing entries, as they do not bring any useful information since they were picked at random.  
    %    These entries can be considered as missing. 
\end{enumerate}
Hence for $q_2 = 0$, all zeros are actually missing and wL1-NMF with $\lambda = 0$ should perform better than $\lambda = 1$ (L1-NMF). On the contrary, for $q_2 > 0$, we expect wL1-NMF with $\lambda > 0$ to perform better since some of the zeros have been obtained by thresholding small values to zero.  
% We fix a target percentage of missing entries for the matrix $X$, $q_1$, so that $c=q_1mn$ entries are replaced by zeros which are missing values. Since we consider two types of missing values, we partition them into two groups according to a ratio $q_2$, which corresponds to the percentage of false zeros among the missing entries. In particular, since in our setting false zeros represents low-frequency data, we remove the smallest $q_2 c$ values and $(1-q_2)c$ entries are also removed at random among the remaining ones. Thus, the choice $q_1=60 \% $, $q_2=25\%$ results in a matrix that has $60\%$ missing values (zeros) and the $25\%$ of those are false zeros (representing the smallest values). \gs{I tried to clarify, not easy} \ngc{Still unclear... let see if I understood}  
%We choose a percentage $q_1$ of the entries in ${X}$ to be missing. We also consider a fraction $q_2$ of $q_1$ to be missing at random, by substituting zeros for randomly selected entries in ${X}$. \ngc{This is very unclear, please clarify this last sentence.} 
%Moreover, we remove the $1-q_2$ smallest entries among the remaining ones. \ngc{This is very unclear, please clarify this last sentence.} 
%Therefore, we have $q_1q_2 \%$ entries that are missing randomly and $(1-q_2)q_1 \%$ of the smallest entries missing as well. %A similar idea was used for Euclidean distance matrix completion in~\cite{gillis2025extrapolated}. 

 Figure~\ref{fig:lambda_mc} shows the final relative errors with the original low-rank matrix, $\lVert \Hat{W} \Hat{H}-W^*H^* \rVert_F/\lVert \Hat{W} \Hat{H} \rVert_F$, 
 where $(W^*,H^*)$ is the  wL1-NMF solution for different values of $\lambda$, and for different combinations of percentages $q_1$ of zeros in the data and fractions $q_2$ of false zeros. 
\begin{figure}[ht!] 
\begin{center}
     \begin{minipage}[h!]{\linewidth}
     \begin{center}
         \resizebox{0.9\linewidth}{!}{\input{Tikz/Rec_sys_false}}
       %\resizebox{\linewidth}{!}{\input{Tikz/Matrix_comp_graph}}
     \end{center}
     \end{minipage}    
       \caption{Relative error averaged over 10 runs between the noiseless data matrix and the wL1-NMFs for different values of~$\lambda$. 
       Each plot corresponds to different percentages $q_1$ of zeros, and each line represents a percentage $q_2$ of false zeros among the missing values. 
       }
       \label{fig:lambda_mc} 
        \end{center} 
\end{figure} 
 %It analyze different scenarios corresponding to different combinations of percentages $q_1$ of zeros in the data and fractions $q_2$ of false zeros. 
 Figure~\ref{fig:lambda_mc} shows  that the choice of $\lambda=1$ (L1-NMF) yields worse results than all the other values of $\lambda$. 
 Moreover, as expected, $\lambda=0$ is a good choice when there are no false zeros among the zero entries (that is, $q_2 = 0$). 
 However, small values of $\lambda$ perform well even when $q_2 =0$, while they perform even better than $\lambda = 0$ for a high level of missing entries (namely, $q_1 = 0.7$, $q_2 = 0$). This is because the second term in  wL1-NMF acts as a regularizer which is well-known to improve the accuracy of low-rank factorization-based recommender systems when a large portion of the data is missing, avoiding overfitting~\cite{koren2009matrix}. Note also that this regularizer would be particularly meaningful  when data is not missing at random and missing entries may correspond to a negative preference~\cite{wang2018modeling}.  It would be interesting to use wL1-NMF in this context, which would be more robust than standard least-squares-based approaches~\cite{koren2009matrix}; this is a topic of further research. 
 
 In the presence of false zeros ($q_2 > 0$), L1-NMF has significantly larger relative errors than wL1-NMF, while wL1-NMF with $\lambda = 0$ 
 gets significantly worse as the number of false zeros increases (that is, $q_2$). A good choice of $\lambda$ is therefore crucial to obtain good factorizations in the presence of false zeros. wL1-NMF does not appear to be too sensitive to this parameter, as values of $\lambda$  between 0.01 and 0.05 provide good results in all cases. 
\begin{comment}
\begin{table}[ht]
\centering
\begin{tabular}{|c|c|c|c|c|c|c|c|c|}
\hline
 \multicolumn{2}{|c}{}  & \multicolumn{7}{|c|}{wL1-NMF} \\
\hline
 &  & \multicolumn{7}{c|}{$\lambda$} \\
\hline
\hspace{-0.2cm} $q_1$ \hspace{-0.2cm} & $q_2$& \hspace{-0.3cm} $0$ \hspace{-0.3cm} &$0.01$ &$0.03$ & $0.05$ &$0.07$ &$0.1$&$1$ \\
\hline
\multirow{5}{*}{$40\%$} & $0\%$& 0.07 & 0.07  & 0.07 & 0.07 & 0.07  & 0.07 & 0.42\\
\cline{2-9}
                       & $25\%$ & 0.09 & 0.08  & 0.08 & 0.08  & 0.08 & 0.08 & 0.39\\
\cline{2-9}
                       &$50\%$  & 0.10 & 0.09  & 0.09 & 0.09 & 0.09 & 0.10 & 0.42\\
\cline{2-9}
                       & $75\%$& 0.13 & 0.13  & 0.12 & 0.11 & 0.12 & 0.14 & 0.45\\
\cline{2-9}
                       & $100\%$& 0.14 & 0.13  & 0.12 & 0.12 & 0.14 & 0.17 & 0.50\\
\hline
\multirow{5}{*}{$50\%$} & $0\%$& 0.08 & 0.08  & 0.08 & 0.08 & 0.08 & 0.09 & 0.61\\
\cline{2-9}
                       & $25\%$ & 0.11 &  0.10 & 0.09 & 0.09 & 0.09 & 0.10& 0.55\\
\cline{2-9}
                       &$50\%$  & 0.16 & 0.13  & 0.13  & 0.13 & 0.14 & 0.16& 0.53\\
\cline{2-9}
                       & $75\%$& 0.16 &  0.14 & 0.13  & 0.15 &  0.18 & 0.20 & 0.55\\
\cline{2-9}
                       & $100\%$& 0.18 & 0.14  & 0.16 & 0.21 & 0.23 & 0.26 & 0.58\\
\hline
\multirow{5}{*}{$60\%$} & $0\%$& 0.09 & 0.10  & 0.09 & 0.09 & 0.09 & 0.09 & 0.79\\
\cline{2-9}
                       & $25\%$ & 0.14 &  0.13 & 0.11 & 0.12 & 0.12 & 0.15 & 0.70 \\
\cline{2-9}
                       &$50\%$  & $10^4$ &  0.18 & 0.16 & 0.16 & 0.19 & 0.25 & 0.65\\
\cline{2-9}
                       & $75\%$& $10^4$ & 0.20  & 0.21 & 0.23 & 0.25 & 0.30 & 0.64 \\
\cline{2-9}
                       & $100\%$& $10^3$ & 0.22  & 0.23 & 0.26 & 0.30 & 0.37 &0.67 \\
\hline
\multirow{5}{*}{$70\%$} & $0\%$& 0.13 &  0.13 & 0.13 & 0.12 & 0.12 & 0.13 & 0.76\\
\cline{2-9}
                       & $25\%$ & 0.39 & 0.19  & 0.17 & 0.19 & 0.22 & 0.26 & 0.75\\
\cline{2-9}
                       &$50\%$  & $10^4$ &  0.26 & 0.26 & 0.26 & 0.32 & 0.38 & 0.77\\
\cline{2-9}
                       & $75\%$& $10^5$ & 0.30  & 0.34 & 0.36 & 0.38 & 0.43& 0.83\\
\cline{2-9}
                       & $100\%$& $10^7$ &  0.35 & 0.38 & 0.43 & 0.43 & 0.48& 0.91\\
\hline
\multirow{5}{*}{$80\%$} & $0\%$& 0.27 &  0.21 & 0.18 & 0.18 & 0.20 & 0.22& 1\\
\cline{2-9}
                       & $25\%$ & 97.8 &  0.33 & 0.29 & 0.33 & 0.37 & 0.42 & 1\\
\cline{2-9}
                       &$50\%$  & $10^4$ & 0.40  & 0.39 & 0.46 & 0.49 & 0.55& 0.88\\
\cline{2-9}
                       & $75\%$& $10^4$ & 0.45  & 0.44 & 0.51 & 0.53 & 0.59& 0.84\\
\cline{2-9}
                       & $100\%$& $10^4$ & 0.51  & 0.51 & 0.51 & 0.59 & 0.61& 0.84\\
\hline
\end{tabular}
\caption{}
\label{tab:false_zeros}
\end{table}  
\end{comment}



\subsection{Topic modeling}
%\gs{this experiments in topic modeling is not averaged over ten runs, it would take a lot to repeat it 10 times. I specified it in the text. Also how to deal with the topics and relevant words in different runs? Do I have to think about it or it is fine on a single run like it is now?  }
We consider the Topic Detection Task (TDT2) dataset, which contains a collection of articles from various sources, collected from January to June of 1998, on $r = 30$ selected topics. It consists of a matrix $X \in \mathbb{R}^{m \times n}$ with $m =19528$ and $n = 9394$, where the $(i,j)$th entry is the number of occurrences of the word $i$ in document $j$.
Such datasets are typically very sparse since only a small portion of the words in the dictionary are used in each document. The percentage of zero entries in TDT2 is $99.37\%$. 
When computing the NMF of $X \approx WH$, the columns of $W$ represent topics, that is, bags of words that are used to reconstruct multiple documents (the columns of $X$), while the matrix $H$ tells us which topics are discussed in which document~\cite{lee1999learning}. 
We present this application for two reasons: 
\begin{enumerate}
    \item[(a)] This extremely sparse scenario is a typical situation where L1-NMF might provide too sparse solutions, and we show that wL1-NMF leads to significantly better  solutions. %by reducing the sparsification effect described in Section~\ref{sec:sparsity_inducing}. 

\item[(b)] The dimensions of the dataset cause the NS and SUB algorithms to run out of memory after few iterations and make the standard CD approach extremely slow (more than 4 hours per iteration, $\sigma$ is equal to 175 for this dataset) on the  computer we used for these experiments. 
Exploiting the sparsity of the data, \name\ has a reasonable cost per iteration (about 70 seconds). 
%In fact, according to the theoretical gain $\sigma$ in~\eqref{eq:comp_fact}, \name\ would be about 175 times faster than the standard CD for TDT2. 
\end{enumerate}

It is beyond the scope of this paper to compare our algorithm to state-of-the-art techniques in topic modeling, and we compare our results to FroNMF.  
 The wL1-NMF factorizations are computed by 15 iterations of \name\, initialized with 3 iterations of HALS, while we use 30 iterations of HALS to compute FroNMF.
 We present the results for a single random initialization to be able to compare the recovered topics. 
 The computational cost of wL1-NMF is consistently larger than that of FroNMF. In fact, \name\ requires approximately 70 seconds, while HALS requires approximately 1 second per iteration. 
 %Nevertheless, according to the factor of compression $\sigma$, which is equal to 175 for this data set, the standard CD requires more than 3 hours to perform a single iteration. 
 We observed that, in such a sparse scenario, initializing the wL1-NMF model with HALS is crucial as it allows to obtain reasonably dense results for larger values of $\lambda$. If random initialization is used, the $\lambda$ parameter needs to be smaller in order to prevent extremely sparse solutions.

Table~\ref{tab:topic_mod} reports the average number of nonzero entries per column in $W$, that is, the average number of words in each topic. It also displays the number of topics with less than 5 words, as we consider them less meaningful. 
\begin{table}[ht!]
\begin{center}
\begin{tabular}{c||c|c|c|c|c|c|c}
\hline
&  \multicolumn{6}{c|}{wL1-NMF} & FroNMF\\
 \midrule
$\lambda$ & 0 & 0.01 & 0.04 & 0.08 & 0.1 & 1.0 & / \\
\hline
Average words per topic & 12898 & 1780 & 417 & 170 & 123 & 4 & 5275  \\
\hline
topics less than 5 words & 0 & 0 & 0 & 0 & 1 & 24 & 0  \\
\hline
\end{tabular}
\caption{Impact of the parameter $\lambda$ on the 
wL1-NMF solution on the TDT2 data.} 
\label{tab:topic_mod}   
\end{center}
\end{table} 
As expected, the larger $\lambda$, the sparser the factor $W$ in wL1-NMF, that is, the fewer the number of words describing each topic. This is a clear example where L1-NMF ($\lambda=1$) produces too sparse solutions since 24 out of 30 topics are described by less than 5 words due to the original sparsity in the data. 
A good choice of the $\lambda$ parameter, such as $\lambda=0.08$, enhances the interpretability of the factorization by providing sparser topics. 
In fact, %as shown in Table~\ref{tab:topic_mod_qual} (see below for a discussion), 
wL1-NMF 
%provides results comparable to FroNMF while 
removes approximately $97 \%$ of the words for each topic compared to FroNMF; from 5275 words on average for FroNMF, to 170 for wL1-NMF with $\lambda = 0.08$.  
A sparser factorization means a more straightforward interpretation and fewer mixed topics. 
%We also compare the extracted topics by wL1-NMF and FroNMF from a qualitative point of view. 

Let us provide a qualitative comparison.  
Table~\ref{tab:topic_mod_qual} shows, for two topics, the ten most important words, that is, the words corresponding to the ten largest entries in each column of $W$. 
The other topics are provided in the supplementary material. 
Overall, the topics extracted by FroNMF and wL1-NMF with $\lambda=0.08$ are similar; thus, we were not able to identify which factorization provides the best topic extraction in general, and the results vary from topic to topic. %Table~\ref{tab:topic_mod_qual} shows an example of the ten most relevant words for two of   %for the first, wL1-NMF provides a clearer overview of the topic, while for the second, FroNMF is more precise. \ngc{first clarisfy what are these words: 10 most important. Second, clarify why you say this?} 
%When  $\lambda$ is not tuned correctly, wL1-NMF yields too sparse (for $\lambda$ close to one) or not well separated topics (for $\lambda$ close to zero). We analyze this behavior by showing in Table~\ref{tab:topic_mod_qual} the results for two choices of $\lambda$: the first, $\lambda=0.01$, not tuned correctly, and the second, $\lambda=0.08$, which is well-tuned. 
\begin{table}[h!]
\begin{center}
\begin{tabular}{c|c|c||c|c|c}
\hline
  \multicolumn{2}{c|}{wL1-NMF} & FroNMF & \multicolumn{2}{c|}{wL1-NMF} & FroNMF\\
 
 \midrule
   0.01  & 0.08   &  /  &  0.01  &   0.08   &  /  \\
\hline
  \textcolor{red}{kaczynski} & gov. & gov. &  \textcolor{red}{european} & \textcolor{red}{court}  & super\\

   \textcolor{red}{algerian} & party  & party & \textcolor{red}{martin} & \textcolor{red}{supreme}  & bowl \\
   \textcolor{red}{algeria} & india  &  \textcolor{red}{algeria} & \textcolor{red}{tour} & \textcolor{black}{bowl}  & game \\
   \textcolor{red}{abacha} & bjp  & election & \textcolor{red}{documents} & \textcolor{black}{super}  & denver \\
  srinagar & election  & elections &  \textcolor{red}{algeria} & bulls & broncos \\
 \textcolor{red}{massacres} & vajpayee  &  \textcolor{red}{algerian} & \textcolor{red}{privilege} & chicago  & packers \\
 gandhi & hindu  & \textcolor{red}{killed} & \textcolor{red}{golf} & jazz & green \\
  \textcolor{red}{algiers} & minister & india &  \textcolor{red}{pga} & utah  & bay \\
 bjp & elections  &  political &\textcolor{red}{supreme} & jordan  &  san\\
 india & voting  & \textcolor{red}{islamic} & \textcolor{red}{court} & game  & football \\
\hline
 \multicolumn{3}{c||}{Topic 6: Vajoayee election} &  \multicolumn{3}{c}{Topic 17: Sport / Super bowl}\\
 \hline
\end{tabular}
\caption{Ten most important  words for two extracted topics (Topic 6 and Topic 17) by  FroNMF and wL1-NMF on the TDT2 dataset. 
In red, we highlight the words that do not have a clear connection with the topic that the majority of the other words refer to. 
%In blue, we highlight the words that are related to the topic of the majority of the other words. \ngc{why is court in blue? This rather looks like 'supreme court' together, no?} \gs{I thought it could be basketball "court" and then it has also detected "supreme" because those two words often appears together in "supreme court"} \ngc{Shouldn't be more words in blue (e.g., all in topic 6 for wL1-NMF with $\lambda = 0.08$ and FromNMF for topic 17)?}  
%\gs{My original idea was to have three categories: the one that are related to the main topic (black), the one that are related to a similar topic, like football instead of basketball (blue), the one that does not have a close connection (red). If you think it is better to simplify I can reduce to just two categories. }
}
\label{tab:topic_mod_qual}
\end{center}
\end{table}
For $\lambda = 0.01$, wL1-NMF provides too dense solutions, with words that have no clear connection with each other (Topic 6) or words that are not related to the topic (Topic 17). 
wL1-NMF with $\lambda = 0.08$ provides a more meaningful Topic 6, in our opinion, related to the 1998 election of Vajpayee of the BJP party as prime minister of India. Even though the same event is covered by FroNMF, there are some unrelated words, such as "Algeria" and "killed". Conversely, Topic 17 extracted by FroNMF discusses the American football game (Super Bowl), in which the Denver Broncos faced the Green Bay Packers.
By contrast, the topic extracted by wL1-NMF with $\lambda = 0.08$ also contains two words  (``court'', ``supreme'') that are not related to the topic and mixes in vocabulary from the NBA finals between the Chicago Bulls and the Utah Jazz.
It therefore provides a more mixed topic than FroNMF. 
 %Although the words "bowl" and "super", highlighted in blue in Table~\ref{tab:topic_mod_qual}, still refers to a sport event, they are not related to basketball but they belong to the lexicon of football. On the other hand, the topic extracted by the FroNMF clearly discusses  All the ten most relevant words refers to this event, resulting in a more meaningful topic. 



\section{Conclusion}

In this paper, we considered L1-NMF that minimizes the component-wise L1 norm between the data matrix and its low-rank approximation. 
We first established the NP-hardness of L1-NMF even in the rank-1 case. Then, we showed that L1-NMF produces sparser solutions than L2-NMF for sparse data. This motivated us to introduce  weighted L1-NMF (wL1-NMF), suitable for sparse data corrupted by false zeros. 
We proposed a CD algorithm for wL1-NMF, 
namely \name, that scales linearly with the number of nonzeros in the data, up to a logarithmic factor. It is, to the best of our knowledge, the first L1-NMF algorithm that can handle large-scale sparse data. 
We performed extensive numerical experiments on synthetic and real datasets, showing that wL1-NMF outperforms other NMF models when the data is  affected by sparse noise or false zeros. Moreover, our new algorithm, \name, performs well against the state of the art for L1-NMF and reduces the computational time of the original CD algorithm consistently when the data is sparse. We finally showed that wL1-NMF can produce meaningful results comparable to FroNMF on a large and sparse document dataset (namely, TDT2). 




\bibliographystyle{spmpsci.bst}
\bibliography{bib.bib}





\newpage 




\section*{Supplementary material}

Table~\ref{tab:topic_mod_qual_sup} provides the extracted topics for the TDT2 dataset for wL1-NMF with $\lambda=0.08$ and for FroNMF. 


\setlength{\tabcolsep}{2pt}
\begin{longtable}[H]{c|c||c|c||c|c}

\hline
wL1 & Fro & wL1 & Fro & wL1 & Fro \\
 \midrule
\endfirsthead

\hline
wL1 & Fro & wL1 & Fro & wL1 & Fro \\
 \midrule
\endhead

spkr & saddam & iraq & weapons & spkr & president \\
president & secretary & iraqi & iraq & president & clinton \\
clinton & hussein & team & inspectors & clinton & house \\
saddam & albright & baghdad & iraqi & china & white \\
iraq & iraqi & sanctions & baghdad & says & clintons \\
hussein & president & weapons & butler & chinese & allegations \\
house & military & iraqis & sites & government & jones \\
white & weapons & ritter & inspections & correspondent & sexual \\
albright & foreign & butler & iraqs & rights & washington \\
diplomatic & madeleine & inspectors & presidential & human & china \\

\hline
\multicolumn{2}{c||}{Topic 1} &
\multicolumn{2}{c||}{Topic 2} &
\multicolumn{2}{c}{Topic 3}\\
\hline
\hline

percent & percent & today & today & government & government \\
market & stock & checking & top & party & party \\
stock & points & stories & stories & india & algeria \\
points & market & top & checking & bjp & election \\
dollar & index & secretary & hour & election & elections \\
stocks & stocks & hour & voice & vajpayee & algerian \\
index & prices & says & tomorrow & hindu & killed \\
yen & investors & washington & peter & minister & india \\
prices & dollar & general & jennings & elections & political \\
investors & markets & week & monica & voting & islamic \\

\hline
\multicolumn{2}{c||}{Topic 4} &
\multicolumn{2}{c||}{Topic 5} &
\multicolumn{2}{c}{Topic 6}\\
\hline
\hline

lewinsky & lewinsky & game & team & military & cable \\
house & starr & team & hockey & indonesia & military \\
white & house & play & game & students & italy \\
grand & white & goal & players & jakarta & italian \\
jury & monica & players & canada & suharto & marine \\
starr & grand & period & play & indonesian & plane \\
president & jury & end & teams & police & car \\
monica & independent & scored & goal & anti & accident \\
clinton & counsel & coach & olympic & troops & officials \\
investigation & investigation & round & czech & protests & ski \\

\hline
\multicolumn{2}{c||}{Topic 7} &
\multicolumn{2}{c||}{Topic 8} &
\multicolumn{2}{c}{Topic 9}\\
\hline
\hline

united & united & suharto & suharto & rudolph & gold \\
states & states & algeria & indonesia & clinic & won \\
japan & nations & algerian & president & bombing & medal \\
nations & american & algiers & jakarta & downhill & olympic \\
japanese & countries & islamic & indonesian & race & race \\
countries & britain & killed & economic & cup & mens \\
china & japan & suhartos & indonesias & eric & cup \\
asia & americans & militants & political & birmingham & womens \\
london & russia & economic & imf & alabama & slalom \\
american & canada & european & students & truck & olympics \\

\hline
\multicolumn{2}{c||}{Topic 10} &
\multicolumn{2}{c||}{Topic 11} &
\multicolumn{2}{c}{Topic 12}\\
\hline
\hline

\newpage

israel & israel & workers & workers & imf & economic \\
netanyahu & netanyahu & plant & general & economic & crisis \\
israeli & israeli & plants & motors & bank & asian \\
palestinian & palestinian & parts & strike & asian & financial \\
peace & peace & auto & plants & economy & asia \\
arafat & minister & flint & flint & financial & economy \\
palestinians & prime & motors & strikes & dlrs & billion \\
talks & talks & jobs & parts & crisis & countries \\
blair & arafat & strike & michigan & billion & south \\
london & palestinians & strikes & plant & government & japan \\

\hline
\multicolumn{2}{c||}{Topic 13} &
\multicolumn{2}{c||}{Topic 14} &
\multicolumn{2}{c}{Topic 15}\\
\hline
\hline

council & annan & court & super & pope & pope \\
weapons & council & supreme & bowl & cuba & cuba \\
annan & security & bowl & game & cuban & cuban \\
security & general & super & denver & castro & visit \\
iraq & iraq & bulls & broncos & visit & john \\
agreement & agreement & chicago & packers & paul & paul \\
inspectors & kofi & jazz & green & john & castro \\
resolution & secretary & utah & bay & havana & havana \\
butler & deal & jordan & san & mass & fidel \\
ambassador & baghdad & game & football & church & church \\

\hline
\multicolumn{2}{c||}{Topic 16} &
\multicolumn{2}{c||}{Topic 17} &
\multicolumn{2}{c}{Topic 18}\\
\hline
\hline

means & american & olympic & nagano & spkr & spkr \\
order & going & nagano & games & starr & voice \\
terms & public & hockey & japan & peter & news \\
view & news & winter & olympic & voice & correspondent \\
certainly & york & gold & olympics & jennings & peter \\
reason & city & olympics & winter & correspondent & announcer \\
interview & political & team & japanese & abc & jennings \\
change & viagra & medal & athletes & camera & abc \\
simply & end & games & sports & tonight & headline \\
actually & country & athletes & japans & rush & voa \\

\hline
\multicolumn{2}{c||}{Topic 19} &
\multicolumn{2}{c||}{Topic 20} &
\multicolumn{2}{c}{Topic 21}\\
\hline
\hline

mckinney & mckinney & iraq & iraq & winfrey & winfrey \\
kaczynski & court & gulf & military & government & show \\
sergeant & sexual & persian & gulf & show & texas \\
martial & major & aircraft & war & claims & beef \\
gene & sergeant & cable & attack & beef & oprah \\
mckinneys & gene & marine & strike & ruled & cattle \\
enlisted & army & italian & force & ranchers & ranchers \\
obstruction & martial & military & action & oprah & prices \\
armys & enlisted & accident & persian & disease & disease \\
soldier & misconduct & italy & kuwait & cattle & cow \\

\hline
\multicolumn{2}{c||}{Topic 22} &
\multicolumn{2}{c||}{Topic 23} &
\multicolumn{2}{c}{Topic 24}\\
\hline
\hline

\newpage

nuclear & nuclear & tobacco & tobacco & economic & says \\
pakistan & india & bill & industry & cohen & headline \\
india & pakistan & industry & bill & defense & voa \\
tests & tests & senate & companies & asia & announcer \\
test & test & smoking & smoking & secretary & correspondent \\
pakistani & pakistani & companies & legislation & william & hes \\
sanctions & indias & legislation & congress & asian & attorney \\
indias & indian & congress & billion & crisis & king \\
testing & testing & republicans & settlement & financial & ginsburg \\
indian & weapons & billion & settlement & says & reports \\

\hline
\multicolumn{2}{c||}{Topic 25} &
\multicolumn{2}{c||}{Topic 26} &
\multicolumn{2}{c}{Topic 27}\\
\hline
\hline

nagano & bombing & school & school & reporter & reporter \\
house & clinic & students & students & king & cnn \\
white & rudolph & jonesboro & arkansas & ray & going \\
tucker & birmingham & boys & jonesboro & cnn & news \\
texas & federal & arkansas & middle & hes & headline \\
death & alabama & police & shooting & martin & reports \\
mrs & womens & shooting & boys & james & martin \\
games & eric & children & police & family & tell \\
ceremony & killed & middle & fire & going & sources \\
execution & atlanta & teacher & teacher & luther & king \\

\hline
\multicolumn{2}{c||}{Topic 28} &
\multicolumn{2}{c||}{Topic 29} &
\multicolumn{2}{c}{Topic 30}\\
\hline
\hline
%\end{tabular}
\caption{Ten most relevant words for the remaining of the extracted topics by the FroNMF and wL1-NMF ($\lambda=0.08$) on the TDT2 dataset.}
\label{tab:topic_mod_qual_sup}
%\end{center}
%\end{table}
\end{longtable}




\end{document}
% This file was created by matlab2tikz.
%
%The latest updates can be retrieved from
%  http://www.mathworks.com/matlabcentral/fileexchange/22022-matlab2tikz-matlab2tikz
%where you can also make suggestions and rate matlab2tikz.
%
\definecolor{mycolor1}{rgb}{0.00000,0.44700,0.74100}%
\definecolor{mycolor2}{rgb}{0.85000,0.32500,0.09800}%
\definecolor{mycolor3}{rgb}{0.92900,0.69400,0.12500}%
\definecolor{mycolor4}{rgb}{0.49400,0.18400,0.55600}%
\definecolor{mycolor5}{rgb}{0.46600,0.67400,0.18800}%
\definecolor{mycolor6}{rgb}{0.30100,0.74500,0.93300}%
\definecolor{mycolor7}{rgb}{0.63500,0.07800,0.18400}%
\tikzset{every mark/.append style={scale=2.5}}

%
\begin{tikzpicture}

\begin{axis}[%
width=4.817in,
height=3.592in,
font=\huge,
at={(0.802in,0.7in)},
scale only axis,
xmin=0,
xmax=50,
xlabel style={font=\color{white!15!black}},
xlabel={\huge $m$},
ymin=0,
ymax=1.05,
ylabel style={font=\color{white!15!black}},
ylabel={\huge $P(\alpha_2>0)$},
axis background/.style={fill=white},
legend style={at={(0.737,0.048)}, anchor=south west, legend cell align=left, align=left, draw=white!15!black}
]
\addplot [color=mycolor1, line width=1.5pt, mark=o, mark repeat=5]
  table[row sep=crcr]{%
1	0.91\\
2	0.9739\\
3	0.994141\\
4	0.99879679\\
5	0.9997623901\\
6	0.999953954119\\
7	0.99999116128261\\
8	0.999998311643696\\
9	0.999999678312302\\
10	0.999999938789337\\
11	0.999999988360974\\
12	0.999999997787685\\
13	0.99999999957957\\
14	0.999999999920109\\
15	0.99999999998482\\
16	0.999999999997116\\
17	0.999999999999452\\
18	0.999999999999896\\
19	0.99999999999998\\
20	0.999999999999996\\
21	0.999999999999999\\
22	1\\
23	1\\
24	1\\
25	1\\
26	1\\
27	1\\
28	1\\
29	1\\
30	1\\
31	1\\
32	1\\
33	1\\
34	1\\
35	1\\
36	1\\
37	1\\
38	1\\
39	1\\
40	1\\
41	1\\
42	1\\
43	1\\
44	1\\
45	1\\
46	1\\
47	1\\
48	1\\
49	1\\
50	1\\
};
\addlegendentry{\huge $p$=0.9}

\addplot [color=mycolor2, line width=1.5pt, mark=star, mark repeat=5]
  table[row sep=crcr]{%
1	0.84\\
2	0.9104\\
3	0.961344\\
4	0.98480384\\
5	0.9942733824\\
6	0.997887217664\\
7	0.99922915835904\\
8	0.999720449009254\\
9	0.999898952043332\\
10	0.999963540815599\\
11	0.999986858309616\\
12	0.999995265714662\\
13	0.999998295001918\\
14	0.999999386069619\\
15	0.999999778958848\\
16	0.999999920419943\\
17	0.999999971350131\\
18	0.999999989685837\\
19	0.999999996286859\\
20	0.999999998663261\\
21	0.999999999518772\\
22	0.999999999826758\\
23	0.999999999937633\\
24	0.999999999977548\\
25	0.999999999991917\\
26	0.99999999999709\\
27	0.999999999998953\\
28	0.999999999999623\\
29	0.999999999999864\\
30	0.999999999999951\\
31	0.999999999999982\\
32	0.999999999999994\\
33	0.999999999999998\\
34	0.999999999999999\\
35	1\\
36	1\\
37	1\\
38	1\\
39	1\\
40	1\\
41	1\\
42	1\\
43	1\\
44	1\\
45	1\\
46	1\\
47	1\\
48	1\\
49	1\\
50	1\\
};
\addlegendentry{\huge $p$=0.8}

\addplot [color=mycolor3, line width=1.5pt, mark=diamond, mark repeat=5]
  table[row sep=crcr]{%
1	0.79\\
2	0.8299\\
3	0.894349\\
4	0.94044799\\
5	0.9679274749\\
6	0.983132712199\\
7	0.99124459322149\\
8	0.99548881554296\\
9	0.99768551782691\\
10	0.998815480661724\\
11	0.999394655108479\\
12	0.999690902096624\\
13	0.999842248466668\\
14	0.999919513237218\\
15	0.999958941706746\\
16	0.99997905725717\\
17	0.999989318297176\\
18	0.999994552060365\\
19	0.999997221469428\\
20	0.999998582925001\\
21	0.999999277284428\\
22	0.999999631412862\\
23	0.999999812019901\\
24	0.999999904129952\\
25	0.999999951106216\\
26	0.999999975064152\\
27	0.999999987282712\\
28	0.999999993514182\\
29	0.999999996692232\\
30	0.999999998313038\\
31	0.99999999913965\\
32	0.999999999561221\\
33	0.999999999776223\\
34	0.999999999885874\\
35	0.999999999941796\\
36	0.999999999970316\\
37	0.999999999984861\\
38	0.999999999992279\\
39	0.999999999996062\\
40	0.999999999997992\\
41	0.999999999998976\\
42	0.999999999999478\\
43	0.999999999999734\\
44	0.999999999999864\\
45	0.999999999999931\\
46	0.999999999999965\\
47	0.999999999999982\\
48	0.999999999999991\\
49	0.999999999999995\\
50	0.999999999999998\\
};
\addlegendentry{\huge $p$=0.7}

\addplot [color=mycolor4, line width=1.5pt, mark=Mercedes star, mark repeat=5]
  table[row sep=crcr]{%
1	0.76\\
2	0.7504\\
3	0.801856\\
4	0.85782784\\
5	0.9028658176\\
6	0.935376523264\\
7	0.95765793488896\\
8	0.972507862328934\\
9	0.982247745490518\\
10	0.988575642553931\\
11	0.992663245410516\\
12	0.99529441073313\\
13	0.996984396337363\\
14	0.998068403043177\\
15	0.998763133702539\\
16	0.999208147871587\\
17	0.999493111558601\\
18	0.999675550165818\\
19	0.999792335613449\\
20	0.999867088195538\\
21	0.999914933806316\\
22	0.999945556580511\\
23	0.999965155789315\\
24	0.999977699536276\\
25	0.999985727635663\\
26	0.999990865659803\\
27	0.999994154011465\\
28	0.999996258563014\\
29	0.9999976054786\\
30	0.999998467505612\\
31	0.999999019203315\\
32	0.999999372290011\\
33	0.999999598265563\\
34	0.999999742889942\\
35	0.999999835449556\\
36	0.999999894687713\\
37	0.999999932600135\\
38	0.999999956864086\\
39	0.999999972393015\\
40	0.999999982331529\\
41	0.999999988692179\\
42	0.999999992762994\\
43	0.999999995368316\\
44	0.999999997035723\\
45	0.999999998102862\\
46	0.999999998785832\\
47	0.999999999222932\\
48	0.999999999502677\\
49	0.999999999681713\\
50	0.999999999796296\\
};
\addlegendentry{\huge $p$=0.6}

\addplot [color=mycolor5, line width=1.5pt, mark=diamond, mark repeat=5]
  table[row sep=crcr]{%
1	0.75\\
2	0.6875\\
3	0.703125\\
4	0.74609375\\
5	0.7939453125\\
6	0.837646484375\\
7	0.87432861328125\\
8	0.903793334960938\\
9	0.926868438720703\\
10	0.944663047790527\\
11	0.958253145217896\\
12	0.968567788600922\\
13	0.976364806294441\\
14	0.982243087142706\\
15	0.986667056567967\\
16	0.989992663031444\\
17	0.992490682576317\\
18	0.994366104583605\\
19	0.995773624763387\\
20	0.996829741735382\\
21	0.997622067882958\\
22	0.998216431702929\\
23	0.998662264172552\\
24	0.998996668327091\\
25	0.999247486344157\\
26	0.999435607307537\\
27	0.999576701755363\\
28	0.999682524453877\\
29	0.999761892409085\\
30	0.999821418841153\\
31	0.999866063898034\\
32	0.99989954780711\\
33	0.999924660797125\\
34	0.99994349556874\\
35	0.999957621662003\\
36	0.999968216239226\\
37	0.999976162175782\\
38	0.999982121630017\\
39	0.999986591221603\\
40	0.999989943415748\\
41	0.999992457561584\\
42	0.999994343171074\\
43	0.999995757378249\\
44	0.999996818033658\\
45	0.999997613525229\\
46	0.999998210143915\\
47	0.999998657607933\\
48	0.999998993205948\\
49	0.99999924490446\\
50	0.999999433678344\\
};
\addlegendentry{\huge $p$=0.5}

\addplot [color=mycolor6, line width=1.5pt, mark=otimes, mark repeat=5]
  table[row sep=crcr]{%
1	0.76\\
2	0.6544\\
3	0.623296\\
4	0.63172864\\
5	0.6595480576\\
6	0.695357968384\\
7	0.73290325344256\\
8	0.76892026889175\\
9	0.80186194746907\\
10	0.831145388834019\\
11	0.856710938396576\\
12	0.878766475318724\\
13	0.897641411507088\\
14	0.91370532900957\\
15	0.927324402374208\\
16	0.938839653598037\\
17	0.948557602384572\\
18	0.956747762020373\\
19	0.963643745707513\\
20	0.969446121760551\\
21	0.974325967498606\\
22	0.978428547830676\\
23	0.981876821256875\\
24	0.98477463450324\\
25	0.9872095557712\\
26	0.989255344520896\\
27	0.990974080001405\\
28	0.992417981563491\\
29	0.99363095713072\\
30	0.994649915560237\\
31	0.995505876012858\\
32	0.996224904016156\\
33	0.996828900272785\\
34	0.997336264768667\\
35	0.997762455529397\\
36	0.998120458518924\\
37	0.998421182680434\\
38	0.998673791966287\\
39	0.998885984360515\\
40	0.999064226328133\\
41	0.999213949794812\\
42	0.99933971763515\\
43	0.999445362698031\\
44	0.999534104597049\\
45	0.999608647819943\\
46	0.999671264143805\\
47	0.999723861865828\\
48	0.999768043958315\\
49	0.999805156919596\\
50	0.999836331809227\\
};
\addlegendentry{\huge $p$=0.4}

\addplot [color=mycolor7, line width=1.5pt, mark=triangle, mark repeat=5]
  table[row sep=crcr]{%
1	0.79\\
2	0.6619\\
3	0.589429\\
4	0.55435039\\
5	0.5440378549\\
6	0.549779747959\\
7	0.56559328064269\\
8	0.587395482384848\\
9	0.612423806870212\\
10	0.638831406781893\\
11	0.665404599942523\\
12	0.691365799787396\\
13	0.71623620749432\\
14	0.739740279601284\\
15	0.761739385984186\\
16	0.782185853328521\\
17	0.801091234986993\\
18	0.81850449975879\\
19	0.834497127924938\\
20	0.8491530096128\\
21	0.862561674988424\\
22	0.874813829608008\\
23	0.885998478701267\\
24	0.896201141248739\\
25	0.905502806477762\\
26	0.913979391453751\\
27	0.921701532514204\\
28	0.92873459499183\\
29	0.935138821725297\\
30	0.940969565967933\\
31	0.946277571769358\\
32	0.951109277027093\\
33	0.955507122796539\\
34	0.959509858236169\\
35	0.963152834538837\\
36	0.966468283911088\\
37	0.969485581495613\\
38	0.972231489356573\\
39	0.974730382451378\\
40	0.977004457026581\\
41	0.979073922191268\\
42	0.980957175602009\\
43	0.982670964283397\\
44	0.984230531637789\\
45	0.985649751688317\\
46	0.986941251564918\\
47	0.988116523194061\\
48	0.989186025095585\\
49	0.990159275129275\\
50	0.991044934972245\\
};
\addlegendentry{\huge $p$=0.3}

\addplot [color=mycolor1, line width=1.5pt, mark=square, mark repeat=5]
  table[row sep=crcr]{%
1	0.84\\
2	0.7184\\
3	0.627264\\
4	0.56025344\\
5	0.5123073024\\
6	0.479386210304\\
7	0.45826772189184\\
8	0.446382581016167\\
9	0.44168373217552\\
10	0.442541546408499\\
11	0.447660015368159\\
12	0.456009719406233\\
13	0.466774214352224\\
14	0.479307152755927\\
15	0.493097992227923\\
16	0.507744573004593\\
17	0.522931190457039\\
18	0.538411063136861\\
19	0.55399231684987\\
20	0.569526781166661\\
21	0.584901035512624\\
22	0.600029254566222\\
23	0.614847492762856\\
24	0.629309119755768\\
25	0.643381176328278\\
26	0.657041466365339\\
27	0.67027623738288\\
28	0.683078331625288\\
29	0.695445713350455\\
30	0.70738029680858\\
31	0.718887014529951\\
32	0.729973077623725\\
33	0.740647389458753\\
34	0.750920081832384\\
35	0.760802148920674\\
36	0.770305159253116\\
37	0.779441029914406\\
38	0.788221850342961\\
39	0.796659745629348\\
40	0.804766771244259\\
41	0.812554832746556\\
42	0.820035625318347\\
43	0.827220589010937\\
44	0.834120876414758\\
45	0.840747330129574\\
46	0.847110467941517\\
47	0.853220474037556\\
48	0.859087194927014\\
49	0.864720139010702\\
50	0.870128478954889\\
};
\addlegendentry{\huge $p$=0.2}

\addplot [color=mycolor2, line width=1.5pt,  mark=asterisk, mark repeat=5]
  table[row sep=crcr]{%
1	0.91\\
2	0.8299\\
3	0.758701\\
4	0.69550399\\
5	0.6394999501\\
6	0.589960850599\\
7	0.54623155209301\\
8	0.50772251557208\\
9	0.473903241516359\\
10	0.444296365091196\\
11	0.418472341831284\\
12	0.396044664764871\\
13	0.376665559833932\\
14	0.360022111780632\\
15	0.345832777453361\\
16	0.333844247790309\\
17	0.323828623612739\\
18	0.315580873846912\\
19	0.308916547931713\\
20	0.303669716993339\\
21	0.299691120910254\\
22	0.296846500679315\\
23	0.29501509755287\\
24	0.294088302269654\\
25	0.293968439370039\\
26	0.294567673087112\\
27	0.295807022655937\\
28	0.297615476099104\\
29	0.299929192640866\\
30	0.302690784886936\\
31	0.305848672793297\\
32	0.309356502245072\\
33	0.313172621784358\\
34	0.317259611672077\\
35	0.321583860050363\\
36	0.326115181495366\\
37	0.330826473721367\\
38	0.335693408621014\\
39	0.340694154207978\\
40	0.345809124371754\\
41	0.351020753663308\\
42	0.356313294608418\\
43	0.361672635295904\\
44	0.367086135213157\\
45	0.372542477504217\\
46	0.378031536008047\\
47	0.383544255598951\\
48	0.389072544498848\\
49	0.394609177364157\\
50	0.400147708069784\\
};
\addlegendentry{\huge $p$=0.1}

\end{axis}

\begin{axis}[%
width=5.833in,
height=4.375in,
at={(0in,0in)},
scale only axis,
xmin=0,
xmax=1,
ymin=0,
ymax=1,
axis line style={draw=none},
ticks=none,
axis x line*=bottom,
axis y line*=left
]
\end{axis}
\end{tikzpicture}%
% This file was created by matlab2tikz.
%
%The latest updates can be retrieved from
%  http://www.mathworks.com/matlabcentral/fileexchange/22022-matlab2tikz-matlab2tikz
%where you can also make suggestions and rate matlab2tikz.
%
\definecolor{mycolor1}{rgb}{0.00000,0.44700,0.74100}%
\definecolor{mycolor2}{rgb}{0.85000,0.32500,0.09800}%
\definecolor{mycolor3}{rgb}{0.92900,0.69400,0.12500}%
\definecolor{mycolor4}{rgb}{0.49400,0.18400,0.55600}%
\definecolor{mycolor5}{rgb}{0.46600,0.67400,0.18800}%
\definecolor{mycolor6}{rgb}{0.30100,0.74500,0.93300}%
\definecolor{mycolor7}{rgb}{0.63500,0.07800,0.18400}%
\tikzset{every mark/.append style={scale=2.5}}

%
\begin{tikzpicture}

\begin{axis}[%
width=4.817in,
height=3.592in,
at={(0.802in,0.7in)},
scale only axis,
font=\huge,
xmin=0,
xmax=50,
xlabel style={font=\color{white!15!black}},
xlabel={\huge $m$},
ymin=0,
ymax=1.05,
ylabel style={font=\color{white!15!black}},
ylabel={\huge $P(\alpha_1>0)$},
axis background/.style={fill=white},
legend style={at={(0.695,0.3)}, anchor=south west, legend cell align=left, align=left, draw=white!15!black}
]
\addplot [color=mycolor1, line width=1.5pt, mark=o, mark repeat=5]
  table[row sep=crcr]{%
1	0.91\\
2	0.9739\\
3	0.974458\\
4	0.98856163\\
5	0.9915644206\\
6	0.99550238779\\
7	0.9970025522338\\
8	0.998269593458244\\
9	0.998892690842791\\
10	0.999335756880245\\
11	0.999581737211259\\
12	0.999744158883816\\
13	0.99983981615486\\
14	0.999900982664557\\
15	0.999938083239264\\
16	0.999961490769981\\
17	0.999975906720906\\
18	0.999984956637076\\
19	0.999990576812469\\
20	0.999994100449988\\
21	0.999996299439098\\
22	0.999997678525945\\
23	0.99999854187481\\
24	0.999999083812895\\
25	0.999999423827856\\
26	0.999999637494925\\
27	0.999999771773457\\
28	0.999999856247708\\
29	0.999999909406013\\
30	0.999999942882355\\
31	0.999999963971928\\
32	0.999999977265601\\
33	0.99999998564842\\
34	0.99999999093699\\
35	0.99999999427469\\
36	0.999999996382007\\
37	0.999999997712957\\
38	0.999999998553861\\
39	0.999999999085318\\
40	0.999999999421307\\
41	0.999999999633782\\
42	0.999999999768187\\
43	0.999999999853229\\
44	0.999999999907052\\
45	0.999999999941124\\
46	0.999999999962699\\
47	0.999999999976363\\
48	0.999999999985019\\
49	0.999999999990503\\
50	0.999999999993978\\
};
\addlegendentry{\huge $p$=0.9}

\addplot [color=mycolor2, line width=1.5pt, mark=star, mark repeat=5]
  table[row sep=crcr]{%
1	0.84\\
2	0.9104\\
3	0.912192\\
4	0.93499648\\
5	0.9452524544\\
6	0.95665301504\\
7	0.9644952461312\\
8	0.971319266410496\\
9	0.976594792623636\\
10	0.980944456576915\\
11	0.984426643267815\\
12	0.987267415097981\\
13	0.989569912525123\\
14	0.991448023357572\\
15	0.992979019224039\\
16	0.994230491405653\\
17	0.995254256273792\\
18	0.996093084280146\\
19	0.996781060156253\\
20	0.99734596054548\\
21	0.997810231829414\\
22	0.99819215017237\\
23	0.998506578719708\\
24	0.998765643294208\\
25	0.998979242903094\\
26	0.999155473025258\\
27	0.999300960830771\\
28	0.999421138282827\\
29	0.999520462051358\\
30	0.999602592268767\\
31	0.99967053752426\\
32	0.999726772917759\\
33	0.999773336219615\\
34	0.999811906353914\\
35	0.999843867518081\\
36	0.999870361639456\\
37	0.999892331334333\\
38	0.999910555127107\\
39	0.999925676351069\\
40	0.999938226885916\\
41	0.999948646670596\\
42	0.99995729975595\\
43	0.999964487520378\\
44	0.999970459557395\\
45	0.999975422650963\\
46	0.999979548178945\\
47	0.999982978223462\\
48	0.999985830616712\\
49	0.999988203109844\\
50	0.999990176818955\\
};
\addlegendentry{\huge $p$=0.8}

\addplot [color=mycolor3, line width=1.5pt, mark=diamond, mark repeat=5]
  table[row sep=crcr]{%
1	0.79\\
2	0.8299\\
3	0.829522\\
4	0.84450403\\
5	0.8553553894\\
6	0.86681576539\\
7	0.8770900631482\\
8	0.886668285828803\\
9	0.895422133080045\\
10	0.903457694717971\\
11	0.910817682939882\\
12	0.917566162462277\\
13	0.923756455579423\\
14	0.929439790135183\\
15	0.934661998655891\\
16	0.9394647316637\\
17	0.943885451104247\\
18	0.947957920623694\\
19	0.951712546527605\\
20	0.955176745212646\\
21	0.958375262979705\\
22	0.961330467795546\\
23	0.964062606869954\\
24	0.966590034938042\\
25	0.968929415456271\\
26	0.971095898035658\\
27	0.973103274909768\\
28	0.974964119088377\\
29	0.976689906515197\\
30	0.978291124274402\\
31	0.979777366621038\\
32	0.981157420374483\\
33	0.982439341005092\\
34	0.983630520563294\\
35	0.984737748444216\\
36	0.985767265846886\\
37	0.986724814672144\\
38	0.987615681505029\\
39	0.988444737243087\\
40	0.989216472859853\\
41	0.989935031730668\\
42	0.990604238894731\\
43	0.991227627581277\\
44	0.991808463288168\\
45	0.992349765666831\\
46	0.992854328437754\\
47	0.993324737534898\\
48	0.993763387654866\\
49	0.994172497367038\\
50	0.994554122923672\\
};
\addlegendentry{\huge $p$=0.7}

\addplot [color=mycolor4, line width=1.5pt, mark=Mercedes star, mark repeat=5]
  table[row sep=crcr]{%
1	0.76\\
2	0.7504\\
3	0.739648\\
4	0.73838848\\
5	0.7396320256\\
6	0.74247399424\\
7	0.7461241397248\\
8	0.750229926608896\\
9	0.754579107716792\\
10	0.75904696435185\\
11	0.763555808078133\\
12	0.768056362289469\\
13	0.772516852283059\\
14	0.77691670170162\\
15	0.781242672210996\\
16	0.785486435028077\\
17	0.789642998143028\\
18	0.793709664319582\\
19	0.797685326931751\\
20	0.801569986104199\\
21	0.805364411503621\\
22	0.809069904545974\\
23	0.812688129082719\\
24	0.816220989923885\\
25	0.819670545193436\\
26	0.823038942872836\\
27	0.826328374801666\\
28	0.829541043380226\\
29	0.832679137578174\\
30	0.83574481580009\\
31	0.838740193826018\\
32	0.8416673365203\\
33	0.844528252343674\\
34	0.847324889951599\\
35	0.850059136343105\\
36	0.852732816158152\\
37	0.855347691820574\\
38	0.857905464297794\\
39	0.860407774303974\\
40	0.862856203815258\\
41	0.865252277797481\\
42	0.867597466070921\\
43	0.869893185255085\\
44	0.872140800750659\\
45	0.874341628726586\\
46	0.876496938088523\\
47	0.878607952411353\\
48	0.880675851823351\\
49	0.882701774833338\\
50	0.884686820095109\\
};
\addlegendentry{\huge $p$=0.6}

\addplot [color=mycolor5, line width=1.5pt, mark=diamond, mark repeat=5]
  table[row sep=crcr]{%
1	0.75\\
2	0.6875\\
3	0.65625\\
4	0.63671875\\
5	0.623046875\\
6	0.61279296875\\
7	0.604736328125\\
8	0.598190307617188\\
9	0.592735290527344\\
10	0.588098526000977\\
11	0.584094047546387\\
12	0.580590128898621\\
13	0.577490508556366\\
14	0.574722990393639\\
15	0.572232224047184\\
16	0.569974967045709\\
17	0.567916879779659\\
18	0.56603029978578\\
19	0.564292660317733\\
20	0.56268534380979\\
21	0.561192835623842\\
22	0.559802089359664\\
23	0.558502043938802\\
24	0.557283251356743\\
25	0.556137586329609\\
26	0.555058017361731\\
27	0.554038424447625\\
28	0.553073452582489\\
29	0.552158393055205\\
30	0.551289086504285\\
31	0.55046184317357\\
32	0.549673376873983\\
33	0.54892074995165\\
34	0.548201327158244\\
35	0.547512736770269\\
36	0.54685283764846\\
37	0.546219691193751\\
38	0.545611537362254\\
39	0.545026774062738\\
40	0.544463939386954\\
41	0.543921696223698\\
42	0.543398818887702\\
43	0.542894181458775\\
44	0.542406747578561\\
45	0.541935561494355\\
46	0.541479740173764\\
47	0.541038466342129\\
48	0.540610982317732\\
49	0.540196584538979\\
50	0.539794618693589\\
};
\addlegendentry{\huge $p$=0.5}

\addplot [color=mycolor6, line width=1.5pt, mark=otimes, mark repeat=5]
  table[row sep=crcr]{%
1	0.76\\
2	0.6544\\
3	0.595648\\
4	0.55652608\\
5	0.5272800256\\
6	0.50380908544\\
7	0.4841065013248\\
8	0.467058900238336\\
9	0.451986717449912\\
10	0.438444734074575\\
11	0.426125294357116\\
12	0.414806833326737\\
13	0.404324432210518\\
14	0.394551972273036\\
15	0.385390802572455\\
16	0.376762262807457\\
17	0.368602588456773\\
18	0.360859342516668\\
19	0.353488856318483\\
20	0.346454355512456\\
21	0.339724562362162\\
22	0.333272636124473\\
23	0.327075357898038\\
24	0.321112495215201\\
25	0.315366300791013\\
26	0.30982111278042\\
27	0.304463032801876\\
28	0.299279664221268\\
29	0.29425989762239\\
30	0.28939373358525\\
31	0.284672135226434\\
32	0.280086904679677\\
33	0.275630578982804\\
34	0.271296341809523\\
35	0.267077948225544\\
36	0.262969660218328\\
37	0.258966191191686\\
38	0.255062657961793\\
39	0.25125453906315\\
40	0.247537638388663\\
41	0.24390805336011\\
42	0.240362146963524\\
43	0.236896523095649\\
44	0.233508004758417\\
45	0.23019361471245\\
46	0.226950558261437\\
47	0.223776207889434\\
48	0.22066808951467\\
49	0.217623870158091\\
50	0.214641346853767\\
};
\addlegendentry{\huge $p$=0.4}

\addplot [color=mycolor7, line width=1.5pt, mark=triangle, mark repeat=5]
  table[row sep=crcr]{%
1	0.79\\
2	0.6619\\
3	0.577522\\
4	0.51767683\\
5	0.4723993894\\
6	0.43631000299\\
7	0.4063726019482\\
8	0.380788636298723\\
9	0.358437793300605\\
10	0.338587005969696\\
11	0.320734055090743\\
12	0.304520449217723\\
13	0.289681014451974\\
14	0.276013568744552\\
15	0.263359966556908\\
16	0.251593811566373\\
17	0.24061221978889\\
18	0.230330128134681\\
19	0.220676254512487\\
20	0.211590161404802\\
21	0.203020076531283\\
22	0.194921245421612\\
23	0.187254665676286\\
24	0.179986100322119\\
25	0.173085298701439\\
26	0.16652537402315\\
27	0.16028230078865\\
28	0.154334505077763\\
29	0.148662527577236\\
30	0.143248744179148\\
31	0.138077132572251\\
32	0.133133075897991\\
33	0.128403196517573\\
34	0.12387521442471\\
35	0.119537825971892\\
36	0.11538059944903\\
37	0.111393884728724\\
38	0.107568734720446\\
39	0.103896836791967\\
40	0.100370452646475\\
41	0.0969823654076687\\
42	0.0937258328771751\\
43	0.0905945461003237\\
44	0.0875825925159015\\
45	0.0846844230797981\\
46	0.0818948228463952\\
47	0.0792088845692064\\
48	0.076621984946753\\
49	0.0741297631934295\\
50	0.0717281016601578\\
};
\addlegendentry{\huge $p$=0.3}

\addplot [color=mycolor1, line width=1.5pt, mark=square, mark repeat=5]
  table[row sep=crcr]{%
1	0.84\\
2	0.7184\\
3	0.624192\\
4	0.54976768\\
5	0.4898284544\\
6	0.44065286144\\
7	0.3996023693312\\
8	0.364786684952576\\
9	0.334836087533077\\
10	0.308745809682749\\
11	0.285769093659265\\
12	0.265343262604492\\
13	0.247038282122928\\
14	0.230520715695894\\
15	0.215528271652412\\
16	0.201851679114886\\
17	0.189321666822267\\
18	0.177799518797821\\
19	0.167170155386822\\
20	0.157337011154553\\
21	0.148218201876027\\
22	0.139743624423944\\
23	0.131852737966492\\
24	0.124492847468055\\
25	0.11761776114038\\
26	0.11118672906333\\
27	0.105163595339066\\
28	0.0995161140420052\\
29	0.0942153920607953\\
30	0.0892354312015211\\
31	0.0845527486744453\\
32	0.0801460600447981\\
33	0.0759960123987395\\
34	0.0720849582159431\\
35	0.0683967625033748\\
36	0.0649166373112255\\
37	0.0616309989511486\\
38	0.0585273441625566\\
39	0.0555941421929088\\
40	0.0528207403226181\\
41	0.0501972808113262\\
42	0.0477146275972913\\
43	0.0453643013660747\\
44	0.0431384218341299\\
45	0.0410296562791148\\
46	0.0390311735007936\\
47	0.037136602521265\\
48	0.0353399954363454\\
49	0.0336357939155047\\
50	0.0320187989191061\\
};
\addlegendentry{\huge $p$=0.2}

\addplot [color=mycolor2, line width=1.5pt,  mark=asterisk, mark repeat=5]
  table[row sep=crcr]{%
1	0.91\\
2	0.8299\\
3	0.758458\\
4	0.69460003\\
5	0.6373964206\\
6	0.58604151259\\
7	0.5398362986338\\
8	0.498173565756804\\
9	0.460525171577671\\
10	0.426431141800139\\
11	0.395490322865646\\
12	0.367352363098272\\
13	0.341710829201177\\
14	0.318297293478915\\
15	0.296876251418208\\
16	0.277240749901824\\
17	0.259208623899942\\
18	0.242619254442168\\
19	0.227330773412911\\
20	0.213217651565541\\
21	0.200168615398985\\
22	0.188084846423805\\
23	0.176878423067047\\
24	0.166470971199214\\
25	0.15679249415953\\
26	0.147780357332239\\
27	0.13937840589318\\
28	0.131536197392604\\
29	0.124208333443958\\
30	0.117353877014492\\
31	0.110935843717636\\
32	0.104920757136497\\
33	0.0992782596027305\\
34	0.0939807710499064\\
35	0.0890031895843762\\
36	0.0843226282945755\\
37	0.0799181835728223\\
38	0.0757707308701219\\
39	0.0718627443596726\\
40	0.0681781374618723\\
41	0.0647021215938722\\
42	0.0614210808597093\\
43	0.0583224607009705\\
44	0.0553946687897894\\
45	0.0526269866717415\\
46	0.050009490860997\\
47	0.0475329822582837\\
48	0.0451889229075618\\
49	0.042969379233009\\
50	0.0408669710067123\\
};
\addlegendentry{\huge $p$=0.1}

\end{axis}

\begin{axis}[%
%width=5.833in,
%height=4.375in,
width=2in,
height=1in,
at={(0in,0in)},
scale only axis,
xmin=0,
xmax=1,
ymin=0,
ymax=1,
axis line style={draw=none},
ticks=none,
axis x line*=bottom,
axis y line*=left
]
\end{axis}
\end{tikzpicture}%
% This file was created by matlab2tikz.
%
%The latest updates can be retrieved from
%  http://www.mathworks.com/matlabcentral/fileexchange/22022-matlab2tikz-matlab2tikz
%where you can also make suggestions and rate matlab2tikz.
%
\definecolor{mycolor1}{rgb}{0.46600,0.67400,0.18800}%
\definecolor{mycolor2}{rgb}{0.49400,0.18400,0.55600}%
\definecolor{mycolor3}{rgb}{0.30100,0.74500,0.93300}%
\definecolor{mycolor4}{rgb}{0.93000,0.69000,0.13000}%
%
\begin{tikzpicture}

\begin{axis}[%
width=8in,
height=3.6in,
at={(0.733in,0.717in)},
scale only axis,
unbounded coords=jump,
font=\huge,
scaled y ticks=false,
    ticklabel style={
        /pgf/number format/fixed,
        /pgf/number format/precision=2
    },
xmin=-0.00725388395662747,
xmax=0.209844561639227,
xtick={   0, 0.04, 0.08, 0.12, 0.16,  0.2},
xlabel style={font=\color{white!15!black}},
xlabel={\huge Noise level ($p$)},
ymin=0.301647047123488,
ymax=1.3647058706529,
ylabel style={font=\color{white!15!black}},
ylabel={\huge Rel.\ err.\ ground truth},
axis background/.style={fill=white},
legend style={at={(0.02,0.58)}, anchor=south west, legend cell align=left, align=left, draw=white!15!black}
]
\addplot [color=red, line width=3.5pt, mark size=7pt, mark=o, mark options={solid, red,,line width=2.5pt}]
  table[row sep=crcr]{%
0	0.463938988879626\\
0.04	0.479124905191728\\
0.08	0.504456862974566\\
0.12	0.541133928421559\\
0.16	0.596806292046503\\
0.2	0.692836597284415\\
};
\addlegendentry{L1-NMF}

\addplot [color=mycolor1, line width=3.5pt, mark size=8pt, mark=diamond, mark options={solid, mycolor1,line width=2.5pt}]
  table[row sep=crcr]{%
0	0.377894274083328\\
0.04	0.442572833249210\\
0.08	0.570810121703690\\
0.12	0.699246833770925\\
0.16	0.815292021664598\\
0.2	0.923262049562748\\
};
\addlegendentry{L2-NMF}

\addplot [color=mycolor2, dashed, line width=5pt, mark size=8.5pt, mark=triangle, mark options={solid, mycolor2,line width=2.5pt}]
  table[row sep=crcr]{%
0	0.392533845650144\\
0.04	0.474994630721814\\
0.08	0.589967425719929\\
0.12	0.711260715970360\\
0.16	0.825521971753015\\
0.2	0.929886181700145\\
};
\addlegendentry{L21-NMF}

\addplot [color=mycolor3, dashdotted, line width=4.5pt, mark size=9.0pt, mark=x, mark options={solid, mycolor3,line width=2.5pt}]
  table[row sep=crcr]{%
0	0.424382181850498\\
0.04	0.561054740555926\\
0.08	0.667690659045612\\
0.12	0.772247074693669\\
0.16	0.875105179094526\\
0.2	0.975588202360953\\
};
\addlegendentry{KL-NMF}

\addplot [color=mycolor4, dotted, line width=4.5pt, mark size=6.5pt, mark=square, mark options={solid, mycolor4,line width=2.5pt}]
  table[row sep=crcr]{%
0	nan\\
0.04	0.583370689260315\\
0.08	0.824307759938737\\
0.12	1.00964329400092\\
0.16	1.16548022699053\\
0.2	1.30310127023990\\
};
\addlegendentry{Noisy data}

\end{axis}

\begin{axis}[%
width=5.833in,
height=4.375in,
at={(0in,0in)},
scale only axis,
xmin=0,
xmax=1,
ymin=0,
ymax=1,
axis line style={draw=none},
ticks=none,
axis x line*=bottom,
axis y line*=left
]
\end{axis}
\end{tikzpicture}%
\begin{tikzpicture}

% Stile comune
\pgfplotsset{
    myplot/.style={
        xmin=-0.3, xmax=6.3,
        xtick={0,1,2,3,4,5,6},
        xticklabels={0,0.01,0.03,0.05,0.07,0.1,1},
        mark options={solid, fill=none, line width=1.2pt},
        grid=both
    }
}

\begin{groupplot}[
    group style={
        group size=2 by 2,
        horizontal sep=2cm,
        vertical sep=2cm
    },
    width=7cm,
    height=6.5cm,
    myplot,
    xlabel={$\lambda$},
    ylabel={Relative error},
    scaled y ticks=false,
    ticklabel style={
        /pgf/number format/fixed,
        /pgf/number format/precision=2
    },
    %font=\fontsize{12pt}{14pt}\selectfont
]

% ---- PLOT 1 ----
\nextgroupplot[
    title={$q_1=40\%$ of zeros},
    ymin=0.05, ymax=0.2,
    legend to name=legendaComune,
    legend style={
        font=\fontsize{12pt}{14pt}\selectfont,
        mark size=3pt,
        line width=1.2pt,
        draw=black,
        fill=white,
        legend columns=-1,
        column sep=1em,
        mark options={solid, fill=none, line width=2pt},
    }
]

\addplot[blue, mark=o, mark size=4pt, line width=2pt] table[x=x,y=yA] {Tikz/Rec_system1_0_4.dat};
\addplot[red, mark=square, dotted, mark size=4pt, line width=2.5pt] table[x=x,y=yB] {Tikz/Rec_system1_0_4.dat};
\addplot[green!60!black, mark=triangle, dashdotted, mark size=5pt, line width=2.5pt] table[x=x,y=yC] {Tikz/Rec_system1_0_4.dat};
\addplot[orange, mark=diamond, dashed, mark size=5pt, line width=2.5pt] table[x=x,y=yD] {Tikz/Rec_system1_0_4.dat};
\addplot[purple!80!black, mark=x, mark size=5.5pt, line width=2pt] table[x=x,y=yE] {Tikz/Rec_system1_0_4.dat};

\addlegendentry{$q_2=100\%$}
\addlegendentry{$q_2=75\%$}
\addlegendentry{$q_2=50\%$}
\addlegendentry{$q_2=25\%$}
\addlegendentry{$q_2=0\%$}

% ---- PLOT 2 ----
\nextgroupplot[
    title={$q_1=50\%$ of zeros},
    ymin=0.05, ymax=0.3
]

\addplot[blue, mark=o, mark size=4pt, line width=2pt] table[x=x,y=yA] {Tikz/Rec_system1_0_5.dat};
\addplot[red, mark=square, dotted, mark size=4pt, line width=2.5pt] table[x=x,y=yB] {Tikz/Rec_system1_0_5.dat};
\addplot[green!60!black, mark=triangle, dashdotted, mark size=5pt, line width=2.5pt] table[x=x,y=yC] {Tikz/Rec_system1_0_5.dat};
\addplot[orange, mark=diamond, dashed, mark size=5pt, line width=2.5pt] table[x=x,y=yD] {Tikz/Rec_system1_0_5.dat};
\addplot[purple!80!black, mark=x, mark size=5.5pt, line width=2pt] table[x=x,y=yE] {Tikz/Rec_system1_0_5.dat};

% ---- PLOT 3 ----
\nextgroupplot[
    title={$q_1=60\%$ of zeros},
    ymin=0.05, ymax=0.4
]

\addplot[blue, mark=o, mark size=4pt, line width=2pt] table[x=x,y=yA] {Tikz/Rec_system1_0_6.dat};
\addplot[red, mark=square, dotted, mark size=4pt, line width=2.5pt] table[x=x,y=yB] {Tikz/Rec_system1_0_6.dat};
\addplot[green!60!black, mark=triangle, dashdotted, mark size=5pt, line width=2.5pt] table[x=x,y=yC] {Tikz/Rec_system1_0_6.dat};
\addplot[orange, mark=diamond, dashed, mark size=5pt, line width=2.5pt] table[x=x,y=yD] {Tikz/Rec_system1_0_6.dat};
\addplot[purple!80!black, mark=x, mark size=5.5pt, line width=2pt] table[x=x,y=yE] {Tikz/Rec_system1_0_6.dat};

% ---- PLOT 4 ----
\nextgroupplot[
    title={$q_1=70\%$ of zeros},
    ymin=0.1, ymax=0.6
]

\addplot[blue, mark=o, mark size=4pt, line width=2pt] table[x=x,y=yA] {Tikz/Rec_System1_0_7.dat};
\addplot[red, mark=square, dotted, mark size=4pt, line width=2.5pt] table[x=x,y=yB] {Tikz/Rec_System1_0_7.dat};
\addplot[green!60!black, mark=triangle, dashdotted, mark size=5pt, line width=2.5pt] table[x=x,y=yC] {Tikz/Rec_System1_0_7.dat};
\addplot[orange, mark=diamond, dashed, mark size=5pt, line width=2.5pt] table[x=x,y=yD] {Tikz/Rec_System1_0_7.dat};
\addplot[purple!80!black, mark=x, mark size=5.5pt, line width=2pt] table[x=x,y=yE] {Tikz/Rec_System1_0_7.dat};

\end{groupplot}

% ---- LEGENDA SOTTO ----
\node at ($(group c1r2.south)!0.5!(group c2r2.south) + (0,-1.5cm)$)
{\pgfplotslegendfromname{legendaComune}};

\end{tikzpicture}
\begin{thebibliography}{10}
\providecommand{\url}[1]{{#1}}
\providecommand{\urlprefix}{URL }
\expandafter\ifx\csname urlstyle\endcsname\relax
  \providecommand{\doi}[1]{DOI~\discretionary{}{}{}#1}\else
  \providecommand{\doi}{DOI~\discretionary{}{}{}\begingroup
  \urlstyle{rm}\Url}\fi

\bibitem{bayar2014probabilistic}
Bayar, B., Bouaynaya, N., Shterenberg, R.: Probabilistic non-negative matrix
  factorization: theory and application to microarray data analysis.
\newblock Journal of Bioinformatics and Computational Biology \textbf{12}(01),
  1450,001 (2014)

\bibitem{bioucas2012hyperspectral}
Bioucas-Dias, J.M., Plaza, A., Dobigeon, N., Parente, M., Du, Q., Gader, P.,
  Chanussot, J.: {H}yperspectral unmixing overview: geometrical, statistical,
  and sparse regression-based approaches.
\newblock IEEE Journal of Selected Topics in Applied Earth Observations and
  Remote Sensing \textbf{5}(2), 354--379 (2012)

\bibitem{blasco2019does}
Blasco-Moreno, A., P{\'e}rez-Casany, M., Puig, P., Morante, M., Castells, E.:
  What does a zero mean? understanding false, random and structural zeros in
  ecology.
\newblock Methods in Ecology and Evolution \textbf{10}(7), 949--959 (2019)

\bibitem{cao2023manhattan}
Cao, C., Tang, S., Zhang, N., Dai, X., Zhang, W., Feng, Y., Xiong, J., Liu, J.,
  Thompson, L.: {M}anhattan nonnegative matrix factorization using the
  alternating direction method of multipliers.
\newblock In: 15th International Conference on Advanced Computational
  Intelligence (ICACI), pp. 1--5 (2023)

\bibitem{chen2008analysis}
Chen, K., Ying, Z., Zhang, H., Zhao, L.: Analysis of least absolute deviation.
\newblock Biometrika \textbf{95}(1), 107--122 (2008)

\bibitem{cichocki2007hierarchical}
Cichocki, A., Zdunek, R., Amari, S.i.: {H}ierarchical {ALS} algorithms for
  nonnegative matrix and {3D} tensor factorization.
\newblock In: International Conference on Independent Component Analysis and
  Signal Separation, pp. 169--176. Springer (2007)

\bibitem{CZA09}
Cichocki, A., Zdunek, R., Phan, A., Amari, S.I.: Nonnegative Matrix and Tensor
  Factorizations: Applications to Exploratory Multi-Way Data Analysis and Blind
  Source Separation.
\newblock John Wiley \& Sons (2009)

\bibitem{croux1998robust}
Croux, C., Filzmoser, P.: Robust factorization of a data matrix.
\newblock In: COMPSTAT: Proceedings in Computational Statistics 13th Symposium
  held in Bristol, Great Britain, pp. 245--250. Springer (1998)

\bibitem{ding2008equivalence}
Ding, C., Li, T., Peng, W.: On the equivalence between non-negative matrix
  factorization and probabilistic latent semantic indexing.
\newblock Computational Statistics \& Data Analysis \textbf{52}(8), 3913--3927
  (2008)

\bibitem{du2012robust}
Du, L., Li, X., Shen, Y.D.: Robust nonnegative matrix factorization via
  half-quadratic minimization.
\newblock In: IEEE 12th International Conference on Data Mining, pp. 201--210
  (2012)

\bibitem{edgeworth1888xxii}
Edgeworth, F.Y.: On a new method of reducing observations relating to several
  quantities.
\newblock The London, Edinburgh, and Dublin Philosophical Magazine and Journal
  of Science \textbf{25}(154), 184--191 (1888)

\bibitem{eriksson2010efficient}
Eriksson, A., Van Den~Hengel, A.: Efficient computation of robust low-rank
  matrix approximations in the presence of missing data using the l1 norm.
\newblock In: IEEE Computer Society Conference on Computer Vision and Pattern
  Recognition, pp. 771--778 (2010)

\bibitem{gillis2020nonnegative}
Gillis, N.: Nonnegative Matrix Factorization.
\newblock SIAM, Philadelphia (2020)

\bibitem{gillis2012accelerated}
Gillis, N., Glineur, F.: Accelerated multiplicative updates and hierarchical
  {ALS} algorithms for nonnegative matrix factorization.
\newblock Neural Computation \textbf{24}(4), 1085--1105 (2012)

\bibitem{gillis2011dimensionality}
Gillis, N., Plemmons, R.J.: Dimensionality reduction, classification, and
  spectral mixture analysis using non-negative underapproximation.
\newblock Optical Engineering \textbf{50}(2), 027,001--027,001 (2011)

\bibitem{gillis2018complexity}
Gillis, N., Vavasis, S.A.: On the complexity of robust {PCA} and $\ell_1$-norm
  low-rank matrix approximation.
\newblock Mathematics of Operations Research \textbf{43}(4), 1072--1084 (2018)

\bibitem{gonzalez2023nectar}
Gonz{\'a}lez-Fern{\'a}ndez, A., Dexter, A., Nikula, C.J., Bunch, J.: {NECTAR}:
  a new algorithm for characterizing and correcting noise in {QT}o{F}-mass
  spectrometry imaging data.
\newblock Journal of the American Society for Mass Spectrometry
  \textbf{34}(11), 2443--2453 (2023)

\bibitem{guan2012mahnmf}
Guan, N., Tao, D., Luo, Z., Shawe-Taylor, J.: {MahNMF: M}anhattan non-negative
  matrix factorization.
\newblock arXiv preprint arXiv:1207.3438  (2012)

\bibitem{guillamet2002non}
Guillamet, D., Vitria, J.: Non-negative matrix factorization for face
  recognition.
\newblock In: Catalonian Conference on Artificial Intelligence, pp. 336--344.
  Springer (2002)

\bibitem{guo2021modified}
Guo, Z., Min, A., Yang, B., Chen, J., Li, H.: A modified {H}uber nonnegative
  matrix factorization algorithm for hyperspectral unmixing.
\newblock IEEE Journal of Selected Topics in Applied Earth Observations and
  Remote Sensing \textbf{14}, 5559--5571 (2021)

\bibitem{gurwitz1990weighted}
Gurwitz, C.: Weighted median algorithms for l1 approximation.
\newblock BIT Numerical Mathematics \textbf{30}(2), 301--310 (1990)

\bibitem{huang2014robust}
Huang, J., Nie, F., Huang, H., Ding, C.: Robust manifold nonnegative matrix
  factorization.
\newblock ACM Transactions on Knowledge Discovery from Data (TKDD)
  \textbf{8}(3), 1--21 (2014)

\bibitem{ke2003robust}
Ke, Q., Kanade, T.: Robust subspace computation using {L1} norm  (2003).
\newblock Carnegie Mellon University,
  \url{https://kilthub.cmu.edu/articles/journal_contribution/Robust_subspace_computation_using_L1_norm/6609131/files/12101213.pdf}

\bibitem{ke2005robust}
Ke, Q., Kanade, T.: Robust $\ell_1$-norm factorization in the presence of
  outliers and missing data by alternative convex programming.
\newblock In: IEEE Conference on Computer Vision and Pattern Recognition
  (CVPR), pp. 739--746 (2005)

\bibitem{kong2011robust}
Kong, D., Ding, C., Huang, H.: Robust nonnegative matrix factorization using
  $\ell_{2,1}$-norm.
\newblock In: Proceedings of the 20th ACM International Conference on
  Information and Knowledge Management, pp. 673--682 (2011)

\bibitem{koren2009matrix}
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for
  recommender systems.
\newblock Computer \textbf{42}(8), 30--37 (2009)

\bibitem{kwak2008principal}
Kwak, N.: Principal component analysis based on l1-norm maximization.
\newblock IEEE Transactions on Pattern Analysis and Machine Intelligence
  \textbf{30}(9), 1672--1680 (2008)

\bibitem{lee1999learning}
Lee, D.D., Seung, H.S.: Learning the parts of objects by non-negative matrix
  factorization.
\newblock Nature \textbf{401}(6755), 788--791 (1999)

\bibitem{li2009fastnmf}
Li, L., Zhang, Y.J.: {FastNMF}: highly efficient monotonic fixed-point
  nonnegative matrix factorization algorithm with good applicability.
\newblock Journal of Electronic Imaging \textbf{18}(3), 033,004 (2009)

\bibitem{li2017robust}
Li, Z., Tang, J., He, X.: Robust structured nonnegative matrix factorization
  for image representation.
\newblock IEEE Transactions on Neural Networks and Learning Systems
  \textbf{29}(5), 1947--1960 (2017)

\bibitem{loeys2012analysis}
Loeys, T., Moerkerke, B., De~Smet, O., Buysse, A.: The analysis of
  zero-inflated count data: Beyond zero-inflated poisson regression.
\newblock British Journal of Mathematical and Statistical Psychology
  \textbf{65}(1), 163--180 (2012)

\bibitem{lopez2023zero}
L{\'o}pez, O.F., Dunlavy, D.M., Lehoucq, R.B.: Zero-truncated poisson
  regression for sparse multiway count data corrupted by false zeros.
\newblock Information and Inference: A Journal of the IMA \textbf{12}(3),
  1573--1611 (2023)

\bibitem{ma2013signal}
Ma, W.K., Bioucas-Dias, J.M., Chan, T.H., Gillis, N., Gader, P., Plaza, A.J.,
  Ambikapathi, A., Chi, C.Y.: A signal processing perspective on hyperspectral
  unmixing: insights from remote sensing.
\newblock IEEE Signal Processing Magazine \textbf{31}(1), 67--81 (2013)

\bibitem{markopoulos2013some}
Markopoulos, P.P., Karystinos, G.N., Pados, D.A.: Some options for l1-subspace
  signal processing.
\newblock In: ISWCS The Tenth International Symposium on Wireless Communication
  Systems, pp. 1--5. VDE (2013)

\bibitem{markopoulos2014optimal}
Markopoulos, P.P., Karystinos, G.N., Pados, D.A.: Optimal algorithms for l1
  subspace signal processing.
\newblock IEEE Transactions on Signal Processing \textbf{62}(19), 5046--5058
  (2014)

\bibitem{moens2025preserving}
Moens, R.A., Migas, L.G., Van~Ardenne, J.M., Skaar, E.P., Spraggins, J.M.,
  Van~de Plas, R.: Preserving full spectrum information in imaging mass
  spectrometry data reduction.
\newblock Bioinformatics \textbf{41}(5), 247 (2025)

\bibitem{nesterov2005smooth}
Nesterov, Y.: Smooth minimization of non-smooth functions.
\newblock Mathematical Programming \textbf{103}, 127--152 (2005)

\bibitem{portnoy1997gaussian}
Portnoy, S., Koenker, R.: The {G}aussian hare and the {L}aplacian tortoise:
  computability of squared-error versus absolute-error estimators.
\newblock Statistical Science \textbf{12}(4), 279--300 (1997)

\bibitem{rahimi2024projected}
Rahimi, M., Ghaderi, S., Moreau, Y., Ahookhosh, M.: Projected subgradient
  methods for paraconvex optimization: Application to robust low-rank matrix
  recovery.
\newblock arXiv preprint arXiv:2501.00427  (2024)

\bibitem{shahnaz2006document}
Shahnaz, F., Berry, M.W., Pauca, V.P., Plemmons, R.J.: Document clustering
  using nonnegative matrix factorization.
\newblock Information Processing \& Management \textbf{42}(2), 373--386 (2006)

\bibitem{shen2014robust}
Shen, B., Liu, B.D., Wang, Q., Ji, R.: Robust nonnegative matrix factorization
  via $\ell_1$ norm regularization by multiplicative updating rules.
\newblock In: IEEE International Conference on Image Processing (ICIP), pp.
  5282--5286 (2014)

\bibitem{song2017low}
Song, Z., Woodruff, D.P., Zhong, P.: Low rank approximation with entrywise
  l1-norm error.
\newblock In: Proceedings of the 49th Annual ACM SIGACT Symposium on Theory of
  Computing, pp. 688--701 (2017)

\bibitem{tseng2001convergence}
Tseng, P.: Convergence of a block coordinate descent method for
  nondifferentiable minimization.
\newblock Journal of Optimization Theory and Applications \textbf{109},
  475--494 (2001)

\bibitem{vavasis2010complexity}
Vavasis, S.A.: On the complexity of nonnegative matrix factorization.
\newblock SIAM Journal on Optimization \textbf{20}(3), 1364--1377 (2010)

\bibitem{wang2018modeling}
Wang, M., Gong, M., Zheng, X., Zhang, K.: Modeling dynamic missingness of
  implicit feedback for recommendation.
\newblock Advances in neural information processing systems \textbf{31} (2018)

\bibitem{wang2012probabilistic}
Wang, N., Yao, T., Wang, J., Yeung, D.Y.: A probabilistic approach to robust
  matrix factorization.
\newblock In: {E}uropean {C}onference on {C}omputer {V}ision, pp. 126--139.
  Springer (2012)

\bibitem{zhang2011robust}
Zhang, L., Chen, Z., Zheng, M., He, X.: Robust non-negative matrix
  factorization.
\newblock Frontiers of Electrical and Electronic Engineering in China
  \textbf{6}, 192--200 (2011)

\bibitem{zhang2007binary}
Zhang, Z., Li, T., Ding, C., Zhang, X.: Binary matrix factorization with
  applications.
\newblock In: IEEE International Conference on Data Mining (ICDM), pp. 391--400
  (2007)

\bibitem{zheng2012practical}
Zheng, Y., Liu, G., Sugimoto, S., Yan, S., Okutomi, M.: Practical low-rank
  matrix approximation under robust $\ell_1$-norm.
\newblock In: IEEE Conference on Computer Vision and Pattern Recognition
  (CVPR), pp. 1410--1417 (2012)

\end{thebibliography}