Central Server Free Federated Learning over Single-sided Trust Social Networks
Extracted source text · arXiv 1910.04956
\documentclass{article}
% if you need to pass options to natbib, use, e.g.:
% \PassOptionsToPackage{numbers, compress}{natbib}
% before loading neurips_2020
% ready for submission
% \usepackage{neurips_2020}
% to compile a preprint version, e.g., for submission to arXiv, add add the
% [preprint] option:
% \usepackage[preprint]{neurips_2020}
% to compile a camera-ready version, add the [final] option, e.g.:
% \usepackage[final]{neurips_2020}
% to avoid loading the natbib package, add option nonatbib:
\PassOptionsToPackage{numbers}{natbib}
\usepackage[preprint]{neurips_2020}
\usepackage[utf8]{inputenc} % allow utf-8 input
\usepackage[T1]{fontenc} % use 8-bit T1 fonts
\usepackage{hyperref} % hyperlinks
\usepackage{url} % simple URL typesetting
\usepackage{booktabs} % professional-quality tables
\usepackage{amsfonts} % blackboard math symbols
\usepackage{nicefrac} % compact symbols for 1/2, etc.
\usepackage{microtype} % microtypography
\usepackage{amsfonts}
\usepackage{multicol}
\usepackage{amsmath}
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator*{\E}{\mathbb{E}}
% If you use BibTeX in apalike style, activate the following line:
% \bibliographystyle{apalike}
\newcommand{\x}{\mathbf{x}}
\newcommand{\z}{\mathbf{z}}
\newcommand{\e}{\mathbf{e}}
\newcommand{\X}{\mathbf{X}}
\newcommand{\Z}{\mathbf{Z}}
\newcommand{\G}{\mathbf{G}}
\newcommand{\bxi}{\bm{\xi}}
\newcommand{\W}{\mathbf{W}}
\newcommand{\0}{\mathbf{0}}
\newcommand{\pushsum}{Online Push-Sum (OPS)}
\usepackage{mathtools}
\usepackage{bm}
\usepackage{caption}
\usepackage{subcaption}
\usepackage{graphicx}
\usepackage{algorithm,algorithmicx,algpseudocode}
\usepackage{hyperref}
\usepackage{subcaption}
\usepackage{amsthm}
\theoremstyle{plain}% Theorem-like structures provided by amsthm.sty
\newtheorem{theorem}{Theorem}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{assumption}[theorem]{Assumption}
\newtheorem{definition}[theorem]{Definition}
\usepackage{color}
\newcommand{\blue}[1]{{\color{blue}#1}}
\def\red{\color{red}}
\newcommand\numberthis{\addtocounter{equation}{1}\tag{\theequation}}
\title{Central Server Free Federated Learning over\\ Single-sided Trust Social Networks}
% The \author macro works with any number of authors. There are two commands
% used to separate the names and addresses of multiple authors: \And and \AND.
%
% Using \And between authors leaves it to LaTeX to determine where to break the
% lines. Using \AND forces a line break at that point. So, if LaTeX puts 3 of 4
% authors names on the first line, and the last on the second line, try using
% \AND instead of \And before the third author name.
\author{%
Chaoyang He \thanks{Equal Contribution} \\
University of Southern California\\
\texttt{chaoyang.he@usc.edu} \\
\And
Conghui Tan \footnotemark[1] \\
WeBank \\
\texttt{tanconghui@gmail.com} \\
\And
Hanlin Tang \\
University of Rochester \\
\texttt{htang14@ur.rochester.edu} \\
\And
Shuang Qiu \\
University of Michigan \\
\texttt{qiush@umich.edu} \\
\And
Ji Liu\\
Kwai Inc. \\
\texttt{ji.liu.uwisc@gmail.com}
% examples of more authors
% \And
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
% \AND
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
% \And
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
% \And
% Coauthor \\
% Affiliation \\
% Address \\
% \texttt{email} \\
}
\begin{document}
\maketitle
\begin{abstract}
Federated learning has become increasingly important for modern machine learning, especially for data privacy-sensitive scenarios. Existing federated learning primarily adopts the central server-based network topology. However, in many social network scenarios, centralized federated learning is not applicable. For instance, a central agent or server connecting all users may not exist, or the communication cost to the central server is not affordable. In this paper, we consider a generic setting: 1) the central server may not exist, and 2) the social network is unidirectional or of single-sided trust (i.e., user A trusts user B but user B may not trust user A). We propose a central server free federated learning algorithm, named Online Push-Sum (OPS) method, to handle this challenging but generic scenario. A rigorous regret analysis is also provided, which shows interesting results on how users can benefit from communication with trusted users in the federated learning scenario. This work builds upon the fundamental algorithm framework and theoretical guarantees for federated learning in the generic social network scenario.
\end{abstract}
\section{Introduction}
Federated learning has been well recognized as a framework able to protect data privacy \cite{konevcny2016federated,smith2017federated,yang2019federated}. State-of-the-art federated learning adopts the centralized network architecture where a centralized node collects the gradients sent from child agents to update the global model. Despite its simplicity, the centralized method suffers from communication and computational bottlenecks in the central node, especially for federated learning, where a large number of clients are usually involved. Moreover, to prevent reverse engineering of the user's identity, a certain amount of noise must be added to the gradient to protect user privacy, which partially sacrifices the efficiency and the accuracy \cite{shokri2015privacy}.
To further protect the data privacy and avoid the communication bottleneck, the decentralized architecture has been recently proposed \cite{vanhaesebrouck2017decentralized,bellet2018personalized}, where the centralized node has been removed, and each node only communicates with its neighbors (with mutual trust) by exchanging their local models. Exchanging local models is usually favored to the data privacy protection over sending private gradients because the local model is the aggregation or mixture of a large sum of data while the local gradient directly reflects only one or a batch of private data samples. Although advantages of decentralized architecture have been well recognized over the state-of-the-art method (its centralized counterpart), it usually can only be run on the network with \emph{mutual trusts} (``trust'' means ``would like to send information to''). That is, two nodes (or users) can exchange their local models only if they trust each other reciprocally (e.g., node A may trust node B, but if node B does not trust node A, they cannot communicate). Given a social
network, one can only use the edges with mutual trust to run decentralized federated learning algorithms. Two immediate drawbacks are: (1) If all mutual trust edges do not form a connected network, the federated learning does not apply; (2) Removing all single-sided edges from the communication network could significantly reduce the efficiency of communication.
These drawbacks lead to the question: \emph{How do we effectively utilize the single-sided trust edges under a decentralized federated learning framework?}
\begin{figure*}[t]
\centering
\begin{subfigure}{.32\textwidth}
\centering
\includegraphics[width=0.7\textwidth]{centralized.png}
\caption{Centralized}
\end{subfigure}
\begin{subfigure}{.32\textwidth}
\centering
\includegraphics[width=0.7\textwidth]{decentralized_mutual.png}
\caption{Decentralized (mutual trust)}
\end{subfigure}
\begin{subfigure}{.32\textwidth}
\centering
\includegraphics[width=0.7\textwidth]{decentralized_single.png}
\caption{Decentralized (singe-sided trust)}
\end{subfigure}
\caption{Different types of architectures.}
\vspace{-0.3cm}
\end{figure*}
In this paper, we consider the social network scenario, where the centralized network is unavailable (e.g., there does not exist a central node that can build up the connection with all users, or the centralized communication cost is not affordable). We make a minimal assumption on the social network: The data may come in a streaming fashion on each user node as the federated learning algorithm runs; the trust between users may be single-sided, where user A trusts user B, but user B may not trust user A.
For the setting mentioned above, we develop a decentralized learning algorithm called online push-sum (OPS) which possesses the following features:
\begin{itemize}
\item Our algorithm removes some constraints imposed by typical decentralized methods, making it more flexible in allowing arbitrary network topology. Each node only needs to know its out neighbors rather than the global topology.
\item Only models rather than local gradients are exchanged among clients in our algorithm. This scheme can reduce the risk of exposing clients' data privacy \cite{aono2017privacy}.
\item We provide the rigorous regret analysis for the proposed algorithm and specifically distinguish two components in the online loss function: the adversary component and the stochastic component, which can model clients' private data and internal connections between clients, respectively.
\end{itemize}
%We specifically distinguish two components in the online loss function: the adversary component and the stochastic component, which can model clients' private data and internal connections between clients, respectively.
\paragraph{Notation}
We adopt the following notation in this paper:
\begin{itemize}
\item
For random variable $\bxi_t^{(i)}$ subject to distribution $D_t^{(i)}$, we use $\Xi_{n,T}$ and $\mathcal{D}_{n,T}$ to denote the set of random variables and distributions, respectively:
\[
\Xi_{n,T}=\big\{\bxi_t^{(i)}\big\}_{1\leq i\leq n, 1\leq t\leq T},\quad \mathcal{D}_{n,T}=\big\{D_t^{(i)}\big\}_{1\leq i\leq n, 1\leq t\leq T}.
\]
Notation $\Xi_{n,T}\sim \mathcal{D}_{n,T}$ implies $\bxi_t^{(i)}\sim D_t^{(i)}$ for any $i\in [n]$ and $t\in [T]$.
\item For a decentralized network with $n$ nodes, we use $\W\in\mathbb{R}^{n\times n}$ to present the confusion matrix, where $W_{ij}\geq 0$ is the weight that node $i$ sends to node $j$ ($i,j\in[n]$). $\mathcal{N}_i^{\text{out}}=\{j\in[n]:W_{ij}>0\}$ and $\mathcal{N}_i^{\text{in}}=\{k\in[n]:W_{ki}>0\}$ are also used for denoting the sets of in neighbors of and out neighbors of node $i$ respectively.
\item Norm $\|\cdot\|$ denotes the $\ell_2$ norm $\|\cdot\|_2$ by default.
\end{itemize}
\section{Related Work}
The concept of \textit{federated learning} was first proposed in \cite{mcmahan_communication-efficient_2016}, which advocates a novel learning setting that learns a shared model by aggregating locally-computed gradient updates without centralizing distributed data on devices. Early examples of research into federated learning also include \cite{konecny_federated_2015, konecny_federated_2016}, and a widespread blog article posted by Google AI \cite{mcmahan_google_2017}. To address both statistical and system challenges, \cite{smith_federated_2017} and \cite{caldas_federated_2018} propose a multi-task learning framework for federated learning and its related optimization algorithm, which extends early works SDCA \cite{shalev-shwartz_stochastic_2013, yang_trading_2013, yang_analysis_2013} and COCOA \cite{jaggi_communication-efficient_2014, ma_adding_2015, smith_cocoa:_2016} to the federated learning setting. Among these optimization methods, \textit{Federated Averaging} (FedAvg), proposed by \cite{mcmahan_communication-efficient_2016}, beats conventional synchronized mini-batch SGD with respect to communication rounds, and it converges on non-IID and unbalanced data. Recent rigorous theoretical analyses \cite{stich_local_2018, wang_cooperative_2018, yu_parallel_2018, lin_dont_2018} show that FedAvg is a special case of averaging periodic SGD (also called “local SGD”) which allows nodes to perform local updates and infrequent synchronization between them to communicate less while converging quickly. However, they cannot be applied to the single-sided trust network (asymmetric topology matrix).
Decentralized learning is a typical parallel strategy in which each worker is only required to communicate with its neighbors, meaning that the communication bottleneck (in the parameter server) is removed. It has already been proven that decentralized learning can outperform the traditional centralized learning when the worker number is comparably large under a poor network condition \cite{lian2017can}. There are two main types of decentralized learning algorithms: fixed network topology \cite{he2018cola}, and time-varying \cite{nedic2015distributed,lian2017asynchronous} during training. \cite{yin_d,pmlr-v80-shen18a} show that the decentralized SGD can converge with a comparable convergence rate to the centralized algorithm with less communication to make large-scale model training feasible. \cite{NIPS2018_8028} provides a systematic analysis of decentralized learning.
Online learning has been studied for decades. It is well known that the lower bounds of online optimization methods are $\mathcal{O}(\sqrt{T})$ and $\mathcal{O}(\log T)$ for convex and strongly convex loss functions, respectively \cite{hazan2016introduction,shalev2012online}. In recent years, due to the increasing volume of data, distributed online learning, especially decentralized methods, has attracted much attention. Examples of these works include \cite{kamp2014communication,shahrampour2017distributed,lee2016coordinate}. Notably, \cite{zhao2019decentralized} shares a similar problem definition and theoretical result as our paper. However, single-sided communication is not allowed in their setting, restricting their results.
\section{Problem Setting}
\label{sec:problem-setting}
In this paper, we consider federated learning with $n$ clients (a.k.a., nodes). Each client can be either an edge server or some other kind of computing device such as a smartphone, which has local private data and the local machine learning model $\x_i$ stored on it. We assume the topological structure of the network of these $n$ nodes can be represented by a directed graph $\mathcal{G}=(\text{nodes}:[n],\ \text{edges}:E)$ with vertex set $[n]=\{1,2,\dots,n\}$ and edge set $E\subset [n]\times [n]$. If there exists an edge $(u,v)\in E$, it means node $u$ and node $v$ have network connection and $u$ can directly send messages to $v$.
Let $\x_t^{(i)}$ denote the local model on the $i$-th node at iteration $t$. In each iteration, node $i$ receives a new sample and computes a prediction for this new sample according to the current model $\x_t^{(i)}$ (e.g., it may recommend some items to the user in the online recommendation system). After that, a loss function, $f_{i,t}(\cdot)$ associated with that new sample is received by node $i$. The typical goal of online learning is to minimize the \emph{regret}, which is defined as the difference between the summation of the losses incurred by the nodes' prediction and the corresponding loss of the global optimal model $\x^*$:
\begin{equation*}
\tilde{\mathcal{R}}_T\coloneqq \sum_{t=1}^T \sum_{i=1}^n \left(f_{i,t}(\x_t^{(i)}) - f_{i,t}(\x^*) \right),
\end{equation*}
where $\x^*=\argmin_{\x} \sum_{t=1}^T\sum_{i=1}^n f_{i,t}(\x)$ is the optimal solution.
However, here we consider a more general online setting: the loss function of the $i$-th node at iteration $t$ is $f_{i,t}(\cdot;\bxi_{i,t})$, which is additionally parametrized by a random variable $\bxi_{i,t}$. This $\bxi_{i,t}$ is drawn from the distribution $D_{i,t}$, and is mutually independent in terms of $i$ and $t$, and we call this part as the \emph{stochastic} component of loss function $f_{i,t}(\cdot;\bxi_{i,t})$. The stochastic component can be utilized to characterize the internal randomness of nodes' data, and the potential connection among different nodes. For example, music preference may be impacted by popular trends on the Internet, which can be formulated by our model by letting $D_{i,t}\equiv D_t$ for all $i\in[n]$ with some time-varying distribution $D_t$. On the other hand, function $f_{i,t}(\cdot;\cdot)$ is the \emph{adversarial} component of the loss, which may include, for example, user's profile, location, etc. Therefore, the objective regret naturally becomes the expectation of all the past losses:
\begin{equation}
\label{eq:def_regret}
\mathcal{R}_T\coloneqq
\E_{\Xi_{n,T}\sim \mathcal{D}_{n,T}}\left\{\sum_{t=1}^T \sum_{i=1}^n \left(f_{i,t}(\x_t^{(i)};\bxi_t^{(i)}) - f_{i,t}(\x^*;\bxi_t^{(i)}) \right)\right\}
\end{equation}
with $\x^*=\argmin_{\x} \E_{\Xi_{n,T}\sim \mathcal{D}_{n,T}}\sum_{t=1}^T\sum_{i=1}^n f_{i,t}(\x;\bxi_t^{(i)})$.
One benefit of the above formulation is that it partially resolves the non-I.I.D. issue in federated learning. A fundamental assumption in many traditional distributed machine learning methods is that the data samples stored on all nodes are I.I.D., which fails to hold for federated learning since the data on each user's device is highly correlated to that user's preferences and habits. However, our formulation does not require the I.I.D. assumption to hold for the adversarial component at all. Even though the random samples for the stochastic component still need to be independent, they are allowed to be drawn from different distributions.
Finally, one should note that online optimization also includes stochastic optimization (i.e., data samples are drawn from a fixed distribution) and offline optimization (i.e., data are already collected before optimization begins) as its typical cases \cite{shalev2012online}. Hence, our setting covers a wide range of applications.
\section{Online Push-Sum Algorithm}
In this section, we define the construction of the confusion matrix and introduce the proposed algorithm.
\subsection{Construction of Confusion Matrix}
% \begin{figure}[t]
% \centering
% \begin{subfigure}{0.4\linewidth}
% \includegraphics[width=0.8\linewidth]{example_graph.pdf}
% \end{subfigure}
% %
% \begin{subfigure}{0.4\linewidth}
% $$
% \W=\begin{pmatrix}
% 0 & 1 & 0 & 0 \\
% 0 & 0 & 1 & 0 \\
% \frac{1}{2} & 0 & 0 & \frac{1}{2} \\
% 0 & 1 & 0 & 0
% \end{pmatrix}
% $$
% \end{subfigure}
% \caption{An example of network topological graph that does not allow doubly stochastic confusion matrix. To make $\W$ doubly stochastic, since $(1,2)$ and $(4,2)$ are the unique out edges for vertices $1$ and $4$ respectively, we must choose $W_{1,2}=1 $ and $W_{4,2}=1$, which makes it impossible for the second column to have summation 1.}
% \label{fig:example_graph}
% \end{figure}
One important parameter of the algorithm is the confusion matrix $\W$. $\W$ is a matrix depending on the network topology $\mathcal{G}$, which means $W_{ij}=0$ if there is no directed edge $(i,j)$ in $\mathcal{G}$. If the value of $W_{ij}$ is large, the node $i$ will have a stronger impact on node $j$. However, $\W$ still allows flexibility where users can specify their weights associated with existing edges, meaning that even if there is a physical connection between two nodes, the nodes can decide against using the channel. For example, even if $(i,j)\in E$, user still can set $W_{ij}=0$ if user $i$ thinks node $j$ is not trustworthy and therefore chooses to exclude the channel from $i$ to $j$.
Of course, there are still some constraints over $\W$. $\W$ must be a row stochastic matrix (i.e., each entry in $\W$ is non-negative, and the summation of each row is 1). This assumption is different from the one in classical decentralized distributed optimization, which typically assumes $\W$ is symmetric and doubly stochastic (e.g., \cite{duchi2011dual}) (i.e., the summations of both rows and columns are all 1). Such a requirement is quite restrictive, because not all networks admit a doubly stochastic matrix (\cite{gharesifard2010does}), and relinquishing double stochasticity can introduce bias in optimization \cite{ram2010distributed,tsianos2012distributed}. As a comparison, our assumption that $\W$ is row stochastic will avoid such concerns since any non-negative matrix with at least one positive entry on each row (which is already implied by the connectivity of the graph) can be easily normalized into row stochastic. The relaxation of this assumption is crucial for federated learning, considering that the federated learning system usually involves complex network topology due to its large number of clients. Moreover, since each node only needs to make sure the summation of its out-weights is 1, there is no need for it to be aware of the global network topology, which significantly benefits the implementation of the federated learning system. Meanwhile, requiring $\W$ to be symmetric rules out the possibility of using asymmetric network topology and adopting sing-sided trust, while our method does not have such restriction.
\begin{algorithm*}[t]
\begin{multicols}{2}
\begin{algorithmic}[1]
\Require Learning rate $\gamma$, number of iterations $T$, and the confusion matrix $\W$.
\State Initialize $\x_{0}^{(i)} =\bm{z}_{0}^{(i)} = \bm{0}$, $\omega_0^{(i)} = 1$ for all $i\in [n]$
\For {$t=0,1, ..., T-1$}
\State $\slash\slash$ For all users (say the $i$-th node $i\in[n]$)
% \State \indent Predict $\x_{t}^{(i)}$.
\State Apply local model $\x_t^{(i)}$ and suffer loss $f_{i,t}(\x_t^{(i)};\bxi_t^{(i)})$
\State Locally computes the intermedia variable
\[\z_{t + \frac{1}{2}}^{(i)} = \z_t^{(i)} - \gamma \nabla f_{i,t}\left(\x_t^{(i)};\bxi_t^{(i)}\right)\]
\State Send $\left(W_{ij}\z_{t + \frac{1}{2}}^{(i)}, W_{ij}\omega_t^{(i)} \right) $ to all $j\in \mathcal{N}_i^\text{out}$
\State Update \begin{align*}
\z_{t+1}^{(i)} =& \sum_{k\in\mathcal{N}_i^\text{in}}W_{ki}\z_{t+\frac{1}{2}}^{(k)}\\
\omega_{t+1}^{(i)} = & \sum_{k\in\mathcal{N}_i^\text{in}}W_{ki}\omega_t^{(k)}\\
\x_{t+1}^{(i)} = & \frac{\z_{t+1}^{(i)}}{\omega_{t+1}^{(i)}}
\end{align*}
\EndFor
\State \Return $\x_T^{(i)}$ to node $i$
\end{algorithmic}
\end{multicols}
\caption{Online Push-Sum (OPS) Algorithm}
\label{alg}
\end{algorithm*}
\subsection{Algorithm Description}
The proposed online push-sum algorithm is presented in Algorithm \ref{alg}. The algorithm design mainly follows the pattern of push-sum algorithm \cite{tsianos2012push}, but here we further generalize it into the online setting.
The algorithm mainly consists of three steps:
\begin{enumerate}
\item Local update: each client $i$ applies the current local model $\x^{(i)}_t$ to obtain the loss function, based on which an intermediate local model $\z^{(i)}_{t+\frac{1}{2}}$ is computed;
\item Push: the weighted variable $W_{ij}\z^{(i)}_{t+\frac{1}{2}}$ is sent to $j$ for all its out neighbors $j$;
\item Sum: all the received $W_{ji}\z^{(j)}_{t+\frac{1}{2}}$ is summed and normalized to obtain the new model $\x^{(i)}_{t+1}$.
\end{enumerate}
It should be noted an auxiliary variables $\z^{(i)}_{t+\frac{1}{2}}$ and $\z^{(i)}_{t+1}$ are used in the algorithm. Actually, they are used in the algorithm to clarify the description but may be easily removed in the practical implementation. Besides, another variable $\omega^{(i)}_{t+1}$ is also introduced, which is the normalizing factor of $\z^{(i)}_{t+1}$. $\omega^{(i)}_{t+1}$ plays an important role in the push-sum algorithm, since $\W$ is not doubly stochastic in our setting, and it is possible that the total weight $i$ receives does not equal to 1. The introduction of the normalizing factor $\omega_t^{(i)}$ helps the algorithm avoid issues brought by that $\W$ is not doubly stochastic. Furthermore, when $\W$ becomes doubly stochastic, it can be easily verified that $\omega_t^{(i)}\equiv 1$ and $\x^{(i)}_t\equiv\z^{(i)}_t$ for any $i$ and $t$, then Algorithm \ref{alg} reduces to the distributed online gradient method proposed by \cite{zhao2019decentralized}.
In the algorithm, the local data, which is encoded in the gradient $f_{i,t}(\x_t^{(i)};\bxi_t)$ \cite{shokri2015privacy}, is only utilized in updating local model. What neighboring nodes exchanges are only limited to the local models.
\subsection{Regret Analysis}
In this subsection, we provide regret bound analysis of OPS algorithm. Due to the limitation of space, the detail proof is deferred to the supplementary material. For convenience, we first denote
\[
F_{i,t}(\x)\coloneqq \E_{\bxi_{i,t}\sim D_{i,t}} f_{i,t}(\x;\bxi_{i,t}).
\]
To carry out the analysis, the following assumptions are required:
%\begin{definition}[B-Strongly Connected] Suppose that $E^{t}$ denotes the edge set of the graph at the $t$-th iteration. A graph is strongly connected if every vertex can be reached from any other vertices. Then, a sequence of directed graphs is B-strongly connected if a graph with an edge set $\cup_{i=\ell B}^{(\ell+1)B-1} E^{t}$ is strongly connected for all $\ell\geq 0$.
%\end{definition}
% With the this definition, we will subsequently make an assumption that the sequence of time varying graph is B-strongly connected, which is shown below. Note that this is a common assumption,which is also found in previous literature \cite{nedic2014distributed,nedic2016stochastic,assran2018asynchronous,assran2018stochastic}.
\begin{assumption} \label{assump}
We make the following assumptions throughout this paper: (1) The topological graph $\mathcal{G}$ is strongly connected; $\W$ is row stochastic; (2) For any $i\in[n]$ and $t\in[T]$, the loss function $f_{i,t}(\x;\bxi_{i,t})$ is convex in $\x$; (3) The problem domain is bounded such that for any two vectors $\x$ and $\bm{y}$ we always have $\|\x-\bm{y}\|^2\leq R$; (4) The norm of the expected gradient $\nabla F_{i,t}(\cdot)$ is bounded, i.e., there exist constant $G>0$ such that $\|\nabla F_{i,t}(\x)\|^2\leq G^2$ for any $i$, $t$ and $\x$; (5) The gradient variance is also bounded by $\sigma^2$, namely,
\[
\E_{\bxi_{i,t}\sim D_{i,t}}\|\nabla f_{i,t}(\x;\bxi_{i,t}) - \nabla F_{i,t}(\x)\|^2\leq \sigma^2.
\]
%\item The sequence of the graph is B-strongly connected.
\end{assumption}
Here constant $G$ provides an upper bound for the adversarial component. On the other hand, $\sigma$ measures the magnitude of stochasticity brought by the stochastic component. When $\sigma=0$, the problem setting simply reduces back to normal distributed online learning. The strong connectivity assumption is necessary to ensure that the information can be exchanged between any two nodes. As for the convexity and the domain boundedness assumptions, they are quite common in online learning literature, such as \cite{hazan2016introduction}.
Equipped with these assumptions, now we are ready to present the convergence result:
% \begin{corollary}
% If we set
% \[
% \gamma=\frac{R^2}{\sqrt{\frac{\sigma^2}{n}+C_2 G}},
% \]
% the regret of OPS can be bounded by:
% \begin{equation}
% \mathcal{R}_T \leq \mathcal{O}\left( \frac{GR}{\sqrt{T}} + \frac{\sigma}{\sqrt{n T}} \right).
% \end{equation}
% \end{corollary}
\begin{theorem}
\label{theory:corollary}
If we set
\begin{equation}
\label{eq:set_gamma}
\gamma=\frac{\sqrt{n}R}{\sigma\sqrt{ 1 + nC_2}+G\sqrt{n C_1 T}},
\end{equation}
the regret of OPS can be bounded by:
\begin{equation}
\mathcal{R}_T \leq \mathcal{O}\left( nGR\sqrt{T} + \sigma R\left( 1 + \sqrt{nC_2}\right)\sqrt{nT} \right),
\end{equation}
where $C_1$ and $C_2$ are two constants defined in the appendix.
\end{theorem}
Note that when $n=1$ and $\sigma=0$, where the problem setting just reduces to normal online optimization, the implied regret bound $\mathcal{O}(GR\sqrt{T})$ exactly matches the lower bound of online optimization \cite{hazan2016introduction}. Moreover, our result also matches the convergence rate of centralized online learning where $q=0$ for fully connected networks. Hence, we can conclude that the OPS algorithm has optimal dependence on $T$.
This bound has a linear dependence on the number of nodes $n$, but it is easy to understand. First, we have defined the regret to be the summation of the losses on all the nodes. Increasing $n$ makes the regret naturally larger. Second, our federated learning setting is different from the typical distributed learning in that I.I.D. assumption does not hold here. Each node contains distinct local data that may be drawn from totally different distributions. Therefore, adding more nodes is not helpful for decreasing the regret of existing clients.
Moreover, we also prove that the difference of the model $\x_t^{(i)}$ on each worker could be bounded using the following theorem:
\begin{theorem}
\label{theory:corollary2}
If we set $\gamma$ as \eqref{eq:set_gamma}, the difference of the model $\x_t^{(i)}$ on each worker admits a faster convergence rate than regret:
\begin{align*}
\frac{1}{T}\sum_i^{n}\sum_{t=0}^T \left\| \x^{(i)}_{t+1} - \overline{\z}_{t+1} \right \|^2 \leq &\mathcal{O}\left(\frac{nGR + nR\sigma}{T}\right).
\end{align*}
\end{theorem}
Hence, the models on all clients' devices will finally converge to the same one with rate $\mathcal{O}(1/T)$.
\subsection{Privacy Protection}
Although the main focus of this work is to provide a solution to deal with the practical scenario with asymmetric social networks, our proposed algorithm also has several advantages concerning privacy protection. First, as we have mentioned, OPS runs in a decentralized way and exchanges models instead of gradients or training samples, which is already proven effective for reducing the risk of privacy leakage \cite{bellet2017personalized}. Second, OPS runs in a decentralized and asymmetric fashion. These properties create difficulties for many attacking methods, such as \cite{nasr2018comprehensive}. In order to infer the data of other clients, the attacker needs to observe the reactions of other nodes after the attack is injected, which is impossible when the connections are single-sided. Even though the attack will spread among the whole network and finally return to the attacker, it is still hard for the attacker to distinguish whether the information he receives from its neighbors is already affected by the attack or not, since he is unaware of the global topology.
%On the other hand, if we look into the averaged regret per iteration of each node, namely,
%\[
%\bar{\mathcal{R}}_T\coloneqq
%\frac{\mathcal{R}_T}{nT} \leq \mathcal{O}\left( \frac{GR}{\sqrt{T}} + \frac{\sigma R\left( 1 + \sqrt{nC_2}\right)}{\sqrt{nT}} \right)
%\]
%we can find the first term is independent of the number of clients $n$. But unlike centralized online learning, where there is no $\mathcal{O}\left(\frac{\sigma R}{\sqrt{T}}\right)$ term of the data variance that also depends on the network topology.
\section{Experiments}
We compare the performance of our proposed \pushsum\ method with that of Decentralized Online Gradient method (DOL) and Centralized
Online Gradient method (COL), and then evaluate the effectiveness of OPS in different network size and network topology density settings.
\subsection{Implementation and Settings}We consider online logistic regression with squared $\ell_{2}$ norm regularization:
$f_{i, t}\left(\mathbf{x} ; \bxi_{i, t}\right)= \log \left(1+\exp \left(-\mathbf{y}_{i, t} \mathbf{A}_{i, t}^{\top} \mathbf{x}\right)\right)+\frac{\lambda}{2}\|\mathbf{x}\|^{2}$,
where regularization coefficient $\lambda$ is set to $10^{-4}$. $\bxi_{i, t}$ is the stochastic component of the function $f_{i, t}$ introduced in Section \textsection~\ref{sec:problem-setting}, which is encoded in the random data sample $(\mathbf{A}_{i, t}, \mathbf{y}_{i,t})$. We evaluate the learning performance by measuring the average loss
$\frac{1}{n T}\mathbb{E}_{\Xi_{n, T}} \sum_{i=1}^{n} \sum_{t=1}^{T} f_{i, t}\left(\mathbf{x}_{i, t} ; \bxi_{i, t}\right)$,
instead of using the dynamic regret \eqref{eq:def_regret} directly, since the optimal reference point $x^*$ is the same for all the methods. The learning rate $\gamma$ in Algorithm \ref{alg} is tuned
to be optimal for each dataset separately. The experiment implementation is based on Python 3.7.0, PyTorch 1.2.0, NetworkX 2.3, and scikit-learn 0.20.3. The source code is developed using \texttt{FedML} \cite{chaoyanghe2020fedml}, an open-source research library for federated learning. Please refer to \texttt{FedML}'s source code for details.
\textbf{Dataset}. Experiments were run on two real-world public datasets: \textit{SUSY}\footnote{\url{https://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/binary.html##SUSY}} and \textit{Room-Occupancy}\footnote{\url{https://archive.ics.uci.edu/ml/datasets/Occupancy+Detection+}}. SUSY and \textit{Room-Occupancy} are both large-scale binary classification
datasets, containing 5,000,000 and 20,566 samples, respectively. Each dataset is split into two subsets: the stochastic data and the adversarial
data. The stochastic data is generated by allocating a fraction of samples (e.g., 50\% of the whole dataset) to nodes randomly and uniformly. The adversarial data is generated by conducting on the remaining dataset to produce $n$ clusters and then allocating every cluster to a node. As we analyzed previously, only the scattered stochastic data can boost the model performance by intra-node communication. For each node, this pre-acquired data is transformed into streaming data to simulate online learning.
\begin{figure*}[t]
\centering
\begin{subfigure}[b]{0.23\textwidth}
\centering
\includegraphics[width=\textwidth]{1-SUSY-beta100-N128.pdf}
\caption{stochastic=100\% \\ ($n=128$, \#Neighb.=32)}
\label{fig:1-SUSY-beta100}
\end{subfigure}
\begin{subfigure}[b]{0.23\textwidth}
\centering
\includegraphics[width=\textwidth]{1-SUSY-beta50-N128.pdf}
\caption{stochastic=50\% \\($n=128$, \#Neighb.=32)}
\label{fig:1-SUSY-beta50}
\end{subfigure}
\begin{subfigure}[b]{0.23\textwidth}
\centering
\includegraphics[width=\textwidth]{1-RoomOccupancy-beta0.pdf}
\caption{stochastic=100\% \\($n=20$, \#Neighb.=10)}
\label{fig:1-Room-beta100}
\end{subfigure}
\begin{subfigure}[b]{0.23\textwidth}
\centering
\includegraphics[width=\textwidth]{1-RoomOccupancy-beta5.pdf}
\caption{stochastic=50\% \\ ($n=20$, \#Neighb.=10)}
\label{fig:1-Room-beta50}
\end{subfigure}
\caption{Comparison among OPS, DOL (Decentralized Online Learning) and COL (Centralized Online Learning)}
\label{fig:comparision-DOL-COL}
\vspace{-0.3cm}
\end{figure*}
\subsection{Comparison with DOL and COL}
To compare OPS with DOL and COL, a network size with 128 nodes and 20 nodes are selected for SUSY and Room-Occupancy, respectively.
For COL, its confusion matrix $\W$ is fully-connected (doubly stochastic matrix).
For DOL and OPS, they are run with the same network topology and the same row stochastic matrix (asymmetric confusion matrix) to maintain a fair comparison.
Such asymmetric confusion is constructed by setting each node's number of neighbors as a random value which is smaller than a fixed upper bound and also ensures the strong connectivity of the whole network (this upper-bound neighbor number is set to 32 for the SUSY dataset, while 10 is set for the Room-Occupancy dataset). Since DOL typically requires the network to be the symmetric and doubly stochastic confusion matrix, DOL is run in two settings for comparison. In the first setting, in order to meet the assumption of the symmetry and doubly stochasticity, all unidirectional connections are removed in the confusion matrix so that the row stochastic confusion matrix degenerates into a doubly stochastic matrix. This setting is labeled as \textit{DOL-Symm} in Figure \ref{fig:comparision-DOL-COL}. In another setting, DOL is forced to run on the asymmetric network where each node naively aggregates its received models without considering whether its sending weights are equal to its receiving weights. \textit{DOL-Asymm} is used to label this setting in Figure \ref{fig:comparision-DOL-COL}.
As illustrated in Figure \ref{fig:comparision-DOL-COL}, in both two datasets, OPS outperforms \textit{DOL-Symm} in the row stochastic confusion matrix. This demonstrates that incorporating unidirectional communication can help to boost the model performance. In other words, OPS gains better performance in the single-sided trust network under the setting of federated learning. OPS also works better than \textit{DOL-Asymm}. Although \textit{DOL-Asymm} utilizes additional unidirectional connections, in some cases its performance is even worse than \textit{DOL-Symm} (e.g., Figure \ref{fig:1-SUSY-beta100}). This phenomenon is most likely attributed to its simple aggregation pattern, which causes decreased performance in \textit{DOL-Asymm} when removing the doubly stochastic matrix assumption. These two observations confirm the effectiveness of OPS in a row stochastic confusion matrix, which is consistent with our theoretical analysis.
Comparing Figure \ref{fig:1-Room-beta100} and Figure \ref{fig:1-Room-beta50}, we also observe that when increasing the ratio of the stochastic component, the average loss (regret) becomes smaller. It is reasonable that OPS achieves slightly worse performance than COL because OPS works in a sparsely connected network where information exchanging is much less than COL. We use the COL as the baseline in all experiments.
Only the number of iterations instead of the actual running time is considered in the experiment. It is redundant to present the actual running time. Because the centralized method requires more time for each iteration due to the network congestion in the central node, OPS usually outperforms COL in terms of running time.
\subsection{Evaluation on Different Network Sizes}
\label{sec:network_size}
\begin{figure*}[t]
\centering
\begin{subfigure}[b]{0.24\textwidth}
\centering
\includegraphics[width=\textwidth]{2-SUSY-beta100.pdf}
\caption{stochastic=100\%}
\label{fig:2-SUSY-beta100-size}
\end{subfigure}
\begin{subfigure}[b]{0.24\textwidth}
\centering
\includegraphics[width=\textwidth]{2-SUSY-beta50.pdf}
\caption{stochastic=50\%}
\label{fig:2-SUSY-beta50-size}
\end{subfigure}
%
\begin{subfigure}[b]{0.24\textwidth}
\centering
\includegraphics[width=\textwidth]{3-SUSY-beta100.pdf}
\caption{stochastic=100\%}
\label{fig:2-SUSY-beta100-density}
\end{subfigure}
\begin{subfigure}[b]{0.24\textwidth}
\centering
\includegraphics[width=\textwidth]{3-SUSY-beta50.pdf}
\caption{stochastic=50\%}
\label{fig:2-SUSY-beta50-density}
\end{subfigure}
\caption{Evaluation on different network sizes and densities}
\label{fig:networksize}
\vspace{-0.3cm}
\end{figure*}
Figure \ref{fig:2-SUSY-beta100-size} and \ref{fig:2-SUSY-beta50-size} summarizes the evaluation of OPS in different network sizes (in the \textit{SUSY} dataset, 128, 256, 512, 1024 are set). The upper-bound neighbor number is aligned to the same value among different network sizes to isolate its impact. As we can see, in every dataset, the average loss (regret) curve in different network sizes is close on a small scale. These observations demonstrate OPS is robust to the network size. Furthermore, the average loss (regret) is smaller in larger network size (i.e., the curve of the $n=1024$ network size is lower than others), which also demonstrates that more stochastic samples provided by more nodes can naturally accelerate the convergence. Due to limitation of space, the results on the other dataset is deferred to the appendix.
\subsection{Evaluation on Network Density}
\label{sec:network_density}
We also evaluate the performance of OPS in different network densities. We fix the network size to 512 for \textit{SUSY} dataset. Network density is defined as the ratio of the upper-bound random neighbor number per node to the size of the network (e.g., if the ratio is 0.5 in \textit{SUSY}, it means 256 is set as the upper-bound neighbor number for each node). We can see from Figure \ref{fig:2-SUSY-beta100-density} and \ref{fig:2-SUSY-beta50-density} that as the network density increased, the average loss (regret) decreased. This observation also proves that our proposed OPS algorithm can work well in different network densities, and can gain more benefits from a denser row stochastic matrix. This benefit can also be understood intuitively: in a federated learning network, a user's model performance will improve if it communicates with more users. The results of \textit{Room Occupancy} are also deferred to the appendix.
\subsection{Comparison with local online gradient descent}
To justify the necessity of communication, we compare OPS with the local online gradient descent (local OGD), where every node trains a local model without communicating with
others. The results are described in the appendix.
% We also evaluate the performance of OPS in different network densities. We fix the network size to 512 and 20 for \textit{SUSY} and \textit{Room Occupancy} dataset, respectively. Network density is defined as the ratio of the upper-bound random neighbor number per node to the size of the network (e.g., if the ratio is 0.5 in \textit{SUSY}, it means 256 is set as the upper-bound neighbor number for each node). We can see from Figure \ref{fig:density} that as the network density increased, the average loss (regret) decreased. This observation also proves that our proposed OPS algorithm can work well in different network densities, and can gain more benefits from a denser row stochastic matrix. This benefit can also be understood intuitively: in a federated learning network, a user's model performance will improve if it communicates with more users.
\section{Conclusions}
Decentralized federated learning with single-sided trust is a promising framework for solving a wide range of problems. In this paper, the online push-sum algorithm is developed for this setting, which is able to handle complex network topology and is proven to have an optimal convergence rate. The regret-based online problem formulation also extends its applications. We tested the proposed OPS algorithm in various experiments, which have empirically justified its efficiency.
\section*{Broader Impact}
The federated learning framework this paper proposed offers an effective tool to cooperatively train machine learning models in the applications where the willing of sharing information is single-sided. Such kind of applications abounds in real lives. Moreover, as we stated, such a framework has the potential of protecting user' privacy in a better way, while we believe privacy protection is one of the most important problem in such an information era. However, in some cases, single-sided communication may increase the information asymmetry among people. Part of the users will always get to know the information faster than the others, which may be considered as a kind of inequality of humans.
\bibliographystyle{unsrt}
\bibliography{ref}
\input{supp.tex}
\end{document}
\newpage
\onecolumn
\vspace{1cm}
\begin{center}
\LARGE{Appendix} \vspace{0.8cm}
\end{center}
\paragraph{Notations:} Below we use the following notation in our proof
\begin{itemize}
\item $\nabla F_t(\X_t) := \left[\nabla F_{1,t}\left(\x_t^{(1)}\right),\cdots, \nabla F_{n,t}\left(\x_t^{(n)}\right)\right] $
\item $\X_{t} := \left[\x^{(1)}_{t},\x^{(2)}_{t},...,\x^{(n)}_{t}\right]$
\item $\G_t := \left[ \nabla f_{1,t}(\x_t^{1};\bxi_t^{1}),\dots, \nabla f_{n,t}(\x_t^{n};\bxi_t^{n})\right]$
\end{itemize}
Here we first present the proof Theorem~\ref{theory:corollary}, then we will present some key lemmas along with the proof of Theorem~\ref{theory:corollary2}.
\section{Proof to Theorem~\ref{theory:corollary} and Theorem~\ref{theory:corollary2}}
The following theorem is the key to prove Theorem~\ref{theory:corollary}:
\begin{theorem}\label{main:theorem}
For the online push-sum algorithm with step size $\gamma>0$, it holds that
\begin{equation}
\mathcal{R}_T \leq G^2Tn\gamma C_1 + \sigma^2T\gamma ( 1 + nC_2) + \frac{nR^2}{2\gamma},
\end{equation}
where
\begin{equation*}
C_1:= \frac{8 Cq}{\delta_{\min}(1-q)} + 1,\quad
C_2:= \frac{2 Cq}{\delta_{\min}(1-q)},
\end{equation*}
and $C$, $q$ and $\delta_{min}$ are some constants defined in later lemmas.
\end{theorem}
\begin{proof}
Since the loss function $f_{i,t}(\cdot)$ is assumed to be convex, which leads to
\begin{align*}
& \mathbb E_{t}\sum_{i=1}^n f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right) - n F_t(\x^{*})\\
= & \mathbb E_{t}\sum_{i=1}^n\left ( f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right) - f_{i,t}\left(\x^{*};\bm{\xi}^{(i)}_t \right) \right)\\
\leq & \mathbb E_{t}\sum_{i=1}^n\left \langle \nabla f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right), \x_t^{(i)} - \x^* \right\rangle \\
= & \underbrace{\mathbb E_{t}\sum_{i=1}^n\left \langle \nabla f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right), \x_t^{(i)} - \overline{\z}_t \right\rangle }_{:=I_{1t}} + \underbrace{\mathbb E_{t}\sum_{i=1}^n\left \langle \nabla f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right), \overline{\z}_t - \x^{*} \right\rangle }_{:=I_{2t}}.
\end{align*}
For $I_{2t}$, we have
\begin{align*}
& \mathbb E_{t}\sum_{i=1}^n\left \langle \nabla f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right), \overline{\z}_t - \x^{*} \right\rangle \\
= & \frac{n}{\gamma} \mathbb E_{t}\left \langle \frac{\gamma}{n}\sum_{i=1}^n\nabla f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right), \overline{\z}_t - \x^{*} \right\rangle \\
= & \frac{n}{2\gamma}\mathbb E_{t}\left( \left\|\frac{\gamma}{n}\sum_{i=1}^n\nabla f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right) \right\|^2 + \left\|\overline{\z}_t - \x^{*}\right\|^2 - \left\| \overline{\z}_t - \x^{*} -\frac{\gamma}{n}\sum_{i=1}^n\nabla f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right) \right\|^2 \right)\\
= & \frac{n}{2\gamma}\mathbb E_{t}\left( \left\|\frac{\gamma}{n}\sum_{i=1}^n\nabla f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right) \right\|^2 + \left\|\overline{\z}_t - \x^{*}\right\|^2 - \left\| \overline{\z}_{t+1} - \x^{*} \right\|^2 \right)\\
\leq & \frac{n}{2\gamma}\mathbb E_{t}\left( \gamma^2G^2 + \frac{\gamma^2\sigma^2}{n} + \left\|\overline{\z}_t - \x^{*}\right\|^2 - \left\| \overline{\z}_{t+1} - \x^{*} \right\|^2 \right)
\end{align*}
Notice that for COL, we have $I_{1t} = 0$ because $\x_t^{(i)} = \overline{\z}_t$. So for DOL, in order to bound $I_{1t}$, we need to bound the difference $\left\|\x_t^{(i)} - \overline{\z}_t\right\|$ (using Lemma~\ref{lem:mat}).
\begin{align*}
&\mathbb E_{t}\sum_{i=1}^n\left \langle \nabla f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right), \x_t^{(i)} - \overline{\z}_t \right\rangle\\
=& \mathbb E_{t}\sum_{i=1}^n\left \langle \nabla F_{i,t}(\x_t^{(i)}), \x_t^{(i)} - \overline{\z}_t \right\rangle\\
\leq & \mathbb E_{t}\sum_{i=1}^n\left(\alpha \left\|\nabla F_{i,t}\left(\x_t^{(i)}\right) \right\|^2 + \frac{1}{\alpha} \left\|\x_t^{(i)} - \overline{\z}_t \right\|^2 \right).
\end{align*}
Summing up the inequality above from $t=1$ to $t=T$, we get
\begin{align*}
&\sum_{t=1}^T\mathbb E_{t}\sum_{i=1}^n\left \langle \nabla f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right), \x_t^{(i)} - \overline{\z}_t \right\rangle\\
= & \sum_{t=1}^T\mathbb E_{t}\sum_{i=1}^n\left \langle \nabla F_{i,t}\left(\x_t^{(i)}\right), \x_t^{(i)} - \overline{\z}_t \right\rangle\\
\leq & \sum_{t=1}^T\mathbb E_{t}\sum_{i=1}^n\left( \alpha \left\| \nabla F_{i,t}\left(\x_t^{(i)}\right)\right\|^2 + \frac{1}{\alpha}\left\| \x_t^{(i)} - \overline{\z}_t \right\|^2\right)\\
= & \sum_{t=1}^T \left(\alpha\mathbb E_{t} \left\|\nabla F_{t}(\X_t) \right\|^2_F + \frac{1}{\alpha}\mathbb E_{t}\left\|\X_t - \overline{\z}_t \right\|^2_F \right)\\
\leq & \alpha\sum_{t=1}^T \mathbb E_{t} \left\|\nabla F_{t}\left(\X_t\right) \right\|^2_F + \frac{4\gamma^2C^2q^2}{\alpha\delta_{\min}^2(1-q)^2}\sum_{t=1}^T \mathbb E_{t} \left\| \G_{t} \right\|^2_F\\
\leq & \alpha\sum_{t=1}^T \mathbb E_{t} \left\|\nabla F_{t}\left(\X_t\right) \right\|^2_F + \frac{4\gamma^2C^2q^2}{\alpha\delta_{\min}^2(1-q)^2}\sum_{t=1}^T \left( \mathbb E_{t} \left\| \nabla F_{t}(\X_t) \right\|^2_F + n\sigma^2 \right).
\end{align*}
Choosing $\alpha = \frac{2\gamma Cq}{\delta_{\min}(1-q)}$, we have
\begin{align*}
\sum_{t=1}^T\mathbb E_{t}\sum_{i=1}^n\left \langle \nabla f_{i,t}\left(\x_t^{(i)};\bm{\xi}_t^{(i)}\right), \x_t^{(i)} - \overline{\z}_t \right\rangle
\leq \frac{8n\gamma CTqG^2}{\delta_{\min}(1-q)} + \frac{2n\gamma Cq\sigma^2T}{\delta_{\min}(1-q)}
\end{align*}
So we have
\begin{align*}
& \sum_{t=1}^T\mathbb E_{t}\sum_{i=1}^n f_{i,t}\left(\bold{z}_t^{(i)};\bm{\xi}_t^{(i)}\right) - n F(\x^{*})\\
\leq & \frac{8n\gamma CTqG^2}{\delta_{\min}(1-q)} + \frac{2\gamma Cq\sigma^2T}{\delta_{\min}(1-q)} + \frac{n}{2n\gamma}\sum_{t=1}^T\left( \gamma^2G^2 + \frac{\gamma^2\sigma^2}{n} + \mathbb E_{t}\left\|\overline{\z}_t - \x^{*}\right\|^2 - \mathbb E_{t}\left\| \overline{\z}_{t+1} - \x^{*} \right\|^2 \right)\\
\leq & G^2Tn\gamma\left( \frac{8 Cq}{\delta_{\min}(1-q)} + 1 \right) + \sigma^2T\gamma\left(1 + \frac{2n Cq}{\delta_{\min}(1-q)} \right) + \frac{n}{2\gamma}\sum_{t=1}^T\left( \mathbb E_{t}\left\|\overline{\z}_t - \x^{*}\right\|^2 - \mathbb E_{t}\left\| \overline{\z}_{t+1} - \x^{*} \right\|^2 \right)\\
\leq &G^2Tn\gamma\left( \frac{8 Cq}{\delta_{\min}(1-q)} + 1 \right) + \sigma^2T\gamma\left(1 + \frac{2n Cq}{\delta_{\min}(1-q)} \right) + \frac{nR^2}{2\gamma}\\
= & C_1nG^2T\gamma + (1 + nC_2)\sigma^2T\gamma + \frac{nR^2}{2\gamma}.
\end{align*}
Notice that Theorem~\ref{theory:corollary} can be easily verified by setting $\gamma=\frac{\sqrt{n}R}{\sqrt{(1 + nC_2)\sigma^2}+\sqrt{n C_1 G^2T}}$.
\end{proof}
Next, we will present two lemmas for our proof of Lemma \ref{lem:mat}. The proofs of following two lemmas can be found in existing literature \cite{nedic2014distributed,nedic2016stochastic,assran2018asynchronous,assran2018stochastic}.
\begin{lemma} \label{lem:prod_lower}
Under the Assumption \ref{assump}, there exists a constant $\delta_{\min} > 0$ such that for any $t$, the following holds
\begin{align}
\sum_{j=1}^n [ \W^{t}{}^\top \W^{t}{}^\top ...\W^{0}{}^\top ]_{ij} \geq \delta_{\min}\geq \frac{1}{n^n},\ \forall i
\end{align}
where $\W^t$ is a row stochastic matrix.
\end{lemma}
\begin{lemma} \label{lem:concentr} Under the Assumption \ref{assump}, for any $t$, there always exists a stochastic vector $\psi(t)$ and two constants $C=4$ and $q= 1-n^{-n} < 1$ such that for any $s$ satisfying $s\leq t$, the following inequality holds
\begin{align*}
\left|[\W^{t}{}^\top \W^{t}{}^\top \cdots \W^{s+1}{}^\top \W^{s}{}^\top]_{ij} - \psi_i(t) \right| \leq C q^{t-s}, \forall i,j
\end{align*}
where $\W^{t}$ is a row stochastic matrix, and $\psi(t)$ is a vector with $\psi_i(t)$ being its $i$-th entry.
\end{lemma}
\begin{lemma}\label{lemma:seq}
Given two non-negative sequences $\{a_t\}_{t=1}^{\infty}$ and $\{b_t\}_{t=1}^{\infty}$ that satisfying
\begin{equation}
a_t = \sum_{s=1}^t\rho^{t-s}b_{s}, \numberthis \label{eqn1}
\end{equation}
with $\rho\in[0,1)$, we have
\begin{align*}
D_k:=\sum_{t=1}^{k}a_t^2 \leq & % \frac{(1+\frac{2\rho}{1-\rho})\sum_{t=1}^kb_t^2-\rho^2a_k^2}{1-\rho^2} \\ \leq &
\frac{1}{(1-\rho)^2} \sum_{s=1}^kb_s^2.
\end{align*}
%where $S_k=\sum_{t=1}^{k}a_t$, and $D_k=\sum_{t=1}^{k}a_t^2$.
\end{lemma}
\begin{proof}
From the definition, we have
\begin{align*}
S_k= & \sum_{t=1}^{k}\sum_{s=1}^t\rho^{t-s}b_{s}
= \sum_{s=1}^{k}\sum_{t=s}^k\rho^{t-s}b_{s}
= \sum_{s=1}^{k}\sum_{t=0}^{k-s}\rho^{t}b_{s}
\leq \sum_{s=1}^{k}{b_{s}\over 1-\rho}, \numberthis \label{eqn3}\\
D_k= & \sum_{t=1}^{k}\sum_{s=1}^t\rho^{t-s}b_{s}\sum_{r=1}^t\rho^{t-r}b_{r}\\
= & \sum_{t=1}^{k}\sum_{s=1}^t\sum_{r=1}^t\rho^{2t-s-r}b_{s}b_{r} \\
\leq & \sum_{t=1}^{k}\sum_{s=1}^t\sum_{r=1}^t\rho^{2t-s-r}{b_{s}^2+b_{r}^2\over2}\\
= & \sum_{t=1}^{k}\sum_{s=1}^t\sum_{r=1}^t\rho^{2t-s-r}b_{s}^2 \\
\leq & {1\over 1-\rho}\sum_{t=1}^{k}\sum_{s=1}^t\rho^{t-s}b_{s}^2\\
\leq & {1\over (1-\rho)^2}\sum_{s=1}^{k}b_{s}^2.
\end{align*}
\end{proof}
Based on the above three lemmas, we can obtain the following lemma.
\begin{lemma} \label{lem:mat} Under the Assumption \ref{assump}, the updating rule of Algorithm \ref{alg} leads to the following inequality
\begin{align*}
\sum_i^{n}\sum_{t=0}^T \left\| \x^{(i)}_{t+1} - \overline{\z}_{t+1} \right \|_2^2 \leq & \frac{4\gamma^2 C^2q^2}{\delta_{\min}^2(1-q)^2}\sum_{s=0}^t \|\G_{s}\|_F^2,
\end{align*}
where $\gamma$ is the step size, and $C=4, \delta_{\min} \geq n^{-n}, q= 1-n^{-n} $ are constants. $\G_{s}$ is the matrix for the stochastic gradient at time $s$ (e.g., the $i$-th column is the stochastic gradient vector on node $i$ at time $s$).
\end{lemma}
\begin{proof}
The updating rule of OPS can be formulated as
\begin{align*}
&\Z_{t+1} = \left ( \Z_{t} - \gamma \G_{t} \right ) \W\\
&\omega_{t+1} = \W{}^\top \omega_{t} \\
&\X_{t+1} = \Z_{t+1} [\mathrm{diag} (\omega_{t+1})]^{-1}
\end{align*}
where $\W$ is a row stochastic matrix. $\X_{t} = [\x^{(1)}_{t},\x^{(2)}_{t},...,\x^{(n)}_{t}]$ is a matrix whose each column is $\x^{(i)}_{t}$. $G_{t}$ is the matrix of gradient, whose each column is the stochastic gradient at $\z^{(i)}_{t}$ on node $i$. $\Z_{t} = [\z^{(1)}_{t},...,\z^{(n)}_{t}]$ is the matrix whose each column is $\z^{(i)}_{t}$.
Assuming $X_{0} = O$ and $\omega_{0} = 1$, then we have
\begin{align}
& \Z_{t+1} = \left ( \Z_{t} - \gamma \G_{t} \right ) \W = ... = - \gamma \sum_{s = 0}^{t}\G_{s} \W^{t-s+1} ,\label{eq:reform1}\\
& \overline{\z}_{t+1} = \overline{\z}_{t} - \gamma \overline{\mathbf{g}}_{t} = ... = - \sum_{s = 0}^{t} \gamma \overline{\mathbf{g}}_{s}, \label{eq:reform2} \\
&\omega_{t+1} = \W^{t+1}{}^\top \omega_{0},\label{eq:reform3}
\end{align}
where $\overline{\x}_{t} = \X_{t} 1$ is the average of all variables on the $n$ nodes, and $\overline{\mathbf{g}}_{t} = \G_{t} 1$ is the averaged gradient. We have $\W 1 = 1$ since $\W$ is a row stochastic matrix.
For $\omega_{t+1}$, according to Lemma \ref{lem:concentr}, we decompose it as follows
\begin{align}
\omega_{t+1} =& \W^{t+1}{}^\top \omega_{0} = [ \W^{t+1}{}^\top - \psi(t) 1^\top ]\omega_{0} + \psi(t) 1^\top \omega_{0}
= [\W^{t+1}{}^\top - \psi(t) 1^\top ]1 + n \psi(t), \label{eq:concentr}
\end{align}
since $\omega_{0} = 1$.
On the other hand, according to Lemma \ref{lem:prod_lower}, we also have
\begin{align}
\omega_{t+1}^{(i)} = [ \W^{t+1}{}^\top 1]^\top \e_i =
\sum_{j=1}^n [\W^{t+1}{}^\top ]_{ij} \geq n\delta_{\min},\label{eq:lower}
\end{align}
where $\e_i$ is a vector with only the $i$-th entry being $1$ and $0$ for others.
We need to further bound the following term
\begin{align*}
\left\| \x^{(i)}_{t+1} - \overline{\z}_{t+1} \right \| =& \gamma \left \| \frac{\z_{t+1}^{(i)}} {\omega_{t+1}^{(i)}} - \overline{\z}_{t+1} \right \| \\
=& \gamma \left \| \sum_{s = 0}^{t}\left ( \frac{\G_{s} \W^{t-s+1} \e_i } {1^\top \W^{t+1}\e_i} - \frac{\G_{s} 1}{n} \right ) \right \| \\
=& \gamma \left \| \sum_{s = 0}^{t} \frac{n\G_{s} \W^{t-s+1} \e_i - \G_{s} 1 1^\top \W^{t+1} \e_i} {n\omega_{t+1}^{(i)}}\right \|,
\end{align*}
where the second equality is by \eqref{eq:reform1}, \eqref{eq:reform2}, and \eqref{eq:reform3}. We turn to bound the following term
\begin{align*}
&\left \| \sum_{s = 0}^{t} \frac{n\G_{s} \W^{t-s+1} \e_i - \G_{s} 1 1^\top \W^{t+1} \e_i} {n\omega_{t+1}^{(i)}}\right \| \\
\leq & \frac{1}{n^2 \delta_{\min}} \left \| \sum_{s = 0}^{t} \left( n\G_{s} \W^{t-s+1} \e_i - \G_{s} 1 1^\top \W^{t+1} \e_i \right) \right \|,
\end{align*}
where the first inequality is accordng to \eqref{eq:lower}. Therefore, combining the results above, we can have
\begin{align*}
\sum_{i=1}^n \left\| \x^{(i)}_{t+1} - \overline{\z}_{t+1} \right \|^2_2 &\leq \frac{\gamma^2}{n^4 \delta_{\min}^2} \sum_{i=1}^n \left \| \sum_{s = 0}^{t} \left( n\G_{s} \W^{t-s+1} \e_i - \G_{s} 1 1^\top \W^{t+1} \e_i \right) \right \|_2^2 \\
&\leq \frac{\gamma^2}{n^4 \delta_{\min}^2} \left \| \sum_{s = 0}^{t} \left( n\G_{s} \W^{t-s+1} - \G_{s} 1 1^\top \W^{t+1} \right) \right \|_F^2
\end{align*}
where the second inequality is due to $\sum_{i=1}^n \|\mathbf{A} \e_i\|_2^2 = \|\mathbf{A}\|_F^2$.
It remains to bound the following term
\begin{align*}
&\left \| \sum_{s = 0}^{t} \left( n\G_{s} \W^{t-s+1} - \G_{s} 1 1^\top \W^{t+1} \right) \right \|_F^2 \\
=& \left \| \sum_{s = 0}^{t} \left(
n\G_{s} \W^{t-s+1} - \G_{s} 1 [1^\top(\W^{t+1} - \psi(t) 1^\top )^\top + n \psi(t)^\top] \right) \right \|_F^2 \\
=& \left \| \sum_{s = 0}^{t} \left(
n \G_{s} [\W^{t-s+1} - 1 \psi(t)^\top] - \G_{s} 1 1^\top [\W^{t+1} - 1 \psi(t){}^\top ] \right) \right \|_F^2 \\
\leq& \left ( \sum_{s = 0}^{t} \left \|
n \G_{s} [\W^{t-s+1} - 1 \psi(t)^\top] \right \|_F + \sum_{s = 0}^{t} \left \| \G_{s} 1 1^\top [\W^{t+1} - 1 \psi(t){}^\top ] \right \|_F \right)^2 \\
\leq& \left ( n \sum_{s = 0}^{t} \left \| \G_{s}\|_F \|[\W^{t-s+1} - 1 \psi(t)^\top] \right \|_F + \sum_{s = 0}^{t} \left \| \G_{s} \|_F \|1 1^\top\|_F \|[\W^{t+1} - 1 \psi(t){}^\top ] \right \|_F \right)^2 \\
\leq& n^2 \left ( \sum_{s = 0}^{t} \left \| \G_{s}\|_F \|[\W^{t-s+1} - 1 \psi(t)^\top] \right \|_F + \sum_{s = 0}^{t} \left \| \G_{s} \|_F \|[\W^{t+1} - 1 \psi(t){}^\top ] \right \|_F \right)^2 \\
\leq& n^2 \left ( \sum_{s = 0}^{t} n C q^{t-s+1} \| \G_{s} \|_F + \sum_{s = 0}^{t} n C q^{t+1} \| \G_{s} \|_F \right)^2 \\\\
\leq& 4 n^4 C^2q^2 \left ( \sum_{s = 0}^{t} q^{t-s} \| \G_{s} \|_F \right)^2
\end{align*}
where the third inequality is due to $\|1 1^\top\|_F = n$ and the fourth inequality is by Lemma \ref{lem:concentr} and the fact that $\|\mathbf{A}\|_F \leq n \cdot \max_{i,j} |A_{ij}|$ if $\mathbf{A}\in \mathbb{R}^{n\times n}$.
Therefore, if we combining all the above inequalities together, we can obtain
\begin{align*}
\sum_{i=1}^n \left\| \x^{(i)}_{t+1} - \overline{\z}_{t+1} \right \|_2^2 \leq \frac{4\gamma^2 C^2q^2 }{\delta^2_{\min}} \left ( \sum_{s = 0}^{t} q^{t-s} \| \G_{s} \|_F \right)^2.
\end{align*}
Using Lemma~\ref{lemma:seq}, we have
\begin{align*}
\sum_{t=0}^T\left ( \sum_{s = 0}^{t} q^{t-s} \| \G_{s} \|_F \right)^2 \leq \frac{1}{(1 - q)^2}\sum_{t=0}^T \| \G_{t} \|_F^2,
\end{align*}
which leads to
\begin{align*}
\sum_{t=0}^T \sum_{i=1}^n \left\| \x^{(i)}_{t+1} - \overline{\z}_{t+1} \right \|_2^2 \leq \frac{4\gamma^2 C^2q^2 }{\delta^2_{\min}(1-q)^2} \sum_{t=0}^T \| \G_{t} \|_F^2,
\end{align*}
which completes the proof.
\end{proof}
Actually, Theorem~\ref{theory:corollary2} is a corollary of Lemma~\ref{lem:mat} by setting $\gamma$ as the appropriate value.
\section{Extra Experiment Results}
\begin{figure*}[t]
\centering
\begin{subfigure}[b]{0.4\textwidth}
\centering
\includegraphics[width=0.8\textwidth]{2-RoomOccupancy-beta0.pdf}
\caption{stochastic=100\%\\ \hspace{0.5cm} {\hspace{0.5cm}\#Neighbors=2}}
\label{fig:2-Room-beta100}
\end{subfigure}
\begin{subfigure}[b]{0.4\textwidth}
\centering
\includegraphics[width=0.8\textwidth]{2-RoomOccupancy-beta5.pdf}
\caption{stochastic=50\%\\ \hspace{0.5cm} {\hspace{0.5cm}\#Neighbors=2}}
\label{fig:2-Room-beta50}
\end{subfigure}
\caption{Evaluation on the Network Sizes}
\label{fig:sizes_2}
\end{figure*}
\begin{figure*}[t]
\centering
\begin{subfigure}[b]{0.4\textwidth}
\centering
\includegraphics[width=0.8\textwidth]{3-RoomOccupancy-beta0.pdf}
\caption{stochastic=100\%}
\label{fig:3-Room-beta100}
\end{subfigure}
\begin{subfigure}[b]{0.4\textwidth}
\centering
\includegraphics[width=0.8\textwidth]{3-RoomOccupancy-beta5.pdf}
\caption{stochastic=50\%}
\label{fig:3-Room-beta50}
\end{subfigure}
\caption{Evaluation on the Network Density}
\label{fig:density_2}
\end{figure*}
\subsection{Evaluation on \textit{Room Occupancy} Dataset}
Due to the limitation of space, we only present the experiment results on \textit{SUSY} dataset in Section \ref{sec:network_size} and \ref{sec:network_density}. Related presents on \textit{Room Occupancy} is shown in Figure~\ref{fig:sizes_2} and Figure~\ref{fig:density_2}.
In Figure~\ref{fig:sizes_2}, we vary the number of clients in the network, from 6 to 20. In Figure~\ref{fig:density_2}, the network density is varied. All the results are consistent with the ones on \textit{SUSY}.
\subsection{Comparison with local online gradient descent}
To justify the necessity of communication, we also compare OPS with the local online gradient descent (local OGD), where every node trains a local model without communicating with
others. We run experiments in different ratios of the adversary and stochastic components based on settings in Figure 2. As we can see in Figure \ref{fig:baseline}, we empirically prove that communication does have benefits in reducing regret. Moreover, as the ratio of the stochastic components increased, the regret of OPS decreases further. This also empirically proves that the stochastic component can benefit from the communication while the adversarial component does not.
\begin{figure}[tb]
\centering
\begin{subfigure}[b]{0.4\textwidth}
\centering
\includegraphics[width=0.8\textwidth]{1-SUSY-beta100-N128-baseline.pdf}
\caption{stochastic=100\% \\ ($n=128$, \#Neighbors=32)}
\label{fig:3-SUSY-beta100}
\end{subfigure}
\begin{subfigure}[b]{0.4\textwidth}
\centering
\includegraphics[width=0.8\textwidth]{1-SUSY-beta50-N128-baseline.pdf}
\caption{stochastic=50\% \\($n=128$, \#Neighbors=32)}
\label{fig:3-SUSY-beta50}
\end{subfigure}
\caption{Comparison between OPS and Local OGD.}
\label{fig:baseline}
\end{figure}
\begin{thebibliography}{10}
\bibitem{konevcny2016federated}
Jakub Kone{\v{c}}n{\`y}, H~Brendan McMahan, Felix~X Yu, Peter Richt{\'a}rik,
Ananda~Theertha Suresh, and Dave Bacon.
\newblock Federated learning: Strategies for improving communication
efficiency.
\newblock {\em arXiv preprint arXiv:1610.05492}, 2016.
\bibitem{smith2017federated}
Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet~S Talwalkar.
\newblock Federated multi-task learning.
\newblock In {\em Advances in Neural Information Processing Systems}, pages
4424--4434, 2017.
\bibitem{yang2019federated}
Qiang Yang, Yang Liu, Tianjian Chen, and Yongxin Tong.
\newblock Federated machine learning: Concept and applications.
\newblock {\em ACM Transactions on Intelligent Systems and Technology (TIST)},
10(2):12, 2019.
\bibitem{shokri2015privacy}
Reza Shokri and Vitaly Shmatikov.
\newblock Privacy-preserving deep learning.
\newblock In {\em Proceedings of the 22nd ACM SIGSAC conference on computer and
communications security}, pages 1310--1321. ACM, 2015.
\bibitem{vanhaesebrouck2017decentralized}
Paul Vanhaesebrouck, Aur{\'e}lien Bellet, and Marc Tommasi.
\newblock Decentralized collaborative learning of personalized models over
networks.
\newblock In {\em International Conference on Artificial Intelligence and
Statistics (AISTATS)}, 2017.
\bibitem{bellet2018personalized}
Aur{\'e}lien Bellet, Rachid Guerraoui, Mahsa Taziki, and Marc Tommasi.
\newblock Personalized and private peer-to-peer machine learning.
\newblock In {\em International Conference on Artificial Intelligence and
Statistics}, pages 473--481, 2018.
\bibitem{aono2017privacy}
Yoshinori Aono, Takuya Hayashi, Lihua Wang, Shiho Moriai, et~al.
\newblock Privacy-preserving deep learning via additively homomorphic
encryption.
\newblock {\em IEEE Transactions on Information Forensics and Security},
13(5):1333--1345, 2017.
\bibitem{mcmahan_communication-efficient_2016}
H.~Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise
Agüera~y Arcas.
\newblock Communication-{Efficient} {Learning} of {Deep} {Networks} from
{Decentralized} {Data}.
\newblock {\em arXiv:1602.05629 [cs]}, February 2016.
\newblock arXiv: 1602.05629.
\bibitem{konecny_federated_2015}
Jakub Konečný, Brendan McMahan, and Daniel Ramage.
\newblock Federated {Optimization}:{Distributed} {Optimization} {Beyond} the
{Datacenter}.
\newblock {\em arXiv:1511.03575 [cs, math]}, November 2015.
\newblock arXiv: 1511.03575.
\bibitem{konecny_federated_2016}
Jakub Konečný, H.~Brendan McMahan, Felix~X. Yu, Peter Richtárik,
Ananda~Theertha Suresh, and Dave Bacon.
\newblock Federated {Learning}: {Strategies} for {Improving} {Communication}
{Efficiency}.
\newblock {\em arXiv:1610.05492 [cs]}, October 2016.
\newblock arXiv: 1610.05492.
\bibitem{mcmahan_google_2017}
Brendan McMahan and Daniel Ramage.
\newblock Google {AI} {Blog}: {Federated} {Learning}: {Collaborative} {Machine}
{Learning} without {Centralized} {Training} {Data}, April 2017.
\bibitem{smith_federated_2017}
Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet~S. Talwalkar.
\newblock Federated multi-task learning.
\newblock In {\em Advances in {Neural} {Information} {Processing} {Systems}},
pages 4424--4434, 2017.
\bibitem{caldas_federated_2018}
Sebastian Caldas, Virginia Smith, and Ameet Talwalkar.
\newblock Federated {Kernelized} {Multi}-{Task} {Learning}.
\newblock {\em The Conference on Systems and Machine Learning}, page~3, 2018.
\bibitem{shalev-shwartz_stochastic_2013}
Shai Shalev-Shwartz and Tong Zhang.
\newblock Stochastic dual coordinate ascent methods for regularized loss
minimization.
\newblock {\em Journal of Machine Learning Research}, 14(Feb):567--599, 2013.
\bibitem{yang_trading_2013}
Tianbao Yang.
\newblock Trading computation for communication: {Distributed} stochastic dual
coordinate ascent.
\newblock In {\em Advances in {Neural} {Information} {Processing} {Systems}},
pages 629--637, 2013.
\bibitem{yang_analysis_2013}
Tianbao Yang, Shenghuo Zhu, Rong Jin, and Yuanqing Lin.
\newblock Analysis of distributed stochastic dual coordinate ascent.
\newblock {\em arXiv preprint arXiv:1312.1031}, 2013.
\bibitem{jaggi_communication-efficient_2014}
Martin Jaggi, Virginia Smith, Martin Takác, Jonathan Terhorst, Sanjay
Krishnan, Thomas Hofmann, and Michael~I. Jordan.
\newblock Communication-efficient distributed dual coordinate ascent.
\newblock In {\em Advances in neural information processing systems}, pages
3068--3076, 2014.
\bibitem{ma_adding_2015}
Chenxin Ma, Virginia Smith, Martin Jaggi, Michael~I. Jordan, Peter Richtárik,
and Martin Takáč.
\newblock Adding vs. {Averaging} in {Distributed} {Primal}-{Dual}
{Optimization}.
\newblock {\em arXiv:1502.03508 [cs]}, February 2015.
\newblock arXiv: 1502.03508.
\bibitem{smith_cocoa:_2016}
Virginia Smith, Simone Forte, Chenxin Ma, Martin Takac, Michael~I. Jordan, and
Martin Jaggi.
\newblock {CoCoA}: {A} {General} {Framework} for {Communication}-{Efficient}
{Distributed} {Optimization}.
\newblock {\em arXiv:1611.02189 [cs]}, November 2016.
\newblock arXiv: 1611.02189.
\bibitem{stich_local_2018}
Sebastian~U. Stich.
\newblock Local {SGD} {Converges} {Fast} and {Communicates} {Little}.
\newblock September 2018.
\bibitem{wang_cooperative_2018}
Jianyu Wang and Gauri Joshi.
\newblock Cooperative {SGD}: {A} unified {Framework} for the {Design} and
{Analysis} of {Communication}-{Efficient} {SGD} {Algorithms}.
\newblock August 2018.
\bibitem{yu_parallel_2018}
Hao Yu, Sen Yang, and Shenghuo Zhu.
\newblock Parallel {Restarted} {SGD} with {Faster} {Convergence} and {Less}
{Communication}: {Demystifying} {Why} {Model} {Averaging} {Works} for {Deep}
{Learning}.
\newblock {\em arXiv:1807.06629 [cs, math]}, July 2018.
\newblock arXiv: 1807.06629.
\bibitem{lin_dont_2018}
Tao Lin, Sebastian~U. Stich, and Martin Jaggi.
\newblock Don't {Use} {Large} {Mini}-{Batches}, {Use} {Local} {SGD}.
\newblock {\em arXiv:1808.07217 [cs, stat]}, August 2018.
\newblock arXiv: 1808.07217.
\bibitem{lian2017can}
Xiangru Lian, Ce~Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji~Liu.
\newblock Can decentralized algorithms outperform centralized algorithms? a
case study for decentralized parallel stochastic gradient descent.
\newblock In {\em Advances in Neural Information Processing Systems}, pages
5330--5340, 2017.
\bibitem{he2018cola}
Lie He, An~Bian, and Martin Jaggi.
\newblock Cola: Decentralized linear learning.
\newblock In {\em Advances in Neural Information Processing Systems}, pages
4541--4551, 2018.
\bibitem{nedic2015distributed}
Angelia Nedi{\'c} and Alex Olshevsky.
\newblock Distributed optimization over time-varying directed graphs.
\newblock {\em IEEE Transactions on Automatic Control}, 60(3):601--615, 2015.
\bibitem{lian2017asynchronous}
Xiangru Lian, Wei Zhang, Ce~Zhang, and Ji~Liu.
\newblock Asynchronous decentralized parallel stochastic gradient descent.
\newblock In {\em International Conference on Machine Learning}, 2018.
\bibitem{yin_d}
Tianyu Wu, Kun Yuan, Qing Ling, Wotao Yin, and Ali H.~Sayed.
\newblock Decentralized consensus optimization with asynchrony and delays.
\newblock {\em IEEE Transactions on Signal and Information Processing over
Networks}, PP:1--1, 04 2017.
\bibitem{pmlr-v80-shen18a}
Zebang Shen, Aryan Mokhtari, Tengfei Zhou, Peilin Zhao, and Hui Qian.
\newblock Towards more efficient stochastic decentralized learning: Faster
convergence and sparse communication.
\newblock In Jennifer Dy and Andreas Krause, editors, {\em Proceedings of the
35th International Conference on Machine Learning}, volume~80 of {\em
Proceedings of Machine Learning Research}, pages 4624--4633,
Stockholmsm{\"a}ssan, Stockholm Sweden, 10--15 Jul 2018. PMLR.
\bibitem{NIPS2018_8028}
Youjie Li, Mingchao Yu, Songze Li, Salman Avestimehr, Nam~Sung Kim, and
Alexander Schwing.
\newblock Pipe-sgd: A decentralized pipelined sgd framework for distributed
deep net training.
\newblock In S.~Bengio, H.~Wallach, H.~Larochelle, K.~Grauman, N.~Cesa-Bianchi,
and R.~Garnett, editors, {\em Advances in Neural Information Processing
Systems 31}, pages 8056--8067. Curran Associates, Inc., 2018.
\bibitem{hazan2016introduction}
Elad Hazan et~al.
\newblock Introduction to online convex optimization.
\newblock {\em Foundations and Trends{\textregistered} in Optimization},
2(3-4):157--325, 2016.
\bibitem{shalev2012online}
Shai Shalev-Shwartz et~al.
\newblock Online learning and online convex optimization.
\newblock {\em Foundations and Trends{\textregistered} in Machine Learning},
4(2):107--194, 2012.
\bibitem{kamp2014communication}
Michael Kamp, Mario Boley, Daniel Keren, Assaf Schuster, and Izchak Sharfman.
\newblock Communication-efficient distributed online prediction by dynamic
model synchronization.
\newblock In {\em Joint European Conference on Machine Learning and Knowledge
Discovery in Databases}, pages 623--639. Springer, 2014.
\bibitem{shahrampour2017distributed}
Shahin Shahrampour and Ali Jadbabaie.
\newblock Distributed online optimization in dynamic environments using mirror
descent.
\newblock {\em IEEE Transactions on Automatic Control}, 63(3):714--725, 2017.
\bibitem{lee2016coordinate}
Soomin Lee, Angelia Nedi{\'c}, and Maxim Raginsky.
\newblock Coordinate dual averaging for decentralized online optimization with
nonseparable global objectives.
\newblock {\em IEEE Transactions on Control of Network Systems}, 5(1):34--44,
2016.
\bibitem{zhao2019decentralized}
Yawei Zhao, Chen Yu, Peilin Zhao, and Ji~Liu.
\newblock Decentralized online learning: Take benefits from others' data
without sharing your own to track global trend.
\newblock {\em arXiv preprint arXiv:1901.10593}, 2019.
\bibitem{duchi2011dual}
John~C Duchi, Alekh Agarwal, and Martin~J Wainwright.
\newblock Dual averaging for distributed optimization: Convergence analysis and
network scaling.
\newblock {\em IEEE Transactions on Automatic control}, 57(3):592--606, 2011.
\bibitem{gharesifard2010does}
Bahman Gharesifard and Jorge Cort{\'e}s.
\newblock When does a digraph admit a doubly stochastic adjacency matrix?
\newblock In {\em Proceedings of the 2010 American Control Conference}, pages
2440--2445. IEEE, 2010.
\bibitem{ram2010distributed}
S~Sundhar Ram, Angelia Nedi{\'c}, and Venugopal~V Veeravalli.
\newblock Distributed stochastic subgradient projection algorithms for convex
optimization.
\newblock {\em Journal of optimization theory and applications},
147(3):516--545, 2010.
\bibitem{tsianos2012distributed}
Konstantinos~I Tsianos and Michael~G Rabbat.
\newblock Distributed dual averaging for convex optimization under
communication delays.
\newblock In {\em 2012 American Control Conference (ACC)}, pages 1067--1072.
IEEE, 2012.
\bibitem{tsianos2012push}
Konstantinos~I Tsianos, Sean Lawlor, and Michael~G Rabbat.
\newblock Push-sum distributed dual averaging for convex optimization.
\newblock In {\em 2012 IEEE 51st IEEE Conference on Decision and Control
(CDC)}, pages 5453--5458. IEEE, 2012.
\bibitem{bellet2017personalized}
Aur{\'e}lien Bellet, Rachid Guerraoui, Mahsa Taziki, and Marc Tommasi.
\newblock Personalized and private peer-to-peer machine learning.
\newblock {\em arXiv preprint arXiv:1705.08435}, 2017.
\bibitem{nasr2018comprehensive}
Milad Nasr, Reza Shokri, and Amir Houmansadr.
\newblock Comprehensive privacy analysis of deep learning: Stand-alone and
federated learning under passive and active white-box inference attacks.
\newblock {\em arXiv preprint arXiv:1812.00910}, 2018.
\bibitem{chaoyanghe2020fedml}
Chaoyang He, Songze Li, Jinhyun So, Mi~Zhang, Hongyi Wang, Xiaoyang Wang,
Praneeth Vepakomma, Abhishek Singh, Hang Qiu, Li~Shen, Peilin Zhao, Yan Kang,
Yang Liu, Ramesh Raskar, Qiang Yang, Murali Annavaram, and Salman Avestimehr.
\newblock Fedml: A research library and benchmark for federated machine
learning.
\newblock {\em arXiv preprint arXiv:2007.13518}, 2020.
\bibitem{nedic2014distributed}
Angelia Nedi{\'c} and Alex Olshevsky.
\newblock Distributed optimization over time-varying directed graphs.
\newblock {\em IEEE Transactions on Automatic Control}, 60(3):601--615, 2014.
\bibitem{nedic2016stochastic}
Angelia Nedi{\'c} and Alex Olshevsky.
\newblock Stochastic gradient-push for strongly convex functions on
time-varying directed graphs.
\newblock {\em IEEE Transactions on Automatic Control}, 61(12):3936--3947,
2016.
\bibitem{assran2018asynchronous}
Mahmoud Assran and Michael Rabbat.
\newblock Asynchronous subgradient-push.
\newblock {\em arXiv preprint arXiv:1803.08950}, 2018.
\bibitem{assran2018stochastic}
Mahmoud Assran, Nicolas Loizou, Nicolas Ballas, and Michael Rabbat.
\newblock Stochastic gradient push for distributed deep learning.
\newblock {\em arXiv preprint arXiv:1811.10792}, 2018.
\end{thebibliography}