Asynchronous Federated Learning with non-convex client objective functions and heterogeneous dataset
Extracted source text · arXiv 2508.01675
%\documentclass[journal,twoside,web]{ieeecolor}
%[journal,twoside,web]{ieeecolor}
%\usepackage{lcsys}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\documentclass[lettersize,journal]{IEEEtran}
\documentclass[journal]{IEEEtai}
\usepackage[colorlinks,urlcolor=blue,linkcolor=blue,citecolor=blue]{hyperref}
\usepackage{color,array}
%\usepackage{enumitem}
\usepackage{bbm}
% << color >>
\usepackage[table]{xcolor}
\definecolor{cyan}{rgb}{0.0,0.6,0.9}
\definecolor{darkred}{rgb}{0.6,0.0,0.0}
\definecolor{darkgreen}{rgb}{0.0,0.5,0.0}
\definecolor{darkblue}{rgb}{0.0,0.0,0.5}
\definecolor{notered}{HTML}{d62728}
\definecolor{noteblue}{HTML}{1f77b4}
\definecolor{notegreen}{HTML}{2ca02c}
\definecolor{noteorange}{HTML}{ff7f0e}
% << annotate >>
\usepackage{soul}
\newenvironment{notecomment}{%
\color{noteblue}%
\fontfamily{ptm}\selectfont
%\fontsize{10pt}{10pt}\selectfont
%\comment%
}{%
%\endcomment
}
\newenvironment{detailcomment}{%
\color{gray}%
\fontfamily{ptm}\selectfont
\comment%
}{%
\endcomment
}
% << color box >>
\usepackage{mdframed}
\usepackage{tcolorbox}
% << table >>
\usepackage{multirow} % set multiple rows
\usepackage{booktabs} % professional-quality tables
\usepackage{threeparttable}
\usepackage{longtable}
% << enumitem >>
%\usepackage{enumitem}
% << algorithm >>
%\usepackage{algorithm}
%\usepackage{algorithmic}
%\usepackage{algpseudocode}
%\usepackage[ruled,vlined,linesnumbered]{algorithm2e}
\let\oldnl\nl% Store \nl in \oldnl
\newcommand{\nonl}{\renewcommand{\nl}{\let\nl\oldnl}}% Remove line number for one line
% << toc of appendix >>
\usepackage{titletoc}
% << hyperref >>
%\usepackage{hyperref}
%\hypersetup{
% colorlinks=true,
% linkcolor={darkred},
% citecolor={darkgreen}
%}
% << graph >>
\usepackage{graphicx}
\usepackage{wrapfig}
\usepackage{subcaption}
% << tikz >>
%\usepackage{tikz}
%% << bib references >>
\usepackage{algorithm}
\usepackage{algorithmicx}
\usepackage{algpseudocode}
\usepackage{caption}
\input{math_macros}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\usepackage{orcidlink}
% << tikz >>
%\usepackage{tikz}
%% << bib references >>
%\usepackage[round, compress]{natbib}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%% custom macros %%%%%%%%%%%%%%%%%
\newcommand{\fl}{\texttt{FL}}
\newcommand{\afl}{\texttt{AFL}}
\newcommand{\pfl}{\texttt{PFL}}
\newcommand{\sfl}{\texttt{SFL}}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{document}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\title{Asynchronous Federated Learning with non-convex client objective functions and heterogeneous dataset}
\author{Ali Forootani, \IEEEmembership{Senior Member, IEEE}, Raffaele Iervolino, \IEEEmembership{Senior Member, IEEE.}
\thanks{Ali Forootani is with Helmholtz Center for Environmental Research-UFZ, Permoserstrasse 15, 04318 Leipzig, Germany (\texttt{aliforootani@ieee.org/ali.forootani@ufz.de}).}
\thanks{Raffaele Iervolino is with Department of Electrical Engineering and Information Technology, University of Naples, 80125 Napoli, Italy
(\texttt{rafierv@unina.it}).}
}
%\markboth{Journal of \LaTeX\ Class Files,~Vol.~18, No.~9, September~2020}%
%{How to Use the IEEEtran \LaTeX \ Templates}
\maketitle
\begin{abstract}
Federated Learning is a distributed machine learning paradigm that enables model training across decentralized devices holding local data, thereby preserving data privacy and reducing the need for centralization. Despite its advantages, traditional FL faces challenges such as communication overhead, system heterogeneity, and straggler effects. Asynchronous Federated Learning has emerged as a promising solution, allowing clients to send updates independently, which mitigates synchronization issues and enhances scalability. This paper extends the Asynchronous Federated Learning framework to scenarios involving clients with non-convex objective functions and heterogeneous dataset, which are prevalent in modern machine learning models like deep neural networks. We provide a rigorous convergence analysis for this setting, deriving bounds on the expected gradient norm and examining the impacts of staleness, variance, and heterogeneity. To address the challenges posed by asynchronous updates, we introduce a staleness-aware aggregation mechanism that penalizes outdated updates, ensuring fresher data has a more significant influence on the global model. Additionally, we propose a dynamic learning rate schedule that adapts to client staleness and heterogeneity, improving stability and convergence.
Our approach effectively manages heterogeneous environments, accommodating differences in client computational capabilities, data distributions, and communication delays, making it suitable for real-world Federated Learning applications. We also analyze the effects of client selection methods—specifically, choosing clients with or without replacement—on variance and model convergence, providing insights for more effective sampling strategies. The practical implementation of our methods using \texttt{PyTorch} and \texttt{Python}'s asyncio library demonstrates their applicability in real-world asynchronous and heterogeneous FL scenarios. Empirical experiments validate the proposed methods, showing improved performance and scalability in handling asynchronous updates, and non-convex client's objective function with associated heterogeneous dataset.
\end{abstract}
\begin{IEEEImpStatement}
Asynchronous Federated Learning addresses key challenges in distributed learning, yet its application to non-convex optimization remains underexplored. This work extends Asynchronous Federated Learning to non-convex client objectives and heterogeneous datasets, providing a rigorous convergence analysis and quantifying the effects of staleness and variance. We introduce a staleness-aware aggregation mechanism and a dynamic learning rate schedule, improving stability and convergence. Empirical results on MNIST and CIFAR-10 demonstrate superior scalability and robustness over synchronous Federated Learning. Implemented using \texttt{PyTorch} and ``asyncio'', our approach enhances real-world applicability in privacy-sensitive domains like healthcare and finance. This work significantly advances Asynchronous Federated Learning's theoretical foundations and practical deployment, enabling more efficient, decentralized learning.
\end{IEEEImpStatement}
\begin{IEEEkeywords}
Federated Learning, Stochastic Gradient Descent, Client Drifts, Asynchronous Federated Learning.
\end{IEEEkeywords}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{comment}
\begin{frontmatter}
\title{Asynchronous Federated Learning with non-convex client objective functions and heterogeneous dataset}
%\author{
%Ali Forootani, \IEEEmembership{Senior Member, IEEE}
%\thanks{To be written (e-mail: aliforootani@ieee.org).}
%\thanks{To be written (e-mail: author@boulder.nist.gov).}
%\thanks{To be written (e-mail: author@lamar.colostate.edu).}
%\thanks{To be written (e-mail: author@nrim.go.jp).}
%}
%\maketitle
\author[UFZ]{Ali Forootani*, Senior Member, IEEE \orcidlink{0000-0001-7612-4016}} \ead{ali.forootani@ufz.de-aliforootani@ieee.org}
\address[UFZ]{Helmholtz Centre for Environmental Research - UFZ, Permoserstraße 15, 04318 Leipzig, Germany}
\begin{abstract}
\end{abstract}
%\begin{IEEEkeywords}
%Federated Learning,
%\end{IEEEkeywords}
\begin{keyword}
Federated Learning, Stochastic Gradient Descent, Client Drifts, Asynchronous Federated Learning.
\end{keyword}
\end{frontmatter}
\end{comment}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Introduction}
\IEEEPARstart{F}{e}derated Learning (FL) is a distributed machine learning paradigm that enables the training of models across multiple decentralized devices or servers holding local data samples, without requiring the exchange of the data itself \cite{yu2019parallel, kairouz2021advances}. This approach addresses privacy concerns and reduces the need for extensive data centralization \cite{mcmahan2017communication}. FL has become increasingly relevant with the growing ubiquity of edge devices and the rising importance of data privacy regulations \cite{bonawitz2019towards, mothukuri2021survey}. Its applications span various domains, including healthcare \cite{xu2021federated}, finance \cite{long2020federated}, renewable energies \cite{forootani2024asynchronous}, and mobile computing \cite{yu2021toward}.
Despite its potential, traditional FL faces significant challenges. These include communication overhead, system heterogeneity, and straggler effects \cite{li2020federated, wang2019adaptive}. Communication overhead arises due to frequent exchanges of model parameters between clients and the central server. System heterogeneity reflects the diversity in computation power, network connectivity, and local data distribution across devices \cite{sattler2020sparse, pang2020realizing}. The straggler effect occurs when slower clients delay the global model aggregation process, impeding efficiency \cite{harlap2016addressing, cipar2013solving, chai2020fedat}.
Asynchronous Federated Learning (\afl) has emerged as a promising solution to mitigate these challenges \cite{chen2020asynchronous}. Unlike synchronous FL, where all participating devices must complete their local training before model aggregation, \afl~allows clients to send updates to the central server independently and at different times \cite{hsieh2020non}. This flexibility reduces waiting times, improves scalability, and accommodates devices with varying computational capabilities. Furthermore, \afl~inherently addresses the straggler problem by not requiring synchronization across all clients, leading to faster convergence and more efficient resource utilization \cite{xie2020asynchronous}.
\subsection{Literature Review}
Federated Learning has garnered significant attention since its inception. Federated Averaging (FedAvg) algorithm is introduced in \cite{mcmahan2017communication}, which became a cornerstone in FL research. They demonstrated the feasibility of FL in training deep learning models while preserving user privacy. However, subsequent studies highlighted its limitations. For example in \cite{bonawitz2019towards} a secure aggregation techniques to enhance privacy in FL has been explored, while the work reported in \cite{karimireddy2020scaffold} proposed adaptive optimizers to tackle the challenges posed by non-IID (non-Independent and Identically Distributed) data distributions across clients.
Addressing communication bottlenecks has been another focus of research. For example in \cite{sattler2020sparse} the sparse updates and compression techniques has been introduced to reduce the volume of data exchanged between clients and the server.
In another line of research the use of personalization layers in FL to accommodate diverse client data distributions without excessive communication was discussed \cite{li2020federated}.
The concept of asynchronous FL was first explored in \cite{hsieh2020non}, who proposed FedAsync, an \afl~approach that allows updates from clients to be aggregated as they arrive. This method showed promise in mitigating the straggler problem and improving convergence speed. Subsequent works, such as \cite{xie2020asynchronous}, investigated the robustness of \afl~under varying client participation rates and proposed strategies to balance stale updates.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
While \afl~addresses many challenges, it introduces new complexities, such as handling stale updates and ensuring fairness in model aggregation. Several works have advanced the field of asynchronous decentralized and parallel stochastic gradient descent (SGD), addressing challenges such as gradient staleness, convergence, and scalability. For example AD-PSGD was introduced in \cite{lian2017asynchronous, wang2021adaptive}, which removes the central server bottleneck while maintaining optimal convergence rates despite staleness. Adaptive frameworks like MindTheStep-AsyncPSGD adjusts step sizes to mitigate the impact of stale gradients \cite{backstrom2022mindthestep}, while instance-based Adaptiveness to Staleness in Asynchronous SGD has been considered to improve performance \cite{backstrom2022asap}.
Theoretical analyses, such as those reported in \cite{dutta2018slow}, highlight error-runtime trade-offs and scenarios where stale gradients can be advantageous \cite{dutta2018slow}. Techniques like delay compensation effectively address delays in asynchronous SGD \cite{zheng2017delay}, while the parallelization limits under gradient staleness has been explored in \cite{zhang2022parallelization}. Acceleration methods have also been integrated into asynchronous SGD to enhance convergence rates \cite{async2022accelerated}. Studies reported in \cite{zhou2018unbounded}, investigated the tolerable limits of unbounded delays in distributed optimization, while in \cite{lian2015nonconvex}, the properties of asynchronous SGD in non-convex settings has been investigated. In \cite{keuper2015asynchronous}, an asynchronous communication algorithm to address stale gradients in parallel updates was proposed \cite{keuper2015asynchronous}.
To address the challenges posed by stragglers in synchronous federated learning (SFL) and improve its efficiency, several \afl~schemes have been proposed \cite{chen2020asynchronous,gu2021privacy,AsyncFed,chen2018lag}. \cite{wu2020safa} introduced the Semi-Asynchronous Federated Learning Algorithm (SAFA), which classifies clients into three categories—sustainable, moderate, and unsustainable—allowing only sustainable clients to operate asynchronously, thereby reducing delays and enhancing system efficiency.
In \cite{FedBuff} a buffering mechanism has been proposed where the server aggregates local updates once a predefined buffer size is reached, balancing asynchronous updates with consistency in global model performance. Meanwhile, \cite{FedSA} developed a two-stage training process that assigns each client a staleness parameter during the convergence phase, ensuring updates arrive synchronously despite varying processing speeds, thereby maintaining robust convergence in asynchronous settings. These strategies collectively mitigate the straggler problem while preserving the effectiveness and scalability of federated learning.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Our previous work \cite{forootani2024asynchronous}, focused on \afl~with convex assumption on local cost functions. In this work, we extend this approach to clients with non-convex objective functions. This extension is significant since non-convex optimization problems are prevalent in machine learning, especially in training deep neural networks. Recent literature has also explored \afl~with non-convex objectives. For instance, in \cite{xie2019asynchronous} an asynchronous federated optimization algorithm was proposed and its convergence for both strongly convex and a restricted family of non-convex problems have been proved. Another relevant work has studied improved convergence rates for asynchronous stochastic gradient descent in non-convex settings \cite{koloskova2022sharper}.
This body of literature underscores the evolution of federated learning from its synchronous roots to asynchronous implementations. \afl~represents a crucial step forward in addressing the inherent limitations of traditional FL, particularly in scenarios involving diverse and large-scale distributed systems. However, further research is needed to refine its algorithms, enhance robustness, and validate its efficacy across diverse real-world applications.
\subsection{Contributions}
\begin{itemize}
\item This paper extends the theoretical and practical framework of asynchronous federated learning to \textit{non-convex objective functions}, addressing the challenges of multiple local minima and saddle points common in modern machine learning models.
\item We provide a rigorous \textit{convergence analysis} for non-convex asynchronous federated learning, deriving bounds on the expected gradient norm and analyzing the impact of key factors such as \textit{staleness}, \textit{variance}, and \textit{heterogeneity}.
\item We introduce and formalize the concept of \textit{drift} due to staleness in asynchronous updates, quantifying its impact on the global model and providing mathematical bounds to mitigate its effects.
\item We propose \textit{staleness-aware aggregation mechanism}, penalizing updates based on their staleness to ensure that fresher updates contribute more significantly to the global model.
\item We introduce a \textit{dynamic learning rate schedule} that adapts to client staleness and heterogeneity, improving stability and convergence in asynchronous federated learning.
\item Our approach effectively handles \textit{heterogeneous environments}, addressing differences in client computation capabilities, data distributions, and communication delays, making it suitable for real-world federated learning applications.
\item We analyze the effect of \textit{choosing clients with replacement} vs. \textit{choosing clients without replacement} on variance and model convergence, providing insights for more effective sampling in federated learning.
\end{itemize}
We emphasize a practical implementation using \texttt{PyTorch} and Python's \texttt{asyncio} library, demonstrating real-world applicability for asynchronous and heterogeneous federated learning scenarios. We validate the proposed methods through \textit{empirical experiments}, demonstrating improved performance and scalability in handling asynchronous updates and non-convex objectives.
This paper is organized as follows: in Section \ref{preliminaries} preliminaries are discussed. Problem formulation and an overview of the proposed \afl~Algorithm is given in Section \ref{problem_formulation}. In section \ref{converegence_analysis} we will provide the convergence analysis of the \afl~Algorithm with non-convex client objective function and heterogeneous dataset. We evaluate the \afl~Algorithm and compare its results with synchronous FL on well known dataset such as MNIST and CIFAR-10 in Section \ref{simulation}. The paper ends with Conclusion in Section \ref{conclusion}.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Preliminaries}\label{preliminaries}
In this section, we establish the necessary notation, definitions, and assumptions required for analyzing the \afl~framework. We begin by introducing the mathematical notation used throughout this paper, followed by a discussion of different sampling methods and their statistical properties. Next, we present key theoretical results related to variance bounds and martingale properties, which form the foundation for our analysis. Finally, we introduce key assumptions that characterize the behavior of local objective functions, stochastic gradients, and model drift in the presence of stale updates.
\subsection{Notation}
Let $\norm{\cdot}$ denote the standard Euclidean norm, applicable to both vectors and matrices. The parameter $\sigma^2$ provides an upper bound on the variance of the stochastic gradients for individual clients participating in the \afl~process. The total number of clients is represented by $C$, with the index $c$ referring to a specific client, and $J$ indicating the subset of clients actively engaged in training during a particular round. Training proceeds over $\mathcal{J}$ global communication rounds, indexed by $j$, and each client performs $I$ local update steps per round, indexed by $i$. The learning rate or step size is denoted by $\lambda$, and the effective learning rate in the \afl~framework is defined as $\tilde\lambda = \lambda C I$.
A permutation of the client indices is represented by $\psi$, defined as $\{\psi_1, \psi_2, \ldots, \psi_C\}$, which maps the set $\{1, 2, \ldots, C\}$ to a reordered sequence. The global objective function is denoted by $\mathcal{L}$, while the local objective function for client $c$ is expressed as $\mathcal{L}_c$. The global model parameters at the $j$-th communication round are denoted by $\theta^{(j)}$, and the local model parameters of client $c$ after $i$ local update steps within the $j$-th round are represented by $\theta_{c,i}^{(j)}$. The stochastic gradients corresponding to the local objective function $\mathcal{L}_{\psi_c}$ of client $\psi_c$, evaluated at $\theta_{c,i}^{(j)}$, are denoted by $\rvq_{\psi_c,i}^{(j)}$, where $\rvq_{\psi_c,i}^{(j)} \coloneqq \nabla l_{\psi_c}(\theta_{c,i}^{(j)}; \xi)$, with $\xi$ representing a stochastic sampling variable.
\subsection{Reviews on some probability and stochastic concepts}
We analyze the properties of the sample mean \( \overline{\theta}_\psi \) under two common sampling approaches: (i) sampling with replacement and (ii) sampling without replacement. First, the population mean \( \overline{\theta} \) and variance \( \nu^2 \) are defined as:
\[
\overline{\theta} \coloneqq \frac{1}{m} \sum_{k=1}^m \theta_k, \quad \nu^2 \coloneqq \frac{1}{m} \sum_{k=1}^m \norm{\theta_k - \overline{\theta}}^2,
\]
\noindent where \( \theta_1, \theta_2, \dots, \theta_m \) represent the fixed vectors in the population. Given a sample of \( s \) vectors, denoted as \( \rvx_{\psi_1}, \rvx_{\psi_2}, \dots, \rvx_{\psi_s} \), drawn from this population, the sample mean is expressed as:
\[
\overline{\theta}_\psi = \frac{1}{s} \sum_{p=1}^s \theta_{\psi_p}.
\]
Below, we evaluate the expected value and variance of \( \overline{\theta}_\psi \) under the two sampling methods:
\paragraph{Sampling with Replacement}
(i) Expected Value : The expected value of the sample mean equals the population mean \cite{forootani_afl_emt}:
\[
\E[\overline{\rvx}_\psi] = \overline{\theta}.
\]
(ii) Variance: The variance of the sample mean is given by:
\[
\E\left[\norm{\overline{\rvx}_\psi - \overline{\theta}}^2\right] = \frac{\nu^2}{s}.
\]
This result shows that the variance decreases inversely with the sample size \( s \) but is independent of the total population size \( m \).
\paragraph{Sampling without Replacement}
(i) Expected Value: The expected value of the sample mean remains equal to the population mean \cite{forootani_afl_emt}: \( \E[\overline{\theta}_\psi] = \overline{\theta}.\)
(ii) Variance: The variance in this case is: \(
\E\left[\norm{\overline{\rvx}_\psi - \overline{\theta}}^2\right] = \frac{m - s}{s (m - 1)} \nu^2.
\)
Here, the variance depends on both the sample size \( s \) and the total population size \( m \), and it is typically smaller than the variance in the case of sampling with replacement.
In summary, while the expected value of the sample mean is identical for both sampling methods, the variance differs, with sampling without replacement usually leading to a smaller variance.
Consider a sequence of random variables \(\{\epsilon_i\}_{i=1}^m\) and associated random vectors \(\{\theta_i\}_{i=1}^m\), where each \(\theta_i \in \mathbb{R}^d\) depends on the history \(\epsilon_1, \epsilon_2, \ldots, \epsilon_i\). Assume the conditional expectation satisfies:
\[
\mathbb{E}_{\epsilon_i}[\theta_i \mid \epsilon_1, \ldots, \epsilon_{i-1}] = \rve_i,
\]
\noindent which implies that \(\{\theta_i - \rve_i\}_{i=1}^m\) forms a martingale difference sequence with respect to the filtration generated by \(\{\epsilon_i\}_{i=1}^m\). Additionally, assume that the conditional variance of each \(\theta_i - \rve_i\) is uniformly bounded:
\[
\mathbb{E}_{\epsilon_i}[\|\theta_i - \rve_i\|^2 \mid \epsilon_1, \ldots, \epsilon_{i-1}] \leq \delta^2,
\]
\noindent for some constant \(\delta > 0\). Then, the total variance of the sum of the martingale difference sequence satisfies:
\begin{equation}\label{martingle}
\mathbb{E}\left[\left\|\sum_{k=1}^m (\theta_k - \rve_k)\right\|^2\right] = \sum_{k=1}^m \mathbb{E}\left[\|\theta_k - \rve_k\|^2\right] \leq m\delta^2.
\end{equation}
This inequality demonstrates that the variance of the sum is bounded by \( m\delta^2 \), where \( m \) is the number of terms in the sequence and \( \delta^2 \) is the uniform variance bound for each term.
Using the approach of sampling clients without replacement, define \cite{karimireddy2020scaffold, forootani_afl_emt}:
\[
p_{c,i}(k) =
\begin{cases}
I-1, & \text{if } k \leq c-1, \\
i-1, & \text{if } k = c,
\end{cases}
\]
\noindent where \( I \) is a fixed integer. For \( J \leq C \) and \( C \geq 2 \), the following inequality holds \cite{karimireddy2020scaffold, forootani_afl_emt}:
\[
\sum_{c=1}^J \sum_{i=0}^{I-1} \mathbb{E} \left\|\sum_{k=1}^c \sum_{j=0}^{p_{c,i}(k)} (\theta_{\psi_k} - \overline{\theta}) \right\|^2 \leq \frac{1}{2} J^2 I^3 \nu^2,
\]
\noindent where \( \overline{\theta} \) is the population mean, and \( \nu^2 \) is the population variance.
%This inequality highlights the dependence of the total variance on the parameters \( J \), \( I \), and \( \nu^2 \).
\subsection{Definitions and main assumptions}
In this subsection we provide definitions, and main assumptions that are required for the convergence proof of the \afl~algorithm.
\begin{definition}
In \afl~, each client updates its local model based on a potentially stale global model. This means that client updates might not align with the current global model, leading to a drift between the models at each client and the global model. Let’s define this drift mathematically:
- At time \( t \), the global model is \( \theta^{(t)} \), and client \( i \) uses a stale model \( \theta^{(t-\tau_i)} \) (with \( \tau_i \) being the staleness of client \( i \)'s model).
- The drift for client \( i \) at time \( t \) can be expressed as:
\[
\Delta \theta_i^{\text{drift}} = \theta^{(t)} - \theta^{(t-\tau_i)}.
\]
\end{definition}
\paragraph{Impact of Drift on Updates} This drift can affect the quality of local updates. The local gradient computed by client \( i \) is based on a stale model \( \theta^{(t-\tau_i)} \), and the direction of the gradient will be influenced by the mismatch between \( \theta^{(t)} \) and \( \theta^{(t-\tau_i)} \).
\begin{assumption}\label{l_smooth}
Each local objective \( \mathcal{L}_i(\theta) \) is \( L \)-smooth:
\[
\mathcal{L}_i(\theta_2) \leq \mathcal{L}_i(\theta_1) + \langle \nabla \mathcal{L}_i(\theta_1), \theta_2 - \theta_1 \rangle + \frac{L}{2} \|\theta_2 - \theta_1\|^2.
\]
\end{assumption}
\begin{assumption}\label{bounded_variance}
The variance of the stochastic gradients is bounded:
\[
\mathbb{E}\left[\|\nabla \mathcal{L}_i(\theta) - \nabla \mathcal{L}(\theta)\|^2\right] \leq \sigma^2.
\]
\end{assumption}
\begin{assumption}\label{max_delay}
The staleness \( \tau_i \) is bounded by \( \tau_{\max} \):
\[
\tau_i \leq \tau_{\max}.
\]
\end{assumption}
\begin{assumption}\label{bounded_drift}
The drift between the local and global models is bounded:
\[
\|\Delta \theta_i^{\text{drift}}\| \leq \Delta \theta_{\text{max}},
\]
where \( \Delta \theta_{\text{max}} \) represents the maximum allowable drift between the global model and the local model due to staleness.
\end{assumption}
\begin{assumption}
Local updates are unbiased estimates of the true gradient:
\[
\mathbb{E}[\nabla \mathcal{L}_i(\theta)] = \nabla \mathcal{L}_i(\theta).
\]
\end{assumption}
\begin{assumption}
Clients are randomly selected at each round, and each client computes the update using the local model \( \theta^{(t-\tau_i)} \).
\end{assumption}
\begin{assumption}\label{asm:heterogeneity:optima}
We assume that, at the global minimizer \( \theta^\ast \), the gradients of the individual local loss functions are closely aligned. Specifically, there exists a constant \( \beta_\ast^2 \) such that:
\[
\frac{1}{C} \sum_{c=1}^C \norm{\nabla \mathcal{L}_c(\theta^\ast)}^2 = \beta_\ast^2,
\]
where \( \beta_\ast \) measures the degree of similarity among the gradients at \( \theta^\ast \), the solution to the global loss \( \mathcal{L}(\theta) \).
\end{assumption}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Problem Formulation}\label{problem_formulation}
The objective of the \fl~framework is to optimize a global loss function, defined as:
\[
\min_{\theta \in \mathbb{R}^d} \left\{ \mathcal{L}(\theta) \coloneqq \frac{1}{C} \sum_{c=1}^{C} \mathcal{L}_c(\theta) \right\},
\]
where \( \mathcal{L}_c(\theta) \coloneqq \mathbb{E}_{\xi \sim \mathcal{D}_c} [\ell_c(\theta; \xi)] \) represents the local objective function for client \( c \). Here, \( \ell_c(\theta; \xi) \) denotes the client-specific loss function evaluated at model parameters \( \theta \) for data point \( \xi \), \( \mathcal{D}_c \) is the data distribution of client \( c \), and \( C \) is the total number of clients.
If \( \mathcal{D}_c \) consists of a finite dataset, \( \mathcal{D}_c = \{\xi_c^j \mid j = 1, 2, \dots, |\mathcal{D}_c|\} \), the local objective function can be expressed equivalently as:
\[
\mathcal{L}_c(\theta) = \frac{1}{|\mathcal{D}_c|} \sum_{j=1}^{|\mathcal{D}_c|} \ell_c(\theta; \xi_c^j).
\]
In federated learning, clients operate independently and often experience asynchronous delays in their updates. The delay for client \( c \) is denoted as \( \tau_c \), representing the time lag between the computation of the local gradient by the client and the incorporation of the gradient into the global model.
During each training round \( t \), a subset of clients \( J^{(t)} = \{c_1, c_2, \dots, c_k\} \subseteq \{1, 2, \dots, C\} \) is selected at random without replacement to participate in training. For each client \( c \in J^{(t)} \), the process is as follows:
\begin{enumerate}
\item Model Initialization: The client initializes its local model parameters using the global parameters available at the time, \( \theta^{(t - \tau_c)} \), where \( \tau_c \) accounts for the update delay.
\item Local Training: The client performs \( I \) iterations of stochastic gradient descent (SGD) on its local dataset, producing intermediate parameter states \( \theta_{c,i}^{(t)} \) for \( i = 0, 1, \dots, I \).
\item Parameter Transmission: After completing local updates, the client sends its final parameters, \( \theta_{c,I}^{(t)} \), to the central server.
\end{enumerate}
The local parameter updates at each iteration \( i \) are computed as:
\[
\theta_{c,i+1}^{(t)} = \theta_{c,i}^{(t)} - \eta \rvq_{c,i}^{(j)},
\]
where \( \eta > 0 \) is the learning rate, and \( \rvq_{c,i}^{(j)}= \nabla \ell_{c}(\theta_{c,i}^{(t)}; \xi) \) is the stochastic gradient of the local loss with respect to \( \theta_{c,i}^{(t)} \), computed using a randomly sampled data point \( \xi \sim \mathcal{D}_c \). The initialization for local training is given by:
\[
\theta_{c,0}^{(t)} = \theta^{(t - \tau_c)}.
\]
Once all participating clients \( c \in J^{(t)} \) complete their local training and transmit their updated parameters to the central server, the global model is updated using an aggregation function:
\[
\theta^{(t+1)} = \text{Aggregate}\left(\{\theta_{c,I}^{(t)} \mid c \in {J}^{(t)}\}\right),
\]
where the aggregation function may involve techniques such as weighted averaging based on the size of local datasets or other strategies to mitigate the effects of heterogeneity across clients.
This iterative process is repeated over multiple rounds, progressively refining the global model parameters \( \theta \) based on asynchronous updates from the distributed clients. The federated learning framework thus enables collaborative training while preserving data privacy and accommodating communication and computation constraints at the client level. Algorithm \ref{afl_non_convex_alg} provides an overview of the proposed \afl.
\begin{algorithm}
\caption{Federated Learning with Asynchronous Client Updates}
\label{afl_non_convex_alg}
\begin{algorithmic}[1]
\State \textbf{Input:} Learning rate \( \eta \), number of local iterations \( I \), number of communication rounds \( \mathcal{J} \), convergence threshold \( \kappa \).
\State \textbf{Initialize:} Global model \( \theta^{(0)} \).
\For{$t = 0, 1, \dots, \mathcal{J}-1$}
\State Randomly select a subset of clients \( J^{(t)} \subseteq \{1, 2, \dots, C\} \).
\For{each client \( c \in J^{(t)} \) \textbf{in parallel}}
\State Initialize local model using the delayed global model:
\[
\theta_{c,0}^{(t)} = \theta^{(t - \tau_c)},
\]
where \( \tau_c \) is the staleness of client \( c \).
\For{$i = 0, 1, \dots, I-1$}
\State Perform local update using stochastic gradient descent:
\[
\theta_{c,i+1}^{(t)} = \theta_{c,i}^{(t)} - \eta \nabla \ell_c(\theta_{c,i}^{(t)}; \xi),
\]
where \( \xi \sim \mathcal{D}_c \) is a randomly sampled data point.
\EndFor
\State Send updated parameters \( \theta_{c,I}^{(t)} \) and staleness \( \tau_c \) to the server.
\EndFor
\State Aggregate updates to compute the new global model:
\[
\theta^{(t+1)} = \text{Aggregate}\left(\{\theta_{c,I}^{(t)} \mid c \in J^{(t)}\}\right).
\]
\If{$\|\theta^{(t+1)} - \theta^{(t)}\| \leq \kappa$}
\State \textbf{Break.}
\EndIf
\EndFor
\State \textbf{Output:} Final global model \( \theta^{(\mathcal{J})} \).
\end{algorithmic}
\end{algorithm}
\section{Convergence Analysis}\label{converegence_analysis}
In this section, we establish theoretical guarantees on the expected improvement in the loss function across training rounds. Specifically, we derive a recursive bound that characterizes the evolution of the model parameters and provides insights into how step size, variance, and system heterogeneity impact convergence. We first present a key recursion bound in Lemma~\ref{lem:non-convex:recursion}, which quantifies the expected descent in the loss function over a single training round. This result is further complemented by Lemma~\ref{lem:non-convex:drift}, where we analyze the drift in model updates due to the decentralized nature of the optimization process. These results collectively provide a rigorous foundation for understanding the convergence properties of our algorithm in the presence of non-convex objectives.
\begin{lemma}[Non-Convex Recursion Bound]\label{lem:non-convex:recursion}
Under Assumptions~\ref{l_smooth}, \ref{bounded_variance}, and \ref{asm:heterogeneity:optima}, if the step size \( \gamma \) satisfies
\[
\gamma \leq \frac{1}{6LJI(1 + \beta^2/J)},
\]
then the following holds for the expected improvement in the loss function \( \mathcal{L} \) after one training round:
\begin{align*}
\E\left[\mathcal{L}(\theta^{(j+1)}) - \mathcal{L}(\theta^{(j)})\right] &\leq -\frac{1}{6}JI\gamma \E\left[\|\nabla \mathcal{L}(\theta^{(j)})\|^2\right] \\
& + 2L\gamma^2JI \sigma^2 \\
& + 2L\gamma^2J^2I^2\frac{C-J}{J(C-1)}\nu^2 \\
& + \frac{5}{6}L^2\gamma \sum_{c=1}^J \sum_{i=0}^{I-1} \E\left[\|\theta_{c,i}^{(j)} - \theta^{(j)}\|^2\right].
\end{align*}
\end{lemma}
\begin{proof}
We focus on a single training round and omit the superscript \( j \) for simplicity. All expectations are conditioned on \( \theta^{(j)} \), unless otherwise stated. Since \( \mathcal{L} \) is \( L \)-smooth (Assumption~\ref{l_smooth}), we begin with:
\[
\E\left[\mathcal{L}(\theta + \Delta \theta) - \mathcal{L}(\theta)\right] \leq \E\left[\langle \nabla \mathcal{L}(\theta), \Delta \theta \rangle\right] + \frac{L}{2} \E\|\Delta \theta\|^2.
\]
We first bound \( \E\left[\langle \nabla \mathcal{L}(\theta), \Delta \theta \rangle\right] \). Substitute the overall update \( \Delta \theta \) into the inner product term:
\begin{multline}
\E\left[ \langle \nabla \mathcal{L}(\theta), \Delta \theta \rangle\right] = -\gamma J I \E\Bigg[\Big\langle \nabla \mathcal{L}(\theta)\\, \frac{1}{J} \sum_{c=1}^J \frac{1}{I} \sum_{i=0}^{I-1} \big(\nabla \mathcal{L}_{\pi_c}(\theta_{c,i}) - \nabla \mathcal{L}_{\pi_c}(\theta) + \nabla \mathcal{L}_{\pi_c}(\theta)\big)\Big\rangle\Bigg].
\end{multline}
\noindent Simplifying the expression:
\begin{multline*}
\E\left[\langle \nabla \mathcal{L}(\theta), \Delta \theta \rangle\right] = -\gamma JI \|\nabla \mathcal{L}(\theta)\|^2 \\ - \gamma JI \E\Big[\Big\langle \nabla \mathcal{L}(\theta), \frac{1}{J} \sum_{c=1}^J \frac{1}{I} \sum_{i=0}^{I-1} \Big(\nabla \mathcal{L}_{\pi_c}(\theta_{c,i}) - \nabla \mathcal{L}_{\pi_c}(\theta)\Big)\Big\rangle\Big].
\end{multline*}
\noindent Using the inequality \( |\langle a, b \rangle| \leq \frac{1}{2} \|a\|^2 + \frac{1}{2} \|b\|^2 \), we obtain:
\begin{multline*}
\E\left[\langle \nabla \mathcal{L}(\theta), \Delta \theta \rangle\right] \leq -\gamma JI \|\nabla \mathcal{L}(\theta)\|^2 + \frac{1}{2} \gamma JI \|\nabla \mathcal{L}(\theta)\|^2 \\ + \frac{1}{2} \gamma \sum_{c=1}^J \sum_{i=0}^{I-1} \E\|\nabla \mathcal{L}_{\pi_c}(\theta_{c,i}) - \nabla \mathcal{L}_{\pi_c}(\theta)\|^2.
\end{multline*}
Using the smoothness assumption (Assumption~ \eqref{l_smooth}):
\begin{multline}
\E\left[\langle \nabla \mathcal{L}(\theta), \Delta \theta \rangle\right] \leq -\frac{1}{2} \gamma JI \|\nabla \mathcal{L}(\theta)\|^2 \\+ \frac{1}{2} L^2 \gamma \sum_{c=1}^J \sum_{i=0}^{I-1} \E\|\theta_{c,i} - \theta\|^2.
\end{multline}
Next, we bound \( \E\|\Delta \theta\|^2 \) as follows:
\begin{multline}\label{eq:recursion-non-convex-1}
\frac{1}{2} L \E\|\Delta \theta\|^2 \leq 2L\gamma^2 \E\left\|\sum_{c=1}^J \sum_{i=0}^{I-1} \left(\rvg_{\pi_c,i} - \nabla \mathcal{L}_{\pi_c}(\theta_{c,i})\right)\right\|^2 \\+ 2L\gamma^2 \E\left\|\sum_{c=1}^J \sum_{i=0}^{I-1} \left(\nabla \mathcal{L}_{\pi_c}(\theta_{c,i}) - \nabla \mathcal{L}_{\pi_c}(\theta)\right)\right\|^2
\\+ 2L\gamma^2 \E\left\|\sum_{c=1}^J \sum_{i=0}^{I-1} \left(\nabla \mathcal{L}_{\pi_c}(\theta) - \nabla \mathcal{L}(\theta)\right)\right\|^2 \\+ 2L\gamma^2 \E\left\|\sum_{c=1}^J \sum_{i=0}^{I-1} \nabla \mathcal{L}(\theta)\right\|^2.
\end{multline}
\noindent We now bound each term on the right-hand side \eqref{eq:recursion-non-convex-1}, for the first term we have
\[
2L\gamma^2 \E\left\|\sum_{c=1}^J \sum_{i=0}^{I-1} \left(\rvg_{\pi_c,i} - \nabla \mathcal{L}_{\pi_c}(\theta_{c,i})\right)\right\|^2 \leq 2L\gamma^2 JI \sigma^2. \tag{see \eqref{martingle}, Assumption~\ref{bounded_variance}}
\]
\noindent For the second term we write from Assumption \ref{l_smooth}:
\begin{multline*}
2L\gamma^2 \E\left\|\sum_{c=1}^J \sum_{i=0}^{I-1} \left(\nabla \mathcal{L}_{\pi_c}(\theta_{c,i}) - \nabla \mathcal{L}_{\pi_c}(\theta)\right)\right\|^2 \\ \leq 2L^3 \gamma^2 JI \sum_{c=1}^J \sum_{i=0}^{I-1} \E\|\theta_{c,i} - \theta\|^2.
\end{multline*}
\noindent For the third and forth term we get:
\begin{multline}
2L\gamma^2 \E\left\|\sum_{c=1}^J \sum_{i=0}^{I-1} \left(\nabla \mathcal{L}_{\pi_c}(\theta) - \nabla \mathcal{L}(\theta)\right)\right\|^2 \\+ 2L\gamma^2 \E\left\|\sum_{c=1}^J \sum_{i=0}^{I-1} \nabla \mathcal{L}(\theta)\right\|^2 \leq 2L\gamma^2 J^2I^2 \tfrac{C-J}{J(C-1)}\nu^2 \\ + 2L\gamma^2 J^2I^2 \left(1 + \tfrac{C-J}{J(C-1)}\right) \|\nabla \mathcal{L}(\theta)\|^2.
\end{multline}
\noindent Substituting the above bounds and applying \( \gamma \leq \frac{1}{6LJI (1 + \beta^2/J)} \), we recover the superscripts and obtain:
\begin{multline}
\E\left[\mathcal{L}(\theta^{(j+1)}) - \mathcal{L}(\theta^{(j)})\right] \leq -\frac{1}{6} JI\gamma \E\|\nabla \mathcal{L}(\theta^{(j)})\|^2 \\+ 2L\gamma^2 JI \sigma^2 + 2L\gamma^2 J^2I^2 \tfrac{C-J}{J(C-1)} \nu^2 \\ + \frac{5}{6} L^2 \gamma \sum_{c=1}^J \sum_{i=0}^{I-1} \E\|\theta_{c,i}^{(j)} - \theta^{(j)}\|^2.
\end{multline}
\end{proof}
\begin{lemma}\label{lem:non-convex:drift}
Suppose Assumptions~\ref{l_smooth}, \ref{bounded_variance}, \ref{asm:heterogeneity:optima} are satisfied. If $\gamma \leq \frac{1}{6LIJ(1+\beta^2/J)}$, then
\begin{align*}
E_j &\leq \frac{9}{4} J^2 I^2 \gamma^2 \nu^2 + \frac{9}{4} J^2 I^3 \gamma^2 \mu^2 \\ &+ \frac{9}{4} \left( \frac{\beta^2}{J} + 1 \right) J^3 I^3 \gamma^2 \mathbb{E} \left[ \left\| \nabla \mathcal{L}(\theta^{(j)}) \right\|^2 \right].
\end{align*}
\end{lemma}
\begin{proof}
According to Algorithm~\ref{afl_non_convex_alg}, the overall updates of \afl~from $\theta^{(j)}$ to $\theta_{c,i}^{(j)}$ are
\begin{align*}
\theta_{c,i}^{(j)} - \theta^{(j)} = -\gamma \sum_{k=1}^{c} \sum_{f=0}^{p_{c,i}(k)} \rvq_{\psi_k,f}^{(j)},\\ \quad p_{c,i}(k) =
\begin{cases}
I-1, & \text{if } k \leq c-1, \\
i-1, & \text{if } k = c.
\end{cases}
\end{align*}
\noindent For the sake of simplicity we consider only one training round and we drop our dependency to superscripts $j$. Unless otherwise stated, the expectation is conditioned on $\theta^{(j)}$.
\noindent We aim to bound $\mathbb{E} \left[ \| \theta_{c,i} - \theta \|^2 \right]$, which is given by
\begin{multline} \label{eq:drift-non-convex-1}
\mathbb{E} \left[ \| \theta_{c,i} - \theta \|^2 \right] \\ \leq 4 \gamma^2 \mathbb{E} \left[ \left\| \sum_{k=1}^{c} \sum_{f=0}^{p_{c,i}(k)} \left( \rvq_{\psi_k,f} - \nabla \mathcal{L}_{\psi_k} (\theta_{k,f}) \right) \right\|^2 \right] \\
+ 4 \gamma^2 \mathbb{E} \left[ \left\| \sum_{k=1}^{c} \sum_{f=0}^{p_{c,i}(k)} \left( \nabla \mathcal{L}_{\psi_k} (\theta_{k,f}) - \nabla \mathcal{L}_{\psi_k} (\theta) \right) \right\|^2 \right] \\
+ 4 \gamma^2 \mathbb{E} \left[ \left\| \sum_{k=1}^{c} \sum_{f=0}^{p_{c,i}(k)} \left( \nabla \mathcal{L}_{\psi_k} (\theta) - \nabla \mathcal{L} (\theta) \right) \right\|^2 \right] \\
+ 4 \gamma^2 \mathbb{E} \left[ \left\| \sum_{k=1}^{c} \sum_{f=0}^{p_{c,i}(k)} \nabla \mathcal{L} (\theta) \right\|^2 \right].
\end{multline}
Next, we bound each term in Eq.~\eqref{eq:drift-non-convex-1}, for the first term we have:
\begin{multline}
4 \gamma^2 \mathbb{E} \left[ \left\| \sum_{k=1}^{c} \sum_{f=0}^{p_{c,i}(k)} \left( \rvq_{\psi_k,f} - \nabla \mathcal{L}_{\psi_k} (\theta_{k,f}) \right) \right\|^2 \right] \\ \leq 4 \gamma^2 \sum_{k=1}^c \sum_{f=0}^{p_{c,i}(k)} \mathbb{E} \left[ \| \rvq_{\psi_k,f} - \nabla \mathcal{L}_{\psi_k} (\theta_{k,f}) \|^2 \right] \leq 4 \gamma^2 \gY_{c,i} \nu^2,
\end{multline}
\noindent For the second term in Eq. \eqref{eq:drift-non-convex-1} we have:
\begin{multline}
4 \gamma^2 \mathbb{E} \left[ \left\| \sum_{k=1}^{c} \sum_{f=0}^{p_{c,i}(k)} \left( \nabla \mathcal{L}_{\psi_k} (\theta_{k,f}) - \nabla \mathcal{L}_{\psi_k} (\theta) \right) \right\|^2 \right]\\ \leq 4 \gamma^2 \gY_{c,i} \sum_{k=1}^c \sum_{f=0}^{p_{c,i}(k)} \mathbb{E} \left[ \| \nabla \mathcal{L}_{\psi_k} (\theta_{k,f}) - \nabla \mathcal{L}_{\psi_k} (\theta) \|^2 \right] \\
\leq 4 L^2 \gamma^2 \gY_{c,i} \sum_{k=1}^c \sum_{f=0}^{p_{c,i}(k)} \mathbb{E} \left[ \| \theta_{k,f} - \theta \|^2 \right],
\end{multline}
\noindent For the third term in Eq. \eqref{eq:drift-non-convex-1} we can write:
\begin{multline} \label{eq:drift-non-convex-2}
4 \gamma^2 \mathbb{E} \left[ \left\| \sum_{k=1}^{c} \sum_{f=0}^{p_{c,i}(k)} \left( \nabla \mathcal{L}_{\psi_k} (\theta) - \nabla \mathcal{L} (\theta) \right) \right\|^2 \right] \\ \leq 4 \gamma^2 \gY_{c,i} \mathbb{E} \left[ \| \nabla \mathcal{L}_{\psi_k} (\theta) - \nabla \mathcal{L} (\theta) \|^2 \right],
\end{multline}
and finally for the fourth term in Eq. \eqref{eq:drift-non-convex-1} we have:
\begin{equation}
4 \gamma^2 \mathbb{E} \left[ \left\| \sum_{k=1}^{c} \sum_{f=0}^{p_{c,i}(k)} \nabla \mathcal{L} (\theta) \right\|^2 \right] \leq 4 \gamma^2 \gY_{c,i}^2 \| \nabla \mathcal{L} (\theta) \|^2.
\end{equation}
\noindent Suppose we denote $\gY_{c,i} = (c-1)I + i$, then by replacing into the Eq.~\eqref{eq:drift-non-convex-1}, we have:
\begin{align*}
E_j &\leq 4 \gamma^2 \nu^2 \sum_{c=1}^J \sum_{i=0}^{I-1} \gY_{c,i} \\ &+ 4 L^2 \gamma^2 \sum_{c=1}^J \sum_{i=0}^{I-1} \gY_{c,i} \sum_{k=1}^c \sum_{f=0}^{p_{c,i}(k)} \mathbb{E} \left[ \| \theta_{k,f} - \theta \|^2 \right] \\
& + 4 \gamma^2 \sum_{c=1}^J \sum_{i=0}^{I-1} \gY_{c,i} + 4 \gamma^2 \sum_{c=1}^J \sum_{i=0}^{I-1} \gY_{c,i}^2 \| \nabla \mathcal{L} (\theta) \|^2.
\end{align*}
\noindent Applying the bounds $\sum_{c=1}^J \sum_{i=0}^{I-1} \gY_{c,i} \leq \frac{1}{2} J^2 I^2$ and $\sum_{c=1}^J \sum_{i=0}^{I-1} \gY_{c,i}^2 \leq \frac{1}{3} J^3 I^3$, we have:
\begin{align*}
E_j &\leq 2 J^2 I^2 \gamma^2 \sigma^2 + 2 L^2 J^2 I^2 \gamma^2 E_j + 2 J^2 I^3 \gamma^2 \nu^2 \\
&\quad + 2 \left( \frac{\beta^2}{J} + 1 \right) J^3 I^3 \gamma^2 \| \nabla \mathcal{L} (\theta) \|^2.
\end{align*}
\noindent Manipulating and forcing the relation $\gamma \leq \frac{1}{6 L I J (1 + \beta^2 / J)}$, we reach the result:
\begin{multline}
E_j \leq \frac{9}{4} J^2 I^2 \gamma^2 \sigma^2 + \frac{9}{4} J^2 I^3 \gamma^2 \nu^2 \\ + \frac{9}{4} \left( \frac{\beta^2}{J} + 1 \right) J^3 I^3 \gamma^2 \| \nabla \mathcal{L} (\theta) \|^2.
\end{multline}
\noindent The claim follows after recovering the superscripts and taking unconditional expectations.
\end{proof}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{comment}
\section{AFL algorithm}
**Algorithm: Asynchronous Federated Learning for Stable Diffusion Models**
1. **Initialization**:
Initialize the global model \( \theta^{(0)} \). Set hyperparameters such as learning rates \( \eta \), staleness penalty \( \lambda \), maximum staleness \( \tau_{\max} \), and number of communication rounds \( T \).
2. **Client Model Updates**:
- At each client \( i \), receive a stale model \( \theta^{(t-\tau_i)} \).
- Perform local training using the local dataset \( \mathcal{D}_i \) for \( E \) epochs:
\[
\theta_i^{(t)} \leftarrow \theta^{(t-\tau_i)} - \eta \nabla \mathcal{L}_i(\theta^{(t-\tau_i)}),
\]
where \( \nabla \mathcal{L}_i(\theta^{(t-\tau_i)}) \) is the gradient of the local loss function.
3. **Staleness-Aware Aggregation**:
- Each client sends \( \theta_i^{(t)} \) and \( \tau_i \) (its staleness) to the server.
- Server aggregates the updates with staleness weights:
\[
\theta^{(t+1)} = \theta^{(t)} - \frac{\eta}{n} \sum_{i=1}^n w_i \nabla \mathcal{L}_i(\theta^{(t-\tau_i)}),
\]
where \( w_i = \frac{1}{1 + \lambda \tau_i} \) penalizes updates with higher staleness.
4. **Convergence Check**:
- Continue until \( t = T \) or \( \|\theta^{(t+1)} - \theta^{(t)}\| \leq \epsilon \), where \( \epsilon \) is a pre-defined convergence threshold.
\end{comment}
\subsection{Tuning the learning rate}
In the next Lemma we drive a bound on the delay aware learning rate for the \afl~algorithm.
\begin{lemma}\label{learning_rate_tune}
We consider two non-negative sequences \(\{d_t\}_{t \geq 0}\), \(\{g_t\}_{t \geq 0}\), which satisfy the relation
\begin{align}
d_{t+1} \leq d_t - b\zeta_t g_t + a_1 \zeta_t^2 + a_2 \zeta_t^3, \label{eq1:lem:dynamic_learning_rate_updated}
\end{align}
for all \(t \geq 0\), where \(b > 0\), \(a_1, a_2 \geq 0\), and \(\zeta_t\) is the learning rate defined as
\[
\zeta_t = \frac{\zeta_0}{\sqrt{t+1} \cdot (1 + \alpha \cdot \tau_t)}.
\]
Then, using \(\pi_t = 1\) and \(\Pi_T \coloneqq \sum_{t=0}^T \pi_t = T+1\), we have:
\begin{multline*}
\Psi_T \coloneqq \frac{1}{\Pi_T} \sum_{t=0}^T g_t \pi_t \leq \frac{d_0}{b\zeta_0(T+1)} \left(\sqrt{T+1} + \alpha \sum_{t=0}^T \tau_t\right) \\ + \frac{a_1\zeta_0}{b} \cdot \frac{\log(T+1)}{\sqrt{T+1}} + \frac{a_2\zeta_0^2}{b} \cdot \frac{\log(T+1)}{T}.
\end{multline*}
\end{lemma}
\begin{proof}
Rearranging Eq.~\eqref{eq1:lem:dynamic_learning_rate_updated} and multiplying both sides by \(\pi_t = 1\), we get:
\[
b g_t \leq \frac{d_t}{\zeta_t} - \frac{d_{t+1}}{\zeta_t} + a_1 \zeta_t + a_2 \zeta_t^2.
\]
Summing over \(t=0\) to \(T\) results in:
\[
b \sum_{t=0}^T g_t \leq \sum_{t=0}^T \frac{d_t - d_{t+1}}{\zeta_t} + a_1 \sum_{t=0}^T \zeta_t + a_2 \sum_{t=0}^T \zeta_t^2.
\]
The first term forms a telescoping sum:
\[
\sum_{t=0}^T \frac{d_t - d_{t+1}}{\zeta_t} = \frac{d_0}{\zeta_0} \left(\sqrt{T+1} + \alpha \sum_{t=0}^T \tau_t \right).
\]
For the second term, substituting \(\zeta_t = \frac{\zeta_0}{\sqrt{t+1} \cdot (1 + \alpha \cdot \tau_t)}\), we use bounds:
\[
\sum_{t=0}^T \zeta_t \leq \zeta_0 \sum_{t=0}^T \frac{1}{\sqrt{t+1}} \leq \zeta_0 (2\sqrt{T+1} - 1).
\]
Similarly, for \(\zeta_t^2\):
\[
\sum_{t=0}^T \zeta_t^2 \leq \zeta_0^2 \sum_{t=0}^T \frac{1}{t+1} \leq \zeta_0^2 \log(T+1).
\]
Dividing by \(\Pi_T = T+1\), we have:
\begin{multline*}
\Psi_T \leq \frac{d_0}{b\zeta_0(T+1)} \left(\sqrt{T+1} + \alpha \sum_{t=0}^T \tau_t \right) \\ + \frac{a_1\zeta_0}{b} \cdot \frac{\log(T+1)}{\sqrt{T+1}} + \frac{a_2\zeta_0^2}{b} \cdot \frac{\log(T+1)}{T}.
\end{multline*}
The first term scales as \( O\left(\frac{1}{\sqrt{T}}\right) \) when \( \tau_t = 0 \) (homogeneous case). The second and third terms decay at \( O\left(\frac{\log(T)}{\sqrt{T}}\right) \) and \( O\left(\frac{\log(T)}{T}\right) \), respectively, showing that higher-order terms become negligible for large \(T\). Thus, the result reveals that the average gradient norm \( \Psi_T \) decreases asymptotically as \( O(1/\sqrt{T}) \), assuming a decaying learning rate schedule.
\end{proof}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\begin{theorem}\label{conv_theorem}
Assume that all the local objective functions are \(L\)-smooth (Assumption~\ref{l_smooth}) and non-convex. For the \afl~algorithm (Algorithm~\ref{afl_non_convex_alg}), there exists a constant effective learning rate \(\tilde{\gamma} = \gamma CI\) and weights \(\{\pi_j\}_{j \geq 0}\) such that the weighted average of the global parameters
\[
\bar{\theta}^{(\mathcal{J})} = \frac{\sum_{j=0}^{\mathcal{J}} \pi_j \theta^{(j)}}{\sum_{j=0}^\mathcal{J} \pi_j},
\]
Under Assumptions~\ref{bounded_variance} (bounded variance) and \ref{asm:heterogeneity:optima} (bounded heterogeneity), if \(\tilde{\gamma} \leq \frac{1}{6L(1 + \beta^2 / C)}\) and \(\pi_j = 1\), then satisfies the following bound:
\begin{multline*}
\E\left[\|\nabla \mathcal{L}(\theta^{(j)})\|^2\right]
\leq \frac{6 (\mathcal{L}(\theta^{(0)}) - \mathcal{L}^\ast) }{\tilde{\eta} \mathcal{J}} \\ + \frac{12L\tilde{\gamma}\sigma^2}{CI} + \frac{45L^2\tilde{\gamma}^2\sigma^2}{4CI} + \frac{45L^2\tilde{\gamma}^2\nu^2}{4C}.
\end{multline*}
\end{theorem}
\begin{proof}
From Lemmas~\ref{lem:non-convex:recursion} and~\ref{lem:non-convex:drift}, and assuming \(\gamma \leq \frac{1}{6LIJ(1+\beta^2/J)}\), the recursion simplifies to (see the Appendix \ref{appendix} for details):
\begin{align}\label{equation_appendix}
\E \left[\mathcal{L}(\theta^{(j+1)}) - \mathcal{L}(\theta^{(j)})\right]
&\leq -\frac{1}{6}IJ\gamma \E\left[\|\nabla \mathcal{L}(\theta^{(j)})\|^2\right]
\\&+ 2L\gamma^2 IJ\sigma^2
+ 2L\gamma^2 J^2 I^2 \frac{C-J}{J(C-1)} \nu^2 \\
& + \frac{15}{8}L^2\gamma^3 J^2 I^2 \sigma^2
+ \frac{15}{8}L^2\gamma^3 J^2 I^3 \nu^2.
\end{align}
\noindent Introducing the substitution \(\tilde{\gamma} \coloneqq IJ\gamma\), we rewrite the recursion after subtracting \(\mathcal{L}^\ast\) from both sides:
\begin{align*}
\E \left[\mathcal{L}(\theta^{(j+1)}) - \mathcal{L}^\ast\right]
&\leq \E\left[\mathcal{L}(\theta^{(j)}) - \mathcal{L}^\ast\right]
\\ &- \frac{\tilde{\gamma}}{6} \E\left[\|\nabla \mathcal{L}(\theta^{(j)})\|^2\right] \\
& + \frac{2L\tilde{\gamma}^2 \sigma^2}{IJ}
+ 2L\tilde{\gamma}^2 \frac{C-J}{J(C-1)} \nu^2
\\& + \frac{15}{8}\frac{L^2\tilde{\gamma}^3 \sigma^2}{IJ}
+ \frac{15}{8}\frac{L^2\tilde{\gamma}^3 \nu^2}{J}.
\end{align*}
Now, applying Lemma~\ref{learning_rate_tune}, we define the following parameters: \(t = j\), \(T = \mathcal{J}\), \(\zeta = \tilde{\gamma}\), \(d_t = \E\left[\mathcal{L}(\theta^{(j)}) - \mathcal{L}^\ast\right]\), \(b = \frac{1}{10}\), \(g_t = \E\left[\|\nabla \mathcal{L}(\theta^{(j)})\|^2\right]\), \(\pi_t = 1\), \(a_1 = \frac{2L\sigma^2}{IJ} + 2L\frac{C-J}{J(C-1)} \nu^2\), and \(a_2 = \frac{15}{8}\frac{L^2 \sigma^2}{IJ} + \frac{15}{8}\frac{L^2 \nu^2}{J}\). Using these substitutions, we derive:
\begin{align}
\E\left[\|\nabla \mathcal{L}(\theta^{(j)})\|^2\right]
&\leq \frac{6\left(\mathcal{L}(\theta^{0}) - \mathcal{L}^\ast\right)}{\tilde{\gamma} \mathcal{J}}
+ \frac{12L\tilde{\gamma} \sigma^2}{IJ}\nonumber \\ &
+ \frac{12L\tilde{\gamma} \nu^2 (C-J)}{J(C-1)} \nonumber \\
&\quad + \frac{45L^2 \tilde{\gamma}^2 \sigma^2}{4IJ}
+ \frac{45L^2 \tilde{\gamma}^2 \nu^2}{4J}. \label{eq:thm:non-convex:bound}
\end{align}
When \(J = C\), the result in the theorem statement follows. Specifically:
1. The first term, \(O(1/\mathcal{J})\), dominates the bound as the number of iterations increases.
2. The second term, \(O(\gamma)\), reflects variance (\(\sigma^2\)) and diminishes as \(\gamma\) decreases.
3. The remaining terms, \(O(\gamma^2)\), arise from both variance and heterogeneity (\(\nu^2\)), and scale quadratically with \(\gamma\).
Choosing a learning rate of \(\gamma \sim O(1/\sqrt{\mathcal{J}})\) ensures the expected gradient norm satisfies:
\[
\E\left[\|\nabla \mathcal{L}(\theta^{(j)})\|^2\right] \sim O\left(\frac{1}{\sqrt{\mathcal{J}}}\right),
\]
which is consistent with the established convergence rate for non-convex optimization.
\end{proof}
\section{Simulations}\label{simulation}
In this section we consider \afl~algorithm to tackle challenges related to heterogeneous client environments and the optimization of non-convex objective functions. We evaluate the performance of our \afl~algorithm using well-known benchmark datasets, namely MNIST and CIFAR, with non-convex optimization objectives. Additionally, we compare the results of the \afl~algorithm with those of a synchronous federated learning (FL) approach.
\subsection{Hardware Setup}
Our deep learning model training was conducted on high-performance hardware to meet the computational demands of wind power simulations. Specifically, we utilized an \texttt{NVIDIA A100 GPU} with 32 \texttt{GB} of Video Random Access Memory (\texttt{VRAM})\footnote{\texttt{VRAM} refers to the dedicated memory on a \texttt{GPU}, optimized for high-speed graphics processing, parallel computations, and deep learning tasks. Unlike standard \texttt{RAM}, which is used by the \texttt{CPU}, \texttt{VRAM} efficiently handles large-scale matrix operations, simulations, and neural network training.}. Additionally, each \texttt{CPU} core was allocated 256 \texttt{GB} of memory to facilitate large-scale data processing.
The training process was managed using the Simple Linux Utility for Resource Management (\texttt{SLURM}) job scheduler\footnote{\texttt{SLURM} is a widely used open-source workload manager for high-performance computing (\texttt{HPC}) environments. Developed by the Lawrence Livermore National Laboratory, it efficiently allocates resources such as \texttt{CPU}s, \texttt{GPU}s, and memory across computing nodes, enabling optimal job scheduling and execution.}. Job-specific constraints were applied to ensure compatibility with the \texttt{A100 GPU}'s extensive memory capacity, optimizing parallel processing and maintaining model training stability.
\subsection{Code and data availability statement.}
The \afl~algorithm is implemented using the \texttt{PyTorch} framework, taking advantage of its advanced tools for constructing and training deep learning models. It is openly available to enable reproducibility and facilitate further research at this \texttt{Github} repository \footnote{\url{https://github.com/Ali-Forootani/AFL_non_convex_non_iid}}. The code for data extraction, interpolation, statistical analysis, visualization, DNN~training and evaluation is provided in the associated repository. It includes detailed comments and instructions for reproducing the results.
Additionally, a version will be archived on \texttt{Zenodo} \footnote{\url{https://zenodo.org/records/14962410}} for reference.
\subsection{Classification task on MNIST dataset}
The dataset used in this experiment is the MNIST dataset, which contains $60,000$ training images and $10,000$ test images of handwritten digits. To simulate realistic federated learning conditions, the data was partitioned among clients in a non-IID manner using a Dirichlet distribution with a concentration parameter \(\alpha = 0.5\). A lower value of \(\alpha\) creates more uneven distributions, meaning some clients may have a disproportionate number of samples for specific classes. Each image is normalized and transformed into a PyTorch tensor for model input.
\paragraph{Training setup} The model used for both clients and the server is a Convolutional Neural Network (CNN). The network architecture consists of three convolutional layers, each followed by a ReLU activation function. The convolutional layers use $32$ filters with a kernel size of \(3 \times 3\), a stride of $1$, and padding of $1$, ensuring the spatial dimensions remain consistent across layers. The weights of the convolutional layers are initialized using Xavier uniform initialization to improve convergence. Following the convolutional layers is an adaptive average pooling layer, which reduces the spatial dimensions to \(1 \times 1\). A fully connected layer maps the resulting features to $10$ output classes, corresponding to the digits $0$ through $9$. The final output layer applies a log-softmax function, making the model suitable for multi-class classification.
The federated learning procedure follows the standard server-client communication protocol. At the start of each communication round, a subset of clients is selected randomly to participate. Each selected client trains its local model on its dataset for a specified number of epochs before sending the updated model weights back to the server. The server aggregates the client models using a weighted average, where the weight corresponds to the size of the client's local dataset. This aggregated model is then broadcast back to the clients for the next round of training.
The learning rate at each client is adjusted dynamically based on the delay using the formula \(\gamma_t = \frac{\gamma_0}{\sqrt{t + 1} \cdot (1 + \alpha \cdot d_t)}\), where \(\gamma_0\) is the initial learning rate, \(t\) is the epoch number, \(\alpha\) is the delay scaling factor, and \(d_t\) is the delay time. This ensures that clients with longer delays make smaller updates, preventing them from destabilizing the global model. An early stopping mechanism is implemented to terminate training if the client's loss does not improve for $10$ consecutive epochs, reducing unnecessary computation.
\texttt{PyTorch} is used for model definition and training, while the \texttt{asyncio} and \texttt{nest\_asyncio} packages enabled asynchronous client updates. The experiment is configured with $C=10$ clients, each training for $I=10$ local epochs per communication round, with a batch size of $64$. The total number of communication rounds is set to $\mathcal{J}=1000$, and in each round. The initial learning rate is set to \(\gamma_0 = 10^{-3}\), and the delay is computed dynamically in the code.
The loss function used for training is negative log-likelihood loss (\texttt{NLLLoss}), appropriate for multi-class classification tasks. The Adam optimizer \cite{Adam} is chosen for its adaptive learning rate capabilities, which complement the delay-aware adjustments.
\paragraph{Results}
In the first experiment $J=5$ clients are randomly selected to participate in the training. The global server model demonstrated consistent improvement across the communication rounds as shown in figure \ref{afl_mnist_1}. The server loss decreased steadily, with diminishing returns toward the later rounds, indicating convergence. This trend suggests that the server is able to generalize well despite the heterogeneous data distributions and asynchronous training times of the clients.
The non-IID data distribution can introduce additional challenges, as clients with highly imbalanced datasets required more rounds to contribute effectively to the global model. Despite this, the model aggregation strategy is able to handle the variability in client updates, resulting in a robust and generalized server model.
Figure~\ref{mnist_client_selection} presents a histogram depicting the frequency of client selection in the federated learning process. The results indicate that some clients, such as client 2, have been selected more frequently than others. Despite imbalance participation of the clients in the training loop, the global convergence of the server is not affected significantly as shown in \ref{afl_mnist_1}.
Figure \ref{mnist_clients_participation}, showing the server loss in the \afl~algorithm for different MNIST experiments, illustrates the impact of varying client participation on the training process. Different curves correspond to experiments where the number of clients participating per round ranges from 3 to 8.
From the plot \ref{mnist_clients_participation}, it is evident that increasing the number of clients per round leads to a faster reduction in server loss, indicating a more rapid convergence of the global model. As the number of clients per round increases, the frequency of model updates also rises, allowing the server to aggregate more information from diverse clients. This leads to faster improvement of the global model, especially in the early rounds of training.
However, after a certain point, the improvement in the server loss starts to diminish, as shown by the flattening of the curves. This indicates the onset of convergence, where further model updates yield smaller improvements. The results also suggest that, despite the asynchronous nature of the client updates, the server can still benefit from increased client participation.
The variations in the curves highlight the balance between the speed of model convergence and the communication cost, as adding more clients per round can improve the server model more quickly but also increase the computational burden on the server to aggregate the larger number of updates. This trade-off is an important consideration when designing federated learning systems, especially in scenarios with high client variability, like the one presented with the MNIST dataset in your experiment.
Figure~\ref{mnist_server_loss_comparison} compares the training loss of asynchronous and synchronous federated learning (FL) on the MNIST dataset. Both approaches show a decreasing loss trend, confirming effective model training. However, synchronous FL exhibits a smoother loss reduction, while \afl~experiences higher variance, particularly in early training rounds. This fluctuation is likely due to staggered client updates and varying staleness in \afl~. Despite this variance, both methods eventually converge to similar loss levels, demonstrating that asynchronous FL remains a viable alternative to synchronous FL.
%\color{blue}[to put it in the conclusion or not]
%This experiment demonstrates that AFL can effectively handle non-IID data and client delays, achieving stable convergence of the global model. The delay-aware learning rate and early stopping mechanisms are particularly effective in ensuring that asynchronous updates did not destabilize the learning process. This approach provides a foundation for further research into scalable and privacy-preserving federated learning systems.
%\color{black}
\begin{figure}
\centering
\includegraphics[width=0.99\columnwidth]{images/asynch_nonconvex_mnist_server_losses_clients_5.png}
\caption{Server loss across rounds in \afl~algorithm for MNIST dataset with participation of $5$ clients in the training loop.}
\label{afl_mnist_1}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=0.85\columnwidth]{images/histogram_afl_mnist_clients_5.png}
\caption{Frequency distribution of different clients during the training loop in \afl~algorithm with participation of $5$ clients in the training loop for the MNIST dataset.}
\label{mnist_client_selection}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=0.99\linewidth]{images/nonconvex_mnist_server_losses_clients_participation.png}
\caption{Server loss across rounds based on the various number of client participation in the \afl~algorithm for the classification task for the MNIST dataset.}
\label{mnist_clients_participation}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=0.99\columnwidth]{images/nonconvex_mnist_server_losses_comparison.png}
\caption{Comparison of the \afl~algorithm with Synchronous FL with participation of 5 clients in the training process.}
\label{mnist_server_loss_comparison}
\end{figure}
%Figure~\ref{mnist_client_selection} presents a histogram depicting the frequency of client selection in the federated learning process. The results indicate that some clients, such as client 2, have been selected more frequently than others. Despite imbalance participation of the clients in the training loop, the global convergence of the server is not affected significantly as shown in \ref{afl_mnist_1}.
%Client selection in \ref{mnist_client_selection} confirmed that the client sampling process is relatively balanced, with each client being selected a similar number of times. While minor variations exist, no significant bias is evident in the selection frequency.
%This implementation demonstrates the effectiveness of asynchronous federated learning in handling delays and data heterogeneity. The delay-aware learning rate adjustment played a crucial role in stabilizing the training process by preventing clients with long delays from making overly large updates. Early stopping also contributed to efficiency by halting client training when improvements plateaued.
%One of the primary challenges encountered was the increased training time for clients with imbalanced datasets, as these clients required more local iterations to reach a stable point. Additionally, the frequent communication of model updates between the server and clients introduces communication overhead, which can become a bottleneck in large-scale deployments.
%The results of the experiment show that client losses varied significantly due to the non-IID nature of their local datasets. Clients with more balanced datasets achieved faster convergence, while clients with more skewed data distributions experienced slower and less stable convergence. Nonetheless, the majority of clients showed a clear trend toward lower losses over the course of the training rounds.
%In a separate set of experiments, we vary the number of clients in the training process, ranging from 30\% to 80\% in the AFL algorithm. For example in each experiment, at every training round, \( J = 3 \) clients are randomly selected to participate. The remaining settings remain the same as in the previous experiment, which involved a total of 5 clients (50\%).
%\color{black}
\subsection{Classification task on CIFAR dataset}
In the next example we employ the \afl~algorithm on the CIFAR-10 dataset using a ResNet architecture. CIFAR-10 contains 50{,}000 training and 10{,}000 test images in 10 classes, each image being $32\times 32$ pixels with three color channels (RGB). To reflect a realistic setup, the data is partitioned in a non-i.i.d.\ manner among 10 clients using a Dirichlet distribution with concentration parameter $\alpha=0.5$, so some clients may have data strongly skewed toward certain classes.
\paragraph{Training setup} Each client maintains a local ResNet model consisting of three layers of stacked blocks, where each block performs two $3\times 3$ convolutions with batch normalization and adds a shortcut path to mitigate vanishing gradients. After these layers, global average pooling reduces the spatial dimension, and a final fully connected layer outputs 10 logits \footnote{the term \emph{logits}, computed as the weighted sum of input features with an added bias, represent the raw decision scores before classification; in this dataset, they are thresholded at zero to assign binary labels.} followed by a log-softmax activation for classification. To mimic real-world conditions, client's delay are computed before training starts, and the code uses \texttt{asyncio} to handle these tasks concurrently.
A \textit{delay-aware learning rate} is adopted, scaling the base learning rate $\gamma_0 = 10^{-3}$ by
\( \frac{1}{\sqrt{t+1}\,\bigl(1 + \alpha \cdot d_t\bigr)},\) where $\alpha=0.01$ and $d_t$ is the client’s delay, so heavily delayed clients contribute smaller updates that reduce the risk of destabilizing the global model. In addition early stopping, with a patience of $10$ epochs, halts local training if the loss fails to improve, saving computation on less productive updates. Once local training finishes, the server aggregates the client's parameters via a weighted average proportional to each client’s dataset size, forming the updated global model.
\paragraph{Results}
At the first experiment in each communication round, a random subset of $J=5$ out of the $C=10$ clients is chosen to train locally for $I=10$ epochs, after which their models are sent back to a global server. Total number of training rounds is considered $\mathcal{J}=200$.
The simulation result for the first experiment is shown in the figure \ref{cifar_1}. Over $\mathcal{J}=200$ communication rounds, the server's average loss converges despite heterogeneous client distributions and client's delays, demonstrating the resilience of asynchronous federated learning with ResNet on CIFAR-10. In the early rounds, the server loss decreases significantly, suggesting that even with a small number of updates, the global model rapidly adapts to the distributed data.
The loss curve exhibits fluctuations at certain points, particularly in the first few rounds. This behavior is characteristic of \afl~, where updates arrive at different times, and clients contribute to the global model asynchronously.
The histogram in \ref{cifar_client_selection} illustrates the frequency of client participation during the \afl~training process on the CIFAR-10 dataset, where 5 out of 10 clients are randomly selected per communication round. The variation in participation frequency among clients highlights the non-uniform selection process inherent in \afl~. Some clients, such as Client 0 and Client 5, are chosen more frequently than others, while some, like Client 7, have fewer training opportunities. This imbalance in client participation can introduce variability in model updates, yet the global model still converges effectively, as observed in the loss curve. The randomness in client selection ensures diversity in training data aggregation, helping the model generalize better despite the non-IID data distribution. These results demonstrate the robustness of \afl~in handling heterogeneous client availability while maintaining stable learning dynamics.
The comparison between \afl~and Synchronous FL on the CIFAR-10 dataset is shown in the figure \ref{cifar_server_loss_comparison} reveals key trade-offs in their training dynamics. While SFL ensures more stable updates by aggregating all selected client models before updating the global model, it introduces delays due to waiting for all clients. In contrast, \afl~enables faster updates by allowing clients to contribute asynchronously, resulting in more fluctuations in the early rounds but maintaining steady convergence over time.
Figure \ref{cifar_clients_participation} presents the impact of varying the number of participating clients per round on server loss in \afl~for the CIFAR-10 dataset. Each curve corresponds to a different number of participating clients per round, ranging from 3 to 8. From this figure, we observe that increasing the number of clients per round leads to a faster reduction in server loss, indicating improved convergence. When more clients participate, the global model receives more frequent and diverse updates, accelerating learning. However, the difference in convergence rates becomes less pronounced after a certain number of rounds, suggesting diminishing returns in increasing client participation. The results also highlight the robustness of \afl~in handling asynchronous updates, as all configurations eventually converge despite fluctuations in early rounds.
This experiment underscores the trade-off between efficiency and convergence speed in federated learning. While selecting more clients per round enhances learning stability, it also increases communication overhead. The results validate that even with a lower number of participating clients, \afl~remains effective, demonstrating its adaptability to real-world federated learning scenarios where client availability may be limited or unpredictable.
Despite initial variations, both \afl~and Synchronous FL achieve similar final loss values, demonstrating that \afl~can effectively learn from decentralized data without requiring strict synchronization. This makes \afl~a more communication-efficient alternative for federated learning settings where client availability varies.
%Clients with balanced subsets typically converge more steadily, whereas those with skewed data see more variability.
%The delayed and non-i.i.d.\ conditions highlight that delay-aware scheduling and early stopping effectively stabilize learning, while random client selection over many rounds promotes robust global convergence.
\begin{figure}
\centering
\includegraphics[width=0.99\columnwidth]{images/nonconvex_cifar_server_losses.png}
\caption{Server loss across rounds in \afl~algorithm for CIFAR dataset with participation of $5$ clients in the training loop.}
\label{cifar_1}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=0.85\columnwidth]{images/histogram_afl_cifar_clients_5.png}
\caption{Frequency distribution of different clients during the training loop in \afl~algorithm with participation of $5$ clients in the training loop for the CIFAR dataset.}
\label{cifar_client_selection}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=0.99\columnwidth]{images/nonconvex_cifar_server_losses_comparison.png}
\caption{Comparison of the \afl~algorithm with Synchronous FL with participation 5 clients in the training loop for CIFAR dataset.}
\label{cifar_server_loss_comparison}
\end{figure}
\begin{figure}
\centering
\includegraphics[width=0.99\linewidth]{images/nonconvex_cifar_server_losses_clients_participation.png}
\caption{Server loss across rounds based on the various number of client participation in the \afl~algorithm for the classification task for the CIFAR dataset.}
\label{cifar_clients_participation}
\end{figure}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\section{Conclusion} \label{conclusion}
In this paper, we extended asynchronous federated learning (\afl~) to the optimization of \textit{non-convex objective functions}, addressing key challenges such as client staleness, heterogeneity, and communication delays. Our approach introduced a \textit{staleness-aware aggregation mechanism} and a \textit{dynamic learning rate schedule} to adapt to the asynchronous nature of client updates, ensuring more stable convergence. The theoretical analysis provided mathematical bounds on the impact of stale gradients, demonstrating that our approach effectively mitigates their negative effects. Through extensive simulations on the MNIST and CIFAR-10 datasets, we validated the effectiveness of \afl~in handling \textit{non-IID data distributions} and \textit{heterogeneous client environments}. The results demonstrated that \afl~maintains robust convergence while reducing the impact of straggler clients, making it a scalable alternative to Synchronous FL. Specifically, our findings highlighted that increasing client participation per round accelerates convergence while preserving stability. The comparison between \afl~and Synchronous FL showed that despite early-round fluctuations, \afl~achieves similar final performance with lower synchronization overhead. This study contributes to the advancement of federated learning in practical, large-scale, and heterogeneous environments. Future work can focus on further refining staleness compensation mechanisms, incorporating adaptive sampling strategies, and exploring \afl~'s applicability to additional real-world tasks such as healthcare, finance, and smart grid systems.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\section{conclusion}
%The experiments demonstrate that AFL can effectively handle non-IID data and client delays, achieving stable convergence of the global model. The delay-aware learning rate and early stopping mechanisms are particularly effective in ensuring that asynchronous updates did not destabilize the learning process. This approach provides a foundation for further research into scalable and privacy-preserving federated learning systems.
%The results highlight that AFL can match the performance of Synchronous FL while offering greater flexibility, making it well-suited for real-world distributed learning environments.
\begin{comment}
\subsection{Simulation Example: Asynchronous Federated Learning for Convex Optimization}
In this simulation, we demonstrate the implementation of an asynchronous \textit{Federated Learning (FL)} system for solving a \textit{convex optimization problem} using a \textit{linear regression model}. The framework simulates a decentralized training environment with multiple clients collaboratively training a global model, where client updates are delayed to mimic real-world network heterogeneity. Below, we outline the simulation setup, the asynchronous FL process, and the results obtained.
\paragraph{Simulation Setup.}
\textbf{Data Generation and Partitioning:} A synthetic dataset is generated with \(n = 1000\) samples and \(d = 10\) input features, drawn uniformly from the interval \([-5, 5]\). The target variable is computed as a linear combination of input features with added Gaussian noise (\(\sigma^2 = 0.2\)). The dataset is partitioned across \textbf{10 clients}, with each client receiving a different subset, introducing non-IID (non-identically distributed) characteristics.
\textbf{Model Description:} A simple linear regression model is deployed:
\[
f(\mathbf{x}) = \mathbf{w}^\top \mathbf{x} + b
\]
where \(\mathbf{w} \in \mathbb{R}^d\) and \(b \in \mathbb{R}\) are the learnable parameters.
\textbf{Federated Learning Process:} The training runs for \textbf{500 communication rounds}. In each round, the server selects a random subset of \textbf{5 clients} to participate in training. Each client performs \textbf{10 local epochs} of Stochastic Gradient Descent (SGD) with a batch size of \textbf{32}. The learning rate \(\gamma_t\) is adjusted dynamically based on the round number \(t\):
\[
\gamma_t = \frac{\gamma_0}{\sqrt{t + 1}(1 + \alpha \cdot \delta_t)}
\]
where \(\delta_t\) is the simulated delay between \textbf{0 and 2 seconds}.
After receiving local model updates from participating clients, the server aggregates the updates proportionally to the dataset sizes:
\[
\mathbf{w}_{\text{global}} = \sum_{i \in S_t} \frac{n_i}{N} \mathbf{w}_i
\]
where \(S_t\) is the set of participating clients in round \(t\), \(n_i\) is the number of samples at client \(i\), and \(N\) is the total number of samples from all selected clients.
\paragraph{Simulation Results.}
\textbf{Server Loss Trend:} The global model’s loss, evaluated using Mean Squared Error (MSE), decreased logarithmically over the communication rounds, indicating effective convergence.
\textbf{Client Participation:} A scatter plot of client participation across rounds shows randomized selection, ensuring that all clients contribute over time.
\textbf{Client Losses:} Loss curves for individual clients indicate gradual convergence despite local data heterogeneity. Minor fluctuations in loss are attributed to asynchronous updates and differing dataset sizes.
\textbf{Execution Time Analysis:} The execution time for each round varied due to simulated delays, with some rounds exhibiting significant differences between the fastest and slowest clients. Despite this, the overall system remained resilient with minimal impact on convergence.
\paragraph{Key Observations.}
1. \textbf{Robust Convergence:} The global model converged smoothly despite asynchronous updates and random delays.
2. \textbf{Fair Participation:} Randomized client selection ensured balanced contributions from all clients.
3. \textbf{Handling Heterogeneity:} Weighted aggregation effectively mitigated the impact of data imbalance and local variability.
\paragraph{Conclusion.}
This simulation demonstrates the robustness of asynchronous Federated Learning for convex optimization tasks. Adaptive learning rates and weighted aggregation play a crucial role in achieving stable convergence despite delays and heterogeneity. These findings highlight the potential of Federated Learning for distributed systems and provide a foundation for further research on non-convex objectives and real-world applications.
\end{comment}
\begin{comment}
\subsubsection{Data Preparation and Partitioning}
The MNIST dataset, consisting of 60,000 training and 10,000 test samples of handwritten digits, was utilized for this study. To simulate real-world heterogeneity, the dataset was partitioned among two clients using a non-IID approach, where each client received a subset with varying class distributions. Preprocessing included normalization of the pixel values to the range \([-1, 1]\). PyTorch's \texttt{DataLoader} was used for efficient data batching during training and evaluation.
\subsubsection{Model Architecture}
The Convolutional Neural Network (CNN) architecture employed in this study consists of four layers:
\begin{itemize}
\item \textbf{Input Layer:} Designed to accept grayscale images with a single channel.
\item \textbf{Hidden Layers:} Two convolutional layers with 32 filters each, followed by ReLU activations and max-pooling layers for feature extraction.
\item \textbf{Fully Connected Layer:} Responsible for classification into 10 classes representing digits (0-9).
\end{itemize}
Both the global and client models utilized the same architecture, ensuring consistency. PyTorch's default initialization methods were applied to model parameters for reproducibility.
\subsubsection{Asynchronous Training Workflow}
Each client trained its local model independently in an asynchronous fashion. At the start of each round, the server initialized and distributed the global model's parameters to all clients. Clients performed local training for a fixed number of epochs (10 per round) and returned their updated model weights to the server upon completion. The asynchronous nature of the workflow was implemented using Python's \texttt{asyncio} library, allowing for non-blocking execution.
%\subsubsection{Delay-Aware Learning Rate Adjustment}
To accommodate disparities in execution times across clients, a delay-aware learning rate adjustment was implemented. The learning rate \(\gamma_t\) at step \(t\) was calculated as:
\[
\gamma_t = \frac{\gamma_0}{\sqrt{t + 1}(1 + \alpha \cdot \text{delay})},
\]
where:
\begin{itemize}
\item \(\gamma_0\): Base learning rate.
\item \(t\): Current iteration or epoch.
\item \(\alpha\): Scaling factor for delay.
\item \(\text{delay}\): Difference in execution times between the fastest and slowest clients.
\end{itemize}
This adjustment ensures that clients with delays contribute effectively without destabilizing the optimization process.
%\subsubsection{Weighted Aggregation of Client Updates}
The server aggregates client updates using a weighted averaging approach, prioritizing clients with larger datasets. For client \(i\), the weight factor is proportional to the size of its dataset, defined as:
\[
p_i = \frac{n_i}{\sum_{j} n_j},
\]
where \(n_i\) is the number of samples held by client \(i\). The updated global model parameters \(w\) are computed as:
\[
w = \sum_{i} p_i w_i,
\]
where \(w_i\) represents the parameters of the \(i\)-th client's model.
%\subsubsection{Loss Function and Optimization}
The Cross-Entropy Loss function was employed to measure classification performance. For parameter updates, the Adam optimizer was utilized. Gradient accumulation was implemented to handle memory constraints, ensuring updates were performed only after a fixed number of gradient steps.
%\subsubsection{Model Evaluation}
The global model was periodically evaluated using the test dataset. Key metrics, including training loss and execution time per client, were recorded to assess the framework's performance.
%Visualizations, such as client participation, execution delays, and convergence trends, were generated to provide insights into the asynchronous training dynamics.
\end{comment}
\section{Appendix}\label{appendix}
This section provides more details related to the intermediate steps for derivation of Eq.\eqref{equation_appendix} in Theorem \ref{conv_theorem}. Starting from the inequality given in Lemma~\ref{lem:non-convex:recursion}:
\begin{multline}
\E\left[\mathcal{L}(\theta^{(j+1)}) - \mathcal{L}(\theta^{(j)})\right] \leq -\frac{1}{6}JI\gamma\E\left[\Norm{\nabla \mathcal{L}(\theta^{(j)})}^2\right]
\\+ 2L\gamma^2JI\sigma^2
+ 2L\gamma^2J^2I^2\frac{C-J}{J(C-1)}\nu^2
+ \frac{5}{6}L^2\gamma E_j,
\end{multline}
and substituting \(E_j\) from Lemma~\ref{lem:non-convex:drift}:
\begin{multline*}
E_j \leq \frac{9}{4}J^2I^2\gamma^2\sigma^2
+ \frac{9}{4}J^2I^3\gamma^2\nu^2
\\ + \frac{9}{4}\left(\frac{\beta^2}{J} + 1\right)J^3I^3\gamma^2\E\left[\Norm{\nabla \mathcal{L}(\theta^{(j)})}^2\right],
\end{multline*}
we substitute \(E_j\) into the recursion to get:
\begin{multline*}
\E\left[\mathcal{L}(\theta^{(j+1)}) - \mathcal{L}(\theta^{(j)})\right]
\leq -\frac{1}{6}JI\gamma\E\left[\Norm{\nabla \mathcal{L}(\theta^{(j)})}^2\right]
\\ + 2L\gamma^2JI\sigma^2
\\+ 2L\gamma^2J^2I^2\frac{C-J}{J(C-1)}\nu^2
+ \frac{15}{8}L^2\gamma^3J^2I^2\sigma^2
+ \frac{15}{8}L^2\gamma^3J^2I^3\nu^2
\\+ \frac{15}{8}L^2\gamma^3\left(\frac{\beta^2}{J} + 1\right)J^3I^3\E\left[\Norm{\nabla \mathcal{L}(\theta^{(j)})}^2\right].
\end{multline*}
Focusing on the gradient-related terms:
\begin{multline*}
-\frac{1}{6}JI\gamma\E\left[\Norm{\nabla \mathcal{L}(\theta^{(r)})}^2\right]
\\ + \frac{15}{8}L^2\gamma^3\left(\frac{\beta^2}{J} + 1\right)J^3I^3\E\left[\Norm{\nabla \mathcal{L}(\theta^{(r)})}^2\right].
\end{multline*}
Substituting \(L = \frac{1}{6\gamma JI\left(1 + \frac{\beta^2}{J}\right)}\), we rewrite \(L^2\) as: \(
L^2 = \frac{1}{36\gamma^2J^2I^2\left(1 + \frac{\beta^2}{J}\right)^2}.\) Substituting \(L^2\) into the second term:
\begin{multline*}
\frac{15}{8}L^2\gamma^3\left(\frac{\beta^2}{J} + 1\right)J^3I^3
\\ = \frac{15}{8} \cdot \frac{\gamma^3}{36\gamma^2J^2I^2\left(1 + \frac{\beta^2}{J}\right)^2} \cdot \left(\frac{\beta^2}{J} + 1\right) J^3I^3\\
= \frac{15}{8} \cdot \frac{\gamma JI}{36\left(1 + \frac{\beta^2}{J}\right)} \E\left[\Norm{\nabla \mathcal{L}(\theta^{(r)})}^2\right]
\end{multline*}
%\[
%\frac{15}{8}L^2\gamma^3\left(\frac{\beta^2}{J} + 1\right)J^3I^3
%= \frac{15}{8} \cdot \frac{\gamma^3}{36\gamma^2J^2I^2\left(1 + \frac{\beta^2}{J}\right)^2} \cdot \left(\frac{\beta^2}{J} + 1\right) J^3I^3.
%\]
%Simplify:
%\[
%= \frac{15}{8} \cdot \frac{\gamma JI}{36\left(1 + \frac{\beta^2}{J}\right)} \E\left[\Norm{\nabla \mathcal{L}(\theta^{(r)})}^2\right].
%\]
Combining with the first term:
\[
JI\gamma\E\left[\Norm{\nabla \mathcal{L}(\theta^{(r)})}^2\right] \left(-\frac{1}{6} + \frac{5}{96\left(1 + \frac{\beta^2}{J}\right)}\right).
\]
Simplify the coefficient:
\[
-\frac{1}{6} + \frac{5}{96\left(1 + \frac{\beta^2}{J}\right)} = \frac{-16\left(1 + \frac{\beta^2}{J}\right) + 5}{96\left(1 + \frac{\beta^2}{J}\right)} = \frac{-11 - \frac{16\beta^2}{J}}{96\left(1 + \frac{\beta^2}{J}\right)}.
\]
To provide a suitable upper bound for this coefficient, note that: \(
\frac{-11 - \frac{16\beta^2}{J}}{96\left(1 + \frac{\beta^2}{J}\right)} \leq -\frac{1}{6}. \) Thus, the final simplified recursion is:
\begin{multline*}
\E\left[\mathcal{L}(\theta^{(r+1)}) - \mathcal{L}(\theta^{(r)})\right]
\leq -\frac{1}{6}JI\gamma\E\left[\Norm{\nabla \mathcal{L}(\theta^{(r)})}^2\right] \\+ \text{higher-order terms}.
\end{multline*}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\bibliographystyle{elsarticle-num-names}
\bibliographystyle{ieeetr}
\bibliography{my_refs_lcss,refs}
%\bio{images/ali_forootani.png}
\end{document}
%%%%% NEW MATH DEFINITIONS %%%%%
\usepackage{amsmath,amsfonts,bm, amssymb}
\usepackage{amsthm}
\usepackage{mathtools}
% Mark sections of captions for referring to divisions of figures
\newcommand{\figleft}{{\em (Left)}}
\newcommand{\figcenter}{{\em (Center)}}
\newcommand{\figright}{{\em (Right)}}
\newcommand{\figtop}{{\em (Top)}}
\newcommand{\figbottom}{{\em (Bottom)}}
\newcommand{\captiona}{{\em (a)}}
\newcommand{\captionb}{{\em (b)}}
\newcommand{\captionc}{{\em (c)}}
\newcommand{\captiond}{{\em (d)}}
% Highlight a newly defined term
\newcommand{\newterm}[1]{{\bf #1}}
% Figure reference, lower-case.
\def\figref#1{figure~\ref{#1}}
% Figure reference, capital. For start of sentence
\def\Figref#1{Figure~\ref{#1}}
\def\twofigref#1#2{figures \ref{#1} and \ref{#2}}
\def\quadfigref#1#2#3#4{figures \ref{#1}, \ref{#2}, \ref{#3} and \ref{#4}}
% Section reference, lower-case.
\def\secref#1{section~\ref{#1}}
% Section reference, capital.
\def\Secref#1{Section~\ref{#1}}
% Reference to two sections.
\def\twosecrefs#1#2{sections \ref{#1} and \ref{#2}}
% Reference to three sections.
\def\secrefs#1#2#3{sections \ref{#1}, \ref{#2} and \ref{#3}}
% Reference to an equation, lower-case.
%\def\eqref#1{equation~\ref{#1}} %liyipeng
% Reference to an equation, upper case
%\def\Eqref#1{Equation~\ref{#1}} %liyipeng
% A raw reference to an equation---avoid using if possible
\def\plaineqref#1{\ref{#1}}
% Reference to a chapter, lower-case.
\def\chapref#1{chapter~\ref{#1}}
% Reference to an equation, upper case.
\def\Chapref#1{Chapter~\ref{#1}}
% Reference to a range of chapters
\def\rangechapref#1#2{chapters\ref{#1}--\ref{#2}}
% Reference to an algorithm, lower-case.
\def\algref#1{algorithm~\ref{#1}}
% Reference to an algorithm, upper case.
\def\Algref#1{Algorithm~\ref{#1}}
\def\twoalgref#1#2{algorithms \ref{#1} and \ref{#2}}
\def\Twoalgref#1#2{Algorithms \ref{#1} and \ref{#2}}
% Reference to a part, lower case
\def\partref#1{part~\ref{#1}}
% Reference to a part, upper case
\def\Partref#1{Part~\ref{#1}}
\def\twopartref#1#2{parts \ref{#1} and \ref{#2}}
\def\ceil#1{\lceil #1 \rceil}
\def\floor#1{\lfloor #1 \rfloor}
\def\1{\bm{1}}
\newcommand{\train}{\mathcal{D}}
\newcommand{\valid}{\mathcal{D_{\mathrm{valid}}}}
\newcommand{\test}{\mathcal{D_{\mathrm{test}}}}
\def\eps{{\epsilon}}
% Random variables
\def\reta{{\textnormal{$\eta$}}}
\def\ra{{\textnormal{a}}}
\def\rb{{\textnormal{b}}}
\def\rc{{\textnormal{c}}}
\def\rd{{\textnormal{d}}}
\def\re{{\textnormal{e}}}
\def\rf{{\textnormal{f}}}
\def\rg{{\textnormal{g}}}
\def\rh{{\textnormal{h}}}
\def\ri{{\textnormal{i}}}
\def\rj{{\textnormal{j}}}
\def\rk{{\textnormal{k}}}
\def\rl{{\textnormal{l}}}
% rm is already a command, just don't name any random variables m
\def\rn{{\textnormal{n}}}
\def\ro{{\textnormal{o}}}
\def\rp{{\textnormal{p}}}
\def\rq{{\textnormal{q}}}
\def\rr{{\textnormal{r}}}
\def\rs{{\textnormal{s}}}
\def\rt{{\textnormal{t}}}
\def\ru{{\textnormal{u}}}
\def\rv{{\textnormal{v}}}
\def\rw{{\textnormal{w}}}
\def\rx{{\textnormal{x}}}
\def\ry{{\textnormal{y}}}
\def\rz{{\textnormal{z}}}
% Random vectors
\def\rvepsilon{{\mathbf{\epsilon}}}
\def\rvtheta{{\mathbf{\theta}}}
\def\rva{{\mathbf{a}}}
\def\rvb{{\mathbf{b}}}
\def\rvc{{\mathbf{c}}}
\def\rvd{{\mathbf{d}}}
\def\rve{{\mathbf{e}}}
\def\rvf{{\mathbf{f}}}
\def\rvg{{\mathbf{g}}}
\def\rvh{{\mathbf{h}}}
\def\rvu{{\mathbf{i}}}
\def\rvj{{\mathbf{j}}}
\def\rvk{{\mathbf{k}}}
\def\rvl{{\mathbf{l}}}
\def\rvm{{\mathbf{m}}}
\def\rvn{{\mathbf{n}}}
\def\rvo{{\mathbf{o}}}
\def\rvp{{\mathbf{p}}}
\def\rvq{{\mathbf{q}}}
\def\rvr{{\mathbf{r}}}
\def\rvs{{\mathbf{s}}}
\def\rvt{{\mathbf{t}}}
\def\rvu{{\mathbf{u}}}
\def\rvv{{\mathbf{v}}}
\def\rvw{{\mathbf{w}}}
\def\rvx{{\mathbf{x}}}
\def\rvy{{\mathbf{y}}}
\def\rvz{{\mathbf{z}}}
% Elements of random vectors
\def\erva{{\textnormal{a}}}
\def\ervb{{\textnormal{b}}}
\def\ervc{{\textnormal{c}}}
\def\ervd{{\textnormal{d}}}
\def\erve{{\textnormal{e}}}
\def\ervf{{\textnormal{f}}}
\def\ervg{{\textnormal{g}}}
\def\ervh{{\textnormal{h}}}
\def\ervi{{\textnormal{i}}}
\def\ervj{{\textnormal{j}}}
\def\ervk{{\textnormal{k}}}
\def\ervl{{\textnormal{l}}}
\def\ervm{{\textnormal{m}}}
\def\ervn{{\textnormal{n}}}
\def\ervo{{\textnormal{o}}}
\def\ervp{{\textnormal{p}}}
\def\ervq{{\textnormal{q}}}
\def\ervr{{\textnormal{r}}}
\def\ervs{{\textnormal{s}}}
\def\ervt{{\textnormal{t}}}
\def\ervu{{\textnormal{u}}}
\def\ervv{{\textnormal{v}}}
\def\ervw{{\textnormal{w}}}
\def\ervx{{\textnormal{x}}}
\def\ervy{{\textnormal{y}}}
\def\ervz{{\textnormal{z}}}
% Random matrices
\def\rmA{{\mathbf{A}}}
\def\rmB{{\mathbf{B}}}
\def\rmC{{\mathbf{C}}}
\def\rmD{{\mathbf{D}}}
\def\rmE{{\mathbf{E}}}
\def\rmF{{\mathbf{F}}}
\def\rmG{{\mathbf{G}}}
\def\rmH{{\mathbf{H}}}
\def\rmI{{\mathbf{I}}}
\def\rmJ{{\mathbf{J}}}
\def\rmK{{\mathbf{K}}}
\def\rmL{{\mathbf{L}}}
\def\rmM{{\mathbf{M}}}
\def\rmN{{\mathbf{N}}}
\def\rmO{{\mathbf{O}}}
\def\rmP{{\mathbf{P}}}
\def\rmQ{{\mathbf{Q}}}
\def\rmR{{\mathbf{R}}}
\def\rmS{{\mathbf{S}}}
\def\rmT{{\mathbf{T}}}
\def\rmU{{\mathbf{U}}}
\def\rmV{{\mathbf{V}}}
\def\rmW{{\mathbf{W}}}
\def\rmX{{\mathbf{X}}}
\def\rmY{{\mathbf{Y}}}
\def\rmZ{{\mathbf{Z}}}
% Elements of random matrices
\def\ermA{{\textnormal{A}}}
\def\ermB{{\textnormal{B}}}
\def\ermC{{\textnormal{C}}}
\def\ermD{{\textnormal{D}}}
\def\ermE{{\textnormal{E}}}
\def\ermF{{\textnormal{F}}}
\def\ermG{{\textnormal{G}}}
\def\ermH{{\textnormal{H}}}
\def\ermI{{\textnormal{I}}}
\def\ermJ{{\textnormal{J}}}
\def\ermK{{\textnormal{K}}}
\def\ermL{{\textnormal{L}}}
\def\ermM{{\textnormal{M}}}
\def\ermN{{\textnormal{N}}}
\def\ermO{{\textnormal{O}}}
\def\ermP{{\textnormal{P}}}
\def\ermQ{{\textnormal{Q}}}
\def\ermR{{\textnormal{R}}}
\def\ermS{{\textnormal{S}}}
\def\ermT{{\textnormal{T}}}
\def\ermU{{\textnormal{U}}}
\def\ermV{{\textnormal{V}}}
\def\ermW{{\textnormal{W}}}
\def\ermX{{\textnormal{X}}}
\def\ermY{{\textnormal{Y}}}
\def\ermZ{{\textnormal{Z}}}
% Vectors
\def\vzero{{\bm{0}}}
\def\vone{{\bm{1}}}
\def\vmu{{\bm{\mu}}}
\def\vtheta{{\bm{\theta}}}
\def\va{{\bm{a}}}
\def\vb{{\bm{b}}}
\def\vc{{\bm{c}}}
\def\vd{{\bm{d}}}
\def\ve{{\bm{e}}}
\def\vf{{\bm{f}}}
\def\vg{{\bm{g}}}
\def\vh{{\bm{h}}}
\def\vi{{\bm{i}}}
\def\vj{{\bm{j}}}
\def\vk{{\bm{k}}}
\def\vl{{\bm{l}}}
\def\vm{{\bm{m}}}
\def\vn{{\bm{n}}}
\def\vo{{\bm{o}}}
\def\vp{{\bm{p}}}
\def\vq{{\bm{q}}}
\def\vr{{\bm{r}}}
\def\vs{{\bm{s}}}
\def\vt{{\bm{t}}}
\def\vu{{\bm{u}}}
\def\vv{{\bm{v}}}
\def\vw{{\bm{w}}}
\def\vx{{\bm{x}}}
\def\vy{{\bm{y}}}
\def\vz{{\bm{z}}}
% Elements of vectors
\def\evalpha{{\alpha}}
\def\evbeta{{\beta}}
\def\evepsilon{{\epsilon}}
\def\evlambda{{\lambda}}
\def\evomega{{\omega}}
\def\evmu{{\mu}}
\def\evpsi{{\psi}}
\def\evsigma{{\sigma}}
\def\evtheta{{\theta}}
\def\eva{{a}}
\def\evb{{b}}
\def\evc{{c}}
\def\evd{{d}}
\def\eve{{e}}
\def\evf{{f}}
\def\evg{{g}}
\def\evh{{h}}
\def\evi{{i}}
\def\evj{{j}}
\def\evk{{k}}
\def\evl{{l}}
\def\evm{{m}}
\def\evn{{n}}
\def\evo{{o}}
\def\evp{{p}}
\def\evq{{q}}
\def\evr{{r}}
\def\evs{{s}}
\def\evt{{t}}
\def\evu{{u}}
\def\evv{{v}}
\def\evw{{w}}
\def\evx{{x}}
\def\evy{{y}}
\def\evz{{z}}
% Matrix
\def\mA{{\bm{A}}}
\def\mB{{\bm{B}}}
\def\mC{{\bm{C}}}
\def\mD{{\bm{D}}}
\def\mE{{\bm{E}}}
\def\mF{{\bm{F}}}
\def\mG{{\bm{G}}}
\def\mH{{\bm{H}}}
\def\mI{{\bm{I}}}
\def\mJ{{\bm{J}}}
\def\mK{{\bm{K}}}
\def\mL{{\bm{L}}}
\def\mM{{\bm{M}}}
\def\mN{{\bm{N}}}
\def\mO{{\bm{O}}}
\def\mP{{\bm{P}}}
\def\mQ{{\bm{Q}}}
\def\mR{{\bm{R}}}
\def\mS{{\bm{S}}}
\def\mT{{\bm{T}}}
\def\mU{{\bm{U}}}
\def\mV{{\bm{V}}}
\def\mW{{\bm{W}}}
\def\mX{{\bm{X}}}
\def\mY{{\bm{Y}}}
\def\mZ{{\bm{Z}}}
\def\mBeta{{\bm{\beta}}}
\def\mPhi{{\bm{\Phi}}}
\def\mLambda{{\bm{\Lambda}}}
\def\mSigma{{\bm{\Sigma}}}
% Tensor
\DeclareMathAlphabet{\mathsfit}{\encodingdefault}{\sfdefault}{m}{sl}
\SetMathAlphabet{\mathsfit}{bold}{\encodingdefault}{\sfdefault}{bx}{n}
\newcommand{\tens}[1]{\bm{\mathsfit{#1}}}
\def\tA{{\tens{A}}}
\def\tB{{\tens{B}}}
\def\tC{{\tens{C}}}
\def\tD{{\tens{D}}}
\def\tE{{\tens{E}}}
\def\tF{{\tens{F}}}
\def\tG{{\tens{G}}}
\def\tH{{\tens{H}}}
\def\tI{{\tens{I}}}
\def\tJ{{\tens{J}}}
\def\tK{{\tens{K}}}
\def\tL{{\tens{L}}}
\def\tM{{\tens{M}}}
\def\tN{{\tens{N}}}
\def\tO{{\tens{O}}}
\def\tP{{\tens{P}}}
\def\tQ{{\tens{Q}}}
\def\tR{{\tens{R}}}
\def\tS{{\tens{S}}}
\def\tT{{\tens{T}}}
\def\tU{{\tens{U}}}
\def\tV{{\tens{V}}}
\def\tW{{\tens{W}}}
\def\tX{{\tens{X}}}
\def\tY{{\tens{Y}}}
\def\tZ{{\tens{Z}}}
% Graph
\def\gA{{\mathcal{A}}}
\def\gB{{\mathcal{B}}}
\def\gC{{\mathcal{C}}}
\def\gD{{\mathcal{D}}}
\def\gE{{\mathcal{E}}}
\def\gF{{\mathcal{F}}}
\def\gG{{\mathcal{G}}}
\def\gH{{\mathcal{H}}}
\def\gI{{\mathcal{I}}}
\def\gJ{{\mathcal{J}}}
\def\gK{{\mathcal{K}}}
\def\gL{{\mathcal{L}}}
\def\gM{{\mathcal{M}}}
\def\gN{{\mathcal{N}}}
\def\gO{{\mathcal{O}}}
\def\gP{{\mathcal{P}}}
\def\gQ{{\mathcal{Q}}}
\def\gR{{\mathcal{R}}}
\def\gS{{\mathcal{S}}}
\def\gT{{\mathcal{T}}}
\def\gU{{\mathcal{U}}}
\def\gV{{\mathcal{V}}}
\def\gW{{\mathcal{W}}}
\def\gX{{\mathcal{X}}}
\def\gY{{\mathcal{Y}}}
\def\gZ{{\mathcal{Z}}}
% Sets
\def\sA{{\mathbb{A}}}
\def\sB{{\mathbb{B}}}
\def\sC{{\mathbb{C}}}
\def\sD{{\mathbb{D}}}
% Don't use a set called E, because this would be the same as our symbol
% for expectation.
\def\sF{{\mathbb{F}}}
\def\sG{{\mathbb{G}}}
\def\sH{{\mathbb{H}}}
\def\sI{{\mathbb{I}}}
\def\sJ{{\mathbb{J}}}
\def\sK{{\mathbb{K}}}
\def\sL{{\mathbb{L}}}
\def\sM{{\mathbb{M}}}
\def\sN{{\mathbb{N}}}
\def\sO{{\mathbb{O}}}
\def\sP{{\mathbb{P}}}
\def\sQ{{\mathbb{Q}}}
\def\sR{{\mathbb{R}}}
\def\sS{{\mathbb{S}}}
\def\sT{{\mathbb{T}}}
\def\sU{{\mathbb{U}}}
\def\sV{{\mathbb{V}}}
\def\sW{{\mathbb{W}}}
\def\sX{{\mathbb{X}}}
\def\sY{{\mathbb{Y}}}
\def\sZ{{\mathbb{Z}}}
% Entries of a matrix
\def\emLambda{{\Lambda}}
\def\emA{{A}}
\def\emB{{B}}
\def\emC{{C}}
\def\emD{{D}}
\def\emE{{E}}
\def\emF{{F}}
\def\emG{{G}}
\def\emH{{H}}
\def\emI{{I}}
\def\emJ{{J}}
\def\emK{{K}}
\def\emL{{L}}
\def\emM{{M}}
\def\emN{{N}}
\def\emO{{O}}
\def\emP{{P}}
\def\emQ{{Q}}
\def\emR{{R}}
\def\emS{{S}}
\def\emT{{T}}
\def\emU{{U}}
\def\emV{{V}}
\def\emW{{W}}
\def\emX{{X}}
\def\emY{{Y}}
\def\emZ{{Z}}
\def\emSigma{{\Sigma}}
% entries of a tensor
% Same font as tensor, without \bm wrapper
\newcommand{\etens}[1]{\mathsfit{#1}}
\def\etLambda{{\etens{\Lambda}}}
\def\etA{{\etens{A}}}
\def\etB{{\etens{B}}}
\def\etC{{\etens{C}}}
\def\etD{{\etens{D}}}
\def\etE{{\etens{E}}}
\def\etF{{\etens{F}}}
\def\etG{{\etens{G}}}
\def\etH{{\etens{H}}}
\def\etI{{\etens{I}}}
\def\etJ{{\etens{J}}}
\def\etK{{\etens{K}}}
\def\etL{{\etens{L}}}
\def\etM{{\etens{M}}}
\def\etN{{\etens{N}}}
\def\etO{{\etens{O}}}
\def\etP{{\etens{P}}}
\def\etQ{{\etens{Q}}}
\def\etR{{\etens{R}}}
\def\etS{{\etens{S}}}
\def\etT{{\etens{T}}}
\def\etU{{\etens{U}}}
\def\etV{{\etens{V}}}
\def\etW{{\etens{W}}}
\def\etX{{\etens{X}}}
\def\etY{{\etens{Y}}}
\def\etZ{{\etens{Z}}}
% The true underlying data generating distribution
\newcommand{\pdata}{p_{\rm{data}}}
% The empirical distribution defined by the training set
\newcommand{\ptrain}{\hat{p}_{\rm{data}}}
\newcommand{\Ptrain}{\hat{P}_{\rm{data}}}
% The model distribution
\newcommand{\pmodel}{p_{\rm{model}}}
\newcommand{\Pmodel}{P_{\rm{model}}}
\newcommand{\ptildemodel}{\tilde{p}_{\rm{model}}}
% Stochastic autoencoder distributions
\newcommand{\pencode}{p_{\rm{encoder}}}
\newcommand{\pdecode}{p_{\rm{decoder}}}
\newcommand{\precons}{p_{\rm{reconstruct}}}
\newcommand{\laplace}{\mathrm{Laplace}} % Laplace distribution
\newcommand{\E}{\mathbb{E}}
\newcommand{\Ls}{\mathcal{L}}
\newcommand{\R}{\mathbb{R}}
\newcommand{\emp}{\tilde{p}}
\newcommand{\lr}{\alpha}
\newcommand{\reg}{\lambda}
\newcommand{\rect}{\mathrm{rectifier}}
\newcommand{\softmax}{\mathrm{softmax}}
\newcommand{\sigmoid}{\sigma}
\newcommand{\softplus}{\zeta}
\newcommand{\KL}{D_{\mathrm{KL}}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\standarderror}{\mathrm{SE}}
\newcommand{\Cov}{\mathrm{Cov}}
% Wolfram Mathworld says $L^2$ is for function spaces and $\ell^2$ is for vectors
% But then they seem to use $L^2$ for vectors throughout the site, and so does
% wikipedia.
\newcommand{\normlzero}{L^0}
\newcommand{\normlone}{L^1}
\newcommand{\normltwo}{L^2}
\newcommand{\normlp}{L^p}
\newcommand{\normmax}{L^\infty}
\newcommand{\parents}{Pa} % See usage in notation.tex. Chosen to match Daphne's book.
\DeclareMathOperator*{\argmax}{arg\,max}
\DeclareMathOperator*{\argmin}{arg\,min}
\DeclareMathOperator{\sign}{sign}
\DeclareMathOperator{\Tr}{Tr}
\let\ab\allowbreak
% liyipeng's new commands
% math commands
\newcommand{\eqdef}{\overset{\text{def}}{=}}
\newcommand\inp[2]{\left\langle #1, #2 \right\rangle} % inner product
\newcommand{\norm}[1]{\left\lVert#1\right\rVert} % norm
\newcommand{\Norm}[1]{\lVert#1\rVert} % norm
\newcommand{\sqn}[1]{{\left\lVert#1\right\rVert}^2} % square
\newcommand{\abs}[1]{\left\lvert#1\right\rvert}
\newcommand{\pfcomment}[1]{
\tag*{$\triangleright$ #1}
}
\newcommand{\Term}[2]{\text{Term}_{#1}\ \text{in}\ #2}
% << color
\newcommand{\tcb}[1]{\textcolor{blue}{#1}}
\newcommand{\tcr}[1]{\textcolor{red}{#1}}
\newcommand{\ulb}[1]{{\setulcolor{blue}\ul{#1}}}
\newcommand{\ulr}[1]{{\setulcolor{red}\ul{#1}}}
\newcommand{\upb}[1]{\textcolor{blue}{\textbf{$\uparrow\!#1$}}}
\newcommand{\upr}[1]{\textcolor{red}{\textbf{$\uparrow\!#1$}}}
% color >>
% expression commands
\newcommand{\eg}{e.g.}
\newcommand{\ie}{i.e.}
\newcommand{\iid}{i.i.d.}
\newcommand{\cf}{cf.}
%\theoremstyle{definition} % change style
\newtheorem{assumption}{Assumption}
\newtheorem{definition}{Definition}
\newtheorem{theorem}{Theorem}
\newtheorem{corollary}{Corollary}
\newtheorem{lemma}{Lemma}
% end new commands
% addtional commands
\newcommand{\fronorm}[1]{\ensuremath{\norm{#1}_{\tiny{\mbox{F}}}}}
\newcommand{\tp}{\ensuremath{^\top}}
\newcommand{\sleff}{\tilde{\eta}_{\text{SL}}}
\newcommand{\fleff}{\tilde{\eta}_{\text{FL}}}
\begin{thebibliography}{10}
\bibitem{yu2019parallel}
H.~Yu, S.~Yang, and S.~Zhu, ``Parallel restarted sgd with faster convergence
and less communication: Demystifying why model averaging works for deep
learning,'' in {\em Proceedings of the AAAI conference on artificial
intelligence}, vol.~33, pp.~5693--5700, 2019.
\bibitem{kairouz2021advances}
P.~Kairouz, H.~B. McMahan, B.~Avent, A.~Bellet, M.~Bennis, A.~N. Bhagoji,
K.~Bonawitz, Z.~Charles, G.~Cormode, R.~Cummings, {\em et~al.}, ``Advances
and open problems in federated learning,'' {\em Foundations and
trends{\textregistered} in machine learning}, vol.~14, no.~1--2, pp.~1--210,
2021.
\bibitem{mcmahan2017communication}
H.~B. McMahan, E.~Moore, D.~Ramage, S.~Hampson, {\em et~al.},
``Communication-efficient learning of deep networks from decentralized
data,'' in {\em Artificial Intelligence and Statistics}, pp.~1273--1282,
PMLR, 2017.
\bibitem{bonawitz2019towards}
K.~Bonawitz, H.~Eichner, W.~Grieskamp, D.~Huba, A.~Ingerman, V.~Ivanov,
C.~Kiddon, J.~Kone{\v{c}}n{\`y}, S.~Mazzocchi, B.~McMahan, {\em et~al.},
``Towards federated learning at scale: System design,'' {\em Proceedings of
the 2nd SysML Conference}, 2019.
\bibitem{mothukuri2021survey}
V.~Mothukuri, R.~M. Parizi, S.~Pouriyeh, Y.~Huang, A.~Dehghantanha, and
G.~Srivastava, ``A survey on security and privacy of federated learning,''
{\em Future Generation Computer Systems}, vol.~115, pp.~619--640, 2021.
\bibitem{xu2021federated}
J.~Xu, B.~S. Glicksberg, C.~Su, P.~Walker, J.~Bian, and F.~Wang, ``Federated
learning for healthcare informatics,'' {\em Journal of healthcare informatics
research}, vol.~5, pp.~1--19, 2021.
\bibitem{long2020federated}
G.~Long, Y.~Tan, J.~Jiang, and C.~Zhang, ``Federated learning for open
banking,'' in {\em Federated learning: privacy and incentive}, pp.~240--254,
Springer, 2020.
\bibitem{forootani2024asynchronous}
A.~Forootani and R.~Iervolino, ``Asynchronous federated learning: A scalable
approach for decentralized machine learning,'' {\em arXiv preprint
arXiv:2412.17723}, 2024.
\bibitem{yu2021toward}
R.~Yu and P.~Li, ``Toward resource-efficient federated learning in mobile edge
computing,'' {\em IEEE Network}, vol.~35, no.~1, pp.~148--155, 2021.
\bibitem{li2020federated}
T.~Li, A.~K. Sahu, A.~Talwalkar, and V.~Smith, ``Federated optimization in
heterogeneous networks,'' in {\em Proceedings of Machine Learning and Systems
2020}, pp.~429--450, 2020.
\bibitem{wang2019adaptive}
S.~Wang, T.~Tuor, T.~Salonidis, K.~K. Leung, C.~Makaya, T.~He, and K.~Chan,
``Adaptive federated learning in resource constrained edge computing
systems,'' {\em IEEE journal on selected areas in communications}, vol.~37,
no.~6, pp.~1205--1221, 2019.
\bibitem{sattler2020sparse}
F.~Sattler, S.~Wiedemann, K.-R. M{\"u}ller, and W.~Samek, ``Sparse binary
compression: Towards distributed deep learning with minimal communication,''
{\em The Journal of Machine Learning Research}, vol.~21, no.~1, pp.~174--223,
2020.
\bibitem{pang2020realizing}
J.~Pang, Y.~Huang, Z.~Xie, Q.~Han, and Z.~Cai, ``Realizing the heterogeneity: A
self-organized federated learning framework for iot,'' {\em IEEE Internet of
Things Journal}, vol.~8, no.~5, pp.~3088--3098, 2020.
\bibitem{harlap2016addressing}
A.~Harlap, H.~Cui, W.~Dai, J.~Wei, G.~R. Ganger, P.~B. Gibbons, G.~A. Gibson,
and E.~P. Xing, ``Addressing the straggler problem for iterative convergent
parallel ml,'' in {\em Proceedings of the seventh ACM symposium on cloud
computing}, pp.~98--111, 2016.
\bibitem{cipar2013solving}
J.~Cipar, Q.~Ho, J.~K. Kim, S.~Lee, G.~R. Ganger, G.~Gibson, K.~Keeton, and
E.~Xing, ``Solving the straggler problem with bounded staleness,'' in {\em
14th Workshop on Hot Topics in Operating Systems (HotOS XIV)}, 2013.
\bibitem{chai2020fedat}
Z.~Chai, Y.~Chen, L.~Zhao, Y.~Cheng, and H.~Rangwala, ``Fedat: A
communication-efficient federated learning method with asynchronous tiers
under non-iid data,'' {\em ArXivorg}, 2020.
\bibitem{chen2020asynchronous}
Y.~Chen, Y.~Ning, M.~Slawski, and H.~Rangwala, ``Asynchronous online federated
learning for edge devices with non-iid data,'' in {\em 2020 IEEE
International Conference on Big Data (Big Data)}, pp.~15--24, IEEE, 2020.
\bibitem{hsieh2020non}
K.~Hsieh, A.~Phanishayee, O.~Mutlu, and P.~B. Gibbons, ``Non-synchronous
federated learning,'' {\em arXiv preprint arXiv:2001.08288}, 2020.
\bibitem{xie2020asynchronous}
C.~Xie, S.~Koyejo, and I.~Gupta, ``Asynchronous federated optimization,'' {\em
arXiv preprint arXiv:2003.10972}, 2020.
\bibitem{karimireddy2020scaffold}
S.~P. Karimireddy, S.~Kale, M.~Mohri, S.~Reddi, S.~Stich, and A.~T. Suresh,
``Scaffold: Stochastic controlled averaging for federated learning,'' in {\em
International Conference on Machine Learning}, pp.~5132--5143, PMLR, 2020.
\bibitem{lian2017asynchronous}
X.~Lian, C.~Zhang, C.-J. Hsieh, W.~Zhang, J.~Liu, J.~T. Kwok, and I.~S.
Dhillon, ``Asynchronous decentralized parallel stochastic gradient descent,''
{\em Advances in Neural Information Processing Systems}, vol.~30, 2017.
\bibitem{wang2021adaptive}
W.~Wang, K.~Zhou, T.~Li, {\em et~al.}, ``Adaptive federated learning in
asynchronous networks,'' {\em arXiv preprint arXiv:2106.00741}, 2021.
\bibitem{backstrom2022mindthestep}
K.~B{\"a}ckstr{\"o}m, A.~Forslund, T.~N. Vu, and A.~Jakobsson,
``Mindthestep-asyncpsgd: Adaptive asynchronous parallel stochastic gradient
descent,'' {\em Advances in Neural Information Processing Systems}, 2022.
\bibitem{backstrom2022asap}
K.~B{\"a}ckstr{\"o}m, A.~Forslund, T.~N. Vu, and A.~Jakobsson, ``Asap sgd:
Instance-based adaptiveness to staleness in asynchronous sgd,'' {\em
Proceedings of the 39th International Conference on Machine Learning}, 2022.
\bibitem{dutta2018slow}
S.~Dutta, G.~Joshi, S.~Ghosh, and S.~Sra, ``Slow and stale gradients can win
the race: Error-runtime trade-offs in distributed sgd,'' {\em Proceedings of
the 22nd International Conference on Artificial Intelligence and Statistics},
2018.
\bibitem{zheng2017delay}
S.~Zheng, H.~Cui, Y.~Li, L.~Chen, A.~Smola, and J.~Liu, ``Asynchronous
stochastic gradient descent with delay compensation,'' {\em Proceedings of
the 34th International Conference on Machine Learning}, 2017.
\bibitem{zhang2022parallelization}
Y.~Zhang, L.~Xiao, T.~Wang, and S.~Wang, ``On the parallelization upper bound
for asynchronous stochastic gradient descent,'' {\em Journal of Optimization
Theory and Applications}, vol.~192, pp.~185--203, 2022.
\bibitem{async2022accelerated}
E.~Author and Others, ``Asynchronous accelerated stochastic gradient descent,''
{\em Journal of Machine Learning Research}, 2022.
\bibitem{zhou2018unbounded}
X.~Zhou, P.~Mertikopoulos, and N.~Bambos, ``Distributed asynchronous
optimization with unbounded delays: How slow can you go?,'' {\em Proceedings
of the 32nd International Conference on Neural Information Processing
Systems}, 2018.
\bibitem{lian2015nonconvex}
X.~Lian, C.~Zhang, C.-J. Hsieh, W.~Zhang, and J.~Liu, ``Asynchronous parallel
stochastic gradient for nonconvex optimization,'' {\em Advances in Neural
Information Processing Systems}, 2015.
\bibitem{keuper2015asynchronous}
J.~Keuper and F.-J. Pfreundt, ``Asynchronous parallel stochastic gradient
descent,'' {\em Proceedings of the IEEE International Parallel and
Distributed Processing Symposium Workshops}, 2015.
\bibitem{gu2021privacy}
B.~Gu, A.~Xu, Z.~Huo, C.~Deng, and H.~Huang, ``Privacy-preserving asynchronous
vertical federated learning algorithms for multiparty collaborative
learning,'' {\em IEEE Transactions on Neural Networks and Learning Systems},
2021.
\bibitem{AsyncFed}
C.~Xie, S.~Koyejo, and I.~Gupta, ``Asynchronous federated optimization,'' {\em
arXiv preprint arXiv:1903.03934}, 2019.
\bibitem{chen2018lag}
T.~Chen, G.~B. Giannakis, T.~Sun, and W.~Yin, ``Lag: Lazily aggregated gradient
for communication-efficient distributed learning,'' in {\em Advances in
Neural Information Processing Systems}, vol.~31, 2018.
\bibitem{wu2020safa}
W.~Wu, L.~He, W.~Lin, R.~Mao, C.~Maple, and S.~Jarvis, ``Safa: a
semi-asynchronous protocol for fast federated learning with low overhead,''
{\em IEEE Transactions on Computers}, vol.~70, no.~5, pp.~655--668, 2020.
\bibitem{FedBuff}
J.~Nguyen, K.~Malik, H.~Zhan, A.~Yousefpour, M.~Rabbat, M.~Malek, and D.~Huba,
``Federated learning with buffered asynchronous aggregation,'' {\em arXiv
preprint arXiv:2106.06639}, 2021.
\bibitem{FedSA}
M.~Chen, B.~Mao, and T.~Ma, ``Fedsa: A staleness-aware asynchronous federated
learning algorithm with non-iid data,'' {\em Future Generation Computer
Systems}, vol.~120, pp.~1--12, 2021.
\bibitem{xie2019asynchronous}
C.~Xie, S.~Koyejo, and I.~Gupta, ``Asynchronous federated optimization,'' {\em
arXiv preprint arXiv:1903.03934}, 2019.
\bibitem{koloskova2022sharper}
A.~Koloskova, S.~U. Stich, and M.~Jaggi, ``Sharper convergence guarantees for
asynchronous sgd for distributed and federated learning,'' in {\em Advances
in Neural Information Processing Systems}, 2022.
\bibitem{forootani_afl_emt}
A.~Forootani and R.~Iervolino, ``Asynchronous federated learning: A scalable
approach for decentralized machine learning,'' {\em arXiv preprint
arXiv:2412.17723}, 2024.
\bibitem{Adam}
D.~P. Kingma and J.~Ba, ``Adam: A method for stochastic optimization,'' {\em
arXiv preprint arXiv:1412.6980}, 2014.
\end{thebibliography}