Convergence analysis of Anderson acceleration for nonlinear equations with H\"older continuous derivatives
Extracted source text · arXiv 2507.15322
\documentclass[a4paper]{article}
\usepackage[left=3cm,right=3cm,top=2.5cm,bottom=2.5cm]{geometry}
\usepackage{amsmath,amsthm,amssymb,bm}
\usepackage{graphicx,subfigure} % about figure
\usepackage{authblk} % about author
\usepackage{cite} % citations of references
\usepackage{booktabs} % Tables with three lines
\usepackage{algorithm}
\usepackage{enumitem} % control layout of itemize, enumerate, description
\usepackage{rotating} % for rotating table
%------------------------------------------------------------------------------
\newtheorem{mytheorem}{Theorem}[section]
\newtheorem{mylemma}{Lemma}[section]
\newtheorem{mycorollary}{Corollary}[section]
\theoremstyle{definition}
\newtheorem{mydefinition}{Definition}[section]
\newtheorem{myremark}{Remark}[section]
\numberwithin{equation}{section} %
\numberwithin{figure}{section} %
\numberwithin{table}{section} %
\numberwithin{algorithm}{section}
%------------------------------------------------------------------------------
\newcommand{\myvec}[1]{\textup{\textbf{#1}}}
\newcommand{\dif}{\mathop{}\!\mathrm{d}}
\newcommand{\RS}{\mathbb{R}}
\newcommand{\NS}{\mathbb{N}}
\newcommand{\DS}{\mathbb{D}}
\newcommand{\diag}{\mathrm{diag}\,} % diagonal matrix
\newcommand{\ball}{\bm{\mathrm{B}}} % open ball
\newcommand{\mylim}{\lim\limits}
\newcommand{\mysum}{\sum\limits}
\newcommand{\mytran}[1]{{#1}^{\top}}
%------------------------------------------------------------------------------
\begin{document}
\title{Convergence analysis of Anderson acceleration for nonlinear equations
with H\"older continuous derivatives}
\author{
Yonghui Ling\thanks{Corresponding author. \textit{E-mail address:} \texttt{yhling@mnnu.edu.cn}}, \
Zikang Xiong and Juan Liang
\and
Department of Mathematics, Minnan Normal University, Zhangzhou 363000, China
}
\maketitle
%------------------------------------------------------------------------------
\begin{abstract}
This work investigates the local convergence behavior of Anderson acceleration in solving nonlinear systems.
We establish local R-linear convergence results for Anderson acceleration with general depth $m$
under the assumptions that the Jacobian of the nonlinear operator is H\"older continuous
and the corresponding fixed-point function is contractive.
In the Lipschitz continuous case, we obtain a sharper R-linear convergence factor.
We also derive a refined residual bound for the depth $m = 1$
under the same assumptions used for the general depth results.
Applications to a nonsymmetric Riccati equation from transport theory demonstrate that
Anderson acceleration yields comparable results to several existing fixed-point methods
for the regular cases, and that it brings significant reductions
in both the number of iterations and computation time,
even in challenging cases involving nearly singular or large-scale problems.
\textbf{Keywords:} Anderson acceleration, local convergence, H\"older continuity, algebraic Riccati equation
\textbf{MSC:} 65H10, 15A24
\end{abstract}
%------------------------------------------------------------------------------
\section{Introduction}
\label{sec:Introduction}
Our aim in this paper is to study convergence acceleration techniques
for nonlinear fixed-point iteration of the form
\begin{equation}
\label{it:FixedPoint}
\myvec{x}_{k+1} = \myvec{g}(\myvec{x}_k), \quad k = 0,1,2,\ldots,
\end{equation}
which is commonly used in solving the solution of nonlinear system
\begin{equation}
\label{eq:f(x)=0}
\myvec{f}(\myvec{x}) = \myvec{g}(\myvec{x}) - \myvec{x} = \myvec{0},
\end{equation}
where $\myvec{g}: \DS \subset \RS^n \to \DS$ is a given Fr\'echet continuously differentiable nonlinear operator,
$\DS$ is an open convex subset of $\RS^n$.
The fixed-point iteration \eqref{it:FixedPoint},
also referred to as the nonlinear Richardson or Picard iteration,
is a fundamental method for solving the fixed-point problem $\myvec{g}(\myvec{x}) = \myvec{x}$.
However, the sequence generated by \eqref{it:FixedPoint} often exhibits slow convergence
or may even fail to converge in certain cases.
Consequently, achieving faster convergence rates has been a central focus
in optimization and numerical analysis communities.
Extrapolation methods \cite{Anderson2019,BrezinskiRS2018,BrezinskiR2019}
provide effective strategies for accelerating the convergence of iterative sequences.
A classical example is Anderson acceleration,
also referred to as Pulay mixing or direct inversion in the iterative subspace
in quantum chemistry and physics communities \cite{Pulay1980}.
Anderson acceleration, originally proposed in \cite{Anderson1965},
has received increasing interest over the past decade as a powerful technique
for accelerating fixed-point iterations \eqref{it:FixedPoint}.
Initially designed to solve nonlinear integral equations,
Anderson acceleration has since demonstrated remarkable efficiency
in accelerating convergence,
particularly for problems arising from discretized partial differential equations
\cite{LipnikovSV2013,AnJW2017,PollockRX2019,PollockR2021,RebholzVX2021,PollockR2023,PollockRTX2025},
smooth and nonsmooth optimization
\cite{ChenK2019,BianCK2021,BianC2022,ScieurAB2020,FuZB2020,ZhangDB2020,DeSterckH2021,BarreTDA2022,BollapragadaSD2023,OuyangLM2024},
data analysis \cite{HighamS2016,Bach2021},
and machine learning \cite{MaiJ2020,WangHS2021,PasiniYRS2021,PasiniYRS2022}.
The underlying idea of Anderson acceleration is to generate new iterates
by constructing an optimized linear combination of the previous iterates
and their corresponding residuals, thereby significantly improving the rate of convergence.
The following algorithm describes the use of Anderson acceleration to solve
the fixed-point problem $\myvec{g}(\myvec{x}) = \myvec{x}$.
The parameter $m$ is referred to as the depth or window-size of Anderson acceleration,
and the coefficients $\alpha_j^{(k)}$ are called acceleration parameters.
\begin{algorithm}
\caption{Anderson acceleration for solving \eqref{eq:f(x)=0}}
\label{alg:AA}
Given the depth $m \in \NS$. Choose an initial point $\myvec{x}_0 \in \RS^n$
and set $\myvec{x}_1 = \myvec{g}(\myvec{x}_0)$.
For $k = 1,2,\ldots$ until convergence, do:
\begin{itemize}[leftmargin=1em,itemindent=3.5em,parsep=0em,itemsep=0em,topsep=0em]
\item[Step 1.]
Set $m_k = \min\{m,k\}$.
\item[Step 2.]
Compute $\myvec{f}_k \triangleq \myvec{f}(\myvec{x}_k) = \myvec{g}(\myvec{x}_k) - \myvec{x}_k$.
\item[Step 3.]
Solve the convex optimization problem
\begin{equation}
\label{optim:alpha}
\min_{\bm{\alpha}_k = \mytran{(\alpha_0^{(k)},\ldots,\alpha_{m_k}^{(k)})}}
\left\|\sum_{j=0}^{m_k} \alpha_j^{(k)}\myvec{f}_{k-m_k+j}\right\| \quad \text{s.t.} \quad
\sum_{j=0}^{m_k} \alpha_j^{(k)} = 1.
\end{equation}
\item[Step 4.]
Set $\myvec{x}_{k+1} = \mysum_{j=0}^{m_k} \alpha_j^{(k)} \myvec{g}(\myvec{x}_{k-m_k+j})$.
\end{itemize}
\end{algorithm}
The update rule in Step 4 of Algorithm \ref{alg:AA} can be
extended to the general Anderson mixing form:
$$
\myvec{x}_{k+1} = (1 - \beta_k) \sum_{j=0}^{m_k} \alpha_j^{(k)} \myvec{x}_{k-m_k+j}
+ \beta_k \sum_{j=0}^{m_k} \alpha_j^{(k)} \myvec{g}(\myvec{x}_{k-m_k+j}),
$$
where $\beta_k \in (0,1]$ is a damping parameter.
In this work, we focus on the undamped case $\beta_k \equiv 1$,
which has received significant attention for its theoretical advantages
in convergence analysis
\cite{TothK2015,ChenK2019,BianCK2021,BianC2022,PollockRX2019,OuyangLM2024}.
Recent developments on the damped case can be found in
\cite{EvansPRX2020,PollockR2021,RebholzX2023,PollockR2023}.
When applied to linearly converging fixed-point iterations,
the minimization step in Anderson acceleration often helps to
improve both convergence rates and robustness.
This observation has motivated the development of convergence analysis
for Anderson acceleration under minimization-based frameworks,
with early results for contractive mappings \cite{TothK2015,ChenK2019}
and further generalizations to include noncontractive cases \cite{PollockR2021}.
However, acceleration is not guaranteed
when the underlying iterations are already quadratically convergent,
as noted in \cite{EvansPRX2020}.
The behavior of Anderson acceleration has been further investigated
by examining its connection to other nonlinear solvers.
It has been demonstrated \cite{FangS2009} that Anderson acceleration
is equivalent to a specialized version of the generalized Broyden method.
In this formulation,
the approximate inverse Jacobian is computed implicitly
by solving a constrained optimization problem that satisfies secant conditions
on recent iterates.
This connection motivates the study of Anderson acceleration
using tools from the convergence theory of quasi-Newton methods.
We refer to \cite{Kelley2018,Saad2025} for comprehensive reviews on this topic.
While Anderson acceleration had demonstrated good numerical performance
in various applications,
its first rigorous local convergence result was provided by Toth and Kelley \cite{TothK2015} in 2015.
They proved that if the nonlinear operator $\myvec{g}$ is continuously differentiable
with Lipschitz continuous derivative,
then the sequence generated by Algorithm \ref{alg:AA}
converges R-linearly (see Definition \ref{def:R-Linear}) to a fixed point,
provided the initial guess is sufficiently close.
In particular, when the Euclidean norm is used and the depth $m = 1$,
the sequence converges Q-linearly (see Definition \ref{def:Q-Linear}).
An improved version of the local convergence result was later established by Kelley \cite{Kelley2018}.
This was further developed by Chen and Kelley \cite{ChenK2019},
who both relaxed the required assumptions and improved the R-linear convergence factor.
Bian et al. \cite{BianCK2021,BianC2022} then extended the convergence results
to the case of nonsmooth fixed-point problems.
Subsequent efforts \cite{PollockRX2019,EvansPRX2020,PollockR2021}
provided a comprehensive resolution to the question of
how Anderson acceleration improves the convergence of linearly converging fixed-point iterations.
Specifically, the study in \cite{PollockRX2019} revealed that the linear convergence of fixed-point iterations
applied to the steady Navier-Stokes equations is improved by Anderson acceleration
via the gain factor from the underlying optimization.
Evans et al. \cite{EvansPRX2020} subsequently generalized to general contractive mappings,
and Pollock and Rebholz \cite{PollockR2021} further extended
and refined the convergence results to include noncontractive mappings.
Recently, Rebholz and Xiao \cite{RebholzX2023} showed that Anderson acceleration
may reduce the order of convergence for superlinearly converging methods, but
it significantly accelerates the convergence for sublinearly converging methods.
The convergence analyses in \cite{TothK2015,EvansPRX2020,PollockR2021}
rely on the assumption that the derivative of the nonlinear operator is Lipschitz continuous.
To generalize these results and extend the applicability of Anderson acceleration,
a natural approach is to weaken this assumption to H\"older continuity.
This relaxation is particularly motivated by optimization theory,
where H\"older continuity provides a bridging framework between smooth and nonsmooth problems
\cite{Nesterov2015,GrapigliaN2017,CartisGT2019,MarumoT2025}.
More specifically, a H\"older exponent of zero indicates a bounded derivative,
an exponent in $(0,1)$ corresponds to a continuous but potentially non-differentiable derivative,
while an exponent of $1$ represents a Lipschitz continuous derivative,
which is again differentiable.
The goal of this paper is to establish local convergence results for Anderson acceleration
applied to the nonlinear system \eqref{eq:f(x)=0},
assuming that the nonlinear operator $\myvec{f}$ has a H\"older continuous Jacobian.
Our main theoretical contribution is to show that,
when the first derivative of the nonlinear operator $\myvec{f}$ is H\"older continuous
and the associated fixed-point function $\myvec{g}$ is contractive,
the sequence generated by Algorithm \ref{alg:AA} with general depth $m \geq 1$
converges R-linearly to the solution $\myvec{x}^*$ of the nonlinear system \eqref{eq:f(x)=0}.
In the special case where the Jacobian is Lipschitz continuous,
we provide an implicit characterization of the R-factor
in terms of the contraction factor, the gain of the optimization problem,
and the condition number of the Jacobian at $\myvec{x}^*$.
Additionally, we obtain a specific convergence rate for Anderson acceleration
with depth $m = 1$ under the same assumptions used for the general depth results.
We further apply our theoretical framework to compute an approximation
of minimal positive solution of a nonsymmetric algebraic Riccati equation (NARE)
arising from transport theory.
Comprehensive numerical results demonstrate that Anderson acceleration
substantially outperforms several existing fixed-point methods,
yielding significant reductions in
both the number of iterations and computation time,
even in challenging cases involving nearly singular or large-scale problems.
The remainder of the paper is organized as follows.
In Section \ref{sec:Preliminaries},
we introduce the necessary notation and present preliminary results for the convergence analysis.
Section \ref{sec:ConvResults} contains our main theoretical results
on the local convergence behavior of Anderson acceleration.
We consider a nonsymmetric algebraic Riccati equation
to illustrate the effectiveness of Anderson acceleration in Section \ref{sec:Application}.
Finally, we conclude the paper in Section \ref{sec:Conclusions}.
%------------------------------------------------------------------------------
\section{Preliminaries}
\label{sec:Preliminaries}
Throughout this paper, vectors are columns by default and are denoted by bold lowercase letters, e.g., $\myvec{v}$,
while matrices are denoted by regular uppercase letters, e.g., $V$, which is clear from the context.
We use $\diag(\myvec{v})$ to denote the diagonal matrix with the vector $\myvec{v}$ on its diagonal,
and use $I$ to denote the identity operator or the identity matrix with proper dimension.
If there is potential confusion, we will use $I_n$ to denote identity matrix of dimension $n$.
The symbol $\myvec{e}_i = (0,\ldots,0,\underset{i}{1},0,\ldots,0)^{\top} \in \RS^n$ is $i$th column of the identity matrix $I_n$.
Let $\myvec{e} = (1,1,\ldots,1)^{\top}$ with proper dimension.
For a square nonsingular matrix $A \in \RS^{n \times n}$, we use $\kappa(A)$ to denote the condition number of $A$.
For $\myvec{x} \in \RS^n$ and real number $r > 0$, we use $\ball(\myvec{x},r)$
to stand for the open ball with center $\myvec{x}$ and radius $r$.
Recall that the Jacobian matrix of a continuously differentiable nonlinear operator
$\myvec{h}: \RS^n \to \RS^n$ at point $\myvec{x} \in \RS^n$ is represented by $\myvec{h}'(\myvec{x})$.
We recall the notions of Q- and R-order of convergence for a convergent sequence $\{\myvec{x}_k\}$ in $\RS^n$.
For more details about these two notions,
see \cite{Jay2001,Potra1989} or the recent review paper \cite{Catinas2021} and the references therein.
\begin{mydefinition}
\label{def:Q-Linear}
A sequence $\{\myvec{x}_k\}$ is said to converge to $\myvec{x}^* \in \RS^n$
with Q-order (at least) $q \geq 1$ if there exist constants $c \geq 0$ and $N \geq 0$
such that
$$
\|\myvec{x}_{k+1} - \myvec{x}^*\| \leq c\|\myvec{x}_k - \myvec{x}^*\|^q
\quad \text{for all} \quad k \geq N.
$$
In the special case $q = 1$, assuming $c < 1$, we said that
the sequence $\{\myvec{x}_k\}$ converges Q-linearly to $\myvec{x}^*$.
\end{mydefinition}
\begin{mydefinition}
\label{def:R-Linear}
A sequence $\{\myvec{x}_k\}$ is said to converge to $\myvec{x}^* \in \RS^n$
with R-order (at least) $q \geq 1$ if there exists a positive real sequence $\{t_k\}$
converging to zero with Q-order at least $q$ such that
$\|\myvec{x}_k - \myvec{x}^*\| \leq t_k$.
When $q = 1$, we said that the sequence $\{\myvec{x}_k\}$ converges R-linearly to $\myvec{x}^*$.
\end{mydefinition}
The notion about H\"older continuous functions is as follows.
\begin{mydefinition}
Let $\myvec{f}: \DS \subset \RS^n \to \RS^n$ be a Fr\'echet continuously differentiable operator,
$\DS$ open and convex.
We say that the Jacobian $\myvec{f}'$ is H\"older continuous with exponent $\nu \in (0,1]$
if there exists a constant $H_\nu > 0$ such that
\begin{equation}
\label{cond:Holder}
\|\myvec{f}'(\myvec{x}) - \myvec{f}'(\myvec{y})\| \leq H_{\nu}\|\myvec{x} - \myvec{y}\|^{\nu}
\quad \text{for all} \quad \myvec{x},\myvec{y} \in \DS.
\end{equation}
\end{mydefinition}
It should be noted that the operator $\myvec{f}$ determines $H_\nu$ for each $\nu \in (0,1]$,
while $\nu$ is not a constant determined by $\myvec{f}$.
In practice, determining the H\"older constant $H_\nu$ of a real-world operator
for a given $\nu \in (0,1]$ is often a challenging problem.
H\"{o}lder continuity, which generalizes Lipschitz continuity,
has been extensively applied in complexity analyses of optimization methods
\cite{DevolderGN2014,Nesterov2015,CartisGT2017,CartisGT2019,GrapigliaN2017,GhadimiLZ2019,GrapigliaN2019,GrapigliaN2020,MarumoT2025}.
Moreover, classical convergence results for Newton's method
under H\"{o}lder-type continuous have been developed in
\cite{Rokne1972,Argyros1992,Hernandez2001,Huang2002,Huang2004,LiS2008,ShenL2008}
for solving general nonlinear operator equations in Banach spaces.
The equation $\myvec{g}(\myvec{x}) = \myvec{x}$,
an alternative formulation of the nonlinear equation \eqref{eq:f(x)=0},
is widely known as the fixed-point problem.
A point $\myvec{x}^*$ satisfying $\myvec{g}(\myvec{x}^*) = \myvec{x}^*$ is called a fixed point of $\myvec{g}$.
The operator $\myvec{g}$ is referred to as the fixed-point function.
It is worth noting that if the nonlinear operator $\myvec{f}$ is H\"older continuous with exponent $\nu \in (0,1]$,
then the fixed-point function $\myvec{g}$ is also H\"older continuous with the same exponent $\nu$.
Indeed, for any $\myvec{x}, \myvec{y} \in \DS$, we have
$$
\|\myvec{g}'(\myvec{x}) - \myvec{g}'(\myvec{y})\|
= \|(\myvec{g}'(\myvec{x}) - I) - (\myvec{g}'(\myvec{y}) - I)\|
= \|\myvec{f}'(\myvec{x}) - \myvec{f}'(\myvec{y})\|
\leq H_\nu \|\myvec{x} - \myvec{y}\|^{\nu}.
$$
We said that the fixed-point function $\myvec{g}$ is contractive
if there exists a constant $\theta \in (0,1)$ such that
\begin{equation}
\label{cond:Contractive}
\|\myvec{g}(\myvec{x}) - \myvec{g}(\myvec{y})\| \leq \theta \|\myvec{x} - \myvec{y}\|
\end{equation}
holds for all $\myvec{x}, \myvec{y} \in \DS$.
The constant $\theta$ is referred the contraction factor of $\myvec{g}$.
This condition implies that
\begin{equation}
\label{cond:Contractive2}
\|\myvec{g}'(\myvec{x})\| \leq \theta < 1
\end{equation}
holds for any $\myvec{x} \in \DS$.
It is guaranteed by the contraction mapping theorem \cite{OrtegaR1970}
that $\myvec{g}$ has a unique fixed point $\myvec{x}^* \in \DS$,
which is the unique solution of the nonlinear system \eqref{eq:f(x)=0}.
The following lemma will be used in the convergence analysis of Anderson acceleration.
\begin{mylemma}
\label{lem:norm_f(x)}
Assume that there is $\myvec{x}^* \in \RS^n$ such that $\myvec{f}(\myvec{x}^*) = \myvec{0}$
and $\myvec{f}'(\myvec{x}^*)^{-1}$ exists.
If the Jacobian $\myvec{f}'$ is H\"older continuous with exponent $\nu$
in $\ball(\myvec{x}^*,r_\nu)$, where
\begin{equation}
\label{cons:r_nu}
r_\nu = \left(\frac{1}{H_\nu \|\myvec{f}'(\myvec{x}^*)^{-1}\|}\right)^{1/\nu},
\end{equation}
then for any $\myvec{x} \in \ball(\myvec{x}^*,r_\nu)$ we have
\begin{equation}
\label{ineq:norm_f(x)}
\frac{\nu}{(1+\nu)\|\myvec{f}'(\myvec{x}^*)^{-1}\|} \|\myvec{x} - \myvec{x}^*\|
\leq \|\myvec{f}(\myvec{x})\|
\leq \frac{(2+\nu)\|\myvec{f}'(\myvec{x}^*)\|}{1+\nu} \|\myvec{x} - \myvec{x}^*\|.
\end{equation}
\end{mylemma}
\begin{proof}
Take $\myvec{x} \in \ball(\myvec{x}^*,r_\nu)$. Since
\begin{align*}
\myvec{f}(\myvec{x})
&= \myvec{f}(\myvec{x}) - \myvec{f}(\myvec{x}^*)
= \int_0^1 \myvec{f}'(\myvec{x}^* + t(\myvec{x} - \myvec{x}^*)) (\myvec{x} - \myvec{x}^*)\dif{t} \\
&= \int_0^1 [\myvec{f}'(\myvec{x}^* + t(\myvec{x} - \myvec{x}^*)) - \myvec{f}'(\myvec{x}^*)] (\myvec{x} - \myvec{x}^*)\dif{t}
+ \myvec{f}'(\myvec{x}^*) (\myvec{x} - \myvec{x}^*),
\end{align*}
it follows from the H\"older condition \eqref{cond:Holder} that
\begin{align*}
\|\myvec{f}(\myvec{x})\|
&\leq \int_0^1 \|\myvec{f}'(\myvec{x}^* + t(\myvec{x} - \myvec{x}^*)) - \myvec{f}'(\myvec{x}^*)\| \|\myvec{x} - \myvec{x}^*\|\dif{t}
+ \|\myvec{f}'(\myvec{x}^*)\| \|\myvec{x} - \myvec{x}^*\| \\
&\leq H_\nu \int_0^1 t^\nu\|\myvec{x} - \myvec{x}^*\|^{1+\nu} \dif{t} + \|\myvec{f}'(\myvec{x}^*)\| \|\myvec{x} - \myvec{x}^*\| \\
&= \left(\frac{H_\nu}{1+\nu} \|\myvec{x} - \myvec{x}^*\|^\nu + \|\myvec{f}'(\myvec{x}^*)\|\right) \|\myvec{x} - \myvec{x}^*\|.
\end{align*}
Noting that $\|\myvec{x} - \myvec{x}^*\| < r_\nu$, we obtain
$$
\frac{H_\nu}{1+\nu} \|\myvec{x} - \myvec{x}^*\|^\nu
< \frac{1}{(1+\nu)\|\myvec{f}'(\myvec{x}^*)^{-1}\|}
\leq \frac{\|\myvec{f}'(\myvec{x}^*)\|}{1+\nu}.
$$
This leads to
$$
\|\myvec{f}(\myvec{x})\|
\leq \left(\frac{2+\nu}{1+\nu}\right) \|\myvec{f}'(\myvec{x}^*)\| \|\myvec{x} - \myvec{x}^*\|.
$$
On the other hand, we apply standard analytical techniques to deduce that
\begin{align*}
\myvec{f}'(\myvec{x}^*)^{-1}\myvec{f}(\myvec{x})
&= \myvec{f}'(\myvec{x}^*)^{-1}
\int_0^1 \myvec{f}'(\myvec{x}^* + t(\myvec{x} - \myvec{x}^*)) (\myvec{x} - \myvec{x}^*)\dif{t} \\
&= (\myvec{x} - \myvec{x}^*) -
\int_0^1 \myvec{f}'(\myvec{x}^*)^{-1}[\myvec{f}'(\myvec{x}^*)
- \myvec{f}'(\myvec{x}^* + t(\myvec{x} - \myvec{x}^*))](\myvec{x} - \myvec{x}^*)\dif{t}.
\end{align*}
Then, the H\"{o}lder condition \eqref{cond:Holder} can be applied again to yield
\begin{align*}
\|\myvec{f}'(\myvec{x}^*)^{-1}\myvec{f}(\myvec{x})\|
&\geq \|\myvec{x} - \myvec{x}^*\| -
H_\nu\|\myvec{f}'(\myvec{x}^*)^{-1}\| \int_0^1 t^\nu\|\myvec{x} - \myvec{x}^*\|^{1+\nu}\dif{t}\\
&= \left(1 - \frac{H_\nu}{1+\nu}\|\myvec{f}'(\myvec{x}^*)^{-1}\| \|\myvec{x} - \myvec{x}^*\|^\nu\right)
\|\myvec{x} - \myvec{x}^*\|.
\end{align*}
Thus, we have
\begin{align*}
\|\myvec{f}(\myvec{x})\|
\geq \frac{\|\myvec{f}'(\myvec{x}^*)^{-1}\myvec{f}(\myvec{x})\|}{\|\myvec{f}'(\myvec{x}^*)^{-1}\|}
&\geq \left(\frac{1}{\|\myvec{f}'(\myvec{x}^*)^{-1}\|} - \frac{H_\nu\|\myvec{x} - \myvec{x}^*\|^\nu}{1+\nu}\right)
\|\myvec{x} - \myvec{x}^*\| \\
&\geq \left(\frac{\nu}{(1+\nu)\|\myvec{f}'(\myvec{x}^*)^{-1}\|}\right)\|\myvec{x} - \myvec{x}^*\|.
\end{align*}
With this, the proof of the lemma is complete.
\end{proof}
\begin{myremark}
By applying the Banach lemma \cite{OrtegaR1970} with standard techniques, as presented in \cite{Ferreira2009},
we can show that the H\"older continuity of $\myvec{f}'$ with exponent $\nu$
in $\ball(\myvec{x}^*,r_\nu)$ guarantees that $\myvec{f}'(\myvec{x})$ is nonsingular for any $\myvec{x} \in \ball(\myvec{x}^*,r_\nu)$,
where $r_\nu$ is given by \eqref{cons:r_nu}.
\end{myremark}
Applying the above lemma, we arrive at the following result.
\begin{mylemma}
Let the assumptions of Lemma $\ref{lem:norm_f(x)}$ hold.
If $\myvec{x}_0 \in \ball(\myvec{x}^*,r_\nu)$, then for any $\myvec{x} \in \ball(\myvec{x}^*,r_\nu)$,
we have
\begin{equation}
\label{ineq:norm_f(x)/f(x0)}
\frac{\nu}{(2+\nu)\kappa(\myvec{f}'(\myvec{x}^*))}
\cdot \frac{\|\myvec{x} - \myvec{x}^*\|}{\|\myvec{x}_0 - \myvec{x}^*\|}
\leq \frac{\|\myvec{f}(\myvec{x})\|}{\|\myvec{f}(\myvec{x}_0)\|}
\leq \frac{(2+\nu)\kappa(\myvec{f}'(\myvec{x}^*))}{\nu}
\cdot \frac{\|\myvec{x} - \myvec{x}^*\|}{\|\myvec{x}_0 - \myvec{x}^*\|}.
\end{equation}
\end{mylemma}
The following lemma is also used in the convergence analysis of Anderson acceleration.
The proof is straightforward and hence omitted.
\begin{mylemma}
\label{lem:q}
Let $m$ be a positive integer and $\tau \in (0,1)$.
If $0 \leq \zeta < 1 - \tau$, then the equation
\begin{equation}
\label{eq:q}
q^{m+1} - \tau q^m - \zeta = 0
\end{equation}
has a unique root in the open interval $\big(m\tau/(m+1),1\big)$.
\end{mylemma}
In \cite{TothK2015}, a key idea in establishing convergence is to assume that the inequality
$$
\left\|\sum_{j=0}^{m_k} \alpha_j^{(k)} \myvec{f}(\myvec{x}_{k-m_k+j})\right\| \leq \|\myvec{f}(\myvec{x}_k)\|
$$
holds when $\{\alpha_j^{(k)}\}_{j=0}^{m_k}$ is the solution of the optimization problem \eqref{optim:alpha}.
This assumption is relaxed in \cite{ChenK2019},
where it is only required that norm of the linear combination of residuals does not exceed
that of the most recent residual.
To understand how Anderson acceleration achieves faster convergence,
Pollock et al. introduced in \cite{PollockRX2019} the following optimization gain
\begin{equation}
\label{cons:eta_k}
\eta_k := \frac{\left\|\mysum_{j=0}^{m_k} \alpha_j^{(k)} \myvec{f}(\myvec{x}_{k-m_k+j})\right\|}{\|\myvec{f}(\myvec{x}_k)\|}.
\end{equation}
This quantity, satisfying $0 \leq \eta_k \leq 1$, was subsequently employed in \cite{EvansPRX2020,PollockR2021,RebholzX2023,PollockR2023}
to demonstrate that Anderson acceleration improves linear convergence rate of fixed-point iterations.
Using the optimization gain, we establish in this work the local convergence of Anderson acceleration
for depth $m=1$, assuming H\"older continuity of the derivative.
%------------------------------------------------------------------------------
\section{Convergence results}
\label{sec:ConvResults}
This section is devoted to the local convergence analysis of Anderson acceleration
for solving the nonlinear system \eqref{eq:f(x)=0} with H\"older continuous derivatives.
%------------------------------------------------------------------------------
\subsection{Local convergence}
We first present a local convergence result for Anderson acceleration with general depths $m$.
The following theorem establishes that the sequence $\{\myvec{x}_k\}$ generated by Algorithm \ref{alg:AA}
converges R-linearly to the solution $\myvec{x}^*$ of the nonlinear system \eqref{eq:f(x)=0}
under the assumptions that the Jacobian $\myvec{f}'$ is H\"older continuous with exponent $\nu \in (0,1]$
and the fixed-point function $\myvec{g}$ is contractive with factor $\theta \in (0,1)$
in a prescribed ball of the solution $\myvec{x}^*$.
\begin{mytheorem}
\label{th:LocalConvAnderson(m)}
Assume that there is an $\myvec{x}^* \in \RS^n$ such that $\myvec{f}(\myvec{x}^*) = \myvec{0}$
and $\myvec{f}'(\myvec{x}^*)$ is nonsingular. Suppose that:
\begin{enumerate}[label=\textup{(\roman*)}]
\item There is a constant $M_\alpha$ such that $\mysum_{j=0}^{m_k} |\alpha_j^{(k)}| \leq M_\alpha$ for all $k \geq 0$.
\item The Jacobian $\myvec{f}'$ is H\"older continuous with exponent $\nu \in (0,1]$ in $\ball(\myvec{x}^*,r)$,
where $r = \min\{r_\nu,\hat{r}_\nu\}$, $r_\nu$ is defined in \eqref{cons:r_nu} and
\begin{equation}
\label{cons:hat_r_nu}
\hat{r}_\nu = \left(\frac{(1-\theta)\nu}{M_\alpha(1 + M_\alpha^\nu)}\right)^{1/\nu}
\cdot\frac{\nu r_\nu}{(2+\nu)\kappa(\myvec{f}'(\myvec{x}^*))}.
\end{equation}
\item The fixed-point function $\myvec{g}$ is contractive with factor $\theta \in (0,1)$ in $\ball(\myvec{x}^*,r)$.
\end{enumerate}
Let $\{\myvec{x}_k\}$ be the sequence generated by Algorithm $\ref{alg:AA}$
with starting point $\myvec{x}_0 \in \ball(\myvec{x}^*,r)$.
Set $\tau := \theta\eta_k$ and
\begin{equation}
\label{cons:zeta}
\zeta := \frac{(2+\nu)^\nu H_\nu M_\alpha (1 + M_\alpha^\nu)}{\nu^{1+\nu}}
\cdot\kappa(\myvec{f}'(\myvec{x}^*)) \|\myvec{f}'(\myvec{x}^*)^{-1}\|\|\myvec{x}_0 - \myvec{x}^*\|^\nu,
\end{equation}
where $\eta_k$ is the optimization gain defined by \eqref{cons:eta_k}.
If $\zeta < 1 - \tau$ and
$$
\frac{2+\nu}{\nu} qM_\alpha \kappa(\myvec{f}'(\myvec{x}^*)) \leq 1,
$$
where $q$ is the unique root of equation \eqref{eq:q} in the interval $\big(m\tau/(m+1),1\big)$,
then the sequence $\{\myvec{x}_k\}$ is contained in $\ball(\myvec{x}^*,r)$
and converges R-linearly to the solution $\myvec{x}^*$
in the sense that
\begin{equation}
\label{ineq:ConvRate}
\limsup_{k \to \infty} \left(\frac{\|\myvec{f}(\myvec{x}_{k})\|}{\|\myvec{f}(\myvec{x}_k)\|}\right)^{1/k} \leq q.
\end{equation}
\end{mytheorem}
\begin{proof}
We proceed by induction. The assumption on the history that
$\|\myvec{f}(\myvec{x}_{\ell})\| \leq q^\ell\|\myvec{f}(\myvec{x}_0)\|$
leads to \eqref{ineq:ConvRate} being satisfied for $0 \leq k \leq m$.
We now assume that $\myvec{x}_k \in \ball(\myvec{x}^*,r)$
and \eqref{ineq:ConvRate} holds for all $0\leq \ell \leq k$ with $k > m$. Set
$$
\myvec{z}_k := \sum_{j=0}^{m_k} \alpha_j^{(k)} \myvec{x}_{k-m_k+j}.
$$
We first show that $\myvec{z}_k \in \ball(\myvec{x}^*,r)$. In fact, since
$$
\myvec{z}_k - \myvec{x}^*
= \sum_{j=0}^{m_k} \alpha_j^{(k)} \myvec{x}_{k-m_k+j} - \myvec{x}^*
= \sum_{j=0}^{m_k} \alpha_j^{(k)} (\myvec{x}_{k-m_k+j} - \myvec{x}^*),
$$
we use the inductive hypothesis and \eqref{ineq:norm_f(x)} to yield
\begin{align*}
\|\myvec{z}_k - \myvec{x}^*\|
&\leq \sum_{j=0}^{m_k} |\alpha_j^{(k)}| \|\myvec{x}_{k-m_k+j} - \myvec{x}^*\| \\
&\leq \frac{1+\nu}{\nu}\|\myvec{f}'(\myvec{x}^*)^{-1}\| \sum_{j=0}^{m_k} |\alpha_j^{(k)}| \|\myvec{f}(\myvec{x}_{k-m_k+j})\| \\
&\leq \frac{1+\nu}{\nu}\|\myvec{f}'(\myvec{x}^*)^{-1}\| \sum_{j=0}^{m_k} |\alpha_j^{(k)}| \cdot q^{k-m_k+j}\|\myvec{f}(\myvec{x}_0)\|.
\end{align*}
In light of the fact that $k - m_k + j = k - \min\{m,k\} + j \geq k - m$,
we further obtain that
\begin{align}
\|\myvec{z}_k - \myvec{x}^*\|
&\leq \frac{1+\nu}{\nu}\|\myvec{f}'(\myvec{x}^*)^{-1}\| \sum_{j=0}^{m_k} |\alpha_j^{(k)}| \cdot q^{k-m}\|\myvec{f}(\myvec{x}_0)\| \label{ineq:norm_zk-x*} \\
&\leq \frac{2+\nu}{\nu}M_\alpha \kappa(\myvec{f}'(\myvec{x}^*)) q^{k-m} \|\myvec{x}_0 - \myvec{x}^*\| \nonumber\\
&\leq \frac{2+\nu}{\nu}q M_\alpha \kappa(\myvec{f}'(\myvec{x}^*)) \|\myvec{x}_0 - \myvec{x}^*\|
\leq \|\myvec{x}_0 - \myvec{x}^*\|. \nonumber
\end{align}
This means that $\myvec{z}_k \in \ball(\myvec{x}^*,r)$.
Next, by using the contractivity of the fixed-point function $\myvec{g}$, one has that
\begin{align*}
\|\myvec{x}_{k+1} - \myvec{x}^*\|
&= \left\|\sum_{j=0}^{m_k} \alpha_j^{(k)} \big[\myvec{g}(\myvec{x}_{k-m_k+j}) - \myvec{g}(\myvec{x}^*)\big]\right\| \\
&\leq \theta \sum_{j=0}^{m_k} |\alpha_j^{(k)}| \|\myvec{x}_{k-m_k+j} - \myvec{x}^*\| \\
&< \sum_{j=0}^{m_k} |\alpha_j^{(k)}| \|\myvec{x}_{k-m_k+j} - \myvec{x}^*\|
\leq \|\myvec{x}_0 - \myvec{x}^*\|,
\end{align*}
which gives that $\myvec{x}_{k+1} \in \ball(\myvec{x}^*,r)$.
To estimate the bound of $\|\myvec{f}(\myvec{x}_{k+1})\|$, we notice that
\begin{equation}
\label{eq:fk+1}
\myvec{f}(\myvec{x}_{k+1}) = \myvec{g}(\myvec{x}_{k+1}) - \myvec{x}_{k+1}
= \big[\myvec{g}(\myvec{x}_{k+1}) - \myvec{g}(\myvec{z}_k)\big]
+ \big[\myvec{g}(\myvec{z}_k) - \myvec{x}_{k+1}\big].
\end{equation}
On the one hand, we have from the contractivity of $\myvec{g}$ that
\begin{align}
\|\myvec{g}(\myvec{x}_{k+1}) - \myvec{g}(\myvec{z}_k)\|
&\leq \theta \|\myvec{x}_{k+1} - \myvec{z}_k\| \nonumber\\
&= \theta \left\|\sum_{j=0}^{m_k} \alpha_j^{(k)}[\myvec{g}(\myvec{x}_{k-m_k+j}) - \myvec{x}_{k-m_k+j}]\right\| \nonumber\\
&= \theta \left\|\sum_{j=0}^{m_k} \alpha_j^{(k)} \myvec{f}(\myvec{x}_{k-m_k+j})\right\| \nonumber \\
&= \theta\eta_k \|\myvec{f}(\myvec{x}_k)\|. \label{ineq:norm_g(xk+1)-g(z_k)}
\end{align}
On the other hand, since
\begin{align*}
\myvec{g}(\myvec{z}_k)
&= \myvec{g}(\myvec{z}_k) - \myvec{g}(\myvec{x}^*) + \myvec{g}(\myvec{x}^*) \\
&= \myvec{g}(\myvec{x}^*) + \myvec{g}'(\myvec{x}^*) (\myvec{z}_k - \myvec{x}^*)
+ \int_0^1 [\myvec{g}'(\myvec{x}^* + t(\myvec{z}_k - \myvec{x}^*)) - \myvec{g}'(\myvec{x}^*)](\myvec{z}_k - \myvec{x}^*)\dif{t} \\
&= \sum_{j=0}^{m_k}\alpha_j^{(k)}
\big[\myvec{g}(\myvec{x}^*) + \myvec{g}'(\myvec{x}^*)(\myvec{x}_{k-m_k+j} - \myvec{x}^*)\big] \\
&\quad + \int_0^1 [\myvec{g}'(\myvec{x}^* + t(\myvec{z}_k - \myvec{x}^*)) - \myvec{g}'(\myvec{x}^*)](\myvec{z}_k - \myvec{x}^*)\dif{t},
\end{align*}
we have
\begin{align*}
\myvec{g}(\myvec{z}_k) - \myvec{x}_{k+1}
&= \sum_{j=0}^{m_k}\alpha_j^{(k)}
\big[\myvec{g}(\myvec{x}^*) + \myvec{g}'(\myvec{x}^*)(\myvec{x}_{k-m_k+j} - \myvec{x}^*)\
- \myvec{g}(\myvec{x}_{k-m_k+j})\big] \\
&\quad + \int_0^1 [\myvec{g}'(\myvec{x}^* + t(\myvec{z}_k - \myvec{x}^*))
- \myvec{g}'(\myvec{x}^*)](\myvec{z}_k - \myvec{x}^*)\dif{t} \\
&= - \sum_{j=0}^{m_k} \alpha_j^{(k)}
\left(\int_0^1 \big[\myvec{g}'(\myvec{x}^* + t(\myvec{x}_{k-m_k+j} - \myvec{x}^*))
- \myvec{g}'(\myvec{x}^*)\big](\myvec{x}_{k-m_k+j} - \myvec{x}^*)\dif{t}\right) \\
&\quad + \int_0^1 [\myvec{g}'(\myvec{x}^* + t(\myvec{z}_k - \myvec{x}^*))
- \myvec{g}'(\myvec{x}^*)](\myvec{z}_k - \myvec{x}^*)\dif{t}.
\end{align*}
Recall that if the nonlinear operator $\myvec{f}$ is H\"older continuous with exponent $\nu \in (0,1)$,
then the corresponding fixed-point function $\myvec{g}$ inherits the same H\"older continuity with the same exponent $\nu$.
Then, the H\"older condition \eqref{cond:Holder} is applicable to deduce that
\begin{align*}
\|\myvec{g}(\myvec{z}_k) - \myvec{x}_{k+1}\|
&\leq H_\nu \sum_{j=0}^{m_k} |\alpha_j^{(k)}| \int_0^1 t^\nu \|\myvec{x}_{k-m_k+j} - \myvec{x}^*\|^{1+\nu}\dif{t} \\
&\quad + H_\nu \int_0^1 t^\nu \|\myvec{z}_k - \myvec{x}^*\|^{1+\nu}\dif{t} \\
&= \frac{H_\nu}{1+\nu} \left(\|\myvec{z}_k - \myvec{x}^*\|^{1+\nu}
+ \sum_{j=0}^{m_k} |\alpha_j^{(k)}| \|\myvec{x}_{k-m_k+j} - \myvec{x}^*\|^{1+\nu}\right).
\end{align*}
It follows from \eqref{ineq:norm_zk-x*} and \eqref{ineq:norm_f(x)} that
\begin{align*}
\lefteqn{\|\myvec{z}_k - \myvec{x}^*\|^{1+\nu}} \\
&\leq \frac{(1+\nu)^{1+\nu}}{\nu^{1+\nu}} \left(q^{k-m}\right)^{1+\nu}
M_\alpha^{1+\nu} \|\myvec{f}'(\myvec{x}^*)^{-1}\|^{1+\nu} \|\myvec{f}(\myvec{x}_0)\|^{1+\nu} \\
&\leq \frac{(1+\nu)^{1+\nu}}{\nu^{1+\nu}} \left(q^{k-m}\right)^{1+\nu}
M_\alpha^{1+\nu} \|\myvec{f}'(\myvec{x}^*)^{-1}\|^{1+\nu}
\left(\frac{2+\nu}{1+\nu}\|\myvec{f}'(\myvec{x}^*)\|\|\myvec{x}_0 - \myvec{x}^*\|\right)^\nu\|\myvec{f}(\myvec{x}_0)\| \\
&= \frac{(1+\nu)(2+\nu)^\nu}{\nu^{1+\nu}}q^{(1+\nu)(k-m)}M_\alpha^{1+\nu}
\|\myvec{f}'(\myvec{x}^*)^{-1}\|\kappa^\nu(\myvec{f}'(\myvec{x}^*))
\|\myvec{x}_0 - \myvec{x}^*\|^\nu \|\myvec{f}(\myvec{x}_0)\|.
\end{align*}
In addition, by using the same argument as above, we have
\begin{align*}
\lefteqn{\sum_{j=0}^{m_k} |\alpha_j^{(k)}| \|\myvec{x}_{k-m_k+j} - \myvec{x}^*\|^{1+\nu} } \\
&\leq \sum_{j=0}^{m_k} |\alpha_j^{(k)}|
\left(\frac{1+\nu}{\nu}\|\myvec{f}'(\myvec{x}^*)^{-1}\|\|\myvec{f}(\myvec{x}_{k-m_k+j})\|\right)^{1+\nu} \\
&\leq \frac{(1+\nu)(2+\nu)^\nu}{\nu^{1+\nu}}q^{(1+\nu)(k-m)}M_\alpha
\|\myvec{f}'(\myvec{x}^*)^{-1}\|\kappa^\nu(\myvec{f}'(\myvec{x}^*))
\|\myvec{x}_0 - \myvec{x}^*\|^\nu \|\myvec{f}(\myvec{x}_0)\|.
\end{align*}
The above two estimates allow us to get
\begin{align*}
\|\myvec{g}(\myvec{z}_k) - \myvec{x}_{k+1}\|
&\leq \frac{H_\nu M_\alpha(1+M_\alpha^\nu)(2+\nu)^\nu}{\nu^{1+\nu}}q^{(1+\nu)(k-m)}
\kappa^\nu(\myvec{f}'(\myvec{x}^*))\|\myvec{f}'(\myvec{x}^*)^{-1}\|
\|\myvec{x}_0 - \myvec{x}^*\|^\nu \|\myvec{f}(\myvec{x}_0)\|.
\end{align*}
This together with \eqref{eq:fk+1} and \eqref{ineq:norm_g(xk+1)-g(z_k)} leads to
\begin{align*}
\lefteqn{\|\myvec{f}(\myvec{x}_{k+1})\| } \\
&\leq \|\myvec{g}(\myvec{x}_{k+1}) - \myvec{x}_{k+1}\| + \|\myvec{g}(\myvec{z}_k) - \myvec{x}_{k+1}\| \\
&\leq q^k\|\myvec{f}(\myvec{x}_0)\|\left[\theta\eta_k
+ \frac{H_\nu M_\alpha(1+M_\alpha^\nu)(2+\nu)^\nu}{\nu^{1+\nu}}q^{(1+\nu)(k-m)-k}
\kappa^\nu(\myvec{f}'(\myvec{x}^*))\|\myvec{f}'(\myvec{x}^*)^{-1}\|
\|\myvec{x}_0 - \myvec{x}^*\|^\nu\right] \\
&\leq q^k\|\myvec{f}(\myvec{x}_0)\|\left[\theta\eta_k
+ \frac{H_\nu M_\alpha(1+M_\alpha^\nu)(2+\nu)^\nu}{\nu^{1+\nu}}q^{-m}
\kappa^\nu(\myvec{f}'(\myvec{x}^*))\|\myvec{f}'(\myvec{x}^*)^{-1}\|
\|\myvec{x}_0 - \myvec{x}^*\|^\nu\right].
\end{align*}
Noting that $\tau = \theta\eta_k \in (0,1)$,
and that $\zeta$, as given in \eqref{cons:zeta}, satisfies $\zeta < 1 - \tau$,
we conclude from Lemma \ref{lem:q} that
$$
\|\myvec{f}(\myvec{x}_{k+1})\|
\leq (\tau + \zeta q^{-m})q^k \|\myvec{f}(\myvec{x}_0)\|
= q \cdot q^k \|\myvec{f}(\myvec{x}_0)\| = q^{k+1} \|\myvec{f}(\myvec{x}_0)\|.
$$
Therefore, by induction, all claims in the theorem are verified.
The proof is complete.
\end{proof}
\begin{myremark}
Theorem \ref{th:LocalConvAnderson(m)} shows that the sequence $\{\myvec{x}_k\}$ converges R-linearly to the solution $\myvec{x}^*$.
The convergence rate is determined by the unique root $q$ of equation \eqref{eq:q} in the interval $\big(m\tau/(m+1),1\big)$.
In particular, if $m=1$, then $q = (\tau + \sqrt{\tau^2 + 4\zeta})/2$.
This means that the convergence rate is determined by
the contraction factor $\theta$, the optimization gain $\eta_k$ and the quantity $\zeta$.
\end{myremark}
\begin{myremark}
The inequality $\zeta < 1 - \tau$ can generally be satisfied
when the initial value $\myvec{x}_0$ is taken sufficiently close to the solution $\myvec{x}^*$.
An important observation is that if $\myvec{f}'(\myvec{x}^*)$ is well conditioned,
then the convergence ball $\ball(\myvec{x}^*,r)$ tends to be large,
making the condition $\zeta < 1 - \tau$ more attainable.
\end{myremark}
The R-linear convergence of the error with R-factor $q$ is a direct consequence of Theorem \ref{th:LocalConvAnderson(m)}.
\begin{mycorollary}
\label{cor:RLinearConvAnderson(m)}
Let the assumptions of Theorem $\ref{th:LocalConvAnderson(m)}$ hold.
Then we have
$$
\limsup_{k \to \infty} \left(\frac{\|\myvec{x}_k - \myvec{x}^*\|}{\|\myvec{x}_0 - \myvec{x}^*\|}\right)^{1/k} \leq q.
$$
\end{mycorollary}
\begin{proof}
Noting from \eqref{ineq:norm_f(x)/f(x0)} that
$$
\frac{\|\myvec{x}_k - \myvec{x}^*\|}{\|\myvec{x}_0 - \myvec{x}^*\|}
\leq \frac{(2+\nu)\kappa(\myvec{f}'(\myvec{x}^*))}{\nu}
\cdot \frac{\|\myvec{f}(\myvec{x}_k)\|}{\|\myvec{f}(\myvec{x}_0)\|}.
$$
This allows us to obtain from \eqref{ineq:ConvRate} that
$$
\limsup_{k \to \infty} \left(\frac{\|\myvec{x}_k - \myvec{x}^*\|}{\|\myvec{x}_0 - \myvec{x}^*\|}\right)^{1/k}
\leq \lim_{k \to \infty} \left[\frac{2+\nu}{\nu}\kappa(\myvec{f}'(\myvec{x}^*))\right]^{1/k}
\limsup_{k \to \infty} \left(\frac{\|\myvec{f}(\myvec{x}_k)\|}{\|\myvec{f}(\myvec{x}_0)\|}\right)^{1/k}
\leq q,
$$
which yields the desired result.
\end{proof}
For the case $\nu = 1$,
the H\"older continuity assumption on the Jacobian $\myvec{f}'$ reduces to the classical Lipschitz continuity:
\begin{equation}
\label{cond:Lipschitz}
\|\myvec{f}'(\myvec{x}) - \myvec{f}'(\myvec{y})\| \leq L\|\myvec{x} - \myvec{y}\|,
\quad \myvec{x}, \myvec{y} \in \ball(\myvec{x}^*,r),
\end{equation}
where
\begin{equation}
\label{cons:r(m=1)}
r = \min\left\{\frac{1}{L\|\myvec{f}'(\myvec{x}^*)^{-1}\|},
\frac{1-\theta}{3LM_\alpha(1+M_a)\|\myvec{f}'(\myvec{x}^*)^{-1}\|\kappa(\myvec{f}'(\myvec{x}^*))}\right\}.
\end{equation}
Then we have the following local convergence result from
Theorem \ref{th:LocalConvAnderson(m)} and Corollary \ref{cor:RLinearConvAnderson(m)}
for Anderson acceleration under Lipschitz condition \eqref{cond:Lipschitz}.
\begin{mycorollary}
\label{cor:LocalConvLipCondAnderson(m)}
Assume that there is an $\myvec{x}^* \in \RS^n$ such that $\myvec{f}(\myvec{x}^*) = \myvec{0}$
and $\myvec{f}'(\myvec{x}^*)$ is nonsingular. Suppose that:
\begin{enumerate}[label=\textup{(\roman*)}]
\item There is a constant $M_\alpha$ such that $\mysum_{j=0}^{m_k} |\alpha_j^{(k)}| \leq M_\alpha$ for all $k \geq 0$.
\item The Jacobian $\myvec{f}'$ satisfies the Lipschitz continuous \eqref{cond:Lipschitz} in $\ball(\myvec{x}^*,r)$,
where $r$ is given in \eqref{cons:r(m=1)}.
\item The fixed-point function $\myvec{g}$ is contractive with constant $\theta \in (0,1)$ in $\ball(\myvec{x}^*,r)$.
\end{enumerate}
Let $\{\myvec{x}_k\}$ be the sequence generated by Algorithm $\ref{alg:AA}$
with starting point $\myvec{x}_0 \in \ball(\myvec{x}^*,r)$.
Set $\tau := \theta\eta_k$ and
$\zeta := 3L M_\alpha (1 + M_\alpha)
\kappa(\myvec{f}'(\myvec{x}^*)) \|\myvec{f}'(\myvec{x}^*)^{-1}\|\|\myvec{x}_0 - \myvec{x}^*\|$,
where $\eta_k$ is the optimization gain defined by \eqref{cons:eta_k}.
If $\zeta < 1 - \tau$ and $qM_\alpha \kappa(\myvec{f}'(\myvec{x}^*)) \leq 1/3$,
where $q$ is the unique root of equation \eqref{eq:q} in the interval $\big(m\tau/(m+1),1\big)$,
then the sequence $\{\myvec{x}_k\}$ is contained in $\ball(\myvec{x}^*,r)$
and converges R-linearly to the solution $\myvec{x}^*$
in the sense that
\begin{equation*}
\limsup_{k \to \infty} \left(\frac{\|\myvec{f}(\myvec{x}_{k})\|}{\|\myvec{f}(\myvec{x}_0)\|}\right)^{1/k}
\leq q \quad \text{and} \quad
\limsup_{k \to \infty} \left(\frac{\|\myvec{x}_k - \myvec{x}^*\|}{\|\myvec{x}_0 - \myvec{x}^*\|}\right)^{1/k}
\leq q.
\end{equation*}
\end{mycorollary}
\begin{myremark}
Toth and Kelley \cite{TothK2015} showed that Anderson acceleration with depth $m$
converges R-linearly with an R-factor in $(\theta,1)$,
provided $\myvec{g}$ is Lipschitz continuously differentiable.
Later, Chen and Kelley \cite{ChenK2019} relaxed this assumption and proved
convergence with R-factor $\theta^{1/(m+1)}$.
Our work reveals that the R-factor depends not only on the contraction factor $\theta$,
but also on the optimization gain $\eta_k$ and the condition number $\kappa(\myvec{f}'(\myvec{x}^*))$,
leading to a more refined characterization of convergence than previous results
based only on the contraction factor $\theta$.
In the special case $m=1$,
the R-factor can be explicitly expressed as $q = (\tau + \sqrt{\tau^2 + 4\zeta})/2$.
\end{myremark}
%------------------------------------------------------------------------------
\subsection{Convergence rate for depth $m=1$}
In this subsection, we obtain the convergence rate of Anderson acceleration for depth $m=1$ with the $\ell^2$ norm.
For the depth $m = 1$, the optimization problem \eqref{optim:alpha} becomes
$$
\alpha_k = \arg\min_{\alpha \in \RS}
\left\|(1-\alpha)\myvec{f}(\myvec{x}_k) + \alpha \myvec{f}(\myvec{x}_{k-1})\right\|,
$$
and admits a closed-form solution
\begin{equation}
\label{cons:alpha_k(m=1)}
\alpha_k = \frac{\mytran{\myvec{f}(\myvec{x}_k)}(\myvec{f}(\myvec{x}_k) - \myvec{f}(\myvec{x}_{k-1}))}
{\|\myvec{f}(\myvec{x}_k) - \myvec{f}(\myvec{x}_{k-1})\|^2}.
\end{equation}
Moreover, we have
$$
\myvec{x}_{k+1} = (1-\alpha_k)\myvec{g}(\myvec{x}_k) + \alpha_k \myvec{g}(\myvec{x}_{k-1}).
$$
The optimization gain $\eta_k$ given by \eqref{cons:eta_k} now becomes
\begin{equation}
\label{cons:eta_k(m=1)}
\eta_k = \frac{\|(1-\alpha_k)\myvec{f}(\myvec{x}_k) + \alpha_k \myvec{f}(\myvec{x}_{k-1})\|}{\|\myvec{f}(\myvec{x}_k)\|}.
\end{equation}
By combining \eqref{cons:alpha_k(m=1)} and \eqref{cons:eta_k(m=1)},
we arrive at
\begin{equation}
\label{eq:norm_f(xk+1)-f(xk)}
|\alpha_k| \|\myvec{f}(\myvec{x}_k) - \myvec{f}(\myvec{x}_{k-1})\|_2
= \sqrt{1 - \eta_k^2} \|\myvec{f}(\myvec{x}_k)\|.
\end{equation}
Details of the derivation can be found in \cite[p. 2848]{PollockR2021}.
As a consequence, we obtain two useful inequalities that provide an upper bound
on the difference between consecutive iterations using the residual $\myvec{f}(\myvec{x}_k)$.
\begin{mylemma}
\label{lem:norm_xk+1-xk}
Let $m = 1$ in Algorithm $\ref{alg:AA}$. For any $k \geq 1$, we have
\begin{align}
\|\myvec{x}_{k} - \myvec{x}_{k-1}\|
&\leq \frac{\sqrt{1 - \eta_k^2}}{|\alpha_k|(1 - \theta)} \|\myvec{f}(\myvec{x}_k)\|, \label{ineq:norm_xk-xk-1_m=1} \\
\|\myvec{x}_{k+1} - \myvec{x}_k\|
&\leq \left(1 + \frac{\theta}{1-\theta}\sqrt{1-\eta_k^2}\right)\|\myvec{f}(\myvec{x}_k)\|, \label{ineq:norm_xk+1-xk_m=1}
\end{align}
where $\alpha_k$ is given by \eqref{cons:alpha_k(m=1)},
$\eta_k$ is given by \eqref{cons:eta_k(m=1)},
and $\theta$ is the contraction factor.
\end{mylemma}
\begin{proof}
By the contractivity of the fixed-point function $\myvec{g}$, we have
\begin{align*}
\|\myvec{f}(\myvec{x}_k) - \myvec{f}(\myvec{x}_{k-1})\|
&= \|(\myvec{g}(\myvec{x}_k) - \myvec{x}_k) - (\myvec{g}(\myvec{x}_{k-1}) - \myvec{x}_{k-1})\| \\
&\geq \|\myvec{x}_k - \myvec{x}_{k-1}\| - \|\myvec{g}(\myvec{x}_k) - \myvec{g}(\myvec{x}_{k-1})\| \\
&\geq (1 - \theta)\|\myvec{x}_k - \myvec{x}_{k-1}\|.
\end{align*}
Combining this with \eqref{eq:norm_f(xk+1)-f(xk)} yields the inequality \eqref{ineq:norm_xk-xk-1_m=1}.
On the other hand, we observe that
\begin{align*}
\|\myvec{x}_{k+1} - \myvec{x}_k\|
&= \left\|[(1-\alpha_k)\myvec{g}(\myvec{x}_k) + \alpha_k \myvec{g}(\myvec{x}_{k-1})] - \myvec{x}_k\right\| \\
&\leq \|\myvec{g}(\myvec{x}_k) - \myvec{x}_k\| + |\alpha_k| \|\myvec{g}(\myvec{x}_{k}) - \myvec{g}(\myvec{x}_{k-1})\| \\
&\leq \|\myvec{f}(\myvec{x}_k)\| + \theta |\alpha_k| \|\myvec{x}_k - \myvec{x}_{k-1}\|,
\end{align*}
which, together with \eqref{ineq:norm_xk-xk-1_m=1}, gives \eqref{ineq:norm_xk+1-xk_m=1}.
\end{proof}
The following theorem provides a refined bound for the residual $\myvec{f}(\myvec{x}_{k+1})$
in terms of the residual $\myvec{f}(\myvec{x}_k)$ and the optimization gain $\eta_k$.
\begin{mytheorem}
\label{th:LocalConvAnderson(m=1)}
Let the assumptions of Theorem $\ref{th:LocalConvAnderson(m)}$ hold.
If $m = 1$, then we have the following bound for the residual $\myvec{f}(\myvec{x}_{k+1})$:
\begin{equation}
\label{ineq:norm_f(xk+1)_m=1}
\begin{aligned}
\|\myvec{f}(\myvec{x}_{k+1})\|
&\leq \theta\left(1 + \frac{1+\theta}{1-\theta}\sqrt{1-\eta_k^2}\right)\|\myvec{f}(\myvec{x}_k)\| \\
&\quad + \frac{H_\nu}{1+\nu}\left[
\left(1+\frac{\theta}{1-\theta}\sqrt{1-\eta_k^2}\right)^{1+\nu}
+ \left(\frac{\sqrt{1-\eta^2_k}}{1-\theta}\right)^{1+\nu}\frac{1}{|\alpha_k|^\nu}
\right]\|\myvec{f}(\myvec{x}_k)\|^{1+\nu},
\end{aligned}
\end{equation}
\end{mytheorem}
\begin{proof}
Since
\begin{align*}
\myvec{f}(\myvec{x}_{k+1})
&= \myvec{g}(\myvec{x}_{k+1}) - \myvec{x}_{k+1} \\
&= [\myvec{g}(\myvec{x}_{k+1}) - \myvec{g}(\myvec{x}_k)]
+ \alpha_k [\myvec{g}(\myvec{x}_{k}) - \myvec{g}(\myvec{x}_{k-1})] \\
&= \int_0^1 \myvec{g}'(\myvec{x}_k + t(\myvec{x}_{k+1} - \myvec{x}_k))(\myvec{x}_{k+1} - \myvec{x}_k)\dif{t} \\
&\quad + \alpha_k \int_0^1 \myvec{g}'(\myvec{x}_{k-1} + t(\myvec{x}_{k} - \myvec{x}_{k-1}))(\myvec{x}_{k} - \myvec{x}_{k-1})\dif{t},
\end{align*}
we get
\begin{align*}
\lefteqn{\myvec{f}(\myvec{x}_{k+1}) - \myvec{g}'(\myvec{x}_k)(\myvec{x}_{k+1} - \myvec{x}_k)
- \alpha_k \myvec{g}'(\myvec{x}_{k-1})(\myvec{x}_{k} - \myvec{x}_{k-1}) } \\
&= \int_0^1 \left[\myvec{g}'(\myvec{x}_k + t(\myvec{x}_{k+1} - \myvec{x}_k))
- \myvec{g}'(\myvec{x}_k)\right](\myvec{x}_{k+1} - \myvec{x}_k)\dif{t} \\
&\quad + \alpha_k \int_0^1 \left[\myvec{g}'(\myvec{x}_{k-1} + t(\myvec{x}_{k} - \myvec{x}_{k-1}))
- \myvec{g}'(\myvec{x}_{k-1})\right](\myvec{x}_{k} - \myvec{x}_{k-1})\dif{t}.
\end{align*}
By the H\"older condition \eqref{cond:Holder}, we have
\begin{align*}
\lefteqn{\|\myvec{f}(\myvec{x}_{k+1}) - \myvec{g}'(\myvec{x}_k)(\myvec{x}_{k+1} - \myvec{x}_k)
- \alpha_k \myvec{g}'(\myvec{x}_{k-1})(\myvec{x}_{k} - \myvec{x}_{k-1})\| } \\
&\leq H_\nu \int_0^1 t^\nu \|\myvec{x}_{k+1} - \myvec{x}_k\|^{1+\nu}\dif{t}
+ H_\nu |\alpha_k| \int_0^1 t^\nu \|\myvec{x}_{k} - \myvec{x}_{k-1}\|^{1+\nu}\dif{t} \\
&= \frac{H_\nu}{1+\nu} \left(\|\myvec{x}_{k+1} - \myvec{x}_k\|^{1+\nu}
+ |\alpha_k| \|\myvec{x}_{k} - \myvec{x}_{k-1}\|^{1+\nu}\right).
\end{align*}
Then we can use \eqref{ineq:norm_xk-xk-1_m=1} and \eqref{ineq:norm_xk+1-xk_m=1} to obtain
\begin{align}
\lefteqn{\|\myvec{f}(\myvec{x}_{k+1}) - \myvec{g}'(\myvec{x}_k)(\myvec{x}_{k+1} - \myvec{x}_k)
- \alpha_k \myvec{g}'(\myvec{x}_{k-1})(\myvec{x}_{k} - \myvec{x}_{k-1})\| } \nonumber \\
&\leq \frac{H_\nu}{1+\nu} \left[\left(1 + \frac{\theta}{1-\theta}\sqrt{1-\eta_k^2}\right)^{1+\nu}
+ \left(\frac{\sqrt{1-\eta^2_k}}{1-\theta}\right)^{1+\nu} \frac{1}{|\alpha_k|^{\nu}}\right]
\|\myvec{f}(\myvec{x}_k)\|^{1+\nu}. \label{ineq:norm_f(xk+1)-g'(xk)(xk+1-xk)}
\end{align}
Recall that the contractivity of $\myvec{g}$ implies that
$$
\|\myvec{g}'(\myvec{x}_k)\| \leq \theta
\quad \text{and}\quad
\|\myvec{g}'(\myvec{x}_{k-1})\| \leq \theta.
$$
This together with \eqref{ineq:norm_xk-xk-1_m=1} and \eqref{ineq:norm_xk+1-xk_m=1} permits us to get
\begin{align}
\lefteqn{\|\myvec{g}'(\myvec{x}_k)(\myvec{x}_{k+1} - \myvec{x}_k)
+ \alpha_k \myvec{g}'(\myvec{x}_{k-1})(\myvec{x}_{k} - \myvec{x}_{k-1})\| } \nonumber \\
&\leq \theta \|\myvec{x}_{k+1} - \myvec{x}_k\| + \theta |\alpha_k| \|\myvec{x}_{k} - \myvec{x}_{k-1}\| \nonumber \\
&\leq \theta \left(1 + \frac{\theta}{1-\theta}\sqrt{1-\eta_k^2}\right)\|\myvec{f}(\myvec{x}_k)\|
+ \frac{\theta}{1-\theta}\sqrt{1-\eta_k^2}\|\myvec{f}(\myvec{x}_k)\| \nonumber \\
&= \theta\left(1 + \frac{1+\theta}{1-\theta}\sqrt{1-\eta_k^2}\right)\|\myvec{f}(\myvec{x}_k)\|. \label{ineq:norm_g'(xk)(xk+1-xk)+alpha_k g'(xk-1)(xk-xk-1)}
\end{align}
Combining \eqref{ineq:norm_f(xk+1)-g'(xk)(xk+1-xk)}
and \eqref{ineq:norm_g'(xk)(xk+1-xk)+alpha_k g'(xk-1)(xk-xk-1)}
yields \eqref{ineq:norm_f(xk+1)_m=1}.
The proof is complete.
\end{proof}
\begin{myremark}
Theorem \ref{th:LocalConvAnderson(m=1)} shows that a smaller optimization gain $\eta_k$ leads to
greater impact from higher-order term.
\end{myremark}
%------------------------------------------------------------------------------
\section{Application to algebraic Riccati equation}
\label{sec:Application}
In this section, we explore Anderson acceleration to solve a special nonlinear equation
which is obtained by a NARE arising from neutron transport theory.
We begin with some new definitions and notations.
For any real matrices $A = (a_{ij})_{m \times n}$ and $B = (b_{ij})_{m \times n}$,
we write $A \geq B$ (respectively, $A > B$) if $a_{ij} \geq b_{ij}$
(respectively, $a_{ij} > b_{ij}$) for all $i = 1,2,\ldots,m$ and $j = 1,2,\ldots,n$.
A real matrix $A = (a_{ij})_{m \times n}$ is called nonnegative
if all its components satisfy $a_{ij} \geq 0$,
and positive if $a_{ij} > 0$ for all $i = 1,2,\ldots,m$ and $j = 1,2,\ldots,n$.
We denote these as $A \geq 0$ and $A > 0$, respectively.
We denote by $A \circ B = (a_{ij}\cdot b_{ij})_{m \times n}$ the Hadamard product of $A$ and $B$.
Moreover, for any real vectors $\myvec{a} = (a_1,a_2,\ldots,a_n)^{\top}$
and $\myvec{b} = (b_1,b_2,\ldots,b_n)^{\top}$,
we write $\myvec{a} \geq \myvec{b}$ (respectively, $\myvec{a} > \myvec{b}$)
if $a_i \geq b_i$ (respectively, $a_i > b_i$) for all $i = 1,2,\ldots,n$.
The vector of all zero components is denoted by $\myvec{0}$.
A vector $\myvec{v} \in \RS^n$ is called nonnegative if $\myvec{v} \geq \myvec{0}$,
and positive if $\myvec{v} > \myvec{0}$.
\subsection{Problem setting}
The form of NARE from neutron transport theory is as follows:
\begin{equation}
\label{eq:NARE}
XCX - XD - AX + B = 0,
\end{equation}
where $X \in \RS^{n \times n}$ is an unknown matrix,
and $A, B, C, D \in \RS^{n \times n}$ are known matrices given by
\begin{equation}
\label{mat:ABCD}
A = \Delta - \myvec{e}\myvec{p}^{\top}, \quad
B = \myvec{e}\myvec{e}^{\top}, \quad
C = \myvec{p}\myvec{p}^{\top}, \quad
D = \hat{\Delta} - \myvec{p}\myvec{e}^{\top}.
\end{equation}
Here $\Delta = \diag(\delta_1,\delta_2,\ldots,\delta_n)$ with $\delta_i = 1/(c\omega_i(1+a)) > 0$,
$\hat{\Delta} = \diag(\hat{\delta}_1,\hat{\delta}_2,\ldots,\hat{\delta}_n)$ with $\hat{\delta}_i = 1/(c\omega_i(1-a)) > 0$,
and $\myvec{p} = (p_1,p_2,\ldots,p_n)^{\top}$ with $p_i = c_i/(2\omega_i) > 0$.
The matrices and vectors above depend on a pair of parameters $(a,c)$ with
\begin{equation}
\label{cons:alpha_c}
a \in [0,1) \quad \text{and} \quad c \in (0,1].
\end{equation}
Moreover, the sets $\{\omega_i\}_{i=1}^n$ and $\{c_i\}_{i=1}^n$
represent the nodes and weights, respectively, of the Gauss-Legendre quadrature
on the interval $[0,1]$, satisfying
$$
0 < \omega_n < \cdots < \omega_2 < \omega_1 < 1 \ \text{and} \ \sum_{i=1}^n c_i = 1 \ \text{with} \ c_i > 0.
$$
Clearly, the sequences $\{\delta_i\}_{i=1}^n$ and $\{\hat{\delta}_i\}_{i=1}^n$ are
strictly monotonically increasing, and
$$
\begin{cases}
\delta_i = \hat{\delta}_i, & \mbox{when $a = 0$},\\
\delta_i \neq \hat{\delta}_i, & \mbox{when $a \neq 0$},
\end{cases}
\quad i = 1,2,\ldots,n.
$$
The NARE \eqref{eq:NARE} is obtained by a discretization of an integrodifferential equation
describing neutron transport during a collision process.
The solution of interest from a physical perspective is the minimal nonnegative solution,
as discussed in previous studies
\cite{Juang1995,JuangL1998,Guo2001}.
Lu \cite{Lu2005a} demonstrated that the solution to equation \eqref{eq:NARE} can be expressed by:
$$
X = T \circ (\myvec{u} \myvec{v}^{\top}) = (\myvec{u} \myvec{v}^{\top}) \circ T,
$$
where $T = (t_{ij})_{n \times n} = \left(\frac{1}{\delta_i + \hat{\delta}_j}\right)_{n\times n}$,
$\myvec{u}$ and $\myvec{v}$ are vectors satisfying
\begin{equation}
\label{eq:VecEq_uv}
\left\{
\begin{aligned}
\myvec{u} &= \myvec{u} \circ (P\myvec{v}) + \myvec{e},\\
\myvec{v} &= \myvec{v} \circ (\widetilde{P}\myvec{u}) + \myvec{e},
\end{aligned}
\right.
\end{equation}
with
\begin{equation}
\label{mat:P_Ptilde}
P
= (p_{ij})_{n\times n}
= \left(\frac{p_j}{\delta_i + \hat{\delta}_j}\right)_{n\times n}, \quad
\widetilde{P}
= (\widetilde{p}_{ij})_{n\times n}
= \left(\frac{p_j}{\hat{\delta}_i + \delta_j}\right)_{n\times n}.
\end{equation}
We set $\myvec{x} = [\myvec{u}^{\top},\myvec{v}^{\top}]^{\top} \in \RS^{2n}$.
Then the objective of finding the minimal nonnegative solution of \eqref{eq:NARE} is equivalent to
finding solutions for the nonlinear system
\begin{equation}
\label{eq:f(u,v)=0}
\myvec{f}(\myvec{x}) =
\myvec{f}(\myvec{u},\myvec{v}) \stackrel{\text{def}}{=}
\begin{bmatrix}
\myvec{u} - \myvec{u} \circ (P\myvec{v}) - \myvec{e} \\
\myvec{v} - \myvec{v} \circ (\widetilde{P}\myvec{u}) - \myvec{e}
\end{bmatrix} = \myvec{0},
\end{equation}
or alternatively, to finding the fixed point of the fixed-point problem
\begin{equation}
\label{eq:g(u,v)=0}
\myvec{x} = \myvec{g}(\myvec{x}) =
\myvec{g}(\myvec{u},\myvec{v}) \stackrel{\text{def}}{=}
\begin{bmatrix}
\myvec{u} \circ (P\myvec{v}) + \myvec{e} \\
\myvec{v} \circ (\widetilde{P}\myvec{u}) + \myvec{e}
\end{bmatrix}.
\end{equation}
The advantage in representing \eqref{eq:VecEq_uv} as the nonlinear system \eqref{eq:f(u,v)=0} is that
we now can use the Newton-type methods to solve it.
Since Lu's Newton-based algorithm was introduced in \cite{Lu2005b},
extensive research has been conducted on Newton-type methods,
focusing either on improving their effectiveness
or on accelerating convergence via higher-order techniques
(see, e.g. \cite{BiniIP2008,LinBW2008,LinB2008,HuangKH2010,LingX2017,LingLL2022}).
Moreover, there has been significant interest in developing more effective fixed-point iterative algorithms
for solving the fixed-point problem \eqref{eq:g(u,v)=0}, such as the ones described in
\cite{BaoLW2006,BaiGL2008,Lin2008,GuoL2010,LinBW2011,HuangM2014,HuangM2020}.
Let $P \in \RS^{n \times n}$
be partitioned column-wise as $P = [\myvec{p}_1,\myvec{p}_2,\ldots,\myvec{p}_n]$, and similarly for
$\tilde{P} = [\tilde{\myvec{p}}_1,\tilde{\myvec{p}}_2,\ldots,\tilde{\myvec{p}}_n] \in \RS^{n \times n}$.
We note that the nonlinear operator $\myvec{f}$ defined by \eqref{eq:f(u,v)=0}
is continuously Fr\'echet differentiable.
The Jacobian at point $(\myvec{u},\myvec{v})$ is given by (see \cite{Lu2005b})
\begin{equation*}
\label{mat:Jacobian}
\myvec{f}'(\myvec{u},\myvec{v}) = I_{2n} - G(\myvec{u},\myvec{v}),
\end{equation*}
where
\begin{equation*}
\label{mat:G}
G(\myvec{u},\myvec{v}) =
\begin{bmatrix}
G_{11}(\myvec{v}) & G_{12}(\myvec{u}) \\
G_{21}(\myvec{v}) & G_{22}(\myvec{u})
\end{bmatrix}
\end{equation*}
with
\begin{align*}
G_{11}(\myvec{v}) &= \diag(P\myvec{v}), \quad
G_{12}(\myvec{u}) = [\myvec{u}\circ \myvec{p}_1,\myvec{u}\circ \myvec{p}_2,\ldots,\myvec{u}\circ \myvec{p}_n], \\
G_{22}(\myvec{u}) &= \diag(\tilde{P}\myvec{u}), \quad
G_{21}(\myvec{v}) = [\myvec{v} \circ \tilde{\myvec{p}}_1,\myvec{v}\circ\tilde{\myvec{p}}_2,\ldots,\myvec{v}\circ\tilde{\myvec{p}}_n].
\end{align*}
It follows that the Jacobian of the fixed-point function $\myvec{g}$ is $G(\myvec{u},\myvec{v})$.
Furthermore, the Jacobian $\myvec{f}'$ is Lipschitz continuous with respect to the $\ell^\infty$ norm.
In fact, for any $\myvec{x}, \myvec{y} \in \RS^{2n}$, we have
$$
\|\myvec{f}'(\myvec{x}) - \myvec{f}'(\myvec{y})\|_{\infty}
= \|G(\myvec{u},\myvec{v}) - G(\myvec{u},\myvec{v})\|_{\infty}
\leq 2\max_{1\leq i \leq n} \left\{\sum_{j=1}^{n}p_{ij}, \sum_{j=1}^{n}\tilde{p}_{ij}\right\} \|\myvec{x} - \myvec{y}\|.
$$
According to Lemma 3 in \cite{Lu2005a}, it holds that
\begin{equation}
\label{ineq:sum_pij}
\sum_{j=1}^{n}p_{ij} < \frac{c(1-a)}{2} \quad \text{and} \quad
\sum_{j=1}^{n}\tilde{p}_{ij} < \frac{c(1+a)}{2}.
\end{equation}
Then we can conclude that
$$
\|\myvec{f}'(\myvec{x}) - \myvec{f}'(\myvec{y})\|_{\infty}
\leq c(1+a) \|\myvec{x} - \myvec{y}\|_{\infty}.
$$
This means that the Jacobian $\myvec{f}'$ satisfies the Lipschitz continuous
with Lipschitz constant $L = c(1+a)$.
It is worth pointing out that the differentiability assumption
is required for the convergence analysis, but not for the implementation of the algorithm.
Let $\myvec{x}^* \in \RS^{2n}$ be the minimal nonnegative solution of the nonlinear system \eqref{eq:f(u,v)=0}.
It was shown in \cite[Theorem 4.1]{BaiGL2008} that
$$
\myvec{e} < \frac{2}{1 + \sqrt{1-4\phi}}\myvec{e} \leq \myvec{x}^*
\leq \frac{2}{1 + \sqrt{1-4\Phi}}\myvec{e} \leq 2\myvec{e},
\quad 0 < \phi < \Phi \leq \frac{1}{4},
$$
which improves upon the previous results obtained in \cite{Lu2005a}.
We observe that the fixed-point function $\myvec{g}$ defined by \eqref{eq:g(u,v)=0} is a contractive operator
with respect to the $\ell^\infty$ norm under some condition depending on the parameters $c$ and $a$.
Indeed, for any $\myvec{x} = \mytran{[\mytran{\myvec{u}},\mytran{\myvec{u}}]}$,
$\myvec{y} = \mytran{[\mytran{\myvec{s}},\mytran{\myvec{t}}]} \in \RS^{2n}$, we have
\begin{align*}
\|\myvec{g}(\myvec{x}) - \myvec{g}(\myvec{y})\|_{\infty}
&= \left\|\begin{bmatrix}
\myvec{u}\circ(P\myvec{v} - P\myvec{t}) + (\myvec{u} - \myvec{s})\circ(P\myvec{t}) \\
\myvec{v}\circ(\tilde{P}\myvec{u} - \tilde{P}\myvec{s}) + (\myvec{v} - \myvec{t})\circ(\tilde{P}\myvec{s})
\end{bmatrix}\right\|_{\infty} \\
&\leq \left\|\begin{bmatrix}
\myvec{u} \\
\myvec{v}
\end{bmatrix}\right\|_{\infty}
\left\|\begin{bmatrix}
\tilde{P} & \\
& P
\end{bmatrix}
\begin{bmatrix}
\myvec{u} - \myvec{s} \\
\myvec{v} - \myvec{t}
\end{bmatrix}\right\|_{\infty}
+ \left\|\begin{bmatrix}
\myvec{u} - \myvec{s} \\
\myvec{v} - \myvec{t}
\end{bmatrix}\right\|_{\infty}
\left\|\begin{bmatrix}
\tilde{P} & \\
& P
\end{bmatrix}
\begin{bmatrix}
\myvec{s} \\
\myvec{t}
\end{bmatrix}\right\|_{\infty}.
\end{align*}
We conclude from \eqref{ineq:sum_pij} that
\begin{equation*}
\label{ineq:norm_g(x)-g(y)}
\|\myvec{g}(\myvec{x}) - \myvec{g}(\myvec{y})\|_{\infty}
\leq \frac{c(1+a)}{2}(\|\myvec{x}\|_{\infty} + \|\myvec{y}\|_{\infty})\|\myvec{x} - \myvec{y}\|_{\infty}.
\end{equation*}
If
$\myvec{x},\myvec{y} \in \ball_+(\myvec{0},r)
:= \{\myvec{z} \in \RS^{2n} \mid \|\myvec{z}\|_\infty < r, \myvec{z} \geq \myvec{0}\}$
with $r = 2/\big(1+\sqrt{1-4\Phi}\big)$, then it follows that
$$
\|\myvec{g}(\myvec{x}) - \myvec{g}(\myvec{y})\|_{\infty}
< \frac{c(1+a)}{2} \cdot \frac{4}{1 + \sqrt{1-4\Phi}} \|\myvec{x} - \myvec{y}\|_{\infty}
= \frac{2c(1+a)}{1+\sqrt{1-4\Phi}}\|\myvec{x} - \myvec{y}\|_{\infty}.
$$
This means that the fixed-point function $\myvec{g}$ is contractive on the ball $\ball_+(\myvec{0},r)$
with contraction factor $\theta = 2c(1+a)/(1+\sqrt{1-4\Phi})$, provided that
$$
2c(1+a) < 1 + \sqrt{1-4\Phi}.
$$
Since the minimal positive solution of the nonlinear system \eqref{eq:f(u,v)=0},
or equivalently, the fixed-point problem \eqref{eq:g(u,v)=0}, is generally unavailable,
Corollary \ref{cor:LocalConvLipCondAnderson(m)} cannot be directly applied.
Nonetheless, we can still verify the convergence results through numerical experiments
comparing Anderson acceleration with several established fixed-point iterative methods.
These experiments, presented in Subsection \ref{subsec:NumericalExperiments},
serve as an indirect validation of our theoretical results
and further illustrate the robustness and efficiency of Anderson acceleration in various problem settings,
particularly in nearly singular and large-scale problems.
%--------------------------------------------------------------------
\subsection{Implementation}
One of the main challenges in implementing the Anderson acceleration described
in Algorithm \ref{alg:AA} is determining the coefficients $\{\alpha_j^{(k)}\}_{j=0}^{m_k}$
by solving the constrained least squares problem \eqref{optim:alpha}.
Following the approach in \cite{FangS2009,WalkerN2011},
this problem can be equivalently reformulated as an unconstrained least squares problem,
which can be efficiently solved using the QR factorization.
To this end, define $\Delta{\myvec{f}_i} = \myvec{f}_{i+1} - \myvec{f}_i$ for any $i \geq 1$
and let
$\mathcal{F}_k = (\Delta{\myvec{f}_{k-m_k}}, \ldots, \Delta{\myvec{f}_{k-1}}) \in \RS^{n\times m_k}$.
If we set $\alpha_0 = \gamma_0$,
$$
\alpha_i = \gamma_i - \gamma_{i-1}, \quad i = 1,2,\ldots,m_k-1,
$$
and $\alpha_{m_k} = 1 - \gamma_{m_k-1}$,
then the constrained least squares problem \eqref{optim:alpha} is equivalent to the following unconstrained least squares problem:
\begin{equation}
\label{optim:alpha_unconstrained}
\min_{\bm{\gamma} = \mytran{(\gamma_0,\ldots,\gamma_{m_k-1})}}
\|\myvec{f}_{k} - \mathcal{F}_k\bm{\gamma}\|_2.
\end{equation}
We denote the least squares solution by
$\bm{\gamma}^{(k)} = \mytran{(\gamma_0^{(k)},\ldots,\gamma_{m_k-1}^{(k)})}$.
In addition, we set $\mathcal{G}_k = (\Delta{\myvec{g}_{k-m_k}},\ldots,\Delta{\myvec{g}_{k-1}})$
with $\Delta{\myvec{g}_i} = \myvec{g}(\myvec{x}_{i+1}) - \myvec{g}(\myvec{x}_i)$.
Then the updated iteration $\myvec{x}_{k+1}$ in Algorithm \ref{alg:AA} can be written as
$$
\myvec{x}_{k+1} = \myvec{g}(\myvec{x}_k) - \mathcal{G}_k \bm{\gamma}^{(k)},
$$
For $\mathcal{F}_k \in \RS^{n \times m_k}$, since $m_k \ll n$,
we can compute the QR factorization of $\mathcal{F}_k$ via thin QR decomposition.
Let $\mathcal{F}_k = Q_kR_k$ be the thin QR factorization,
where $Q_k \in \RS^{n \times m_k}$ has orthogonal columns
and $R_k \in \RS^{m_k \times m_k}$ is an upper triangular matrix.
Then the unconstrained least squares problem \eqref{optim:alpha_unconstrained}
reduces to
$$
\min_{\bm{\gamma} = \mytran{(\gamma_0,\ldots,\gamma_{m_k-1})}}
\|\mytran{Q}_k\myvec{f}_{k} - R_k\bm{\gamma}\|_2.
$$
Therefore, the least squares solution $\bm{\gamma}^{(k)} \in \RS^{m_k}$ can be obtained by
solving the upper triangular system $R_k\bm{\gamma} = \mytran{Q}_k\myvec{f}_{k}$.
We point out that each matrix $\mathcal{F}_k \in \RS^{n \times m_k}$
in \eqref{optim:alpha_unconstrained} is obtained from $\mathcal{F}_{k-1}$
by appending the new column on the right and,
if the resulting number of columns exceeds the depth $m$,
also deleting the first column on the left.
This means that the QR factorization of $\mathcal{F}_k$ can be efficiently updated
from that of $\mathcal{F}_{k-1}$ using standard QR updating techniques \cite{GolubV1996},
with a computational cost of only $\mathcal{O}(m_kn)$.
It is worth noting that $\mathcal{F}_k$ does not need to be explicitly
constructed in the algorithm implementation.
To delete the first column on the left,
we utilize MATLAB's \texttt{qrdelete} function for efficient QR factorization updating.
For appending a column on the right, we assume the QR factorization of $\mathcal{F}_{k-1} \in \RS^{n \times (m_k-1)}$
is given by $\mathcal{F}_{k-1} = Q_{k-1}R_{k-1}$,
with $Q_{k-1} \in \RS^{n \times (m_k-1)}$ and $R_{k-1} \in \RS^{(m_k-1)\times(m_k-1)}$.
Then the updated matrix $\mathcal{F}_k$ has the form
$$
Q_kR_k = \mathcal{F}_{k}
= [\mathcal{F}_{k-1}, \Delta{\myvec{f}_{k-1}}]
= [Q_{k-1}R_{k-1}, \Delta{\myvec{f}_{k-1}}].
$$
If we set $Q_k = [Q_{k-1},\myvec{q}_{m_k}]$
and $R_k = \begin{bmatrix}
R_{k-1} & \myvec{r}_{m_k} \\ 0 & r_{m_km_k}
\end{bmatrix}$, then the new column $\Delta{\myvec{f}_{k-1}}$ admits the decomposition
$$
\Delta{\myvec{f}_{k-1}} = Q_{k-1}\myvec{r}_{m_k} + r_{m_km_k}\myvec{q}_{m_k}
= [Q_{k-1},\myvec{q}_{m_k}]\begin{bmatrix}
\myvec{r}_{m_k} \\ r_{m_km_k}
\end{bmatrix}.
$$
This means that updating the QR factorization from $\mathcal{F}_{k-1}$ to $\mathcal{F}_k$
requires only computing the QR factorization of the new column $\Delta{\myvec{f}_{k-1}}$
against the existing orthogonal basis $Q_{k-1}$,
which can be efficiently accomplished using a single modified Gram-Schmidt sweep \cite{GolubV1996}.
The implementation of Anderson acceleration for solving the fixed-point problem \eqref{eq:g(u,v)=0}
is summarized in Algorithm \ref{alg:AA_Implementation}.
\begin{algorithm}[t]
\caption{Anderson acceleration for solving the fixed-point problem \eqref{eq:g(u,v)=0}}
\label{alg:AA_Implementation}
\textit{Initialization}.
Given parameters $a \in [0,1)$ and $c \in (0, 1]$.
Choose initial point
$\myvec{x}_0 = [\myvec{u}^{\top}_0,\myvec{v}^{\top}_0]^{\top} = \myvec{0} \in \RS^{2n}$
and the depth $m \geq 1$.
\begin{itemize}[leftmargin=1em,itemindent=3.5em,parsep=0em,itemsep=0em,topsep=0em]
\item[Step 1.]
Form the matrices $P$ and $\tilde{P}$ as defined in \eqref{mat:P_Ptilde}.
\item[Step 2.]
Compute $[\myvec{u}^{\top}_1,\myvec{v}^{\top}_1]^{\top} = \myvec{g}_0 \triangleq \myvec{g}(\myvec{u}_0,\myvec{v}_0)$,
where $\myvec{g}$ is defined by \eqref{eq:g(u,v)=0}.
\item[Step 3.]
Compute the initial residual
$\myvec{f}_0 = \myvec{g}_0 - \myvec{x}_0$.
\end{itemize}
\textit{Iterative process}.
For $k = 1,2,\ldots$ until convergence, do:
\begin{itemize}[leftmargin=1em,itemindent=3.5em,parsep=0em,itemsep=0em,topsep=0em]
\item[Step 1.]
Set $m_k = \min\{m,k\}$.
\item[Step 2.]
Compute $\myvec{g}_k = \myvec{g}(\myvec{u}_k,\myvec{v}_k)$
and the residual $\myvec{f}_k = \myvec{g}_k - \myvec{x}_k$.
\item[Step 3.]
Set $\Delta{\myvec{g}_{k-1}} = \myvec{g}_k - \myvec{g}_{k-1}$
and $\Delta{\myvec{f}_{k-1}} = \myvec{f}_k - \myvec{f}_{k-1}$.
\item[Step 4.]
Update the matrix $\mathcal{G}_k$:
\begin{itemize}
\item If $k = 1$, set $\mathcal{G}_k = \Delta{\myvec{g}_{k-1}}$.
\item If $k \leq m$, set $\mathcal{G}_k = [\mathcal{G}_{k-1},\Delta{\myvec{g}_{k-1}}]$.
\item If $k > m$, set $\mathcal{G}_k = [\mathcal{G}_{k-1}(:,2:m_k),\Delta{\myvec{g}_{k-1}}]$.
\end{itemize}
\item[Step 5.]
Update the QR factorization $\mathcal{F}_k = Q_kR_k$:
\begin{itemize}
\item If $k = 1$, set $R_k = \|\Delta{\myvec{f}_{k-1}}\|_2$
and $Q_k = \Delta{\myvec{f}_{k-1}}/R_k$.
\item If $k > 1$ and $k > m$,
update the QR factorization by deleting the first column of $\mathcal{F}_{k-1} = Q_{k-1}R_{k-1}$
using the MATLAB's \texttt{qrdelete} function:
$[Q_k,R_k] = \texttt{qrdelete}(Q_{k-1},R_{k-1},1)$.
\item If $k > 1$, append the new column $\Delta{\myvec{f}_{k-1}}$ to $\mathcal{F}_{k-1}$
by using a single modified Gram-Schmidt sweep.
\end{itemize}
\item[Step 6.]
Solve the upper triangular system $R_k\bm{\gamma} = \mytran{Q}_k\myvec{f}_{k}$
to obtain the least squares solution $\bm{\gamma}^{(k)}$.
\item[Step 7.]
Update the next iteration
$\myvec{x}_{k+1} = [\myvec{u}_{k+1}^{\top},\myvec{v}_{k+1}^{\top}]^{\top}
= \myvec{g}_k - \mathcal{G}_k\bm{\gamma}^{(k)}$.
\end{itemize}
\end{algorithm}
%--------------------------------------------------------------------
\subsection{Numerical experiments}
\label{subsec:NumericalExperiments}
This subsection presents numerical experiments that demonstrate
the effectiveness of Anderson acceleration in reducing both the number of iterations
and the overall execution time of the fixed-point iterative methods
for computing the minimal nonnegative solution of the nonlinear system \eqref{eq:f(u,v)=0},
or equivalently, the fixed-point problem \eqref{eq:g(u,v)=0},
which stems from a nonsymmetric algebraic Riccati equation in neutron transport theory.
All experiments were conducted in MATLAB R2024b on a MacBook Pro
equipped with an Apple M3 8-core CPU and 24 GB of RAM.
The algorithms tested are listed below,
with abbreviations used in the corresponding tables and figures.
\begin{itemize}
\item AA($m$) (for Anderson acceleration with depth $m$)
is our implementation of Algorithm \ref{alg:AA_Implementation}.
\item FP is the algorithm from \cite{Lu2005a} by using the fixed-point iteration.
\item MFP is the modified fixed-point iteration algorithm from \cite{BaoLW2006}.
\item NBJ is the nonlinear block Jacobi iteration algorithm proposed in \cite{BaiGL2008}.
\item NBGS is the nonlinear block Gauss-Seidel iteration algorithm from \cite{BaiGL2008}.
\end{itemize}
The initial point $\myvec{x}_0$ is set to $\myvec{0}$ for all the algorithms considered above.
Following Example 5.2 of \cite{GuoL2000b},
the constants $c_i$ and $\omega_i$ are obtained using a numerical quadrature on $[0,1]$,
by dividing the interval into $n/4$ equal subintervals
and applying four nodes Gauss-Legendre quadrature on each.
The stopping criterion for all algorithms considered above is given by
$$
\text{RES} := \max\left\{\frac{\|\myvec{u}_{k+1} - \myvec{u}_k\|_{\infty}}{\|\myvec{u}_{k+1}\|_{\infty}},
\frac{\|\myvec{v}_{k+1} - \myvec{v}_k\|_{\infty}}{\|\myvec{v}_{k+1}\|_{\infty}}\right\}
\leq n\cdot\texttt{eps},
$$
where $n$ is the matrix size from \eqref{eq:NARE}
and $\texttt{eps} = 2^{-52} \approx 2.2204 \times 10^{-16}$
denotes the double-precision machine epsilon.
The CPU time is measured in seconds using MATLAB's \texttt{tic/toc} commands.
Each experiment is repeated 10 times, and the average runtime are reported
in the tables and figures below.
The number of iterations is denoted by IT.
We note that the fixed-point iterative methods proposed in \cite{Lu2005a,BaoLW2006,BaiGL2008}
exhibit linear convergence when the pair of parameters $(a,c) \neq (0,1)$.
A detailed theoretical analysis comparing the convergence rates of these methods
is provided in \cite{GuoL2010}.
The singular case $(a,c) = (0,1)$, known to be particularly challenging,
has been effectively solved by various Newton-type methods
\cite{LinBW2008,LinB2008,HuangKH2010,LingLL2022,LiangL2025}
or alternative approaches \cite{BiniIP2008,MehrmannX2008}.
Such techniques are generally recommended when $(a,c)$ is close to $(0,1)$,
while fixed-point iterative methods are more suitable for the regular cases.
Our numerical results below demonstrate that AA performs efficiently in both cases.
We begin with the regular case $(a,c) = (0.5,0.5)$.
Figure \ref{fig:IterHistory_alphaC0.5} shows the iteration histories
for the problem sizes $n = 1024, 2048, 4096, 8192$.
It is observed that AA with various depths requires fewer iterations than FP, MFP and NBJ,
although it still requires more iterations than NBGS.
As $(a,c)$ approaches the singular case $(0,1)$,
AA becomes increasingly efficient and eventually outperforms NBGS in terms of iteration count,
as illustrated in Figures \ref{fig:IterHistory_alphaC10-1} and \ref{fig:IterHistory_alphaC10-3}.
\begin{figure}
\centering
\subfigure{\includegraphics[width=0.42\textwidth]{alpha0.5c0.5_n1024}}\quad
\subfigure{\includegraphics[width=0.42\textwidth]{alpha0.5c0.5_n2048}} \\
\subfigure{\includegraphics[width=0.42\textwidth]{alpha0.5c0.5_n4096}}\quad
\subfigure{\includegraphics[width=0.42\textwidth]{alpha0.5c0.5_n8192}} \\
\caption{Iteration histories for $(a, c) = (0.5,0.5)$ with various problem sizes $n = 1024, 2048, 4096, 8192$.}
\label{fig:IterHistory_alphaC0.5}
\end{figure}
\begin{figure}
\centering
\subfigure{\includegraphics[width=0.42\textwidth]{alpha10-1c1-10-1_n1024}}\quad
\subfigure{\includegraphics[width=0.42\textwidth]{alpha10-1c1-10-1_n2048}} \\
\subfigure{\includegraphics[width=0.42\textwidth]{alpha10-1c1-10-1_n4096}}\quad
\subfigure{\includegraphics[width=0.42\textwidth]{alpha10-1c1-10-1_n8192}} \\
\caption{Iteration histories for $(a, c) = (10^{-1},1-10^{-1})$ with various problem sizes $n = 1024, 2048, 4096, 8192$.}
\label{fig:IterHistory_alphaC10-1}
\end{figure}
\begin{figure}
\centering
\subfigure{\includegraphics[width=0.42\textwidth]{alpha10-3c1-10-3_n1024}}\quad
\subfigure{\includegraphics[width=0.42\textwidth]{alpha10-3c1-10-3_n2048}} \\
\subfigure{\includegraphics[width=0.42\textwidth]{alpha10-3c1-10-3_n4096}}\quad
\subfigure{\includegraphics[width=0.42\textwidth]{alpha10-3c1-10-3_n8192}} \\
\caption{Iteration histories for $(a, c) = (10^{-3},1-10^{-3})$ with various problem sizes $n = 1024, 2048, 4096, 8192$.}
\label{fig:IterHistory_alphaC10-3}
\end{figure}
Tables \ref{tab:n1024}, \ref{tab:n2048}, \ref{tab:n4096} and \ref{tab:n8192}
report the overall numerical results for seven test cases
with problem sizes $n = 1024, 2048, 4096$ and $8192$,
further illustrating the significant improvement in the performance of AA
over other fixed-point iterative methods.
Specifically, AA requires fewer iterations and less computation time than
FP, MFP and NBJ for each problem size across all seven test cases.
The only exceptions are the regular cases $(a,c) = (0.9,0.1)$ and $(0.1,0.9)$,
where AA slightly underperforms NBGS in terms of iteration count and computation time.
In all other cases, AA demonstrates superior performance compared to
the four fixed-point iterative methods,
with its advantages becoming particularly significant
as the pair of parameters $(a,c)$ approaches the singular case $(0,1)$.
In such cases, AA achieves significantly fewer iterations, reduced computation time,
and improved numerical accuracy.
For instance, compared to NBGS, even with depth $m = 1$,
AA achieves a 720-fold decrease in the number of iterations
for the case $(a,c) = (10^{-9},1-10^{-9})$ when $n = 1024$,
and a 970-fold decrease for the same case when $n = 8192$.
\begin{sidewaystable}
\centering
\caption{Numerical results for $n = 1024$}
\label{tab:n1024}
\begin{tabular}{cccccccccc}
\toprule
$(a,c)$ & Item & AA(1) & AA(3) & AA(5) & AA(8) & FP & MFP & NBJ & NBGS \\
\midrule
$(0.9,0.1)$ & IT & 7 & 6 & 6 & 6 & 9 & 8 & 7 & 5 \\
& CPU & 0.0119 & 0.0111 & 0.0127 & 0.0100 & 0.0111 & 0.0117 & 0.0117 & 0.0107 \\
& RES & 3.7180e-15 & 8.5951e-14 & 6.2769e-14 & 6.2769e-14 & 6.3425e-15 & 1.7322e-13 & 6.2987e-14 & 4.4191e-14 \\
$(0.1,0.9)$ & IT & 37 & 22 & 20 & 19 & 71 & 58 & 39 & 21 \\
& CPU & 0.0159 & 0.0141 & 0.0177 & 0.0171 & 0.0203 & 0.0163 & 0.0158 & 0.0139 \\
& RES & 1.2577e-13 & 7.3995e-14 & 5.6501e-14 & 3.7234e-14 & 1.9562e-13 & 2.1107e-13 & 1.9645e-13 & 1.2801e-13 \\
$(10^{-2},1-10^{-2})$ & IT & 70 & 29 & 25 & 23 & 242 & 194 & 117 & 61 \\
& CPU & 0.0196 & 0.0154 & 0.0162 & 0.0173 & 0.0453 & 0.0303 & 0.0244 & 0.0195 \\
& RES & 1.1239e-13 & 7.4448e-14 & 2.0664e-13 & 1.0577e-13 & 2.2173e-13 & 2.1277e-13 & 1.9614e-13 & 1.7487e-13 \\
$(10^{-4},1-10^{-4})$ & IT & 119 & 42 & 34 & 34 & 2100 & 1667 & 955 & 494 \\
& CPU & 0.0293 & 0.0176 & 0.0171 & 0.0169 & 0.2723 & 0.2225 & 0.1309 & 0.0620 \\
& RES & 1.2653e-13 & 1.4410e-14 & 5.3285e-14 & 3.4963e-14 & 2.2656e-13 & 2.2689e-13 & 2.2176e-13 & 2.1726e-13 \\
$(10^{-6},1-10^{-6})$ & IT & 114 & 57 & 41 & 42 & 16528 & 13143 & 7531 & 3915 \\
& CPU & 0.0274 & 0.0207 & 0.0191 & 0.0185 & 2.5279 & 1.8203 & 1.1512 & 0.4395 \\
& RES & 6.2976e-14 & 1.0726e-13 & 1.1131e-14 & 9.1490e-16 & 2.2722e-13 & 2.2676e-13 & 2.2554e-13 & 2.2646e-13 \\
$(10^{-8},1-10^{-8})$ & IT & 108 & 53 & 41 & 48 & 119319 & 95406 & 55398 & 29168 \\
& CPU & 0.0282 & 0.0237 & 0.0173 & 0.0260 & 18.1731 & 13.2879 & 6.8167 & 3.2711 \\
& RES & 1.5058e-13 & 7.3612e-14 & 1.7471e-13 & 1.9487e-13 & 2.2733e-13 & 2.2733e-13 & 2.2687e-13 & 2.2626e-13 \\
$(10^{-9},1-10^{-9})$ & IT & 106 & 62 & 49 & 52 & 304534 & 244626 & 143488 & 76421 \\
& CPU & 0.0283 & 0.0283 & 0.0188 & 0.0241 & 43.7372 & 36.4205 & 16.3861 & 10.2075 \\
& RES & 6.7664e-14 & 1.0402e-13 & 2.0956e-13 & 2.7799e-14 & 2.2715e-13 & 2.2730e-13 & 2.2730e-13 & 2.2730e-13 \\
\bottomrule
\end{tabular}
\end{sidewaystable}
\begin{sidewaystable}
\centering
\caption{Numerical results for $n = 2048$}
\label{tab:n2048}
\begin{tabular}{cccccccccc}
\toprule
$(a,c)$ & Item & AA(1) & AA(3) & AA(5) & AA(8) & FP & MFP & NBJ & NBGS \\
\midrule
$(0.9,0.1)$ & IT & 6 & 6 & 6 & 6 & 8 & 8 & 7 & 5 \\
& CPU & 0.0593 & 0.0554 & 0.0554 & 0.0554 & 0.0583 & 0.0608 & 0.0589 & 0.0556 \\
& RES & 3.7858e-13 & 8.6170e-14 & 6.2769e-14 & 6.2769e-14 & 3.8689e-13 & 1.7321e-13 & 6.2987e-14 & 4.4191e-16 \\
$(0.1,0.9)$ & IT & 37 & 22 & 19 & 18 & 69 & 57 & 38 & 21 \\
& CPU & 0.1070 & 0.0836 & 0.0780 & 0.0768 & 0.1528 & 0.1362 & 0.1054 & 0.0800 \\
& RES & 1.2564e-13 & 7.3519e-14 & 3.3556e-13 & 2.3628e-13 & 4.3260e-13 & 3.4456e-13 & 4.2847e-13 & 1.2863e-13 \\
$(10^{-2},1-10^{-2})$ & IT & 69 & 28 & 25 & 23 & 236 & 189 & 114 & 59 \\
& CPU & 0.1595 & 0.0932 & 0.0883 & 0.0853 & 0.4168 & 0.3377 & 0.2213 & 0.1389 \\
& RES & 3.0851e-13 & 4.4862e-13 & 2.0830e-13 & 1.0773e-13 & 4.1767e-13 & 4.1541e-13 & 3.8832e-13 & 4.3851e-13 \\
$(10^{-4},1-10^{-4})$ & IT & 115 & 42 & 33 & 34 & 2031 & 1613 & 925 & 478 \\
& CPU & 0.2363 & 0.1179 & 0.1024 & 0.1048 & 3.2430 & 2.5565 & 1.4444 & 0.7820 \\
& RES & 2.4404e-13 & 3.2872e-13 & 4.3675e-14 & 3.7916e-14 & 4.5448e-13 & 4.5219e-13 & 4.4473e-13 & 4.4831e-13 \\
$(10^{-6},1-10^{-6})$ & IT & 100 & 45 & 38 & 41 & 15833 & 12598 & 7230 & 3766 \\
& CPU & 0.2118 & 0.1365 & 0.1111 & 0.1173 & 24.9623 & 19.9647 & 11.0088 & 5.8172 \\
& RES & 2.7612e-13 & 4.2402e-13 & 4.0557e-14 & 1.5095e-14 & 4.5472e-13 & 4.5456e-13 & 4.5410e-13 & 4.5196e-13 \\
$(10^{-8},1-10^{-8})$ & IT & 109 & 43 & 39 & 52 & 112384 & 89976 & 52336 & 27666 \\
& CPU & 0.2277 & 0.1305 & 0.1127 & 0.1381 & 174.8984 & 142.5888 & 79.3217 & 42.3485 \\
& RES & 1.8417e-13 & 2.6724e-13 & 1.9501e-13 & 1.5734e-13 & 4.5462e-13 & 4.5462e-13 & 4.5386e-13 & 4.5401e-13 \\
$(10^{-9},1-10^{-9})$ & IT & 108 & 55 & 55 & 50 & 282652 & 227488 & 134008 & 71720 \\
& CPU & 0.2248 & 0.1393 & 0.1401 & 0.1329 & 433.1914 & 353.6441 & 204.5411 & 109.9595 \\
& RES & 3.9098e-14 & 3.4700e-13 & 1.0996e-14 & 1.0370e-13 & 4.5472e-13 & 4.5441e-13 & 4.5472e-13 & 4.5426e-13 \\
\bottomrule
\end{tabular}
\end{sidewaystable}
\begin{sidewaystable}
\centering
\caption{Numerical results for $n = 4096$}
\label{tab:n4096}
\begin{tabular}{cccccccccc}
\toprule
$(a,c)$ & Item & AA(1) & AA(3) & AA(5) & AA(8) & FP & MFP & NBJ & NBGS \\
\midrule
$(0.9,0.1)$ & IT & 6 & 6 & 6 & 6 & 8 & 8 & 7 & 5 \\
& CPU & 0.2382 & 0.2259 & 0.2301 & 0.2347 & 0.2487 & 0.2503 & 0.2455 & 0.2290 \\
& RES & 3.7858e-13 & 8.6389e-14 & 6.2987e-14 & 6.2987e-14 & 3.8711e-13 & 1.7321e-13 & 6.2768e-14 & 4.4191e-16 \\
$(0.1,0.9)$ & IT & 36 & 21 & 19 & 18 & 68 & 56 & 37 & 20 \\
& CPU & 0.4272 & 0.3255 & 0.3156 & 0.3157 & 0.6352 & 0.5604 & 0.4404 & 0.3205 \\
& RES & 4.9122e-13 & 6.7939e-13 & 3.3733e-13 & 2.3734e-13 & 6.4263e-13 & 5.6174e-13 & 8.8138e-13 & 5.7073e-13 \\
$(10^{-2},1-10^{-2})$ & IT & 68 & 26 & 24 & 23 & 229 & 184 & 111 & 58 \\
& CPU & 0.6622 & 0.3572 & 0.3515 & 0.3493 & 1.6709 & 1.3866 & 0.9160 & 0.5644 \\
& RES & 7.3474e-13 & 6.2237e-13 & 5.6842e-13 & 1.0826e-13 & 8.7396e-13 & 8.1203e-13 & 7.7411e-13 & 6.9077e-13 \\
$(10^{-4},1-10^{-4})$ & IT & 112 & 37 & 32 & 32 & 1962 & 1559 & 895 & 463 \\
& CPU & 0.9254 & 0.4330 & 0.4048 & 0.4094 & 12.8604 & 10.3231 & 5.9628 & 3.1721 \\
& RES & 7.0958e-13 & 8.2890e-13 & 1.3132e-13 & 4.6285e-13 & 9.0652e-13 & 9.0419e-13 & 8.8695e-13 & 9.0046e-13 \\
$(10^{-6},1-10^{-6})$ & IT & 100 & 52 & 38 & 40 & 15140 & 12057 & 6928 & 3615 \\
& CPU & 0.8389 & 0.5318 & 0.4469 & 0.4637 & 97.8450 & 77.7533 & 45.1426 & 23.4178 \\
& RES & 4.7920e-13 & 2.1132e-13 & 7.0744e-14 & 1.3569e-14 & 9.0909e-13 & 9.0817e-13 & 9.0878e-13 & 9.0725e-13 \\
$(10^{-8},1-10^{-8})$ & IT & 104 & 52 & 241 & 50 & 105468 & 84556 & 49331 & 26165 \\
& CPU & 0.8636 & 0.5297 & 0.4672 & 0.5346 & 678.0004 & 543.9350 & 319.1383 & 168.3982 \\
& RES & 7.3068e-13 & 8.3467e-13 & 6.6746e-13 & 1.2801e-13 & 9.0905e-13 & 9.0874e-13 & 9.0920e-13 & 9.0737e-13 \\
$(10^{-9},1-10^{-9})$ & IT & 107 & 56 & 49 & 60 & 260734 & 210351 & 124431 & 66969 \\
& CPU & 0.8816 & 0.5588 & 0.5203 & 0.5971 & 1683.4272 & 1353.8946 & 801.2046 & 561.6521 \\
& RES & 2.2160e-13 & 6.7625e-13 & 4.9518e-13 & 3.8368e-13 & 9.0909e-13 & 9.0833e-13 & 9.0940e-13 & 9.0864e-13 \\
\bottomrule
\end{tabular}
\end{sidewaystable}
\begin{sidewaystable}
\centering
\caption{Numerical results for $n = 8192$}
\label{tab:n8192}
\begin{tabular}{cccccccccc}
\toprule
$(a,c)$ & Item & AA(1) & AA(3) & AA(5) & AA(8) & FP & MFP & NBJ & NBGS \\
\midrule
$(0.9,0.1)$ & IT & 6 & 6 & 6 & 6 & 8 & 8 & 7 & 5 \\
& CPU & 1.0840 & 1.0987 & 1.0951 & 1.0877 & 1.1754 & 1.1855 & 1.1403 & 1.0552 \\
& RES & 3.7858e-13 & 8.6389e-14 & 6.3206e-14 & 6.3206e-14 & 3.8711e-13 & 1.7321e-13 & 6.2768e-14 & 4.4191e-16 \\
$(0.1,0.9)$ & IT & 33 & 21 & 18 & 17 & 66 & 54 & 37 & 20 \\
& CPU & 2.0959 & 1.6639 & 1.5476 & 1.5151 & 3.3465 & 3.3578 & 2.5705 & 1.6243 \\
& RES & 1.7853e-12 & 6.8092e-13 & 1.7769e-12 & 9.6635e-13 & 1.4162e-12 & 1.4974e-12 & 8.8421e-13 & 5.7260e-13 \\
$(10^{-2},1-10^{-2})$ & IT & 63 & 25 & 22 & 21 & 223 & 178 & 108 & 56 \\
& CPU & 3.1793 & 1.8143 & 1.6897 & 1.6514 & 9.1696 & 9.0883 & 5.7975 & 2.9814 \\
& RES & 1.8084e-12 & 1.7286e-12 & 1.3659e-12 & 1.2474e-12 & 1.6488e-12 & 1.8117e-12 & 1.5294e-12 & 1.7226e-12 \\
$(10^{-4},1-10^{-4})$ & IT & 109 & 34 & 31 & 32 & 1893 & 1505 & 864 & 448 \\
& CPU & 4.9374 & 2.1441 & 2.0285 & 2.0497 & 71.3948 & 70.9941 & 41.1794 & 20.8786 \\
& RES & 9.8466e-13 & 6.5734e-13 & 4.6889e-13 & 1.9392e-13 & 1.8175e-12 & 1.8035e-12 & 1.8150e-12 & 1.7896e-12 \\
$(10^{-6},1-10^{-6})$ & IT & 100 & 49 & 37 & 39 & 14448 & 11514 & 6628 & 3466 \\
& CPU & 4.5839 & 2.6971 & 2.2662 & 2.3174 & 184.0034 & 164.5215 & 95.1652 & 50.3798 \\
& RES & 6.5132e-13 & 1.3125e-12 & 6.6824e-13 & 1.0122e-12 & 1.8180e-12 & 1.8157e-12 & 1.8181e-12 & 1.8102e-12 \\
$(10^{-8},1-10^{-8})$ & IT & 87 & 49 & 42 & 46 & 98540 & 79120 & 46324 & 24660 \\
& CPU & 4.1214 & 2.6954 & 2.4590 & 2.6037 & 990.0935 & 746.2888 & 386.8558 & 233.7374 \\
& RES & 1.4841e-12 & 7.6747e-13 & 1.0326e-12 & 8.0241e-13 & 1.8188e-12 & 1.8174e-12 & 1.8174e-12 & 1.8178e-12 \\
$(10^{-9},1-10^{-9})$ & IT & 64 & 49 & 44 & 49 & 238855 & 193179 & 114927 & 62217 \\
& CPU & 3.2395 & 2.7019 & 2.5261 & 2.6625 & 2146.4811 & 1724.1584 & 895.6699 & 517.4697 \\
& RES & 1.6310e-12 & 1.7158e-12 & 6.5997e-13 & 1.5136e-12 & 1.8188e-12 & 1.8185e-12 & 1.8183e-12 & 1.8189e-12 \\
\bottomrule
\end{tabular}
\end{sidewaystable}
It is worth noting that, for each test case,
the number of iterations tends to stagnate as the problem size $n$ grows.
Furthermore, for each fixed problem size,
as the pair of parameters $(a,c)$ approaches the singular case $(0,1)$,
the number of iterations and computation time required by AA increase only slightly,
whereas those of the other four fixed-point iterative methods increase significantly.
This advantage of AA is further illustrated in
Figures \ref{fig:IterTimeHistory_alphaC10-3}, \ref{fig:IterTimeHistory_alphaC10-5}
and \ref{fig:IterTimeHistory_alphaC10-7},
which depict the iteration histories for the nearly singular cases
$(a,c) = (10^{-3},1-10^{-3})$, $(10^{-5},1-10^{-5})$ and $(10^{-7},1-10^{-7})$,
respectively, with problem sizes $n = 4096$ and $8192$.
\begin{figure}
\centering
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-3c1-10-3_n4096_iter}}\quad
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-3c1-10-3_n4096_time}} \\
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-3c1-10-3_n8192_iter}}\quad
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-3c1-10-3_n8192_time}} \\
\caption{Comparison of Anderson acceleration and other fixed-point methods
for $(a, c) = (10^{-3},1-10^{-3})$ with problem sizes $n = 4096, 8192$.
Left: number of iterations. Right: elapsed time.}
\label{fig:IterTimeHistory_alphaC10-3}
\end{figure}
\begin{figure}
\centering
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-5c1-10-5_n4096_iter}}\quad
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-5c1-10-5_n4096_time}} \\
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-5c1-10-5_n8192_iter}}\quad
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-5c1-10-5_n8192_time}} \\
\caption{Comparison of Anderson acceleration and other fixed-point methods
for $(a, c) = (10^{-5},1-10^{-5})$ with problem sizes $n = 4096, 8192$.
Left: number of iterations. Right: elapsed time.}
\label{fig:IterTimeHistory_alphaC10-5}
\end{figure}
\begin{figure}
\centering
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-7c1-10-7_n4096_iter}}\quad
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-7c1-10-7_n4096_time}} \\
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-7c1-10-7_n8192_iter}}\quad
\subfigure{\includegraphics[width=0.48\textwidth]{alpha10-7c1-10-7_n8192_time}} \\
\caption{Comparison of Anderson acceleration and other fixed-point methods
for $(a, c) = (10^{-7},1-10^{-7})$ with problem sizes $n = 4096, 8192$.
Left: number of iterations. Right: elapsed time.}
\label{fig:IterTimeHistory_alphaC10-7}
\end{figure}
To conclude, compared to the other four fixed-point iterative methods,
we have found that AA for the NARE,
with varying depths $m$, yields comparable performance in the regular cases.
In addition, for nearly singular cases and large-scale problems,
AA achieves a significant reduction in both the number of iterations and execution time,
while exhibiting better desired accuracy in most cases.
%------------------------------------------------------------------------------
\section{Conclusions}
\label{sec:Conclusions}
In this paper, we have presented a new local convergence analysis
for Anderson acceleration applied to nonlinear equations under the assumptions that
the first derivative of nonlinear operator is H\"{o}lder continuous,
and that the associated fixed-point operator is contractive.
The main results are encapsulated in Theorem \ref{th:LocalConvAnderson(m)},
which shows that Anderson acceleration is R-linear convergent.
This convergence rate is explicitly characterized by the unique root of equation \eqref{eq:q}.
In particular, when H\"{o}lder continuity reduces to Lipschitz continuity,
the resulting convergence factor is not determined only by the contraction factor $\theta$.
It also depends on the optimization gain $\eta_k$ defined in \eqref{cons:eta_k}
and the condition number $\kappa(\myvec{f}'(\myvec{x}^*))$.
Consequently, our analysis yields a more refined characterization of convergence behavior
compared to previous studies that rely exclusively on the contraction factor $\theta$.
Moreover, we obtained a new convergence rate of Anderson acceleration
for depth $m=1$.
We further demonstrated the applicability and efficiency of Anderson acceleration
by solving the approximation of minimal positive solution of
the special nonlinear equation \eqref{eq:f(u,v)=0},
which is derived from NARE \eqref{eq:NARE} arising from neutron transport theory.
The numerical results confirm that Anderson acceleration
performs efficiently in both regular and nearly singular cases,
especially for large-scale problems.
One goal of our future work is to explore whether Anderson acceleration
can be applied to aid in the convergence of Newton's method
for special singular nonlinear equations.
%------------------------------------------------------------------------------
\section*{Acknowledgments}
This work was supported by the Fujian Province Natural Science Foundation of China (Grant No. 2022J01896),
the Education Research Projects for Young Teachers of Fujian Provincial Education Department (Grant No. JAT220197),
Fujian Key Laboratory of Granular Computing and Applications,
and Fujian Key Laboratory of Data Science and Statistics.
%------------------------------------------------------------------------------
% \bibliography{LingBib}
% \bibliographystyle{mysiam}
%----------------------------------------------------------
\begin{thebibliography}{10}
\bibitem{AnJW2017}
{\sc H.~An, X.~Jia, and H.~F. Walker}, {\em Anderson acceleration and
application to the three-temperature energy equations}, J. Comput. Phys., 347
(2017), pp.~1--19.
\bibitem{Anderson1965}
{\sc D.~G. Anderson}, {\em Iterative procedures for nonlinear integral
equations}, J. ACM, 12 (1965), pp.~547--560.
\bibitem{Anderson2019}
{\sc D.~G. Anderson}, {\em {Comments on “Anderson acceleration, mixing and
extrapolation”}}, Numer. Algorithms, 80 (2019), pp.~135--234.
\bibitem{Argyros1992}
{\sc I.~K. ARGYROS}, {\em {Remarks on the convergence of Newton's method under
H{\"o}lder continuity conditions}}, Tamkang J. Math., 23 (1992),
pp.~269--277.
\bibitem{Bach2021}
{\sc F.~Bach}, {\em {On the effectiveness of Richardson extrapolation in data
science}}, SIAM J. Math. Data Sci., 3 (2021), pp.~1251--1277.
\bibitem{BaiGL2008}
{\sc Z.~Bai, Y.~Gao, and L.~Lu}, {\em {Fast iterative schemes for nonsymmetric
algebraic Riccati equations arising from transport theory}}, SIAM J. Sci.
Comput., 30 (2008), pp.~804--818.
\bibitem{BaoLW2006}
{\sc L.~Bao, Y.~Lin, and Y.~Wei}, {\em {A modified simple iterative method for
nonsymmetric algebraic Riccati equations arising in transport theory}}, Appl.
Math. Comput., 181 (2006), pp.~1499--1504.
\bibitem{BarreTDA2022}
{\sc M.~Barr{\'e}, A.~Taylor, and A.~d'Aspremont}, {\em Convergence of a
constrained vector extrapolation scheme}, SIAM J. Math. Data Sci., 4 (2022),
pp.~979--1002.
\bibitem{BianC2022}
{\sc W.~Bian and X.~Chen}, {\em Anderson acceleration for nonsmooth fixed point
problems}, SIAM J. Numer. Anal., 60 (2022), pp.~2565--2591.
\bibitem{BianCK2021}
{\sc W.~Bian, X.~Chen, and C.~T. Kelley}, {\em Anderson acceleration for a
class of nonsmooth fixed-point problems}, SIAM J. Sci. Comput., 43 (2021),
pp.~S1--S20.
\bibitem{BiniIP2008}
{\sc D.~A. Bini, B.~Iannazzo, and F.~Poloni}, {\em {A fast Newton's method for
a nonsymmetric algebraic Riccati equation}}, SIAM J. Matrix Anal. Appl., 30
(2008), pp.~276--290.
\bibitem{BollapragadaSD2023}
{\sc R.~Bollapragada, D.~Scieur, and A.~d'Aspremont}, {\em Nonlinear
acceleration of momentum and primal-dual algorithms}, Math. Program., 198
(2023), pp.~325--362.
\bibitem{BrezinskiR2019}
{\sc C.~Brezinski and M.~Redivo-Zaglia}, {\em {The genesis and early
developments of Aitken's process, Shanks' transformation, the
$\varepsilon$--algorithm, and related fixed point methods}}, Numer.
Algorithms, 80 (2019), pp.~11--133.
\bibitem{BrezinskiRS2018}
{\sc C.~Brezinski, M.~Redivo-Zaglia, and Y.~Saad}, {\em {Shanks sequence
transformations and Anderson acceleration}}, SIAM Rev., 60 (2018),
pp.~646--669.
\bibitem{CartisGT2017}
{\sc C.~Cartis, N.~I. Gould, and P.~L. Toint}, {\em {Worst-case evaluation
complexity of regularization methods for smooth unconstrained optimization
using H{\"o}lder continuous gradients}}, Optim. Methods Softw., 32 (2017),
pp.~1273--1298.
\bibitem{CartisGT2019}
{\sc C.~Cartis, N.~I. Gould, and P.~L. Toint}, {\em Universal regularization
methods: varying the power, the smoothness and the accuracy}, SIAM J. Optim.,
29 (2019), pp.~595--615.
\bibitem{ChenK2019}
{\sc X.~Chen and C.~T. Kelley}, {\em {Convergence of the EDIIS algorithm for
nonlinear equations}}, SIAM J. Sci. Comput., 41 (2019), pp.~A365--A379.
\bibitem{Catinas2021}
{\sc E.~C\u{a}tina\c{s}}, {\em How many steps still left to $x^*$?}, SIAM Rev.,
63 (2021), pp.~585--624.
\bibitem{DeSterckH2021}
{\sc H.~De~Sterck and Y.~He}, {\em {On the asymptotic linear convergence speed
of Anderson acceleration, Nesterov acceleration, and nonlinear GMRES}}, SIAM
J. Sci. Comput., 43 (2021), pp.~S21--S46.
\bibitem{DevolderGN2014}
{\sc O.~Devolder, F.~Glineur, and Y.~Nesterov}, {\em First-order methods of
smooth convex optimization with inexact oracle}, Math. Program., 146 (2014),
pp.~37--75.
\bibitem{EvansPRX2020}
{\sc C.~Evans, S.~Pollock, L.~G. Rebholz, and M.~Xiao}, {\em {A proof that
Anderson acceleration improves the convergence rate in linearly converging
fixed-point methods (but not in those converging quadratically)}}, SIAM J.
Numer. Anal., 58 (2020), pp.~788--810.
\bibitem{FangS2009}
{\sc H.-r. Fang and Y.~Saad}, {\em Two classes of multisecant methods for
nonlinear acceleration}, Numer. Linear Algebra Appl., 16 (2009),
pp.~197--221.
\bibitem{Ferreira2009}
{\sc O.~P. Ferreira}, {\em Local convergence of {Newton's} method in {Banach}
space from the viewpoint of the majorant principle}, IMA J. Numer. Anal., 29
(2009), pp.~746--759.
\bibitem{FuZB2020}
{\sc A.~Fu, J.~Zhang, and S.~Boyd}, {\em {Anderson accelerated
Douglas--Rachford splitting}}, SIAM J. Sci. Comput., 42 (2020),
pp.~A3560--A3583.
\bibitem{GhadimiLZ2019}
{\sc S.~Ghadimi, G.~Lan, and H.~Zhang}, {\em Generalized uniformly optimal
methods for nonlinear programming}, J. Sci. Comput., 79 (2019),
pp.~1854--1881.
\bibitem{GolubV1996}
{\sc G.~H. Golub and C.~F.~V. Loan}, {\em {Matrix Computations, Fourth
Edition}}, The Johns Hopkins University Press, Baltimore and London, 2013.
\bibitem{GrapigliaN2017}
{\sc G.~N. Grapiglia and Y.~Nesterov}, {\em {Regularized Newton methods for
minimizing functions with H\"{o}lder continuous Hessians}}, SIAM J. Optim.,
27 (2017), pp.~478--506.
\bibitem{GrapigliaN2019}
{\sc G.~N. Grapiglia and Y.~Nesterov}, {\em {Accelerated regularized Newton
methods for minimizing composite convex functions}}, SIAM J. Optim., 29
(2019), pp.~77--99.
\bibitem{GrapigliaN2020}
{\sc G.~N. Grapiglia and Y.~Nesterov}, {\em {Tensor methods for minimizing
convex functions with H\"{o}lder continuous higher-order derivatives}}, SIAM
J. Optim., 30 (2020), pp.~2750--2779.
\bibitem{Guo2001}
{\sc C.-H. Guo}, {\em {Nonsymmetric algebraic Riccati equations and
Wiener--Hopf factorization for M-matrices}}, SIAM J. Matrix Anal. Appl., 23
(2001), pp.~225--242.
\bibitem{GuoL2000b}
{\sc C.-H. Guo and A.~J. Laub}, {\em {On the iterative solution of a class of
nonsymmetric algebraic Riccati equations}}, SIAM J. Matrix Anal. Appl., 22
(2000), pp.~376--391.
\bibitem{GuoL2010}
{\sc C.-H. Guo and W.-W. Lin}, {\em {Convergence rates of some iterative
methods for nonsymmetric algebraic Riccati equations arising in transport
theory}}, Linear Algebra Appl., 432 (2010), pp.~283--291.
\bibitem{Hernandez2001}
{\sc M.~A. Hern{\'a}ndez}, {\em {The Newton method for operators with
H{\"o}lder continuous first derivative}}, J. Optim. Theory Appl., 109 (2001),
pp.~631--648.
\bibitem{HighamS2016}
{\sc N.~J. Higham and N.~Strabi{\'c}}, {\em Anderson acceleration of the
alternating projections method for computing the nearest correlation matrix},
Numer. Algorithms, 72 (2016), pp.~1021--1042.
\bibitem{HuangM2020}
{\sc B.~Huang and C.~Ma}, {\em {Some accelerated iterative algorithms for
solving nonsymmetric algebraic Riccati equations arising in transport
theory}}, Int. J. Comput. Math., 97 (2020), pp.~1819--1839.
\bibitem{HuangM2014}
{\sc N.~Huang and C.~Ma}, {\em {Some predictor--corrector-type iterative
schemes for solving nonsymmetric algebraic Riccati equations arising in
transport theory}}, Numer. Linear Algebra Appl., 21 (2014), pp.~761--780.
\bibitem{Huang2002}
{\sc Z.~Huang}, {\em On {Newton's} method under {H\"{o}lder} continuous
derivative}, J. Math. Anal. Appl., 270 (2002), pp.~332--229.
\bibitem{Huang2004}
{\sc Z.~Huang}, {\em The convergence ball of {Newton's} method and the
uniqueness ball of equations under {H\"{o}lder}-type continuous derivatives},
Comput. Math. Appl., 47 (2004), pp.~247--251.
\bibitem{HuangKH2010}
{\sc Z.~Huang, X.~Kong, and W.~Hu}, {\em {The King--Werner method for solving
nonsymmetric algebraic Riccati equation}}, Appl. Math. Comput., 216 (2010),
pp.~1790--1804.
\bibitem{Jay2001}
{\sc L.~O. Jay}, {\em {A note on Q-order of convergence}}, BIT Numer. Math., 41
(2001), pp.~422--429.
\bibitem{Juang1995}
{\sc J.~Juang}, {\em {Existence of algebraic matrix Riccati equations arising
in transport theory}}, Linear Algebra Appl., 230 (1995), pp.~89--100.
\bibitem{JuangL1998}
{\sc J.~Juang and W.-W. Lin}, {\em {Nonsymmetric algebraic Riccati equations
and Hamiltonian-like matrices}}, SIAM J. Matrix Anal. Appl., 20 (1998),
pp.~228--243.
\bibitem{Kelley2018}
{\sc C.~T. Kelley}, {\em Numerical methods for nonlinear equations}, Acta
Numer., 27 (2018), pp.~207--287.
\bibitem{LiS2008}
{\sc C.~Li and W.~Shen}, {\em Local convergence of inexact methods under the
{H\"{o}lder} condition}, J. Comput. Appl. Math., 222 (2008), pp.~544--560.
\bibitem{LiangL2025}
{\sc J.~Liang and Y.~Ling}, {\em {On the convergence of two-step modified
Newton method for nonsymmetric algebraic Riccati equations from transport
theory}}, Numer. Algorithms, (2025), pp.~1--37.
\newblock https://doi.org/10.1007/s11075-025-02154-1 (to appear).
\bibitem{Lin2008}
{\sc Y.~Lin}, {\em {A class of iterative methods for solving nonsymmetric
algebraic Riccati equations arising in transport theory}}, Comput. Math.
Appl., 56 (2008), pp.~3046--3051.
\bibitem{LinB2008}
{\sc Y.~Lin and L.~Bao}, {\em {Convergence analysis of the Newton--Shamanskii
method for a nonsymmetric algebraic Riccati equation}}, Numer. Linear Algebra
Appl., 15 (2008), pp.~535--546.
\bibitem{LinBW2008}
{\sc Y.~Lin, L.~Bao, and Y.~Wei}, {\em {A modified Newton method for solving
non-symmetric algebraic Riccati equations arising in transport theory}}, IMA
J. Numer. Anal., 28 (2008), pp.~215--224.
\bibitem{LinBW2011}
{\sc Y.~Lin, L.~Bao, and Q.~Wu}, {\em {On the convergence rate of an iterative
method for solving nonsymmetric algebraic Riccati equations}}, Comput. Math.
Appl., 62 (2011), pp.~4178--4184.
\bibitem{LingLL2022}
{\sc Y.~Ling, J.~Liang, and W.~Lin}, {\em {On semilocal convergence analysis
for two-step Newton method under generalized Lipschitz conditions in Banach
spaces}}, Numer. Algorithms, 90 (2022), pp.~577--606.
\bibitem{LingX2017}
{\sc Y.~Ling and X.~Xu}, {\em {On one-parameter family of Newton-like
iterations for solving nonsymmetric algebraic Riccati equation from transport
theory}}, J. Nonlinear Convex Anal., 18 (2017), pp.~1833--1848.
\bibitem{LipnikovSV2013}
{\sc K.~Lipnikov, D.~Svyatskiy, and Y.~Vassilevski}, {\em Anderson acceleration
for nonlinear finite volume scheme for advection-diffusion problems}, SIAM J.
Sci. Comput., 35 (2013), pp.~A1120--A1136.
\bibitem{Lu2005b}
{\sc L.-Z. Lu}, {\em {Newton iterations for a non-symmetric algebraic Riccati
equation}}, Numer. Linear Algebra Appl., 12 (2005), pp.~191--200.
\bibitem{Lu2005a}
{\sc L.-Z. Lu}, {\em {Solution form and simple iteration of a nonsymmetric
algebraic Riccati equation arising in transport theory}}, SIAM J. Matrix
Anal. Appl., 26 (2005), pp.~679--685.
\bibitem{MaiJ2020}
{\sc V.~Mai and M.~Johansson}, {\em Anderson acceleration of proximal gradient
methods}, in International Conference on Machine Learning, PMLR, 2020,
pp.~6620--6629.
\bibitem{MarumoT2025}
{\sc N.~Marumo and A.~Takeda}, {\em {Universal heavy-ball method for nonconvex
optimization under H{\"o}lder continuous Hessians}}, Math. Program., 212
(2025), pp.~147--175.
\bibitem{MehrmannX2008}
{\sc V.~Mehrmann and H.~Xu}, {\em {Explicit solutions for a Riccati equation
from transport theory}}, SIAM J. Matrix Anal. Appl., 30 (2008),
pp.~1339--1357.
\bibitem{Nesterov2015}
{\sc Y.~Nesterov}, {\em Universal gradient methods for convex optimization
problems}, Math. Program., 152 (2015), pp.~381--404.
\bibitem{OrtegaR1970}
{\sc J.~M. Ortega and W.~C. Rheinboldt}, {\em {Iterative Solution of Nonlinear
Equations in Several Variables}}, Academic Press, New York, 1970.
\bibitem{OuyangLM2024}
{\sc W.~Ouyang, Y.~Liu, and A.~Milzarek}, {\em {Descent properties of an
Anderson accelerated gradient method with restarting}}, SIAM J. Optim., 34
(2024), pp.~336--365.
\bibitem{PasiniYRS2021}
{\sc M.~L. Pasini, J.~Yin, V.~Reshniak, and M.~Stoyanov}, {\em {Stable Anderson
acceleration for deep learning}}, arXiv preprint arXiv:2110.14813, (2021).
\bibitem{PasiniYRS2022}
{\sc M.~L. Pasini, J.~Yin, V.~Reshniak, and M.~K. Stoyanov}, {\em Anderson
acceleration for distributed training of deep learning models}, in
SoutheastCon 2022, IEEE, 2022, pp.~289--295.
\bibitem{PollockR2021}
{\sc S.~Pollock and L.~G. Rebholz}, {\em Anderson acceleration for contractive
and noncontractive operators}, IMA J. Numer. Anal., 41 (2021),
pp.~2841--2872.
\bibitem{PollockR2023}
{\sc S.~Pollock and L.~G. Rebholz}, {\em {Filtering for Anderson
acceleration}}, SIAM J. Sci. Comput., 45 (2023), pp.~A1571--A1590.
\bibitem{PollockRTX2025}
{\sc S.~Pollock, L.~G. Rebholz, X.~Tu, and M.~Xiao}, {\em {Analysis of the
Picard-Newton iteration for the Navier-Stokes equations: global stability and
quadratic convergence}}, J. Sci. Comput., 104 (2025).
\newblock Article number 25, 23 pp.
\bibitem{PollockRX2019}
{\sc S.~Pollock, L.~G. Rebholz, and M.~Xiao}, {\em {Anderson-accelerated
convergence of Picard iterations for incompressible Navier--Stokes
equations}}, SIAM J. Numer. Anal., 57 (2019), pp.~615--637.
\bibitem{Potra1989}
{\sc F.~A. Potra}, {\em {On Q-order and R-order of convergence}}, J. Optim.
Theory Appl., 63 (1989), pp.~415--431.
\bibitem{Pulay1980}
{\sc P.~Pulay}, {\em {Convergence acceleration of iterative sequences. The case
of SCF iteration}}, Chem. phys. lett., 73 (1980), pp.~393--398.
\bibitem{RebholzVX2021}
{\sc L.~G. Rebholz, D.~Vargun, and M.~Xiao}, {\em {Enabling convergence of the
iterated penalty Picard iteration with $O(1)$ penalty parameter for
incompressible Navier--Stokes via Anderson acceleration}}, Comput. Methods
Appl. Mech. Eng., 387 (2021), p.~114178.
\bibitem{RebholzX2023}
{\sc L.~G. Rebholz and M.~Xiao}, {\em {The effect of Anderson acceleration on
superlinear and sublinear convergence}}, J. Sci. Comput., 96 (2023).
\newblock Article number 34, 23 pp.
\bibitem{Rokne1972}
{\sc J.~Rokne}, {\em {Newton's} method under mild differentiability conditions
with error analysis}, Numer. Math., 18 (1972), pp.~401--412.
\bibitem{Saad2025}
{\sc Y.~Saad}, {\em Acceleration methods for fixed point iterations}, Acta
Numer., 34 (2025), pp.~805--890.
\bibitem{ScieurAB2020}
{\sc D.~Scieur, A.~d'Aspremont, and F.~Bach}, {\em Regularized nonlinear
acceleration}, Math. Program., 179 (2020), pp.~47--83.
\bibitem{ShenL2008}
{\sc W.~Shen and C.~Li}, {\em Convergence criterion of inexact methods for
operators with {H\"{o}lder} continuous derivatives}, Taiwan. J. Math., 12
(2008), pp.~1865--1882.
\bibitem{TothK2015}
{\sc A.~Toth and C.~T. Kelley}, {\em {Convergence analysis for Anderson
acceleration}}, SIAM J. Numer. Anal., 53 (2015), pp.~805--819.
\bibitem{WalkerN2011}
{\sc H.~F. Walker and P.~Ni}, {\em Anderson acceleration for fixed-point
iterations}, SIAM J. Numer. Anal., 49 (2011), pp.~1715--1735.
\bibitem{WangHS2021}
{\sc D.~Wang, Y.~He, and H.~De~Sterck}, {\em {On the asymptotic linear
convergence speed of Anderson acceleration applied to ADMM}}, J. Sci.
Comput., 88 (2021), p.~38.
\bibitem{ZhangDB2020}
{\sc J.~Zhang, B.~O'Donoghue, and S.~Boyd}, {\em {Globally convergent type-I
Anderson acceleration for nonsmooth fixed-point iterations}}, SIAM J. Optim.,
30 (2020), pp.~3170--3197.
\end{thebibliography}
%------------------------------------------------------------------------------
\end{document}