Benign Landscape of Quadratic Programs with Orthogonality Constraints and Its Application to Heteroscedastic Probabilistic PCA
Extracted source text · arXiv 2606.27189
\documentclass[11pt]{article}
%% ===========================================================================
% Packages and page formatting, adapted from arxiv_new.tex
\usepackage[a4paper]{geometry}
\geometry{margin=1in}
\renewcommand\baselinestretch{1.2}
\usepackage{adjustbox}
\usepackage{amssymb}
\usepackage{amsmath,amsfonts,amsthm,amsopn}
\usepackage{latexsym}
\usepackage{mathrsfs}
\usepackage{xcolor}
% \usepackage{refcheck}
% \usepackage{comment}
\usepackage{algorithm}
\usepackage{algorithmic}
\usepackage{subcaption}
\usepackage{dsfont}
\usepackage{multirow}
\usepackage[overload]{empheq}
\usepackage{enumitem}
\usepackage{setspace}
\usepackage{url}
\usepackage{booktabs}
\usepackage{bm}
\usepackage{graphicx}
\usepackage{epstopdf}
\usepackage{ifpdf}
\usepackage[numbers]{natbib}
\usepackage{makecell}
\usepackage{placeins}
\usepackage{soul}
\usepackage{tikz}
\usetikzlibrary{decorations.pathreplacing,calligraphy,calc}
\usetikzlibrary{positioning}
\usetikzlibrary{shapes.geometric,arrows.meta}
%\usepackage[hidelinks]{hyperref}
\usepackage[colorlinks=true,
linkcolor=blue,
citecolor=blue,
urlcolor=blue]{hyperref}
\usepackage{cleveref}
\hfuzz=21pt
\AtBeginDocument{\graphicspath{{./}{../}}}
\ifpdf
\DeclareGraphicsExtensions{.pdf,.png,.jpg,.eps}
\else
\DeclareGraphicsExtensions{.eps}
\fi
\hypersetup{
pdftitle={On the Benign optimization landscape of Quadratic Programs with Orthogonality Constraints and Its Application to Heteroscedastic Probabilistic PCA},
pdfauthor={Peng Wang, Po Chen, Rujun Jiang, and Laura Balzano}
}
%%% Convenience macros from arxiv_new.tex
\newcommand{\beqa}{\begin{eqnarray}}
\newcommand{\eeqa}{\end{eqnarray}}
\newcommand{\beqas}{\begin{eqnarray*}}
\newcommand{\eeqas}{\end{eqnarray*}}
\newcommand{\ba}{\begin{array}}
\newcommand{\ea}{\end{array}}
\newcommand{\bi}{\begin{itemize}}
\newcommand{\ei}{\end{itemize}}
\newcommand{\nn}{\nonumber}
\newcommand{\RN}[1]{%
\textup{\uppercase\expandafter{\romannumeral#1}}%
}
% caligraphic letters
\newcommand{\mcK}{{\mathcal K}}
\newcommand{\mcX}{{\mathcal X}}
\newcommand{\mcL}{{\mathcal L}}
\newcommand{\mcA}{{\mathcal A}}
\newcommand{\mcE}{{\mathcal E}}
\newcommand{\mcN}{{\mathcal N}}
\newcommand{\mcJ}{{\mathcal J}}
\newcommand{\mcT}{{\mathcal T}}
\newcommand{\mcI}{{\mathcal I}}
\newcommand{\mcS}{{\mathcal S}}
% math operators
\newcommand{\Rank}{\mathop{Rank}}
\DeclareMathOperator{\Tr}{Tr}
\DeclareMathOperator{\diag}{diag}
\newcommand{\prox}{\mathrm{prox}}
\newcommand{\ve}{\mathrm{vec}}
\newcommand{\dom}{\mathrm{dom}}
\newcommand{\dist}{\mathrm{dist}}
\DeclareMathOperator{\argmin}{\arg\min}
\DeclareMathOperator{\argmax}{\arg\max}
\providecommand{\st}{\mbox{s.t.}}
% operators with arguments
\newcommand{\ceil}[1]{\left\lceil #1 \right\rceil}
\newcommand{\floor}[1]{\left\lfloor #1 \right\rfloor}
\newcommand{\Inner}[2]{\left\langle #1, #2 \right\rangle}
% italic shorts
\newcommand{\cf}{{\it cf.}}
\newcommand{\eg}{{\it e.g.}}
\newcommand{\ie}{{\it i.e.}}
\newcommand{\etc}{{\it etc.}}
\newcommand{\res}{{\it res.}}
% theorem environments for the article template
\theoremstyle{plain}
\newtheorem{theorem}{Theorem}
\newtheorem{lemma}{Lemma}
\newtheorem{corollary}{Corollary}
\newtheorem{proposition}{Proposition}
\newtheorem{fact}{Fact}
\newtheorem{claim}{Claim}
\theoremstyle{definition}
\newtheorem{definition}{Definition}
\newtheorem{assumption}{Assumption}
\newtheorem{example}{Example}
\theoremstyle{remark}
\newtheorem{remark}{Remark}
\newtheorem{observation}{Observation}
\crefname{theorem}{Theorem}{Theorems}
\crefname{lemma}{Lemma}{Lemmas}
\crefname{corollary}{Corollary}{Corollaries}
\crefname{proposition}{Proposition}{Propositions}
\crefname{definition}{Definition}{Definitions}
\crefname{assumption}{Assumption}{Assumptions}
\crefname{fact}{Fact}{Facts}
\crefname{figure}{Figure}{Figures}
\crefname{table}{Table}{Tables}
\newcommand{\creflastconjunction}{, and~}
\newenvironment{keywords}{\par\medskip\noindent{\bf Keywords:} }{\par}
\newenvironment{MSCcodes}{\par\medskip\noindent{\bf Mathematics Subject Classification:} }{\par}
\def \openbox{$\sqcup\llap{$\sqcap$}$}
\newcommand{\email}[1]{\protect\href{mailto:#1}{#1}}
%%%%%%%%%%%%%%%%% define some convenient notations %%%%%%%%%%%%%%%%%%%%%%%
\newcommand{\edit}[1]{{\color{blue}#1}}
\newcommand{\gedit}[1]{{\color{green!50!black}#1}}
\newcommand{\redit}[1]{{\color{red}#1}}
\newcommand{\reit}[1]{\redit{#1}}
\newcommand{\todo}[1]{\hl{\textbf{TODO:} #1}}
%% bm capital letters
\def\bmw{\bm{W}}
\def\nablag{\nabla G(\bm W)}
\def\bA{\bm{A}}
\def\bB{\bm{B}}
\def\bC{\bm{C}}
\def\bCp{\bC^\prime}
\def\bbC{\bar{\bC}}
\def\bD{\bm{D}}
\def\bE{\bm{E}}
\def\bG{\bm{G}}
\def\bGp{\bG^\prime}
\def\bH{\bm{H}}
\def\bbH{\bar{\bm{H}}}
\def\bHp{\bm{H}^\prime}
\def\bHs{\bm{H}^*}
\def\bHsT{\bH^{*^T}}
\def\bI{\bm{I}}
\def\bLambda{\bm{\Lambda}}
\def\bM{\bm{M}}
\def\bO{\bm{O}}
\def\bP{\bm{P}}
\def\bPp{\bm{P}^\prime}
\def\bQ{\bm{Q}}
\def\bR{\bm{R}}
\def\bS{\bm{S}}
\def\bbU{\bar{\bU}}
\def\bU{\bm{U}}
\def\bV{\bm{V}}
\def\bVp{\bm{V}^\prime}
\def\bW{\bm{W}}
\def\bX{\bm{X}}
\def\bY{\bm{Y}}
\def\bZ{\bm{Z}}
\def\bZp{\bZ^\prime}
% mathcal
\def\mO{\mathcal{O}}
\def\mP{\mathcal{P}}
\def\mQ{\mathcal{Q}}
\def\mR{\mathcal{R}}
\def\mS{\mathcal{S}}
\def\mT{\mathcal{T}}
\def\mX{\mathcal{X}}
\def\mZ{\mathcal{Z}}
\def\mW{\mathcal{W}}
% others
\def\bbA{\bar{\bm{A}}}
\def\Diag{\mathbf{Diag}}
\def\C{\mathbb{C}}
\def\E{\mathbb{E}}
\def\M{\mathbb{M}}
\def\O{\mathbb{O}}
\def\P{\mathbb{P}}
\def\R{\mathbb{R}}
\def\S{\mathbb{S}}
\def\bo{\bm{1}}
\def\tK{\tilde{K}}
\newcommand{\blk}{\mathrm{BlkD}}
\begin{document}
\title{Benign Landscape of Quadratic Programs with Orthogonality Constraints and Its Application to Heteroscedastic Probabilistic PCA\thanks{The first and second authors equally contribute to this work.}
}
\author{
Peng Wang\thanks{Department of Computer and Information Science, University of Macau, Macao SAR, China (\email{pengw@um.edu.mo}).}
\and Po Chen\thanks{School of Data Science, Fudan University, Shanghai, China (\email{chenp24@m.fudan.edu.cn}).}
\and Rujun Jiang\thanks{School of Data Science, Fudan University, Shanghai, China (\email{rjjiang@fudan.edu.cn}).}
\and Laura Balzano\thanks{Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, USA (\email{girasole@umich.edu}).}
}
\date{\today}
\maketitle
\begin{abstract}
In this work, we study the optimization landscape of homogeneous quadratic programs with orthogonality constraints (QPOC) and apply the resulting theory to heteroscedastic probabilistic PCA (HePPCA). For QPOC, we establish a complete characterization of the benign optimization landscape by showing that every critical point is either a global maximizer or a strict saddle point. Our analysis builds on a closed-form characterization of the critical point set, from which we derive a necessary and sufficient condition for global optimality and show that every non-optimal critical point has a direction of positive curvature. As an application, we show that the population version of HePPCA is a special instance of QPOC and therefore has a benign optimization landscape; moreover, it satisfies local geodesic strong concavity near every global maximizer. We furthermore prove that, when the sample size is sufficiently large, the sample version of HePPCA inherits these favorable properties with high probability. Together with existing theory on avoiding strict saddle points, our results provide a theoretical justification for the observed local linear convergence of retraction-based optimization methods to global solutions for both QPOC and HePPCA. Finally, we present numerical experiments to corroborate our theoretical results.
\end{abstract}
% REQUIRED
% \begin{keywords}
% Quadratic programs with orthogonality constraints, heteroscedastic probabilistic PCA, critical points, optimization landscape, strict-saddle property
% \end{keywords}
% REQUIRED
% \vspace{-0.1in}
% \begin{MSCcodes}
% 90C26, 90C30, 62H25
% \end{MSCcodes}
\vspace{-0.1in}
\section{Introduction}\label{sec:intro}
In this work, we study the following homogeneous quadratic program with orthogonality constraints (QPOC):
\begin{align}\label{eq:QPOC}
\max_{\bm Q \in \R^{d \times K}} F(\bm Q) := \mathrm{Tr}\left(\bm Q^T\bm A\bm Q \bm B \right)\qquad \mathrm{s.t.}\ \bm Q \in \mathrm{St}(d,K).
\end{align}
Here, $\mathrm{St}(d,K) = \left\{\bm Q \in \R^{d\times K}: \bm Q^T\bm Q = \bm I_K \right\}$ denotes the Stiefel manifold with $d \ge K$, $\bm I_K$ is the $K \times K$ identity matrix, and $\bm A \in \R^{d\times d}$ and $\bm B \in \R^{K\times K}$ are symmetric matrices. This is a classical and important class of matrix optimization problems, which has been widely studied in applied mathematics, machine learning, and signal processing. It includes several well-known formulations as special cases, such as eigenvalue problems \cite{golub2013matrix}, generalized eigenvalue problems \cite{horn2012matrix}, and orthogonal relaxations of quadratic assignment problems \cite{lawler1963quadratic}.
Over the past few decades, Problem~\eqref{eq:QPOC} has been extensively studied, with particular focus on algorithm design and the analysis of its optimization properties. A variety of algorithms have been proposed for solving this problem by exploiting its structure, including Riemannian gradient descent \cite{boumal2023optimization,smith2014optimization}, projected gradient descent \cite{wen2013feasible}, and exact penalty function methods \cite{xiao2021solving,xiao2022class}, to name a few. These methods leverage the geometry of the Stiefel manifold or incorporate penalty terms to enforce the constraints, providing both theoretical convergence guarantees and strong empirical performance. An influential work \cite{liu2019quadratic} demonstrated that the \L{}ojasiewicz exponent at any critical point of Problem~\eqref{eq:QPOC} is $1/2$. This result implies that a broad class of first-order retraction-based methods for solving this problem enjoys local linear convergence to a critical point. Interestingly, empirical observations demonstrate that these methods often converge to a global maximizer of Problem~\eqref{eq:QPOC}, rather than merely a generic critical point (see \Cref{fig:1}). This naturally raises the question of whether the optimization landscape of Problem~\eqref{eq:QPOC} is benign, in the sense that each critical point is either a global maximizer or a strict saddle point (see \Cref{def:crit}).
Our first contribution is to provide an affirmative answer to the above question by showing that Problem~\eqref{eq:QPOC} indeed has a benign optimization landscape. To this end, we first review the closed-form characterization of the critical point set in \cite{liu2019quadratic}. Building on this characterization, we identify a sufficient condition for global optimality (see \Cref{prop:global}). We then show that any critical point violating this condition is a strict saddle point by constructing a direction of positive curvature (see \Cref{prop:saddle}). Consequently, this condition is also necessary. These results yield a complete characterization of the optimization landscape (see \Cref{thm:land}). Combined with strict-saddle avoidance \cite{lee2019first} and the \L{}ojasiewicz exponent in \cite{liu2019quadratic}, this explains why retraction-based methods usually converge to a global maximizer at a locally linear rate.
Our second contribution is the application of the above results to the HePPCA problem \cite{gilman2025semidefinite,hong2021heppcat,wang2023estimation}. The HePPCA model extends the classical PCA model by allowing samples from different groups to have different noise variances, enabling more accurate modeling of data with heterogeneous variability across groups (see \eqref{eq:HPCA}). We study the maximum likelihood estimation problem of the HePPCA model, as formulated in Problem~\eqref{eq:QPOC1}. Under the HePPCA model assumptions, we show that the population version of Problem~\eqref{eq:QPOC1} is a special instance of Problem~\eqref{eq:QPOC} and therefore admits a benign optimization landscape. Moreover, we prove that it is locally geodesically strongly concave on the Stiefel manifold near every global maximizer. Building on this, we conduct a perturbation analysis and show that, when the sample size is sufficiently large, Problem~\eqref{eq:QPOC1} admits the same properties as its population counterpart with high probability (see \Cref{thm:theorem5}). This result is significant, as it shows that retraction-based optimization methods converge to global solutions of the HePPCA problem at a locally linear rate.
\subsection{Related Work}\label{sec:related}
% We now review prior work on optimization landscape analysis for manifold optimization and on theoretical developments related to HePPCA.
% \paragraph{optimization landscape analysis in non-convex optimization}
% In non-convex optimization, a key approach to understanding the behavior of optimization algorithms is through the analysis of the optimization landscape.
In non-convex optimization, a common approach to understanding the behavior of optimization algorithms is through the analysis of the optimization landscape
\cite{achour2024loss,chen2025complete,chi2019nonconvex,li2019symmetry,sun2018geometric,pmlr-v235-wang24as}. By characterizing the landscape, one can identify different types of critical points, including local minimizers, global minimizers, strict saddle points, and non-strict saddle points (see \Cref{def:crit}).
In particular, a non-convex problem is said to admit a {\em benign optimization landscape} if every critical point is either a strict saddle point or a global minimizer.\footnote{This definition is stated for minimization problems. In our setting, we consider a maximization problem, and the definition is adapted accordingly.} Typical examples of such problems include neural collapse problems \cite{yaras2022neural,zhu2021geometric}, low-rank matrix recovery \cite{Bhojanapalli2016,ge2017no}, and group synchronization \cite{mcrae2025nonconvex,mcrae2024benign}. Moreover, the foundational work \cite{lee2019first} showed that first-order methods, including gradient descent, manifold gradient descent, block coordinate descent, and their variants, avoid strict saddle points for almost all initializations. Consequently, for a problem with a benign landscape, randomly initialized first-order methods almost always converge to globally optimal points.
We next review recent advances in the analysis of optimization landscapes for smooth problems over manifolds. McRae and Boumal \cite{mcrae2024benign} studied the Burer-Monteiro relaxation of the orthogonal group synchronization problem, which maximizes a quadratic problem over the Stiefel manifold. They proved that it has a benign optimization landscape when the relaxation rank is larger than a threshold. Ling \cite{ling2023solving,ling2025local} studied the same relaxation problem and showed that it possesses a unique local minimizer, which is also global under mild conditions. Recently, McRae et al. \cite{mcrae2025nonconvex} derived a sufficient condition under which the Burer-Monteiro relaxation of the $Z_2$ synchronization problem, which lies on the oblique manifold, has a benign optimization landscape under mild conditions on the measurement graph and noise. Yaras et al. \cite{yaras2022neural} studied the cross-entropy loss for optimizing two-layer linear networks over the oblique manifold and showed that this problem has a benign optimization landscape. % More recently, Chen et al. \cite{chen2025complete} conducted a comprehensive study of the optimization landscape of the regularized deep matrix factorization problem by characterizing precise conditions under which a critical point is a local minimizer, a global minimizer, a strict saddle point, or a non-strict saddle point.
%\paragraph{Theoretical studies on HePPCA}
Now we turn to literature on the heteroscedastic PCA problem. HePPCA extends classical PCA by incorporating heterogeneous probabilistic priors into the dimensionality reduction framework. Hong et al. \cite{hong2021heppcat} developed a HePPCA model that accounts for heteroscedastic noise across samples and derived the corresponding maximum likelihood estimation formulation. Based on this, they proposed efficient alternating maximization algorithms to jointly estimate both underlying factors and noise variances. Gilman et al. \cite{gilman2025semidefinite} considered the setting where the noise variances for each data group are known and formulated HePPCA as a sum of heterogeneous quadratic functions on the Stiefel manifold. They studied a semidefinite relaxation of this problem and established a certificate for global optimality. Recently, Wang et al. \cite{wang2023estimation} proposed a generalized power method for solving the same formulation and provided estimation guarantees based on an error-bound property.
Beyond the aforementioned line of research, many other works have also explored different aspects and formulations of HePPCA. Zhang et al. \cite{zhang2022heteroskedastic} proposed a HeteroPCA algorithm for principal component analysis under heteroscedastic noise across features (as opposed to heteroscedasticity across samples) and proved its optimality under a generalized spiked covariance model via robust subspace perturbation analysis. Building on this work, Yan et al. \cite{yan2024inference} developed non-asymptotic distributional guarantees for HeteroPCA and demonstrated that it enables accurate inference for the principal subspace, even in the presence of missing data. For heteroscedasticity across samples, further extensions and algorithmic developments include the ALPCAH method, which addresses sample-wise heteroscedastic PCA with robust subspace estimation \cite{cavazos2023alpcah}; streaming HePPCA, which is designed for online or sequential data with heteroscedastic noise \cite{gilman2023streaming}; and the HeMPPCAT mixture model, which generalizes HePPCA to mixtures of probabilistic PCA components with heterogeneous noise across components \cite{xu2023hemppcat}.
\vspace{-0.05in}
\subsection{Notation and Organization} We use $\S^n$ to denote the set of all $n\times n$ symmetric matrices. For a positive integer $n$, let $[n] := \{1,\dots,n\}$. For a finite set $\mathcal S$, we use $|\mathcal S|$ to denote its cardinality. Given scalars $a_1,\dots,a_n$, we define $a_{\max} := \max\{a_i: i \in [n]\}$ and $a_{\min} := \min\{a_i: i \in [n]\}$. Given a vector $\bm{a}$, let $\|\bm{a}\|$ denote its Euclidean norm, $a_i$ its $i$-th entry, and $\mathrm{diag}(\bm{a})$ the diagonal matrix with $\bm{a}$ on its diagonal. For a matrix $\bm{A}$, we define $\|\bm{A}\|$ as its spectral norm, $\|\bm{A}\|_F$ as its Frobenius norm, $a_{ij}$ or $\bm A(i,j)$ as its $(i,j)$-th entry, and $\sigma_i(\bm{A})$ as its $i$-th largest singular value. For an integer $n$, let $\mathcal{P}^{n}$ denote the set of all $n \times n$ permutation matrices.
We associate each $\bm\Pi \in \mathcal{P}^{n}$ with a permutation $\pi : [n] \rightarrow [n]$ through the convention that
$\bm\Pi(i,j)=1$ if $j=\pi(i)$ and $\bm\Pi(i,j)=0$ otherwise for all $i,j\in[n]$. Under this convention, \(\bm \Pi \bm e_i = \bm e_{\pi^{-1}(i)}\), \(\bm \Pi^T \bm e_i = \bm e_{\pi(i)}\), and, for any diagonal matrix \(\bm D\), \((\bm \Pi^T \bm D\bm \Pi)(i,i)=\bm D(\pi^{-1}(i),\pi^{-1}(i))\).
We use $\mathcal{O}^{n}$ to denote the set of $n \times n$ orthogonal matrices. The symbol $\bm a \circ \bm b$ denotes the Hadamard product of two vectors $\bm a, \bm b \in \R^n$, i.e., $\bm a \circ \bm b=(a_1b_1,\dots,a_nb_n)$. We denote by $\mathrm{BlkD}(\bm{X}_1, \dots, \bm{X}_n)$ the block diagonal matrix with diagonal blocks $\bm{X}_1, \dots, \bm{X}_n$. Blocks with zero rows or zero columns are allowed and are simply omitted from displayed block matrices. In particular, $\bm X_i\in\mathbb R^{m_i\times 0}$ contributes only $m_i$ rows and $\bm X_i\in\mathbb R^{0\times n_i}$ contributes only $n_i$ columns. % We also use the convention $\bm A\bm B=\bm 0_{m\times n}$ for $\bm A\in\mathbb R^{m\times 0}$ and $\bm B\in\mathbb R^{0\times n}$.
Given $\bm Q \in \mathrm{St}(d,K)$, the tangent space to $\mathrm{St}(d,K)$ at $\bm Q$ is $$\mathrm{T}(\bm Q) = \left\{\bm X \in \R^{d\times K}: \mathrm{sym}(\bm Q^T\bm X) = \bm 0 \right\},$$
where $\mathrm{sym}(\bm C):= ({\bm C+\bm C^T})/{2}$. According to \cite[page 162]{boumal2023optimization}, the projection onto the tangent space of $\mathrm{St}(d,K)$ is $\mathrm{Proj}_{\mathrm{T}(\bm Q)}(\bm C)=\bm C-\bm Q\mathrm{sym}(\bm Q^T\bm C)$. Given a smooth function $F:\R^{d \times K} \to \R$, its Riemannian gradient on the Stiefel manifold at $\bm Q$ can be calculated as
\begin{align*}
\mathrm{grad}\ F(\bm Q) &= \mathrm{Proj}_{\mathrm{T}(\bm Q)}(\nabla F(\bm Q)).
\end{align*}
The Riemannian Hessian at $\bm Q$ along the direction $\bm D\in \mathrm{T}(\bm Q)$ can be calculated as
\begin{align}\label{eq:Re He}
\mathrm{Hess}\ F(\bm Q)[\bm D] &=\mathrm{Proj}_{\mathrm{T}(\bm Q)}\left(\nabla^2 F(\bm Q)[\bm D] - \bm D \mathrm{sym}(\bm Q^T \nabla F(\bm Q))\right)
\end{align}
We also write $\mathrm{Hess}\ F(\bm Q)[\bm D,\bm D]
:= \left\langle \bm D, \mathrm{Hess}\ F(\bm Q)[\bm D]\right\rangle$.
Now, we formally introduce the definitions of global maximizers, local maximizers, critical points, and strict saddle points for the problem of maximizing a smooth function $H(\bm Q)$ over the Stiefel manifold:
\begin{align}\label{eq:mani}
\max_{\bm Q \in \R^{d\times K}} H(\bm Q)\qquad \mathrm{s.t.}\ \bm Q\in \mathrm{St}(d,K).
\end{align}
\begin{definition}\label{def:crit}
For Problem \eqref{eq:mani}, we say that \\
(i) $\bm Q^* \in \mathrm{St}(d,K)$ is a global maximizer if $H(\bm Q^*) \ge H(\bm Q)$ for all $\bm Q \in \mathrm{St}(d,K)$. \\
(ii) $\bm Q^*$ is a local maximizer if there exists a neighborhood $\cal U$ of $\bm Q^*$ such that $H(\bm Q^*) \ge H(\bm Q)$ for all $\bm Q \in \mathcal{U}\cap \mathrm{St}(d,K)$. \\
(iii) $\bm Q^* \in \mathrm{St}(d,K)$ is a critical point if $\mathrm{grad}\ H(\bm Q^*) = \bm 0$. \\
(iv) $\bm Q^* \in \mathrm{St}(d,K)$ is a strict saddle point\footnote{For maximization problems, a direction of positive curvature is an escaping direction \cite{lee2019first}. Note that this definition of strict saddle points therefore includes minima.} if it is a critical point and \\
$\mathrm{Hess}\ H(\bm Q^*)[\bm D,\bm D] > 0$ for a nonzero direction $\bm D\in \mathrm{T}(\bm Q^*)$.
\end{definition}
The rest of this paper is organized as follows. In \Cref{sec:pre}, we review existing closed-form characterizations of the critical point set of Problem~\eqref{eq:QPOC}. In \Cref{sec:QPOC}, we present the optimization landscape result for Problem~\eqref{eq:QPOC}. In \Cref{sec:PPCA}, we extend these results to study the optimization landscape of the HePPCA problem. Finally, \Cref{sec:exp} presents numerical experiments, and \Cref{sec:con} concludes the paper.
\section{Preliminaries}\label{sec:pre}
In this section, we present the problem setup and review a key result from \cite{liu2019quadratic} that gives a closed-form characterization of the critical point set of Problem~\eqref{eq:QPOC} when $\bm B$ is nonsingular. We then extend it to the singular case.
To proceed, let $\bm A = \bm U_A \bm \Lambda_A \bm U_A^T $ and $\bm B = \bm U_B \bm \Lambda_B \bm U_B^T $ be spectral decompositions of $\bm A \in \S^d$ and $\bm B \in \S^{K}$, respectively, where $\bm U_A \in \mathcal{O}^d$ and $\bm U_B \in \mathcal{O}^K$. It is straightforward to verify that $ \mathrm{Tr}\left( \bm Q^T \bm A \bm Q \bm B \right) = \mathrm{Tr} \left( \bar{\bm Q}^T \bm \Lambda_A \bar{\bm Q} \bm \Lambda_B \right) $, where $ \bar{\bm Q} = \bm U_A^T \bm Q \bm U_B \in \mathrm{St}(d,K)$. The map $\bm Q\mapsto \bm U_A^T\bm Q\bm U_B$ is a bijective isometry of the Stiefel manifold under the embedded Euclidean metric. Therefore, we assume without loss of generality that
\begin{equation}\label{eq:simpleAB}
\bm A = \mathrm{diag}(a_1, \ldots, a_d),\quad \bm B = \mathrm{diag}(b_1, \ldots, b_K),
\end{equation}
where $ a_1 \geq a_2 \geq \cdots \geq a_d$ and $ b_1 \geq b_2 \geq \cdots \geq b_K$.
For ease of exposition, we denote the critical point set of Problem~\eqref{eq:QPOC} by
$\mathcal{Q} := \left\{ \bm Q \in \mathrm{St}(d,K): \mathrm{grad} F(\bm Q) = \bm 0\right\}.$
By \cite[Proposition 1]{liu2019quadratic}, this set can be written as
\begin{align}\label{eq:crit set}
\mathcal{Q} = \left\{ \bm Q \in \mathrm{St}(d,K): \bm A\bm Q \bm B - \bm Q \bm B\bm Q^T\bm A\bm Q = \bm 0\right\}.
\end{align}
\paragraph{Nonsingular $\bm B$}
We first consider the case where $\bm B$ is nonsingular, i.e., $ b_i \neq 0 $ for each $i \in [K]$. In this case, Liu et al. \cite{liu2019quadratic} derived a closed-form expression for $\mathcal{Q}$. Specifically, let $n_A$ and $ n_B $ be the numbers of distinct eigenvalues of $\bm A$ and $\bm B$, respectively. We assume that $ n_A \geq 2 $, as the objective function is a constant when $n_A = 1$ and all feasible points are trivially optimal.
Then, there exist indices $ s_0, s_1, \ldots, s_{n_A} $ and $ t_0, t_1, \ldots, t_{n_B} $ that partition the eigenvalues into groups of equal value, such that $ 0 = s_0 < s_1 < \cdots < s_{n_A} = d $, $ 0 = t_0 < t_1 < \cdots < t_{n_B} = K $, and
\begin{align*}
&a_{s_0+1} = \cdots = a_{s_1} > a_{s_1+1} = \cdots = a_{s_2} > \cdots > a_{s_{n_A-1}+1} = \cdots = a_{s_{n_A}}, \\
&b_{t_0+1} = \cdots = b_{t_1} > b_{t_1+1} = \cdots = b_{t_2} > \cdots > b_{t_{n_B -1}+1}=\cdots=b_{t_{n_B}}.
\end{align*}
This, together with \eqref{eq:simpleAB}, allows us to express $ \bm A \in \S^d $ and $ \bm B \in \S^K$ as
\begin{align}
&\bm A = \operatorname{BlkD}\left(a_{s_1} \bm I_{s_1-s_0}, \ldots, a_{s_{n_A}} \bm I_{s_{n_A}-s_{n_A-1}}\right) \in \S^d,\label{eq:block a}\\
&\bm B= \operatorname{BlkD}\left(b_{t_1} \bm I_{t_1-t_0}, \ldots, b_{t_{n_B}} \bm I_{t_{n_B}-t_{n_B-1}}\right) \in \S^K, \label{eq:block b}
\end{align}
respectively. We work with this form as opposed to \eqref{eq:simpleAB} in order to handle rotational invariance that arises when eigenvalues have multiplicity greater than 1. Define
\begin{equation*}
\mathcal{H} := \left\{ (h_1, \ldots, h_{n_A}): \sum_{i=1}^{n_A} h_i = K, \; h_i \in \{0, 1, \ldots, s_i - s_{i-1}\},\ i \in [n_A] \right\}.
\end{equation*}
In addition, given any $ \bm h = (h_1, \ldots, h_{n_A}) \in \mathcal{H} $, define
\begin{align*}
\bm E_i(\bm h) &:= \left[\bm e_{s_{i-1}+1} \cdots \bm e_{s_{i-1}+h_i}\right] \in \mathbb{R}^{d \times h_i} \quad \text{for } i = 1, \ldots, n_A, \nonumber \\
\bm E(\bm h) &:= \left[\bm E_1(\bm h), \cdots, \bm E_{n_A}(\bm h) \right] \in \mathbb{R}^{d \times K},
\end{align*}
where $ \{\bm e_i\}_{i=1}^d $ is the standard basis of $ \mathbb{R}^d $.\footnote{By convention, $h_i = 0$ means that $\bm E_i(\bm h)$ is not included in $\bm E(\bm h)$ for each $i \in [n_A]$.} Intuitively, \(\bm h\) records how many basis vectors are selected from each
eigenspace of \(\bm A\). The matrix \(\bm E(\bm h)\) acts as a selector: for any
compatible matrix \(\bm W\), the product \(\bm W\bm E(\bm h)\) keeps the columns of
\(\bm W\) indexed by these selected basis vectors. % Notably, $\mathcal{H}$ specifies how many columns of a critical point $\bm Q$ lie in each eigenspace of $\bm A$ corresponding to the distinct eigenvalues, and $\bm E(\bm h)$ will be used to select the corresponding basis vectors to form its canonical representation.
According to \cite{liu2019quadratic}, the critical point set of Problem~\eqref{eq:QPOC} admits the following closed-form expression.
\begin{fact}(\cite[Proposition 3]{liu2019quadratic})\label{fact:1}
Suppose that $\bm A \in \S^d$ and $\bm B \in \S^K$ are given by \eqref{eq:block a} and \eqref{eq:block b}, respectively, and that $b_i \neq 0$ for each $i \in [K]$. Then the critical point set $\mathcal{Q}$ can be expressed as
\begin{align*}
\mathcal{Q} = \bigcup_{\bm h \in \mathcal{H}, \bm \Pi \in \mathcal{P}^K} \mathcal{Q}_{\bm h,\bm \Pi},
\end{align*}
where every $\bm Q \in \mathcal{Q}_{\bm h,\bm \Pi}$ can be written as
\begin{equation*}
\bm Q = \mathrm{BlkD}(\bm U_1, \ldots,\bm U_{n_A}) \bm E(\bm h) \bm\Pi \mathrm{BlkD}\left(\bm V_1^T, \ldots, \bm V_{n_B}^T\right)
\end{equation*}
for some $\bm U_i \in \mathcal{O}^{s_i - s_{i-1}}$ for each $i \in [n_A]$ and $\bm V_j \in \mathcal{O}^{t_j - t_{j-1}}$ for each $j \in [n_B]$.
\end{fact}
\paragraph{Singular \(\bm B\)}
We now extend the above result to the case where \(\bm B\) is singular but nonzero, and derive a closed-form expression for \(\mathcal Q\). With a slight abuse of notation, let $n_B$ denote the number of distinct nonzero eigenvalues of $\bm B$. After reordering the diagonal blocks, we write the zero block last and set $t_{n_B}=\operatorname{rank}(\bm B)$. Similar to the nonsingular case, there exist indices $t_0,t_1,\ldots,t_{n_B},t_{n_B+1}$ and $p \in \{0,1,\dots,n_B\}$ such that $0=t_0<t_1<\cdots <t_{n_B}<t_{n_B+1}=K$ and
\begin{equation*}
\begin{aligned}
& b_{t_0+1}=\cdots=b_{t_1}>\cdots>b_{t_{p-1}+1}=\cdots=b_{t_p}>0,\\
& 0 > b_{t_p+1}=\cdots=b_{t_{p+1}}>\cdots>b_{t_{n_B-1}+1}=\cdots=b_{t_{n_B}},\\
&\ b_{t_{n_B}+1}=\cdots=b_{t_{n_B+1}}=0.
\end{aligned}
\end{equation*}
In particular, $p=0$ (resp., $p=n_B$) indicates that there does not exist an index $i$ satisfying $b_i > 0$ (resp., $b_i < 0$). This partition differs from the decreasing-order partition only in that all zero entries are placed in the last block; either sign part may be empty.
%We note that this partition is slightly different from the decreasing-order partition, as all zero elements are moved to the last block. In particular, the positive or negative part is allowed to be empty.
Therefore, we have
\begin{align}\label{eq:block b1}
\bm B = \mathrm{BlkD}(\widehat{\bm B},\bm 0_{K-t_{n_B}}),\ \ \widehat{\bm B} =
\mathrm{BlkD}\left( b_{t_1} \bm I_{t_1-t_0},\dots,b_{t_{n_B}} \bm I_{t_{n_B}-t_{n_B-1}} \right).
\end{align}
Substituting this form into \eqref{eq:crit set}, together with the argument in \cite[Section 3.1.4]{liu2019quadratic}, yields that the critical point set $\mathcal{Q}$ of Problem~\eqref{eq:QPOC} is equivalent to
\begin{align}
\label{eq:crit 1}
\mathcal{Q}
&=
\left\{\bm Q=[\bm Q_1,\bm Q_2]\in\mathrm{St}(d,K):
\bm Q_1\in\mathcal{Q}_1\right\},\\
\mathcal{Q}_1
&:=
\left\{\bm Y\in\mathrm{St}(d,t_{n_B}):
\bm A\bm Y\widehat{\bm B}
-\bm Y\widehat{\bm B}\bm Y^T\bm A\bm Y=\bm0
\right\}.
\notag
\end{align}
Now, we derive an explicit expression for $\cal Q$ based on \Cref{fact:1}. Define
\begin{align*}
\widetilde{\mathcal{H}} := \left\{ (\bm h_1,\bm h_2):
\begin{array}{l}
\bm h_1=(h_{1,1},h_{1,2},\ldots,h_{1,n_A}),\
\bm h_2 = (h_{2,1},h_{2,2},\ldots,h_{2,n_A}), \\
h_{1,i}\in \{0,1,\ldots,s_i-s_{i-1}\},\ i \in [n_A],\
\sum_{i=1}^{n_A} h_{1,i}=t_{n_B}, \\
h_{2,i} = s_i-s_{i-1}-h_{1,i},\ i \in [n_A]
\end{array}
\right\}.
\end{align*}
In addition, given any $\bm h =(\bm h_1,\bm h_2)\in \widetilde{\mathcal{H}}$, define
\begin{align}
&\bar{\bm E}_i(\bm h_1)
:= \left[\bm e_{s_{i-1}+1} \ \cdots\ \bm e_{s_{i-1}+h_{1,i}}\right]
\in \mathbb{R}^{d \times h_{1,i}}\ \text{for each}\ i \in [n_A] \notag \\
&\widehat{\bm E}_i(\bm h_2)
:= \left[\bm e_{s_{i-1}+h_{1,i}+1} \ \cdots\ \bm e_{s_{i}}\right]
\in \mathbb{R}^{d \times h_{2,i}}\ \text{for each}\ i \in [n_A], \notag\\
& \bar{\bm E}(\bm h_1) := \left[\bar{\bm E}_1(\bm h_1)\ \cdots\ \bar{\bm E}_{n_A}(\bm h_1)\right] \in \R^{d \times t_{n_B}},\label{eq:Eh1}\\
& \widehat{\bm E}(\bm h_2) := \left[\widehat{\bm E}_1(\bm h_2) \ \cdots\ \widehat{\bm E}_{n_A}(\bm h_2)\right] \in \R^{d \times (d-t_{n_B})}, \notag\\
& \bm E(\bm h)
:= \left[\bar{\bm E}(\bm h_1)\ \widehat{\bm E}(\bm h_2)\right]
\in \mathbb{R}^{d \times d}.
\label{eq:basedef}
\end{align}
To illustrate this construction, we provide a schematic illustration of the matrix $\bm E(\bm h)$ in Figure \ref{fig:blockplot}.
\begin{figure}[t]
\centering
\makebox[\textwidth][c]{
\begin{tikzpicture}[
scale=0.85, transform shape,
every node/.style={scale=0.85},
brace/.style={decorate, decoration={calligraphic brace, amplitude=4pt}, thick},
braceMirror/.style={decorate, decoration={calligraphic brace, amplitude=4pt, mirror}, thick},
smallbrace/.style={decorate, decoration={calligraphic brace, amplitude=2.5pt}, thick},
smallbraceMirror/.style={decorate, decoration={calligraphic brace, amplitude=2.5pt, mirror}, thick},
guideline/.style={densely dotted, gray, thin}
]
% ==========================================================
% 1. Coordinates and dimensions
% ==========================================================
\pgfmathsetmacro{\TotalW}{14.0}
\pgfmathsetmacro{\SplitX}{7.5}
\pgfmathsetmacro{\TotalH}{5.60}
\pgfmathsetmacro{\yTop}{5.60}
\pgfmathsetmacro{\yOne}{4.45}
\pgfmathsetmacro{\yTwo}{3.55}
\pgfmathsetmacro{\yThree}{2.40}
\pgfmathsetmacro{\yFour}{1.15}
\pgfmathsetmacro{\yBot}{0.0}
\pgfmathsetmacro{\LabelYOffset}{0.22}
% --- X-axis coordinates: Blue part ---
\pgfmathsetmacro{\wBOne}{2.0}
\pgfmathsetmacro{\gapBOne}{0.8}
\pgfmathsetmacro{\wBMid}{2.2}
\pgfmathsetmacro{\gapBTwo}{0.8}
\pgfmathsetmacro{\xBEndOne}{\wBOne}
\pgfmathsetmacro{\xBStartMid}{\xBEndOne + \gapBOne}
\pgfmathsetmacro{\xBEndMid}{\xBStartMid + \wBMid}
\pgfmathsetmacro{\xBStartBot}{\xBEndMid + \gapBTwo}
\pgfmathsetmacro{\xBEndBot}{\SplitX}
% --- X-axis coordinates: Orange part ---
\pgfmathsetmacro{\wOOne}{1.5}
\pgfmathsetmacro{\gapOOne}{0.8}
\pgfmathsetmacro{\wOMid}{2.0}
\pgfmathsetmacro{\gapOTwo}{0.8}
\pgfmathsetmacro{\xOStartOne}{\SplitX}
\pgfmathsetmacro{\xOEndOne}{\SplitX + \wOOne}
\pgfmathsetmacro{\xOStartMid}{\xOEndOne + \gapOOne}
\pgfmathsetmacro{\xOEndMid}{\xOStartMid + \wOMid}
\pgfmathsetmacro{\xOStartBot}{\xOEndMid + \gapOTwo}
\pgfmathsetmacro{\xOEndBot}{\TotalW}
% ==========================================================
% 2. Colored blocks and entries
% ==========================================================
% --- Block 1 ---
\draw[fill=cyan!20] (0, \yOne) rectangle (\xBEndOne, \yTop);
\node at (0.5*\xBEndOne, 0.5*\yOne + 0.5*\yTop + \LabelYOffset)
{$\bm{I}_{h_{1,1}}$};
\node at (0.5*\xBEndOne, 0.5*\yOne + 0.5*\yTop - \LabelYOffset)
{$\bm{0}$};
\draw[fill=orange!20] (\xOStartOne, \yOne) rectangle (\xOEndOne, \yTop);
\node at (0.5*\xOStartOne + 0.5*\xOEndOne, 0.5*\yOne + 0.5*\yTop + \LabelYOffset)
{$\bm{0}$};
\node at (0.5*\xOStartOne + 0.5*\xOEndOne, 0.5*\yOne + 0.5*\yTop - \LabelYOffset)
{$\bm{I}_{h_{2,1}}$};
% --- Gap 1 ---
\draw[fill=cyan!20] (\xBEndOne, \yTwo) rectangle (\xBStartMid, \yOne);
\draw[fill=orange!20] (\xOEndOne, \yTwo) rectangle (\xOStartMid, \yOne);
\pgfmathsetmacro{\GapOneX}{(\xBEndOne + \xBStartMid)/2}
\pgfmathsetmacro{\GapOneY}{(\yOne + \yTwo)/2}
\node at (\GapOneX, \GapOneY) {\Large $\ddots$};
\pgfmathsetmacro{\GapOneOX}{(\xOEndOne + \xOStartMid)/2}
\node at (\GapOneOX, \GapOneY) {\Large $\ddots$};
% --- Block i ---
\draw[fill=cyan!20] (\xBStartMid, \yThree) rectangle (\xBEndMid, \yTwo);
\node at (0.5*\xBStartMid + 0.5*\xBEndMid, 0.5*\yThree + 0.5*\yTwo + \LabelYOffset)
{$\bm{I}_{h_{1,i}}$};
\node at (0.5*\xBStartMid + 0.5*\xBEndMid, 0.5*\yThree + 0.5*\yTwo - \LabelYOffset)
{$\bm{0}$};
\draw[fill=orange!20] (\xOStartMid, \yThree) rectangle (\xOEndMid, \yTwo);
\node at (0.5*\xOStartMid + 0.5*\xOEndMid, 0.5*\yThree + 0.5*\yTwo + \LabelYOffset)
{$\bm{0}$};
\node at (0.5*\xOStartMid + 0.5*\xOEndMid, 0.5*\yThree + 0.5*\yTwo - \LabelYOffset)
{$\bm{I}_{h_{2,i}}$};
\draw[smallbraceMirror] (\xBStartMid, \yThree-0.07) -- (\xBEndMid, \yThree-0.07)
node[midway, below=1pt] {\scriptsize $h_{1,i}$};
\draw[smallbraceMirror] (\xOStartMid, \yThree-0.07) -- (\xOEndMid, \yThree-0.07)
node[midway, below=1pt] {\scriptsize $h_{2,i}$};
% --- Gap 2 ---
\draw[fill=cyan!20] (\xBEndMid, \yFour) rectangle (\xBStartBot, \yThree);
\draw[fill=orange!20] (\xOEndMid, \yFour) rectangle (\xOStartBot, \yThree);
\pgfmathsetmacro{\GapTwoX}{(\xBEndMid + \xBStartBot)/2}
\pgfmathsetmacro{\GapTwoY}{(\yThree + \yFour)/2}
\node at (\GapTwoX, \GapTwoY) {\Large $\ddots$};
\pgfmathsetmacro{\GapTwoOX}{(\xOEndMid + \xOStartBot)/2}
\node at (\GapTwoOX, \GapTwoY) {\Large $\ddots$};
% --- Block n_A ---
\draw[fill=cyan!20] (\xBStartBot, \yBot) rectangle (\xBEndBot, \yFour);
\node at (0.5*\xBStartBot + 0.5*\xBEndBot, 0.5*\yBot + 0.5*\yFour + \LabelYOffset)
{$\bm{I}_{h_{1,n_A}}$};
\node at (0.5*\xBStartBot + 0.5*\xBEndBot, 0.5*\yBot + 0.5*\yFour - \LabelYOffset)
{$\bm{0}$};
\draw[fill=orange!20] (\xOStartBot, \yBot) rectangle (\xOEndBot, \yFour);
\node at (0.5*\xOStartBot + 0.5*\xOEndBot, 0.5*\yBot + 0.5*\yFour + \LabelYOffset)
{$\bm{0}$};
\node at (0.5*\xOStartBot + 0.5*\xOEndBot, 0.5*\yBot + 0.5*\yFour - \LabelYOffset)
{$\bm{I}_{h_{2,n_A}}$};
% ==========================================================
% 3. Alignment guidelines
% ==========================================================
\draw[guideline] (\xBEndOne, \TotalH) -- (\xBEndOne, 0);
\draw[guideline] (\xBStartMid, \TotalH) -- (\xBStartMid, 0);
\draw[guideline] (\xBEndMid, \TotalH) -- (\xBEndMid, 0);
\draw[guideline] (\xBStartBot, \TotalH) -- (\xBStartBot, 0);
\draw[guideline] (\xOEndOne, \TotalH) -- (\xOEndOne, 0);
\draw[guideline] (\xOStartMid, \TotalH) -- (\xOStartMid, 0);
\draw[guideline] (\xOEndMid, \TotalH) -- (\xOEndMid, 0);
\draw[guideline] (\xOStartBot, \TotalH) -- (\xOStartBot, 0);
% ==========================================================
% 4. Frame and partition lines
% ==========================================================
\draw[thick] (0, 0) rectangle (\TotalW, \TotalH);
\draw[thick, dashed] (\SplitX, 0) -- (\SplitX, \TotalH);
\draw[dotted] (0, \yOne) -- (\TotalW, \yOne);
\draw[dotted] (0, \yTwo) -- (\TotalW, \yTwo);
\draw[dotted] (0, \yThree) -- (\TotalW, \yThree);
\draw[dotted] (0, \yFour) -- (\TotalW, \yFour);
% ==========================================================
% 5. Axis ticks and labels
% ==========================================================
\pgfmathsetmacro{\tickL}{0.07}
% --- Left vertical axis labels ---
\draw[thick] (0, \yTop) -- ++(-\tickL, 0) node[left, inner sep=2pt] {$1$};
\draw[thick] (0, \yOne) -- ++(-\tickL, 0) node[left, inner sep=2pt] {$s_1$};
\node at (-0.32, \GapOneY) {$\vdots$};
\draw[thick] (0, \yTwo) -- ++(-\tickL, 0) node[left, inner sep=2pt] {$s_{i-1}$};
\draw[thick] (0, \yThree) -- ++(-\tickL, 0) node[left, inner sep=2pt] {$s_i$};
\node at (-0.32, \GapTwoY) {$\vdots$};
\draw[thick] (0, \yFour) -- ++(-\tickL, 0) node[left, inner sep=2pt] {$s_{n_A-1}$};
\draw[thick] (0, 0) -- ++(-\tickL, 0) node[left, inner sep=2pt] {$d$};
% --- Top horizontal axis labels ---
\draw[thick] (0, \TotalH) -- ++(0, \tickL)
node[above=1pt, inner sep=1pt] {$1$};
\draw[thick] (\xBEndOne, \TotalH) -- ++(0, \tickL)
node[above=1pt, inner sep=1pt, font=\scriptsize] {$h_{1,1}$};
\draw[thick] (\xBStartMid, \TotalH) -- ++(0, \tickL);
\draw[thick] (\xBEndMid, \TotalH) -- ++(0, \tickL);
\draw[thick] (\xBStartBot, \TotalH) -- ++(0, \tickL);
\draw[thick] (\SplitX, \TotalH) -- ++(0, \tickL)
node[above=1pt, inner sep=1pt] {$t_{n_B}$};
\draw[thick] (\xOEndOne, \TotalH) -- ++(0, \tickL)
node[above=1pt, inner sep=1pt, font=\scriptsize] {$t_{n_B}+h_{2,1}$};
\draw[thick] (\xOStartMid, \TotalH) -- ++(0, \tickL);
\draw[thick] (\xOEndMid, \TotalH) -- ++(0, \tickL);
\draw[thick] (\xOStartBot, \TotalH) -- ++(0, \tickL);
\draw[thick] (\TotalW, \TotalH) -- ++(0, \tickL)
node[above=1pt, inner sep=1pt] {$d$};
% --- Top braces with labels attached directly ---
\pgfmathsetmacro{\yBraceTop}{\TotalH + 0.42}
\draw[brace] (0, \yBraceTop) -- (\SplitX, \yBraceTop)
node[midway, above=3pt] {$\bar{\bm E}(\bm h_1)$};
\draw[brace] (\SplitX, \yBraceTop) -- (\TotalW, \yBraceTop)
node[midway, above=3pt] {$\widehat{\bm E}(\bm h_2)$};
% --- Bottom braces ---
\draw[braceMirror] (0, -0.08) -- (\SplitX, -0.08)
node[midway, below=1pt] {\scriptsize $t_{n_B}$ columns};
\draw[braceMirror] (\SplitX, -0.08) -- (\TotalW, -0.08)
node[midway, below=1pt] {\scriptsize $d-t_{n_B}$ columns};
\end{tikzpicture}
}
\caption{Visualization of the block structure of matrix $\bm E(\bm h)$ defined in \eqref{eq:basedef}. All entries in the unshaded regions are zero.}
\label{fig:blockplot}
\end{figure}
% \begin{figure}[t]
% \centering
% % === 核心修改：使用 makebox 强制居中超宽的图片 ===
% \makebox[\textwidth][c]{
% \begin{tikzpicture}[
% scale=0.8, transform shape,
% scale=0.85, % 适当调小比例，防止超出纸张边缘太多
% every node/.style={scale=0.9},
% brace/.style={decorate, decoration={calligraphic brace, amplitude=5pt}, thick},
% braceMirror/.style={decorate, decoration={calligraphic brace, amplitude=5pt, mirror}, thick},
% smallbrace/.style={decorate, decoration={calligraphic brace, amplitude=3pt}, thick},
% smallbraceMirror/.style={decorate, decoration={calligraphic brace, amplitude=3pt, mirror}, thick},
% axisline/.style={thick},
% guideline/.style={densely dotted, gray, thin} % 辅助线样式
% ]
% % ==========================================================
% % 1. 坐标与尺寸定义 (保证省略号方块宽度严格一致)
% % ==========================================================
% \pgfmathsetmacro{\TotalW}{14.0}
% \pgfmathsetmacro{\TotalH}{8.10}
% \pgfmathsetmacro{\SplitX}{7.5}
% % Y轴坐标
% \pgfmathsetmacro{\yTop}{8.10}
% \pgfmathsetmacro{\yOne}{6.40}
% \pgfmathsetmacro{\yTwo}{5.10}
% \pgfmathsetmacro{\yThree}{2.98}
% \pgfmathsetmacro{\yFour}{1.70}
% \pgfmathsetmacro{\yBot}{0.0}
% \pgfmathsetmacro{\LabelYOffset}{0.32}
% % --- X轴坐标 (Blue 区域) ---
% \pgfmathsetmacro{\wBOne}{2.0}
% \pgfmathsetmacro{\gapBOne}{0.8} % 第一个省略号方块的宽度
% \pgfmathsetmacro{\wBMid}{2.2}
% \pgfmathsetmacro{\gapBTwo}{0.8} % 第二个省略号方块的宽度 (强制与第一个相等)
% \pgfmathsetmacro{\wBBot}{1.7} % 剩余宽度 (7.5 - 2.0 - 0.8 - 2.2 - 0.8 = 1.7)
% \pgfmathsetmacro{\xBEndOne}{\wBOne}
% \pgfmathsetmacro{\xBStartMid}{\xBEndOne + \gapBOne}
% \pgfmathsetmacro{\xBEndMid}{\xBStartMid + \wBMid}
% \pgfmathsetmacro{\xBStartBot}{\xBEndMid + \gapBTwo}
% \pgfmathsetmacro{\xBEndBot}{\SplitX}
% % --- X轴坐标 (Orange 区域) ---
% \pgfmathsetmacro{\wOOne}{1.5}
% \pgfmathsetmacro{\gapOOne}{0.8} % 第一个省略号方块的宽度
% \pgfmathsetmacro{\wOMid}{2.0}
% \pgfmathsetmacro{\gapOTwo}{0.8} % 第二个省略号方块的宽度 (强制与第一个相等)
% \pgfmathsetmacro{\wOBot}{1.4} % 剩余宽度 (6.5 - 1.5 - 0.8 - 2.0 - 0.8 = 1.4)
% \pgfmathsetmacro{\xOStartOne}{\SplitX}
% \pgfmathsetmacro{\xOEndOne}{\SplitX + \wOOne}
% \pgfmathsetmacro{\xOStartMid}{\xOEndOne + \gapOOne}
% \pgfmathsetmacro{\xOEndMid}{\xOStartMid + \wOMid}
% \pgfmathsetmacro{\xOStartBot}{\xOEndMid + \gapOTwo}
% \pgfmathsetmacro{\xOEndBot}{\TotalW}
% % ==========================================================
% % 2. 绘制色块与内部元素
% % ==========================================================
% % --- Block 1 ---
% \draw[fill=cyan!20] (0, \yOne) rectangle (\xBEndOne, \yTop);
% \node at (0.5*\xBEndOne, 0.5*\yOne + 0.5*\yTop + \LabelYOffset) {$\bm{I}_{h_{1,1}}$};
% \node at (0.5*\xBEndOne, 0.5*\yOne + 0.5*\yTop - \LabelYOffset) {$\bm{0}$};
% \draw[fill=orange!20] (\xOStartOne, \yOne) rectangle (\xOEndOne, \yTop);
% \node at (0.5*\xOStartOne + 0.5*\xOEndOne, 0.5*\yOne + 0.5*\yTop + \LabelYOffset) {$\bm{0}$};
% \node at (0.5*\xOStartOne + 0.5*\xOEndOne, 0.5*\yOne + 0.5*\yTop - \LabelYOffset) {$\bm{I}_{h_{2,1}}$};
% % \node at (\xBEndOne + 0.5*\SplitX - 0.5*\xBEndOne, 0.5*\yOne + 0.5*\yTop) {\Large $0$};
% % \node at (\xOEndOne + 0.5*\TotalW - 0.5*\xOEndOne, 0.5*\yOne + 0.5*\yTop) {\Large $0$};
% % --- Gap 1 填充与省略号 ---
% \draw[fill=cyan!20] (\xBEndOne, \yTwo) rectangle (\xBStartMid, \yOne);
% \draw[fill=orange!20] (\xOEndOne, \yTwo) rectangle (\xOStartMid, \yOne);
% \pgfmathsetmacro{\GapOneX}{(\xBEndOne + \xBStartMid)/2}
% \pgfmathsetmacro{\GapOneY}{(\yOne + \yTwo)/2}
% \node at (\GapOneX, \GapOneY) {\Huge $\ddots$};
% \pgfmathsetmacro{\GapOneOX}{(\xOEndOne + \xOStartMid)/2}
% \node at (\GapOneOX, \GapOneY) {\Huge $\ddots$};
% % --- Block i ---
% \draw[fill=cyan!20] (\xBStartMid, \yThree) rectangle (\xBEndMid, \yTwo);
% \node at (0.5*\xBStartMid + 0.5*\xBEndMid, 0.5*\yThree + 0.5*\yTwo + \LabelYOffset) {$\bm{I}_{h_{1,i}}$};
% \node at (0.5*\xBStartMid + 0.5*\xBEndMid, 0.5*\yThree + 0.5*\yTwo - \LabelYOffset) {$\bm{0}$};
% \draw[fill=orange!20] (\xOStartMid, \yThree) rectangle (\xOEndMid, \yTwo);
% \node at (0.5*\xOStartMid + 0.5*\xOEndMid, 0.5*\yThree + 0.5*\yTwo + \LabelYOffset) {$\bm{0}$};
% \node at (0.5*\xOStartMid + 0.5*\xOEndMid, 0.5*\yThree + 0.5*\yTwo - \LabelYOffset) {$\bm{I}_{h_{2,i}}$};
% % \node at (0.5*\xBStartMid, 0.5*\yThree + 0.5*\yTwo) {\Large $0$};
% % \node at (\xBEndMid + 0.5*\SplitX - 0.5*\xBEndMid, 0.5*\yThree + 0.5*\yTwo) {\Large $0$};
% % \node at (\SplitX + 0.5*\xOStartMid - 0.5*\SplitX, 0.5*\yThree + 0.5*\yTwo) {\Large $0$};
% % \node at (\xOEndMid + 0.5*\TotalW - 0.5*\xOEndMid, 0.5*\yThree + 0.5*\yTwo) {\Large $0$};
% \draw[smallbraceMirror] (\xBStartMid, \yThree-0.1) -- (\xBEndMid, \yThree-0.1) node[midway, below=2pt] {\scriptsize $h_{1,i}$};
% \draw[smallbraceMirror] (\xOStartMid, \yThree-0.1) -- (\xOEndMid, \yThree-0.1) node[midway, below=2pt] {\scriptsize $h_{2,i}$};
% % --- Gap 2 填充与省略号 ---
% \draw[fill=cyan!20] (\xBEndMid, \yFour) rectangle (\xBStartBot, \yThree);
% \draw[fill=orange!20] (\xOEndMid, \yFour) rectangle (\xOStartBot, \yThree);
% \pgfmathsetmacro{\GapTwoX}{(\xBEndMid + \xBStartBot)/2}
% \pgfmathsetmacro{\GapTwoY}{(\yThree + \yFour)/2}
% \node at (\GapTwoX, \GapTwoY) {\Huge $\ddots$};
% \pgfmathsetmacro{\GapTwoOX}{(\xOEndMid + \xOStartBot)/2}
% \node at (\GapTwoOX, \GapTwoY) {\Huge $\ddots$};
% % --- Block nA ---
% \draw[fill=cyan!20] (\xBStartBot, \yBot) rectangle (\xBEndBot, \yFour);
% \node at (0.5*\xBStartBot + 0.5*\xBEndBot, 0.5*\yBot + 0.5*\yFour + \LabelYOffset) {$\bm{I}_{h_{1,n_A}}$};
% \node at (0.5*\xBStartBot + 0.5*\xBEndBot, 0.5*\yBot + 0.5*\yFour - \LabelYOffset) {$\bm{0}$};
% \draw[fill=orange!20] (\xOStartBot, \yBot) rectangle (\xOEndBot, \yFour);
% \node at (0.5*\xOStartBot + 0.5*\xOEndBot, 0.5*\yBot + 0.5*\yFour + \LabelYOffset) {$\bm{0}$};
% \node at (0.5*\xOStartBot + 0.5*\xOEndBot, 0.5*\yBot + 0.5*\yFour - \LabelYOffset) {$\bm{I}_{h_{2,n_A}}$};
% % \node at (0.5*\xBStartBot, 0.5*\yBot + 0.5*\yFour) {\Large $0$};
% % \node at (\SplitX + 0.5*\xOStartBot - 0.5*\SplitX, 0.5*\yBot + 0.5*\yFour) {\Large $0$};
% % ==========================================================
% % 3. 辅助对齐线 (贯穿全高)
% % ==========================================================
% % Blue Lines
% \draw[guideline] (\xBEndOne, \TotalH) -- (\xBEndOne, 0);
% \draw[guideline] (\xBStartMid, \TotalH) -- (\xBStartMid, 0);
% \draw[guideline] (\xBEndMid, \TotalH) -- (\xBEndMid, 0);
% \draw[guideline] (\xBStartBot, \TotalH) -- (\xBStartBot, 0);
% % Orange Lines
% \draw[guideline] (\xOEndOne, \TotalH) -- (\xOEndOne, 0);
% \draw[guideline] (\xOStartMid, \TotalH) -- (\xOStartMid, 0);
% \draw[guideline] (\xOEndMid, \TotalH) -- (\xOEndMid, 0);
% \draw[guideline] (\xOStartBot, \TotalH) -- (\xOStartBot, 0);
% % ==========================================================
% % 4. 框架与分割线
% % ==========================================================
% \draw[thick] (0, 0) rectangle (\TotalW, \TotalH);
% \draw[thick, dashed] (\SplitX, 0) -- (\SplitX, \TotalH);
% \draw[dotted] (0, \yOne) -- (\TotalW, \yOne);
% \draw[dotted] (0, \yTwo) -- (\TotalW, \yTwo);
% \draw[dotted] (0, \yThree) -- (\TotalW, \yThree);
% \draw[dotted] (0, \yFour) -- (\TotalW, \yFour);
% % ==========================================================
% % 5. 坐标轴与刻度线标注
% % ==========================================================
% \pgfmathsetmacro{\tickL}{0.08}
% % --- 左侧纵轴 ---
% \draw[thick] (0, \yTop) -- ++(-\tickL, 0) node[left, inner sep=3pt] {$1$};
% \draw[thick] (0, \yOne) -- ++(-\tickL, 0) node[left, inner sep=3pt] {$s_1$};
% \node at (-0.35, \GapOneY) {$\vdots$};
% \draw[thick] (0, \yTwo) -- ++(-\tickL, 0) node[left, inner sep=3pt] {$s_{i-1}$};
% \draw[thick] (0, \yThree) -- ++(-\tickL, 0) node[left, inner sep=3pt] {$s_i$};
% \node at (-0.35, \GapTwoY) {$\vdots$};
% \draw[thick] (0, \yFour) -- ++(-\tickL, 0) node[left, inner sep=3pt] {$s_{n_A-1}$};
% \draw[thick] (0, 0) -- ++(-\tickL, 0) node[left, inner sep=3pt] {$d$};
% % --- 顶部横轴 ---
% % 1. Start Point
% \draw[thick] (0, \TotalH) -- ++(0, \tickL) node[above, inner sep=3pt] {$1$};
% % 2. Blue Gap 1
% \draw[thick] (\xBEndOne, \TotalH) -- ++(0, \tickL) node[above, inner sep=3pt] {$h_{1,1}$};
% % \node at (\GapOneX, \TotalH + 0.2) {$\cdots$};
% % 3. Blue Block i (恢复刻度，对接辅助线)
% \draw[thick] (\xBStartMid, \TotalH) -- ++(0, \tickL);
% \draw[thick] (\xBEndMid, \TotalH) -- ++(0, \tickL);
% % 4. Blue Gap 2
% \pgfmathsetmacro{\GapTwoXTop}{(\xBEndMid + \xBStartBot)/2}
% % \node at (\GapTwoXTop, \TotalH + 0.2) {$\cdots$};
% \draw[thick] (\xBStartBot, \TotalH) -- ++(0, \tickL);
% % 5. Split Line (Center)
% \draw[thick] (\SplitX, \TotalH) -- ++(0, \tickL) node[above, inner sep=3pt] {$t_{n_B}$};
% % --- Right Side (Orange) ---
% % 6. Orange Gap 1
% \draw[thick] (\xOEndOne, \TotalH) -- ++(0, \tickL) node[above, inner sep=3pt] {$t_{n_B}+h_{2,1}$};
% % \node at (\GapOneOX, \TotalH + 0.2) {$\cdots$};
% % 7. Orange Block i (恢复刻度，对接辅助线)
% \draw[thick] (\xOStartMid, \TotalH) -- ++(0, \tickL);
% \draw[thick] (\xOEndMid, \TotalH) -- ++(0, \tickL);
% % 8. Orange Gap 2
% \pgfmathsetmacro{\GapTwoOXTop}{(\xOEndMid + \xOStartBot)/2}
% % \node at (\GapTwoOXTop, \TotalH + 0.2) {$\cdots$};
% \draw[thick] (\xOStartBot, \TotalH) -- ++(0, \tickL);
% % 9. End Point
% \draw[thick] (\TotalW, \TotalH) -- ++(0, \tickL) node[above, inner sep=3pt] {$d$};
% % --- 顶部大括号 ---
% \pgfmathsetmacro{\yBraceTop}{\TotalH + 0.5}
% \node[above] at (0.5*\SplitX, \yBraceTop + 0.1) {\large $\bar{\bm E}(\bm h_1)$};
% \node[above] at (\SplitX + 0.5*\TotalW - 0.5*\SplitX, \yBraceTop + 0.1) {\large $\widehat{\bm E}(\bm h_2)$};
% \draw[brace] (0, \yBraceTop) -- (\SplitX, \yBraceTop);
% \draw[brace] (\SplitX, \yBraceTop) -- (\TotalW, \yBraceTop);
% % --- 底部大括号 ---
% \draw[braceMirror] (0, -0.2) -- (\SplitX, -0.2) node[midway, below=3pt] {$t_{n_B}$ columns};
% \draw[braceMirror] (\SplitX, -0.2) -- (\TotalW, -0.2) node[midway, below=3pt] {$d - t_{n_B}$ columns};
% \end{tikzpicture}
% } % === 结束 makebox ===
% \caption{Visualization of the structure of matrix $\bm E(\bm h)$ defined in \eqref{eq:basedef} with strictly aligned indices. Vertical dotted lines illustrate the alignment between the column partitions and the row blocks. All entries in the unshaded regions are zero.}
% \label{fig:blockplot}
% \end{figure}
\begin{theorem}\label{thm:singular-case}
Suppose that \(\bm A\in\S^d\) is in the form \eqref{eq:block a} and
\(\bm B\in\S^K\) is singular but nonzero, and is in the form
\eqref{eq:block b1}.
%Suppose that $\bm A \in \S^d$ and $\bm B \in \S^K$ are in the forms given by \eqref{eq:block a} and \eqref{eq:block b1}.
Then the set $\mathcal{Q}$ can be expressed as
\begin{align*}
\mathcal{Q} = \bigcup_{\bm h\in \widetilde{\mathcal{H}},\ \bm \Pi\in \mathcal{P}^{t_{n_B}}} \mathcal{Q}_{\bm h,\bm \Pi},
\end{align*}
where every $\bm Q \in \mathcal{Q}_{\bm h,\bm \Pi}$ takes the form
\begin{align}
\bm Q = \mathrm{BlkD}(\bm U_1,\ldots,\bm U_{n_A}) \left[\bar{\bm E}(\bm h_1) \bm \Pi\ \widehat{\bm E}(\bm h_2)\bm V_{n_B+1}\right] \mathrm{BlkD}\left(\bm V_1^T,\ldots,\bm V_{n_B}^T,\bm I_{K-t_{n_B}} \right)
\label{eq:exp_singular}
\end{align}
for some $\bm U_i\in \mathcal{O}^{s_i-s_{i-1}}$ for each $i\in [n_A]$, $\bm V_j \in \mathcal{O}^{t_j -t_{j-1}}$ for each $j\in[n_B]$, and $\bm V_{n_B+1} \in \mathrm{St}(d-t_{n_B}, K-t_{n_B})$. % \redit{The two blocks correspond to the nonzero and zero parts of $\bm B$, respectively.}
\end{theorem}
\begin{proof}
Using $\bar{\bm E}_i(\bm h_1)^T\widehat{\bm E}_i(\bm h_2) = \bm 0$ for each $i\in [n_A]$, it is straightforward to verify that every $\bm Q \in \mathcal{Q}_{\bm h,\bm \Pi}$ of the form~\eqref{eq:exp_singular} satisfies $\bm A\bm Q \bm B - \bm Q \bm B\bm Q^T\bm A\bm Q = \bm 0$, and thus $\mathrm{grad} F(\bm Q) = \bm 0$.
Using the equivalence between \eqref{eq:crit set} and \eqref{eq:crit 1}, it suffices to take an arbitrary \(\bm Q=[\bm Q_1,\bm Q_2]\in\mathcal Q\) as in \eqref{eq:crit 1} and show that it admits the representation \eqref{eq:exp_singular}.
% Using the equivalence between \eqref{eq:crit set} and \eqref{eq:crit 1}, it remains to show that every point in~\eqref{eq:crit 1} can be written in the form of \eqref{eq:exp_singular}. Take any $\bm Q = [\bm Q_1,\, \bm Q_2] \in \mathcal{Q}$ as defined in \eqref{eq:crit 1}.
We note that $\mathcal{Q}_1$ in \eqref{eq:crit 1} coincides with the critical point set of the following QPOC problem
\[
\max_{\bm{Q} \in \mathrm{St}(d, t_{n_B})} \mathrm{Tr}\bigl( \bm{Q}^T \bm{A} \bm{Q} \widehat{\bm{B}} \bigr).
\]
By \Cref{fact:1}, every $\bm{Q}_1 \in \mathcal{Q}_1$ admits the representation
\begin{equation}
\bm{Q}_1 = \mathrm{BlkD}(\bm{U}_1, \ldots, \bm{U}_{n_A}) \bar{\bm{E}}(\bm{h}_1) \, \bm{\Pi} \mathrm{BlkD}(\bm{V}_1^T, \ldots, \bm{V}_{n_B}^T),\notag
\end{equation}
where $\bm{U}_i \in \mathcal{O}^{s_i - s_{i-1}}$ for all $i \in [n_A]$, $\bm{V}_j \in \mathcal{O}^{t_j - t_{j-1}}$ for all $j \in [n_B]$, and $\bar{\bm{E}}(\bm{h}_1)$ is defined in~\eqref{eq:Eh1}. Then $\bm U := \mathrm{BlkD}(\bm U_1, \ldots, \bm U_{n_A})$ is an orthogonal matrix whose columns form an orthonormal basis for $\mathbb{R}^d$ and $\bar{\bm E}(\bm h_1)$ selects $t_{n_B}$ columns of $\bm U$.
Moreover, right multiplication by $\bm \Pi$ and $\mathrm{BlkD}(\bm V_1^T,\ldots,\bm V_{n_B}^T)$ does not change the column space of $\bm Q_1$. Hence, with $\mathrm{col}(\cdot)$ denoting the column space, $\mathrm{col}(\bm Q_1)
= \mathrm{col}\bigl(\bm U\bar{\bm E}(\bm h_1)\bigr).$ Since $\bm Q=[\bm Q_1,\bm Q_2]\in \mathrm{St}(d,K)$, we have $\bm Q_1^T\bm Q_2=\bm 0$. %Therefore, the column space of $\bm Q_2$ is contained in the orthogonal complement of $\mathrm{col}(\bm Q_1)$. This orthogonal complement is spanned by the columns of \(\bm U\widehat{\bm E}(\bm h_2)\).
Therefore,
\(\mathrm{col}(\bm Q_2)\subseteq \mathrm{col}(\bm Q_1)^\perp
=
\mathrm{col}(\bm U\widehat{\bm E}(\bm h_2))\).
Hence, there exists \(\bm V_{n_B+1}\in\mathrm{St}(d-t_{n_B},K-t_{n_B})\) such that $\bm Q_2 = \bm U\widehat{\bm E}(\bm h_2)\bm V_{n_B+1}.$ Consequently, \(\bm Q\) is in the form \eqref{eq:exp_singular}.
\end{proof}
% This closed-form characterization of the critical point set enables a systematic study of the different types of critical points and thus lays the foundation for the subsequent analysis of the optimization landscape of \edit{Problem~\eqref{eq:QPOC}}.
\section{Optimization Landscape Analysis of QPOC}\label{sec:QPOC}
In this section, we study the optimization landscape of Problem~\eqref{eq:QPOC}. Without loss of generality, we focus on the case where $\bm B$ is singular, since the nonsingular case corresponds to the special case $t_{n_B}=K$, where the zero block is absent. Moreover, for any scalar $\alpha$, replacing $\bm A$ in \eqref{eq:block a} by $\bm A+\alpha\bm I$ changes the objective value on $\mathrm{St}(d,K)$ only by the additive constant $\alpha\mathrm{Tr}(\bm B)$. Therefore, we assume without loss of generality that $\bm A \succ \bm 0$, i.e., $a_i > 0$ for all $i \in [d]$.
To begin, we identify a condition, expressed in terms of the permutation and the signs of the nonzero diagonal entries of $\bm B$, under which a critical point is a global maximizer of Problem~\eqref{eq:QPOC}.
Before proceeding with the analysis, we introduce some notation. By \Cref{thm:singular-case}, any $\bm Q \in \mathcal{Q}_{\bm h,\bm \Pi}$ takes the form
\begin{align}\label{eq1:prop saddle}
\bm Q = \mathrm{BlkD}(\bm U_1,\ldots,\bm U_{n_A}) \left[\bar{\bm E}(\bm h_1)\bm \Pi\ \ \widehat{\bm E}(\bm h_2)\bm V_{n_B+1}\right] \mathrm{BlkD}\left(\bm V_1^T,\ldots,\bm V_{n_B}^T,\bm I\right)
\end{align}
for some $\bm U_i\in \mathcal{O}^{s_i-s_{i-1}}$ for each $i\in[n_A]$, $\bm V_j \in \mathcal{O}^{t_j -t_{j-1}}$ for each $j\in[n_B]$, and $\bm V_{n_B+1} \in \mathrm{St}(d-t_{n_B}, K - t_{n_B})$. For ease of exposition, define
\begin{align}
\label{eq:defuv}
\bm U:= \mathrm{BlkD}(\bm U_1,\ldots,\bm U_{n_A}),\quad
\bm V := \mathrm{BlkD}\left(\bm V_1^T,\ldots,\bm V_{n_B}^T,\bm I_{K-t_{n_B}}\right).
\end{align}
Moreover, define
\begin{align}\label{eq:defPsi}
\bm \Psi := \mathrm{diag}\left(a_1,\ldots,a_{t_p}, a_{d+t_p-t_{n_B}+1}, \ldots, a_d\right) \in \S^{t_{n_B}},\ \forall p \in \{0,1,\dots,n_B\},
\end{align}
where the diagonal entries of $\bm \Psi$ are sorted in descending order. In particular, $p = 0$ (resp., $p = n_B$) indicates $\bm \Psi := \mathrm{diag}(a_{d+1-t_{n_B}},\ldots,a_d)$ (resp., $ \bm \Psi := \mathrm{diag}(a_1,\ldots,a_{t_p})$). Further define
\begin{equation}
\label{eq:deftildeA}
\widetilde{\bm A} := \bm \Pi^T \bar{\bm{E}}(\bm h_1)^T \bm{A} \bar{\bm{E}}(\bm h_1)\bm \Pi \in \R^{t_{n_B} \times t_{n_B}}.
\end{equation}
Since $\bm A$ is diagonal and $\bar{\bm E}(\bm h_1)$ is a coordinate-selection matrix, $\widetilde{\bm A}$ is diagonal and admits the block partition
\[
\widetilde{\bm A} = \mathrm{BlkD}\!\left(\mathrm{diag}(\tilde{\bm a}_1),\dots,\mathrm{diag}(\tilde{\bm a}_{n_B})\right),\ \text{where}\ \tilde{\bm a}_i \in \mathbb{R}^{t_i-t_{i-1}},\ \forall i\in[n_B].
\]
For each $i\in[n_B]$, choose $\bar{\bm \Pi}_i \in \mathcal{P}^{t_i-t_{i-1}}$ such that
\begin{align*}
\bar{\bm \Pi}_i^T\mathrm{diag}(\tilde{\bm a}_i)\bar{\bm \Pi}_i = \mathrm{diag}\left(\bar{\bm a}_{i} \right),
\end{align*}
where $\bar{\bm a}_i=(\bar a_{i,1},\ldots,\bar a_{i,t_i-t_{i-1}})$ is sorted in decreasing order $\bar a_{i,1}\ge \bar a_{i,2}\ge \cdots\ge \bar a_{i,t_i-t_{i-1}}$ for each $i \in [n_B]$.
Let $\bar{\bm\Pi}:=\mathrm{BlkD}(\bar{\bm\Pi}_1,\ldots,\bar{\bm\Pi}_{n_B})$ and
\begin{align}\label{eq:Omega_def_rewrite}
\bm\Omega :=\mathrm{BlkD}(\mathrm{diag}\left(\bar{\bm a}_1),\ldots,\mathrm{diag}(\bar{\bm a}_{n_B})\right)= \bar{\bm \Pi}^T\widetilde{\bm A}\bar{\bm \Pi}.
\end{align}
By construction, $\bm\Omega$ is a blockwise sorted version of
$\widetilde{\bm A}$, with the blocks corresponding to the equal-eigenvalue blocks of $\widehat{\bm B}$ defined in \eqref{eq:block b1}. In contrast, $\bm\Psi$ represents the optimal spectral
assignment dictated by the signs of the nonzero eigenvalues of $\bm B$: the
positive entries of $\widehat{\bm B}$ are paired with the largest diagonal
entries of $\bm A$, whereas the negative entries are paired with the
smallest ones. The following proposition shows that if this pattern is realized, then the corresponding critical point is globally optimal.
\begin{proposition}\label{prop:global}
Fix $\bm h\in \widetilde{\mathcal{H}}$ and
$\bm \Pi\in \mathcal{P}^{t_{n_B}}$, and let
$\bm Q \in \mathcal{Q}_{\bm h,\bm \Pi}$. Let $\bm\Psi$ and $\bm\Omega$
be defined as in \eqref{eq:defPsi} and \eqref{eq:Omega_def_rewrite}, respectively.
Suppose that $\bm\Psi=\bm\Omega$. Then $\bm Q$ is a global maximizer.
\end{proposition}
% \begin{proposition}\label{prop:global}
% Let $\bm h\in \widetilde{\mathcal{H}}$, $\bm \Pi\in \mathcal{P}^{t_{n_B}}$ be arbitrary, and $\bm Q \in \mathcal{Q}_{\bm h,\bm \Pi}$ be any point. Suppose that there exists $\widehat{\bm \Pi} =\mathrm{BlkD}(\widehat{\bm \Pi}_1,\ldots,\widehat{\bm \Pi}_{n_B})\in \mathcal{P}^{t_{n_B}}$ with $ \widehat{\bm \Pi}_k \in \mathcal{P}^{t_k-t_{k-1}}$ for each $k \in [n_B]$ such that
% \begin{align}\label{eq:global}
% &\ \widehat{\bm \Pi}^T \widetilde{\bm A} \widehat{\bm \Pi}=\bm \Psi,
% \end{align}
% where $\bm \Psi$ and $\widetilde{\bm A}$ are defined in \eqref{eq:defPsi} and \eqref{eq:deftildeA}, respectively. Then $\bm Q$ is a global maximizer.
% \end{proposition}
\begin{proof}
By \Cref{thm:singular-case}, any $\bm Q \in \mathcal{Q}_{\bm h,\bm \Pi}$ takes the form \eqref{eq1:prop saddle}. Define \(\bm \alpha:=\mathrm{diag}\left(\bar{\bm E}(\bm h_1)^T\bm A\bar{\bm E}(\bm h_1)\right)\in \mathbb{R}^{t_{n_B}}\). Since $\bm A$ is diagonal and $\bar{\bm E}(\bm h_1)$ is a coordinate-selection matrix, the entries of $\bm\alpha$ are precisely the $t_{n_B}$ diagonal entries of $\bm A$ selected by $\bm h_1$. Substituting \eqref{eq1:prop saddle} into the objective of Problem~\eqref{eq:QPOC} gives
\begin{align}
F(\bm Q)
&=
\mathrm{Tr}\left(
\bm \Pi^T\bar{\bm E}(\bm h_1)^T
\bm A
\bar{\bm E}(\bm h_1)\bm \Pi
\widehat{\bm B}
\right) =
\mathrm{Tr}\left(
\bm \Pi^T\mathrm{diag}(\bm\alpha)\bm \Pi
\widehat{\bm B}
\right).
\label{eq:plusFX}
\end{align}
Thus, the value of $F(\bm Q)$ depends only on the selected diagonal entries of $\bm A$, determined by $\bm h_1$, and on their ordering, determined by $\bm\Pi$.
We next identify the largest possible value of \eqref{eq:plusFX}. By the rearrangement inequality and the fact that all diagonal entries of $\bm A$ are positive, this inner product is maximized by pairing the positive entries of $\widehat{\bm B}$ with the largest $t_p$ diagonal entries of $\bm A$ and the negative entries with the smallest $t_{n_B}-t_p$ diagonal entries.
Hence, for any feasible $\bm h_1$ and any permutation $\bm\Pi$,
\begin{align}
F(\bm Q)
\le
\sum_{i=1}^{t_p} a_i b_i
+
\sum_{i=t_p+1}^{t_{n_B}} a_{d-t_{n_B}+i} b_i .
\label{eq:global_upper_bound}
\end{align}
It remains to show that $\bm Q$ attains this upper bound. Since
$\bar{\bm\Pi}$ only permutes coordinates within the equal-eigenvalue blocks
of $\widehat{\bm B}$, it commutes with $\widehat{\bm B}$. Therefore, by
cyclicity of the trace, (\ref{eq:defPsi}--\ref{eq:Omega_def_rewrite}), and $\bm\Psi=\bm\Omega$, we obtain
\begin{align}\label{eq:FQ}
F(\bm Q)
&=
\mathrm{Tr}\left(
\bm \Pi^T\bar{\bm E}(\bm h_1)^T
\bm A
\bar{\bm E}(\bm h_1)\bm \Pi
\widehat{\bm B}
\right) =
\mathrm{Tr}\left(
\bar{\bm\Pi}^T
\bm \Pi^T\bar{\bm E}(\bm h_1)^T
\bm A
\bar{\bm E}(\bm h_1)\bm \Pi
\bar{\bm\Pi}
\widehat{\bm B}
\right) \notag\\
&=
\mathrm{Tr}\left(\bm \Psi \widehat{\bm B}\right) =
\sum_{i=1}^{t_p} a_i b_i
+
\sum_{i=t_p+1}^{t_{n_B}} a_{d-t_{n_B}+i} b_i .
\end{align}
Thus $\bm Q$ attains the upper bound in \eqref{eq:global_upper_bound} and is therefore a global maximizer.
\end{proof}
We next show that any critical point $\bm Q \in \mathcal{Q}_{\bm h,\bm\Pi}$
violating $\bm\Psi=\bm\Omega$ is a strict saddle point. The proof is to construct
an ascent direction along which the Riemannian Hessian has
positive curvature. This relies on Givens rotations \cite{givens1958computation}.
\begin{definition}[\bf Givens rotations]\label{def:rota}
Given a positive integer $n$, for any indices $1 \le i \neq j \le n$ and angle $\theta \in \mathbb{R}$, the Givens rotation matrix $\bm G(i,j,\theta)\in\mathbb{R}^{n\times n}$
is defined as the identity matrix except for the $2\times2$ block indexed by
rows and columns $(i,j)$, where \(g_{ii}=\cos\theta,\ g_{ij}=\sin\theta,\ g_{ji}=-\sin\theta,\ g_{jj}=\cos\theta\).
\end{definition}
The next lemma provides an elementary identity that can be verified by a direct expansion of the affected $(i,j)$ block.
\begin{lemma} \label{lem:givens_effect}
Let $\bm C=\mathrm{diag}(c_1,\dots,c_n)$ and $\bm D=\mathrm{diag}(d_1,\dots,d_n)$ be diagonal matrices.
Then it holds that
\[
\mathrm{Tr}\left(\bm G(i,j,\theta)^{T}\bm D\bm G(i,j,\theta)\,\bm C\right)-\mathrm{Tr}(\bm D\bm C)
=(c_j-c_i)(d_i-d_j)\sin^2\theta.
\]
\end{lemma}
We next present a counting lemma that identifies a useful mismatched coordinate for the subsequent construction of an improving rotation direction.
\begin{lemma}\label{lem:first_mismatch_counting}
Suppose that $\bm\Omega\neq\bm\Psi$, where $\bm\Psi$ and $\bm\Omega$ are
defined in \eqref{eq:defPsi} and \eqref{eq:Omega_def_rewrite}, respectively.
Write $\omega_i:=\bm\Omega(i,i)$ and $\psi_i:=\bm\Psi(i,i)$ for
$i\in[t_{n_B}]$. If $\omega_i\neq\psi_i$ for some $i\le t_p$, let
$i^*:=\min\{i\le t_p:\omega_i\neq\psi_i\}$; otherwise, let
$i^*:=\max\{i>t_p:\omega_i\neq\psi_i\}$. Then, we have
\[
i^*\le t_p \Rightarrow \omega_{i^*}<\psi_{i^*},
\qquad
i^*>t_p \Rightarrow \omega_{i^*}>\psi_{i^*}.
\]
Moreover, let $k\in[n_A]$ be such that
$\psi_{i^*}=a_{s_{k-1}+1}=\cdots=a_{s_k}$. If $\omega_j\neq \psi_{i^*}$ for all $j>i^*$
when $i^*\le t_p$, or $\omega_j\neq \psi_{i^*}$ for all $j<i^*$ when $i^*>t_p$,
then $h_{2,k}>0$.
\end{lemma}
\begin{proof}We first consider the case $i^*\le t_p$. By the
definition of $i^*$, $\omega_i=\psi_i$ for all $i<i^*$. Since
$\psi_1,\ldots,\psi_{t_p}$ are the largest $t_p$ diagonal entries of
$\bm A$ in nonincreasing order, all diagonal entries of $\bm A$ that are
strictly larger than $\psi_{i^*}$ appear among $\psi_1,\ldots,\psi_{i^*-1}$.
Hence, if $\omega_{i^*}>\psi_{i^*}$, then the number of entries strictly larger
than $\psi_{i^*}$ in $\omega_1,\ldots,\omega_{i^*}$ would exceed the corresponding
multiplicity in the spectrum of $\bm A$, a contradiction.
Since $\omega_{i^*}\neq \psi_{i^*}$, we obtain
$\omega_{i^*}<\psi_{i^*}$.
The case $i^*>t_p$ is analogous.
It remains to prove the last assertion. We argue by contradiction. Set
$m_k:=s_k-s_{k-1}$. Suppose, to the contrary, that $h_{2,k}=0$. Since
$h_{1,k}+h_{2,k}=m_k$, all $m_k$ copies of the eigenvalue $\psi_{i^*}$ are selected
by $\bar{\bm E}(\bm h_1)$. Hence $\bm\Omega$, which is obtained by sorting
the selected diagonal entries blockwise, must contain exactly $m_k$
diagonal entries equal to $\psi_{i^*}$.
Consider the case $i^*\le t_p$. Since $\omega_i=\psi_i$ for all $i<i^*$ and
$\psi_{i^*}=\psi_{i^*}$, we have $ |\{i<i^*:\omega_i=\psi_{i^*}\}|
=
|\{i<i^*:\psi_i=\psi_{i^*}\}|
\le m_k-1.$ Moreover, $\omega_{i^*}\neq \psi_{i^*}$. Hence, if no $j>i^*$ satisfies $\omega_j=\psi_{i^*}$, then the diagonal entries of
$\bm\Omega$ contain at most $m_k-1$ copies of $\psi_{i^*}$, contradicting $h_{2,k}=0$. % there are at most $m_k-1$ such copies before $i^*$, none at $i^*$, and none after $i^*$.
Therefore $h_{2,k}>0$.
The case $i^*>t_p$ is analogous, with the order reversed.
\end{proof}
Based on the above setup, we now construct an ascent direction for each critical point that does not satisfy $\bm\Psi=\bm\Omega$ in the following proposition.
\begin{proposition}\label{prop:saddle}
Let $\bm h\in \widetilde{\mathcal{H}}$, $\bm \Pi\in \mathcal{P}^{t_{n_B}}$,
and $\bm Q \in \mathcal{Q}_{\bm h,\bm \Pi}$ be arbitrary.
Let $\bm\Psi$ and $\bm\Omega$ be defined as in \eqref{eq:defPsi} and
\eqref{eq:Omega_def_rewrite}, respectively. Suppose that $\bm\Psi\ne\bm\Omega$. Then $\bm Q$ is a strict saddle point.
\end{proposition}
\begin{proof}
% Let $\pi$ and $\bar{\pi}$ denote the permutations induced by $\bm\Pi$ and
% $\bar{\bm\Pi}$, respectively. {\color{blue}By assumption,
% $\bm\Omega\ne\bm\Psi$.} We divide the proof into two cases based on the different ways in which $\bm\Omega\neq \bm\Psi$ can occur. To this end, we introduce the following definitions. If there exists $i \in [t_{p}]$ such that \(\bm \Omega(i,i) \neq \bm \Psi(i,i)\), let \( i^* \in [t_{p}]\) be the smallest index such that \(\bm \Omega(i^*,i^*) \neq \bm \Psi(i^*,i^*)\). Otherwise, let \( i^* \in \{t_p + 1, t_p + 2, \ldots, t_{n_B}\}\) be the largest index such that \(\bm \Omega(i^*,i^*) \neq \bm \Psi(i^*,i^*)\). This $i^*$ satisfies \Cref{lem:first_mismatch_counting}.
Let $\pi$ and $\bar{\pi}$ denote the permutations induced by $\bm\Pi$ and
$\bar{\bm\Pi}$, respectively. By assumption, $\bm\Omega\neq\bm\Psi$.
Write $\omega_i:=\bm\Omega(i,i)$ and $\psi_i:=\bm\Psi(i,i)$ for
$i\in[t_{n_B}]$. Choose $i^*$ as in Lemma~\ref{lem:first_mismatch_counting}: if a mismatch
occurs among the first $t_p$ entries, set $ i^*:=\min\{i\le t_p:\omega_i\neq\psi_i\};$ otherwise, set $i^*:=\max\{i>t_p:\omega_i\neq\psi_i\}.$ We distinguish two cases according to whether the value $\psi_{i^*}$ appears on the opposite side of $i^*$ in
$\bm\Omega$.
{\bf Case 1:} Either $i^* \le t_p$ and there exists $j>i^*$ such that
$\omega_j=\psi_{i^*}$, or $i^*>t_p$ and there exists $j<i^*$ such that
$\omega_j=\psi_{i^*}$. If $i^*\le t_p$, then by Lemma~\ref{lem:first_mismatch_counting},
$\omega_{i^*}<\psi_{i^*}=\omega_j$. Since $j>i^*$ and the diagonal entries of $\widehat{\bm B}$ are
nonincreasing, it suffices to rule out the possibility that
$b_{i^*}=b_j$. If $b_{i^*}=b_j$, then $i^*$ and $j$ belong to the same
equal-eigenvalue block of $\widehat{\bm B}$. This contradicts
$\omega_{i^*}<\omega_j$, since $\bm\Omega$ is sorted in nonincreasing order
within each block. Therefore, $b_{i^*}>b_j$.
If $i^*>t_p$, then by Lemma~\ref{lem:first_mismatch_counting},
$\omega_{i^*}>\psi_{i^*}=\omega_j$. A similar argument gives
$b_j>b_{i^*}$. Therefore, in both situations, we have
\begin{equation}\label{eq3:prop_saddle}
\left( b_j - b_{i^*}\right)\left(\omega_{i^*}-\omega_j\right) > 0 .
\end{equation}
% {\bf Case 1:} Either $i^* \le t_p$ and there exists an index $j > i^*$ satisfying \edit{$\omega_j=\psi_{i^*}$}, or \redit{$i^* > t_p$ and there exists an index $j < i^*$ satisfying \edit{$\omega_j=\psi_{i^*}$}}. On one hand, suppose that $i^* \le t_p$ and there exists an index $j > i^*$ satisfying \edit{$\omega_j=\psi_{i^*}$}. Recall that $a_1\ge \cdots \ge a_{t_p}$ and $i^* \in [t_p]$ is the smallest index such that \(\edit{\omega_{i^*}\neq\psi_{i^*}}\). \redit{By \Cref{lem:first_mismatch_counting}, \edit{$\omega_{i^*}<\psi_{i^*}=\omega_j$}.}
% Moreover, we claim that $b_{i^*} > b_j$, which implies that $i^*$ and $j$ belong to two different blocks in \eqref{eq:block b1}.
% Suppose that this is not true. This, together with $b_{i^*} \ge b_j$ due to $j > i^*$, implies \( b_{i^*} = b_j \). Then the block structure in \eqref{eq:block b1} yields that there exists \( l \in [n_B] \) such that \( t_{l-1} < i^* < j \le t_l \). But \(\edit{\omega_{i^*}<\omega_j}\) contradicts the fact that \( \bar{\bm a}_l \) is in decreasing order in this block, which proves the claim.
% On the other hand, suppose that $i^* > t_p$ and there exists an index $j < i^*$ satisfying \edit{$\omega_j=\psi_{i^*}$}. \redit{By \Cref{lem:first_mismatch_counting}, \edit{$\omega_{i^*}>\psi_{i^*}=\omega_j$}.} Using the definition of $i^*$ and an argument similar to the above, we obtain $b_j>b_{i^*}$ and \edit{$\omega_j<\omega_{i^*}$}. In either case, we have
% \begin{align}\label{eq3:prop saddle}
% \edit{\left( b_j - b_{i^*}\right)\left(\omega_{i^*}-\omega_j\right) > 0.}
% \end{align}
Set $i':=\bar\pi^{-1}(i^*)$ and $j':=\bar\pi^{-1}(j)$. Since
$\bar{\bm\Pi}$ is block diagonal with respect to the equal-eigenvalue
blocks of $\widehat{\bm B}$, we have $b_{i'}=b_{i^*}$ and $b_{j'}=b_j$.
Moreover, by \eqref{eq:Omega_def_rewrite},
\begin{equation}\label{eq2:prop_saddle}
\widetilde{\bm A}(i',i')=\omega_{i^*},
\quad
\widetilde{\bm A}(j',j')=\omega_j .
\end{equation}
Moreover, we define $\bm G(\theta):=\bm G(i',j',\theta)\in\mathbb R^{t_{n_B}\times t_{n_B}}$ and the Stiefel trajectory
\[
\bm Q(\theta) := \bm U\begin{bmatrix}\bar{\bm E}(\bm h_1)\bm \Pi \bm G(\theta)&\widehat{\bm E}(\bm h_2)\bm V_{n_B+1}\end{bmatrix}\bm V.
\]
Obviously, $\bm Q(0) = \bm Q$ due to $\bm G(0) = \bm I$, \eqref{eq1:prop saddle}, and \eqref{eq:defuv}, and $\bm Q(\theta) \in \mathrm{St}(d,K)$.
Moreover, for a sufficiently small $\theta\neq0$, we obtain
\begin{align*}
F(\bm Q(\theta))-F(\bm Q)
&= \mathrm{Tr}\left(\bm{Q}(\theta)^T{\bm A}\bm{Q}(\theta){\bm B}\right) - \mathrm{Tr}(\bm{Q}^T{\bm A}\bm{Q}{\bm B})\notag \\
& = \mathrm{Tr}\left(\bm{G}(\theta)^T\widetilde{\bm{A}}
\bm{G}(\theta)
\widehat{\bm{B}}\right) - \mathrm{Tr}(\widetilde{\bm{A}}\widehat{\bm{B}}) \notag \\
& =(b_{j'}-b_{i'})\left(\widetilde{\bm A}(i',i')-\widetilde{\bm A}(j',j')\right)\sin^2\theta \notag \\
&= \left( b_j - b_{i^*}\right)\left(\omega_{i^*}-\omega_j\right)\sin^2{\theta} > 0,
\end{align*}
where the second equality uses \eqref{eq:defuv}, \eqref{eq:block b1}, and \eqref{eq:deftildeA}, the third equality uses Lemma~\ref{lem:givens_effect} with $\bm D=\widetilde{\bm A}$ and $\bm C=\widehat{\bm B}$, the last equality uses \eqref{eq2:prop_saddle}, and the inequality follows from \eqref{eq3:prop_saddle}.
% According to~\cite[(5.26) in page 107]{boumal2023optimization} and $\sin(\theta)=\theta+o(\theta)$, there exists $\bm{D} \in T(\bm{Q})$ with $\bm D=\bm Q'(0)$ such that $\langle \bm{D}, \mathrm{Hess}\,F(\bm{Q})[\bm{D}] \rangle > 0$. Hence, $\bm{Q}$ is a strict saddle point.
Let $\bm D:=\bm Q'(0)\in \mathrm{T}(\bm Q)$. Since
$\sin^2\theta=\theta^2+o(\theta^2)$ and $\bm Q$ is critical, the standard
second-derivative formula on the Stiefel manifold
\cite[(5.26), p.~107]{boumal2023optimization} yields
$
\langle \bm D,\mathrm{Hess}\,F(\bm Q)[\bm D]\rangle>0 .
$
Hence $\bm Q$ is a strict saddle point.
{\bf Case 2:} Case~1 does not occur. Choose
$k\in[n_A]$ such that $\psi_{i^*}=a_{s_{k-1}+1}=\cdots=a_{s_k}$. Then there is no
$j>i^*$ with $\omega_j=\psi_{i^*}$ if $i^*\le t_p$, and no $j<i^*$ with
$\omega_j=\psi_{i^*}$ if $i^*>t_p$. Hence Lemma~\ref{lem:first_mismatch_counting} gives
$h_{2,k}>0$. Choose the last column of $\widehat{\bm E}_k(\bm h_2)$, i.e., let $g_k:=\sum_{i=1}^{k}h_{2,i}$. Then $\widehat{\bm E}(\bm h_2)\bm e_{g_k}$ lies in the $k$-th eigenspace of $\bm A$.
% {\bf Case 2:} \redit{No index $j$ satisfying the condition in {\bf Case 1} exists.} Since \edit{$\psi_{i^*}$} is a diagonal entry of $\bm A$, there exists $k\in[n_A]$ such that \edit{$\psi_{i^*}=a_{s_{k-1}+1} =\cdots = a_{s_k}$}. Moreover, \redit{\Cref{lem:first_mismatch_counting} gives $h_{2,k}>0$; choose the last column of $\widehat{\bm E}_k(\bm h_2)$, i.e., let $g_k:=\sum_{i=1}^{k}h_{2,i}$. Then $\widehat{\bm E}(\bm h_2)\bm e_{g_k}$ lies in the $k$-th eigenspace of $\bm A$.}
Let $i':=\bar{\pi}^{-1}(i^*)$, so $i'$ and $i^*$ lie in the same block of \eqref{eq:Omega_def_rewrite} and hence $b_{i^*}=b_{i'}$. By \Cref{lem:first_mismatch_counting}, if $i^*\le t_p$ then $b_{i'}>0$ and $\psi_{i^*}>\omega_{i^*}$, while if $i^*>t_p$ then $b_{i'}<0$ and $\psi_{i^*}<\omega_{i^*}$. Then we have
\begin{align}\label{eq:case2_key_sign_rewrite}
b_{i'}\big(\psi_{i^*}-\omega_{i^*}\big)>0.
\end{align}
Set $\bm u:=\bar{\bm E}(\bm h_1)\bm e_{\pi^{-1}(i')}$ and
$\bm v:=\widehat{\bm E}(\bm h_2)\bm e_{g_k}$. Then
\[
\bm u^T\bm A\bm u=\widetilde{\bm A}(i',i')=\omega_{i^*},
\quad
\bm v^T\bm A\bm v=a_{s_k}=\psi_{i^*},
\]
where the identities for $\bm u$ follow from \eqref{eq:deftildeA},
\eqref{eq:Omega_def_rewrite}, and the choice $i'=\bar\pi^{-1}(i^*)$, while
the identity for $\bm v$ follows from the choice of $g_k$.
For $\theta\in\mathbb R$, define $\bar{\bm E}(\bm h_1,\theta)$ and
$\widehat{\bm E}(\bm h_2,\theta)$ by changing only the
$\pi^{-1}(i')$-th column of $\bar{\bm E}(\bm h_1)$ and the $g_k$-th
column of $\widehat{\bm E}(\bm h_2)$ as follows:
\[
\begin{aligned}
\bar{\bm E}(\bm h_1,\theta)_{:,\pi^{-1}(i')}
=
\bm u\cos\theta
+
\bm v\sin\theta,\quad \widehat{\bm E}(\bm h_2,\theta)_{:,g_k} =
-\bm u\sin\theta
+
\bm v\cos\theta,
\end{aligned}
\]
with all other columns unchanged.
% For $\theta \in \mathbb{R}$, define $\bar{\bm E}(\bm h_1,\theta)$ and $\widehat{\bm E}(\bm h_2,\theta)$ by
% \begin{equation*}
% \begin{bmatrix} \bar{\bm E}(\bm h_1,\theta)\bm e_{\pi(i')} & \widehat{\bm E}(\bm h_2,\theta)\bm e_{g_k} \end{bmatrix} = \begin{bmatrix} \bar{\bm E}(\bm h_1)\bm e_{\pi(i')} & \widehat{\bm E}(\bm h_2)\bm e_{g_k} \end{bmatrix} \begin{bmatrix} \cos\theta & -\sin\theta \\ \sin\theta & \cos\theta \end{bmatrix},
% \end{equation*}
% with all other columns of $\bar{\bm E}(\bm h_1)$ and $\widehat{\bm E}(\bm h_2)$ remaining unchanged.
Let
\[
\bm Q(\theta) :=
\bm U\begin{bmatrix}
\bar{\bm E}(\bm h_1,\theta)\bm \Pi & \widehat{\bm E}(\bm h_2,\theta)\bm V_{n_B+1}
\end{bmatrix}\bm V.
\]
By \eqref{eq1:prop saddle} and \eqref{eq:defuv}, $\bm Q(0)=\bm Q$.
Since the two changed columns are obtained by an orthogonal rotation of the
orthonormal pair $\bm u,\bm v$, we also have
$\bm Q(\theta)\in\mathrm{St}(d,K)$. Using \eqref{eq:block b1}, \eqref{eq1:prop saddle}, \eqref{eq:defuv},
and the fact that the zero block of $\bm B$ contributes nothing, the objective
function can be written as
\[
F(\bm Q(\theta))
=
\sum_{l=1}^{t_{n_B}} b_l
\left[
\big(\bar{\bm E}(\bm h_1,\theta)\bm e_{\pi^{-1}(l)}\big)^T
\bm A
\big(\bar{\bm E}(\bm h_1,\theta)\bm e_{\pi^{-1}(l)}\big)
\right].
\]
Since only the $\pi^{-1}(i')$-th column of $\bar{\bm E}(\bm h_1)$ is
changed among the selected columns, all terms with $l\ne i'$ cancel in
$F(\bm Q(\theta))-F(\bm Q)$. Hence, for sufficiently small $\theta\ne0$,
\begin{align*}
F(\bm Q(\theta))-F(\bm Q)
&=
b_{i'}
\left[
(\bm u\cos\theta+\bm v\sin\theta)^T
\bm A
(\bm u\cos\theta+\bm v\sin\theta)
-
\bm u^T\bm A\bm u
\right] \\
&=
b_{i'}
\left[
\bm u^T\bm A\bm u(\cos^2\theta-1)
+
\bm v^T\bm A\bm v\sin^2\theta
\right] \\
&=
b_{i'}(\psi_{i^*}-\omega_{i^*})\sin^2\theta
>0,
\end{align*}
where the second equality uses $\bm u^T\bm A\bm v=0$, and the
last inequality follows from \eqref{eq:case2_key_sign_rewrite}.
As in Case~1, setting $\bm D:=\bm Q'(0)\in\mathrm{T}(\bm Q)$ and using
$\sin^2\theta=\theta^2+o(\theta^2)$, together with the criticality of
$\bm Q$, yields
$ \langle \bm D,\mathrm{Hess}\,F(\bm Q)[\bm D]\rangle>0.$ Thus $\bm Q$ is a strict saddle point.
\end{proof}
Building on \Cref{lem:givens_effect,lem:first_mismatch_counting,prop:global,prop:saddle}, we obtain the following result directly.
\begin{theorem}[\bf Benign optimization landscape of QPOC]\label{thm:land}
Let $\bm h\in \widetilde{\mathcal{H}}$, $\bm \Pi\in \mathcal{P}^{t_{n_B}}$, and $\bm Q \in \mathcal{Q}_{\bm h,\bm \Pi}$ be arbitrary.
Let $\bm\Psi$ and $\bm\Omega$ be defined as in \eqref{eq:defPsi} and
\eqref{eq:Omega_def_rewrite}, respectively. Suppose that $\bm\Psi=\bm\Omega$. Then $\bm Q$ is a global maximizer.
Otherwise, $\bm Q$ is a strict saddle point.
\end{theorem}
We close this section with two comments. First, \Cref{thm:land} gives an
exact classification of all critical points of Problem~\eqref{eq:QPOC}:
the condition $\bm\Psi=\bm\Omega$ characterizes global maximizers, while every other critical point is a strict saddle point. Hence Problem~\eqref{eq:QPOC}
has a benign optimization landscape. Second, combined with strict-saddle avoidance
for randomly initialized first-order methods \cite{lee2019first} and the
\L{}ojasiewicz exponent $1/2$ for Problem~\eqref{eq:QPOC}
\cite{liu2019quadratic}, this result provides a theoretical explanation
for the observed convergence of retraction-based gradient methods to global
maximizers at a local linear rate. The same conclusion extends to the
general symmetric case by orthogonally diagonalizing $\bm B$, grouping its
nonzero eigenvalues by sign, and applying the shift
$\bm A\mapsto \bm A+\alpha\bm I$ for some scalar $\alpha$. If \(\bm B = \bm 0\) or $\bm A=\alpha\bm I_d$ for some scalar $\alpha$, the objective is constant on
\(\operatorname{St}(d,K)\), and every feasible point is a global maximizer.
%We now make several remarks on the significance of this result. First, this theorem provides a precise criterion that distinguishes global maximizers from strict saddle points among all critical points of \edit{Problem~\eqref{eq:QPOC}}. Thus, \edit{Problem~\eqref{eq:QPOC}} admits a benign optimization landscape. Second, combined with the fact that gradient-based methods with random initialization almost surely escape strict saddle points \cite{lee2019first} and the fact that the \L{}ojasiewicz exponent of \edit{Problem~\eqref{eq:QPOC}} is $1/2$ \cite{liu2019quadratic}, this result explains why retraction-based gradient methods almost always converge to a global maximizer of Problem~\eqref{eq:QPOC} at a local linear rate. Third, our proof technique constructs an explicit ascent direction to establish positive curvature of the Riemannian Hessian at strict saddle points. This may provide a useful framework for analyzing the optimization landscapes of other Riemannian optimization problems. Finally, \Cref{thm:land} extends to the general symmetric case by orthogonally diagonalizing $\bm B$, grouping its nonzero eigenvalues by sign, and applying the shift $\bm A\mapsto \bm A+\alpha\bm I$; the remaining cases, where the objective is constant, are trivial.
\section{Application to Heteroscedastic Probabilistic PCA}\label{sec:PPCA}
In this section, we apply the optimization landscape characterization of Problem~\eqref{eq:QPOC} to HePPCA. After introducing the model in Section \ref{subsec:setup}, we use the QPOC formulation of the population HePPCA problem from \cite{wang2023estimation} to establish its benign landscape in Section \ref{subsec:popu}. We then show in Section \ref{subsec:empi} that this benign landscape persists for the empirical problem under mild conditions.
%In this section, we leverage the optimization landscape characterization of Problem~\eqref{eq:QPOC} to analyze the optimization landscape of the HePPCA problem. We formally introduce the HePPCA model in \Cref{subsec:setup}, where data points are collected from heterogeneous sources or sensors with different noise levels. Existing work on this topic was reviewed in \Cref{sec:related}. The work of \cite{wang2023estimation} showed that the population version of HePPCA can be formulated as a QPOC. Therefore, in \Cref{subsec:popu}, we first apply our results to establish the benign optimization landscape of the population problem. Then, in \Cref{subsec:empi}, we perform a perturbation analysis showing that the benign landscape persists for the empirical loss in two settings: under sufficiently small noise in \Cref{thm:Delta}, and with sufficiently many samples in \Cref{thm:theorem5}.
\subsection{Problem Setup}\label{subsec:setup}
Consider the HePPCA problem with $L$ data groups, where the $l$-th group contains $n_l$ samples for each $l \in [L]$. For ease of exposition, let $n := \sum_{l=1}^L n_l$ and denote the $i$-th sample in the $l$-th group by $\bm y_{l,i} \in \R^d$. The samples $\{\bm y_{l,i}\} \subseteq \R^d$ are generated according to the model
\begin{align}\label{eq:HPCA}
\bm y_{l,i} = \bm Q^\star \mathrm{diag}(\sqrt{\bm \lambda}) \bm z_{l,i} + \bm \delta_{l,i},\quad \forall l \in [L],\ i \in [n_l],
\end{align}
where $\bm Q^\star \in \mathrm{St}(d,K)$ denotes the ground-truth subspace to be estimated, $\bm \lambda = (\lambda_1,\dots,\lambda_K)$ with $\lambda_1 > \cdots > \lambda_K > 0$ denotes the signal strengths, $\bm z_{l,i} \overset{i.i.d.}{\sim} \mathcal{N}(\bm 0, \bm I_K)$ are latent variables, and $\bm \delta_{l,i} \sim \mathcal{N}(\bm 0, v_l\bm I_d)$ are additive heteroscedastic noise terms with $v_l > 0$. We assume that the noise terms are independent across all samples and independent of all latent variables. In this work, we assume that the signal and noise strengths are known. We note that \cite{hong2021heppcat} provides a practical procedure for estimating these parameters from observed data.
According to \cite{gilman2025semidefinite,hong2021heppcat}, a maximum likelihood estimator of the ground-truth subspace $\bm Q^\star$ can be obtained by solving the following non-convex problem:
\begin{align}\label{eq:QPOC1}
\max_{\bm Q \in \R^{d\times K}} F(\bm Q) := \sum_{k=1}^K \bm q_k^T \bm A_k \bm q_k
\qquad \mathrm{s.t.}\quad \bm Q \in \mathrm{St}(d,K),
\end{align}
where $\bm q_k$ denotes the $k$-th column of $\bm Q$, $\bm Y_l = [\bm y_{l,1},\dots,\bm y_{l,n_l}] \in \R^{d\times n_l}$ for all $l \in [L]$, and
\begin{align}\label{eq:Ak}
w_{l,k} := \frac{\lambda_k}{\lambda_k+v_l},\quad
\bm A_k := \sum_{l=1}^L \frac{w_{l,k}}{n v_l} \bm Y_l \bm Y_l^T.
\end{align}
Using the data model above, the expectation of $\bm A_k$ is given as follows, whose proof is provided in \cite[Lemma 1]{wang2023estimation}.
\begin{lemma}\label{lem:EA}
For each $k \in [K]$, it holds that
\begin{align}\label{eq:EA}
\E[\bm A_k] = \left( \sum_{l=1}^L \frac{w_{l,k} n_l}{n v_l} \right) \bm Q^\star \mathrm{diag}(\bm \lambda) \bm Q^{\star T} + \left(\sum_{l=1}^L \frac{w_{l,k} n_l}{n} \right) \bm I_d.
\end{align}
\end{lemma}
\subsection{Benign Optimization Landscape of Population HePPCA}\label{subsec:popu}
Replacing each $\bm A_k$ in Problem \eqref{eq:QPOC1} with its expectation $\E[\bm A_k]$ and dropping the resulting additive constant, define $\bm w := (w_1,\dots,w_K)$ with $w_k := \sum_{l=1}^L \frac{n_l w_{l,k}}{n v_l}$, and let
\[
\bW:=\mathrm{diag}(\bm w),\quad \bLambda:=\mathrm{diag}(\bm\lambda).
\]
Then we obtain the following population objective:
\begin{align}\label{eq:QPOC2}
\max_{\bm Q \in \mathbb{R}^{d\times K}}
G(\bm Q) := \mathrm{Tr}\left( \bm Q^T \bm Q^\star{\bLambda}\bm Q^{\star T} \bm Q \bW \right)
\qquad \mathrm{s.t.}\quad \bm Q \in \mathrm{St}(d,K).
\end{align}
It follows from $\lambda_1 > \cdots > \lambda_K$ and \eqref{eq:Ak} that $w_1 > \cdots > w_K$. We denote the critical point set of Problem \eqref{eq:QPOC2} by
\begin{align*}
\mathcal{Q}_G := \left\{ \bm Q \in \mathrm{St}(d,K): \mathrm{grad}\ G(\bm Q) = \bm 0 \right\}.
\end{align*}
Thus, Problem \eqref{eq:QPOC2} is a special case of Problem~\eqref{eq:QPOC}. %\edit{When $K<d$, the signal matrix is singular; adding a large multiple of the identity only shifts $G$ by the constant $\alpha\mathrm{Tr}(\bW)$ and leaves the landscape unchanged.}
% Moreover, $\bm Q^\star \bLambda\bm Q^{\star T}$ admits the eigen-decomposition
% \begin{align*}
% \bm Q^\star \bLambda\bm Q^{\star T} = \begin{bmatrix}
% \bm Q^\star & \bar{\bm Q}^\star
% \end{bmatrix} \begin{bmatrix}
% \bLambda & \bm 0 \\
% \bm 0 & \bm 0
% \end{bmatrix}\begin{bmatrix}
% \bm Q^\star & \bar{\bm Q}^\star
% \end{bmatrix}^T,
% \end{align*}
% where $\left[\bm Q^\star\ \bar{\bm Q}^\star\right] \in \mathcal{O}^d$.
Moreover, let \(\bar{\bm Q}^\star\in\mathbb R^{d\times(d-K)}\) be any matrix
such that \([\bm Q^\star\ \bar{\bm Q}^\star]\in\mathcal O^d\). Then
\(\bm Q^\star \bLambda\bm Q^{\star T}\) admits the
eigen-decomposition
\[
\bm Q^\star \bLambda\bm Q^{\star T}
=
\begin{bmatrix}
\bm Q^\star & \bar{\bm Q}^\star
\end{bmatrix}
\begin{bmatrix}
\bLambda & \bm 0 \\
\bm 0 & \bm 0
\end{bmatrix}
\begin{bmatrix}
\bm Q^\star & \bar{\bm Q}^\star
\end{bmatrix}^T .
\]
Using this decomposition, $\lambda_1 > \cdots > \lambda_K > 0$, and \Cref{fact:1}, one can verify that the critical point set of Problem \eqref{eq:QPOC2} can be expressed as follows. Define
\begin{equation*}
\mathcal{H}_G\! :=\! \left\{\!(h_1,\dots,h_{K+1})\!:\!\! \sum_{i=1}^{K+1} h_i\! =\! K,\ h_i\! \in\! \{0,1\},\ \!\forall i \!\in\! [K],\ h_{K+1}\!\in\! \{0,1,\dots,d\!-\!K\}\! \right\}.
\end{equation*}
In addition, for any $\bm h = (h_1, \ldots, h_K, h_{K+1}) \in \mathcal{H}_G$, we define \(\bm E(\bm h)\in\mathbb R^{d\times K}\) by \(\bm E(\bm h):=[\bm E_1(\bm h)\ \cdots\ \bm E_K(\bm h)\ \bm E_{K+1}(\bm h)]\),
where for each $i \in [K]$,
\begin{align*}
\bm E_i(\bm h) :=
\begin{cases}
\bm e_i, & \text{if } h_i = 1,\\
\emptyset, & \text{if } h_i = 0,
\end{cases} \quad
\bm E_{K+1}(\bm h) :=
\begin{cases}
\begin{bmatrix}
\bm e_{K+1} & \cdots & \bm e_{K+h_{K+1}}
\end{bmatrix}, & \text{if } h_{K+1} \neq 0,\\
\emptyset, & \text{if } h_{K+1} = 0.
\end{cases}
\end{align*}
Here, $\emptyset$ denotes an empty matrix of size $d \times 0$, which is omitted from the concatenation, and $\{\bm e_i\}_{i=1}^d \subseteq \mathbb{R}^d$ is the standard basis of $\mathbb{R}^d$. Using \Cref{fact:1} and the setup above, one can verify the following closed-form expression for the critical point set of Problem~\eqref{eq:QPOC2}.
\begin{proposition}\label{prop:crit}
It holds that \(\mathcal{Q}_G = \bigcup_{\bm h \in \mathcal{H}_G,\ \bm \Pi \in \mathcal{P}^K} \mathcal{Q}_{\bm h,\bm \Pi}\),
where every $\bm Q \in \mathcal{Q}_{\bm h,\bm \Pi}$ takes the form
\begin{equation}\label{eq:expresshpi 1}
\bm Q = [\bm Q^\star\ \bar{\bm Q}^\star] \mathrm{BlkD}\left(\mathrm{diag}(\bm s),\bm S_{K+1} \right) \bm E(\bm h) \cdot \bm\Pi \cdot \mathrm{diag}(\bm t)
\end{equation}
for some $\bm s, \bm t \in \R^K$ with $s_i, t_i \in \{1,-1\}$ for each $i\in [K]$ and $\bm S_{K+1} \in \mathcal{O}^{d-K}$. % \edit{When $d=K$, the factor $\bm S_{K+1}$ and the corresponding zero-dimensional block are omitted.}
\end{proposition}
\begin{proof}
Let \(\bm U_\star:=[\bm Q^\star\ \bar{\bm Q}^\star]\in\mathcal O^d\) and
set \(\widetilde{\bm Q}:=\bm U_\star^T\bm Q\). The map \(\bm Q\mapsto \bm U_\star^T\bm Q\) is a bijection from
\(\mathrm{St}(d,K)\) onto itself. Hence \(\bm Q\) is a critical point of
the population problem if and only if \(\widetilde{\bm Q}\) is a critical
point of the diagonal QPOC
\begin{equation}\label{eq:diag_QPOC}
\max_{\widetilde{\bm Q}\in\mathbb R^{d\times K}}
\operatorname{Tr}\!\left(
\widetilde{\bm Q}^T
\mathrm{BlkD}(\bLambda,\bm 0)
\widetilde{\bm Q}
\bW
\right)\qquad \mathrm{s.t.} \quad
\widetilde{\bm Q}\in\mathrm{St}(d,K).
\end{equation}
Since \(\lambda_1>\cdots>\lambda_K>0\) and \(w_1>\cdots>w_K>0\),
applying \Cref{fact:1} to \eqref{eq:diag_QPOC} gives
\[
\widetilde{\bm Q}
=
\mathrm{BlkD}\left(\mathrm{diag}(\bm s),\bm S_{K+1}\right)
\bm E(\bm h)\bm\Pi\mathrm{diag}(\bm t),
\]
where \(\bm h\in\mathcal H_G\), \(\bm\Pi\in\mathcal P^K\),
\(s_i,t_i\in\{1,-1\}\) for \(i\in[K]\), and
\(\bm S_{K+1}\in\mathcal O^{d-K}\). Substituting this into
\(\bm Q=\bm U_\star\widetilde{\bm Q}\) gives \eqref{eq:expresshpi 1}. The converse inclusion follows
from the same characterization in \Cref{fact:1}. Hence the stated
representation of \(\mathcal Q_G\) holds.
\end{proof}
% \begin{proof}
% \edit{In the basis $[\bm Q^\star\ \bar{\bm Q}^\star]$, the signal matrix has simple nonzero eigenvalues $\lambda_1>\cdots>\lambda_K>0$ and, if $d>K$, one zero eigenspace of dimension $d-K$; $\bW$ also has simple positive eigenvalues. Applying \Cref{fact:1} gives $h_i\in\{0,1\}$ for $i\le K$, an arbitrary rotation $\bm S_{K+1}$ in the zero eigenspace, and sign choices $\bm s,\bm t\in\{\pm1\}^K$, yielding \eqref{eq:expresshpi 1}.}
% \end{proof}
We next show that, for fixed $\bm h \in \mathcal{H}_G$ and $\bm \Pi \in \mathcal{P}^K$, all points in $\mathcal{Q}_{\bm h,\bm \Pi}$ have the same type of criticality.
\begin{lemma}\label{lem:equallemma}
Let $\bm h \in \mathcal{H}_G$ and $\bm \Pi \in \mathcal{P}^K$ be arbitrary, and let $\bm Q, \widehat{\bm Q} \in \mathcal{Q}_{\bm h, \bm \Pi}$. Then $\bm Q$ is a local minimizer, local maximizer, global minimizer, global maximizer, or strict saddle point if and only if $\widehat{\bm Q}$ has the same classification.
\end{lemma}
\begin{proof}
By \Cref{prop:crit}, $\bm Q$ can be written in the form \eqref{eq:expresshpi 1} for some $\bm s, \bm t \in \R^K$ with $s_i,t_i\in\{1,-1\}$ for each $i\in[K]$ and some $\bm S_{K+1}\in \mathcal{O}^{d-K}$.
Set $ \bm Q' := [\bm Q^\star\ \bar{\bm Q}^\star]\bm E(\bm h)\bm\Pi .$ Define the linear mapping $\psi$ by
\[
\psi(\bm Z)
:= [\bm Q^\star\ \bar{\bm Q}^\star]
\mathrm{BlkD}\left(\mathrm{diag}(\bm s),\bm S_{K+1}\right)
[\bm Q^\star\ \bar{\bm Q}^\star]^T
\bm Z \mathrm{diag}(\bm t).
\]
Then $\psi(\bm Q')=\bm Q$. Moreover, since the matrices multiplying $\bm Z$ from the left and right are orthogonal, $\psi$ is an isometric bijection from $\mathrm{St}(d,K)$ onto itself.
Now, we verify that $G(\psi(\bm Z)) = G(\bm Z)$ holds for all $\bm Z$:
\begin{align*}
G(\psi(\bm Z)) & = \mathrm{Tr}\left(\psi(\bm Z)^T\bm Q^\star { \bLambda} \bm Q^{\star T} \psi(\bm Z) { \bW}\right)\\
&=\mathrm{Tr}\left(\psi(\bm Z)^T[\bm Q^\star\ {\bar{\bm Q}^\star}]\blk\left({ \bLambda},\bm 0\right) [\bm Q^\star\ {\bar{\bm Q}^\star}]^T\psi(\bm Z){ \bW}\right)\\
&=\mathrm{Tr}\left(\mathrm{diag}(\bm t) \bm Z^T [\bm Q^\star\ {\bar{\bm Q}^\star}] \mathrm{BlkD}\left(\mathrm{diag}(\bm s),\bm S_{K+1}^T\right)\blk\left({ \bLambda},\bm 0\right)\right.\\
&\quad\quad\quad~~~\left. \mathrm{BlkD}\left(\mathrm{diag}(\bm s),\bm S_{K+1}\right)[\bm Q^\star\ {\bar{\bm Q}^\star}]^T \bm Z \mathrm{diag}(\bm t){ \bW}\right)\\
&=\mathrm{Tr}\left(\bm Z^T[\bm Q^\star\ {\bar{\bm Q}^\star}]\ \blk\left({ \bLambda},\bm 0\right)\ [\bm Q^\star\ {\bar{\bm Q}^\star}]^T \bm Z\ { \bW}\right)= G(\bm Z),
\end{align*}
where the fourth equality uses $s_i,t_i \in \{1,-1\}$ for each $i \in [K]$.
% Since $\psi$ is an isometric bijection of the feasible set and preserves the objective value, $\bm Q'$ and $\bm Q$ have the same classification. Moreover, $D\psi$ maps tangent spaces isometrically and $G\circ\psi=G$, so
% \[
% \mathrm{Hess}\,G(\psi(\bm Z))[D\psi(\bm Z)\bm D,D\psi(\bm Z)\bm D]
% =
% \mathrm{Hess}\,G(\bm Z)[\bm D,\bm D],\ \forall \bm D\in T(\bm Z).
% \]
% Therefore, positive-curvature directions are preserved under $\psi$.
Since \(\psi\) is an isometry of \(\mathrm{St}(d,K)\) and \(G\circ\psi=G\),
it preserves local and global optimality. Moreover, since \(\psi\) is a Riemannian isometry and \(G\circ\psi=G\),
the Riemannian Hessian quadratic form is invariant under \(\psi\). Indeed,
for any geodesic \(\gamma\) with \(\gamma(0)=\bm Z\) and
\(\gamma'(0)=\bm D\), the curve \(\psi\circ\gamma\) is a geodesic with
initial velocity \(\mathrm{D}\psi(\bm Z)[\bm D]\), and
\(G(\psi(\gamma(t)))=G(\gamma(t))\). Taking second derivatives at
\(t=0\) gives the claim.
Applying the same argument to $\widehat{\bm Q}$ gives the claim.
\end{proof}
Using this lemma, to determine the type of each critical point in
$\mathcal{Q}_{\bm h,\bm \Pi}$, it suffices to study the representative
$\bm Q \in \mathcal{Q}_{\bm h,\bm \Pi}$ of the form
\begin{align}\label{eq:Q}
\bm Q = \left[\bm Q^\star\ \bar{\bm Q}^\star\right] \cdot \bm E(\bm h) \cdot \bm\Pi.
\end{align}
Using \Cref{thm:land}, we directly obtain the optimization landscape of Problem \eqref{eq:QPOC2} and explicitly characterize its set of global maximizers as follows.
\begin{proposition}\label{prop:opt}
Each critical point of Problem \eqref{eq:QPOC2} is either a global maximizer or a strict saddle point. In particular, the set of global maximizers is
\begin{align}\label{eq:opti}
\mathcal{Q}_{\mathrm{opt}}: = \left\{ \bm Q^\star \mathrm{diag}(\bm s): s_i \in \{1,-1\},\ i \in [K]\right\}.
\end{align}
\end{proposition}
\begin{proof}
The first statement follows from \Cref{thm:land}. It remains to characterize the global maximizers. By \Cref{lem:equallemma}, it suffices to consider \eqref{eq:Q}. Let $\bm\mu := (\lambda_1,\ldots,\lambda_K,0,\ldots,0)\in\mathbb{R}^d$. For any $\bm h \in \mathcal{H}_G$ and $\bm \Pi \in \mathcal{P}^K$, substituting \eqref{eq:Q} into \eqref{eq:QPOC2} yields
\begin{align*}
G(\bm Q)
&= \mathrm{Tr}\left(\bm \Pi^T \bm{E}^T(\bm h)
\mathrm{diag}(\bm\mu) \bm E(\bm h)\bm \Pi { \bW}\right).
\end{align*}
Since $\lambda_1>\cdots>\lambda_K>0$ and $w_1>\cdots>w_K>0$, the rearrangement inequality implies
$
G(\bm Q) \le \sum_{k=1}^K \lambda_k w_k,
$
and equality holds for \eqref{eq:Q} if and only if $ \bm E(\bm h)\bm \Pi =
\begin{bmatrix}
\bm I_K& \bm 0
\end{bmatrix}^T.$
Substituting this condition into \eqref{eq:expresshpi 1}, we obtain $ \bm Q = \bm Q^\star \mathrm{diag}(\bm s)\mathrm{diag}(\bm t).$ Since $s_i,t_i\in\{1,-1\}$ for each $i\in[K]$, this is equivalent to the form in \eqref{eq:opti}.
\end{proof}%Hence, $\mathcal{Q}_{\mathrm{opt}}$ is exactly the set in \eqref{eq:opti}.
Building on \Cref{thm:land} and \Cref{prop:opt}, we further show that the population objective has a negative-definite Riemannian Hessian at every global maximizer and a positive-curvature direction at every other critical point.
\begin{proposition}
\label{prop:opt_saddle_G}
The following hold for the objective \(G\) in Problem \eqref{eq:QPOC2}:
\begin{enumerate}
\item[(i)] There exists $\delta_1>0$ such that, for every $\bm Q \in \mathcal{Q}_{\mathrm{opt}}$ and every $\bm D \in \mathrm{T}(\bm Q)$,
\begin{align}\label{eq:opt}
\left\langle\bm D, \mathrm{Hess}\ G(\bm Q)[\bm D]\right\rangle
\le -\delta_1 \|\bm D\|_F^2.
\end{align}
\item[(ii)] % If $\mathcal{Q}_G \setminus \mathcal{Q}_{\mathrm{opt}}$ is nonempty, then
There exists $\delta_2>0$ such that, for every $\bm Q \in \mathcal{Q}_G \setminus \mathcal{Q}_{\mathrm{opt}}$, there exists a nonzero $\bm D \in \mathrm{T}(\bm Q)$ satisfying
\begin{align}\label{eq:strict-saddle}
\left\langle\bm D, \mathrm{Hess}\ G(\bm Q)[\bm D]\right\rangle
\ge \delta_2 \|\bm D\|_F^2.
\end{align}
% \edit{If $\mathcal{Q}_G \setminus \mathcal{Q}_{\mathrm{opt}}=\emptyset$, the statement is vacuous and we fix any positive $\delta_2$.}
\end{enumerate}
\end{proposition}
\begin{proof}
(i) It suffices to consider $\bm Q=\bm Q^\star$, since the map
$\bm Q\mapsto \bm Q\mathrm{diag}(\bm s)$ is an isometry on $\mathrm{St}(d,K)$ and leaves $G$ invariant for any $\bm s\in\{1,-1\}^K$.
For any direction $\bm D \in \mathrm{T}(\bm Q^\star)$, decompose it as
\begin{align*}
\bm D = \underbrace{(\bm I - \bm Q^\star \bm Q^{\star T})\bm D}_{\bm D_1} + \underbrace{\bm Q^\star \bm Q^{\star T}\bm D}_{\bm D_2}.
\end{align*}
Let $\widehat{\bm D} := \bm Q^{\star T} \bm D$. Since $\bm D\in \mathrm{T}(\bm Q^\star)$, $\widehat{\bm D}$ is skew-symmetric and $\bm D_2=\bm Q^\star\widehat{\bm D}$. We have
\begin{align*}
\nabla G(\bm Q) &= 2\bm Q^{\star} \bLambda \bm Q^{\star T} \bm Q \bW,\ \nabla^2 G(\bm Q)[\bm D]=2\bm Q^{\star} \bLambda \bm Q^{\star T} \bm D \bW.
\end{align*}
Therefore, by the formula for the Riemannian Hessian on the Stiefel manifold,
\begin{align*}
\mathrm{Hess}\ G(\bm Q^\star)[\bm D]
&= \mathrm{Proj}_{\mathrm{T}(\bm Q^\star)}
\left(\nabla^2 G(\bm Q^\star)[\bm D]
- \bm D \mathrm{sym}\left(\bm Q^{\star T} \nabla G(\bm Q^\star)\right)\right) \\
&=2\left(\bm Q^\star \bLambda\widehat{\bm D} \bW
- \bm D\bLambda\bW \right)
-2\bm Q^\star\mathrm{sym}\left(\bLambda \widehat{\bm D} \bW - \widehat{\bm D}{\bLambda\bW}
\right).
\end{align*}
For any matrix $\bm M$, we have $\langle \bm D, \bm Q^\star \mathrm{sym}(\bm M)\rangle
= \langle \widehat{\bm D}, \mathrm{sym}(\bm M)\rangle = 0,$ where the second equality follows from the fact that the inner product of a symmetric matrix and a skew-symmetric matrix is zero. Hence,
\begin{align}\label{eq:inner_product}
\left\langle\bm D,\mathrm{Hess}\ G(\bm Q^\star)[\bm D]\right\rangle
=2\mathrm{Tr}\left(
\widehat{\bm D}^T{\bLambda}\widehat{\bm D}\bW
- (\bm D_1^T \bm D_1 + \widehat{\bm D}^T\widehat{\bm D})\bLambda\bW\right).
\end{align}
Since $\lambda_1 > \cdots > \lambda_K$ and $w_1 > \cdots > w_K$, we first have
\begin{align}
\label{eq:part1_inequality}
-\mathrm{Tr}\left(\bm{D}_1^T \bm{D}_1 \bLambda\bW\right)
\le -\lambda_K w_K \| \bm{D}_1 \|_F^2.
\end{align}
Writing $\widehat{\bm D}=(\widehat d_{ij})_{i,j=1}^K$, and using the
skew-symmetry of $\widehat{\bm D}$, we compute
\begin{align}\label{eq:part2_inequality}
\mathrm{Tr}\left(\widehat{\bm{D}}^T \bLambda \widehat{\bm{D}} \bW
- \widehat{\bm{D}}^T \widehat{\bm{D}} \bLambda\bW\right)
& = \sum_{i,j} \left(\lambda_j w_i\widehat{d}_{ji}^2 - \lambda_i w_i\widehat{d}_{ji}^2 \right) \notag\\
& = -\frac{1}{2}\sum_{i,j}(w_j-w_i)(\lambda_j-\lambda_i)\widehat{d}_{ji}^2 \le 0.
\end{align}
Set
\[
\delta_{\rm rot}:=
\begin{cases}
\displaystyle \left(\min_{i\in[K-1]}(w_i-w_{i+1})\right)
\left(\min_{i\in[K-1]}(\lambda_i-\lambda_{i+1})\right),& K\ge 2,\\[1ex]
+\infty,& K=1.
\end{cases}
\]
For $K=1$, we have $\widehat{\bm D}= \bm 0$. Plugging \eqref{eq:part1_inequality} and \eqref{eq:part2_inequality} into \eqref{eq:inner_product} yields
\begin{align*}
\left\langle\bm D,\mathrm{Hess}\ G(\bm Q^\star)[\bm D]\right\rangle
&\le - \sum_{i,j}(w_j-w_i)(\lambda_j-\lambda_i)\widehat{d}_{ji}^2
- 2\lambda_K w_K\|\bm D_1\|_F^2 \\
&\le -\min\left\{\delta_{\rm rot},2\lambda_K w_K \right\}\|\bm D\|_F^2,
\end{align*}
where we used the skew-symmetry of $\widehat{\bm D}$, the fact that its diagonal entries are zero, and
$\|\bm D\|_F^2=\|\widehat{\bm D}\|_F^2+\|\bm D_1\|_F^2$. This proves \eqref{eq:opt} with $\delta_1=\min\{\delta_{\rm rot},2\lambda_Kw_K\}$.
(ii) If \(\mathcal{Q}_G\setminus\mathcal{Q}_{\mathrm{opt}}=\emptyset\), the claim is vacuous and we set \(\delta_2=1\). Otherwise, define
\[
\rho(\bm Q):=
\max_{\bm D\in \mathrm{T}(\bm Q),\ \|\bm D\|_F=1}
\left\langle
\bm D,\mathrm{Hess}\,G(\bm Q)[\bm D]
\right\rangle .
\]
The maximum is attained by compactness of the unit sphere in \(\mathrm{T}(\bm Q)\). By \Cref{prop:saddle}, \(\rho(\bm Q)>0\) on \(\mathcal{Q}_G\setminus\mathcal{Q}_{\mathrm{opt}}\). This set is compact by \Cref{prop:crit,lem:equallemma,prop:opt}, and \(\rho\) is continuous because the Riemannian Hessian depends continuously on \(\bm Q\). Hence
$
\delta_2:=
\frac12
\min_{\bm Q\in\mathcal{Q}_G\setminus\mathcal{Q}_{\mathrm{opt}}}
\rho(\bm Q)>0.
$
For each \(\bm Q\in\mathcal{Q}_G\setminus\mathcal{Q}_{\mathrm{opt}}\), choosing a unit maximizer in the definition of \(\rho(\bm Q)\) gives \eqref{eq:strict-saddle}.
\end{proof}
\subsection{Benign Optimization Landscape of HePPCA}\label{subsec:empi}
In this subsection, we study the empirical problem of HePPCA, namely Problem~\eqref{eq:QPOC1}. It is generally impossible to obtain a closed-form expression for the critical points of \( F(\bm Q) \). Fortunately, the critical points of its population counterpart \( G(\bm Q) \) can be fully characterized, as shown in \Cref{prop:crit}. For ease of exposition, we denote the critical point set of \(F\) by
\begin{align*}
\mathcal{Q}_F := \{\bm Q \in \mathrm{St}(d,K):\mathrm{grad}\ F(\bm Q) =\bm 0\}.
\end{align*}
For each $k \in [K]$, define
\begin{align}\label{eq:Delta}
\bm \Delta_k := \bm A_k - \E\left[ \bm A_k\right].
\end{align}
We first recall a global error bound obtained from \cite[Corollary 1]{liu2019quadratic}.
\begin{lemma}
\label{lem:plg}
There exists a constant $\kappa_1 > 0$ such that, for all $\bm Q \in \mathrm{St}(d,K)$,
\begin{align*}
\mathrm{dist}(\bm Q, \mathcal{Q}_G) \le \kappa_1 \|\mathrm{grad}\ G(\bm Q)\|_F.
\end{align*}
\end{lemma}
% \begin{proof}
% \edit{By \cite[Corollary 1]{liu2019quadratic}, each connected critical component $\mathcal C_j$ of $\mathcal Q_G$ has a local error bound. Since \Cref{prop:crit} makes $\mathcal Q_G$ a finite union of compact components, these local bounds imply \eqref{eq:pl_g} on a neighborhood $\mathcal U'$ of $\mathcal Q_G$. On the compact complement, $\|\mathrm{grad}\,G\|_F$ has a positive minimum $m$, while $\mathrm{dist}(\bm Q,\mathcal Q_G)\le\mathrm{diam}(\mathrm{St}(d,K))$. Taking $\kappa_1$ as the maximum of the local constant and $\mathrm{diam}(\mathrm{St}(d,K))/m$ proves the global bound.}
% \end{proof}
If $\mathcal{Q}_G\setminus\mathcal{Q}_{\mathrm{opt}}=\emptyset$, we set $\mathrm{dist}(\bm Q,\emptyset)=+\infty$ so the corresponding neighborhood assertions are vacuous.
\Cref{lem:plg} controls the distance to \(\mathcal Q_G\) by the norm of Riemannian gradient of \(G\). We next apply this bound to critical points of \(F\), using the following perturbation identity. By \eqref{eq:QPOC1}, \eqref{eq:EA}, and \eqref{eq:QPOC2}, for any \(\bm Q\in\mathrm{St}(d,K)\),
\begin{align}\label{eq:F-G}
F(\bm Q)
=
G(\bm Q)
+
\sum_{k=1}^K
\mathrm{Tr}\!\left(\bm Q^T\bm\Delta_k\bm Q\bm E_{kk}\right)
+
\sum_{k=1}^K\sum_{l=1}^L \frac{w_{l,k}n_l}{n},
\end{align}
where \(\bm E_{kk}:=\bm e_k\bm e_k^T \in \mathbb{R}^{K \times K}\).
\begin{lemma}
\label{lem:dist_twosets}
For each \( \bm Q \in \mathcal{Q}_F \), we have
\begin{align}
\label{eq:distpl}
\mathrm{dist}(\bm Q, \mathcal{Q}_G) \le {2\kappa_1\sum_{k=1}^K \|\bm\Delta_k\|},
\end{align}
where \( \kappa_1 \) is defined in \Cref{lem:plg}.
\end{lemma}
\begin{proof}
For each $\bm Q \in \mathcal{Q}_F$, \eqref{eq:F-G} gives
\begin{align*}
\bm 0 = \mathrm{grad}\ F(\bm Q)
= \mathrm{grad}\ G(\bm Q)
+ \sum_{k=1}^K \mathrm{grad}\,\mathrm{Tr}\left(\bm Q^T\bm \Delta_k \bm Q \bm E_{kk}\right).
\end{align*}
Since \(\bm\Delta_k\) and \(\bm E_{kk}\) are symmetric, we have
\begin{align*}
\|\mathrm{grad}\ G(\bm Q)\|_F
&= 2\left\|\sum_{k=1}^K \mathrm{Proj}_{\mathrm{T}(\bm Q)}
\left( \bm \Delta_k \bm Q \bm E_{kk}\right)\right\|_F \\
&\le 2\sum_{k=1}^K \left\|\bm \Delta_k \bm Q \bm E_{kk}\right\|_F
= 2\sum_{k=1}^K \left\|\bm \Delta_k \bm q_k\right\|
\le 2\sum_{k=1}^K \left\|\bm \Delta_k\right\|,
\end{align*}
where \(\bm q_k\) is the \(k\)-th column of \(\bm Q\). Together with \Cref{lem:plg}, this gives \eqref{eq:distpl}.
\end{proof}
We next perturb the landscape of Problem \eqref{eq:QPOC2} to analyze Problem \eqref{eq:QPOC1} when $\bm A_k$ is close to $\E[\bm A_k]$.
\begin{proposition}
\label{prop:F_classify}
Suppose that \( \{\bm \Delta_k\}_{k=1}^K \) satisfy
\begin{align}\label{eq:delta_condition}
\sum_{k=1}^K \| \bm \Delta_k \| \le \frac{\min\{\delta_1,\delta_2\}}{12},
\end{align}
where $\delta_1$ and $\delta_2$ are defined in \Cref{prop:opt_saddle_G}. Then the following statements hold: \\
(i) There exists \( \gamma_1 > 0\) such that, for every $\bm Q \in \mathrm{St}(d,K)$ satisfying
\[
\dist(\bm Q, \mathcal Q_{\rm opt})\le \gamma_1,
\]
we have
\begin{align}\label{eq1:prop F class}
\langle\bm D, \mathrm{Hess}\ F(\bm Q)[\bm D]\rangle\le -\frac{\delta_1}{3} \|\bm D\|_F^2,\quad \forall \bm D \in \mathrm{T}(\bm Q),
\end{align}
where $\mathcal Q_{\rm opt}$ is defined in \Cref{prop:opt}.\\
(ii) There exists \( \gamma_2 > 0\) such that, for every $\bm Q \in \mathrm{St}(d,K)$ satisfying $\mathrm{dist}(\bm Q, \mathcal{Q}_G\setminus\mathcal{Q}_{\mathrm{opt}}) \le \gamma_2$, there exists a nonzero $\bm D \in \mathrm{T}(\bm Q)$ such that
\begin{align}\label{eq2:prop F class}
\langle \bm D, \mathrm{Hess}\ F(\bm Q)[\bm D]\rangle
\ge \frac{\delta_2}{3} \|\bm D\|_F^2.
\end{align}
\end{proposition}
\begin{proof}
From \eqref{eq:F-G}, we obtain
\begin{align}\label{eq1:prop F}
\mathrm{Hess}\ F(\bm Q)
=
\mathrm{Hess}\ G(\bm Q)
+\sum_{k=1}^K \mathrm{Hess}\,
\mathrm{Tr}(\bm Q^T\bm \Delta_k\bm Q \bm E_{kk}).
\end{align}
For each $\bm D \in \mathrm{T}(\bm Q)$, we have
\begin{align*}
\nabla \mathrm{Tr}(\bm Q^T\bm \Delta_k\bm Q\bm E_{kk})
&= 2\bm \Delta_k\bm Q\bm E_{kk},\\
\nabla^2 \mathrm{Tr}(\bm Q^T\bm \Delta_k\bm Q\bm E_{kk})[\bm D]
&= \bm \Delta_k\bm D\bm E_{kk} + \bm \Delta_k^T\bm D\bm E_{kk}^T
= 2 \bm \Delta_k\bm D \bm E_{kk},
\end{align*}
where $\bm\Delta_k$ and $\bm E_{kk}$ are symmetric. By \eqref{eq:Re He}, nonexpansiveness of the tangent projection, $\|\bm\Delta_k\bm D\bm E_{kk}\|_F\le\|\bm\Delta_k\|\,\|\bm D\|_F$ and $\|\mathrm{sym}(\bm Q^T\bm\Delta_k\bm Q\bm E_{kk})\|_F\le \|\bm\Delta_k\|$, we obtain
\[
\left|\left\langle\bm D, \mathrm{Hess}\,
\mathrm{Tr}(\bm Q^T\bm \Delta_k\bm Q \bm E_{kk})[\bm D]\right\rangle \right|
\le 4\|\bm \Delta_k\|\|\bm D\|_F^2.
\]
This, together with \eqref{eq1:prop F}, implies
\begin{equation}
\label{eq:hessian_def}
\begin{aligned}
&\left|\mathrm{Hess}(F\!-\!G)(\bm Q)\![\bm D,\bm D]\right|\le 4\|\bm D\|_F^2\!\sum_{k=1}^K\|\bm \Delta_k\|.
\end{aligned}
\end{equation}
(i) It follows from \Cref{prop:opt_saddle_G}(i), compactness of \( \mathcal{Q}_{\mathrm{opt}} \), and smoothness of \(G\) that there exists \( \gamma_1 > 0 \) such that for any \( \widetilde{\bm Q} \) in a neighborhood of \( \mathcal{Q}_{\mathrm{opt}} \), i.e.,
\[
\widetilde{\bm Q} \in \left\{ \bm Q \in \mathrm{St}(d, K) : \mathrm{dist}(\bm Q, \mathcal{Q}_{\mathrm{opt}}) \le \gamma_1 \right\},
\]
the following inequality holds:
\begin{align}\label{eq2:prop F}
\langle \widetilde{\bm D}, \mathrm{Hess}\ G(\widetilde{\bm Q})[\widetilde{\bm D}] \rangle
\le -\frac{2}{3} \delta_1 \| \widetilde{\bm D} \|_F^2,
\quad \forall \widetilde{\bm D} \in \mathrm{T}(\widetilde{\bm Q}).
\end{align}
For each $\bm Q\in \mathrm{St}(d,K)$ such that $\mathrm{dist}(\bm Q, \mathcal{Q}_{\mathrm{opt}}) \le \gamma_1$, we have for all $\bm D \in \mathrm{T}(\bm Q)$,
\begin{align*}
\langle \bm D, \mathrm{Hess}\ F(\bm Q)[\bm D]\rangle
&\le 4\sum_{k=1}^K \|\bm \Delta_k\|\cdot \|\bm D\|_F^2
+ \langle \bm D, \mathrm{Hess}\ G(\bm Q)[\bm D]\rangle \le -\frac{1}{3}\delta_1 \|\bm D\|_F^2,
\end{align*}
where the first inequality follows from \eqref{eq:hessian_def}, and the last inequality uses \eqref{eq:delta_condition} and \eqref{eq2:prop F}.
(ii) If \(\mathcal{Q}_G\setminus\mathcal{Q}_{\mathrm{opt}}=\emptyset\), fix any \(\gamma_2>0\). Otherwise, set \(\mathcal S:=\mathcal{Q}_G\setminus\mathcal{Q}_{\mathrm{opt}}\). By \Cref{prop:opt_saddle_G}(ii), for each \(\bm Q_0\in\mathcal S\), there is a unit direction \(\bm D_0\in \mathrm{T}(\bm Q_0)\) with \(\langle \bm D_0,\mathrm{Hess}\,G(\bm Q_0)[\bm D_0]\rangle\ge \delta_2\). For \(\bm Q\in\mathrm{St}(d,K)\), define \(\bm D_{\bm Q_0}(\bm Q):=\mathrm{Proj}_{\mathrm{T}(\bm Q)}(\bm D_0)\). Since \(\bm D_{\bm Q_0}(\bm Q_0)=\bm D_0\neq \bm 0\), from continuity of the tangent projection and the Riemannian Hessian, it follows that there exists \(r_{\bm Q_0}>0\) such that \(\bm D_{\bm Q_0}(\bm Q)\neq \bm 0\) and\[
\frac{\left\langle
\bm D_{\bm Q_0}(\bm Q),
\mathrm{Hess}\,G(\bm Q)[\bm D_{\bm Q_0}(\bm Q)]
\right\rangle}
{\|\bm D_{\bm Q_0}(\bm Q)\|_F^2}
\ge
\frac{2\delta_2}{3},\quad \forall \bm Q\in U_{\bm Q_0},
\]
where \(U_{\bm Q_0}:=\{\bm Q\in\mathrm{St}(d,K):\|\bm Q-\bm Q_0\|_F<r_{\bm Q_0}\}\). The sets $\{U_{\bm Q_0}: \bm Q_0 \in \mathcal S\}$ form an open cover of compact \(\mathcal S\), so there exist \(\bm Q_1,\ldots,\bm Q_m\in\mathcal S\) such that \(\mathcal S \subseteq \bigcup_{i=1}^m U_{\bm Q_i}\). Set \(U:=\bigcup_{i=1}^m U_{\bm Q_i}\). Since \(U\) is an open neighborhood of compact \(\mathcal S\) in \(\mathrm{St}(d,K)\), choose \(\gamma_2>0\) such that
\[
\left\{\bm Q\in\mathrm{St}(d,K):\mathrm{dist}(\bm Q,\mathcal S)\le \gamma_2\right\} \subseteq U.
\]
Thus, if \(\mathrm{dist}(\bm Q,\mathcal S)\le\gamma_2\), then \(\bm Q\in U_{\bm Q_i}\) for some \(i\). Taking \(\bm D:=\bm D_{\bm Q_i}(\bm Q)=\mathrm{Proj}_{\mathrm{T}(\bm Q)}(\bm D_0^i)\), where \(\bm D_0^i\in\mathrm{T}(\bm Q_i)\) is the corresponding unit direction, gives \(\bm D\neq\bm0\) and \(\langle \bm D,\mathrm{Hess}\,G(\bm Q)[\bm D]\rangle\ge(2\delta_2/3)\|\bm D\|_F^2\). Combining this with \eqref{eq:delta_condition} and \eqref{eq:hessian_def} gives \eqref{eq2:prop F class}.
\end{proof}
% {\color{red}
% We use the standard fact that every point on a finite-dimensional Riemannian manifold admits sufficiently small geodesic balls that are normal neighborhoods and geodesically convex \cite[Section~5.3 and Theorem~6.17]{lee2018introduction}. Here ``normal'' means that the exponential map is locally diffeomorphic, while ``strongly geodesically convex'' means that any two points in the neighborhood are connected by a unique minimizing geodesic lying entirely in the neighborhood. Since $\mathrm{St}(d,K)$ is a finite-dimensional Riemannian manifold, this fact applies here.}
To proceed, we introduce the following definitions. Let \(\mathcal B_{\rm geo}(\bm Q_0,\eta)\) denote a geodesic ball in \(\mathrm{St}(d,K)\) centered at $\bm Q_0$ with radius $\eta$ with respect to the Riemannian metric. A subset \(S\subseteq\mathrm{St}(d,K)\) is called geodesically
convex if any two points in \(S\) can be joined by a geodesic segment lying
entirely in \(S\) (see \cite[Chapter~11]{boumal2023optimization}).
\begin{lemma}\label{lem:local_unique_critical}
Let \(H:\mathrm{St}(d,K)\to\mathbb R\) be a smooth function. Let \(\eta>0\)
be such that the geodesic ball
\(\mathcal B_{\rm geo}(\bm Q_0,\eta)\) is geodesically convex. Suppose that there exists \(\mu>0\) such that
\[
\left\langle \bm D,\mathrm{Hess}\,H(\bm Z)[\bm D]\right\rangle
\le -\mu\|\bm D\|_F^2,
\quad
\forall \bm Z\in\mathcal B_{\rm geo}(\bm Q_0,\eta),\
\bm D\in\mathrm{T}(\bm Z).
\]
Then \(\mathcal B_{\rm geo}(\bm Q_0,\eta)\) contains at most one critical point
of \(H\).
\end{lemma}
\begin{proof}
{Suppose that two distinct critical points $\bm Q_a,\bm Q_b$ of $H$ lie
in $\mathcal B_{\rm geo}(\bm Q_0,\eta)$. By geodesic convexity, they
are connected by a nonconstant geodesic $\gamma$ contained in this ball. For
$\varphi:=H\circ\gamma$, the Hessian bound gives
\[
\varphi''(t)
=
\left\langle \dot\gamma(t),
\mathrm{Hess}\,H(\gamma(t))[\dot\gamma(t)]\right\rangle
\le -\mu\|\dot\gamma(t)\|_F^2 <0 .
\]
Hence $\varphi'$ is strictly decreasing. On the other hand, since
$\bm Q_a$ and $\bm Q_b$ are critical points of $H$, we have
$\varphi'(0)=\varphi'(1)=0$, a contradiction.}
\end{proof}
% We next fix several constants used later.
% Since $\mathcal Q_{\mathrm{opt}}$ is finite, there exists $\eta>0$ such that, for
% every $\bm Q_1\in\mathcal Q_{\mathrm{opt}}$, the geodesic ball
% $\mathcal B_{\rm geo}(\bm Q_1,\eta)$ is a strongly geodesically convex normal
% neighborhood contained in the $\gamma_1$-neighborhood of
% $\mathcal Q_{\mathrm{opt}}$. By decreasing $\eta$ if necessary, we may also
% choose $\tau>0$ such that
% \[
% \mathbb B(\bm Q_1,\tau)\cap\mathrm{St}(d,K)
% \subseteq
% \mathcal B_{\rm geo}(\bm Q_1,\eta),
% \qquad
% \forall \bm Q_1\in\mathcal Q_{\mathrm{opt}} .
% \]
% Set
% \[
% \sigma_0:=
% \begin{cases}
% \operatorname{dist}\!\left(\mathcal Q_{\mathrm{opt}},
% \mathcal Q_G\setminus\mathcal Q_{\mathrm{opt}}\right),
% &\mathcal Q_G\setminus\mathcal Q_{\mathrm{opt}}\neq\emptyset,\\
% 1,&\mathcal Q_G\setminus\mathcal Q_{\mathrm{opt}}=\emptyset,
% \end{cases}
% \qquad
% c_0:=1/2.
% \]
% {\color{red}
% Note that \(\sigma_0>0\). In the nontrivial case \(\mathcal Q_G\setminus\mathcal Q_{\mathrm{opt}}\neq\emptyset\), by \Cref{prop:opt_saddle_G}(i) and the continuity of the Riemannian Hessian, each \(\bm Q_1\in\mathcal Q_{\mathrm{opt}}\) admits a sufficiently small strongly geodesically convex normal neighborhood on which \(\mathrm{Hess}\,G\) remains negative definite. Applying \Cref{lem:local_unique_critical} with \(H=G\), we conclude that each \(\bm Q_1\in\mathcal Q_{\mathrm{opt}}\) is isolated in \(\mathcal Q_G\). Since \(\mathcal Q_{\mathrm{opt}}\) is finite and \(\mathcal Q_G\) is compact, it follows that $\sigma_0>0$.}
We next fix several constants used later. Since $\mathcal Q_{\mathrm{opt}}$ is finite, by the local geodesic convexity of sufficiently small geodesic balls, \Cref{prop:opt_saddle_G}(i), and the continuity of the Riemannian Hessian of $G$, we can choose $\eta>0$ such that, for every $\bm Q_1\in\mathcal Q_{\mathrm{opt}}$, the geodesic ball
$\mathcal B_{\rm geo}(\bm Q_1,\eta)$ is geodesically convex, is contained in the $\gamma_1$-neighborhood of $\mathcal Q_{\mathrm{opt}}$, and satisfies
\[
\mathrm{Hess}\,G(\bm Z)[\bm D,\bm D]
\le
-\frac{\delta_1}{2}\|\bm D\|_F^2,
\quad
\forall \bm Z\in\mathcal B_{\rm geo}(\bm Q_1,\eta),\
\bm D\in\mathrm T(\bm Z).
\]
Moreover, since each $\mathcal B_{\rm geo}(\bm Q_1,\eta)$ is an open neighborhood of $\bm Q_1$ in $\mathrm{St}(d,K)$ and $\mathcal Q_{\mathrm{opt}}$ is finite, there exists $\tau>0$ such that
\[
\mathbb B(\bm Q_1,\tau)\cap\mathrm{St}(d,K)
\subseteq
\mathcal B_{\rm geo}(\bm Q_1,\eta),
\quad
\forall \bm Q_1\in\mathcal Q_{\mathrm{opt}},
\]
where $\mathbb B(\bm Q_1,\tau) := \left\{ \bm Q: \|\bm Q - \bm Q_1\|_F \le \tau \right\}$.
Set
\[
\sigma_0:=
\begin{cases}
\operatorname{dist}\!\left(\mathcal Q_{\mathrm{opt}},
\mathcal Q_G\setminus\mathcal Q_{\mathrm{opt}}\right),
&\mathcal Q_G\setminus\mathcal Q_{\mathrm{opt}}\neq\emptyset,\\
1,&\mathcal Q_G\setminus\mathcal Q_{\mathrm{opt}}=\emptyset,
\end{cases}
\qquad
c_0:=1/2.
\]
Then, we prove that \(\sigma_0>0\). If
\(\mathcal Q_G\setminus\mathcal Q_{\mathrm{opt}}=\emptyset\), this follows
directly from the definition. Otherwise, fix any
\(\bm Q_1\in\mathcal Q_{\mathrm{opt}}\). By the choice of \(\eta\), the ball
\(\mathcal B_{\rm geo}(\bm Q_1,\eta)\) is geodesically convex and
\(\mathrm{Hess}\,G\) is negative definite on this ball. Applying \Cref{lem:local_unique_critical} with \(H=G\), we conclude that each \(\bm Q_1\in\mathcal Q_{\mathrm{opt}}\) is isolated in \(\mathcal Q_G\). Since \(\mathcal Q_{\mathrm{opt}}\) is finite and \(\mathcal Q_G\) is compact, it follows that $\sigma_0>0$. We now state the perturbation result for Problem \eqref{eq:QPOC1}.
\begin{theorem}\label{thm:Delta}
Suppose that \( \{\bm \Delta_k\}_{k=1}^K \) satisfies
\begin{align}\label{eq:final_condition}
\sum_{k=1}^K \| \bm \Delta_k \| \le \delta_3 := \min \left\{\frac{\min\{\gamma_1,\gamma_2,\tau,\sigma_0/3,c_0\}}{2\kappa_1}, \frac{\min\{\delta_1,\delta_2\}}{12}\right\},
\end{align}
where $\gamma_1,\gamma_2$ are defined in \Cref{prop:F_classify},
$\delta_1,\delta_2$ are defined in \Cref{prop:opt_saddle_G}, $\tau,\sigma_0,c_0$ are fixed above, and $\kappa_1$ is defined in
\Cref{lem:plg}. Then every \( \bm{Q} \in \mathcal{Q}_F \) is either a global
maximizer or a strict saddle point. In particular, if \( \bm Q \) is a global
maximizer, then \(F\) is locally geodesically strongly concave near
$\bm Q$.
\end{theorem}
\begin{proof}
Obviously, it follows from \eqref{eq:final_condition} that \eqref{eq:delta_condition} holds. This, together with \Cref{prop:F_classify}, yields that \eqref{eq1:prop F class} and \eqref{eq2:prop F class} hold. Moreover, using \Cref{lem:dist_twosets} and \eqref{eq:final_condition}, we have \(\mathrm{dist}(\bm{Q}, \mathcal{Q}_G)\le \min\left\{\gamma_1,\gamma_2,c_0,\tau, {\sigma_0}/{3}\right\}\) for any $\bm Q \in \mathcal{Q}_F$. For simplicity, let $r := \min\{\gamma_1,\gamma_2,c_0,\tau,\sigma_0/3\}$. Thus every \( \bm Q\in\mathcal Q_F\) is \(r\)-close to \(\mathcal Q_{\rm opt}\) or to \(\mathcal Q_G\setminus\mathcal Q_{\rm opt}\).
The choice $r\le\sigma_0/3$ separates the two population critical sets when both are nonempty.
\textbf{Case 1.} Suppose that \( \bm{Q} \in \mathcal{Q}_F \) satisfies $\mathrm{dist}(\bm Q,\mathcal{Q}_{G}\setminus\mathcal{Q}_{\rm opt})\le r.$ This, together with \eqref{eq2:prop F class}, implies that $\bm Q$ is a strict saddle point, and thus not a global maximizer.
\textbf{Case 2.} Suppose that \( \bm{Q} \in \mathcal{Q}_F \) satisfies \({\mathrm{dist}\!(\bm Q,\mathcal{Q}_{\rm opt})\!\le\! r}\).
Since \(\mathcal Q_{\mathrm{opt}}\) consists of finitely many isolated points as defined in \eqref{eq:opti}, whose pairwise distances are at least \(2\) due to \(\bm s,\bm t\in\{1,-1\}^K\), and \(r\le c_0=1/2<1\), there exists a unique \(\bm Q_\star\in\mathcal Q_{\mathrm{opt}}\) such that
\[
\|\bm Q_\star-\bm Q\|_F
=
\min\left\{\|\bm U-\bm Q\|_F:\bm U\in\mathcal Q_{\mathrm{opt}}\right\}.
\]
By \(r\le \tau\), we have
\(\bm Q\in\mathcal B_{\rm geo}(\bm Q_\star,\eta)\).
Since \eqref{eq1:prop F class} holds throughout
\(\mathcal B_{\rm geo}(\bm Q_\star,\eta)\),
\Cref{lem:local_unique_critical} shows that
\(\bm Q\) is the unique critical point of \(F\) in this neighborhood.
It remains to show that \(\bm Q\) is a global maximizer. Suppose, for contradiction, that it is not. Since \(\mathrm{St}(d,K)\) is compact and \(F\) is continuous, there exists a global maximizer \(\widehat{\bm Q}\). Then \(\widehat{\bm Q}\in\mathcal Q_F\) and
$ F(\widehat{\bm Q})>F(\bm Q).$
The point \(\widehat{\bm Q}\) cannot fall under \textbf{Case 1}, since every point in \textbf{Case 1} is a strict saddle point. Therefore, $ \operatorname{dist}(\widehat{\bm Q},\mathcal Q_{\mathrm{opt}})\le r.$
Using the same argument as above, there exists a unique $\widehat{\bm Q}_\star\in\mathcal{Q}_{\rm opt}$ such that
\[
\|\widehat{\bm Q}_\star-\widehat{\bm Q}\|_F
=
\min\left\{\|\bm U-\widehat{\bm Q}\|_F:\bm U\in\mathcal{Q}_{\rm opt}\right\}.
\]
By \Cref{prop:opt}, there exists \( \bm{s} \in \{1,-1\}^K \) such that $\bm Q_\star = \widehat{\bm Q}_\star \mathrm{diag}(\bm{s}).$ Moreover, by the definition of \(F\) in \eqref{eq:QPOC1} and \eqref{eq:Ak}, we have
\begin{align}\label{eq:FQdiag}
F(\bm Z) = F(\bm Z\mathrm{diag}(\bm{s})),\quad \forall \bm Z \in \mathrm{St}(d,K).
\end{align}
%\edit{This sign-flip map is an isometry and leaves $F(\bm Z)=\sum_{k=1}^K \bm z_k^T\bm A_k\bm z_k$ invariant.}
Thus, we obtain $F(\widehat{\bm Q})=F(\widehat{\bm Q}\mathrm{diag}(\bm{s})).$ Also, by \( \bm{s} \in \{1,-1\}^K \), we have $\| \bm Q_\star - \widehat{\bm Q} \mathrm{diag}(\bm{s}) \|_F=\|\widehat{\bm Q}_\star-\widehat{\bm Q}\|_F\le r.$
Therefore, \( \widehat{\bm Q}\mathrm{diag}(\bm{s}) \) is also a global maximizer by \eqref{eq:FQdiag} and {$\widehat{\bm Q}\mathrm{diag}(\bm{s}) \in\mathcal B_{\rm geo}(\bm Q_\star,\eta)$ because $r\le\tau$}. Moreover, \( \widehat{\bm Q}\mathrm{diag}(\bm{s})\neq \bm Q \); otherwise, \eqref{eq:FQdiag} would imply $F(\widehat{\bm Q})=F(\bm Q),$ contradicting \(F(\widehat{\bm Q})>F(\bm Q)\). Thus, {$\mathcal B_{\rm geo}(\bm Q_\star,\eta)$} contains two distinct critical points, namely $\bm Q$ and $\widehat{\bm Q}\mathrm{diag}(\bm{s})$, contradicting the uniqueness established above. Hence, $\bm Q$ must be a global maximizer.
Finally, let $\bm Q$ be any global maximizer of \(F\). {{\bf By Case 2}, $\bm Q$ lies near some $\bm Q_\star\in\mathcal Q_{\rm opt}$; applying \eqref{eq1:prop F class} on a smaller neighborhood gives local geodesic strong concavity.} This completes the proof.
\end{proof}
It is worth noting that \cite[Lemma F.14]{gilman2025semidefinite} and \cite[Lemma 3]{wang2023estimation} have established concentration bounds for the deviation of the sample covariance matrix of Gaussian random vectors from its expectation as follows. Recall that $\bm y_{l,i} \sim \mathcal{N}\left(\bm 0, \bm Q^\star { \bLambda} \bm Q^{\star T} + v_l\bm I_d\right),$ as defined in \eqref{eq:HPCA} and let
\begin{align}\label{eq:Cl}
\bm C_l := \frac{1}{n}\sum_{i=1}^{n_l} \bm y_{l,i}\bm y_{l,i}^T,\qquad
\xi_{l} := \frac{\sum_{k=1}^{K}\lambda_{k}+v_{l}d}{\lambda_1+v_l}.
\end{align}
\begin{lemma}
\label{lem:delta_lemma}
For each fixed $l\in[L]$ and each $t>0$, it holds with probability at least $1-e^{-t}$ that
\begin{equation*}
\|\bm C_{l}-\mathbb{E}\left[\bm{C}_{l}\right]\| \leq \frac{c_1 n_l}{n}(\lambda_1+v_l)\max\left\{\sqrt{\frac{\xi_l \log d+t}{n_l}}, \frac{\xi_{l}\log d+t}{n_l}\log n\right\},
\end{equation*}
where $c_{1}>0$ is a universal constant.
\end{lemma}
%Applying the groupwise bound to $\sum_i\bm y_{l,i}\bm y_{l,i}^T/n_l$ and rescaling by $n_l/n$ gives the following estimate for $\bm C_l$ defined in \eqref{eq:Cl}.
By setting $t=\log n$ in the above lemma and applying the union bound, we obtain the following corollary.
\begin{corollary}\label{coro:concen}
It holds with probability at least $1-L/n$ that for all $l \in [L]$,
\begin{align}\label{eq:cov}
\|\bm C_{l}-\mathbb{E}\left[\bm{C}_{l}\right]\| \le \frac{c_1 n_l(\lambda_1+v_l)}{n}\max\left\{\sqrt{\frac{\xi_l \log d + \log n}{n_l}}, \frac{\xi_{l}\log d + \log n}{n_l}\log n\right\},
\end{align}
where $c_{1}>0$ is a universal constant.
\end{corollary}
Before proceeding, we note that for every $l\in[L]$,
\begin{equation}\label{eq:xi}
\xi_l = \frac{\sum_{k=1}^K \lambda_k + v_l d}{\lambda_1+v_l}
\le
\frac{d\lambda_1+v_l d}{\lambda_1+v_l}
= d,
\end{equation}
where the inequality follows from $\sum_{k=1}^K\lambda_k\le K\lambda_1\le d\lambda_1$. Using \Cref{thm:Delta} and \Cref{coro:concen}, we are ready to show the optimization landscape of the HePPCA problem \eqref{eq:QPOC1} when the number of training samples satisfies a suitable lower bound. % \todo{delete? Here, $w_{l,k}$ is computed using the true known values of $\lambda_k$ and $v_l$, and we define $n_{\min}:=\min_l n_l$ and $v_{\min}:=\min_l v_l$.}
\begin{theorem}\label{thm:theorem5}Let $n_{\min}:=\min_l n_l$ and $v_{\min}:=\min_l v_l$.
Suppose that
\begin{align}\label{eq:n}
n_{\min} \ge
\max\left\{
\log^2 n,
\left( \frac{c_1\lambda_1 K}{v_{\min}\delta_3} \right)^2
\right\}\left(d\log d+\log n\right),
\end{align}
where $\delta_3$ is defined in \Cref{thm:Delta} and $c_1$ in \Cref{coro:concen}. Then, with probability at least $1-L/n$, every \( \bm{Q} \in \mathcal{Q}_F \) is either a global maximizer or a strict saddle point. In particular, if \( \bm Q \) is a global maximizer, then \(F\) is locally geodesically strongly concave near $\bm Q$.
\end{theorem}
% \edit{Since $n=\sum_l n_l$, \eqref{eq:n} is an implicit sufficient perturbative condition, not a sharp sample-complexity characterization.}
\begin{proof}
Suppose that \eqref{eq:cov} holds for each $l \in [L]$, which happens with probability at least $1-L/n$. According to \eqref{eq:Delta} and \eqref{eq:Cl}, we have
\[\|\bm \Delta_k\|= \left\|\sum_{l\in[L]} \frac{w_{l,k}}{v_l}(\bm C_l-\E[\bm C_l])\right\|.
\]
This implies
\begin{align*}
\sum_{k=1}^K \|\bm \Delta_k\|
& \le \sum_{k=1}^K\sum_{l=1}^L \frac{w_{l,k}}{v_l}\left\| \bm C_l -\mathbb{E}[\bm C_l] \right\| \\
& \le \frac{c_1}{n} \sum_{k=1}^K\sum_{l=1}^L \frac{w_{l,k}n_l}{v_l}(\lambda_1+v_l)
\max\left\{\sqrt{\frac{\xi_l \log d + \log n}{n_l}}, \frac{\xi_{l}\log d + \log n}{n_l}\log n\right\} \\
%& \le \frac{c_1 \lambda_1 K}{n} \sum_{l=1}^L \frac{ n_l}{v_l}
%\max\left\{\sqrt{\frac{\xi_l \log d + \log n}{n_l}}, \frac{\xi_{l}\log d + \log n}{n_l}\log n\right\} \\
& \le \frac{c_1 \lambda_1 K}{n} \sum_{l=1}^L \frac{ n_l}{v_l}
\max\left\{\sqrt{\frac{d \log d + \log n}{n_l}}, \frac{d \log d + \log n}{n_l}\log n\right\},
\end{align*}
where the second inequality uses \eqref{eq:cov}, the third inequality follows from $w_{l,k} \le \lambda_1/(\lambda_1 + v_l)$ by \eqref{eq:Ak} and \eqref{eq:xi}.
Using the above inequality and $n_l\ge n_{\min} \ge (d\log d + \log n)\log^2 n$ from \eqref{eq:n} yields
\begin{align*}
\sum_{k=1}^K \|\bm \Delta_k\|
& \le \frac{c_1\lambda_1 K}{n} \sum_{l=1}^L \frac{n_l}{v_l}\sqrt{\frac{d \log d + \log n}{n_l}} \le \frac{c_1\lambda_1 K}{v_{\min}}\sqrt{\frac{d \log d + \log n}{n_{\min}}}.
\end{align*}
This, together with \eqref{eq:n}, implies $\sum_{k=1}^K \| \bm \Delta_k \| \le \delta_3.$ It follows from this and \Cref{thm:Delta} that the desired result holds.
\end{proof}
\section{\texorpdfstring{Experimental Results}{Experimental Results}}\label{sec:exp}
In this section, we conduct numerical experiments on the QPOC and HePPCA problems to illustrate the convergence behavior predicted by our theoretical results.
% \begin{figure}
% \centering
% \includegraphics[width=\linewidth]{figures/A3.eps}
% \caption{Enter Caption}
% \label{fig:placeholder}
% \end{figure}
\subsection{Benign optimization landscape of QPOC Problems}\label{subsec:QPOC}
\begin{figure*}[t]
\begin{center}
\begin{subfigure}{0.32\textwidth}
\centering\includegraphics[width = 1\linewidth]{figs/F1.pdf}
\caption{$d=100, K=20$}
\end{subfigure} \hfill
\begin{subfigure}{0.32\textwidth}
\centering\includegraphics[width = 1\linewidth]{figs/F2.pdf}
\caption{$d=100, K=40$}
\end{subfigure}\hfill
\begin{subfigure}{0.32\textwidth}
\centering\includegraphics[width = 1\linewidth]{figs/F3.pdf}
\caption{$d=100, K=60$}
\end{subfigure}
\caption{\textbf{Convergence performance of Riemannian GA for solving QPOC.} The $x$-axis is number of iterations $k$, and $y$-axis is the function value gap $F^*-F^k$. Here, \(F^k:=F(\bm Q^k)\) denotes the function value at the \(k\)-th iterate \(\bm Q^k\) generated by Riemannian GA, and \(F^*\) is the optimal value of Problem~\eqref{eq:QPOC}. % \laura{Make the plot labels and axes larger, should be close to caption size. Also, same comment as in the text about computing $F^*$.}
}
\label{fig:1}
\end{center}
\end{figure*}
In this subsection, we conduct numerical experiments on synthetic data to validate the theoretical results in \Cref{thm:land}. To this end, we apply Riemannian gradient ascent (RGA) to solve {Problem~\eqref{eq:QPOC}} and examine whether it converges to a global maximizer from different random initializations. For each random initialization, we generate a Gaussian random matrix $\bm Z \in \R^{d\times K}$ and compute its compact SVD $\bm Z = \bm U \bm \Sigma \bm V^T$, where $\bm U \in \mathrm{St}(d,K)$ and $\bm V \in \mathcal{O}^{K}$. We then set $\bm Q^0 = \bm U \bm V^T$.
In our experiments, we set $d=100$ and $K \in \{20,40,60\}$, and generate the matrices $\bm A \in \S^{d}$ and $\bm B \in \S^K$ as follows. Specifically, we generate a vector $\bm a \in \R^d$ by sampling its entries independently from the uniform distribution on $[5,10]$ and then sorting them in descending order. Similarly, we generate a vector $\bm b \in \R^K$ by sampling the first $K/2$ entries from the uniform distribution on $[1,1.1]$ and the remaining entries from the uniform distribution on $[-1,-0.9]$, after which the entries are sorted in descending order. We then set $\bm A = \mathrm{diag}(\bm a)$ and $\bm B = \mathrm{diag}(\bm b)$.
Based on the above setup, we run RGA from four different random initializations and visualize the convergence behavior in terms of the optimality gap \(F^*-F^k\) for solving {Problem~\eqref{eq:QPOC}}; see \Cref{fig:1}. {Here, \(F^k:=F(\bm Q^k)\) denotes the function value at the \(k\)-th iterate \(\bm Q^k\) generated by RGA, and \(F^*\) is the optimal value of {Problem~\eqref{eq:QPOC}}, computed according to \eqref{eq:FQ}.} It can be observed that, regardless of the random initialization, RGA consistently converges to a global maximizer at a linear rate. This behavior empirically supports the theoretical results in \Cref{thm:land} regarding the benign optimization landscape of {Problem~\eqref{eq:QPOC}}.
\subsection{Experiments on HePPCA}
In this subsection, we conduct numerical experiments on synthetic data to validate the theoretical results in \Cref{thm:theorem5}. First, we generate the samples $\{\bm y_{l,i}\}$ according to the HePPCA model \eqref{eq:HPCA}. Specifically, we set the total dimension $d=20$, the subspace dimension $K = 3$, the number of groups $L=2$, $\bm \lambda = (3, 2, 0.5)$ and $\bm v = (0.2, 0.1)$. The ground-truth subspace matrix $\bm Q^\star \in \mathrm{St}(d,K)$ is generated by first sampling a Gaussian random matrix $\bm G \in \mathbb{R}^{d\times K}$ with i.i.d. standard normal entries and then computing its QR decomposition $\bm G = \bm Q \bm R$, where $\bm Q \in \mathrm{St}(d,K)$ and $\bm R\in \mathbb{R}^{K\times K}$ is upper triangular. Then, we set $\bm Q^\star = \bm Q$, which has orthonormal columns. Next, we generate three independent sets of samples $\{\bm y_{l,i}\}$ and construct the corresponding data matrices $\{\bm A_k\}_{k=1}^K$ according to \eqref{eq:Ak}.
Based on the above setup, we run RGA from four different random initializations, as described in \Cref{subsec:QPOC}, and stop the algorithm when the Riemannian gradient norm is smaller than $10^{-6}$. %We first visualize the convergence performance in terms of $\hat F - F(\bm Q^k)$, where $\hat F$ denotes the function value at the final iterate, for solving Problem \eqref{eq:QPOC1} in \Cref{fig:2}.
{We first visualize the convergence performance of Riemannian GA for solving Problem~\eqref{eq:QPOC1} in terms of $\hat F-F^k$ in \Cref{fig:2}, where $F^k=F(\bm Q^k)$ denotes the function value at the $k$-th iterate $\bm Q^k$, and $\hat F$ denotes the function value at the final iterate.}
It can be observed that RGA appears to converge to an optimal solution, reaching the same function value across different runs with a linear rate. This behavior empirically supports the theoretical results in \Cref{thm:theorem5} regarding the benign optimization landscape of Problem \eqref{eq:QPOC1}.
% We next report the averaged estimation error $\|\hat{\bm Q}\hat{\bm Q}^T - \bm Q^\star\bm Q^{\star T}\|_F$ over 20 runs in \Cref{tab:1}, where $\hat{\bm Q}$ is the last iterate returned by Riemannian GA, along with the corresponding standard deviation. The consistently low estimation error across runs indicates that the algorithm reliably recovers the underlying subspace, providing empirical support for the estimation error bound established in \Cref{lem:dist_twosets} due to $\|\hat{\bm Q}\hat{\bm Q}^T - \bm Q^\star\bm Q^{\star T}\|_F \le \sqrt{2}\mathrm{dist}(\hat{\bm Q}, \bm Q^*)$.
% We next report the averaged estimation error
% $\|\hat{\bm Q}\hat{\bm Q}^T-\bm Q^\star\bm Q^{\star T}\|_F$ over 20 runs in \Cref{tab:1}, where $\hat{\bm Q}$ denotes the last iterate returned by Riemannian gradient ascent.
% We also report the perturbation size $\sum_{k=1}^K\|\bm\Delta_k\|$, which is the quantity appearing in the perturbation bound in \Cref{lem:dist_twosets}.
% Across all sample sets, the estimation errors remain relatively small, indicating that the algorithm reliably recovers the underlying subspace. This also provides empirical support for the estimation error bound established in \Cref{lem:dist_twosets} due to $\mathrm{dist}(\hat{\bm Q}, \bm Q^*) \le \|\hat{\bm Q}\hat{\bm Q}^T - \bm Q^\star\bm Q^{\star T}\|_F$.
% \begin{table}[!htbp]
% \centering
% \caption{Estimation errors of the ground-truth subspace matrix $\bm Q^\star$ for different sample sets over 20 runs.}
% \label{tab:1}
% \begin{tabular}{c c c}
% \hline
% Sample set & $\|\hat{\bm Q}\hat{\bm Q}^T - \bm Q^\star\bm Q^{\star T}\|_F$ & $\sum_{k=1}^K \|\bm \Delta_k\|$ \\
% \hline
% Set 1 & 0.41 & $18.7$ \\
% Set 2 & 0.44 & $11.9$ \\
% Set 3 & 0.28 & $15.0$ \\
% \hline
% \end{tabular}
% \end{table}
\begin{figure*}[t]
\begin{center}
\begin{subfigure}{0.32\textwidth}
\centering\includegraphics[width = 1\linewidth]{figs/F4.pdf}
\caption{First set of samples}
\end{subfigure} %\quad
\begin{subfigure}{0.32\textwidth}
\centering\includegraphics[width = 1\linewidth]{figs/F5.pdf}
\caption{Second set of samples}
\end{subfigure}
\begin{subfigure}{0.32\textwidth}
\centering\includegraphics[width = 1\linewidth]{figs/F6.pdf}
\caption{Third set of samples}
\end{subfigure}
\caption{\textbf{Convergence performance of Riemannian GA for solving HePPCA.} The $x$-axis is number of iterations $k$, and $y$-axis is the function value gap $\hat F - F^k$. Here, $F^k=F(\bm Q^k)$ denotes the function value at the $k$-th iterate $\bm Q^k$, and $\hat F$ denotes the function value at the final iterate. % \laura{I guess you only need to write $F^*$ once for each plot, perhaps in the title? Also, make the labels and axes font larger (good to be same as the caption font, but close is also good enough).}
}
\label{fig:2}
\end{center}
\end{figure*}
\section{Conclusion}\label{sec:con}
In this work, we establish a complete characterization of the optimization landscape of QPOC by showing that every critical point is either a global maximizer or a strict saddle point. As an application, we show that the population objective of HePPCA is a special instance of QPOC and enjoys both a benign landscape and local geodesic strong concavity near every global maximizer. We further prove that, under a sufficiently large groupwise sample-size condition, the finite-sample HePPCA objective inherits these properties with high probability, thereby supporting the observed local linear convergence of retraction-based methods. A natural future direction is to extend the benign landscape analysis to other manifold optimization problems arising in modern machine learning, such as maximal coding rate reduction \cite{chan2022redunet} and neural collapse \cite{yaras2022neural}. Another interesting direction is to investigate nonhomogeneous QPOC problems that include an additional linear term in the objective.
\bibliographystyle{abbrvnat}
\bibliography{reference}
\end{document}